Submitted to:
Department of Health and Human Services
Assistant Secretary for Planning and Evaluation
Submitted by:
Westat
Chapin Hall Center for Children
James Bell Associates
1. Study Overview
1.1 Background
In 1980, the Adoption Assistance and Child Welfare Act of 1980 (P.L. 96-272) required states to make "reasonable efforts" to prevent children from entering foster care and to reunify children who were placed out of the home with their families. A major focus of policy and planning in state child welfare systems was the development of family preservation programs. The emphasis on family preservation culminated in 1993 in the Family Preservation and Family Support provision of the Omnibus Budget Reconciliation Act (OBRA) (Title IV, subpart 2 of the Social Security Act), which encouraged states to institute or further develop family preservation and family support.
As part of the legislation, the Department of Health and Human Services (DHHS) was authorized to set aside funds to evaluate state family preservation and family support programs. In support of this, DHHS funded three separate studies in September 1994:
· Family Preservation and Family Support Services Implementation Study. This study was awarded to James Bell Associates and is a process analysis of the implementation of the legislation, focusing on the types of programs developed and the barriers encountered. The interim report, "Family Preservation and Family Support (FP/FS) Services Implementation Study," was released March 1999. Special topic reports were completed in 2001 and a final report on implementation should be complete in December 2003.
· National Evaluation of Family Support Programs. This study was awarded to Abt Associates, Inc. and is an outcome evaluation of family support programs. Volume A, a meta-analysis evaluation of family support, and Volume B, a research studies final report, were both completed in April 2001.
· The Evaluation of Family Preservation and Reunification Services. This study was awarded to Westat, Chapin Hall Center for Children, and James Bell Associates, and is the subject of this report. It is an outcome evaluation of family preservation and reunification programs.
The three projects are designed to be complementary. Although each focuses on a different aspect of the 1993 legislation, taken together they represent a comprehensive examination of the programs authorized.
More recently, the enactment of the Adoption and Safe Families Act of 1997 (P.L. 105-89) changed and clarified a number of policies established in the 1980 Act with a renewed emphasis on safety, permanency, and adoption. This legislation placed Federal family preservation initiatives under the rubric of "Promoting Safe and Stable Families" and extended funding for FY 2001. The law made safety of children the paramount concern in service delivery. The law increased the need to understand how family preservation services strengthen families and prevent foster care placement and subsequent abuse and neglect allegations.
Public Law 107-133, the "Promoting Safe and Stable Families Amendments of 2001" was signed into law in January 2002. This legislation reauthorized family preservation services through 2006. Additionally, the legislation authorized the Court Improvement Program, and offered states flexibility in defining family preservation services to allow states to support infant safe haven programs and strengthen parental relationships and promote healthy marriages.
Concurrent with the development of legislation have been program initiatives in family preservation at the state and local levels. Since the 1970s, a number of programs have been developed to provide services to children and families who are experiencing serious problems that may eventually lead to the placement of children in foster care or otherwise result in the dissolution of the family unit. Although these programs share a common philosophy of family- centered services, they differ in their treatment theory, level of intensity of services, and length of service provision. Three models emerged (Nelson et al., 1990):
1. Crisis intervention model. This model, based on crisis theory and intervention, stresses the situation of everyday people confronted with unstable and unsecure circumstances from precipitating events, and the belief that symptoms can be worked through in a brief amount of time (Barth, 1990). Crisis theory also holds that those experiencing a crisis - that is, families about to have a child placed in foster care - will be more amenable to receiving services and learning new behaviors (Nelson et al., 1990, citing Kinney et al., 1988). Homebuilders, a foster care placement prevention program developed in 1974 in Tacoma, Washington, is the prototype program for the crisis intervention model. The program calls for short-term, time-limited services provided to the entire family in the home. Services are provided to families with children who are at risk of an imminent placement into foster care. Social learning theory also plays a part in defining the Homebuilders program, providing the theoretical base for interventions employed (Nelson et al., 1990). Social learning theory stresses that behavior, beliefs, and expectations influence each other in a reciprocal manner, and rejects the belief that changes in thinking and feeling must precede changes in behavior (Barth, 1990). Concrete and supportive services are an important element of the Homebuilders program. Key program characteristics include: contact with the family within 24 hours of the crisis, caseload sizes of one or two families per worker, service duration of four to six weeks, provision of both concrete services and counseling, and up to 20 hours of service per family per week (Nelson et al., 1990).
2. Home-based model. This model focuses on the behavior of the family overall, how members interact with one another, and attempts to change the way in which the family functions as a whole and within the community. Programs using the home-based model stress longer-term interventions based on family systems theory. The FAMILIES program, which began in Iowa in 1974, is the original program using the home-based model. Under the original program in Iowa, teams of workers carry a caseload of 10 to 12 families whom they see in the families' homes for an average of four and one-half months. Both concrete and therapeutic services are provided (Nelson et al., 1990).
3. Family treatment model. This model focuses less on the provision of concrete and supportive services and more on family therapy (Nelson et al., 1990, citing Tavantzis et al., 1986). Services are provided in an office as well as in the home and are less intensive than those using the crisis intervention model. The Intensive Family Services (IFS) Program, which began in Oregon in 1980, is based on the family treatment model. The IFS program also uses family systems theory, which views individual behavioral problems as a reflection of other family problems. Therefore, treatment focuses on the family as a whole. Workers carry a caseload of approximately 11 families. Services are provided for 90 days with weekly followup services provided for three to five and one-half months (Nelson et al., 1990).
Over the years, various states have adopted these family preservation models, sometimes with variations. The growth in family preservation can be partly attributed to early evaluations that were "unequivocally positive and reported high placement prevention successes" (Bath, Howard, and Haapala, 1993). Primarily, these studies only measure family outcomes such as placement prevention for families who receive the treatment. No comparison was made to families who did not receive the services. It was assumed that nearly all children would be taken into foster care placement. However, it cannot be assumed that a high proportion of children receiving family preservation services were at imminent risk without observing the experiences of a comparison group that did not receive the intervention. More recent studies using experimental designs have shown that most of the cases referred were not at imminent risk of placement, as many children in the control groups did not become part of the foster care population.
Although many nonexperimental studies have suggested that high percentages of families remain intact after intensive family preservation services, the results of randomized experiments are mixed. Seven of eleven studies reviewed in A Synthesis of Research on Family Preservation and Family Reunification (Littell and Schuerman, 1995) found that the programs did not produce significant overall reductions in placement. In less than half of the control or comparison cases, placements did not occur within a short period of time after group assignment, which suggests that these programs were generally not delivered to families with children at risk of placement. When the risk of placement among family preservation clients is low, it is unlikely that a program will demonstrate significant reductions in placement.
Despite these findings, placement prevention remains a primary goal of family preservation programs. A review of family preservation programs was conducted in 1995 as part of the Evaluation of Family Preservation and Reunification Services. Information from that study was updated in 1997. As part of the update, 32 family preservation state coordinators were asked if placement prevention was the primary purpose of their program. The majority (78 percent) indicated that it was still the primary purpose, with the remaining coordinators identifying child safety (18 percent) and family functioning (4 percent) as the primary purpose. These goals broaden when county public agency and family preservation administrators were asked about the objectives of local family preservation progress. From the 32 states, 58 county public agency administrators and family preservation program administrators were asked to describe their family preservation objectives. Of the 58 administrators contacted, most offered multiple service objectives. The most frequently reported objective was placement prevention, followed by strengthening families and child and family safety. The purpose of the Evaluation of Family Preservation and Reunification Services is to test whether these service delivery objectives are attained.
1.2 Study Objectives
The Evaluation of Family Preservation and Reunification Services is intended to estimate the impact of family preservation and reunification services. The design of the evaluation was guided by the following objectives:
- To identify and describe the range of existing placement prevention, family preservation, and reunification programs;
- To determine the extent to which family preservation and reunification programs are effective in safely reducing unnecessary foster care placement;
- To determine the extent to which family preservation programs are effective in meeting the basic needs of children and in promoting improved family functioning;
- To explore the extent to which family preservation/reunification programs have varying degrees of success with different target populations;
- To determine the extent to which program variables, child welfare system variables, and other factors in the service delivery environment affect the success of family preservation and reunification programs;
- To identify the effects of each family preservation/reunification program on its related child welfare system; and
- To compare the costs of family preservation/reunification services to those of control groups.
The evaluation was conducted through randomized experiments in four family preservation sites: Kentucky, New Jersey, Tennessee, and Pennsylvania and the evaluation of an earlier implemented reunification program in New York City. The classic experimental design of this study is the best way to determine causal connections between interventions and outcomes. The control group received the "regular services" of the child welfare system; it was not a no-treatment control group. We studied the effects of the experimental services relative to ordinary services, i.e., services that would have been provided in the absence of family preservation services.
1.2.1 Site Selection and Recruitment
Site selection was based on a number of criteria, including selecting programs which were based on well-articulated theories, in place long enough to operate in the way expected by program managers, consistently implemented, and with sufficient numbers of families to provide adequate sample sizes. It was also important that programs have a primary focus on a population of children involved in abuse and neglect reports and that key policymakers, managers, and line staff were willing to allow evaluation. Initially, it was proposed that of the six sites to be evaluated, at least two would be placement prevention programs, two broader family preservation programs, and two reunification programs.
Emphasis was placed on selecting well-defined programs and those with characteristics useful for the development of knowledge (e.g., serving clientele with substance abuse problems). It was decided to evaluate three programs that use relatively "pure" versions of the Homebuilders model of service. These include Memphis, Tennessee; Louisville and Lexington, (8) Kentucky; and seven counties in New Jersey. The fourth family preservation site, Philadelphia, has a program in which the goal of family preservation services is defined more broadly than prevention placement, compares family preservation services to less intensive in-home services, and has an explicit focus on substance abuse.
Our program review established that there were few reunification programs, and those that existed served small numbers of clients. Most reunification programs were part of family preservation programs and served families after discharge from foster care. We decided to examine the HomeRebuilders reunification program in New York City, by conducting the data collection for the experiment started by the New York State Department of Social Services. We were not able to identify a suitable site for a second experimental evaluation of reunification.
1.2.2 Sample Size
Each site was evaluated separately. We initially set a goal of 500 cases in each site, about 250 in each group. To detect a difference of 15 percentage points between the experimental and controls groups in such characteristics as placement rates with a probability of 0.8 (directional hypothesis, centered on 50%) we would require a total of about 275 cases in both groups. We set our goal higher in order to be able to do some subgroup analyses with adequate power. Initially we hoped to enroll 500 families in each site over a one-year period. However, the sample accumulation in sites in this report, Kentucky, New Jersey, Tennessee, and Philadelphia was slower than expected. A 349-case sample size was achieved in Kentucky after enrolling families for two years. In New Jersey, 442 net cases were enrolled over an 18-month period and in Tennessee, 147 net cases were enrolled over a 21-month period. In Philadelphia, we obtained a sample of 353 cases over a 26-month period.
1.3 Data Elements and Measures
Outcome measures relate to the goals of the programs and require multiple measures, including placement, subsequent maltreatment, family problems, and child and family functioning. Outcome measures are the heart of the experiment, but other types of measures were also needed in order to carry out the study and to more fully understand the observed overall impact in specific sites. Other measures include mediating and conditioning variables. Mediating variables reflect intervening factors that may be the underlying mechanism for achieving change in the more general outcomes, including parents' coping skills, the family's social isolation or embeddedness, and the general quality of interactions in the home environment. There is not always a clear dividing line between mediating and outcome measures. Moreover, an outcome in one realm may be a mediator in another. For instance, adequacy of the parent's attention to a child's health may be considered an outcome as itself, but it is also a key mediating variable in relation to other outcomes.
Measures that may "condition" the effects of the treatment, such as demographic and household composition variables, were examined for their potential influence. For example, family preservation services may emerge as more effective for families with certain characteristics (e.g., single parent families or families with younger children). We also used check measures to ensure that the treatment that was intended actually occurred and to determine whether control group families received services that are supposed to be reserved for members of the experimental group. Finally, the study used service variables to identify at the program level those variables necessary for understanding the results at the family level.
1.4 Data Sources
To obtain these measures, we used multiple data sources, including administrative data, interviews with investigating workers, caseworkers and caretakers, and qualitative data collection on program operation and context.
For family preservation/placement prevention sites, the study used a longitudinal design in which caretakers were interviewed at three points in time: when they entered the study, at the end of services, and at one year after entry to the study. Caseworkers were interviewed at two points in time, when the family entered the study and at the end of services. Investigating workers completed a self-administered form as quickly after assignment as possible. They were asked to provide a description of the allegation and the investigation findings. Caseworkers were asked to provide information on the actual services provided during in-person contacts with the family during treatment for the experimental cases and during a comparable time period for the control cases. Administrative data on placement and subsequent maltreatment were collected for 18 months after enrollment on each case.
An interim evaluation report was released in October 2000. The interim report presented description, service, and outcome analyses for the Homebuilders study sites. This report expands on the interim report by including description, service, and outcome analyses of the non-Homebuilders site. Additionally, analyses on sample attrition, social support, investigating worker questionnaires, staff questionnaires, and secondary analyses are included in this report. (9)
To preserve the distinct nature between the Homebuilders programs (Kentucky, New Jersey, and Tennessee) and non-Homebuilders programs (Philadelphia), the description and analysis are presented separately. This report consists of three volumes. The Executive Summary and Study Overviews are provided in both Volumes One and Two. In addition, each volume provides the following:
Volume One - Study implementation, descriptions of each study site, and a description of the families for the Homebuilders sites.
Volume Two - Services for the Homebuilders sites, outcome analysis for the Homebuilders sites, description and analysis on the Philadelphia family preservation, attrition analysis for the study; social support; investigating worker questionnaire analysis; staff questionnaire analysis; and study conclusions.
Volume Three - Appendices A through K, which include study protocols, forms, secondary analysis and questionnaires.
Barth, Richard P. (1990) "Theories Guiding Home-Based Intensive Family Preservation Services," in J.K. Whittaker, J. Kinney, E.M. Tracey, and C. Booth, eds., Reaching High-Risk Families: Intensive Family Preservation in Human Services. New York: Aldine de Gruyter, 1990.
James Bell Associates. (1997). "Interim Report: The Family Preservation and Family Support Services (FP/FS) Implementation Study Volume I," submitted to DHHS, ACF, contract no. 105-94-8103.
Nelson, K. E.; Landsman, M. J.; and Deutelbaum, W. (Jan.-Feb. 1990) "Three Models of Family-Centered Placement Prevention Services." Child Welfare vol. 69, 3-21.
Nelson K. (1994). Family-based Services for Families and Children at Risk of Out-of-Home Placement. Child Welfare Research Review I, 83-108.
Westat, Chapin Hall Center for Children, and James Bell Associates. (1998). "A Review of Family Preservation and Reunification Programs" submitted to DHHS, ASPE, contract no. HHS-100-94-0020.
Westat, Chapin Hall Center for Children, and James Bell Associates. (1998) "Evaluation of the New York City HomeRebuilders Demonstration," submitted to DHHS, ASPE, contract No HHS-100-94-0020.
References
8. Lexington, Kentucky, remained in the study only a short time. Further details on Lexington are presented in Chapter 3, Kentucky Overview.
9. As to be expected with any program, some of the families assigned to family preservation programs did not receive the services or received a minimal dosage of the services. In addition, a small number of the families in the control group were actually provided family preservation services. To address these issues, analyses were conducted in which these cases were dropped (secondary analysis).
2. The Services
In this chapter, we describe the services provided to families in both the family preservation and control groups in Kentucky, New Jersey, and Tennessee. Services in Philadelphia are described in Volume 2, Chapter 4. We are concerned with describing the experiences of these families in these programs and with making comparisons between the experimental and control groups. We must determine whether the experimental group received more services and more intense services in order to assess the extent to which the intended experimental conditions held. The interpretation of outcome information depends on a demonstration that experimental services were more extensive than "regular" services. We also attempt to compare the services received by the experimental group to the Homebuilders model, to get at questions of the extent to which the model was implemented. Finally, we describe how the families experienced these programs.
Most of the data come from the second interview with caretakers and caseworkers in which we asked questions about services offered and received during the period since random assignment and from the contact forms completed by workers serving both groups. In the followup interview a year after random assignment, caretakers were also asked about services received since the post-treatment interview and we report on analyses of those data at the end of this chapter. Comparisons were made between experimental and control group families as they were initially randomly assigned (the "primary analysis"). "Secondary" analyses, where violations of random assignment and cases receiving minimal service are dropped, were also performed. Tables showing secondary analyses are provided in Appendix H, Volume 3. In most cases, secondary analyses show similar results to those of the primary analyses. Differences are highlighted in footnotes to the following text.
2.1 Caseworker Activities
Caretakers were asked to indicate whether the caseworker provided help with a number of specific problems. Table 2-1 shows the number of affirmative responses in each group.
Kentucky. According to caretakers, the most common activities in which workers engaged were discussing discipline and anger management and telling caretakers about other agencies that offer services. On the 19 items on which caretakers were questioned, never did the control group workers reportedly engage in an activity more than the experimental group
| Kentucky | New Jersey | Tennessee | |||||||
|---|---|---|---|---|---|---|---|---|---|
| C | E | p | C | E | p | C | E | p | |
| % | % | % | % | % | % | ||||
| Caseworker helped with money for rent/elect./phone | 3 | 17 | 0.001 | 5 | 4 | 5 | 10 | ||
| Caseworker helped with money for other things | 9 | 35 | 0.001 | 10 | 14 | 11 | 19 | ||
| Caseworker provided transportation | 16 | 42 | 0.001 | 12 | 25 | 0.003 | 19 | 34 | 0.1 |
| Caseworker discussed proper feeding of child | 14 | 20 | 5 | 11 | 0.06 | 16 | 28 | ||
| Caseworker talked with you about discipline | 35 | 55 | 0.001 | 39 | 60 | 0.001 | 46 | 70 | 0.01 |
| Caseworker talked with you on relations with spouse | 16 | 18 | 8 | 14 | 0.09 | 11 | 34 | 0.01 | |
| Caseworker helped you clean house | 2 | 6 | 2 | 5 | 11 | 9 | |||
| Caseworker helped with painting/house repairs | 1 | 1 | 1 | 1 | 5 | 1 | |||
| Caseworker discussed how to get childcare | 15 | 18 | 15 | 1 | 14 | 24 | |||
| Caseworker helped with welfare/food stamps | 8 | 14 | 5 | 7 | 11 | 8 | |||
| Caseworker advised how to get medical care | 12 | 16 | 14 | 13 | 22 | 20 | |||
| Caseworker talked with you how to handle anger | 28 | 43 | 0.005 | 29 | 53 | 0.001 | 42 | 70 | 0.004 |
| Caseworker advised you on substance abuse | 3 | 7 | 11 | 12 | 11 | 18 | |||
| Caseworker discussed with you how to get a better place | 11 | 15 | 12 | 6 | 0.06 | 11 | 19 | ||
| Caseworker advised on job training programs | 9 | 19 | 0.009 | 7 | 10 | 8 | 16 | ||
| Caseworker talked about how to get a paying job | 6 | 17 | 0.004 | 5 | 8 | 11 | 18 | ||
| Caseworker advised on how to continue school | 9 | 18 | 0.04 | 5 | 8 | 14 | 23 | ||
| Caseworker arranged for some childcare | 1 | 3 | 5 | 7 | 6 | 13 | |||
| Caseworker told you about other agencies | 38 | 43 | 42 | 56 | 0.01 | 19 | 33 | ||
| Note: C = Control Group, E = Experimental Group | |||||||||
workers. In the primary analysis, for 7 of the 19 items, experimental group workers reportedly engaged in the activity significantly more often than control group workers (all at p = .01 or less). One additional item showed significant differences in the same direction at p = .05 or lower. (10) A total count of the number of these 19 caseworker activities reported by caretakers also shows significant differences between the experimental and control groups. Caretakers in the experimental group reported an average of 3.9 caseworker activities (n = 148) while caretakers in the control group reported an average of 2.2 caseworker activities (n = 146, p = .001). (11) Caretakers were asked which of the caseworker activities were especially helpful. Experimental group caretakers judged significantly more activities to be helpful than did control group caretakers (1.7 vs. 1.0, p = .001). The services most often cited as helpful by experimental group caretakers were, in order, "the caseworker talked with you about discipline," "the caseworker talked with you about how to handle anger," "the caseworker told you about other agencies," and "the caseworker helped you with money for other things [other than rent, electricity, or phone]." For control group caretakers the most often cited helpful items were "the caseworker told you about other agencies," "the caseworker talking with you about discipline," and "the caseworker talked with you about how to handle anger."
New Jersey. The most common activities of workers (according to the caretakers) were discussions of discipline and the handling of anger and referrals to other agencies. In the primary analysis, in 2 of the 19 items control group workers more often engaged in the activity (ignoring those items with 1% differences): discussing getting a better place to live (p = .055) and discussing child care (not significant). For 4 of the 19 items, experimental group workers significantly more often engaged in the activity: discussion of discipline, transportation, discussion of how to handle anger, and discussion of other agencies. A fifth item, discussion of proper feeding of the child, was nearly significant at p = .06.
There were significant differences between the experimental and control groups in the average number of activities reported by caretakers, 3.25 for the experimental group (n = 210) and 2.31 for the control group (n = 134, p = .001). (12) When asked which of these activities were especially helpful, experimental group respondents judged significantly more activities to be helpful than did the control group respondents (1.97 vs. 1.11, p = .0001). The items cited most often as helpful were remarkably similar to those in Kentucky. The services most often cited as helpful by experimental group caretakers were, in order, "the caseworker talked with you about discipline," "the caseworker talked with you about how to handle anger," "the caseworker told you about other agencies," "the caseworker provided transportation," and "the caseworker helped you with money for other things [other than rent, electricity, or phone]." For control group caretakers the most often cited helpful items were exactly the same as in Kentucky: "the caseworker told you about other agencies," "the caseworker talking with you about discipline," and "the caseworker talked with you about how to handle anger."
Tennessee. Activities most often engaged in were similar to those in Kentucky and New Jersey: talk about discipline and talk about handling of anger. In 15 of the 19 items, experimental group workers were reported to have engaged in the activity more often than control group workers, although the differences were significant on only three of these items (talk about discipline, talk about handling anger, and talk about relations with spouse). For the four items control group workers more often engaged in, differences between the groups were small and not significant.
As in Kentucky and New Jersey, there was a significant difference between the groups in the average number of activities reported by caretakers, 4.6 for the experimental group (n = 80) vs. 2.89 for the control group (n = 37, p = .02). (13) Experimental group respondents also judged more activities as especially helpful, an average of 1.34 vs. .84 (p = .04). Again, the items cited most often as helpful were similar to those in Kentucky and New Jersey. Both the experimental and control groups most often listed talk about discipline, talk about how to handle anger, and transportation as most helpful, although experimental group respondents cited these activities far more often.
2.2 Social Program Participation
In the second interview, at the completion of family preservation services for the experimental group, caretakers were asked about their participation in the same set of social programs they were asked about in the initial interview (see Volume 1, Chapter 7), except this time they were asked to report their participation since the time of the first interview (Table 2-2).
| Program | Kentucky | New Jersey | Tennessee | ||||||
|---|---|---|---|---|---|---|---|---|---|
| C % | E % | p | C % | E % | p | C % | E % | p | |
| Food stamps | 60 | 66 | 51 | 48 | 65 | 64 | |||
| Job training | 3 | 8 | 0.09 | 2 | 3 | 3 | 4 | ||
| WIC | 32 | 45 | 0.02 | 22 | 20 | 51 | 41 | ||
| AFDC | 47 | 49 | 38 | 40 | 49 | 50 | |||
| Housing vouchers | 15 | 20 | 17 | 16 | 11 | 11 | |||
| Social security disability | 39 | 34 | 32 | 28 | 22 | 36 | |||
| Alcoholism program | 5 | 5 | 5 | 5 | 5 | 6 | |||
| Drug treatment program | 3 | 1 | 6 | 9 | 8 | 9 | |||
| Marriage counseling | 0 | 7 | 0.006 (FE) | 2 | 2 | 0 | 1 | ||
| Community mental health program | 11 | 15 | 21 | 28 | 14 | 18 | |||
| Head Start/pre-school | 26 | 21 | 33 | 32 | 25 | 38 | |||
| Note: "FE" indicates significance determined by Fisher's exact test. C = Control Group, E = Experimental Group | |||||||||
Kentucky. The proportions of participation in social programs at post-treatment were remarkably similar to those at the initial interview for combined-group data. There was less than a 2 percent change in participation of most programs. Exceptions to this were a 2.6 percent decrease in participation in WIC services (from 41% to 38%), a 4 percent decrease in the proportion of respondents receiving food stamps (from 67% to 63%), and a 6.3 percent decrease for participation in Head Start or Pre-school programs (from 30% to 24%).
Looking at post-interview data in Table 2-2, there were significant differences in experimental and control group participation in the WIC program, with 45 percent of the experimental group reporting participation at the post-treatment interview (n = 148) compared to 32 percent of the control group (n = 146; p = .021). Differences between experimental and control groups were also found with respect to participation in job training and marriage counseling. For job training, 8 percent of the experimental group reported participation (n = 148) compared to 3 percent of the control group (n = 146; p = .085). Seven percent of the experimental group (n = 102) but none of the control group respondents (n = 105) reported participation in marriage counseling (Fisher's exact p = .006). No significant differences were found with respect to the total number of income support programs or treatment programs in which respondents participated since the time of the first interview.
New Jersey. As in Kentucky, the proportions of social program participation for combined groups at initial and post-treatment interviews were similar except for community mental health programs (26% in the second interview compared to 31% in the first for both groups combined) and using Head Start or another pre-school program (32% vs. 42%). There were no significant differences between the experimental and control groups. There were no significant differences between groups in the number of income support programs and treatment programs.
Tennessee. Participation in social programs at the post-treatment interview was similar to that at the initial interview for both groups combined except for declines in use of food stamps (from 72% to 64%), AFDC (from 61% to 50%) and head start/preschool (38% to 34%). There were no significant differences between the experimental and control groups in participation in any program post-treatment, nor were there significant differences in the average number of income support or treatment programs.
2.3 Caretakers' Reports of Services
In the second interviews, caretakers were asked if they had received any of a set of specific services in the time since the first interview. Results are shown in Table 2-3.
| Kentucky | New Jersey | Tennessee | |||||||
|---|---|---|---|---|---|---|---|---|---|
| C | E | p | C | E | p | C | E | p | |
| % | % | % | % | % | % | ||||
| Daycare | 5 | 19 | 0.001 | 10 | 7 | 26 | 26 | ||
| Help in finding a place to live | 1 | 4 | 5 | 2 | 17 | 5 | 0.04 | ||
| Staying at an emergency shelter | 1 | 1 | 2 | 1 | 6 | 0 | 0.03 | ||
| Medical or dental care | 8 | 15 | 0.07 | 36 | 42 | 34 | 16 | 0.03 | |
| Transportation | 7 | 16 | 0.02 | 14 | 12 | 17 | 19 | ||
| Education services/GED | 1 | 4 | 2 | 2 | 9 | 8 | |||
| Parent education/training classes | 13 | 19 | 8 | 11 | 20 | 8 | 0.06 | ||
| Legal services | 7 | 11 | 11 | 7 | 9 | 5 | |||
| Counseling | 35 | 52 | 0.003 | 50 | 56 | 9 | 17 | ||
| Respite care | 1 | 1 | 0 | 1 | 0 | 0 | |||
| Homemaker services | 1 | 3 | 6 | 3 | 14 | 3 | 0.02 | ||
| A parent aide to help you | 1 | 4 | 7 | 4 | 11 | 5 | |||
| Note: C = Control Group, E = Experimental Group | |||||||||
Kentucky. A significantly greater proportion of caretakers from the experimental group reported receiving such services as day care (19% vs. 5%), transportation (16% vs. 7%), and counseling (52% vs. 35%). All were significant at p = .05 or less. Reported receipt of medical or dental care was also a higher for the experimental group than for the control group (15% vs. 8%, p = .07). (14) In a separate question, caretakers were asked whether the agency provided homemaker services or the assistance of parent aide. Approximately two percent of all caretakers reported having a homemaker and about three percent reported receiving assistance from a parent aide, with slightly greater but not significantly different percentages reported in the experimental group as compared to the control group. When caretakers were asked whether they did not receive any services they felt were needed, 27 percent of the control group responded affirmatively compared to 19 percent of the experimental group, a difference that was not statistically significant in either the primary or secondary analyses.
New Jersey. There were no significant differences between the experimental and control groups in receipt of any of these services in the primary analyses. (15) About 4 percent of the caretakers reported having a homemaker, with no significant difference between the experimental and control groups. Control group caretakers significantly more often reported they did not receive services that were needed (56% vs. 42%, p = .01).
Tennessee. On most of the services items, control group caretakers more often reported receiving the service, these differences being significant at .06 or lower for five of the items listed in Table 2-3. Experimental group percentages were higher for only two items, counseling (reported by far fewer caretakers in Tennessee than in Kentucky and New Jersey) and transportation, neither difference being significant. These rather surprising results, indicating more services for the control group, contradict data on caseworker activities presented above and
data from the Tennessee caseworkers (discussed in Section 2.5). Control group caretakers more often reported they did not receive services that were needed (39% vs. 24%, p = .1).
2.4 Relationship with Caseworker
Table 2-4 shows results from a number of questions in which caretakers were asked about their relationships with caseworkers. In all three states, for most of these questions, caretakers in the experimental group rated their workers significantly more positively than did caretakers in the control group. A greater proportion of experimental group caretakers felt their workers listened to their concerns "most of the time" other responses were "some of the time" and "not very often." Also, a greater proportion of experimental group caretakers felt their workers understood their situation "very well" as compared to "not very well." A greater proportion of caretakers in the experimental group reported reaching agreement with their workers on goals "most of the time."
In all three states, experimental group caretakers significantly more often than control group caretakers reported that workers talked with them about problems that were not easy to talk about, helped caretakers to "see your problems" (p = .1 in Kentucky), and helped them see their good qualities. With regard to the frequency of contact with the workers, in Kentucky, approximately 20 percent of caretakers from both the experimental and control groups indicated they did not see their caseworkers often enough. A greater proportion of caretakers in the experimental group indicated they saw their workers "more often than [they] wanted" (18% vs. 9%) and a greater proportion of caretakers in the control group indicated they saw their workers "as often as [they] wanted" (70% vs. 62%). In New Jersey, a greater proportion of caretakers in the experimental group responded that they saw their workers "as often as [they] wanted" (59% vs. 43%) and a greater proportion of caretakers in the control group responded that they saw their workers "not often enough" (45% vs. 27%). In Tennessee, more experimental group caretakers said they saw their workers more often than they wanted (27% vs. 19%) and more control group caretakers said they did not see their workers often enough (36% vs. 25%), but the differences between the groups on this item were not significant. In none of the three states did the groups differ in the extent to which they called workers when they had problems.
2.5 Caseworkers' Reports of Services
In the second interview, caseworkers were asked whether they had made referrals to any of 25 services, such as childcare, homemaker services, income programs, treatment programs of various sorts, and health care. Results from these 25 items are shown in Table 2-5.
| Kentucky | New Jersey | Tennessee | |||||||
| C | E | p | C | E | p | C | E | p | |
| % | % | % | % | % | % | ||||
| Worker listened to your concerns most of the time | 71 | 87 | 0.001 | 56 | 78 | 0.001 | 71 | 91 | 0.02 |
| Worker understood your situation very well | 75 | 90 | 0.002 | 62 | 79 | 0.001 | 64 | 81 | 0.09 |
| You and worker agreed on goals most of the time | 66 | 76 | 0.06 | 40 | 72 | 0.001 | 38 | 58 | 0.09 |
| Did worker sometimes talk with you about issues that were not easy to talk about? | 27 | 34 | 29 | 44 | 0.01 | 22 | 51 | 0.003 | |
| Caseworker helped you to see your good qualities | 67 | 79 | 0.03 | 47 | 70 | 0.001 | 53 | 82 | 0.001 |
| Caseworker helped you to see your problems | 66 | 76 | 0.1 | 52 | 72 | 0.001 | 50 | 82 | 0.001 |
| Did you see your caseworker | 0.09 | 0.003 | |||||||
| More often than you wanted | 9 | 18 | 12 | 14 | 19 | 27 | |||
| As often as you wanted | 70 | 62 | 43 | 59 | 44 | 48 | |||
| Not often enough | 21 | 20 | 45 | 27 | 36 | 25 | |||
| Note: C = Control Group, E = Experimental Group | |||||||||
| Kentucky | New Jersey | Tennessee | |||||||
|---|---|---|---|---|---|---|---|---|---|
| C | E | p | C | E | p | C | E | p | |
| % | % | % | % | % | % | ||||
| Childcare or baby sitting | 8.7 | 8.5 | 9.3 | 7.7 | 2.1 | 8.9 | |||
| AFDC or other public income (except SSI) | 4.3 | 4.8 | 4.3 | 4 | 4.2 | 5.6 | |||
| SSI for adult or child | 1.9 | 0.6 | 1.2 | 0.7 | 2.1 | 0 | |||
| Food stamps | 2.5 | 6.1 | 4.3 | 3.3 | 4.2 | 6.7 | |||
| Drug treatment | 3.8 | 4.2 | 11 | 5.5 | 0.05 | 0 | 10 | 0.03 | |
| Alcoholism treatment | 3.8 | 2.4 | 8 | 5.5 | 0 | 6.7 | 0.09 | ||
| Legal aid | 3.7 | 7.9 | 1.8 | 4.8 | 0.08 | 0 | 5.6 | ||
| Help with education | 13 | 22 | 0.03 | 14 | 26 | 0.001 | 4.2 | 15.6 | 0.05 |
| Respite care | 5 | 6.1 | 5.5 | 5.5 | 4.2 | 4.4 | |||
| Parent training | 29 | 67 | 0.001 | 28 | 67 | 0.0001 | 31 | 68 | 0.001 |
| Health care | 12 | 22 | 0.02 | 15 | 18 | 8.3 | 22 | 0.04 | |
| Inpatient mental health | 4.3 | 2.4 | 2.5 | 5.1 | 0.14 | 2.1 | 0 | ||
| Outpatient mental health/counseling | 39 | 36 | 25 | 37 | 0.01 | 17 | 20 | ||
| Health assessment | 13 | 24 | 0.01 | 17 | 21 | 13 | 13 | ||
| Housing financial assistance | 3.7 | 12 | 0.005 | 5.5 | 4.8 | 0 | 5.6 | ||
| Other housing services | 1.2 | 10 | 0.01 | 1.9 | 3.7 | 0 | 5.6 | ||
| W.I.C. | 1.9 | 2.4 | 3.1 | 2.6 | 2.1 | 3.3 | |||
| Emergency financial assistance other than housing | 5.6 | 33 | 0.001 | 18 | 22 | 6.3 | 23 | 0.01 | |
| Job training | 0.6 | 1.8 | 1.2 | 2.2 | 0 | 5.6 | |||
| Emergency shelter | 3.7 | 1.8 | 6.2 | 1.5 | 0.02 | 2.1 | 2.2 | ||
| Recreational services | 7.5 | 24 | 0.001 | 11 | 23 | 0.001 | 4.2 | 21 | 0.008 |
| Family planning | 9.9 | 15 | 11 | 20 | 0.009 | 8.3 | 10 | ||
| Self help groups | 10 | 3.6 | 0.02 | 4.3 | 8.8 | 0.056 | 2.1 | 8.9 | |
| Household management | 10 | 32 | 0.01 | 12 | 28 | 0.0001 | 17 | 29 | |
| Homemaker services | 3.7 | 13 | 0.003 | 6.8 | 1.5 | 0.01 | 8.3 | 3.3 | |
| Other | 9.3 | 16 | 0.08 | 15 | 16 | 17 | 14 | ||
| N | 161 | 165 | 162 | 272 | 48 | 90 | |||
| Note: C = Control Group, E = Experimental Group | |||||||||
Kentucky. Caseworkers for the experimental group reported that their clients were provided with an average of 3.7 of these services, while caseworkers from the control group reported their clients were provided with an average of 2.1 of these services (p = .0001). (16) In the primary analyses, 10 specific services were provided significantly more often to the experimental group than to the control group (significance levels were all at p = .05 or less). These services include help with education, parent training, recreation services, health care, health assessment, housing financial assistance, other housing assistance, emergency financial assistance, household management, and homemaker services. A greater proportion of caseworkers from the experimental group selected the unspecified category of "other" services provided (p = .08). One service, self help groups, was provided significantly more often to the control group than the experimental group (p = .02). (17)
New Jersey. Experimental group caseworkers reported that on average their clients were provided 3.5 of these services, while control group families were provided 2.4, a difference significant at .0001. (18) When individual services are examined, there were six services that were provided significantly more often to the experimental group (education services, parent training, outpatient mental health, recreational services, family planning, and household management). Three services were provided significantly more often to the control group: drug treatment, emergency shelter, and homemakers. (19)
Tennessee. Experimental group caseworkers reported providing an average of 3.2 services, compared with 1.6 for the control group, significantly different at .0002. (20) Six individual services were significantly more often provided to experimental group families (drug treatment, help with education, parent training, health care, emergency financial assistance, and recreational services). (21) No services were significantly more often provided to the control group.
Workers serving clients in both the experimental and control groups were asked to complete a one-page contact report following each in-person contact with a family member (see Appendix K, Volume 3). The report was a simple check-off form, asking about who was present in the visit and about the content of the conversation. Although these forms were quite simple and easy to fill out, it proved difficult to get workers to complete them. We implored workers who did not fill out these forms to do so, and we have at least one on a fair proportion of the cases. However, it is likely that for at least some cases on which we have forms that we do not have them for all of the contacts. We are unable to determine how many contacts occurred for which we have no forms. Furthermore, the quality of information may be affected by the fact that some of the forms were submitted after many calls from our office and after long delay. The following analyses were limited to those families with contact reports. Only "primary" analyses are reported for contact reports.
Some data on contact forms are presented in Table 2-6. Forms were received on between 71 percent and 91 percent of the experimental groups and between 51 percent and 71 percent of the control groups. It should be noted that the lower rate for control group cases is partially due to the fact that there was no contact in the four weeks after the date of random assignment, the period of time for which we requested contact forms for the control group (a period comparable to the 4 week period of services for the experimental group). On average, more contact forms were submitted for the experimental group than for the control group. In addition to the overall number of reports submitted, in all three states the experimental group received significantly more home visits, visits with caretakers, visits with the other parent, and visits with children. The experimental group workers were more likely to involve other adults in the family, non-family members, and other workers. As experimental group families received significantly more contacts than the control group families, they also received significantly more individual activities. For both experimental and control families, in all three states the most common concrete service was the provision of transportation. Purchasing food, child care, and providing clothing, furnishings, and supplies were also common forms of concrete services.
| Kentucky | New Jersey | Tennessee | |||||||
|---|---|---|---|---|---|---|---|---|---|
| C % | E % | p | C % | E % | p | C % | E % | p | |
| Number of cases with at least one form submitted | 111 (63%) | 124 (71%) | 119 (71%) | 250 (91%) | 25 (51%) | 73 (74%) | |||
| Average number of forms per case | 3.1 | 13.8 | .001 | 4.4 | 12.4 | .001 | 2.5 | 9.5 | .001 |
| Average number of home visits | 1.9 | 10.3 | .001 | 3.6 | 10.8 | .001 | 2.0 | 8.0 | .001 |
| Average number of visits with caretakers | 2.4 | 12.8 | .001 | 3.8 | 10.3 | .001 | 2.2 | 8.2 | .001 |
| Average number of visits with the other parent | 0.4 | 2.2 | .001 | 0.6 | 1.9 | .001 | 0.2 | 1.7 | .01 |
| Average number of visits with children | 2.1 | 10.3 | .001 | 3.6 | 9.4 | .001 | 1.9 | 7.2 | .001 |
| Concrete Services | |||||||||
| Transportation | 0.5 | 3.5 | .001 | 0.9 | 2.3 | .001 | 0.2 | 1.9 | .001 |
| Buying food | 0.1 | 1.0 | .001 | 0.4 | 0.8 | .003 | 0.0 | 0.5 | .01 |
| Child care | 0.3 | 0.4 | 0.2 | 1.0 | .001 | .04 | .21 | .03 | |
| Clothing, furnishings, and supplies | 0.2 | 0.9 | .001 | 0.2 | 0.6 | .01 | 0.0 | 0.3 | .01 |
| Topics of Discussion | |||||||||
| Discipline of children | 1.5 | 7.1 | .001 | 2.2 | 6.0 | .001 | 1.4 | 4.7 | .001 |
| Goals | 1.8 | 6.0 | .001 | 2.1 | 6.6 | .001 | 1.8 | 4.2 | .001 |
| Caretaker's interaction with children | 1.5 | 6.0 | .001 | 2.2 | 5.7 | .001 | 1.7 | 4.8 | .001 |
| Child's anger management | 1.1 | 3.9 | .001 | 1.6 | 4.8 | .001 | 0.8 | 1.5 | .06 |
| Supervision of children | 1.1 | 4.0 | .001 | 1.4 | 2.9 | .001 | 1.6 | 2.9 | .001 |
| Note: C = Control Group, E = Experimental Group | |||||||||
The contact forms contained additional concrete services and topics of discussion (see Appendix K, Volume 3). Only those that were most often reported are shown here. Entries are average numbers of times per family that an item was reported, for those families with at least one form submitted.
Contact forms also captured general information about the topic of discussion, counseling, or instruction. In all three states, for both experimental and control families, the most common topics of discussion were the discipline of children, goals, and the caretaker's interaction with the children. Other common topics were the child's anger management and supervision of children.
Experimental Group Contacts. We examined further the contact forms for the experimental group to explore some issues in the adherence of programs to the Homebuilders model of service, subscribed to in all three states (see Table 2-7). In addition to other critical elements of family preservation, the Homebuilders model specifies that workers should provide an in-home contact within 72 hours of referral, and family preservation workers should be available seven days per week. Substantial contact should take place within the first week; Kinney, Haapala, and Booth suggest that the typical case receive 11 hours of service in that time. (22) Concrete services are also an important component of service, particularly early in the case.
In Kentucky, of the 124 experimental families with submitted contact forms, 55 (44%) received an in-home contact within 72 hours, 97 (78%) had contact in the first week. Those 97 families had an average of 5.1 hours of face-to-face contact in the first week. Regarding availability of worker, 18 (1%) of contacts occurred on either Saturday or Sunday. Finally, 34 (27%) of the experimental families received some type of concrete service within the first seven days.
| Kentucky | New Jersey | Tennessee | ||||
|---|---|---|---|---|---|---|
| N | % | N | % | N | % | |
| Number of families with contact data | 124 | 250 | 73 | |||
| Total number of contact forms submitted | 1713 | 308 | 690 | |||
| Contacts in week 1 | 280 | 16 | 753 | 24 | 169 | 25 |
| Contacts in week 2 | 353 | 21 | 667 | 22 | 142 | 21 |
| Contacts in week 3 | 322 | 19 | 601 | 19 | 133 | 19 |
| Contacts in week 4 | 322 | 19 | 515 | 17 | 111 | 16 |
| In-home contact within 72 hours | 55 | 44 | 183 | 73 | 42 | 57 |
| In-home contact with 7 days | 97 | 78 | 219 | 88 | 53 | 73 |
| Concrete service within 7 days | 34 | 27 | 95 | 38 | 21 | 29 |
In New Jersey, of the 250 experimental families with submitted contact forms, 73 percent received an in-home contact within 72 hours, 219 (88%) in the first week, and those families had an average of 6.5 hours of face-to-face contact in the first week. Regarding availability of the worker, only 196 (6%) of submitted contacts occurred on Saturday or Sunday. Finally, 38 percent of the experimental families received some type of concrete service within the first seven days.
In Tennessee, of the 73 experimental families with submitted contact forms, 42 (57%) received an in-home contact within 72 hours, 53 (73%) had contact in the first week. We are able to calculate hours of contact for 45 of these 53 cases and these cases had an average of 8.3 hours of face-to-face contact in the first week. Regarding the availability of the worker, 60 (9%) contacts occurred on either Saturday or Sunday. Finally, 21 (29%) of the experimental families received some type of concrete service within the first seven days.
These data seem to indicate that some "structural" aspects of the Homebuilders model (contact within 72 hours of referral, amount of contact in the first week, services provided at all hours, including weekends, and concrete services early in the case) are not always upheld in these states. However, it is not possible to draw firm conclusions about this, because of issues in the quality of the contact form data discussed earlier.
2.6 Summary of Services
In all three states, the caretaker interview, the caseworker interview, and the contact data generally confirmed the expectation that the experimental group would receive more services and more intensive services that the control group. An exception is the caretaker reports of services received in Tennessee. Table 2-8 shows a summary of those items on which there were significant differences between experimental and control groups on the primary analyses in any state.
In all three states, the number of experimental group caseworker activities reported by caretakers was greater than that reported by control group respondents, and this was also true of "helpful" caseworker activities. As for specific caseworker activities, experimental group workers in all three states were more likely to provide transportation, talk about discipline, and talk about how to handle anger. In all three states, the number of specific services received by experimental group families was greater than the number received by control group families. Contact from data confirmed that there was far more contact with experimental group families. The most common concrete service reported on contact forms was transportation; the most common topics of discussion were discipline of children, goals, and caretaker's interaction with children.
| Caseworker Activities: | Kentucky | New Jersey | Tennessee | ||||||
|---|---|---|---|---|---|---|---|---|---|
| Proportion of affirmative answers by caretakers to yes/no questions | C | E | C | E | C | E | |||
| % | % | p | % | % | p | % | % | p | |
| Is caseworker still working with family | 79 | 64 | 0.006 | 75 | 31 | 0.001 | 57 | 34 | 0.02 |
| Caseworker helped with money for rent, electricity, phone | 3 | 17 | 0.001 | 5 | 4 | 5 | 10 | ||
| Caseworker helped with money for other things | 9 | 35 | 0.001 | 10 | 14 | 11 | 19 | ||
| Caseworker provided transportation | 16 | 42 | 0.001 | 12 | 25 | 0.003 | 19 | 34 | 0.10 |
| Caseworker discussed proper feeding of child | 14 | 20 | 5 | 11 | 0.06 | 16 | 28 | ||
| Caseworker talked with you about discipline | 35 | 55 | 0.001 | 39 | 60 | 0.001 | 46 | 70 | 0.01 |
| Caseworker talked with you on relationship with spouse | 16 | 18 | 8 | 14 | 0.09 | 11 | 34 | 0.01 | |
| Caseworker talked with you about how to handle anger | 28 | 43 | 0.005 | 29 | 53 | 0.001 | 42 | 70 | 0.004 |
| Caseworker told you about other agencies | 38 | 43 | 42 | 56 | 0.01 | 19 | 33 | ||
| Caseworker advised on job training programs | 9 | 19 | 0.009 | 7 | 10 | 8 | 16 | ||
| Caseworker talked about how to get paying job | 6 | 17 | 0.004 | 5 | 8 | 11 | 18 | ||
| Caseworker advised on how to continue school | 9 | 18 | 0.04 | 5 | 8 | 14 | 23 | ||
| Caseworker talked about uneasy issues | 27 | 34 | 29 | 44 | 0.008 | 22 | 51 | 0.003 | |
| Caseworker helped you see good qualities | 67 | 79 | 0.03 | 47 | 70 | 0.001 | 53 | 82 | 0.001 |
| Caseworker helped you see your problem | 66 | 76 | 0.10 | 52 | 72 | 0.001 | 50 | 82 | 0.001 |
| Caseworker understood your situation | 75 | 90 | 0.002 | 62 | 79 | 0.001 | 64 | 79 | 0.08 |
| Note: C = Control Group, E = Experimental Group This table only includes items with a primary analysis p-value less than .05 in at least one of the states; p-values greater than .10 are not reported. Items in bold indicate significant findings in favor of the experimental group whereas italicized items indicate significant findings in favor of the control group. | |||||||||
| Kentucky | New Jersey | Tennessee | |||||||
|---|---|---|---|---|---|---|---|---|---|
| C Mean | E Mean | p | C Mean | E Mean | p | C Mean | E Mean | p | |
| CT report of # of caseworker activities | 2.18 | 3.90 | 0.0001 | 2.31 | 3.25 | 0.001 | 2.89 | 4.60 | 0.02 |
| CT report of # of "helpful" caseworker activities | 1.04 | 1.68 | 0.0001 | 1.11 | 1.97 | 0.0001 | 0.83 | 1.33 | 0.04 |
| C = Control Group, E = Experimental Group | |||||||||
| Services Provided | Kentucky | New Jersey | Tennessee | ||||||
|---|---|---|---|---|---|---|---|---|---|
| Proportion of affirmative answers by caretakers to yes/no questions | C % | E % | p | C % | E % | p | C % | E % | p |
| Anyone been in job training program | 3 | 8 | 0.09 | 2 | 3 | 3 | 4 | ||
| Anyone been in WIC | 32 | 45 | 0.02 | 22 | 20 | 51 | 41 | ||
| Been in a marriage counseling program | 0 | 7 | 0.006 | 2 | 2 | 0 | 1 | ||
| Anyone receive daycare | 5 | 19 | 0.001 | 10 | 7 | 26 | 26 | ||
| Anyone receive transportation | 7 | 16 | 0.02 | 14 | 12 | 17 | 19 | ||
| Anyone receiving parent education/training | 13 | 19 | 6 | 10 | 20 | 8 | 0.06 | ||
| Anyone receive counseling | 35 | 52 | 0.003 | 50 | 56 | 9 | 17 | ||
| Anyone receive help finding a place to live | 1 | 4 | 5 | 2 | 17 | 5 | 0.04 | ||
| Anyone stay at an emergency shelter | 1 | 1 | 2 | 1 | 6 | 0 | 0.03 | ||
| Anyone receive medical or dental care | 8 | 15 | 0.07 | 36 | 42 | 34 | 16 | 0.03 | |
| Anyone receive homemaker services | 1 | 3 | 6 | 3 | 14 | 3 | 0.02 | ||
| Were any needed services not gotten | 27 | 19 | 56 | 42 | 0.01 | 39 | 24 | 0.10 | |
| Kentucky | New Jersey | Tennessee | |||||||
| C Mean | E Mean | p | C Mean | E Mean | p | C Mean | E Mean | p | |
| Caseworker report of # of services provided | 3.16 | 4.99 | 0.001 | 2.31 | 3.17 | 0.001 | 1.58 | 3.19 | 0.0002 |
| Note: C = Control Group, E = Experimental Group This table only includes items with a primary p-value less than .05 in at least one of the states; p-values greater than .10 are not reported. Items in bold indicate significant findings in favor of the experimental group whereas italicized items indicate significant findings in favor of the control group. | |||||||||
It is of interest that transportation is a theme in a number of sources of information about services. We do not have information on where workers were transporting parents and children, but it is evident that needs for transportation are common in these families, needs that workers are able to respond to. This is a concrete service that provides immediate help and builds relationships. Furthermore, workers told us that they often use the time in the car to good advantage in discussing problems of the family.
The most common subject of counseling, interaction with children and in particular their discipline, reflect central problems in these families, problems of paramount concern to the child protective system. It is, therefore, not surprising that workers were focused on altering parent-child interaction patterns. Experimental group caseworkers in all three states were more often reported to have talked about difficult issues, to have helped the caretaker to see her/his good qualities and problems, and to have "understood your situation."
Insofar as there are differences between groups, we can be reasonably sure that the experimental conditions held. Conclusions regarding adherence to the Homebuilders model are less clear cut. Families did not always receive contact within 72 hours, fewer than expected contacts occurred in the first week of the program, and few contacts occurred on weekends. There was relatively little provision of concrete services early on. These results are not entirely surprising. Social programs are never implemented precisely as they are designed. Perhaps the test of a program conception is that it achieves desired outcomes even when it is not implemented exactly as intended.
2.7 Services During the Followup Period
When caretakers were interviewed a year after random assignment, they were asked some of the same questions about services received, this time since the last interview (since the end of family preservation services for the experimental group and during a comparable period for the control group). Tables 2-9, 2-10, and 2-11 show analyses of these questions.
Caseworker Activities. Caretaker reports of caseworker activities since the post-treatment interview are shown in Table 2-9. In all three states, the experimental group respondents reported more caseworker activities than did control group respondents. In Kentucky, there were five activities the experimental group caretakers significantly more often reported: help with money for rent, electricity, or phone; help with money for other things; transportation; advice on getting medical care; and information about other agencies. In New Jersey, there were two such activities, help in cleaning the house and talk about how to handle anger, with a third
| Kentucky | New Jersey | Tennessee | |||||||
|---|---|---|---|---|---|---|---|---|---|
| C | E | p | C | E | p | C | E | p | |
| % | % | % | % | % | % | ||||
| Caseworker helped with money for rent/elect./phone | 1 | 8 | .008 | 4 | 5 | 8 | 13 | ||
| Caseworker helped with money for other things | 8 | 16 | .05 | 8 | 11 | 14 | 37 | .01 | |
| Caseworker provided transportation | 11 | 23 | .01 | 11 | 18 | 17 | 42 | .007 | |
| Caseworker discussed proper feeding of child | 3 | 8 | 3 | 3 | 11 | 23 | |||
| Caseworker talked with you about discipline | 24 | 32 | 24 | 34 | .08 | 42 | 62 | .05 | |
| Caseworker talked with you on relations with spouse | 4 | 9 | 8 | 8 | 19 | 39 | .04 | ||
| Caseworker helped you clean house | 1 | 1 | 0 | 5 | .03 (FE) | 8 | 10 | ||
| Caseworker helped with painting/house repairs | 0 | 0 | 0 | 0 | 3 | 4 | |||
| Caseworker discussed how to get childcare | 8 | 11 | 8 | 5 | 6 | 16 | |||
| Caseworker helped with welfare/food stamps | 2 | 4 | 3 | 2 | 3 | 4 | |||
| Caseworker advised how to get medical care | 2 | 9 | .01 | 6 | 6 | 11 | 14 | ||
| Caseworker talked with you how to handle anger | 24 | 33 | 16 | 28 | .03 | 36 | 59 | .02 | |
| Caseworker advised you on substance abuse | 6 | 6 | 7 | 6 | 0 | 18 | .009 (FE) | ||
| Caseworker discussed with you how to get a better place | 8 | 8 | 7 | 4 | 11 | 18 | |||
| Caseworker advised on job training programs | 7 | 9 | 4 | 3 | 8 | 15 | |||
| Caseworker talked about how to get a paying job | 6 | 9 | 3 | 4 | 8 | 18 | |||
| Caseworker advised on how to continue school | 5 | 6 | 5 | 4 | 17 | 22 | |||
| Caseworker arranged for some childcare | 1 | 0 | 2 | 2 | 0 | 1 | |||
| Caseworker told you about other agencies | 14 | 24 | .05 | 30 | 41 | .06 | 8 | 30 | .01 |
| Caseworker talked with you about family planning | 9 | 16 | 5 | 7 | 6 | 7 | |||
| Note: "FE" indicates significance determined by Fisher's exact test C = Control Group, E = Experimental Group | |||||||||
item nearly significant, information about other agencies (p = .06). In Tennessee, there were seven activities significantly more often reported by experimental group caretakers: help with money for other things, transportation, talk about discipline, advice on substance abuse, help with relations with spouse, talk about how to handle anger, and information about other agencies. Differences between the groups were not as great as those reported for the treatment period, as is to be expected, since the treatment did not continue during this period.
Participation in Social Programs. As indicated in Table 2-10, there were no significant differences between the experimental and control groups in any state in involvement in social programs during the post-treatment period.
Caretaker Report of Services. Table 2-11 indicates that there was only one service in the three states on which there was a significant difference between groups in receipt post-treatment; in Tennessee more control group respondents reported having a parent aide. In Kentucky and New Jersey, the proportions of the two groups receiving each service are remarkably similar. Control group families in Tennessee more often received a couple of other services, but the differences were not significant. For the most part, the superiority of the control group in Tennessee in receipt of services observed at the post-treatment interview dissipated at the time of the followup interview.
Summary of Post-treatment Services. A summary of the significant differences between experimental and control groups on report of services at the followup caretaker interview is shown in Table 2-12. In the questions about caseworker activities, there is some indication that experimental group families received more services during the post-treatment period. Since caretakers were asked about the period of time following the last interview, we assume that for experimental group respondents the activities were undertaken by workers other than family preservation workers, perhaps workers in the public agency or workers in other private agency programs to which they might have been referred. Hence, the data may be taken as indicating receipt of somewhat more services by the experimental group families after the end of family preservation services, in accordance with the goal of these programs to connect families with ongoing services. However, this finding was not confirmed by data on social programs or services. It is possible that the finding also reflects something that we have often heard from public agency workers working with family preservation programs, that the family preservation
| Kentucky | New Jersey | Tennessee | |||||||
|---|---|---|---|---|---|---|---|---|---|
| C % | E % | p | C % | E % | p | C % | E % | p | |
| Food stamps | 50 | 61 | .10 | 49 | 45 | 56 | 54 | ||
| Job training | 7 | 13 | .10 | 10 | 5 | .07 | 11 | 12 | |
| WIC | 24 | 31 | 21 | 18 | 28 | 34 | |||
| AFDC | 34 | 39 | 37 | 39 | 31 | 34 | |||
| Housing vouchers | 13 | 13 | 16 | 18 | 19 | 18 | |||
| Social security disability | 32 | 32 | 27 | 27 | 19 | 36 | .07 | ||
| Alcoholism program | 5 | 6 | 10 | 8 | 6 | 8 | |||
| Drug treatment program | 3 | 4 | 10 | 11 | 3 | 9 | |||
| Marriage counseling | 4 | 4 | 5 | 6 | 3 | 3 | |||
| Community mental health program | 6 | 7 | 29 | 32 | 19 | 22 | |||
| Head Start/pre-school | 29 | 35 | 41 | 45 | 50 | 60 | |||
| Note: C = Control Group, E = Experimental Group | |||||||||
| Kentucky | New Jersey | Tennessee | |||||||
|---|---|---|---|---|---|---|---|---|---|
| C | E | p | C | E | P | C | E | p | |
| % | % | % | % | % | % | ||||
| Daycare | 13 | 12 | 14 | 13 | 31 | 33 | |||
| Help in finding a place to live | 2 | 2 | 2 | 2 | 14 | 5 | |||
| Staying at an emergency shelter | 2 | 1 | 2 | 3 | 11 | 3 | .09 (FE) | ||
| Medical or dental care | 6 | 7 | 64 | 62 | 17 | 19 | |||
| Transportation | 13 | 17 | 13 | 14 | 9 | 18 | |||
| Education services/GED | 3 | 2 | 3 | 4 | 9 | 8 | |||
| Parent education/training classes | 14 | 13 | 7 | 8 | 17 | 16 | |||
| Legal services | 6 | 6 | 14 | 18 | 3 | 1 | |||
| Counseling | 50 | 48 | 52 | 57 | 19 | 25 | |||
| Respite care | 1 | 2 | 2 | 4 | 0 | 0 | |||
| Homemaker services | 1 | 2 | 6 | 5 | 6 | 5 | |||
| A parent aide to help you | 1 | 3 | 3 | 2 | 14 | 3 | .04 (FE) | ||
| Family planning services | 2 | 4 | 2 | 4 | 3 | 1 | |||
| Note: "FE" indicates significance determined by Fisher's exact test Control = Control Group, E = Experimental Group | |||||||||
| Caseworker Activities: | Kentucky | New Jersey | Tennessee | ||||||
|---|---|---|---|---|---|---|---|---|---|
| Proportion of affirmative answers to yes/no questions | C % | E % | p | C % | E % | p | C % | E % | p |
| Caseworker helped with money for rent, electricity, phone | 1 | 8 | .008 | 4 | 5 | 8 | 13 | ||
| Caseworker helped with money for other things | 8 | 16 | .05 | 8 | 11 | 14 | 37 | .01 | |
| Caseworker provided transportation | 11 | 23 | .01 | 11 | 18 | 17 | 42 | .007 | |
| Caseworker talked with you about discipline | 24 | 32 | 24 | 34 | .08 | 42 | 62 | .05 | |
| Caseworker talked with you on relationship with spouse | 4 | 9 | 8 | 8 | 19 | 39 | .04 | ||
| Caseworker helped you clean house | 1 | 1 | 0 | 5 | .03 (FE) | 8 | 10 | ||
| Caseworker talked with you about how to handle anger | 24 | 33 | 16 | 28 | .03 | 36 | 59 | .02 | |
| Caseworker advised you on substance abuse | 6 | 6 | 7 | 6 | 0 | 18 | .009 (FE) | ||
| Caseworker told you about other agencies | 14 | 24 | .05 | 30 | 41 | .06 | 8 | 30 | .01 |
| Kentucky | New Jersey | Tennessee | |||||||
|---|---|---|---|---|---|---|---|---|---|
| C Mean | E Mean | p | C Mean | E Mean | p | C Mean | E Mean | p | |
| CT report of # of caseworker activities | .97 | 1.65 | .01 | 1.0 | 1.3 | 1.6 | 3.3 | .002 | |
| Services Provided: | Kentucky | New Jersey | Tennessee | ||||||
|---|---|---|---|---|---|---|---|---|---|
| Proportion of affirmative answers to yes/no questions | C | E | p | C | E | p | C | E | p |
| % | % | % | % | % | % | ||||
| Anyone been in job training program | 7 | 13 | .10 | 10 | 5 | .07 | 11 | 12 | |
| Anyone receive a parent aide to help you | 1 | 3 | 3 | 2 | 14 | 3 | .04(FE) | ||
| Were any needed services not gotten | 22 | 9 | .006 | 48 | 38 | .10 | 44 | 32 | |
| Note: C = Control Group, E = Experimental Group. "FE" indicates significance determined by Fisher's exact test. Tables only include items with a primary p-value less than .05 in at least one of the states; p-values greater than .10 are not reported. Items in bold indicate significant findings in favor of the experimental group whereas italicized items indicate significant findings in favor of the control group. | |||||||||
involvement gave them more information about the family and enabled them to plan better for services after family preservation.
10. The results of the secondary analyses show slightly greater differences between the experimental and control groups. Here, 8 of the 19 items show significant differences in favor of the experimental group at p = .01 or lower, and an additional 2 items show significant differences in the same direction at p = .05 or lower. See Appendix H-1.
11. These differences were even larger when violations and minimal service cases were excluded from the analyses (4.7 vs. 2.1, ns of 138 and 109, p = .001).
12. Differences were even greater when the violations and minimal service cases were excluded (3.57 vs. 1.90, ns of 181 and 115, p = .0001).
13. In the secondary analysis, the experimental group had an average of 4.99 activities, compared to 2.88 for the control group.
14. When violations and minimal service cases are excluded from the analyses, the differences remained significant and most p-values decreased even further. The secondary analyses showed a significantly greater proportion of the experimental group caretakers report attendance at parent education classes (p = .04). See Appendix H-3.
15. In the secondary analysis (dropping violations and minimal service cases) there was a difference on one item: experimental group caretakers more often received counseling (59% vs. 46%, p = .03). See Appendix H-3.
16. When violations and minimal services cases were excluded, the difference between the groups was even larger (4.6 vs. 2.0, p = .0001).
17. In the secondary analyses, excluding violations and minimal service cases, 13 services were provided significantly more often to the experimental group than to the control group (again, significance levels were all at p = .05 or less). In addition to the 11 primary analysis items showing differences in favor of the experimental group (ten significant items plus the category of "other"), these were: food stamps (p = .01) , and family planning (p = .04). Again, in the secondary analyses, self-help groups was the only service provided significantly more often to the control group than the experimental group. See Appendix H-5.
18. The difference between groups was even larger when violations and minimal service cases were excluded: 3.8 vs. 2.0 (p < .0001).
19. Excluding violations and minimal service cases, only one service was provided significantly more often to the control group, emergency shelter, while nine services were significantly more often provided to the experimental group, the above six plus legal aid, emergency financial assistance, and self help groups.
20. The difference between groups was even larger when violations and minimal service cases were excluded: 3.4 vs. 1.4 (p < .0001).
21. In the secondary analysis, there was one additional service provided significantly more often to the experimental group: household management.
22. Jill Kinney, David Haapala, and Charlotte Booth, (1991). Keeping Families Together: The Homebuilders Model, New York: Aldine de Gruyter.
3. The Outcomes
This chapter describes the outcomes of the programs in Kentucky, New Jersey, and Tennessee, the three Homebuilders states in the evaluation. Outcomes for Philadelphia are presented in Chapter 4 of this volume.
The outcomes we examined were the placement of children in substitute care following random assignment to the experimental or control group, subsequent reports of maltreatment and a number of measures of child and family functioning. The focus is on comparisons between the experimental and control groups. Analyses we have designated as "primary" were conducted on all randomly assigned cases except those that were determined to be inappropriate referrals. This includes cases in which the assignment was violated (cases assigned to the control group that were given family preservation services) and cases assigned to family preservation that received no or little such service. Insofar as family preservation services have effects, inclusion of these cases in the analysis will tend to reduce the observed differences between the groups. However, the most rigorous approach to analysis requires that we retain these cases in the group to which they were assigned in order to maintain the statistical equivalence of the groups at the outset of the experiment, which is the reason for random assignment in the first place.
It is likely that violations and minimal service cases differ in systematic ways from other cases (perhaps not detected in the measurements of the study), hence, switching them to the other group would result in groups that were not equivalent at the beginning. It can be argued that inclusion of minimal service cases in the experimental group is quite proper on other grounds: the implementation of any program will involve some cases that do not receive the service, and estimates of impact ought to take that into account. We did conduct analyses ("secondary" analyses) in which the violations and minimal service cases were dropped, so as to examine differences between cases that actually received the intended treatment (family preservation or regular services). This analysis must be viewed as only suggestive, since it does not preserve the initial statistical equivalence of the groups created by random assignment. In fact, the results of the secondary analyses were usually similar to those of the primary analysis. We note in footnotes when the secondary analysis differed substantially from the primary analysis. The secondary analysis tables are presented in Appendix I, Volume 3.
Some analyses were also conducted on a more "refined" sample in which we attempted to focus on cases that approached a conception of "ideal" family preservation cases. Family preservation services are designed for families in crisis, presumably the crisis surrounding a recent allegation of maltreatment, the investigation of that allegation, and the threat of removal of a child. Theoretically, this state of crisis makes families more willing to seek and respond to help. As indicated in Chapter 7 in Volume One of the report, many of the families did not appear to conform to this specification of the target group. In Kentucky and New Jersey we looked at two subgroups of cases, those with a recent substantiated allegation recorded in the administrative data (within three months prior to referral to family preservation services) and those in which an investigative worker was involved. In Tennessee, nearly all of the cases came from investigating workers, so we looked at those cases with an allegation within 30 days prior to referral.
3.1 Substitute Care Placement Following Random Assignment
A principal goal of family preservation services is the prevention of placement into substitute care, so that must be the first (though not the last) outcome examined. Placement included foster care, institutions and residential treatment programs, group homes, and adoptive placements. (23) We are initially concerned with the character and timing of the first placement of a child following random assignment. We collected data on placement prior to November 30, 1999 in Kentucky, September 30, 1999 in New Jersey, and August 31, 1999 in Tennessee. (24) Although data were provided at the individual level, most of the analyses are presented at the family level. (25) In Kentucky, the administrative files contained data on 1130 children in 345 families, 172 in the experimental group and 171 families in the control group. One hundred thirty-nine children in 61 families (36%) in the experimental group experienced placement subsequent to random assignment compared to 96 children in 55 families (32%) in the control group. In New Jersey, administrative data were available on 1290 children in 442 families, 275 in the experimental group and 167 in the control group. One hundred sixty-six children in 109 families (40%) in the experimental group were placed compared to 55 children in 48 families (29%) in the control group.
In Tennessee, multiple sources of data were used to calculate the rate of subsequent placement. A statewide management information system (CORS) provided information on formal paid placements. Additionally, case record reviews provided information on unpaid relative placements. In Tennessee, placement data were available on 468 children in 140 families, 93 in the experimental group and 47 in the control group. In the analysis of CORS data, forty-six children in 23 families (25%) in the experimental group experienced placement subsequent to random assignment compared to 25 children in 10 families (21%) in the control group. Including unpaid relative placements, 60 children in 29 families (31%) in the experimental group experienced placement subsequent to random assignment compared to 31 children in 13 families (28%) in the control group. These differences were not statistically significant at the family level in Kentucky, New Jersey or Tennessee (see Table 3-1 for types of placements after random assignment).
A comparison of these percents is, however, misleading, because of varying periods of risk of placement. The proper approach to the analysis of such data is survival analysis, in which the proportions of cases placed at each point in time following random assignment in each group are compared, accounting for the numbers of cases that "survive" to that point. We examined survival curves for each group and determined whether these curves were statistically different. Family level analyses were based on the first date of placement of any child in the family if a placement occurred.
| Kentucky | ||
|---|---|---|
| Type | N | % |
| Foster care | 144 | 64.0 |
| Private institution | 69 | 30.7 |
| Foster care, medically fragile | 6 | 2.7 |
| Child psychiatric hospital | 4 | 1.8 |
| Not specified | 2 | 0.8 |
| Total | 225 | 100 |
| New Jersey | ||
|---|---|---|
| Type | N | % |
| Foster care | 102 | 46.1 |
| Juvenile family crisis | 47 | 21.2 |
| Residential treatment | 35 | 15.9 |
| Group home | 17 | 7.7 |
| Public institution | 8 | 3.6 |
| Shelter care | 5 | 2.3 |
| Adoptive | 4 | 1.8 |
| Relative | 3 | 1.4 |
| Total | 221 | 100 |
| Tennessee | ||
|---|---|---|
| Type | N | % |
| Foster care | 31 | 44.3 |
| Relative home | 9 | 12.9 |
| Trial home | 6 | 8.5 |
| Residential | 6 | 8.5 |
| Continuum contract | 4 | 5.7 |
| Non-relative home | 4 | 5.7 |
| Adoptive home | 3 | 4.3 |
| Runaway | 2 | 2.8 |
| Shelter | 2 | 2.8 |
| Independent living | 2 | 2.8 |
| Detention | 1 | 1.4 |
| Total | 71 | 100 |
| Note: Includes only placements recorded in administrative data. There were additional unpaid relative placements (see text). | ||
Kentucky. The family level analysis of subsequent placement is displayed in Figure 3-1. (26) These survival curves show the proportion of families remaining intact (without placement of a child) at each point in time following random assignment. The curves begin at 1, indicating that at the time of random assignment, all children were at home. The curves then decline as children enter care. The higher curve at any point represents the group with fewer placed children at that point. The curves are adjusted for cases that are "right censored." For example, cases that were not observed for a full year following random assignment are dropped in the calculation of the percentage remaining intact ("surviving") at one year. The Wilcoxon statistic indicates that the survival rates for the experimental and control groups are not statistically different. At the one-year interval, 25 percent of experimental group families and 24 percent of control group families experienced substitute care placement. At the end of two years, 32 percent of the experimental group and 27 percent of the control group families experienced substitute care placement.
"Refined" groups analyses were also conducted, limiting the sample to cases referred by investigative workers and to those families with substantiated allegations within the three months prior to random assignment. Significant differences did emerge for families with a substantiated allegation within three months prior to random assignment. In the primary analysis of those families coming from an investigative worker, 26 percent of the experimental group and 15 percent of the control group experienced subsequent placement within one year after the random assignment date. For those with recent substantiated allegations, 29 percent of the experimental group and 13 percent of the control group experienced subsequent placement within one year (significant at .05 level).
An additional "refined" group was available for analysis in Kentucky. Prior to random assignment, workers submitted petitions to the court for placement or some other court ordered intervention on 66 families. Administrative data were available for all 66 families (32 in the experimental group, 34 in the control group). Survival analyses were conducted to explore the relationship between family preservation services and subsequent placement. At one year after random assignment, 22 percent of the experimental group and 29 percent of the control group experienced placement; a nonsignificant difference.
Figure 3-1
First Placement after Random Assignment (Families)
In addition to the administrative data on placement, in Kentucky the Westat site coordinator attempted to document all placements subsequent to random assignment, based on her contacts with caseworkers. The administrative data file contained placements not recorded by the site coordinator, and vice versa. The only systematic difference between these data sources was the documentation of relative placement. Relatives are generally not paid for placements in Kentucky, so these data were not recorded in the administrative files. Survival analyses were conducted with a combination of caseworker and administrative placement records. If either data source recorded a placement event, that family was coded as experiencing subsequent placement. The first documented date of placement, taken from either source, was selected for analysis. The patterns of placement in these analyses are similar to those reported above. At one year, 27 percent of the experimental group and 32 percent of the control group families experienced placement, a nonsignificant difference.
In addition to survival analyses, placement can be examined in terms of the proportion of time in substitute care subsequent to random assignment. If family preservation services are effective in preventing placements, we would expect them to result in lower numbers of days in foster care. Family preservation might also result in shorter stays in care, once children are placed. Comparison of days in care provides a beginning look at the question of whether family preservation results in lower costs of foster care (of course, a complete cost-effectiveness analysis must also factor in the differential costs of family preservation and regular services).
The proportion of time in care is calculated by dividing the number of days in care by the number of days of possible care (number of days between random assignment and the date of administrative data collection). As the proportions are calculated at the family level, the number of days in care represents the total number of care days summed across all children within a particular family. Similarly, the number of possible care days represents the total number of possible care days summed across all children within a particular family. The number of possible care days is adjusted for a child's eighteenth birthday and for births since random assignment. For both primary and secondary analyses, in both the experimental and control groups children spent an average of 6 percent of the days subsequent to random assignment in care.
New Jersey. The family level analysis of placements is shown in Figure 3-1. (27) More families in the experimental group experienced placement of a child than in the control group (at one year, 29% of the experimental group vs. 22 percent of the control group; at two years, 39 percent of the experimental group vs. 28 percent of the control group) although the differences are not significant. It might be noted that in the analyses the survival curves for the two groups tend to begin to diverge at about 6-7 months, that is, at about that time more children in the experimental group are being placed. We do not have a ready explanation for this divergence.
Refined groups analyses in New Jersey revealed statistically significant differences. In the primary analysis of those families coming from an investigative worker, 25 percent of the experimental group and 15 percent of the control group experienced subsequent placement within one year of the random assignment date. For those with recent substantiated allegations, 25 percent of the experimental group and 14 percent of the control group experienced subsequent placement.
As to the proportion of time that children spent in care in New Jersey, experimental group children spent an average of 6 percent of that time in placement, compared to 5 percent for the control group children (not a significant difference).
Tennessee. Survival rates at the family level were first calculated using only the CORS and then including relative placement (the "any evidence" analysis) data. The family level analyses of subsequent placement is displayed in Figure 3-1. (28) The Wilcoxon statistic indicates that the survival rates for the experimental and control groups are not statistically different. In the analysis of CORS data, 23 percent of experimental group families and 19 percent of control group families experienced substitute care placement within one year subsequent to random assignment. In the "any evidence" analysis, 28 percent of the experimental group families and 23 percent of control group families experienced placement within one year subsequent to random assignment.
As in Kentucky and New Jersey, a "refined" group was available for analysis in Tennessee. Ninety-three families had an allegation within 30 days prior to random assignment. The Wilcoxon statistic for the survival analysis of placement in these families indicates that the survival rates of the two groups are not statistically different. In the analysis of CORS data, 17 percent of the experimental group and 15 percent of the control group experienced subsequent placement within one year of random assignment. In the "any evidence" analysis, 22 percent of the experimental group and 21 percent of the control group experienced subsequent placement within one year of random assignment.
As to the proportion of time that children spent in care in Tennessee, experimental group children spent an average of 10 percent of that time in placement, compared to 5 percent for the control group children. This difference is nonsignificant.
In a number of analyses of subsequent placement in these states, more experimental group families experienced placement than did control group families. In a few analyses, fewer experimental group families experienced placement. However, none of these analyses were statistically significant; in none of these states can the data be taken as firm evidence that family preservation resulted in more placements. Nor is there evidence that it resulted in fewer.
Imminent Risk of Placement. The family preservation programs in these states are designed to prevent the unnecessary removal of children by serving families with children who are at imminent risk of out-of-home placement. (29) One way to explore the accuracy of the "imminent risk" designation is to examine the proportion of control group families that experienced placement within a short time after random assignment. Since the control and experimental groups were randomly assigned and are expected to be statistically equivalent before services are begun, the proportion of families experiencing placement in the control group indicates the proportion of referred families that would have experienced placement in the absence of receiving family preservation services. We looked at control group placement rates 30 days after random assignment, believing that time period provided a liberal interpretation of "imminent risk." If a significant proportion of the control group experienced placement within 30 days of random assignment, one could argue that the program was appropriately targeted. At the time of random assignment, referring workers were asked to designate those children who were considered "at risk."
In Kentucky, in the first 30 days following random assignment, in the primary analysis 4 percent of at risk children in the experimental group were placed compared to 3 percent of control group at risk children. At the family level, 6 percent of the experimental group families and 5 percent of the control group families experienced placement within the first 30 days subsequent to random assignment. The percentages were similar in the investigative group (8% of the experimental compared with 5% of the control group), and among those with recent substantiated allegations (6% of the experimental group compared with 3% of the control group).
In New Jersey, of those children judged to be at risk, 4 percent of the control group and 3 percent of the experimental group were placed in 30 days. At the family level, 5 percent of the families in the experimental group experienced placement of at least one child within one month of random assignment, compared to 6 percent of the control group. Rates of imminent placement were similar in the "refined" group analyses. Of those families coming from an investigative worker, 3 percent of the experimental group and 5 percent of the control group experienced placement within 30 days of random assignment. For those families with a substantiated allegation within three months prior to random assignment, 8 percent of the experimental group and 5 percent of the control group experienced placement within 30 days.
In Tennessee, rates of placement within one month were somewhat higher than in Kentucky and New Jersey. Of those children judged to be at risk, 13 percent of the control group and 11 percent of the experimental group were placed in 30 days. There were no relative placements within the first 30 days subsequent to random assignment. Thus, there are no differences between the CORS and "any evidence" analysis. At the family level in Tennessee, the CORS administrative data indicates that 11 percent of both the experimental and control groups experience placement within 30 days subsequent to random assignment. Rates of imminent placement were similar in the "refined" group analyses. Of those families with a recent allegation (within 30 days prior to random assignment), 7 percent of the experimental group and 12 percent of the control group experienced a CORS placement within 30 days of random assignment.
Although the percentages of placement within one month were somewhat higher in Tennessee, in all three states, these percentages were quite low. The numbers of interest here are those for the control group, indicating the targeting efficiency of the program in these three sites is very low.
3.2 Hazard Analyses of Placement
Hazard analyses permit the examination of the effects of multiple independent variables (in addition to experimental group membership) on rates of placement. They also provide somewhat more precise estimates of the effect of experimental group membership, since they control for the effects of the other variables in examining experimental-control group differences. In addition, they allow for the examination of "interactions" between other variables and experimental-control group membership, to see if the effects of experimental group membership differ for subgroups of the sample. We conducted Cox regression analyses of placement hazards using as predictor variables case characteristics available in the administrative data. Case characteristics in the administrative data are quite limited. Unlike the survival analyses which were conducted at the family level, hazard analyses were done at the child level because we wanted to include in them characteristics of children.
Kentucky. We examined the effects of the child's age, race, prior placement, and prior substantiated allegation of maltreatment, together with experimental group membership, on rates of placement. Regarding main effects, there were no significant predictors of placement. However, there was a significant interaction between experimental group membership and prior placement. Examining the differences in placement rates between the experimental and control groups by whether or not the child had had a prior placement indicates that among those with a prior placement, there is little difference in placement rates (32% for the family preservation group and 34% for the control group) while there is a significant difference for those without prior placement (22% for the experimental group and 14% for the control group). The interactions between experimental group membership and age and prior substantiated allegations were not significant.
New Jersey. New Jersey hazard analyses indicate that older age and prior placement increase the hazard rate significantly (p < .05; prior placement by 88% and each year of age by 3%). Experimental group membership was also significant (p < .08; experimental group membership increases the hazard rate by 97%). The interactions between experimental group membership and race, age, prior placement, and prior substantiated allegations were not significant.
Hazard analyses were also performed to examine the effect of county on placement. These analyses were conducted at the family level. Burlington county was chosen as the reference category, as it had the highest rate of placement. Thus, rates of placement in the other New Jersey counties are compared to the placement rates of Burlington. In addition to the county variables, experimental group and interactions of county with experimental group were entered into the regression equation. The hazard of placement for families was decreased by 67 percent for Ocean county, 73 percent for Monmouth county, 47 percent for Essex county, 57 percent for Bergen county, and 74 percent for Passaic county. The coefficient associated with Camden county was non-significant. There were no significant effects of experimental group or of county-experimental group interactions. This indicates that even after removing county variation, there are no significant differences between the experimental and control groups, nor does the effect of experimental group vary by county.
Tennessee. We examined the effects of the child's age, race, prior placement, prior allegation within 30 days of random assignment, prior substantiated allegation within 30 days of random assignment, and experimental group membership on rates of placement. Similar to Kentucky and New Jersey, we also explored interactions between experimental group membership and child characteristics. No significant interactions emerged. Only prior substantiated allegation had a significant effect on the likelihood of placement subsequent to random assignment. In the analysis of the CORS administrative data, a substantiated allegation within the last 30 days prior to random assignment increased the hazard rate by 209 percent. When unpaid relative placements were included ("any evidence") prior substantiation increased the hazard by 173 percent.
3.3 Allegations of Maltreatment Following Random Assignment
Subsequent maltreatment of children is a second important outcome to be examined. Family preservation programs are intended to lower the risk of harm to children while keeping them at home, and subsequent maltreatment is an indicator of such risk. Furthermore, the justification for family preservation programs rests on the belief that the safety of children is not compromised when their families are referred to these programs, so examination of subsequent maltreatment rates is important to determine whether children, in fact, are safe in these programs.
As with placement, data on subsequent maltreatment come from the administrative data files of the states. As is almost always the case in studies like this, our data do not record actual maltreatment, but only investigated reports of maltreatment. Some abuse and neglect goes unreported, and, because not every report is investigated, there are cases of harm that are reported but not investigated.
As with the analyses of subsequent placement, survival graphs were developed to compare the timing of subsequent, substantiated allegations of maltreatment. (30) Again, survival analyses were conducted at the family level for both the primary and secondary analysis groups as well as for the "refinement" groups.
Kentucky. Two hundred twenty-three children in 99 families (58%) in the experimental group were the subjects of investigated allegations of maltreatment following random assignment, compared with 206 children in 100 families (58%) in the control group. The distribution of the various types of allegations is as follows: 2 percent dependency, 7 percent emotional, 63 percent neglect, 25 percent physical abuse, and 8 percent sexual maltreatment. As families can be the subjects of multiple allegations on any given day, these percentages do not sum to 100. Not all investigations result in substantiated allegations. One hundred forty-three children in 63 families (37%) in the experimental group were the subjects of substantiated allegations of maltreatment compared with 118 children in 57 families (33%) in the control group. The difference was not statistically significant at the family level. The distribution of substantiated allegations is as follows: 1 percent dependency, 4 percent emotional, 72 percent neglect, 20 percent physical abuse, and 3 percent sexual maltreatment.
Figure 3-2 displays the survival curves for substantiated allegations in the primary analysis. At one year subsequent to random assignment, 24 percent of the experimental group and 21 percent of the control group families experienced substantiated reports of maltreatment. Although a higher percentage of families in the "refined" analyses experienced substantiated allegations of maltreatment, similar patterns emerged. For the investigative group, 27 percent of the experimental group and 24 percent of the control group experienced a substantiated allegation of maltreatment in the one-year interval. For those families with a substantiated allegation within the three months prior to random assignment, 25 percent of the experimental and 21 percent of the control group experienced substantiated allegations of maltreatment within a year subsequent to random assignment. For the group on which petitions had been submitted to court for placement or other orders, 22 percent of the experimental group and 33 percent of the control group experienced a substantiated allegation within one year subsequent to random assignment, a nonsignificant difference.
Figure 3-2
First Substantial Allegation after Random Assignment (Families)
The survival analyses indicate that experimental and control group families had a very similar likelihood of substantiated reports of maltreatment subsequent to random assignment.
New Jersey. One hundred seventy-eight children in 94 families (34%) in the experimental group were the subjects of investigated allegations of maltreatment following random assignment, compared to 101 children in 60 families (36%) in the control group. Fifty-eight children in 34 families (12%) in the experimental group experienced a substantiated allegation of maltreatment following random assignment, compared to 42 children in 29 families (17%) in the control group. In none of the survival analyses conducted were there significant differences between the experimental and control groups. Figure 3-2 shows substantiated allegations at the family level. About 11 percent of families in both groups have substantiated allegations within one year.
Patterns of substantiated allegations were similar for the "refined" group analyses, none of which showed significant differences between groups. Of those families coming from an investigative worker, 7 percent of the experimental group and 10 percent of the control group had a substantiated allegation within one year subsequent to random assignment. For those families with a substantiated allegation within three months prior to random assignment, 10 percent of the experimental group and 16 percent of the control group had a substantiated allegation within one year subsequent to random assignment.
Tennessee. Allegation data were available for 482 children in 144 families. Sixty-four children in 36 families (38%) in the experimental group were the subjects of investigated allegations of maltreatment following random assignment, compared with 61 children in 26 families (54%) in the control group. The differences were not statistically significant at the family level. The distribution of the various types of allegations is: 66 percent physical abuse, 20 percent supervision/neglect, 2 percent sexual abuse/medical, and 12 percent other (includes allegations such as failure to thrive, truancy, and unruly child). Forty-four children in 25 families (26%) in the experimental group were the subjects of substantiated allegations of maltreatment compared with 42 children in 18 families (38%) in the control group. These differences were not statistically significant at the family level. The distribution of the various types of substantiated allegations is: 66 percent physical abuse, 20 percent supervision/neglect, 1 percent sexual abuse/medical, and 13 percent other.
Figure 3-2 displays the survival curves for substantiated allegations in the primary analysis. At one year subsequent to random assignment, 24 percent of the experimental group and 25 percent of the control group families experienced substantiated reports of maltreatment. Survival rates were also calculated for those families with an allegation within 30 days prior to random assignment. Significant differences emerged for subsequent allegations and near significant differences emerged for subsequent substantiated allegations. Of those families with a recent allegation, 28 percent of the experimental group and 52 percent of the control group experienced an allegation within one year subsequent to random assignment. Similarly, 18 percent of the experiment group and 30 percent of the control group experienced a substantiated allegation within one year subsequent to random assignment. These differences suggest that in Tennessee family preservation reduced the likelihood of subsequent maltreatment for those families with recent allegations.
3.4 Sub-group Analysis
In Kentucky and New Jersey, we examined a number of subgroups of cases to determine whether we could detect differences between experimental and control groups on placement and substantiated allegations subsequent to random assignment within each subgroup. The number of cases in Tennessee was not sufficient to support subgroup analysis. The results are shown in Table 3-2. Most of the subgroups were defined in terms of problems existing at the time of the initial interview. For both placement and substantiated allegations the table shows the number of cases in each subgroup and the percentage of cases in the subgroup experiencing the event within 12 months. The analysis involved determining the significance of the difference between the experimental and control groups in the occurrence of the event within twelve months, and in the survival curves as a whole. (31) The first row of the table shows the results for the Kentucky and New Jersey samples as a whole. Except for substance abuse, the definitions of the subgroups were taken from the initial caretaker interview. Very few caretakers acknowledged substance use in the first interview, so that subgroup was determined from information in both the caretaker and caseworker initial interviews.
None of the 36 experimental-control group comparisons were significant at the .05 level. In the analysis so far, efforts to find subgroups for which family preservation service was related to reduced placement have been unsuccessful.
3.5 Case Closing Subsequent to Random Assignment
Family preservation services are sometimes thought to lead to quicker case closings in the public agency and less frequent subsequent involvement with the child welfare agency. Administrative data on case closings and subsequent case openings were examined to determine the effects of these services on case closings and subsequent reopenings.
Kentucky. Of the 255 cases that were open in the public agency at the time of the referral to family preservation services, 180 (71%) were closed some time after the referral and 75 (30%) remained open as of November 30, 1999 (the last date of observation for these analyses). Survival analyses were performed to examine the lengths of time between the referral to family preservation services and the first closing of the case. As shown in Figure 3-3, significant differences were found between the experimental and control groups.
Of the 180 cases that were open at the time of the referral to family preservation services and closed some time after that referral, 10 cases were re-opened again before November 30, 1999. Five of these 10 cases were in the experimental group, and five were in the control group.
| Subgroup | Kentucky | New Jersey | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Placement | Substantiated allegations | Placement | Substantiated allegations | |||||||||||
| In 12 mos | In 12 mos | In 12 mos | In 12 mos | |||||||||||
| N | Overall % | pa | Survival p | Overall % | pa | Survival p | N | Overall % | pa | Survival p | Overall % | pa | Survival p | |
| Overall | 345 | 25 | 23 | 442 | 27 | .09b | 11 | |||||||
| Substance abuse | 37 | 29 | 29 | 53 | 26 | 20 | ||||||||
| No substance abuse | 244 | 23 | 19 | 326 | 28 | 12 | ||||||||
| Problems with bills | 157 | 26 | 25 | 195 | 27 | 13 | ||||||||
| Problems with daycare | 99 | 22 | 25 | 111 | 26 | 14 | ||||||||
| Depressionc | 152 | 26 | 24 | 168 | 33 | 11 | ||||||||
| Problems with punishment | 205 | 26 | 23 | 262 | 28 | 11 | ||||||||
| Problems with school | 145 | 23 | 22 | 202 | 31 | 11 | ||||||||
| Problems with employment | 193 | 24 | 22 | 192 | 26 | 14 | ||||||||
| Single mother | 129 | 21 | 19 | 116 | 26 | 17 | ||||||||
| a Fisher exact, two tail b Experimental group more likely to experience placement c Caretakers with depression scores above median for the state | ||||||||||||||
Figure 3-3
First Case Closing after Random Assignment (Families)
New Jersey. Of the 441 cases with administrative case closing data, 427 were open at the time of random assignment. Of these 427 cases, 263 (62%) were closed subsequent to the referral to family preservation services. Figure 3-3 shows the results of a survival analysis in which we examined the time to case closing for the 421 cases that were open at the time of the referral to family preservation services. There was no significant difference between the experimental and control groups in the rate or timing of case closings after the referral date. Of the 263 cases that were closed after the referral to family preservation services, 81 cases (31%) were reopened. There was no significant difference between the experimental and control groups in the proportion of cases that were reopened (32% in the experimental group, 29% in the control group).
Tennessee. Of the 147 families with case opening and closing data, 111 were open in the public agency at the time of the referral to family preservation services. Of these 111 families, 96 (87%) were closed some time after the referral and 15 (14%) remained open as of August 31, 1999 (the last date of observation for these analyses). Survival analyses were performed to examine the lengths of time between the referral to family preservation services and the first closing of the case. As indicated by the survival curves in Figure 3-3, no significant differences were found between the experimental and control groups. Of the 96 cases that were open at the time of the referral to family preservation services and closed some time after that referral, 17 cases were re-opened again before August 31, 1999. There was a significant difference in the rate of reopening. Eight (12%) of the 66 experimental group cases that were closed subsequently reopened, compared with 9 (30%) of the 30 control group cases (p < .05).
3.6 Family and Child Functioning-Caretaker Interviews
Family preservation services are intended to result in improved functioning of children and families. This goal is sought both for its own intrinsic value as well as an intermediate objective in the prevention of subsequent maltreatment and placement; parents who are functioning better and better parent-child relationships should result in lower risk of abuse or neglect.
In our interviews with caretakers and caseworkers we asked a number of questions tapping various aspects of functioning. We asked most of these questions in all three interviews with caretakers (at the beginning of service, four to six weeks after service began, and one year after the beginning of service) and in the two interviews with caseworkers (at the beginning of service and four to six weeks later). In the initial interview, we usually asked respondents to answer in terms of circumstances in the last three months. In the post-treatment and followup interviews, we asked in terms of "since we last talked to you [at the time of the initial interview or the post-treatment interview]." To indicate the effects of family preservation services, we can compare the experimental and control groups on the responses to these questions in the second and third interviews and on change between interviews. We report on the responses to a number of individual items in our interviews. In addition, we combined the responses to many questions into summated scales. (32) We examined differences between experimental and control groups in each state in the average levels of these scales at post-treatment and at followup and we examined changes over time in these averages using multivariate repeated measures analysis. (33) The results of the analyses of the scales are shown in Tables 3-3, 3-4, and 3-5 and in Figure 3-4. (34)
| Post-Treatment | followup | Multivariate Repeated Measures | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Means | Multivariate ps | Univariate ps-time | Univariate ps-Grp-time interaction | |||||||||||||||
| N | Ma | pb | N | M | p | N | Initial | Post | Follow | Grpc | Timed | Time -Grpe | Initial v. laterf | Post v. Followg | Initial v. later | Post v. Follow | ||
| Positive life events | C | 146 | .12 | 119 | .22 | 108 | .16 | .12 | .21 | .42 | .001 | .41 | .001 | |||||
| E | 148 | .14 | 130 | .21 | 117 | .18 | .14 | .20 | ||||||||||
| Negative life events | C | 146 | .04 | 119 | .07 | 108 | .08 | .03 | .07 | .40 | .001 | .27 | .001 | .001 | ||||
| E | 148 | .03 | 130 | .09 | 117 | .10 | .03 | .09 | ||||||||||
| Life events depression | C | 145 | .35 | 119 | .37 | 107 | .45 | .36 | .38 | .40 | .001 | .38 | .001 | |||||
| E | 147 | .36 | 130 | .40 | 117 | .50 | .37 | .42 | ||||||||||
| Economic functioning | C | 142 | .17 | 118 | .17 | 105 | .22 | .15 | .18 | .08 | .001 | .27 | .001 | |||||
| E | 144 | .22 | 127 | .20 | 111 | .32 | .23 | .20 | ||||||||||
| Punishment | C | 143 | .16 | 113 | .15 | 101 | .22 | .17 | .15 | .49 | .001 | .36 | .001 | .09 | ||||
| E | 147 | .17 | 121 | .15 | 109 | .25 | .18 | .15 | ||||||||||
| Child aggression | C | 146 | 1.36 | 119 | 1.24 | 108 | 1.56 | 1.44 | 1.29 | .84 | .001 | .96 | .001 | .05 | ||||
| E | 148 | 1.33 | 130 | 1.29 | 117 | 1.57 | 1.44 | 1.32 | ||||||||||
| School problems | C | 112 | .22 | 96 | .25 | 78 | .29 | .24 | .27 | .14 | .03 | .41 | .01 | |||||
| E | 101 | .20 | 97 | .19 | 77 | .26 | .21 | .18 | ||||||||||
| Child withdrawn | C | 146 | .89 | 119 | .99 | 108 | 1.08 | .89 | 1.00 | .99 | .001 | .29 | .001 | |||||
| E | 148 | .93 | 130 | .93 | 117 | 1.08 | .97 | .93 | ||||||||||
| Stolen things or arrested | C | 146 | .34 | 119 | .39 | 108 | .31 | .34 | .39 | .76 | .75 | .07 | .03 | |||||
| E | 148 | .32 | 130 | .31 | 117 | .44 | .35 | .32 | ||||||||||
| Child substance abuse | C | 146 | .07 | 119 | .03 | 108 | .05 | .07 | .04 | .40 | .35 | .67 | ||||||
| E | 148 | .04 | 130 | .02 | 117 | .03 | .03 | .03 | ||||||||||
| Child problems | C | 146 | 2.20 | 119 | 1.89 | 108 | 2.44 | 2.23 | 2.18 | .87 | .001 | .47 | .001 | |||||
| E | 148 | 2.05 | 130 | 2.04 | 117 | 2.56 | 2.14 | 2.07 | ||||||||||
| Negative child behaviors | C | 140 | .34 | 107 | .33 | 98 | .37 | .34 | .34 | .95 | .001 | .47 | .001 | |||||
| E | 139 | .34 | 120 | .34 | 106 | .39 | .34 | .33 | ||||||||||
| Positive child behaviors | C | 142 | .71 | 109 | .67 | 99 | .70 | .69 | .67 | .78 | .31 | .84 | ||||||
| E | 142 | .71 | 121 | .68 | 109 | .69 | .69 | .67 | ||||||||||
| Household condition | C | 142 | .02 | 119 | .02 | 102 | .05 | .02 | .02 | .29 | .002 | .45 | .001 | |||||
| E | 147 | .02 | 129 | .01 | 111 | .03 | .01 | .01 | ||||||||||
| Depression (SCL-90) | C | 145 | .79 | 119 | .67 | 107 | .95 | .77 | .70 | .67 | .001 | .31 | .001 | |||||
| E | 146 | .74 | 130 | .79 | 115 | .96 | .74 | .83 | ||||||||||
| Positive child care practices | C | 140 | .85 | 107 | .82 | 94 | .87 | .85 | .83 | .55 | .09 | .97 | .06 | |||||
| E | 143 | .82 | 116 | .81 | 103 | .85 | .84 | .81 | ||||||||||
| Negative child care practices | C | 141 | .14 | 109 | .12 | 97 | .17 | .15 | .12 | .57 | .001 | .22 | .001 | .05 | ||||
| E | 144 | .13 | 117 | .13 | 104 | .20 | .14 | .13 | ||||||||||
| a Means of control and experimental groups b Test of hypothesis of equivalent group means c Test of hypothesis that group means, averaged over time, are equal d Test of hypothesis that means at three points in time, averaged over the groups, are equal e Test of hypothesis of no interaction between group and time, that is, that the pattern of means over time is the same for both groups f Test of hypothesis that time one is equal to average of time two and time three g Test of hypothesis that time two is equal to time three | ||||||||||||||||||
| Post-Treatment | followup | Multivariate Repeated Measures | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| N | Ma | pb | N | M | p | N | Means | Multivariate ps | Univariate ps-Time | Univariate ps-Grp-time interaction | ||||||||
| Initial | Post | Follow | Grpc | Timed | Time -Grpe | Initial v. laterf | Post v. Followg | Initial v. later | Post v. Follow | |||||||||
| Positive life events | C | 133 | .15 | 107 | .23 | 83 | .19 | .16 | .25 | .05 | .001 | .99 | .001 | |||||
| E | 210 | .13 | 166 | .21 | 129 | .17 | .13 | .22 | ||||||||||
| Negative life events | C | 133 | .04 | 107 | .09 | 83 | .13 | .04 | .10 | .46 | .001 | .49 | .001 | .001 | ||||
| E | 210 | .05 | 166 | .11 | 129 | .11 | .04 | .09 | ||||||||||
| Life events depression | C | 133 | .42 | 106 | .47 | 83 | .48 | .38 | .44 | .93 | .001 | .33 | .001 | |||||
| E | 210 | .39 | 165 | .42 | 128 | .52 | .39 | .41 | ||||||||||
| Economic functioning | C | 132 | .34 | .02 | 107 | .36 | 83 | .39 | .33 | .33 | .07 | .06 | .31 | .06 | ||||
| E | 209 | .25 | 167 | .31 | 129 | .29 | .24 | .30 | ||||||||||
| Punishment | C | 131 | .25 | .04 | 105 | .21 | 80 | .31 | .25 | .22 | .03 | .001 | .93 | .001 | .03 | |||
| E | 209 | .20 | 167 | .18 | 129 | .27 | .20 | .17 | ||||||||||
| Child aggression | C | 134 | 1.68 | .09 | 107 | 1.38 | 84 | 1.89 | 1.77 | 1.37 | .05 | .001 | .26 | .001 | .001 | |||
| E | 210 | 1.48 | 167 | 1.32 | 130 | 1.62 | 1.44 | 1.28 | ||||||||||
| School problems | C | 118 | .22 | 96 | .34 | 69 | .36 | .26 | .36 | .009 | .001 | .84 | .01 | .003 | ||||
| E | 197 | .20 | 157 | .28 | 121 | .29 | .19 | .26 | ||||||||||
| Child withdrawn | C | 134 | .40 | 107 | .41 | 84 | .62 | .42 | .37 | .38 | .001 | .51 | .001 | |||||
| E | 210 | .40 | 167 | .38 | 130 | .50 | .41 | .33 | ||||||||||
| Stolen things or arrested | C | 134 | .37 | 107 | .42 | 84 | .58 | .35 | .44 | .60 | .001 | .99 | .001 | .05 | ||||
| E | 210 | .29 | 167 | .48 | 130 | .55 | .31 | .41 | ||||||||||
| Child substance abuse | C | 134 | .16 | 107 | .20 | 84 | .26 | .13 | .18 | .94 | .01 | .71 | .02 | .10 | ||||
| E | 210 | .17 | 167 | .27 | 130 | .23 | .14 | .22 | ||||||||||
| Child problems | C | 134 | 2.64 | 107 | 2.88 | 84 | 3.00 | 2.70 | 2.99 | .57 | .001 | .45 | .003 | .09 | ||||
| E | 210 | 2.64 | 167 | 2.83 | 130 | 3.03 | 2.62 | 2.73 | ||||||||||
| Negative child behaviors | C | 130 | .33 | .04 | 105 | .34 | 81 | .41 | .34 | .35 | .005 | .001 | .84 | .001 | .001 | |||
| E | 207 | .28 | 163 | .32 | 126 | .35 | .28 | .30 | ||||||||||
| Positive child behaviors | C | 132 | .74 | 106 | .75 | 83 | .73 | .75 | .77 | .63 | .01 | .69 | .02 | .02 | ||||
| E | 208 | .73 | 163 | .76 | 128 | .72 | .73 | .77 | ||||||||||
| Household condition | C | 134 | .06 | 107 | .04 | 84 | .06 | .06 | .03 | .78 | .05 | .38 | .04 | |||||
| E | 210 | .05 | 167 | .05 | 129 | .06 | .05 | .04 | ||||||||||
| Depression (SCL-90) | C | 134 | 1.00 | .08 | 105 | .85 | 83 | 1.01 | .84 | .82 | .92 | .001 | .71 | .001 | ||||
| E | 209 | .83 | 166 | .82 | 127 | 1.04 | .89 | .77 | ||||||||||
| Positive child care practices | C | 128 | .76 | 103 | .76 | 77 | .79 | .77 | .77 | .43 | .11 | .67 | .06 | |||||
| E | 206 | .77 | 163 | .80 | 124 | .82 | .78 | .79 | ||||||||||
| Negative child care practices | C | 129 | .18 | .02 | 101 | .14 | 76 | .21 | .18 | .14 | .07 | .001 | .34 | .001 | .06 | |||
| E | 207 | .14 | 162 | .13 | 124 | .19 | .13 | .13 | ||||||||||
a Means of control and experimental groups | ||||||||||||||||||
| Post-Treatment | followup | Multivariate Repeated Measures | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| N | Ma | pb | N | M | p | N | Means | Multivariate ps | Univariate ps-Time | Univariate ps-Grp-time interaction | ||||||||
| Initial | Post | Follow | Grpc | Timed | Time -Grpe | Initial v. laterf | Post v. Followg | Initial v. later | Post v. Follow | |||||||||
| Positive life events | C | 37 | .14 | 36 | .31 | 28 | .20 | .15 | .32 | .53 | .001 | .76 | .001 | |||||
| E | 80 | .18 | 74 | .31 | 61 | .24 | .19 | .31 | ||||||||||
| Negative life events | C | 37 | .05 | 36 | .14 | 28 | .14 | .06 | .15 | .13 | .001 | .77 | .10 | .001 | ||||
| E | 80 | .05 | 74 | .11 | 61 | .10 | .04 | .12 | ||||||||||
| Life events depression | C | 31 | .36 | 36 | .35 | 28 | .49 | .40 | .34 | .85 | .001 | .66 | .001 | |||||
| E | 80 | .34 | 74 | .34 | 61 | .50 | .35 | .34 | ||||||||||
| Economic functioning | C | 37 | .25 | 36 | .33 | 28 | .43 | .25 | .31 | .06 | .001 | .86 | .001 | |||||
| E | 80 | .18 | 73 | .22 | 59 | .31 | .16 | .19 | ||||||||||
| Punishment | C | 37 | .13 | 36 | .10 | 28 | .21 | .12 | .11 | .45 | .001 | .11 | .001 | .09 | .10 | |||
| E | 76 | .13 | 71 | .07 | 54 | .28 | .15 | .08 | ||||||||||
| Child aggression | C | 37 | .81 | 36 | .86 | 28 | .89 | .86 | .86 | .97 | .11 | .18 | .09 | |||||
| E | 80 | .86 | 74 | .59 | 61 | 1.05 | .93 | .61 | ||||||||||
| School problems | C | 32 | .17 | 35 | .20 | 22 | .35 | .25 | .25 | .11 | .003 | .65 | .002 | |||||
| E | 65 | .15 | 63 | .18 | 46 | .23 | .15 | .20 | ||||||||||
| Child withdrawn | C | 37 | .27 | 36 | .28 | 28 | .68 | .25 | .32 | .94 | .001 | .10 | .01 | |||||
| E | 80 | .38 | 74 | .23 | 61 | .59 | .41 | .23 | ||||||||||
| Stolen things or arrested | C | 37 | .19 | 36 | .47 | 28 | .50 | .25 | .50 | .27 | .004 | .66 | .04 | .007 | ||||
| E | 80 | .19 | 74 | .34 | 61 | .43 | .18 | .31 | ||||||||||
| Child substance abuse | C | 37 | .03 | 36 | .03 | 28 | .00 | .04 | .04 | .44 | .97 | .31 | ||||||
| E | 80 | .03 | 74 | .04 | 61 | .08 | .03 | .05 | ||||||||||
| Child problems | C | 37 | 2.08 | 36 | 2.03 | 28 | 2.39 | 2.14 | 2.04 | .49 | .02 | .70 | .005 | |||||
| E | 80 | 1.72 | 74 | 1.70 | 61 | 2.33 | 1.84 | 1.80 | ||||||||||
| Negative child behaviors | C | 34 | .21 | 35 | .22 | 26 | .33 | .23 | .23 | .52 | .001 | .81 | .001 | |||||
| E | 72 | .21 | 71 | .19 | 55 | .30 | .22 | .20 | ||||||||||
| Positive child behaviors | C | 34 | .81 | 36 | .90 | .07 | 26 | .83 | .80 | .90 | .69 | .03 | .26 | .01 | ||||
| E | 72 | .83 | 70 | .86 | 54 | .81 | .83 | .85 | ||||||||||
| Household condition | C | 36 | .07 | 34 | .10 | 25 | .13 | .08 | .14 | .09 | .28 | .38 | ||||||
| E | 78 | .06 | 71 | .06 | 58 | .09 | .07 | .06 | ||||||||||
| Depression (SCL-90) | C | 37 | .76 | 36 | .83 | 28 | 1.00 | .81 | .72 | .89 | .008 | .64 | .002 | |||||
| Positive child care practices | E | 80 | .70 | 74 | .73 | 61 | 1.03 | .71 | .73 | |||||||||
| C | 36 | .82 | 34 | .96 | 24 | .90 | .89 | .94 | .70 | .05 | .71 | .01 | ||||||
| E | 71 | .88 | 67 | .93 | 48 | .90 | .86 | .95 | ||||||||||
| Negative child care practices | C | 35 | .09 | 33 | .07 | 25 | .11 | .09 | .08 | .38 | .001 | .09 | .001 | .03 | ||||
| E | 72 | .09 | 66 | .06 | 50 | .18 | .09 | .07 | ||||||||||
| a Means of control and experimental groups b Test of hypothesis of equivalent group means c Test of hypothesis that group means, averaged over time, are equal d Test of hypothesis that means at three points in time, averaged over the groups, are equal e Test of hypothesis of no interaction between group and time, that is, that the pattern of means over time is the same for both groups f Test of hypothesis that the average initial score is equal to the average of post-treatment and final scores g Test of hypothesis that average post-treatment score is equal to the average final score | ||||||||||||||||||
We report on the differences between the experimental and control groups as they were initially formed (the "primary analysis"). Results of the secondary analyses (dropping violations and minimal service cases) were usually similar. They are reported in footnotes when they were materially different.
We looked at a large number of differences between groups on functioning variables and as will be seen, some of these comparisons revealed statistically significant differences, usually favoring the experimental group. However, as will also be seen, the differences are not consistent across states or across time.
3.6.1 Life Events
In the caretaker interviews, there was a 15 item "life events" inventory asking about the occurrence of both positive and negative events in the last 3 months in the initial interview or since the previous interview in the post-treatment and followup interviews (Appendix K, Volume 3, Initial Caretaker Interview, p. 7). We formed three scales from this inventory, positive life events, negative life events, and a scale of those life events that might reflect depression in the caretaker (we had a more formal depression measure as well, described below). In none of the three states were there statistically significant differences between the experimental and control groups in the scores on these measures at the post-treatment or followup interviews nor were there differences between the groups in changes over time (see Tables 3-3, 3-4, and 3-5). It will be noted that generally the levels of these scales dropped in the second interview compared to the first and then increased again in the third interview. This is likely due to the differences in the time periods referenced in the questions asked at the three points in time (the last three months in the initial interview, last 4 to 6 weeks in the post-treatment interview, and last 10 to 11 months in the followup interview).
3.6.2 Problems
In the second and third interviews, caretakers were again asked questions about problems in the family. These questions paralleled those asked in the first interview, except this time caretakers were asked to respond to questions with regard to the time "since we last spoke to you" (see Tables 3-6 and 3-7). We analyze each of these problem items separately.
| Kentucky | New Jersey | Tennessee | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Control | Experimental | Control | Experimental | Control | Experimental | ||||||||||
| N | % | N | % | p | N | % | N | % | p | N | % | N | % | p | |
| Problems | |||||||||||||||
| Felt blue or depressed | 145 | 41 | 147 | 37 | 133 | 44 | 210 | 48 | 37 | 46 | 80 | 40 | |||
| Felt nervous or tense | 145 | 46 | 147 | 46 | 131 | 49 | 210 | 44 | 37 | 41 | 80 | 40 | |||
| Just wanted to give up | 145 | 17 | 147 | 15 | 133 | 26 | 210 | 20 | 35 | 29 | 79 | 23 | |||
| Overwhelmed with work or family responsibility | 145 | 37 | 148 | 47 | 0.08 | 133 | 50 | 210 | 44 | 37 | 32 | 80 | 33 | ||
| Felt you had few or no friends | 145 | 14 | 147 | 18 | 133 | 20 | 209 | 18 | 37 | 38 | 79 | 19 | 0.03 | ||
| Not enough money for food, rent, or clothing | 145 | 39 | 148 | 44 | 133 | 52 | 210 | 40 | 0.03 | 37 | 46 | 80 | 43 | ||
| Gotten in trouble with the law | 144 | 3 | 147 | 3 | 133 | 2 | 210 | 1 | 37 | 3 | 80 | 0 | |||
| Had too much to drink in a week | 145 | 2 | 148 | 1 | 133 | 2 | 209 | 2 | 37 | 5 | 80 | 3 | |||
| Used drugs several times a week | 145 | 1 | 148 | 0 | 133 | 2 | 209 | 1 | 37 | 5 | 80 | 3 | |||
| Economic Items | |||||||||||||||
| Had difficulty paying rent | 141 | 13 | 144 | 20 | 133 | 29 | 209 | 18 | 0.02 | 37 | 24 | 80 | 15 | ||
| Had difficulty paying electric/heat | 142 | 20 | 144 | 28 | 132 | 30 | 209 | 26 | 37 | 27 | 80 | 21 | |||
| Had difficulty buying enough food | 145 | 15 | 146 | 17 | 132 | 28 | 209 | 22 | 37 | 22 | 80 | 11 | |||
| Had difficulty buying clothes | 145 | 17 | 146 | 21 | 132 | 47 | 208 | 33 | 0.01 | 37 | 27 | 80 | 24 | ||
| Positive Items | |||||||||||||||
| Have you felt happy | 145 | 82 | 147 | 89 | 0.09 | 131 | 82 | 210 | 81 | 37 | 84 | 80 | 93 | ||
| Gotten together with anyone to have fun/relax | 145 | 64 | 148 | 64 | 133 | 65 | 210 | 59 | 37 | 38 | 80 | 75 | 0.001 | ||
| Doing a pretty good job raising kids | 144 | 94 | 147 | 91 | 130 | 88 | 209 | 91 | 36 | 92 | 79 | 96 | |||
| Kentucky | New Jersey | Tennessee | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Control | Experimental | p | Control | Experimental | p | Control | Experimental | p | |||||||
| N | % | N | % | N | % | N | % | N | % | N | % | ||||
| Problems | |||||||||||||||
| Felt blue or depressed | 119 | 44 | 130 | 42 | 107 | 55 | 166 | 48 | 36 | 44 | 74 | 47 | |||
| Felt nervous or tense | 119 | 47 | 130 | 53 | 105 | 53 | 166 | 47 | 36 | 31 | 73 | 38 | |||
| Just wanted to give up | 119 | 18 | 130 | 23 | 105 | 27 | 164 | 25 | 36 | 19 | 73 | 14 | |||
| Overwhelmed with work or family responsibility | 119 | 39 | 130 | 42 | 105 | 51 | 165 | 50 | 36 | 44 | 74 | 36 | |||
| Felt you had few or no friends | 119 | 24 | 130 | 25 | 107 | 15 | 166 | 24 | .07 | 36 | 8 | 74 | 19 | ||
| Not enough money for food, rent, or clothing | 118 | 44 | 130 | 46 | 106 | 46 | 166 | 45 | 36 | 42 | 74 | 49 | |||
| Gotten in trouble with the law | 119 | 6 | 130 | 4 | 107 | 1 | 166 | 4 | 36 | 3 | 74 | 3 | |||
| Had too much to drink in a week | 119 | 3 | 130 | 3 | 106 | 0 | 166 | 2 | 36 | 3 | 74 | 7 | |||
| Used drugs several times a week | 119 | 3 | 130 | 0 | 106 | 0 | 166 | 1 | 36 | 3 | 74 | 4 | |||
| Economic Items | |||||||||||||||
| Had difficulty paying rent | 118 | 20 | 127 | 20 | 107 | 34 | 167 | 27 | 36 | 39 | 74 | 20 | .04 | ||
| Had difficulty paying electric/heat | 118 | 19 | 126 | 25 | 107 | 36 | 167 | 37 | 36 | 42 | 74 | 32 | |||
| Had difficulty buying enough food | 119 | 14 | 129 | 15 | 107 | 35 | 167 | 26 | 36 | 19 | 73 | 14 | |||
| Had difficulty buying clothes | 119 | 15 | 128 | 19 | 107 | 42 | 167 | 35 | 36 | 31 | 73 | 21 | |||
| Positive Items | |||||||||||||||
| Have you felt happy | 119 | 89 | 130 | 89 | 106 | 83 | 166 | 87 | 36 | 92 | 73 | 89 | |||
| Gotten together with anyone to have fun/relax | 119 | 73 | 130 | 69 | 107 | 57 | 166 | 64 | 36 | 61 | 74 | 69 | |||
| Doing a pretty good job raising kids | 112 | 91 | 123 | 93 | 104 | 90 | 166 | 92 | 36 | 92 | 74 | 96 | |||
| NOTE: "FE" indicates significance determined by Fisher's exact test | |||||||||||||||
Kentucky. At the post-treatment interview, on 8 of the 9 problem questions, there were no differences between the experimental and control groups in responses. On the question of whether the caretaker felt overwhelmed with work or family responsibilities, a greater proportion of caretakers in the experimental group responded affirmatively at post-treatment than did caretakers in the control group (47% vs. 37%, p = .08). (35) In addition to the items about problems, caretakers were asked three questions about positive aspects of their lives: "gotten together with anyone to have fun or relax," "felt happy," and "felt that considering everything you're doing a pretty good job raising your kids." The primary analysis revealed that 89 percent of caretakers in the experimental group responded that they "felt happy" since the last interview as compared to 82 percent of caretakers in the control group (p = .09). For both the experimental and control groups, 64 percent responded affirmatively to the question of "getting together with anyone to have fun or relax," and over 90 percent responded affirmatively that they were "doing a pretty good job raising [their] kids." There were no significant differences between experimental and control groups on these last two items in either the primary or secondary analyses.
At the followup interview, there were no significant differences between experimental and control groups on any of these items.
New Jersey. On 8 of the 9 problem questions, there were no significant differences between the experimental and control groups in responses at post-treatment. On the overall question about the economic condition of family, "have you felt you just didn't have enough money for food, rent, or clothing?" 52 percent of the control group said yes, compared to 40 percent of the experimental group significantly different at p = .03, although in the secondary analysis the difference was not significant. (36) On none of the positive questions were there significant differences between groups in either the primary or secondary analyses.
At followup differences between groups approached significance for only one item, "felt you had few or no friends." More experimental group respondents replied affirmatively to this item (24% vs. 15% p = .07).
Tennessee. There was a significant difference between groups on only one of the nine problem questions at post-treatment. Fewer experimental group respondents reported they felt they had few or no friends (19% vs. 38%, p = .03). As to positive items, experimental group respondents far more often reported they "had gotten together with someone to have fun or relax" (75% vs. 38%, p < .001). At followup there were no differences between groups on any of these items.
3.6.3 Economic Functioning
In addition to the general item in the problem inventory on not having enough money for food, rent, or clothing, we asked four specific questions about difficulties in paying for the essentials of living (rent, electric service and heating, food, and clothes) (see Tables 3-6 and 3-7). These items were examined individually and were combined into an overall economic functioning scale. (See Tables 3-3, 3-4, and 3-5 for analyses of the scale).
Kentucky. On the scale, primary and secondary analyses revealed no significant differences in the average proportion of affirmative responses to the four items either at post-treatment or at followup. (37), (38) There also were no differences between groups in changes over time.
New Jersey. On the economic functioning scale, the experimental group had a lower average proportion of affirmative responses to these items at post-treatment (.25 vs. .34, p = .02) although the difference was not significant in the secondary analysis. The difference was not significant at followup nor were there significant differences in change over time. There were, however, significant differences on two of the specific items at post-treatment. Control group respondents more often reported difficulties paying rent (29% vs. 18%, p = .02) and also more often reported difficulties in buying clothes (47% vs. 33%, p = .01). These differences were not significant in the secondary analysis. At followup, there continued to be a difference in regard to buying food, although it was not significant (26% of the experimental group vs. 35% of the control group p = .12). At followup, the groups were similar on the other three items.
Tennessee. Control group respondents more often reported problems on the economic functioning scale at both post-treatment and followup, although the differences were not significant. There were also no significant differences between the groups in change over time. On individual items, there were no significant differences between groups at the post-treatment interview. At followup, significantly fewer experimental group respondents reported difficulties paying rent (20% vs. 39%, p = .04). There were no significant differences on the other three items at followup.
3.6.4 Household Condition
Caretakers were asked 10 questions about problematic conditions in the home (e.g., nonfunctioning heating, plumbing, or electrical systems; peeling paint; broken windows or doors).
Kentucky. In Kentucky, the experimental and control groups did not differ on the average proportions of the presence of such conditions at post-treatment or followup, nor did they differ on change in these proportions over time (see Table 3-3). On only one of the specific items were there any differences at post-treatment. Five percent of caretakers in the control group reported having broken windows or doors that were not fixed as compared to 1 percent in the experimental group (Fisher's exact p-value = .034). (39) There were no significant differences on any of the individual items at followup.
New Jersey. In New Jersey the experimental and control groups did not differ on the average proportions of the presence of such conditions at post-treatment or followup, nor did they differ on change in these proportions between interviews (see Table 3-4). There were no significant differences between the groups on any of the ten individual items at post-treatment. At followup, 8 percent of the experimental group respondents reported "a lot of peeling paint" compared to 2 percent of the control group respondents (p = .04). Also, 4 percent of the experimental group and none of the control group respondents reported that cooking appliances did not work (Fisher's exact p-value = .09).
Tennessee. There were no significant differences between experimental and control groups on the overall scales of household condition at either post-treatment or followup, nor were there differences in change over time (see Table 3-5). At post-treatment, more control group respondents reported bare electrical wires (8% vs. 0%, p = .08; Fisher's exact p-value = .03) while more experimental group respondents reported living in an "unsafe building because of illegal acts (5% vs. 0%, p = .05). There were no significant differences on any individual items at followup.
3.6.5 Child Care Practices
Caretakers were asked a series of yes-no questions about child care practices at the end of treatment and in the last three months at followup (both positive and negative). The results from these questions are shown in Table 3-8 and 3-9. In addition, three scales were formed using these items: positive child care practices (5 items), negative child care practices (10 items), and punishment (5 items, all of which are also included in the negative child care practices scale). Results from the scales are shown in Tables 3-3, 3-4 and 3-5.
Kentucky. In the post-treatment interview, "punishment for not finishing food" was the only item for which there were significant differences between the experimental and control groups, with a greater proportion of the control group responding affirmatively. There were no items on which there were significant differences at followup.
There were no significant differences between the experimental and control groups with regard to the positive and negative child care practice scales at the time of the second or third interviews. Both groups responded affirmatively to over 80 percent of the positive items and less than 15 percent of the negative items at both points in time. There were also no significant differences in change over time, in both groups there was a decline in the number of negative child care practices over time. With regard to the 5 items that pertain to punishment, caretakers in the experimental group responded affirmatively to a greater average proportion of punishment items than did the caretakers from the control group (.25 vs .20, p = .067) at the initial interview. At post-treatment, the average proportion of punishment items answered affirmatively were nearly the same (.17 for experimental group vs. .16 for the control group). Thus, from the first interview to the second, the reduction in the average proportion of punishment items endorsed was greater for the experimental group than for the control group (.09 fewer vs. .04 fewer, p = .054). (40) However, across the three points in time, there were no significant differences in change.
| Kentucky | New Jersey | Tennessee | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Control | Experimental | p | Control | Experimental | p | Control | Experimental | p | |||||||
| N | % | N | % | N | % | N | % | N | % | N | % | ||||
| Lost temper when child got on nerves | 145 | 43 | 147 | 46 | 132 | 63 | 209 | 59 | 37 | 27 | 77 | 35 | |||
| Found that hitting child was good | 143 | 6 | 147 | 7 | 131 | 12 | 208 | 8 | 37 | 19 | 76 | 8 | 0.09 | ||
| Hitting child harder that meant to | 143 | 8 | 147 | 6 | 131 | 10 | 208 | 5 | 37 | 5 | 75 | 8 | |||
| Out of control when punishing child | 144 | 24 | 146 | 24 | 131 | 40 | 209 | 30 | 0.05 | 36 | 11 | 76 | 12 | ||
| Have you praised your children | 144 | 94 | 146 | 92 | 132 | 92 | 209 | 92 | 37 | 92 | 76 | 91 | |||
| Listened to music together w/child | 144 | 86 | 146 | 86 | 132 | 86 | 209 | 82 | 37 | 92 | 77 | 90 | |||
| Tied child with cord- string-belt | 142 | 0 | 146 | 0 | 132 | 1 | 209 | 0 | 37 | 0 | 76 | 1 | |||
| Gone to amusement park, pool, picnic | 143 | 71 | 145 | 65 | 130 | 46 | 208 | 37 | 0.08 | 37 | 43 | 76 | 68 | 0.01 | |
| Uncomfortable hugging child | 126 | 5 | 134 | 6 | 95 | 5 | 144 | 6 | 33 | 9 | 72 | 4 | |||
| Encouraged child to read book | 137 | 92 | 140 | 90 | 126 | 82 | 202 | 91 | 0.02 | 34 | 94 | 72 | 96 | ||
| Have children handled household chores | 127 | 79 | 125 | 80 | 126 | 75 | 200 | 83 | 0.09 | 34 | 91 | 72 | 93 | ||
| Not let children into the house | 138 | 2 | 137 | 2 | 127 | 8 | 202 | 4 | 33 | 3 | 72 | 1 | |||
| Punished for not finishing food | 139 | 7 | 139 | 1 | 0.02 | 126 | 6 | 204 | 5 | 33 | 0 | 71 | 0 | ||
| Blamed child w/ things not their fault | 142 | 39 | 142 | 33 | 128 | 28 | 204 | 20 | 0.07 | 35 | 31 | 75 | 44 | ||
| Let child to play where not allowed | 138 | 2 | 138 | 4 | 123 | 1 | 200 | 4 | 33 | 0 | 71 | 1 | |||
| Unable to find someone to watch children | 144 | 9 | 146 | 12 | 131 | 21 | 206 | 12 | 0.04 | 35 | 20 | 73 | 27 | ||
| NOTE: "FE" indicates significance determined by Fisher's exact test | |||||||||||||||
| Kentucky | New Jersey | Tennessee | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Control | Experimental | Control | Experimental | Control | Experimental | ||||||||||
| N | % | N | % | p | N | % | N | % | p | N | % | N | % | p | |
| Lost temper when child got on nerves | 113 | 44 | 121 | 41 | 106 | 60 | 167 | 57 | 36 | 22 | 72 | 18 | |||
| Found that hitting child was good | 113 | 6 | 121 | 4 | 105 | 5 | 167 | 4 | 36 | 8 | 71 | 6 | |||
| Hitting child harder that meant to | 113 | 3 | 121 | 5 | 105 | 6 | 167 | 1 | .06 | 36 | 6 | 71 | 3 | ||
| Out of control when punishing child | 112 | 21 | 121 | 23 | 105 | 32 | 167 | 25 | 36 | 14 | 71 | 7 | |||
| Have you praised your children | 113 | 89 | 121 | 91 | 106 | 92 | 167 | 94 | 36 | 97 | 72 | 93 | |||
| Listened to music together w/child | 113 | 82 | 121 | 84 | 106 | 81 | 167 | 86 | 36 | 94 | 69 | 93 | |||
| Tied child with cord- string-belt | 113 | 0 | 121 | 2 | 106 | 1 | 167 | 1 | 36 | 0 | 70 | 0 | |||
| Gone to amusement park, pool, picnic | 113 | 74 | 121 | 70 | 106 | 48 | 167 | 44 | 35 | 94 | 70 | 90 | |||
| Uncomfortable hugging child | 102 | 4 | 110 | 5 | 74 | 5 | 116 | 6 | 35 | 6 | 66 | 3 | |||
| Encouraged child to read book | 107 | 83 | 115 | 81 | 103 | 87 | 160 | 89 | 34 | 100 | 67 | 96 | |||
| Have children handled household chores | 101 | 75 | 104 | 75 | 103 | 70 | 160 | 83 | .02 | 34 | 94 | 63 | 89 | ||
| Not let children into the house | 106 | 0 | 115 | 0 | 103 | 3 | 162 | 3 | 33 | 3 | 64 | 3 | |||
| Punished for not finishing food | 109 | 3 | 116 | 2 | 103 | 2 | 162 | 2 | 33 | 3 | 65 | 3 | |||
| Blamed child w/ things not their fault | 109 | 34 | 116 | 42 | 103 | 21 | 162 | 23 | 33 | 15 | 66 | 14 | |||
| Let child to play where not allowed | 109 | 1 | 114 | 0 | 101 | 2 | 160 | 1 | 33 | 0 | 65 | 3 | |||
| Unable to find someone to watch children | 111 | 3 | 122 | 0 | 103 | 17 | 165 | 18 | 33 | 21 | 67 | 19 | |||
New Jersey. On two of the items there were significant differences between the experimental and control groups at post-treatment: "have things sometimes gotten out of control when you punished your children?" happened more often in the control group and "have you encouraged your child to read a book?" which was done more often by experimental group respondents. At followup, there were two other items with differences: more control group respondents said they hit their child harder than they meant to (6% vs. 1%, p = .06) while more experimental group respondents said they had the children handle household chores (83% vs. 70%, p = .02).
On the scales, there were no significant differences between the experimental and control groups with respect to the positive child care practice items at either post-treatment or followup. At both points in time, both groups responded affirmatively to over 75 percent of the items. There were significant differences between the experimental and control groups with respect to the negative child care practice items at post-treatment. Caretakers in the experimental group responded affirmatively to 14 percent of the items whereas caretakers in the control group responded affirmatively to 18 percent of the items (p = .02). The difference disappeared at followup. At post-treatment, the experimental group significantly less often used punishment (.20 vs. .25; p = .04), but the difference was not statistically significant in the secondary analysis (p = .08) or in the primary analysis at followup (p = .15). There were no significant differences between groups in the change in changes over time in proportion of negative child care practice or punishment items.
Tennessee. At post-treatment, on one item there was a significant difference between experimental and control groups; more experimental group respondents indicated they had gone to an amusement park, pool, or picnic (68% vs. 43%, p = .01). There were no items with significant differences at followup. There were no significant differences in any of the three scales at either post-treatment or followup, nor in changes over time. The experimental group had a higher negative child care practices score at the first interview, and declined more than the control group, resulting in a nearly significant multivariate time-group interaction (p = .09) and a significant univariate comparison of the first interview score with the average of the scores from the later two interviews (p = .03).
3.6.6 Caretaker Depression
In both the initial and post-treatment interviews we administered the SCL-90 (41) depression scale to measure the level of depression of the caretaker. (42) In none of the three states were there significant differences between the groups in scores on this scale at the post-treatment or followup interviews or in changes over time (see Tables 3-3, 3-4, and 3-5). Scores at post-treatment were, on average, less than those in the initial interview for both groups and the reduction was greater for the experimental group (Kentucky: .23 less for the experimental group and .14 less for the control group; New Jersey: .19 less for the experimental group, .09 less for the control group; Tennessee: .32 less for the experimental group, .19 less for the control group), though the differences were not statistically significant.
3.6.7 Child Behavior
In all three interviews, we asked 35 questions about specific child behaviors, both positive and negative. Questions were phrased in terms of "any of the children" and some questions were age specific. Responses to these questions were used to form various scales: aggression (3 items), school problems (5 items), positive child behaviors (10 items), and negative child behaviors (21 items, including the aggression and school problems items). Analyses of these scales are shown in Tables 3-3, 3-4, and 3-5.
Kentucky. Neither the primary nor the secondary analyses revealed any significant differences between the groups in scores on any of these scales at post-treatment or at followup or in the change over time. Specific items on whether the child was withdrawn, had stolen things or been arrested, or had engaged in substance abuse did not reveal significant differences between groups at post-treatment or followup. For the scale of having stolen things or been arrested, the experimental group scored higher at the initial interview and declined between the first and second interview, while the control group increased between the first and second interviews and again at the followup interview. As a result, the multivariate interaction between time and group was nearly significant (p = .07) and the univariate difference between the groups in the difference between the first interview and the average of the later two interviews was significant at p = .03.
New Jersey. In the primary analysis there were no significant differences between groups on these scales at post-treatment or at followup, except for the overall negative child behaviors scale at post-treatment, on which the experimental group was lower (an average of 28% of the items vs. 33%; p = .04). (43) For none of these scales was there a significant difference between groups on change over time.
Specific items on whether the child was withdrawn, had stolen things or been arrested, or had engaged in substance abuse did not reveal significant differences between groups at post-treatment or followup or in change over time.
Tennessee. There were no significant differences between the experimental and control groups in the average scores on these scales at post-treatment or at followup, nor were there significant differences in change over time. The difference for positive child behaviors at followup was nearly significant, with the control group scoring higher (p = .07). The specific items on whether the child was withdrawn, had stolen things or been arrested, or had engaged in substance abuse did not reveal significant differences between groups at post-treatment or followup or in change over time.
3.7 Overall Assessment of Improvement by Caretakers
In the post-treatment interview, caretakers were asked about general changes in their families' lives since entering the study. Results are shown in Table 3-10 and 3-11. At post-treatment, in Kentucky and New Jersey, relative to control group caretakers, a significantly larger proportion of experimental group caretakers generally thought there was "great improvement" in their lives. This difference was significant in both the primary and secondary analyses. In the Tennessee secondary analysis, results
| Kentucky | New Jersey | Tennessee | ||||
|---|---|---|---|---|---|---|
| Control % | Experimental % | Control % | Experimental % | Control % | Experimental % | |
| Primary analysis: | p = .02 | p = .001 | p = n.s. | |||
| Great improvement | 16 | 22 | 9 | 16 | 32 | 32 |
| Some improvement | 31 | 42 | 41 | 52 | 32 | 42 |
| Same | 42 | 29 | 34 | 20 | 22 | 14 |
| Somewhat or a great deal worse | 12 | 6 | 16 | 12 | 14 | 13 |
| Kentucky | New Jersey | Tennessee | ||||
|---|---|---|---|---|---|---|
| Control % | Experimental % | Control % | Experimental % | Control % | Experimental % | |
| Primary analysis: | p = n.s. | p = n.s. | p = n.s. | |||
| Great improvement | 34 | 36 | 30 | 28 | 53 | 36 |
| Some improvement | 37 | 38 | 36 | 42 | 31 | 41 |
| Same | 18 | 16 | 17 | 16 | 8 | 15 |
| Somewhat or a great deal worse | 11 | 9 | 16 | 13 | 3 | 7 |
tended in the same direction, though not significantly (p = .09). At followup, differences
between the groups in Kentucky and New Jersey had nearly disappeared. In Tennessee, control group respondents more often thought there was "great improvement," although it was not a significant difference.
3.8 Information from Caseworkers on Family and Child Functioning
The caseworker interviews also contained questions regarding child and family functioning, in an effort to provide another perspective on these issues. In interpreting caseworker reports, it should be noted that experimental group caseworkers were Homebuilders workers, while control group respondents were the public agency workers responsible for the cases at the time of the interview. It is likely that there are differences between these groups of caseworkers in the knowledge they have of the cases, since Homebuilders workers had much more intensive involvement and that involvement began before the first research interview. In addition, it may be that there are systematic differences in these groups of workers in the approaches they take to the assessment of family problems. Hence, interpretations of comparisons between responses of workers serving each of the groups must be made with caution.
3.8.1Caretaker Functioning
Caseworkers were asked nine questions tapping various aspects of caretaker functioning on a five-point scale from 0 for not adequate to 4 for very adequate. Table 3-12 provides a list of these nine questions and a summary of the results from the initial and post-treatment interviews.
Kentucky. At the initial interview, significant or nearly significant differences were found on three items, with the experimental group scoring more adequate on average: ability to provide food (p = .02), responding patiently to child's questions (p = .06), and attending to children's health needs (p = .08). On a scale averaging the nine ratings for each case, the difference between means of the experimental and control groups approached significance, with the experimental group having a higher mean (p = .06). At post-treatment there were no significant differences in the primary analysis. However, in the secondary analysis, caretakers from the experimental group were rated higher (more adequate) than those from the control group with respect to whether they attended to the children's health needs (p = .04). As for the scale averaging the nine ratings, no differences were found between the experimental and
| Kentucky | Control | Experimental | p | ||
|---|---|---|---|---|---|
| N | Mean | N | Mean | ||
| Initial: | |||||
| Caretaker ability to provide food | 130 | 2.68 | 114 | 2.96 | 0.02 |
| Caretaker ability giving affection | 132 | 2.63 | 125 | 2.82 | |
| Caretaker respect for child's opinions | 119 | 2.38 | 106 | 2.58 | |
| Respond patiently to child's questions | 122 | 2.16 | 110 | 2.44 | 0.06 |
| Respond to child's emotional needs | 137 | 2.15 | 122 | 2.35 | |
| Provide learning opportunities | 127 | 2.17 | 110 | 2.35 | |
| Setting firm/consistent limits/rules | 130 | 1.68 | 116 | 1.88 | |
| Adequate supervisor/responsible childcare | 140 | 2.14 | 123 | 2.39 | 0.10 |
| Attending to children's health needs | 135 | 2.76 | 114 | 3.00 | 0.08 |
| Caretaker functioning, 9 items, average of nonmissing items, higher = better | 118 | 2.25 | 102 | 2.48 | 0.06 |
| Post-treatment: | |||||
| Caretaker ability to provide food | 145 | 2.88 | 154 | 2.97 | |
| Caretaker ability giving affection | 147 | 2.82 | 157 | 2.81 | |
| Caretaker respect for child's opinions | 135 | 2.58 | 144 | 2.45 | |
| Respond patiently to child's questions | 138 | 2.43 | 148 | 2.34 | |
| Respond to child's emotional needs | 145 | 2.28 | 156 | 2.28 | |
| Provide learning opportunities | 144 | 2.38 | 154 | 2.42 | |
| Setting firm/consistent limits/rules | 145 | 2.09 | 150 | 1.99 | |
| Adequate supervisor/responsible childcare | 152 | 2.50 | 158 | 2.59 | |
| Attending to children's health needs | 150 | 2.93 | 157 | 3.08 | |
| Caretaker functioning, 9 items, average of nonmissing items, higher=better | 142 | 2.56 | 151 | 2.55 | |
| Note: Scale for individual items: 0-4, where 0 = not adequate, 4 = very adequate | |||||
| New Jersey | Control | Experimental | p | ||
|---|---|---|---|---|---|
| N | Mean | N | Mean | ||
| Initial: | |||||
| Caretaker ability to provide food | 119 | 3.24 | 224 | 3.20 | |
| Caretaker ability giving affection | 120 | 2.88 | 229 | 2.62 | 0.03 |
| Caretaker respect for child's opinions | 118 | 2.42 | 219 | 2.32 | |
| Respond patiently to child's questions | 117 | 2.44 | 220 | 2.27 | |
| Respond to child's emotional needs | 118 | 2.37 | 228 | 2.23 | |
| Provide learning opportunities | 114 | 2.83 | 220 | 2.50 | 0.005 |
| Setting firm/consistent limits/rules | 126 | 2.11 | 228 | 1.93 | |
| Adequate supervisor/responsible childcare | 130 | 2.80 | 238 | 2.71 | |
| Attending to children's health needs | 125 | 3.34 | 214 | 3.17 | |
| Caretaker functioning, 9 items, average of nonmissing items, higher = better | 107 | 2.65 | 211 | 2.44 | 0.02 |
| Post-treatment: | |||||
| Caretaker ability to provide food | 137 | 3.36 | 246 | 3.34 | |
| Caretaker ability giving affection | 141 | 2.93 | 256 | 2.70 | 0.04 |
| Caretaker respect for child's opinions | 130 | 2.55 | 247 | 2.42 | |
| Respond patiently to child's questions | 140 | 2.51 | 248 | 2.37 | |
| Respond to child's emotional needs | 149 | 2.43 | 258 | 2.37 | |
| Provide learning opportunities | 137 | 2.89 | 247 | 2.60 | 0.01 |
| Setting firm/consistent limits/rules | 147 | 2.37 | 252 | 2.14 | 0.06 |
| Adequate supervisor/responsible childcare | 149 | 2.95 | 258 | 2.79 | |
| Attending to children's health needs | 148 | 3.35 | 252 | 3.25 | |
| Caretaker functioning, 9 items, average of nonmissing items, higher=better | 140 | 2.79 | 249 | 2.66 | 0.10 |
| Note: Scale for individual items: 0-4, where 0 = not adequate, 4 = very adequate | |||||
| Tennessee | Control | Experimental | p | ||
|---|---|---|---|---|---|
| N | Mean | N | Mean | ||
| Initial: | |||||
| Caretaker ability to provide food | 38 | 2.79 | 53 | 3.11 | |
| Caretaker ability giving affection | 42 | 2.76 | 60 | 2.92 | |
| Caretaker respect for child's opinions | 34 | 2.23 | 52 | 2.77 | 0.01 |
| Respond patiently to child's questions | 32 | 2.22 | 53 | 2.57 | |
| Respond to child's emotional needs | 40 | 2.05 | 59 | 2.47 | 0.04 |
| Provide learning opportunities | 39 | 2.64 | 56 | 2.55 | |
| Setting firm/consistent limits/rules | 36 | 2.33 | 57 | 2.01 | |
| Adequate supervisor/responsible childcare | 44 | 2.32 | 61 | 2.95 | 0.005 |
| Attending to children's health needs | 43 | 2.65 | 59 | 3.18 | 0.03 |
| Caretaker functioning, 9 items, average of nonmissing items, higher = better | 30 | 2.53 | 51 | 2.60 | |
| Post-treatment: | |||||
| Caretaker ability to provide food | 41 | 2.98 | 74 | 3.32 | 0.06 |
| Caretaker ability giving affection | 45 | 2.73 | 80 | 2.95 | |
| Caretaker respect for child's opinions | 40 | 2.35 | 74 | 2.84 | 0.01 |
| Respond patiently to child's questions | 38 | 2.26 | 76 | 2.67 | 0.04 |
| Respond to child's emotional needs | 42 | 2.26 | 81 | 2.59 | 0.06 |
| Provide learning opportunities | 44 | 2.64 | 78 | 2.64 | |
| Setting firm/consistent limits/rules | 43 | 2.04 | 79 | 2.38 | |
| Adequate supervisor/responsible childcare | 46 | 2.52 | 82 | 2.93 | 0.04 |
| Attending to children's health needs | 45 | 2.96 | 78 | 3.13 | |
| Caretaker functioning, 9 items, average of nonmissing items, higher=better | 42 | 2.51 | 77 | 2.82 | 0.04 |
| Note: Scale for individual items: 0-4, where 0 = not adequate, 4 = very adequate | |||||
control groups at post-treatment. Looking at change over time, on one item, respecting child's opinions, the ratings for the control group increased over time (.19 change), whereas the ratings for the experimental group decreased slightly over time (-.06 change), a difference that is significant (p = .05). The differences between groups in change on the overall scale averaging the nine ratings was not significant.
New Jersey. At the initial interview, on two items there were significant differences between the experimental and control groups, the control group scoring more adequate on average: caretaker's ability in giving affection (p = .03) and the caretaker's ability to provide learning opportunities (p = .005). On the scale averaging the nine ratings for each case, there was a significant difference between means of the experimental and control groups, the control group having a higher mean (p = .02). At post-treatment, the control group scored higher (more adequate functioning) on the same two items as before. On the scale of nine items the control group scored slightly higher, although the difference was nonsignificant. As to change over time, on one item ("respecting child's opinions"), the control group had, on average, more positive change than the experimental group. The difference in degree of change was significant at .05 (this result also held in the secondary analysis, p = .05). Differences between groups in change on the overall scale were not significant.
Tennessee. At the initial interview there were four items on which the groups were significantly different, the experimental group scoring higher on all four: caretaker respect for child's opinions (p = .01), response to child's emotional needs (p = .04), adequate supervision (p = .005), and attending to the child's health needs (p = .03). At post-treatment, five items had differences between groups significant at .06 or lower, all favoring the experimental group: caretaker ability to provide food, respect for child's opinions, response to child's emotional needs, adequate supervision, and respond patiently to child's questions. The average of all nine items was also significantly different for the groups. On one item, setting firm and consistent limits, there was a significant difference in the amount of change over time, the experimental group increased by an average of .31, while the control group declined by an average of .29 (p = .01). On the scale of nine items there was no significant difference between the groups in change over time.
3.8.2 Household Condition
As in the caretaker interview, we asked caseworkers about conditions in the home. Caseworkers were asked 13 yes-no questions, some positive and some negative. These items were combined in a scale which indicated that in Kentucky and New Jersey at post-treatment, control group families had, on average, a significantly better household condition than did experimental group families (Kentucky: p = .014; New Jersey: p = .02). In both states, for both groups the analysis of change over time indicated a slight improvement in the condition of the household. The difference between the experimental and control groups in change over time was not significant in either state. In Tennessee, there was no difference between groups at post-treatment or in change over time (both groups declined by .01).
3.8.3 Caretaker Problems
Caseworkers were asked a number of questions about problems experienced by children, caretakers, or other adult household members (question 19 on the initial caseworker interview, question 17 on the post-treatment caseworker interview). Twenty-one of these problems concerned the caretakers.
Kentucky. At post-treatment, in the primary analysis, caseworkers reported that the experimental group caretakers had, on average, 31 percent of the problems compared to 25 percent for the control group, a difference significant at p = .0005. (44) There were no significant differences in change in caretaker problems between the interviews in either the primary or secondary analyses.
New Jersey. At the post-treatment interview, on average, in the primary analysis caseworkers reported that experimental group caretakers had 23 percent of the problems compared to 21 percent of the control group, a nonsignificant difference. (45) There were no significant differences in change in caretaker problems between the interviews in either the primary or secondary analyses.
Tennessee. At post-treatment, caseworkers reported that experimental group caretakers had 18 percent of the problems compared to 21 percent of the control group, a nonsignificant difference. There was a significant difference between the groups in change over time, the experimental group improving more than the control group (-.08 vs. -.03, p = .05).
3.8.4 Child Problems
Twelve of the items on the caseworker problem inventory concerned the children. In Kentucky at post-treatment, the percentage of child problems for the experimental group was, on average, 27 percent compared to an average of 25 percent for the control group, a nonsignificant difference. (46) There were no significant differences in change in child problems between interviews in either the primary or secondary analyses.
In New Jersey at post-treatment, the average of the percentages of child problems was 25 percent for the experimental group and 27 percent for the control group, a nonsignificant difference. (47) There were no significant differences in change in child problems between interviews in either the primary or secondary analyses.
In Tennessee, the average percentages of child problems in the two groups at post-treatment were very close (18% for the control group, 19% for the experimental group). The difference between the groups in change over time was not significant.
3.9 Predictors of Outcomes
We performed regression analyses on a number of family functioning outcomes measured at the post-treatment interview and at followup. The analyses were intended to control for the effects of a number of variables, thereby providing more sensitive tests of the effects of family preservation, and to examine the effects of the variables on the outcomes. The dependent variables in these analyses were some of the scales of functioning discussed above: caretaker depression, child aggression, punishment, child school problems, difficulty paying bills, positive life events, negative life events, positive child behaviors, negative child behaviors, household condition, positive child care practices, and negative child care practices. Independent variables in these analyses were assignment group (experimental or control), caretaker's age, caretaker's race, family composition, caretaker's educational attainment, caretaker's employment status, residential stability, use of income support programs, caretaker's history of abuse and/or neglect, regular access to an automobile, and time to interview (days between random assignment and post-treatment/followup interview). The analyses also included the initial scores for the dependent variable, thereby controlling the level at post-treatment or followup for the initial value. Interactions between control variables and experimental group were also examined, only a few were found to be significant. (48)
Caretaker's age, caretaker's race, family composition, caretaker's educational attainment, caretaker's employment status, use of income support programs, caretaker's history of abuse and/or neglect have all been examined in previous studies of outcomes in child welfare and have often been found to be predictive. Residential stability and regular access to a car have been less often examined. Since transportation and housing assistance are commonly provided in family preservation service models, the inclusion of such variables seems justifiable. Moreover, prior research does support a relationship between residential stability and major depression (49) and child adjustment. (50) Similarly, transportation (or lack there of) has been found to be related to participation in social programs (51) and family functioning. (52) We included time to interview because of the fact that that varied considerably and might have affected the degree of change that we were observing.
Regression analyses were conducted at the family level for both the post-treatment and followup measures. The coefficients are displayed in Table 3-13 and 3-14. All of the coefficients are shown for the initial measure of the outcome variable and for experimental group. Coefficients for other variables are shown if they were significant at p = .1 or lower. Most of the analyses are ordinary least squares regressions, logistic regressions were used for dichotomous or highly skewed variables. Generally, the initial measure was the strongest predictor. Although the size of these coefficients decreased between the post-treatment and followup interview, the majority of such coefficients remained significant. The positive direction of the coefficients indicates that caretakers with higher initial values also had higher post-treatment and followup values.
In regard to the post-treatment analyses, experimental group families generally had better outcomes, but the differences were significant in only three analyses. In New Jersey, the experimental group had lower depression scores and lower negative child care practices than the control group when controlled for the other independent variables. In the analysis without controlling for the other variables, the result for depression was in the same direction, but not significant (p = .08). The result for negative child care practices without the control variables was also in the same direction and significant (p = .02). In Tennessee the experimental group had fewer negative life events in the regression analysis. The difference between groups in the uncontrolled analysis was not significant. (53) Three differences significantly in favor of the experimental group in the uncontrolled analyses were no longer significant in the regression analysis, all in New Jersey: caretaker use of punishment, negative life events, and positive child behaviors.
At the followup interview, the regression analysis indicates that family preservation clients had lower levels of child aggression in Tennessee, fewer school problems in Kentucky, and fewer problematic conditions in the home in Tennessee. There were no significant differences between groups in the uncontrolled comparisons.
Regarding the remaining independent variables, there was little consistency in whether or not a variable had an effect and even in the direction of the effect. The following discussion focuses on those variables significant at p = .05 or lower. At post-treatment, the variables that most often showed effects were education and the caretaker having a history of being maltreated.
| Dependent Measures | Initial Measure | Experim. Group Assignment | Care-taker Age | Single Mother | Ethnic Minority | Abuse Neglect History | Education | Employ- ment | Income Support | Access to Car | Housing Stability | Time to Interview |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Caretaker depression | ||||||||||||
| Kentucky2 | .596** | -.053 | .083 | .144** | .078 | |||||||
| New Jersey2 | .690** | -.202** | -.048 | .102* | -.170** | |||||||
| Tennessee2 | .606** | -.064 | ||||||||||
| Child aggression | ||||||||||||
| Kentucky | .522** | -.051 | ||||||||||
| New Jersey | .589** | -.044 | .101* | -.079 | ||||||||
| Tennessee | .533** | -.004 | ||||||||||
| Punishment | ||||||||||||
| Kentucky1 | 9.81** | 1.16 | 2.02* | |||||||||
| New Jersey | .529** | -.079 | ||||||||||
| Tennessee | .281** | -.012 | ||||||||||
| Child school problems | ||||||||||||
| Kentucky | .539** | -.020 | .111 | |||||||||
| New Jersey | .381** | -.041 | .107 | |||||||||
| Tennessee | .654** | -.073 | ||||||||||
| Difficulty paying bills | ||||||||||||
| Kentucky | .608** | .023 | ||||||||||
| New Jersey | .632** | -.061 | -.080 | -.088 | .107* | |||||||
| Tennessee | .513** | -.020 | .155 | .142 | ||||||||
| Positive life events | ||||||||||||
| Kentucky | .218** | .055 | .160** | .124* | ||||||||
| New Jersey | .330** | -.074 | -.108* | .104 | -.139* | |||||||
| Tennessee | .330** | .020 | ||||||||||
| Negative life events | ||||||||||||
| Kentucky1 | 1.85* | .833 | 2.69** | |||||||||
| New Jersey | .278** | .008 | .132* | .097 | ||||||||
| Tennessee | .100 | -.768** | -.290** | -.707** | ||||||||
| Positive child behaviors | ||||||||||||
| Kentucky | .489** | -.002 | -.089 | .120* | ||||||||
| New Jersey | .579** | -.032 | ||||||||||
| Tennessee | .525** | .098 | -.187* | |||||||||
| Negative child behaviors | ||||||||||||
| Kentucky | .592** | -.012 | ||||||||||
| New Jersey | .581** | -.078 | ||||||||||
| Tennessee | .647** | -.016 | ||||||||||
| Household condition | ||||||||||||
| Kentucky1 | 9.01** | .961 | .414* | .948 | ||||||||
| New Jersey1 | 5.66** | .744 | 2.01* | |||||||||
| Tennessee | .429** | -.004 | ||||||||||
| Positive child care | ||||||||||||
| Kentucky | .401** | -.041 | .113* | .128* | ||||||||
| New Jersey | .566** | .007 | -.081 | |||||||||
| Tennessee | .575** | .069 | .268** | |||||||||
| Negative child care | ||||||||||||
| Kentucky | .569** | -.078 | -.115* | |||||||||
| New Jersey | .571** | -.119* | .083 | .116* | ||||||||
| Tennessee | .371** | -.136 | -.177 | |||||||||
1 Logistic regression, Exp (B) displayed | ||||||||||||
| Dependent Measures | Initial Measure | Experim. Group Assignment | Care- taker Age | Single Mother | Ethnic Minority | Abuse Neglect History | Education | Employ-ment | Income Support | Access to Car | Housing Stability | Time to Interview |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Caretaker depression | ||||||||||||
| Kentucky2 | .552** | .068 | .116* | |||||||||
New Jersey2 | .518** | -.051 | -.124* | |||||||||
| Tennessee2 | .443** | -.011 | ||||||||||
| Child aggression | ||||||||||||
| Kentucky | .363** | .022 | ||||||||||
| New Jersey | .417** | .031 | -.120* | |||||||||
| Tennessee | .347** | -.226* | .173 | |||||||||
| Punishment | ||||||||||||
| Kentucky1 | 6.67** | .750 | .977* | |||||||||
| New Jersey | .288** | -.085 | -.126 | -.108 | ||||||||
| Tennessee | .271* | -.146 | .293** | |||||||||
| Child school problems | ||||||||||||
| Kentucky | .254** | -.147* | .253** | .180** | ||||||||
| New Jersey | .272** | -.098 | ||||||||||
| Tennessee | .451** | -.002 | .290** | |||||||||
| Difficulty paying bills | ||||||||||||
| Kentucky | .396** | -.016 | .127* | |||||||||
| New Jersey | .537** | -.001 | .108* | -.108* | ||||||||
| Tennessee | .290** | -.132 | ||||||||||
| Positive life events | ||||||||||||
| Kentucky | .260** | -.044 | .117 | .227** | ||||||||
| New Jersey | .081 | -.049 | -.165* | |||||||||
| Tennessee | .261* | -.081 | -.198 | .304* | ||||||||
| Negative life events | ||||||||||||
| Kentucky1 | 1.71* | 1.18 | ||||||||||
| New Jersey | .356** | .023 | -.176** | |||||||||
| Tennessee | .116 | -.127 | .272* | .236* | ||||||||
| Positive child behaviors | ||||||||||||
| Kentucky | .250** | .028 | .182** | |||||||||
| New Jersey | .294** | -.002 | .165* | |||||||||
| Tennessee | .192 | -.064 | -.231* | .301** | ||||||||
| Negative child behaviors | ||||||||||||
| Kentucky | .385** | -.058 | .137* | |||||||||
| New Jersey | .404** | -.016 | -.121* | |||||||||
| Tennessee | .344** | -.097 | .200 | |||||||||
| Household condition | ||||||||||||
| Kentucky1 | 3.86* | 1.38 | .271 | |||||||||
| New Jersey1 | 3.59** | 1.40 | 2.51* | |||||||||
| Tennessee | .045 | -1.24** | -.317 | .207 | ||||||||
| Positive child care | ||||||||||||
| Kentucky | .370** | .014 | -.193** | .133* | ||||||||
| New Jersey | .164* | .044 | -.200** | |||||||||
| Tennessee | .110 | -.056 | ||||||||||
| Negative child care | ||||||||||||
| Kentucky | .340** | .020 | ||||||||||
| New Jersey | .311** | -.050 | -.135* | -.143* | ||||||||
| Tennessee | .195 | -.085 | .296* | |||||||||
| 1 Logistic Regression, Exp (B) displayed 2 Depression scores transformed using log transformation * p < .05, ** p < .01. All coefficients for experimental group assignment and initial measure are shown, regardless of significance. All other entries without stars are significant at .1. | ||||||||||||
Caretaker education was related to three post-treatment outcomes in Kentucky. More education was associated with more punishment, more positive life events, and worse household condition. In Tennessee more education was related to fewer negative life events. In Kentucky, having a history of maltreatment was related to higher depression and more negative life events. In New Jersey, history of maltreatment was related to higher depression and children being more aggressive. Income support, ethnic minority, and caretaker employment all were predictors in 3 of the 36 post-treatment regressions. Time to interview was significant in only one of the regressions.
At followup, time to interview emerged as a predictor in 4 of the 36 regression equations, in all cases related to an increase (worsening). Other variables often related to outcome were caretaker age and education. In New Jersey, older caretakers had fewer positive life events and had children who were less aggressive. In Tennessee, older caretakers had children with fewer positive behaviors. In Kentucky, older caretakers had higher depression scores, less often engaged in positive child care practices, and had children with more school problems.
In New Jersey at followup, caretaker education was related to 3 outcomes. More education was related to lower depression, less difficulty paying bills, and more positive child behaviors. In Kentucky, more education was related to more difficulty paying bills, more positive life events, and more positive child behaviors. More education in Tennessee is related to more child school problems. In Kentucky, caretaker's employment is related to more negative child behaviors, more child school problems, and more positive child care practices. Caretaker employment in New Jersey is associated with fewer negative child care practices. Ethnic minority caretakers in New Jersey had fewer negative life events, engaged in fewer negative child care practices, and had children with fewer negative behaviors.
The 72 regression equations for post-treatment and followup contain a fair number of significant coefficients, but there is little consistency across states or across outcomes.
In summary, regression models were constructed to explore the relationship between caretaker demographic characteristics and experimental group and family functioning. Other than the initial value of the measures, relatively few significant relationships emerged. Moreover, these relationships were not consistent across the states. As to the effects of family preservation services, these data do not support a strong relationship between these services and better family functioning.
3.10 Relationship of Placement and Subsequent Harm to Amount of Service
The preceding analysis focuses on differences between cases assigned to family preservation and the control group. But some cases in the experimental group received relatively low levels of service while cases in the control group may have received more services than usual. One can examine the relationship between the amount of service provided, regardless of whether the case is in the experimental or control group, and outcomes. In this analysis we measured level of service in two ways, both drawn from the post-treatment interview with the caretaker: the caretaker's report of the number of contacts with the worker and the caretaker's
| KY, NJ, TN | |||
|---|---|---|---|
| N | Exp(B) | p | |
| Placement within 18 months and caseworker contact | 749 | ||
| Constant | .39 | ||
| Number of times met with worker since last interview | 1.000 | .89 | |
| Number of placements within 18 months after random assignment date | 208 | ||
| Placement within 18 months and caseworker activities | 749 | ||
| Constant | .40 | ||
| Number of caseworker activities (as reported by caregiver) | .99 | .68 | |
| Number of placements w/in 18 months after random assignment date | 208 | ||
| Substantiated allegation within 18 months and caseworker contact | 753 | ||
| Constant | .25 | ||
| Number of times met with worker since last interview | 1.003 | .12 | |
| Number of substantiated allegations 18 months after random assignment date | 161 | ||
| Substantiated allegation within 18 months and caseworker activities | 753 | ||
| Constant | .26 | ||
| Number of CW activities (as reported by caregiver) | 1.01 | .68 | |
| Number of substantiated allegations 18 months after random assignment date | 161 | ||
report of the number of caseworker activities. The relationship between these variables and placement and subsequent substantiated allegations of harm was examined through logistic regression, reported in Table 3-15. As can be seen in the table, there were no significant relationships between the level of service and these outcomes, all of the odds ratios for the predictive variables were quite close to one.
3.11 Matching of Services to Problems
The analysis to this point has examined the effects of services in an undifferentiated way, by looking at the relationship between the amount of services and outcomes, by either comparing the outcomes of the experimental and control groups or, in the last section, examining the relationship between amount of services as determined by the number of contacts and outcomes. But it is possible that the services provided were not responsive to the particular problems of families. To the extent that this is the case, it would explain the relatively small effects of services on outcomes. Furthermore, an examination of specific problems and specific services might reveal effects that are obscured in the global analyses presented thus far.
We explored this possibility in a limited way by looking at three prominent problems experienced by families in the study, financial difficulties, problems with discipline of children, and depression. After identifying families with these problems, we determined the extent to which services provided might have addressed these issues, the extent of "match" between problems and services provided. Finally, we determined whether there was a relationship between the extent of match of services and problems with outcomes. We conducted the analyses only on cases in the family preservation group in the three Homebuilders states for which we had interviews at the beginning and end of service and contact forms giving us information on services provided. We limited the sample to the Homebuilders group so as to look at the implementation of a particular, well defined model, thus limiting extraneous variance due to variation in approach. It was thought that within that group we would most likely find a match between problems and services. We also believed that the contact form data, on which this analysis depended, was more complete and of a higher quality for the experimental group. Furthermore, in general, control group cases did not receive a high enough volume of services to reveal a match.
We combined the samples from the three states, a total of 292 families with 886 children. Women were the caretakers in 89 percent of these families and the caretakers were, on average, 36 years old. Forty-eight percent were African American, 47 percent were white.
Economic problems were determined from responses on the caretaker interviews to questions about difficulties paying rent, paying electric and heating bills, buying food, and buying clothes for the children. Any family responding positively to any one of these four questions at the initial interview was classified as having economic problems (n = 157, 54%). The presence of disciplinary problems was determined from seven questions on the initial caretaker interview. (54) Again, if any one of these questions was answered affirmatively, the case was classified as having disciplinary problems (n = 221, 76%). Caretaker depression was determined from the SCL-90 depression scale. Individuals with scores higher than the median (determined separately for men and women) were categorized as having some problems with depression.
Services specific to particular problems were determined from the contact forms completed by family preservation caseworkers. Services responding to economic problems were providing emergency cash or paying bills; buying food; helping to find housing; providing clothing, furniture, or supplies; and discussions of money management. Services directed at disciplinary practices were discussions of discipline of children, child's anger management, dealing with violence in the family, caretaker interaction with the child, supervision of children, and child development. Services directed at caretaker depression were discussions of depression, other caretaker emotional problems, social skills, and adult companionship. The extent of service response to particular problems was calculated as the simple count of the number of times caseworkers checked a particular item on the contact forms submitted for the family.
There are a number of limitations in this procedure. First, we looked at only three problem areas, areas that we thought we could identify relatively easily. Further, the measurement of service response is clearly not ideal, service data were not constructed in a way that would make them straightforwardly parallel to problems, so the development of service measures in this analysis is quite post-hoc. In addition, obviously caseworkers may have chosen, for good reason, not to respond to a particular problem, perhaps because another problem was more pressing or more tractable, so that the failure to respond to a particular problem should not be viewed as an indication of the failure of casework in the case. There was also considerable overlap of these problems (83 families reported all three problems at the initial interview, 82 reported two of them, while 24 reported none of them). Despite these limitations, tendencies toward the matching of services and problems should show up in the data, though perhaps not as prominently as would have been the case had data collection been explicitly directed at exploring these issues.
To determine the match between problems and services, we calculated the average number of times a service was provided to cases with the problem (the number of contact forms recording the service) and compared that to the average number of times the service was provided to cases that did not have the problem. The results are shown in Table 3-16.
Families with economic problems significantly more often than families without such problems received three of the five services identified as responding to these problems. We conclude that there is moderate evidence of a match between problems and services in the economic area. Families with discipline problems received two of five services significantly more often than those without these problems, some indication of a match. As to depression, caseworkers significantly more often discussed depression with caretakers with high levels of depression than with those with lower levels. For the other three services identified as possibly responding to caretaker depression there were no significant differences. Discussion of "other caretaker emotional problems" occurred more often with those with higher levels of depression, although the difference is not significant. We conclude, therefore, that there is some match between problems and casework response, a match that might have shown itself more strongly had measurement of services been designed to reveal it.
| Cases with the problem | Cases not having the problem | |
|---|---|---|
| Economic problems, n = | 157 | 134 |
| Services: | ||
| Emergency cash or paying bills | 0.57** | 0.23 |
| Buying food | 1.26** | 0.63 |
| Helping to find housing | 0.37 | 0.36 |
| Clothing, furniture, supplies | 0.92 | 0.66 |
| Money management discussion | 2.05** | 0.93 |
| Discipline problems, n = | 221 | 68 |
| Services, discussion of: | ||
| Discipline of child | 7.35** | 5.08 |
| Child anger management | 4.98* | 3.33 |
| Family violence | 2.53 | 1.73 |
| Caretaker interaction with child | 6.32 | 5.82 |
| Supervision of children | 3.45 | 3.76 |
| Depression, n = | 152 | 140 |
| Services, discussion of: | ||
| Caretaker depression | 2.83** | 1.43 |
| Other caretaker emotional problems | 2.37 | 1.80 |
| Adult companionship | 1.32 | 0.99 |
| Social skills | 2.42 | 2.85 |
| * p < .05, ** p < .01. | ||
Effect of Match of Services to Problems on Outcomes.
We next attempted to determine whether, for cases experiencing each of the three problem categories, service responses specific to the problem made a difference in outcome. The outcomes we examined were improvements in the specific problem at the time of the post-treatment caretaker interview and subsequent placement and maltreatment. Because families in the experimental group were not randomly assigned to varying levels of specific kinds of services, it can be assumed that there are selection biases operating in determining levels of service. Hence, for the examination of effects on the level of the problem at the post-treatment interview, two-stage least squares techniques were used in which the provision of specific services was modeled in the first stage and the effects of services on outcomes were determined in the second stage. Demographic characteristics were included in both stages and the level of the problem at the initial interview was included in the second stage. A variable reflecting the caretaker's response in the post-treatment interview to a question on whether additional services were needed was also included in both stages. First stage instruments were chosen based on their prediction of levels of service and lack of association with the outcome variables. In all of the analyses the instruments were jointly significant in the prediction of levels of service.
We report here on the results of the second stage of these analyses, the determination of the effect of specific services on problem level at the post-treatment interview. For all three problems, as would be expected, the initial level of the problem was significantly and positively related to the post-treatment level. For the group experiencing economic problems, of the five services thought to respond to the problem, only one was significantly related to post-treatment problem levels, the provision of cash assistance was positively related to level of economic problems post-treatment. That is, the more cash assistance provided, the higher the levels of economic problems. It is likely that this seemingly contrary finding simply reflects the fact that families with considerable economic difficulties are more likely to get cash assistance but are also more likely to continue to experience those problems. The variable reflecting need for additional concrete services was also positively related to post-treatment levels of economic problems.
For the group with discipline problems, of the five services in the second stage equation, two were significantly related to the level of the problem at post-treatment. The higher the level of discussion of discipline of children, the lower the level of the problem while the greater the discussion of child anger management, the higher the level of the problem. For cases with above median levels of depression, two of the four services thought to respond to the problem were significantly related to level of the problem at post treatment. More discussion of depression in caseworker contacts was related to higher levels of depression post-treatment while more discussion of adult companionship was related to decreased post-treatment depression.
The sparseness of positive findings in this analysis leads us to conclude that there is little evidence here of positive effects of concrete or clinical services on these three problem areas.
To examine the effects of specific services on placement and maltreatment following entry into the study, hierarchical linear modeling was used in order to account for the fact that we have multiple children in some families and their outcomes are not independent. (55) To deal with selection issues, predicted values of services, determined from the first stage of the 2SLS analyses, were entered at level two (the family level) and these terms were used to determine the effects of services on outcomes. Again, separate analyses were done for each problem group.
For the economic difficulties subgroup, two of the five problem specific services were significantly and negatively related to the likelihood of subsequent maltreatment: provision of clothing/furniture/supplies and housing assistance. Cash assistance and provision of clothing/furniture/supplies were significantly and negatively related to the likelihood of subsequent placement. For families with problems in discipline of children, none of the six services were related to subsequent maltreatment, while discussions of violence in the family were related to a decrease in likelihood of placement and discussions of child anger management were related to an increase. In families in which the caretaker was measured as having higher than median levels of depression at the beginning of service, none of the problem specific services were related to subsequent maltreatment. However, discussion of depression was related to an increase in likelihood of placement while discussions of social skills and of companionship/friendship were related to a decrease in placement.
No consistent patterns emerge in the analysis of the effects of specific services on subsequent maltreatment and placement. There is some indication that within the economic problems group, services directed at these problems have some beneficial effects on these outcomes. Within the discipline problems and depression groups, results are mixed, some services are related to increases in subsequent maltreatment and placement while others are associated with decreases. No clear conclusions can be drawn.
3.12 Summary of Outcome Data
Information from the caretaker interviews, the caseworker interviews, and the administrative data were analyzed for indications of differences between the experimental and control groups subsequent to the referral to the family preservation program. Tables 3-15 and 3-16 contain a summary of those outcomes on which we found significant differences between the experimental and control groups in any state for the primary analyses (p < .05). Items in bold are those on which the experimental group had better outcomes, those in italics are those on which the control group had better outcomes.
In none of the three states were there significant differences between the experimental and control groups on family level rates of placement or case closings. Subsequent maltreatment was generally not related to experimental group membership, except for one subgroup in Tennessee. In Tennessee, in those families with an allegation within 30 days prior to random assignment, the experimental group children experienced fewer substantiated allegations than children in the control group.
In Tables 3-17 and 3-18 there are a number of child and family functioning items in which the experimental group displayed better outcomes than the control group in one of the states. It should be noted that the results have not been adjusted for the multiplicity of significance tests performed. That is, these significant items surfaced out of a large number of items and scales examined. In such a situation it is to be expected that some items will show significant differences simply by chance, so the appearance of a few significant differences should not be taken as an indication of superiority of one group over another, particularly when the results are not confirmed in more than one state. On only two items were differences found in two states: caretakers' assessment of whether goals had been accomplished and their assessment of overall change. We are inclined to believe that family preservation programs as represented in these states do result in higher assessments by clients of the extent to which goals have been accomplished and of overall change, since differences on those items were found in both states. Beyond that, we are unable to claim consistent evidence of positive effects of family preservation services. (56)
| Caretaker Interview: Proportion of affirmative answers to yes/no questions | |||||||||
|---|---|---|---|---|---|---|---|---|---|
| Kentucky | New Jersey | Tennessee | |||||||
| Control % | Exp % | p | Control % | Exp % | p | Control % | Exp % | p | |
| Is apartment/house rented (vs. owned) | 75 | 89 | 0.005 | 70 | 68 | 69 | 75 | ||
| Got together with anyone to have fun | 64 | 64 | 65 | 59 | 38 | 75 | 0.001 | ||
| Felt had few or no friends | 14 | 18 | 20 | 18 | 38 | 19 | 0.03 | ||
| Had difficulty buying clothes | 17 | 21 | 47 | 33 | 0.008 | 27 | 24 | ||
| Out of control when punishing child | 24 | 24 | 40 | 30 | 0.05 | 11 | 12 | ||
| Punished for not finishing food | 7 | 1 | 0.02 | 6 | 5 | 0 | 0 | ||
| Unable to find someone to watch child | 9 | 12 | 21 | 12 | 0.04 | 20 | 27 | ||
| Encouraged child to read a book | 92 | 90 | 82 | 91 | 0.02 | 94 | 96 | ||
| Have goals been accomplished | 63 | 77 | 0.02 | 52 | 71 | 0.001 | 81 | 84 | |
| Assessment of overall change: | 0.02 | 0.001 | |||||||
| Great improvement | 16 | 22 | 9 | 16 | 32 | 32 | |||
| Some improvement | 31 | 42 | 41 | 52 | 32 | 42 | |||
| Same | 42 | 29 | 34 | 20 | 22 | 14 | |||
| Somewhat or a great deal worse | 12 | 6 | 16 | 12 | 14 | 13 | |||
| Caretaker Scales: | |||||||||
| Difficulty paying bills (proportion of 4 items) | 0.17 | 0.22 | 0.34 | 0.25 | 0.02 | 0.25 | 0.18 | ||
| Negative child care practices (proportion of 10 items) | 0.14 | .0.13 | 0.18 | 0.14 | 0.02 | 0.09 | 0.09 | ||
| Punishment (proportion of 5 items) | 0.16 | 0.17 | 0.25 | 0.20 | 0.04 | 0.13 | 0.13 | ||
| Negative child behaviors (proportion of 21 items) | 0.34 | 0.34 | 0.33 | 0.28 | 0.04 | 0.21 | 0.21 | ||
| Change in proportion of punishment items from initial to post-treatment interviews | -0.04 | -0.09 | 0.05 | -0.05 | -0.07 | -0.07 | -0.13 | ||
| Change in proportion of negative child care practices from Initial to post-treatment interviews | -0.02 | -0.06 | 0.04 | -0.04 | -0.05 | -0.01 | -0.08 | 0.02 | |
| Ability giving affection (higher = more adequate) | 2.83 | 2.83 | 2.93 | 2.70 | 0.04 | 2.73 | 2.95 | ||
| Providing learning opportunities for child (higher = more adequate) | 2.38 | 2.42 | 2.89 | 2.60 | 0.008 | 2.64 | 2.64 | ||
| Respecting child's opinions (higher = more adequate) | 2.58 | 2.45 | 2.55 | 2.42 | 2.35 | 2.84 | 0.01 | ||
| Responding patiently to child's questions (higher = more adequate) | 2.43 | 2.34 | 2.44 | 2.27 | 2.26 | 2.67 | 0.04 | ||
| Adequate supervision / Responsible child care (higher = more adequate) | 2.50 | 2.59 | 2.80 | 2.71 | 2.52 | 2.93 | 0.04 | ||
| Household condition (proportion of 13 items, higher = worse condition) | 0.10 | 0.13 | 0.01 | 0.09 | 0.11 | 0.02 | 0.12 | 0.12 | |
| Caretaker problems (proportion of 21 items, higher = more problems) | 0.25 | 0.31 | 0.0005 | 0.21 | 0.23 | 0.21 | 0.18 | ||
| Caretaker functioning (higher = better) | 2.56 | 2.55 | 2.79 | 2.66 | 0.10 | 2.51 | 2.82 | 0.04 | |
| Respecting child's opinions (change in average ratings from Time 1 to Time 2)** | 0.19 | -0.06 | 0.05 | 0.27 | 0.04 | 0.05 | 0.06 | 0.14 | |
| Setting firm/consistent limits/rules (change in average ratings from Time 1 to Time 2) ** | 0.35 | 0.22 | 0.33 | 0.25 | -0.29 | 0.29 | 0.01 | ||
| Caretaker Problems (Change in proportion of 21 items; lower = less at Time 2) | -0.06 | -0.04 | -0.05 | -0.04 | -0.03 | -0.08 | 0.05 | ||
| NOTE: This table only includes items with a primary analysis p-value less than .05 in at least one of the states; p-values greater than .10 are not reported. Items in bold indicate significant findings in favor of the experimental group whereas italicized items indicate significant findings in favor of the control group. ** Scale for change in ratings: -4 = ability decreased greatly over time, 0 = no change in ability over time, +4 = ability increased greatly over time | |||||||||
| Proportion of affirmative answers to yes/no questions | Kentucky | New Jersey | Tennessee | ||||||
|---|---|---|---|---|---|---|---|---|---|
| Control % | Exp % | p | Control % | Exp % | p | Control % | Exp % | p | |
| Has spouse held full time job | 81 | 78 | 86 | 68 | .05 | 100 | 85 | ||
| Had difficulty paying rent | 20 | 20 | 34 | 27 | 39 | 20 | .04 | ||
| Have children handled household chores | 75 | 75 | 70 | 83 | .02 | 94 | 89 | ||
| NOTE: This table only includes items with either a primary p-value less than .05 in at least one of the states; p-values greater than .10 are not reported Items in bold indicate significant findings in favor of the experimental group whereas italicized items indicate significant findings in favor of the control group. | |||||||||
There are a few items on which the control group had better outcomes, nearly all of them on measures provided by caseworkers. We are not inclined to read too much into these results, since experimental group caseworkers generally knew the families better and there may well have been significant differences in the ways that workers serving the two groups saw families and judged their functioning.
23. The full list of New Jersey service codes that were included is: public institution, teaching family placement, para-foster care income maintenance, juvenile-family crisis shelter placement, relative placement, foster care placement, residential treatment placement, finalized adoption placement, selected adoption placement - pending, maternity home care, group home placement, independent living, and shelter care placement. Four of these categories did not actually occur in the data: teaching family placement, para-foster care income maintenance, finalized adoption placement, and selected adoption placement - pending. In Kentucky placement (as reflected in the variable FACTYPE), included: adoption, foster care, private institution/boarding schools, family treatment home, unmarried parent, other, children's psychiatric hospital, and foster care medically fragile. The data did not include adoption, family treatment home, and unmarried parent. In Tennessee, placements included: foster care, relative home, trial home, residential care, continuum contract, non-relative home, adoptive home, runaway, shelter, independent living, and detention.
24. Cases entered the study at varying points in time. In Kentucky, cases entered between May 7, 1996 and February 13, 1998; in New Jersey, cases entered between November 6, 1996 and February 26, 1998; and in Tennessee, between November 19, 1996 and May 26, 1998.
25. There are two reasons for focusing on family-level analyses. First, we are not confident that the administrative data allow for accurate identification of children to be included in the risk pool (what would be the denominator in a rate of placement calculation). Children are identified as belonging to a family through a case number. The analysis requires that we identify children who are in the home at the time of random assignment (or who are born or return to the home subsequently). In these states, children apparently often retain a family case number even when they are not in the home, and the administrative data do not allow us to verify the location of the child at the time of random assignment (or even sometimes at the time of an event such as placement). This problem is alleviated in analyses at the family level, since we know that the family is at risk of having a child placed (as long as there are any children in the family).
As to the accuracy of the "numerator" in our analyses, we focus on the first event (e.g., placement) in the family, subsequent to random assignment. It is possible that the first event occurs with regard to a child identified with a family but not living in that family at the time of the event. We judge the likelihood of that occurring to be small (the effects of this source of error would be similar in a family and child level analysis). In addition, subsequent events involving other children identified with the family but not in the family at the time of the event would not affect the family level analysis, while they would create inaccuracies in a child level analysis.
The second reason for focusing on the family level has to do with a "clustering" effect in the child level analysis. Clustering refers to the lack of independence between children within the same family of observations of such things as placement. If one child is removed from the home, the remaining children are more likely to experience placement. The "clustering effect" leads to an underestimate of the significance levels when analyses are conducted at the child level. Conducting the analyses at the family level is one approach to resolving this dilemma.
We did conduct a few analyses at the child level, when we wanted to take into account child characteristics, but it should be remembered that significance levels in those analyses are downwardly biased.
26. In Kentucky, the ratio of assignment to experimental and control groups was 50-50.
27. In New Jersey, approximately 60 percent of the cases were assigned to the experimental group.
28. In Tennessee, approximately two-thirds of the cases were assigned to the experimental group.
29. Kentucky policy specifies that imminent risk includes children who are at risk of commitment as dependent, abused, or neglected; who are identified through the Regional Interagency Council, an interdepartmental unit, as severely emotionally disturbed; or whose families are in conflict such that they are unable to exercise reasonable control of the child. Both the referring worker and family members shall believe that without immediate intensive intervention, out-of-home placement is imminent. At the time of this study, New Jersey targeted family preservation services for families at imminent risk of having at least one child enter placement. The referring worker must have based the assessment of imminent risk on a face-to-face interview with the family no more than 5 days prior to the referral. The family must need services immediately and the worker must determine that other, less intensive, services have been used, are not appropriate, or are not available. In Tennessee, CPS intake workers complete a risk assessment form to identify high, intermediate, low, or no risk. High risk cases are identified as cases where "the child or children in the home are at imminent risk of serious harm if there is no intervention in the situation." A typical high risk case might involve such factors as: 1) a vulnerable child; 2) a history of previous maltreatment; 3) an active perpetrator who has continued access to the child; and 4) no available support or family strengths to offset the stated risks.
30. Analyses were also done on all allegations, whether substantiated or not. The results were very similar, although, of course, rates for all allegations were higher.
31. The six months analyses and survival analyses are obviously not independent.
32. Often we used average responses or proportions of positive responses rather than sums of responses to items. This was done in order to have scores for individuals when there were a few missing items on the scales. If an individual had too many missing items (usually 1/3rd or more) the score was declared missing. Rules for the calculation of all scales are given in Appendix J.
33. In multivariate repeated measures analysis, three main hypotheses are tested, first, that the scores for the experimental group, averaged over the three points in time are equal to those of the control group, (the "group" hypothesis); second, that the averages of the groups at each point in time are the same (the "time" hypothesis); and third, that there is no interaction between time and group. It is the third hypothesis that is central, indicating whether the groups change in different ways.
34. Variables in Tables 3-3, 3-4 and Figure 3-4 are described in Vol. 3, Appendix J.
35. This difference was slightly greater and statistically significant in the secondary analysis (48% vs. 35%, p= .04).
36. In the secondary analysis, fewer experimental group respondents reported health problems (12% vs. 21% for the control group, p = .04).
37. The control group had a slightly lower average proportion of affirmative responses to these items at post-treatment (.17 vs. .22, p = .16).
38. In the primary analysis, at post-treatment, a greater proportion of the experimental group reported difficulties paying rent (20% vs 13%, p = .13) and electric or heat bills (28% vs. 20%, p = .11). In the secondary analysis, differences were smaller and p-values for both items were above .20.
39. This difference was maintained but not significant in the secondary analysis (5% vs. 1%, Fisher's exact p-value = .078).
40. In the secondary analysis, there was again a .09 reduction in the average proportion of punishment items endorsed by the experimental group and a .04 reduction for the control group (p = .03).
41. Derogatis, L. R., Lipman, R. S., & Covi, L. (1973) SCL-90: An outpatient psychiatric rating scale -- preliminary report. Psychopharmacology Bulletin, 9 (1), 13 - 28.
42. Reliability analysis yielded a Cronbach's alpha of .92 at initial, .93 at post-treatment, and .92 at follow-up in Kentucky; .95 at initial, .94 at post-treatment, and .95 at follow-up in New Jersey; and .91 at both initial and post-treatment, and .90 at follow-up in Tennessee.
43. This difference was also significant for the secondary analysis (28% vs. 33%, p = .006).
44. In the secondary analysis, the difference was maintained and remained significant (31% vs. 24%, p = .0004).
45. In the secondary analysis, the average percents were 24 percent for the experimental group and 21% for the control group (p = .06).
46. In the secondary analysis, however, the difference increased and approached significance with 29 percent for the experimental group and 24 percent for the control group, p = .06.
47. The difference for the secondary analysis was also not significant (25% vs. 28%, p = .12).
48. The significant interactions with experimental group were as follows. For depression at post-treatment in New Jersey, there was an interaction of experimental group with single motherhood; for single mothers, there was no relationship between experimental group and depression, for other caretakers, the control group had higher depression scores. Also for depression at post-treatment in New Jersey, there was an interaction with employment; for those employed at the initial interview, there was no difference between the experimental and control groups, for those unemployed, the control group had higher depression scores. For negative life events at post-treatment in Tennessee, there was an interaction with income support; for those not receiving income support the control group had more negative life events, for those receiving income support, there was no difference between the experimental and control groups in negative life events. For household condition at follow-up in Tennessee, there was an interaction between age of caretaker and experimental group; in the control group there was no relationship between age and household condition while in the experimental group, older caretakers had worse household conditions.
49. Brown, D., Ahmed, F., Gary, L., & Milburn, N. (1995) Major depression in a community sample of African Americans. American Journal of Psychiatry 152(3), March 373-378.
50. Humke, C. & Schaefer, C. (1995) Relocation: A review of the effects of residential mobility on children and adolescents. Psychology; a quarterly journal of human behavior, 32(1), 16-24.
51. Honig, A. & Pfannestiel, A. (1991) Difficulties in reaching low-income new fathers: Issues and cases. Early Child Development & Care 77, 115-125.
52. Baxter, A., & Kahn, J. (1999) Social support, needs and stress in urban families with children enrolled in an early intervention program. Infant-Toddler Intervention 9(3), September 239-257.
53. The differing results for the uncontrolled analysis and the regression analysis may be due to the significant interaction in the regression equation of experimental group and income support.
54. The questions were: have you lost your temper when your children got on your nerves, have you found that hitting your child was a good way to get him/her to listen, have you sometimes found yourself hitting your child harder than you meant to, have things sometimes gotten out of control when you punished your child, have you punished your child by tying him/her up with a rope, cord, string, or belt, have you sometimes punished your child by not letting him/her into the house, have you punished your child for not finishing the food on his/her plate.
55. Because the dependent variable was dichotomous, the logit link function was used, transforming the outcome into log-odds. Hence, the analysis actually used a hierarchical non-linear model.
56. The reader is reminded of the findings reported in Chapter 7 indicating that experimental group caretakers generally had more positive views of service and of their relationships with workers than control group caretakers.
4. Philadelphia
4.1 Introduction
Philadelphia was selected for the evaluation because it employed an alternative, somewhat less intensive, longer term approach, and therefore provides some contrast with the Homebuilders sites. In addition, the programs in Philadelphia that were examined were thought to focus on families with substance abuse problems which was not the case in the other sites.
As in the other states, the design for the evaluation in Philadelphia was an experiment in which families were randomly assigned to either the family preservation program (the experimental group) or to other, "regular," services of the child welfare system (the control group). In Philadelphia, both the experimental group and control group received services from private agencies under contract with the public child welfare agency. The public agency has a specialized family preservation unit that develops selection criteria, approves families to receive family preservation services and works closely with the private providers. All other in-home services, known as SCOH (Services to Children in their Own Homes), are delivered by private providers and monitored by the Department's caseworkers. During the evaluation period, experimental cases received family preservation services and control cases received SCOH services. Family preservation is a three-month program that requires workers to spend at least 5 to 10 hours per week with the family.
A description of data collection and sample size in Philadelphia is found in Volume 1, Chapter 6 of this report.
4.2 The Philadelphia Families
Descriptive information about the Philadelphia families was gathered from the initial interviews with caretakers (n = 263) and is summarized in Table 4-1. Because families were randomly assigned, we would expect the families in the experimental and control groups to be similar at the time of random assignment, and for that reason, the sample is described as a whole. Differences between the two groups were examined and there were no characteristics on which the groups differed to a statistically significant degree.
The respondents were primarily women (95%). Most (91%) of the respondents were birth mothers, 5 percent were biological fathers, just under 3 percent were grandparents, and
| N | % | |
|---|---|---|
| Gender of caretaker/respondent | 263 | |
| Male | 5.3 | |
| Female | 94.7 | |
| Race of caretaker/respondent | 263 | |
| African American (not Hispanic) | 80.0 | |
| Caucasian (not Hispanic) | 15.0 | |
| Hispanic | 2.3 | |
| Other | 1.9 | |
| Respondent's education level | 263 | |
| Elementary school or less | 3.8 | |
| Some high school | 61.0 | |
| High school graduate or obtained GED | 18.7 | |
| College | 11.0 | |
| Special education or vocational schooling | 4.2 | |
| Respondent's marital status | 263 | |
| Married | 9.5 | |
| Divorced | 6.8 | |
| Separated | 11.0 | |
| Widowed | 3.0 | |
| Never married | 69.0 | |
| Respondent's relationship to youngest child | 263 | |
| Birth mother | 90.5 | |
| Biological father | 4.6 | |
| Grandparent | 2.7 | |
| Other relative | 2.3 | |
| Household composition | 263 | |
| Birth mother, no other adults | 49.8 | |
| Birth mother & 1 male adult | 19.8 | |
| Birth mother & extended family* | 18.6 | |
| Biological father* | 4.6 | |
| Other relative caretaker* | 4.6 | |
| Other** | 2.7 | |
| N | Mean | |
| Age of respondent | 260 | 31.78 |
| Age of youngest child | 263 | 3.36 |
| Age of oldest child | 263 | 9.83 |
| Number of kids | 263 | 3.40 |
| Number of adults | 263 | 1.60 |
| * These categories may also include other non-related adults in the home **Includes: nonrelative caretaker, adoptive or step-parent, birth mother & non-related females, or birth mother, and more than one non-related male | ||
just over 2 percent were other relatives. The racial composition of the respondents was mostly African American (not Hispanic), at 80 percent, with 15 percent Caucasian (not Hispanic), and the remainder Hispanic and other. The average age of the respondents was 32 (s.d. = 9.11). Just under 4 percent of the respondents had less than a high school level education, 61 percent had some high school, 19 percent had graduated from high school or obtained a GED, 11 percent had at least some college education, and 4 percent had special education or vocational schooling. Approximately 10 percent of the respondents indicated they were married, 7 percent divorced, 11 percent separated, 3 percent widowed, and 69 percent never married. Twenty-two percent reported that they were living with a spouse or partner. At the time of the first interview, 17 percent of the respondents indicated they were employed, 43 percent were unemployed and looking for work, and 40 percent were unemployed and not looking for work. Overall, 65 percent of the respondents rented their homes.
On average, these families were comprised of 1.6 adults and 3.4 children for an overall average family size of 5.0 persons. The average age of the youngest child in the family was 3.45 years (s.d. = 3.75), and the average age of the oldest child in the family was 9.8 years (s.d. = 4.47). Respondents were also asked to provide information regarding the relationship of other adults in the home relative to the youngest child in the home. This information was then used to determine household composition for these families. Approximately one half of the households were headed by a single birth mother, 20 percent had a birth mother residing with one male adult, 19 percent had a birth mother and extended family, 5 percent were headed by a biological father, and 5 percent were headed by another relative caretaker.
4.2.1 Family Problems
Problems and strengths identified by Philadelphia caretakers are summarized in Table 4-2. Most (96%) respondents felt they were "doing a pretty good job raising [their] kids." Still, data from the time one interviews provides us with some sense of the difficulties these families faced as caretakers were asked whether or not they had experienced certain problems in the last month. With regard to emotional problems, 62 percent of the respondents reported feeling "blue or depressed," 53 percent reported feeling nervous or tense, 52 percent were overwhelmed by work or family responsibilities, 33 percent said they had just wanted to give up at some point in the last month, and 35 percent felt they had few or no friends.
With regard to financial difficulties, 56 percent responded that in the past month they did not feel they had enough money for food, rent, or clothing. In response to more specific
| Problems | Control | Experimental | p |
|---|---|---|---|
| Felt blue or depressed | 58 | 65 | |
| Felt nervous or tense | 50 | 54 | |
| Just wanted to give up | 35 | 32 | |
| Overwhelmed with work or family responsibility | 48 | 55 | |
| Felt you had few or no friends | 35 | 35 | |
| Not enough money for food, rent, or clothing | 60 | 54 | |
| Gotten in trouble with the law | 0 | 3 | |
| Had too much to drink in a week | 7 | 3 | |
| Used drugs several times a week | 6 | 10 | |
| Economic Items | |||
| Had difficulty paying rent | 25 | 25 | |
| Had difficulty paying electric/heat | 38 | 35 | |
| Had difficulty buying enough food | 18 | 21 | |
| Had difficulty buying clothes | 40 | 40 | |
| Positive Items | |||
| Have you felt happy | 69 | 77 | |
| Gotten together with anyone to have fun/relax | 0 | 3 | |
| Doing a pretty good job raising kids | 95 | 97 | |
questions about difficulties paying bills in the past 3 months, 25 percent reported difficulty paying rent, 36 percent reported difficulty paying electric or heating bills, 20 percent difficulty buying food for the family, and 40 percent difficulty buying clothes for their children.
Although the Philadelphia program was intended to be focused on substance abuse, only five percent of respondents acknowledged having too much to drink several times a week, and 8 percent reported using drugs several times a week. When caretakers were asked whether a child or children they care for went through alcohol or drug withdrawal when born, 8 percent responded affirmatively. (57) Less than 2 percent of respondents indicated they had gotten in trouble with the law in the past month.
4.2.2 Child Problems
Table 4-3 shows caretakers' responses to questions about problems of children in the families. About three-fifths of respondents reported that their child "gets upset easily," and almost three-fourths reported that at least one child throws tantrums. Approximately one-fifth of the caretakers reported school problems for a child in the family; 19 percent had been absent a lot, 25 percent had failed a class, and 22 percent had been temporarily suspended from school. Approximately one-third of the caretakers reported aggressive behavior by the child; 31 percent indicated a child in their family "fights a lot with other kids" and 33 percent reported that the child is aggressive toward the caretaker.
4.2.3 Caretaker Abuse or Neglect as a Child
When asked two separate questions about whether they had been abused or neglected as a child, 32 percent of the respondents reported having been abused and 23 percent neglected. Eighteen percent responded affirmatively to both questions, and overall, 37 percent of the caretakers reported having either been abused, neglected, or both as a child. Thirteen percent of caretakers had been in a foster home or institution. Experimental and control groups did not differ significantly with respect to these previous experiences. (58)
4.2.4 Previous Allegations and Placement
Historical reports of maltreatment and historical records of placement in substitute care were contained within the administrative data files. Three hundred and thirty-one (95%) of the families had been investigated for maltreatment prior to random assignment. Two hundred and eighty-three (81%) of the families had experienced at least one substantiated allegation prior to random assignment. (59) The administrative files reported four types of allegations; physical abuse, neglect, sexual maltreatment, and other. The allegation just prior to random assignment was of primary interest. This particular allegation provides some indication of reason for referral to
| N | % | |
|---|---|---|
| Asked about all children | ||
| Child went through alcohol withdrawal at birth | 262 | 3 |
| Child went through drug withdrawal when born | 262 | 8 |
| Child doesn't show much interest in what is going on | 259 | 17 |
| Child is smaller/lighter than other children | 262 | 21 |
| Child get(s) upset easily | 259 | 59 |
| Asked for children over 3 months old | ||
| Is/Are funny and makes you laugh | 251 | 99 |
| Like(s) to share things with others | 248 | 90 |
| Throw(s) tantrums | 251 | 70 |
| Is/are shy and withdrawn | 251 | 36 |
| Is/are outgoing and friendly | 252 | 97 |
| Is/are good looking | 252 | 100 |
| Fight(s) a lot with other kids | 247 | 31 |
| Has/have language problems | 246 | 18 |
| Asked for children over 4 years old | ||
| Is/are very aggressive toward you | 217 | 33 |
| Has/have a special talent in music | 217 | 51 |
| Like(s) animals | 217 | 95 |
| Is/are good at sports | 216 | 79 |
| Usually does the right thing | 217 | 86 |
| Hangs with friends you don't like | 215 | 25 |
| In the past 3 months, has any child you care for | ||
| Gone to church regularly | 216 | 42 |
| Been absent from school a lot | 207 | 19 |
| Run away from home overnight | 199 | 5 |
| Been temporarily suspended from school | 205 | 22 |
| Been expelled from school | 205 | 4 |
| Taken care of younger children | 204 | 36 |
| Took something that didn't belong | 216 | 24 |
| Absent from school/no good reason | 205 | 9 |
| Received special education at school | 206 | 25 |
| Failed any classes | 205 | 25 |
| Received counseling | 205 | 28 |
| Asked for any child over age 7 | ||
| In the last 3 months, has any child been arrested | 174 | 7 |
| Asked only for children over age 10 | ||
| Has child age 11 or older had alcohol problems | 115 | 0 |
| Has child age 11 or older had a drug problem | 114 | 4 |
| Has any girl age 12 to 18 been pregnant | 57 | 4 |
| Has any boy age 14 to 18 fathered a child | 19 | 0 |
family preservation. The distribution of last substantiated allegation prior to random assignment is as follows: 29 percent physical abuse, 66 percent neglect, 2 percent sexual maltreatment, and 3 percent other.
Regarding substitute care placement, 131 children in 63 (18%) families had experienced placement prior to random assignment. For these cases, on average, 42.5 months elapsed between the last day of care and random assignment. In the placement spell just prior to random assignment the average length of time in substitute care was 6.7 months. (60)
4.2.5 Social Program Participation
In the initial interview, respondents were asked whether they or anyone else in the household had participated in various social programs within the past 3 months. The overall rates of participation are provided in Table 4-4. Approximately four-fifths indicated that they received food stamps, just over two-thirds received AFDC, slightly less than half received WIC, about one quarter received social security disability, and less than one-tenth received a housing voucher. Overall, respondents indicated that they participated in an average of 2.3 of the 5 income support programs listed (s.d. = 1.2) and 90 percent of the sample participated in at least one of the five programs. There were no significant differences between experimental and control groups in the rate of participation in income support programs. Reports of participation in alcoholism, drug treatment, marriage counseling, and job training programs were less than 10 percent for each. Slightly less than a third of the sample reported participation in Head Start or another pre-school program.
4.3 Summary of Sample Description
Most of the respondents to the first interview were women and birth mothers of the youngest child in the home. Eighty percent of the respondents were African-American (not Hispanic) and 15 percent were Caucasian. Approximately two-thirds of the respondents had not graduated from high school, slightly more than two-thirds were never married, and over four-fifths were unemployed. About half of the households were headed by a single birth mother, and
| Program | Percent |
|---|---|
| Food stamps | 80.2 |
| Job training | 23.3 |
| WIC | 46.2 |
| AFDC | 70.1 |
| Housing vouchers | 7.3 |
| Social security disability | 25.2 |
| Alcoholism program | 5.7 |
| Drug treatment program | 14.5 |
| Marriage counseling | 0.8 |
| Community mental health program | 11.5 |
| Head Start/pre-school | 36.9 |
the average age of the respondents was just under 32 years. On average there were 1.6 adults and 3.4 children in the home, with the average age of the youngest child 3.4 years and the average age of the oldest child 9.8 years.
Over half of the respondents answered affirmatively to each of three questions about emotional difficulties: "feeling blue or depressed," "feeling nervous or tense," and "feeling overwhelmed with work or family responsibility." More than half of the respondents also indicated that they did not have enough money for food, rent, or clothing. Eleven percent said they had problems with alcohol or drugs, and just over one-third reported that they had been abused, neglected, or both as a child.
Ninety percent of the respondents indicated that they participated in at least one of 5 income support programs: AFDC, food stamps, WIC, Social Security disability, and housing vouchers. The rate of participation was less than 10 percent for alcoholism treatment, drug treatment, marriage counseling, and job training. About one-third of the respondents indicated participation in Head Start or another pre-school program.
Ninety-five percent of the families in the study had an investigation prior to their referral for family preservation services and 80 percent had at least one substantiated allegation. Seventeen percent of the families experienced placement of at least one child prior to the referral for family preservation services. With respect to sample characteristics, there were no significant differences between the experimental and control groups.
4.4 Services
4.4.1 Services During the Treatment Period
In the second interview with caretakers and caseworkers, we asked questions about services offered and received during the period since random assignment. Experimental and control group responses to these questions were compared. This analysis documents the services received by both groups (thereby beginning to get into the "black box" of services) and determines whether the experimental group in fact did receive more services and more intense services than the control group.
Caseworker Activities. Caretakers were asked to indicate whether the caseworker provided help with a number of specific problems. Table 4-5 shows the number of affirmative responses in each group. According to caretakers, the most common activities in which workers engaged were discussing discipline, providing transportation, and telling caretakers about other agencies that offer services. Of the 19 items on which caretakers were questioned, the control group workers reportedly engaged in one activity, advising on substance abuse, more than the experimental group workers (26% vs. 18%), however, the difference was not statistically significant (p = .16). For 8 of the 19 items, experimental group workers reportedly engaged in the activity significantly more often than control group workers (all at p = .05 or less). (61) The activities engaged in more often by experimental group workers include: help with money for other things, providing transportation, talking with caretaker about discipline, advising how to get medical care, discussing how to get a better place, advising on job training programs, talking about how to get a paying job, and advising on how to continue school.
| Control % | Experimental % | p | |
|---|---|---|---|
| Caseworker helped with money for rent/elect./phone | 3 | 4 | |
| Caseworker helped with money for other things | 5 | 22 | .001 |
| Caseworker provided transportation | 35 | 50 | .03 |
| Caseworker discussed proper feeding of child | 22 | 28 | |
| Caseworker talked with you about discipline | 32 | 53 | .002 |
| Caseworker talked with you on relations with spouse | 13 | 20 | |
| Caseworker helped you clean house | 6 | 7 | |
| Caseworker helped with painting/house repairs | 01 | 4 | |
| Caseworker discussed how to get childcare | 21 | 32 | .08 |
| Caseworker helped with welfare/food Stamps | 4 | 9 | |
| Caseworker advised how to get medical care | 10 | 23 | .02 |
| Caseworker talked with you how to handle anger | 31 | 37 | |
| Caseworker advised you on substance abuse | 26 | 18 | |
| Caseworker discussed with you how to get a better place | 25 | 38 | .05 |
| Caseworker advised on job training programs | 23 | 36 | .04 |
| Caseworker talked about how to get a paying job | 19 | 33 | .02 |
| Caseworker advised on how to continue school | 21 | 34 | .03 |
| Caseworker arranged for some childcare | 5 | 8 | |
| Caseworker told you about other agencies | 39 | 47 |
A total count of the number of these 19 caseworker activities reported by caretakers also shows significant differences between the experimental and control groups. Caretakers in the experimental group reported an average of 4.6 caseworker activities (n = 148, s.d. = 3.8) while caretakers in the control group reported an average of 2.9 caseworker activities (n = 113, s.d. = 3.1) (p = .001). (62) When asked which of the caseworker activities were especially helpful, experimental group caretakers judged significantly more activities to be helpful than did control group caretakers (2.2 vs. 1.5, p = .02). (63)
Social Program Participation. In the second interview, caretakers were again asked about their participation in the set of social programs listed in Table 4-6, except this time they were asked to report their participation since the time of the first interview. The proportions of involvement were remarkably similar to those in the first interview with a change of 2 percent or
| Program | C % | E % | p |
|---|---|---|---|
| Food stamps | 78 | 80 | |
| Job training | 20 | 26 | |
| WIC | 40 | 44 | |
| AFDC | 65 | 70 | |
| Housing vouchers | 10 | 5 | |
| Social security disability | 29 | 23 | |
| Alcoholism program | 8 | 7 | |
| Drug treatment program | 13 | 16 | |
| Marriage counseling | 2 | 2 | |
| Community mental health program | 13 | 13 | |
| Head Start/pre-school | 29 | 32 | |
| Note: C = Control Group, E = Experimental Group | |||
less for most programs. Exceptions to this were a 4.1 percent decrease in participation in WIC services, and a 6.1 percent decrease for participation in Head Start or Pre-school programs, perhaps due to the aging of youngest children. There were no significant differences between experimental and control groups for participation in any of the social programs at the time of the second interview. No significant differences were found with respect to the total number of income support programs or treatment programs in which respondents participated since the time of the first interview.
Caretakers' Reports of Services. In the second interviews, caretakers were asked if they had received any of a set of specific services in the time since the first interview. As shown in Table 4-7, there were no significant differences between the experimental and control groups in receipt of any of these services in the primary analysis. (64) In a separate question, caretakers were asked whether the agency provided homemaker services or the assistance of a parent aide.
| Control % | Experimental % | p | |
|---|---|---|---|
| Daycare | 11 | 15 | |
| Help in finding a place to live | 9 | 9 | |
| Staying at an emergency shelter | 4 | 3 | |
| Medical or dental care | 33 | 39 | |
| Transportation | 25 | 39 | |
| Education services/GED | 9 | 11 | |
| Parent education/training classes | 16 | 37 | |
| Legal services | 7 | 10 | |
| Counseling | 21 | 26 | |
| Respite care | 3 | 3 | |
| Homemaker services | 1 | 1 | |
| A parent aide to help you | 1 | 3 |
Fewer than 3 percent of all caretakers reported having a homemaker or receiving assistance from a parent aide, with no significant differences reported between the experimental and control groups (again, see Table 4-7 for details). When caretakers were asked whether they did not receive any services they felt were needed, 21 percent of the control group responded affirmatively and 18 percent of the experimental group responded affirmatively, a difference that was not statistically significant. (65)
Relationship with Caseworker. Table 4-8 shows results from a number of questions in which caretakers were asked about their relationships with caseworkers. Reports of the quality of the relationship were positive overall, with more than half of the respondents from both the experimental and control group indicating that the caseworker listened to their concerns, understood their situation, and agreed on the goals most of the time. A greater proportion of
| Control % | Experimental % | p | |
|---|---|---|---|
| Worker listened to your concerns most of the time | 81 | 80 | |
| Worker understood your situation very well | 82 | 82 | |
| You and worker agreed on goals most of the time | 67 | 71 | .10 |
| Did worker sometimes talk with you about issues that were not easy to talk about? | 27 | 36 | |
| Caseworker helped you to see your good qualities | 68 | 82 | .01 |
| Caseworker helped you to see your problems | 74 | 76 | |
| Did you see your caseworker | |||
| More often than you wanted | 21 | 25 | |
| As often as you wanted | 59 | 57 | |
| Not often enough | 20 | 18 |
experimental group caretakers felt their workers helped them see their good qualities (82% vs. 68%, p = .01). With respect to the frequency of contact with the workers, approximately 20 percent of caretakers from both the experimental and control groups indicated they did not see their caseworkers often enough. A slightly greater proportion of caretakers in the experimental group indicated they saw their workers "more often than [they] wanted" (25% vs. 21%) and a slightly greater proportion of caretakers in the control group indicated they saw their workers "as often as [they] wanted" (59% vs. 57%). (66)
Caseworkers' Reports of Services. In the second interview, caseworkers were asked whether they had helped any member of the family with any of 25 services, such as child care, homemaker services, income programs, treatment programs of various sorts, and health care. Table 4-9 provides a list of these 25 items and the proportion of caseworkers who indicated this service was provided. Caseworkers from the experimental group reported helping their clients with an average of 4.9 of these services (s.d. = 3.5), while caseworkers from the control group reported helping their clients with an average of 3.8 of these services (s.d. = 3.0; p = .0004). (67) In the primary analyses, an examination of individual services reveals 6 services that were provided significantly more often to the experimental group than to the control group (significance levels were all at p = .05 or less). These services include: childcare or babysitting, parent training, other housing assistance, emergency financial assistance, recreational services, and household management. No services were provided significantly more often to the control group. (68)
| Control % | Experimental % | p | |
|---|---|---|---|
| Childcare or baby sitting | 6 | 16 | .02 |
| AFDC or other public income (except SSI) | 6 | 12 | |
| SSI for adult or child | 3 | 0 | .06 (FE) |
| Food stamps | 6 | 9 | |
| Drug treatment | 17 | 16 | |
| Alcoholism treatment | 8 | 8 | |
| Legal aid | 4 | 9 | |
| Help with education | 19 | 24 | |
| Respite care | 6 | 6 | |
| Parent training | 34 | 62 | .001 |
| Health care | 28 | 35 | |
| Inpatient mental health | 2 | 2 | |
| Outpatient mental health/counseling | 20 | 24 | |
| Health assessment | 23 | 31 | |
| Housing financial assistance | 3 | 8 | .08 |
| Other housing services | 10 | 21 | .02 |
| W.I.C. | 8 | 6 | |
| Emergency financial assistance other than housing | 23 | 54 | .001 |
| Job training | 7 | 11 | |
| Emergency shelter | 1 | 2 | |
| Recreational services | 26 | 38 | .05 |
| Family planning | 23 | 21 | |
| Self help groups | 9 | 16 | |
| Household management | 21 | 38 | .004 |
| Homemaker services | 6 | 9 | |
| Other | 16 | 13 | |
| N | 99 | 151 | |
| Note: "FE" indicates significance determined by Fisher's exact test | |||
Contact Data. One page contact reports were to be completed by all caseworkers for each face-to-face contact with a family member during the time period designated for family preservation services. On these forms, workers provided information about the date, time, persons involved, and the services delivered during each contact. Some data on contact forms are presented in Table 4-10. At least one contact form was submitted for 85 (59%) of the control group cases and 124 (59%) of the experimental group cases. The following analyses were limited
| C | E | p | |
|---|---|---|---|
| Number of cases with at least one form submitted | 85 | 124 | |
| Average number of forms per case | 9.9 | 17.6 | .01 |
| Average number of home visits | 7.8 | 13.9 | .01 |
| Average number of visits with caretakers | 8.2 | 15.4 | .01 |
| Average number of visits with the other parent | 1.2 | 1.5 | |
| Average number of visits with children | 7.3 | 13.3 | .01 |
| Average number of contacts with service | |||
| Concrete Services | C | E | P |
| Transportation | 2.5 | 5.2 | .01 |
| Buying food | 1.3 | 1.0 | |
| Child care | 2.1 | 1.2 | .06 |
| Clothing, furnishings, and supplies | 0.9 | 1.5 | .07 |
| Topics of Discussion | |||
| Discipline of children | 3.5 | 4.4 | |
| Goals | 5.7 | 7.2 | .10 |
| Caretaker's interaction with children | 4.4 | 4.2 | |
| Child's anger management | 1.8 | 2.0 | |
| Supervision of children | 4.1 | 4.3 | |
| Money management | 3.4 | 5.1 | .05 |
| Employment | 2.7 | 4.6 | .01 |
| Conflict among adults | 1.9 | 2.7 | .10 |
| Note: C = Control Group, E = Experimental Group The contact forms contained additional concrete services and topics of discussion (see Appendix K, Volume 3). Only those that were most often reported are shown here. Entries are average numbers of times per family that an item was reported, for those families with at least one form submitted. | |||
to those families with contact reports. On average, more contact forms were submitted for the experimental group than for the control group (17.6 vs. 9.9; p < .001). In addition to the overall number of reports submitted, the experimental group received significantly more home visits (13.9 vs. 7.8; p < .001), visits with caretakers (15.4 vs. 8.3; p < .001), and visits with children (13.3 vs. 7.3; p < .001). As experimental group families received significantly more contacts than the control group families, they also received significantly more individual activities. The most common concrete service was the provision of transportation (an average of 5.2 times for the experimental group vs. 2.5 times for the control group; p < .001). Child care was the second most common concrete service (an average of 1.2 times for the experimental group vs. 2.1 times for the control group; p < .1). Additional concrete services included clothing, furnishings, and supplies (1.5 times for the experimental group vs. 0.95 for the control group; p < .1).
Contact forms also captured general information about the topic of discussion, counseling, or instruction. The most common topics of discussion were the goals of working together (7.2 times in the experimental group vs. 5.7 times in the control group; p < .1), money management (5.1 vs. 3.4; p < .05), employment (4.6 vs. 2.7; p < .01), and conflict among adults (2.7 vs. 1.9; p < .1).
Additional data on experimental group contacts are shown in Table 4-11. These data confirm the fact that services to the experimental group often did not begin until sometime after random assignment. Only 8 percent of the cases had an in-home contact within 7 days and a relatively small proportion of contacts occurred in the first month.
| N | % | |
|---|---|---|
| Number of families with contact data | 124 | 60 |
| Total number of contact forms submitted | 2182 | |
| Contacts in week 1 | 18 | 1 |
| Contacts in month 1 | 479 | 16 |
| Contacts in month 2 | 912 | 30 |
| Contacts in month 3 | 825 | 27 |
| In-home contact within 72 hours | 3 | 2 |
| In-home contact with 7 days | 10 | 8 |
| Concrete service within 7 days | 8 | 7 |
| Hours of contact | N | Mean |
| Average hours of contact overall | 122 | 34.1 |
| Average hours contact in month 1 | 89 | 8.8 |
4.4.2 Summary of Services During the Treatment Period
The caretaker interview, the caseworker interview, and the contacts data all confirmed the expectation that the experimental group would receive more and more intensive services than the control group. At time two, caretakers in the experimental group reported an average of 4.6 caseworker activities as compared to 2.9 for the control group. In addition to caseworker activities, caretakers were asked about specific services received. The following services were among those most often reported by caretakers: counseling, transportation, parent education or training, and medical or dental services. Differences between the experimental and control groups for the caretaker interview data include a significantly greater proportion of experimental group caretakers responding affirmatively to nine questions regarding caseworker activities. These nine activities and the response rates are provided in Table 4-12.
Consistent with the information provided by caretakers, caseworkers reported providing more services to families in the experimental group (an average of 4.9 services) than those in the control group (an average of 3.8 services, p = .001). For 6 of the specific services listed, a greater proportion of caseworkers in the experimental group reported providing services as compared to caseworkers in the control group (see Table 4-12).
The contact form data were consistent with both the caretaker and caseworker interview data in supporting the conclusion that the experimental group families received more services than did the control group families. An average of 17.6 contact forms were received for the experimental group as compared to an average of 9.9 contact forms for the control group. The contact forms also indicate that the experimental group received home visits, visits with the caretaker, and visits with the children significantly more often than did the control group. In response to questions pertaining to the nature of the relationship with the caseworker, caretakers from the experimental group were significantly more likely to indicate their workers helped them to see their good qualities.
4.4.3 Services During the Followup Period
4.4.4 Summary of Followup Services.
When caretakers were interviewed a year after random assignment, they were asked some of the same questions about services received, this time since the last interview (since the end of family preservation services for the experimental group and during a comparable period for the control group). Tables 4-13, 4-14, and 4-15 show analyses of these questions.
| Caseworker Activities: | |||
|---|---|---|---|
| Proportion of affirmative answers by caretakers to yes/no questions | Control % | Experimental % | p |
| Caseworker helped with money for other things | 5 | 22 | .001 |
| Caseworker provided transportation | 35 | 50 | .03 |
| Caseworker talked with you about discipline | 32 | 53 | .002 |
| Caseworker advised how to get medical care | 10 | 23 | .02 |
| Caseworker discussed with you how to get a better place | 25 | 38 | .05 |
| Caseworker advised on job training programs | 23 | 36 | .04 |
| Caseworker talked about how to get paying job | 19 | 33 | .02 |
| Caseworker advised on how to continue school | 21 | 34 | .03 |
| Caseworker helped you see good qualities | 68 | 82 | .01 |
| Caseworker talked about how to get paying job | 19 | 33 | .02 |
| Control Mean | Experimental Mean | p | |
|---|---|---|---|
| Caretaker report of number of caseworker activities | 2.9 | 4.6 | .0001 |
| Caretaker report of number of "helpful" caseworker activities | 1.5 | 2.2 | .02 |
| Services Provided: | |||
|---|---|---|---|
| Proportion of affirmative answers by caretakers to yes/no questions | Control % | Experimental % | p |
| Childcare or baby sitting | 6 | 16 | .02 |
| Parent training | 34 | 62 | .001 |
| Other housing services | 10 | 21 | .02 |
| Emergency financial assistance | 23 | 54 | .001 |
| Recreational services | 26 | 38 | .05 |
| Household management | 21 | 38 | .004 |
| Control Mean | Experimental Mean | p | |
|---|---|---|---|
| Caseworker report of number of services provided | 3.4 | 4.9 | .0004 |
| Note: This table only includes items with a primary p-value less than or equal to .05 Items in bold indicate significant findings in favor of the experimental group. | |||
| Control % | Experimental % | p | |
|---|---|---|---|
| Caseworker helped with money for rent/electricity/phone | 4 | 4 | |
| Caseworker helped with money for other things | 18 | 16 | |
| Caseworker provided transportation | 29 | 32 | |
| Caseworker discussed proper feeding of child | 16 | 13 | |
| Caseworker talked with you about discipline | 32 | 24 | |
| Caseworker talked with you on relations with spouse | 16 | 8 | .08 |
| Caseworker helped you clean house | 3 | 6 | |
| Caseworker helped with painting/house repairs | 2 | 1 | |
| Caseworker discussed how to get child care | 14 | 17 | |
| Caseworker helped with welfare/food Stamps | 9 | 10 | |
| Caseworker advised how to get medical care | 10 | 14 | |
| Caseworker talked with you how to handle anger | 28 | 19 | .10 |
| Caseworker advised you on substance abuse | 18 | 19 | |
| Caseworker discussed with you how to get a better place | 24 | 18 | |
| Caseworker advised on job training programs | 22 | 21 | |
| Caseworker talked about how to get a paying job | 23 | 16 | |
| Caseworker advised on how to continue school | 29 | 18 | .05 |
| Caseworker arranged for some child care | 2 | 6 | |
| Caseworker told you about other agencies | 33 | 29 |
| Program | C % | E % | p |
|---|---|---|---|
| Food stamps | 79 | 75 | |
| Job training | 21 | 31 | .09 |
| WIC | 38 | 40 | |
| AFDC | 64 | 68 | |
| Housing vouchers | 12 | 10 | |
| Social security disability | 33 | 22 | .07 |
| Alcoholism program | 10 | 04 | |
| Drug treatment program | 17 | 13 | |
| Marriage counseling | 1 | 2 | |
| Community mental health program | 9 | 14 | |
| Head Start/pre-school | 57 | 52 | |
| Note: C = Control Group, E = Experimental Group | |||
| Control % | Experimental % | p | |
|---|---|---|---|
| Day care | 17 | 25 | |
| Help in finding a place to live | 11 | 10 | |
| Staying at an emergency shelter | 4 | 6 | |
| Medical or dental care | 36 | 38 | |
| Transportation | 23 | 28 | |
| Education services/GED | 9 | 16 | |
| Parent education/ training classes | 17 | 29 | .03 |
| Legal services | 7 | 10 | |
| Counseling | 23 | 29 | |
| Respite care | 1 | 1 | |
| Homemaker services | 0 | 1 | |
| A parent aide to help you | 2 | 3 |
Caseworker Activities. Caretaker reports of caseworker activities since the post-treatment interview are shown in Table 4-13. Only one item showed significant differences between experimental and control groups in the primary analysis. Compared to caretakers in the experimental group, a significantly greater proportion of caretakers in the control group reported that their caseworkers advised them on how to continue school (29% vs. 18%; p = .05). (69)
Participation in Social Programs. As indicated in Table 4-14, there were no significant differences between the experimental and control groups with respect to involvement in social programs during the post-treatment period. (70)
Caretaker Report of Services. Table 4-15 indicates that there was only one service in which there is a significant difference between experimental and control groups in reported receipt of services during the post-treatment period. A greater proportion of experimental group respondents reported receiving parent education/training classes (29% vs. 17%; p = .03). (71)
4.4.4 Summary of Followup Services.
There were few significant differences between experimental and control groups on report of service as shown in Table 4-16. A significantly greater proportion of caretakers in the control group reported that, in the period of time since the post-treatment interview, their caseworker advised them on how to continue school. On the other hand, a significantly greater proportion of caretakers from the experimental group reported receiving parent education/training classes since the time of the post-treatment interview.
| Caseworker Activities: (Proportion of affirmative answers to yes/no questions) | Control % | Experimental % | p |
|---|---|---|---|
| Caseworker advised on how to continue school | 29 | 18 | .05 |
| Services Provided: (Proportion of affirmative answers to yes/no questions) | Control % | Experimental % | p |
|---|---|---|---|
| Parent education/training classes | 17 | 29 | .03 |
| Note: Table only includes items with a primary p-value of .05 or less. Items in bold indicate significant findings in favor of the experimental group whereas italicized items indicate significant findings in favor of the control group. | |||
4.5 Outcomes
The following outcome analyses compare the experimental and control groups. As previously discussed, there were a few cases in which the original random assignments were violated, that is control group families were given FPS services or minimal services were provided. There were also cases which are thought to have not received services before the post-treatment interview due to a delay in the assignment of a worker to that case. Therefore, analyses were conducted for the primary analysis group (the original random assignment group), the secondary analysis group (dropping the violations and minimal services cases from the original random assignment group), and, for interview data, the tertiary analysis group (dropping additional cases which appear to have not had a worker assigned in time for the post-treatment interview to determine effects of service). (72) Secondary and tertiary analyses are reported in footnotes.
4.5.1 Substitute Care Placement Following Random Assignment
Family preservation is believed to prevent unnecessary placement in substitute care. Prevention of placement is not as central an objective of family preservation in Philadelphia as in other locations, but it is, nonetheless, an outcome of interest. Table 4-17 provides data at the individual level for type of first placement after random assignment. (73) The administrative files contained subsequent placement data on 349 families, 205 in the experimental group and 144 families in the control group. Although the data were at the child level, the analyses are presented at the family level. (74)
| Philadelphia | ||
|---|---|---|
| Type | N | Percentage |
| Foster care, institution | 98 | 42.1 |
| Foster care, home | 92 | 39.5 |
| Emergency shelter | 29 | 12.4 |
| Foster care, group | 14 | 6.0 |
| Total | 233 | 100 |
In the experimental group, 148 children in 65 families (32%) experienced placement subsequent to random assignment. In the control group, 85 children in 37 families (26%) experienced placement subsequent to random assignment. The differences were not statistically significant at the family level.
As in the other states, a simple comparison of overall percentages is not the most appropriate way to analyze these placement data. As families were randomly assigned at various points in time (between March 12, 1997 and June 23, 1999), the risk periods (amount of time eligible to experience placement) varied between families. The administrative data were collected on October 31, 2000 and therefore the minimum risk period was approximately sixteen months, and the maximum was more than 44 months. Hence, survival analyses were conducted to account for the varying risk periods.
Child level data were aggregated to the family level for the following survival analyses. The family level survival analyses were developed based on all 1,212 children in the administrative data. Families survived if no child experienced subsequent placement. For those families with subsequent placement, the first placement date of any child in that particular family was used to calculate the time interval between random assignment and first subsequent placement. If multiple children were removed from a single home, the date of first placement was selected.
The family level analysis of subsequent placement is displayed in Figure 4-1. The survival curves and Wilcoxon statistic indicate that the survival rates are not statistically different between the experimental and control groups. At the one year interval, 18 percent of experimental group families and 15 percent of control group families experienced substitute care placement in the primary analysis. (75) The survival analyses suggest that there were no differences between the rates of placement in the experimental and control groups. A summary of placement rates at various points in time following random assignment is shown in Table 4-18.
In addition to survival analyses, placement can be examined in terms of the proportion of time in substitute care subsequent to random assignment. The proportion is calculated by dividing the number of days in care by the number of days of possible care (number of days between random assignment and the date of administrative data collection). As the proportions are calculated at the family level, the number of days in care represents the total number of care days summed across all children within a particular family. Similarly, the number of possible care days represents the total number of possible care days summed across all children within a
Figure 4-1
First Placement after Random Assignment (Families)
| 1 month | 6 months | 12 months | 18 months | |||||
|---|---|---|---|---|---|---|---|---|
| C % | E % | C % | E % | C % | E % | C % | E % | |
| Primary analysis | 1 | 1 | 12 | 10 | 15 | 18 | 20 | 24 |
| Secondary analysis | 1 | 1 | 13 | 9 | 16 | 15 | 19 | 21 |
| Note: C = Control Group, E = Experimental Group | ||||||||
particular family. The number of possible care days is adjusted for a child's eighteenth birthday. In the experimental group, children spent an average of 6 percent of the days subsequent to random assignment in care. In the control group, children spent an average of 4 percent of the days subsequent to random assignment in care. This difference is not statistically significant.
4.5.2 Allegations of Maltreatment Following Random Assignment
Two hundred sixty-eight children in 110 families (54%) in the experimental group were the subjects of investigated allegations of maltreatment following random assignment, compared with 161 children in 69 families (50%) in the control group. The difference was not statistically significant at the family level. The distribution of the various types of allegations is as follows: 43 percent physical abuse, 53 percent neglect, and 4 percent sexual maltreatment. One hundred twenty-three children in 60 families (29%) in the experimental group were the subjects of substantiated allegations of maltreatment compared with 67 children in 32 families (22%) in the control group. This difference was not statistically significant at the family level. The distribution of substantiated allegations is as follows: 53 percent physical abuse, 43 percent neglect, and 4 percent sexual maltreatment.
As with the analyses of subsequent placement, survival graphs were developed to compare the timing of subsequent substantiated allegations of maltreatment. (76) Again, survival analyses were conducted for both the primary and secondary analysis groups. Child level data were aggregated at the family level. There were no significant differences between the experimental and control groups. Figure 4-2 displays the survival curves for the primary analysis group. At one year, 20 percent of the experimental group and 13 percent of the control group families experienced substantiated reports of maltreatment subsequent to random assignment. At two years, 25 percent of the experimental group and 18 percent of the control group families experienced substantiated reports of maltreatment subsequent to random assignment. (77) The survival analyses indicate that experimental families did not experience fewer substantiated reports of maltreatment subsequent to random assignment.
Sub-group Analysis. We examined a number of subgroups of cases to determine whether we could detect differences between experimental and control groups on placement and substantiated allegations subsequent to random assignment within each subgroup. The results are shown in Table 4-19. Most of the subgroups were defined in terms of problems existing at the time of the initial interview. For both placement and substantiated allegations the table shows the number of cases in each subgroup, the percentage of cases in the subgroup experiencing the event within 6 months, the significance of the difference between the experimental and control groups in the occurrence of the event within six months, and the p value for the analysis of differences in survival curves. The first row of the table shows the results for the Philadelphia sample as a whole. Except for substance abuse, the definitions of the subgroups were determined from the initial caretaker interview. Very few caretakers acknowledged substance use in the first interview, so that subgroup was determined from information in both the caretaker and caseworker initial interviews.
As can be seen, nearly all of the experimental-control group comparisons shown are not significant. Of the 18 comparisons in the table, only one is significant at .05, that for
Figure 4-2
First Substantiated Allegation after Random Assignment (Families)
| Placement | Substantiated allegations | ||||||
|---|---|---|---|---|---|---|---|
| In 6 mos | In 6 mos | ||||||
| Subgroup | N | Overall % | pa | Survival p | Overall % | pa | Survival p |
| Overall | 349 | 9.5 | 10.0 | .098 | |||
| Substance abuse | 72 | 11.1 | 11.1 | ||||
| No substance abuse | 186 | 9.1 | .076 | 8.6 | |||
| Problems with bills | 151 | 11.9 | 10.6 | ||||
| Problems with daycare | 143 | 9.8 | 10.5 | ||||
| Depressionc | 139 | 12.9 | 8.6 | .089 | |||
| Problems with punishment | 163 | 11.0 | 9.8 | ||||
| Problems with school | 98 | 13.3 | 6.1 | .038b | |||
| Problems with employment | 56 | 10.7 | 7.1 | ||||
| Single mother | 129 | 11.6 | 13.2 | ||||
| a Fisher exact, two tail b Experimental group more likely to experience subsequent allegation c Caretakers with depression scores above median for the site | |||||||
problems with school. Among those who identified a child having problems with school, those in the experimental group were more likely to have a substantiated allegation than those in the control group. In the analysis so far, efforts to find subgroups for which family preservation service was related to reduced placement have been unsuccessful.
4.5.3 Family and Child Functioning - Caretaker Interviews
Life Events. In both the initial and second interviews, caretakers were asked to respond to a 15 item "life events" inventory asking about the occurrence of both positive and negative
events (see Appendix K, Volume 3, Initial Caretaker Interview, p. 7, and Interim Caretaker Interview, p. 8). Three scales were formed from this inventory: positive life events, negative life events, and a scale of those life events that might reflect depression in the caretaker (we had a more formal depression measure as well, described below). In the post-treatment interview, the proportion of positive life events reported by caretakers in the experimental group was significantly higher than the proportion reported by caretakers in the control group (.19 vs. .15; p = .05). (78) The proportion of positive life events reported by caretakers in the experimental group remained higher in the followup interview (.23 vs. .20), however, the difference was not statistically significant. On the measures of negative life events and life events reflecting depression there were no statistically significant differences between the experimental and control groups at the time of the post-treatment or followup interviews. (79)
Problems. In the post-treatment and followup interviews, caretakers were again asked questions about problems in the family. These questions paralleled those asked in the first interview (see Section 4.2.1 Family Problems above, under Section 4.2 The Philadelphia Families), except this time caretakers were asked to respond to questions with regard to the time "since we last spoke to you." Tables 4-21 and 4-22 display these items and the proportion of affirmative responses at the time of the post-treatment and followup interviews. At the time of the post-treatment interview, there were no significant differences between the experimental and control groups responses to any of the nine items about problems in the family. (80) At the time of the followup interview, no significant differences were found on eight of the nine items in the primary analysis. However, on the question about the overall economic condition of the family,
| Post-Treatment | Followup | Multivariate Repeated Measures | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| N | Ma | pb | N | M | p | N | Means | Multivariate ps | Univariate ps-Time | Univariate ps-Grp-time interaction | ||||||||
| Initial | Post | Follow | Grpc | Timed | Time -Grpe | Initial v. laterf | Post v. Followg | Initial v. later | Post v. Follow | |||||||||
| Positive life events | C | 113 | .15 | .05 | 90 | .20 | 70 | .13 | .15 | .21 | .001 | .003 | .001 | |||||
| E | 148 | .19 | 135 | .23 | 102 | .17 | .18 | .23 | ||||||||||
| Negative life events | C | 113 | .06 | 90 | .08 | 70 | .13 | .11 | .14 | .02 | .03 | |||||||
| E | 148 | .05 | 135 | .08 | 102 | .10 | .09 | .12 | ||||||||||
| Life events depression | C | 113 | .38 | 89 | .35 | 69 | .43 | .34 | .33 | .09 | .001 | .001 | ||||||
| E | 148 | .41 | 135 | .41 | 102 | .53 | .40 | .42 | ||||||||||
| Paying bills | C | 113 | .27 | 90 | .21 | .08 | 70 | .33 | .23 | .20 | .02 | .005 | ||||||
| E | 148 | .31 | 134 | .28 | 100 | .32 | .31 | .30 | ||||||||||
| Income support | C | 113 | 2.20 | 90 | 2.25 | 70 | 2.40 | 2.38 | 2.43 | |||||||||
| E | 148 | 2.22 | 135 | 2.14 | 102 | 2.24 | 2.23 | 2.15 | ||||||||||
| Treatment programs | C | 113 | .36 | 90 | .37 | 70 | .39 | .41 | .40 | |||||||||
| E | 148 | .37 | 135 | .32 | 102 | .28 | .39 | .32 | ||||||||||
| Punishment | C | 112 | .16 | 89 | .16 | 69 | .20 | .15 | .15 | .06 | .002 | .001 | ||||||
| E | 148 | .19 | 132 | .17 | 98 | .25 | .20 | .17 | ||||||||||
| Child aggression | C | 113 | 1.32 | 90 | 1.13 | 70 | 1.34 | 1.30 | 1.14 | |||||||||
| E | 148 | 1.25 | 135 | 1.21 | 102 | 1.16 | 1.24 | 1.24 | ||||||||||
| School problems | C | 93 | .14 | 80 | .16 | 56 | .15 | .14 | .17 | .07 | ||||||||
| E | 123 | .13 | 116 | .17 | 79 | .18 | .14 | .19 | ||||||||||
| Child withdrawn | C | 113 | .52 | 90 | .42 | 70 | .51 | .51 | .43 | |||||||||
| E | 148 | .61 | 135 | .54 | 102 | .48 | .63 | .56 | ||||||||||
| Stolen things or arrested | C | 113 | .20 | 90 | .13 | .02 | 70 | .27 | .19 | .10 | .03 | .04 | .01 | .02 | ||||
| E | 148 | .16 | 135 | .26 | 102 | .26 | .18 | .27 | ||||||||||
| Child substance abuse | C | 113 | .03 | 90 | .01 | 70 | .03 | .01 | .01 | |||||||||
| E | 148 | .01 | 135 | .02 | 102 | .00 | .00 | .03 | ||||||||||
| Child problems | C | 113 | 1.59 | 90 | 1.68 | 70 | 1.61 | 1.56 | 1.67 | |||||||||
| E | 148 | 1.76 | 135 | 1.67 | 102 | 1.61 | 1.84 | 1.66 | ||||||||||
| Negative child behaviors | C | 108 | .25 | 88 | .22 | 65 | .26 | .25 | .23 | |||||||||
| E | 143 | .25 | 131 | .25 | 96 | .25 | .25 | .25 | ||||||||||
| Positive child behaviors | C | 109 | .80 | 89 | .79 | 66 | .78 | .79 | .79 | |||||||||
| E | 145 | .81 | 134 | .80 | 97 | .81 | .81 | .79 | ||||||||||
| Household condition | C | 112 | .09 | 90 | .06 | .05 | 69 | .12 | .08 | .06 | ||||||||
| E | 147 | .09 | 135 | .10 | 101 | .10 | .10 | .10 | ||||||||||
| Depression (SCL-90) | C | 113 | .96 | 89 | .79 | 69 | 1.05 | .89 | .78 | .006 | .003 | |||||||
| E | 148 | 1.00 | 135 | .83 | 102 | .98 | .95 | .88 | ||||||||||
| Positive child care practices | C | 108 | .90 | 87 | .91 | 66 | .90 | .90 | .91 | |||||||||
| E | 142 | .88 | 129 | .88 | 93 | .90 | .88 | .88 | ||||||||||
| Negative child care practices | C | 108 | .13 | 86 | .13 | 66 | .15 | .13 | .13 | .008 | .002 | |||||||
| E | 147 | .15 | 130 | .15 | 94 | .19 | .15 | .14 | ||||||||||
| a Means of control and experimental groups b Test of hypothesis of equivalent group means c Test of hypothesis that group means, averaged over time, are equal d Test of hypothesis that means at three points in time, averaged over the groups, are equal e Test of hypothesis of no interaction between group and time, that is, that the pattern of means over time is the same for both groups f Test of hypothesis that time one is equal to average of time two and time three g Test of hypothesis that time two is equal to time three | ||||||||||||||||||
| Control | Experimental | p | |||
|---|---|---|---|---|---|
| N | % | N | % | ||
| Problems | |||||
| Felt blue or depressed | 113 | 45 | 148 | 43 | |
| Felt nervous or tense | 113 | 43 | 147 | 46 | |
| Just wanted to give up | 113 | 22 | 148 | 29 | |
| Overwhelmed with work or family responsibility | 113 | 41 | 146 | 47 | |
| Felt you had few or no friends | 112 | 22 | 148 | 25 | |
| Not enough money for food, rent, or clothing | 113 | 48 | 148 | 59 | .08 |
| Gotten in trouble with the law | 113 | 2 | 148 | 3 | |
| Had too much to drink in a week | 112 | 4 | 148 | 2 | |
| Used drugs several times a week | 113 | 8 | 148 | 6 | |
| Economic Items | |||||
| Had difficulty paying rent | 113 | 19 | 148 | 20 | |
| Had difficulty paying electric/heat | 113 | 28 | 148 | 33 | |
| Had difficulty buying enough food | 113 | 26 | 148 | 31 | |
| Had difficulty buying clothes | 113 | 34 | 148 | 42 | |
| Positive Items | |||||
| Have you felt happy | 112 | 83 | 148 | 86 | |
| Gotten together with anyone to have fun/relax | 113 | 51 | 148 | 53 | |
| Doing a pretty good job raising kids | 112 | 93 | 148 | 95 | |
| Control | Experimental | p | |||
|---|---|---|---|---|---|
| N | % | N | % | ||
| Problems | |||||
| Felt blue or depressed | 90 | 46 | 135 | 49 | |
| Felt nervous or tense | 90 | 38 | 135 | 44 | |
| Just wanted to give up | 89 | 18 | 135 | 25 | |
| Overwhelmed with work or family responsibility | 89 | 38 | 135 | 46 | |
| Felt you had few or no friends | 90 | 31 | 135 | 27 | |
| Not enough money for food, rent, or clothing | 90 | 33 | 135 | 49 | .02 |
| Gotten in trouble with the law | 90 | 0 | 135 | 1 | |
| Had too much to drink in a week | 90 | 3 | 135 | 2 | |
| Used drugs several times a week | 90 | 0 | 135 | 2 | |
| Economic Items | |||||
| Had difficulty paying rent | 90 | 18 | 134 | 20 | |
| Had difficulty paying electric/heat | 90 | 29 | 134 | 29 | |
| Had difficulty buying enough food | 90 | 16 | 134 | 27 | .05 |
| Had difficulty buying clothes | 90 | 21 | 134 | 37 | .01 |
| Positive Items | |||||
| Have you felt happy | 89 | 89 | 135 | 90 | |
| Gotten together with anyone to have fun/relax | 90 | 57 | 135 | 57 | |
| Doing a pretty good job raising kids | 88 | 97 | 134 | 96 | |
"have you felt you did not have enough money for food, rent, or clothing?" 49 percent of the experimental group and 33 percent of the control group responded affirmatively (p = .02). (81)
In addition to the items about problems, caretakers were asked three questions about positive aspects of their lives: "getting together with anyone to have fun or relax," "felt happy and "felt that considering everything you're doing a pretty good job raising your kids." For the experimental and control groups combined, at post-treatment, 85 percent responded affirmatively to the question of whether they "felt happy," 53 percent responded affirmatively to the question of "getting together with anyone to have fun or relax," and 94 percent responded affirmatively that they were "doing a pretty good job raising [their] kids." At followup, 90 percent of respondents (experimental and control groups combined) reported that they "felt happy," 57 percent responded affirmatively to the question of "getting together with anyone to have fun or relax," and 96 percent reported that they were "doing a pretty good job raising [their] kids." (82)
Economic Functioning. In addition to the general item on not having enough money for food or rent, caretakers were asked four specific questions about difficulties in paying for the essentials of living (rent, electricity and heating, food, and clothing). When these items were combined into a scale, all analyses (primary, secondary, and tertiary) revealed no significant differences in the average proportion of affirmative responses to the four items at the time of the post-treatment interview. At the time of the followup interview, the average proportion of affirmative responses to the four items was greater for the experimental group than the control group (.28 vs. .21) but the difference was not statistically significant in the primary analysis (p = .08). (83) Using repeated measures to look at changes in the scale responses over time, results indicate a decline in the average proportion of affirmative responses to this scale of economic functioning for both groups (p = .02). These changes over time did not differ significantly for the experimental and control groups.
Looking at the 4 individual items that comprised the scale of economic functioning, there were no significant differences between experimental and control groups at the time of the post-treatment interview. At the time of the followup interview, primary analyses revealed significant differences between experimental and control group respondents on 2 of the 4 items. A greater proportion of the experimental group respondents reported having difficulty buying enough food (27% vs. 16%; p = .05), and difficulty buying clothes (37% vs. 21%; p = .01). (84)
Household Condition. Caretakers were asked 10 questions about problematic conditions in the home (e.g., nonfunctioning heating, plumbing, or electrical systems; peeling paint; broken windows or doors). The experimental and control groups did not differ on the average proportions of the presence of such conditions at the time of the post-treatment interview. At the time of the followup interview, the average proportion of problematic conditions present was greater for the experimental group than for the control group (.21 vs. .06; p = .05). Repeated measures analysis revealed no significant changes over time and no significant differences between the two groups averaged over time.
On only one of the specific items regarding problematic conditions in the home were there any differences in the primary analysis of the post-treatment interview. Twenty-three percent of caretakers in the experimental group and 13 percent of the caretakers in the control group reported that "there were not enough basic necessities such as chairs, tables, beds, cribs, mattresses, or not enough basic necessities such as blankets, sheets, pots or dishes" (p = .03). (85)At the time of the followup interview, primary analysis revealed significant differences between the experimental and control group on one of the 10 specific household condition items. Four percent of the experimental group and none of the control group caretakers reported that the electricity did not work for more than a day at a time since the post-treatment interview (Fisher's exact p-value = .05). (86)
Child Care Practices. In both the post-treatment and followup interviews, caretakers were asked a series of yes-no questions about child care practices in the last three months (both positive and negative). The results from these questions are shown in Tables 4-23 and 4-24.
| Control | Experimental | p | |||
|---|---|---|---|---|---|
| N | % | N | % | ||
| Lost temper when child got on nerves | 112 | 53 | 148 | 52 | |
| Found that hitting child was good | 112 | 4 | 148 | 6 | |
| Hitting child harder that meant to | 112 | 5 | 148 | 11 | |
| Out of control when punishing child | 111 | 18 | 147 | 24 | |
| Have you praised your children | 112 | 95 | 147 | 98 | |
| Listened to music together w/child | 112 | 95 | 148 | 92 | |
| Tied child with cord- string-belt | 112 | 0 | 147 | 1 | |
| Gone to amusement park, pool, picnic | 111 | 85 | 147 | 77 | |
| Uncomfortable hugging child | 105 | 10 | 138 | 13 | |
| Encouraged child to read book | 108 | 98 | 142 | 99 | |
| Have children handled household chores | 105 | 76 | 140 | 71 | |
| Not let children into the house | 105 | 1 | 140 | 1 | |
| Punished for not finishing food | 107 | 6 | 140 | 3 | |
| Blamed child w/ things not their fault | 107 | 21 | 139 | 22 | |
| Let child to play where not allowed | 107 | 15 | 139 | 12 | |
| Unable to find someone to watch children | 111 | 42 | 144 | 46 | |
| Control | Experimental | p | |||
|---|---|---|---|---|---|
| N | % | N | % | ||
| Lost temper when child got on nerves | 89 | 52 | 132 | 44 | |
| Found that hitting child was good | 89 | 6 | 132 | 9 | |
| Hitting child harder that meant to | 89 | 4 | 132 | 7 | |
| Out of control when punishing child | 89 | 17 | 132 | 26 | |
| Have you praised your children | 89 | 99 | 132 | 96 | |
| Listened to music together w/child | 89 | 93 | 132 | 92 | |
| Tied child with cord- string-belt | 89 | 0 | 130 | 0 | |
| Gone to amusement park, pool, picnic | 90 | 79 | 133 | 73 | |
| Uncomfortable hugging child | 87 | 10 | 131 | 11 | |
| Encouraged child to read book | 87 | 99 | 129 | 98 | |
| Have children handled household chores | 85 | 84 | 128 | 80 | |
| Not let children into the house | 84 | 1 | 126 | 2 | |
| Punished for not finishing food | 86 | 1 | 130 | 7 | .05 (FE) |
| Blamed child w/ things not their fault | 86 | 21 | 130 | 24 | |
| Let child play where not allowed | 86 | 19 | 129 | 16 | |
| Unable to find someone to watch children | 88 | 47 | 133 | 33 | .04 |
| NOTE: "FE" indicates significance determined by Fisher's exact test | |||||
Three scales were formed using items that appear in Tables 4-23 and 4-24: positive child care practices (5 items), negative child care practices (10 items), and punishment (5 items, all of which were also in the negative child care practices scale).
At the time of the post-treatment interview, primary analyses revealed no significant differences between experimental and control groups on any of the items. (87) At followup, a significantly greater proportion of experimental group respondents responded affirmatively that they "punished [child] for not finishing food" (7% vs. 1%; Fisher's exact p-value = .05). (88)
There were no significant differences between the experimental and control groups with regard to the positive and negative child care practice scales at the time of either the post-treatment or the followup interview. At each point in time, caretakers from both groups responded affirmatively to over 80 percent of the positive items and less than 15 percent of the negative items. Repeated measures analyses revealed no significant changes over time in the positive child care practices scale. There was a small but significant decrease in the proportion of negative child care practices and in the proportion of affirmative answers to the punishment items for both groups averaged over time (see Table 4-20 and Figure 4-3). For all scales, there were no significant interactions between group and time variables, indicating that the pattern of means over time was similar for both the experimental and control groups.
Caretaker Depression. In all three interviews, we administered the SCL-90 depression scale to measure the level of depression of the caretaker. (89) There were no differences between the groups in scores on this scale at the time of the post-treatment or followup interview. Results of the repeated measures analysis indicate significant decreases over time in the depression scores for both groups averaged together (see Table 4-20 and Figure 4-3). The pattern of declining depression scores was the same for both the experimental and control group. (90)
Child Behavior. We asked 35 questions about specific child behaviors, both positive and negative. Questions were phrased in terms of "any of the children" and some questions were age specific. Responses to these questions were used to form various scales: aggression (3 items), school problems (5 items), positive child behaviors (10 items), and negative child behaviors (21 items, including the aggression and school problems items). Neither the primary nor the secondary analyses revealed any significant differences between the groups in scores on any of these scales at the time of the post-treatment or followup interviews (see Table 4-20 and Figure 4-3). Furthermore, none of the hypotheses tested in the repeated measures analysis resulted in significant effects for any of the levels of analyses (primary, secondary, or tertiary).
Figure 4-3
Child and Family Functioning Over Time (Families)
Specific items on whether the child was withdrawn, or had engaged in substance abuse also did not reveal significant differences between groups at either point in time (post-treatment or followup) or in any of the analysis (primary, secondary, or tertiary). A scale measuring two items asking whether any of the children had stolen things or been arrested did result in significant differences in the primary analysis of the followup interview and of the effects over time. This scale was calculated by summing the "yes" responses to the two items, resulting in scale scores ranging from 0 to 2. At the time of the followup interview, the scale score was significantly higher for the experimental group than for the control group (.26 vs. .13; p = .02). Repeated measures analysis indicated that the pattern of scores over time was significantly different for the experimental and control groups, particularly in the time period between the post-treatment interview and the followup interview. In the control group, the average score for caretakers responding that their child had stolen things or been arrested consistently declined over time, but in the experimental group, the average score declined between the initial and post-treatment interviews and returned to the original level in the followup interview (see Table 4-20). For further interpretation of these results, scale scores of 1 and 2 were collapsed and Chi-square analyses were used to examine the proportion of caretakers from each group responding affirmatively to either item at each point in time.
At the time of the initial interview, 22 percent of caretakers from both the control and experimental groups reported that their child had stolen things and/or been arrested in the last three months. At the time of the post-treatment interview, 17 percent of caretakers from both the control and the experimental groups reported that their child had stolen things and/or been arrested since the time of the initial interview. At the time of the followup interview, a significantly greater proportion of caretakers from the experimental group reported that their child had stolen things and/or been arrested since the time of the post-treatment interview (24% vs. 13%; p = .04).
Overall Assessment of Improvement by Caretakers. In both the post-treatment and followup interviews, caretakers were asked about general changes in their family lives since entering the study (see Tables 4-25 and 4-26). At the time of the post-treatment interview, 27 percent of experimental group caretakers generally thought there was "great improvement" in
| Control % | Experimental % | |
|---|---|---|
| p = .07 | ||
| Great improvement | 17 | 27 |
| Some improvement | 46 | 51 |
| Same | 27 | 17 |
| Somewhat or a great deal worse | 6 | 5 |
| Not ascertained | 4 | 1 |
| Control % | Experimental % | |
|---|---|---|
| p = n.s. | ||
| Great improvement | 38 | 36 |
| Some improvement | 38 | 40 |
| Same | 17 | 21 |
| Somewhat or a great deal worse | 7 | 4 |
| Not ascertained | 1 | 0 |
their lives, compared to 17 percent of control group caretakers ( p = .07). (91) When response categories were collapsed to reflect "some or great improvement," things are "just the same," or "somewhat or a great deal worse," a significantly greater proportion of experimental group caretakers reported "some or great improvement" (77% vs. 63%; p = .05). (92)
At the time of the followup interview, slightly more than a third of respondents reported "great improvement" and three quarters of respondents reported "some or great improvement," with no significant differences between the experimental and control groups in any of the levels of analysis (primary, secondary, or tertiary).
4.5.4 Information from Caseworkers on Functioning
In interpreting caseworker reports, it should be noted that while both SCOH and family preservation services were provided by private agency workers, it is likely that there are differences between these groups of caseworkers in the knowledge they have of the cases. It is expected that family preservation workers had much more intensive involvement with the families they worked with. Hence, interpretations of comparisons between responses of workers serving each of the groups must be made with caution.
Given the low response rate on caseworker initial interviews (35 percent for the control group and 54 percent for the experimental group), analysis of initial caseworker interview items was not conducted. This lack of data from the caseworker initial interview makes it difficult to interpret differences between the experimental and control groups at the time of the post-treatment interview. It is unknown whether those differences existed at the time of random assignment or whether they reflect differential changes during the treatment period.
Caretaker Functioning. Caseworkers were asked nine questions tapping various aspects of caretaker functioning on a five-point scale from 0 for not adequate to 4 for very adequate. Table 4-27 provides a list of these nine questions. At the time of the post-treatment interview, there were no significant differences between experimental and control group caseworkers on any of these nine items or on the scale averaging the nine items. (93)
| Control | Experimental | p | |||
|---|---|---|---|---|---|
| N | Mean | N | Mean | ||
| Caretaker ability to provide food | 89 | 2.90 | 145 | 3.05 | |
| Caretaker ability giving affection | 92 | 2.95 | 145 | 3.05 | |
| Caretaker respect for child's opinions | 83 | 2.59 | 131 | 2.80 | |
| Respond patiently to child's questions | 86 | 2.50 | 138 | 2.64 | |
| Respond to child's emotional needs | 89 | 2.56 | 144 | 2.69 | |
| Provide learning opportunities | 90 | 2.30 | 143 | 2.57 | .07 |
| Setting firm/consistent limits/rules | 89 | 2.19 | 140 | 2.45 | .10 |
| Adequate supervisor/responsible childcare | 92 | 2.54 | 148 | 2.80 | .10 |
| Attending to children's health needs | 89 | 2.90 | 144 | 3.03 | |
| Caretaker functioning, 9 items, average of nonmissing items, higher=better | 88 | 2.63 | 139 | 2.79 | |
| Note: Scale for individual items: 0-4, where 0 = not adequate, 4 = very adequate | |||||
Household Condition. As in the caretaker interview, we asked caseworkers about conditions in the home. Caseworkers were asked 13 yes-no questions, some positive and some negative. These items were combined in a scale which indicated the proportion of household condition problems. At the time of the post-treatment interview, experimental group families had, on average, significantly fewer problems with household condition than did control group families (.13 vs. .16; p = .05). (94)
Caretaker Problems. Caseworkers were asked a number of questions about problems experienced by children, caretakers, or other adult household members (question 19 on the first caseworker interview, question 17 on the second caseworker interview). Twenty-one of these problems concerned the caretakers. At the time of the post-treatment interview, caseworkers reported that the caretakers had, on average, 26 percent of the 21 problems. There were no significant differences between the experimental and control groups. (95)
Child Problems. Twelve of the items on the caseworker problem inventory concerned the children. At the time of the post-treatment interview, the percentage of child problems for the experimental group was, on average, 19 percent compared to an average of 15 percent for the control group, a nonsignificant difference. (96)
4.5.5 Summary of Outcome Data
Information from the caretaker interviews, the caseworker interviews, and the administrative data were analyzed for indications of differences between the experimental and control groups subsequent to the referral to the family preservation program. Tables 4-28 and 4-29 contain a summary of those outcomes on which we found significant differences between the experimental and control groups in the primary analysis (p < .05). Items in bold are those on which the experimental group had better outcomes, those in italics are those on which the control group had better outcomes.
There were no significant differences between experimental and control groups on family level rates of placement. Subsequent maltreatment was generally not related to experimental group membership either.
| Caretaker Scales: | Control Mean | Experimental Mean | p |
|---|---|---|---|
| Positive life events | .15 | .19 | .05 |
| Caseworker Scales: | |||
| Household Condition (higher = worse condition) | .16 | .13 | .05 |
| NOTE: This table only includes items with a primary p-value less than or equal to .05. Items in bold indicate significant findings in favor of the experimental group. | |||
| Proportion of affirmative answers to yes/no questions | |||
|---|---|---|---|
| Control % | Experimental % | p | |
| Not enough money for food, rent, or clothing | 33 | 49 | .02 |
| Had difficulty buying enough food | 16 | 27 | .05 |
| Had difficulty buying clothes | 21 | 37 | .01 |
| Punished children for not finishing food | 1 | 7 | .05 (FE) |
| NOTE: This table only includes items with a primary p-value less than or equal to .05. Italicized items indicate significant findings in favor of the control group. "FE" indicates significance determined by Fisher's exact test. | |||
| Caretaker Scales: | Control Mean | Experimental Mean | p |
|---|---|---|---|
| Stolen things or arrested | .13 | .26 | .02 |
| Number of 10 household condition problems | .06 | .10 | .05 |
| NOTE: This table only includes items with a primary p-value less than or equal to .05. Italicized items indicate significant findings in favor of the control group. | |||
As shown in Tables 4-28 and 4-29, there were few significant differences between experimental and control groups in analyses of child and family functioning items. It should also be noted that the results have not been adjusted for the multiplicity of significance tests performed. That is, these significant items surfaced out of a large number of items and scales examined. In such a situation it is to be expected that some items will show significant differences simply by chance, so the appearance of a few significant differences should not be taken as an indication of superiority of one group over another. Overall, we are unable to claim consistent evidence of positive effects of the family preservation services in Philadelphia that were examined in this study.
57. There was a small, non-significant difference between the experimental group, of whom 5.8 percent said yes, and the control group, of whom 11.2 percent responded yes (p = .11).
58. Although not significantly different, compared to the control group, the experimental group did have a somewhat higher proportion of respondents who reported having been neglected as a child (27% vs. 19%; p = .15).
59. The state of Philadelphia reports 8 possible outcomes for reports of maltreatment; (1) indicated, perpetrator admitted, (2) indicated, medical evidence, (3) not substantiated, (4) pending determination, (5) substantiated, (6) unfounded, (7) indicated - investigating, and (8) unable to complete.
60. Placement spells are defined as any consecutive period of time in substitute care and may consist of several distinct placements (i.e., several different foster homes).
61. The results of "secondary" analyses, dropping violations of random assignment and cases receiving minimal service show slightly greater differences between the experimental and control groups (see Appendix). Here, the same 8 significant primary analysis items show significant differences in favor of the experimental group at p = .01 or lower, and one additional item showed significant differences in the same direction at p < .05. All nine items that showed significant differences in the secondary analysis remained significantly different in the tertiary analysis (see Appendix).
62. These differences remained significant and in the same direction for both secondary and tertiary analyses (for magnitude and significance levels, see Appendix).
63. Differences remained significant and in the same direction for both secondary and tertiary analyses (for magnitude and significance levels, see Appendix).
64. In the secondary analysis (dropping violations and minimal service cases) and the tertiary analysis (dropping additional cases that may not have had a worker assigned), a significantly greater proportion of experimental group caretakers reported receiving transportation and parent education/training classes (see Appendix for magnitude and significant levels).
65. The difference was, however, significant in the secondary analysis (21% control vs. 14% experimental, p = .001) and the tertiary analysis (22% control vs. 13% experimental, p = .003).
66. In addition to slight changes in the magnitude of the difference in whether workers helped caretakers see their good qualities, secondary analysis revealed that a significantly greater proportion of experimental group caretakers reported that they and their worker agreed on goals most of the time (75% vs. 70%, p = .03). Tertiary analysis revealed no additional items with significant differences (see Appendix).
67. When violations and minimal services cases were excluded, the difference between the groups was even larger (5.3 vs. 3.1, p = .0001).
68. SSI for adult or child was reportedly provided more often to the control group than the experimental group, and the difference was marginally significant by Fisher's Exact test (3% vs. 0%; p = .06). In the secondary analyses, excluding violations and minimal service cases, 10 services were provided significantly more often to the experimental group than to the control group (again, significance levels were all at p = .05 or less). In addition to the 6 primary analysis items showing differences in favor of the experimental group, secondary analyses indicate that the following services were also provided significantly more often by the experimental group: health assessment, housing financial assistance, self help groups, and homemaker services (see Appendix for magnitude of difference and significance levels). In the secondary analyses no services were provided significantly more often to the control group than the experimental group.
69. None of the items showed significant differences between experimental and control group caretakers in the secondary analysis or the tertiary analysis.
70. Secondary and tertiary analyses did not result in any significant differences either.
71. This difference remained significant in the secondary analysis (31% vs. 16%; p = .02) and was marginally significant in the tertiary analysis (31% vs. 18%; p = .06).
72. Tertiary analyses were not performed on caseworker interview data due to the fact that all of the 29 additional cases dropped for this level of analysis were missing both caseworker interviews and results would therefore be the same as for the secondary analysis.
73. In determining placements, we depended on the variable "factype" in the administrative data. The specific categories for this variable included: adoption, foster care, private institution/boarding schools, family treatment home, unmarried parent, other, children's psychiatric hospital, and foster care medically fragile.
74. Due to the "clustering effect," analyses at the child level are misleading. Clustering refers to the lack of independence between children within the same family of observations of such things as placement. One could argue that if one child is removed from the home, the remaining children are more likely to experience placement. The "clustering effect" leads to an underestimate of the significance levels when analyses are conducted at the child level. Conducting the analyses at the family level is one approach to resolving this dilemma.
75. Fifteen percent of the experimental group, and 16 percent of the control group experienced substitute care placement within a year in the secondary analysis.
76. Analyses were also done on all allegations, whether substantiated or not. The results were very similar, although, of course, rates for all allegations were higher.
77. Figure 4-2 also displays the survival curves for the secondary analysis group. At one year, 22 percent of the experimental group and 13 percent of the control group experienced substantiated reports of maltreatment.
78. This difference remained significant in both the secondary and tertiary analyses (see Appendix for magnitude of difference and significant levels).
79. These results held for the secondary and tertiary analyses.
80. These results were maintained in the secondary and tertiary analyses.
81. This difference was slightly larger and more significant in the secondary analysis (53% experimental and 33% control; p = .006) and the tertiary analysis (53% experimental and 31% control; p = .005). Tertiary analysis also revealed significant differences in the proportion of respondents indicating they "just wanted to give up," with a greater proportion of experimental group respondents answering affirmatively (29% vs. 15%; p = .04).
82. There were no significant differences between experimental and control groups on these items in the secondary or tertiary analyses.
83. The difference was greater and marginally significant in the secondary analysis (.30 vs. .21; p = .06). In the tertiary analysis, the difference was greater still and it was statistically significant (.31 for the experimental group and .19 for the control group; p = .02).
84. Differences on both items remained significant in the secondary and tertiary analyses (see Appendix for magnitude of differences and significance levels).
85. This difference increased and remained significant in the secondary analysis (25% vs. 11%; p = .009) and the tertiary analysis (26% vs. 11%; p = .01). In the tertiary analysis, one additional item resulted in significant differences at the time of the post-treatment interview. Six percent of the experimental group caretakers and none of the control group caretakers reported that "there were bare electric wires" (Fisher's exact p-value = .02).
86. This difference was not significant in either the secondary or the tertiary analysis. However, tertiary analyses revealed that two different items resulted in significant differences between experimental and control group caretakers at the time of the follow-up interview. Nine percent of the experimental group caretakers and none of the control group caretakers reported that the "plumbing did not work" (Fisher's exact p-value = .01). Seventeen percent of the experimental group caretakers and six percent of the control group caretakers reported that "a lot of paint was peeling" (p = .04).
87. In the secondary analysis, "hitting child harder than meant to" was the only item for which there were significant differences between the experimental and control groups, with a greater proportion of the experimental group responding affirmatively (13% vs. 5%, p = .03). The difference was not significant in the tertiary analysis.
88. This difference was not significant in the secondary or tertiary analysis.
89. Reliability analysis yielded a Cronbach's alpha of .92 at initial interview, .90 at post-treatment, and .94 at follow-up.
90. These results held for the secondary and tertiary analyses.
91. This difference was greater and statistically significant in the secondary analysis (31% vs. 18%; p = .02) and the tertiary analysis (31% vs. 16%; p = .02).
92. These results were also maintained in the secondary analysis (80% vs. 63%; p = .02) and tertiary analysis (79% vs. 62%; p = .03).
93. Secondary analyses resulted in significant differences on three of the individual items. Relative to caseworkers in the control group, caseworkers in the experimental group rated caretakers higher (more adequate) on "respecting children's opinions" (2.84 vs. 2.56; p = .05), "setting firm limits for children" (2.50 vs. 2.18; p = .05), and "providing adequate personal supervision" (2.86 vs. 2.51; p = .04).
94. These results were maintained in the secondary analysis (.13 vs. .17; p = .03).
95. This was also true in the secondary analysis.
96. This result held in the secondary analysis.
5. Attrition Analysis: Caretaker Interviews
Longitudinal studies almost always encounter sample attrition. Not all respondents will be interviewed at all points in time. In this study, some caretakers responded to all three interviews, some caretakers responded to the initial interview but not the post-treatment or followup interview, and some not interviewed earlier were interviewed later. Numbers of cases with caretaker interviews at each of the three points in time are shown in Table 5-1. Percentages are shown in Table 5-2.
| Kentucky | New Jersey | Tennessee | Philadelphia | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| C % |
E % |
Total | C % |
E % |
Total | C % |
E % |
Total | C % |
E % |
Total | |
| Time 1 only | 9 | 5 | 14 | 5 | 9 | 14 | 2 | 3 | 5 | 7 | 20 | 27 |
| Time 2 only | 2 | 1 | 3 | 4 | 15 | 19 | 1 | 1 | 4 | 9 | 13 | |
| Time 3 only | 5 | 5 | 10 | 4 | 5 | 9 | 2 | 7 | 9 | 1 | 6 | 7 |
| Time 1 and time 2 | 34 | 28 | 62 | 34 | 46 | 80 | 5 | 14 | 19 | 25 | 22 | 47 |
| Time 1 and time 3 | 4 | 6 | 10 | 7 | 13 | 20 | 2 | 2 | 4 | 5 | 12 | 17 |
| Time 2 and time 3 | 2 | 2 | 4 | 12 | 19 | 31 | 4 | 4 | 8 | 14 | 15 | 29 |
| All three interviews | 108 | 117 | 225 | 84 | 130 | 214 | 28 | 61 | 89 | 70 | 102 | 172 |
| Totals | 164 | 164 | 328 | 150 | 237 | 387 | 43 | 92 | 135 | 126 | 186 | 312 |
| Kentucky | New Jersey | Tennessee | Philadelphia | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| C % |
E % |
Total | C % |
E % |
Total | C % |
E % |
Total | C % |
E % |
Total | |
| Net Study Cases | 175 | 174 | 349 | 167 | 275 | 442 | 49 | 98 | 147 | 144 | 209 | 353 |
| Time 1 only | 5.1 | 2.9 | 4.0 | 3.0 | 3.3 | 3.2 | 4.1 | 3.1 | 3.4 | 4.9 | 9.6 | 7.6 |
| Time 2 only | 1.1 | 0.5 | 0.9 | 2.4 | 5.5 | 4.3 | 1.0 | 0.7 | 2.8 | 4.3 | 3.7 | |
| Time 3 only | 2.8 | 2.9 | 2.9 | 2.4 | 1.8 | 2.0 | 4.1 | 7.1 | 6.1 | 0.7 | 2.9 | 2.0 |
| Time 1 and time 2 | 19.4 | 16.1 | 17.8 | 20.4 | 16.7 | 18.1 | 10.2 | 14.3 | 12.9 | 17.4 | 10.5 | 13.3 |
| Time 1 and time 3 | 2.3 | 3.4 | 2.9 | 4.2 | 4.7 | 4.5 | 4.1 | 2.0 | 2.3 | 3.5 | 5.7 | 4.8 |
| Time 2 and time 3 | 1.1 | 1.1 | 1.1 | 7.2 | 6.9 | 7.0 | 8.2 | 4.1 | 5.4 | 9.7 | 7.2 | 8.2 |
| All three interviews | 61.7 | 67.2 | 64.5 | 50.3 | 47.3 | 48.4 | 57.1 | 62.2 | 60.5 | 48.6 | 48.8 | 48.7 |
Of the cases randomly assigned, the proportion of respondents who completed both the initial and followup interviews ranged from approximately 53 percent to 67 percent. Of primary concern here is that those who did not complete the followup interviews might vary in systematic ways from those who did, thus potentially affecting any analyses of change over time that rely on the interview data. Several steps were taken to examine the sample attrition for differences in those who did not see the study through to the end, particularly in regard to whether there were differences between the experimental and control groups with respect to who completed the followup interview.
Starting with the sample characteristics for those who responded to each of the interviews at each point in time, no significant differences were found in the distributions of the following characteristics: respondent's age, youngest child's age, oldest child's age, number of persons in household, number of adults in household, or number of children in household.
In addition to looking at demographic characteristics, we examined initial interview responses on the family and child functioning scales that were used as primary outcome variables. (97) This was done to detect whether the group that was analyzed in our change analysis was functioning better or worse at the outset compared to the whole sample of cases that were interviewed at the outset. Those who completed the initial interview but did not complete the followup interview were compared to those who completed the initial interview and the followup interview on scale measures at the initial interview. (98) Items on which there were significant differences between those who responded to the initial but not to the followup interview and those who responded to both interviews are reported in Table 5-3.
For those caretaker scale comparisons indicating significant differences between those who remained in the interview sample through the followup period and those who did not, t-tests were conducted to assess differences between experimental and control groups for that particular scale at the time of the initial interview. For example, in Kentucky the cases where respondents did not complete the followup interview were analyzed for differences between experimental and control groups in reports of average child aggression at the time of the initial interview. The
| Completed initial but not followup interview | Completed initial and followup interview | p | |||
|---|---|---|---|---|---|
| N | Mean | N | Mean | ||
| Kentucky | |||||
| Child aggression | 76 | 1.18 | 235 | 1.53 | .005 |
| Positive child behaviors | 69 | .75 | 228 | .70 | .02 |
| New Jersey | |||||
| Negative life events | 93 | .08 | 234 | .12 | .007 |
| Stolen things or arrested | 94 | .41 | 234 | .58 | .02 |
| Tennessee | |||||
| Child problems | 24 | 1.45 | 93 | 2.32 | .01 |
| Philadelphia | |||||
| Caretaker depression | 74 | 1.25 | 189 | .97 | .03 |
| Note: Means represent average scores on the scales at the time of the initial interview (i.e. at the outset of the study). | |||||
results were not significant, thus, while those who did not complete the followup interview appear to have reported a lower proportion of child aggression problems at the outset than those who did complete the followup interview, this did not occur differentially for experimental and control groups. None of the t-test comparisons for the items listed in Table 5-3 revealed any significant differences in the average initial scores for the experimental and control groups.
In summary, of the 68 comparisons (4 states, 17 for each state) on initial levels for the family and child functioning scales of the group completing both initial and followup interviews with those completing initial interviews but not followup, only five showed significant differences in means at the initial interview. All five of these measures indicate lower initial functioning of those who were interviewed at both points in time, but none of the comparisons of experimental and control groups on initial levels of these measures were significant.
In conclusion, there is no substantial evidence that attrition resulted in an analytic sample that is unrepresentative of the initial interview sample.
Endnotes
97. There were 17 scales in all: positive life events, negative life events, life events depression, economic functioning, punishment, child aggression, school problems, child withdrawn, stolen things or arrested, child substance abuse, child problems, negative child behaviors, positive child behaviors, household condition, depression, positive child care practices, and negative child care practices.
98. Respondents were categorized regardless of completion of post-treatment interview. Furthermore, those who did not complete an initial interview were excluded all together as information about functioning at the outset of the study was unavailable. The proportion of net study cases without initial interview data and thus excluded from these analyses are as follows: 5 percent in Kentucky, 13 percent in New Jersey, 12 percent in Tennessee, and 14 percent in Philadelphia.
6. Social Support
Because the support that families receive from relatives and friends is widely thought to contribute to family and child well-being, at each of the three interviews, we asked caretakers about the kinds of informal supports that were available to them. Caretakers were asked how frequently they had contact with their mothers and fathers, adult brothers and sisters, and up to four friends. For each person that the caretaker reported having contact with at least once a year, they were asked whether the person could be relied on for each of three kinds of support -- emotional (talking over problems), instrumental (help with money or housework), and informational (advice on how to handle problems). In addition, caretakers who were residing with partners were asked about the support that they received from their partners.
In the following analyses, we examine several aspects of the caretakers' informal support systems. We first examine the extent to which caretakers had family and friends available who might provide support, and the proportion of partners, siblings, parents, and friends that the caretakers could rely on for support. Second, the levels of emotional, informational, and instrumental support available from each group of relatives and friends is assessed. Finally, because increasing the level of informal social support is sometimes thought to be a useful outcome of family preservation services, we examine whether there was change in the levels of support that were available to the caretakers.
6.1 Composition of Caretakers' Support Networks
A concern raised by early investigations into the informal support systems of maltreating families was the extent to which they are isolated from relatives and friends. Families may be socially isolated if they don't have relatives and friends, or don't have much contact with them. Furthermore, families may have relatives and friends with whom they have regular contact but not rely on them for support. Hence, in order to assess the extensiveness of caretakers' informal supports, we first asked them whether they had living parents, siblings, partners, and friends, then determined how often they had contact with each of them, and finally asked if each could be relied on for support.
Table 6-1 summarizes the proportion of caretakers in each states' control and experimental groups who had partners, siblings, parents, and friends from whom they might receive emotional, instrumental, and informational support. The percentages of caretakers reporting that they had contact with particular relatives and friends at least once a year are
| New Jersey | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Relationship | Caretakers with relative/friend | Support Available | ||||||||
| Emotional | Instrumental | Informational | Any type of support | |||||||
| C N=131 N(%) | E N=198 N(%) | C % | E % | C % | E % | C % | E % | C % | E % | |
| Partner | 56 (43) | 69 (35) | 79 | 73 | 100 | 97 | 88 | 74 | 100 | 97 |
| Sisters | 103 (79) | 162 (82) | 59 | 60 | 38 | 41 | 58 | 61 | 63 | 65 |
| Brothers | 88 (67) | 146 (74) | 43 | 47 | 33 | 35 | 42 | 45 | 48 | 49 |
| Mother | 93 (71) | 116 (59) | 66 | 54 | 44 | 45 | 52 | 48 | 68 | 62 |
| Father | 61 (47) | 89 (45) | 49 | 40 | 34 | 42 | 44 | 39 | 51 | 53 |
| Friends | 96 (73) | 161 (81) | 95 | 95 | 69 | 70 | 92 | 94 | 96 | 98 |
| Overall | 131 (100) | 197 (99) | 92 | 93 | 85 | 87 | 92 | 93 | 93 | 95 |
| C = Control E = Experimental | ||||||||||
| Kentucky | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Relationship | Caretakers with relative/friend | Support Available | ||||||||
| Emotional | Instrumental | Informational | Any type of support | |||||||
| C N=155 N(%) | E N=156 N(%) | C % | E % | C % | E % | C % | E % | C % | E % | |
| Partner | 56 (36) | 52 (33) | 80 | 92 | 100 | 94 | 75 | 92 | 100 | 98 |
| Sisters | 117 (76) | 100 (64) | 73 | 58 | 49 | 42 | 73 | 51 | 77 | 63 |
| Brothers | 112 (72) | 110 (71) | 50 | 47 | 36 | 39 | 45 | 45 | 56 | 51 |
| Mother | 132 (85) | 124 (80) | 61 | 58 | 52 | 49 | 61 | 50 | 68 | 67 |
| Father | 92 (59) | 97 (62) | 40 | 42 | 43 | 42 | 40 | 38 | 48 | 49 |
| Friends | 144 (93) | 142 (91) | 99 | 97 | 84 | 73 | 97 | 93 | 99 | 98 |
| Overall | 155 (100) | 155 (99) | 98 | 97 | 97 | 87 | 96 | 96 | 99 | 98 |
| C = Control E = Experimental | ||||||||||
| Tennessee | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
Relationship | Caretakers with relative/friend | Support Available | ||||||||
| Emotional | Instrumental | Informational | Any type of support | |||||||
| C N=37 N(%) | E N=80 N(%) | C % | E % | C % | E % | C % | E % | C % | E % | |
| Partner | 7 (19) | 29 (36) | 86 | 90 | 100 | 90 | 71 | 86 | 100 | 93 |
| Sisters | 28 (76) | 58 (73) | 71 | 81 | 71 | 78 | 64 | 74 | 71 | 84 |
| Brothers | 30 (81) | 57 (71) | 60 | 61 | 67 | 58 | 60 | 60 | 70 | 65 |
| Mother | 25 (68) | 58 (73) | 72 | 62 | 76 | 67 | 64 | 60 | 88 | 79 |
| Father | 23 (62) | 39 (49) | 35 | 54 | 39 | 49 | 35 | 54 | 39 | 59 |
| Friends | 33 (89) | 70 (88) | 94 | 96 | 79 | 94 | 91 | 94 | 94 | 97 |
| Overall | 37 (100) | 79 (99) | 95 | 100 | 92 | 98 | 89 | 96 | 95 | 100 |
| C = Control E = Experimental | ||||||||||
| Pennsylvania | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Relationship | Caretakers with relative/friend | Support Available | ||||||||
| Emotional | Instrumental | Informational | Any type of support | |||||||
| C N=107 N% | E N=156 N% | C % | E % | C % | E % | C % | E % | C % | E % | |
| Partner | 25 (23) | 34 (22) | 88 | 68 | 96 | 94 | 76 | 71 | 100 | 97 |
| Sisters | 79 (74) | 117 (75) | 71 | 71 | 62 | 60 | 66 | 64 | 75 | 73 |
| Brothers | 82 (77) | 118 (76) | 58 | 49 | 49 | 45 | 57 | 48 | 63 | 58 |
| Mother | 87 (81) | 120 (77) | 68 | 63 | 60 | 65 | 63 | 63 | 74 | 75 |
| Father | 66 (62) | 87 (56) | 42 | 42 | 39 | 40 | 42 | 44 | 53 | 54 |
| Friends | 84 (79) | 127 (81) | 100 | 97 | 88 | 95 | 95 | 98 | 100 | 98 |
| Overall | 107 (100) | 156 (100) | 94 | 97 | 94 | 91 | 93 | 95 | 97 | 97 |
| C = Control E = Experimental | ||||||||||
reported under the column heading "Caretakers with relative/friend." The remaining columns report the proportions of caretakers who had minimal contact (at least once a year) with specific relatives and friends and could rely on them for support.
At least 99 percent of all caretakers in every state reported having minimal contact with at least one relative or friend, and a large majority believed that they could count on at least one person for some type of support. In Kentucky and Pennsylvania, only one to three percent of caretakers in either the control or experimental groups felt that they had no one to count on for any kind of support. The proportion of caretakers with no support from anyone was slightly larger in New Jersey's control (7%) and experimental (5%) groups, and in Tennessee's control group (5%).
Although their numbers are small (over all states, only 36 caretakers reported that no support was available to them), caretakers who report that they have no support may be of particular interest since they could be easily identified and targeted for services linking them to informal and community supports. In addition, caretakers without any informal support may benefit the most from efforts to establish linkages to support. Importantly, although they did not rely on them for support, this group of caretakers reported having, on average, nine relatives and friends. In addition, 33 percent of these caretakers were employed. The presence of relatives, friends, and coworkers in the caretakers' social networks may improve the prospects of successfully increasing the levels of informal support that are available to these families. (99)
Caretakers across all states had, on average, 9.4 (s.d. 2.5) friends and relatives on whom they might call for various kinds of assistance. Within the set of family members and friends that they were asked about, caretakers most often cited their friends as people they can go to for help. Overall, about three-quarters of the caretakers reported having mothers and siblings and had contact with them at least once a year. Across all support areas, caretakers perceived mothers and sisters as support providers more often than brothers. Still, half or more of the caretakers with brothers said that they could turn to them for support. Fewer caretakers (less than 63 percent across all states) reported having a father. Furthermore, fathers were less likely to be relied on for support than were other relatives or friends.
Even fewer caretakers reported living with a partner. For instance, more New Jersey caretakers reported co-residing partners than caretakers in any other state, and only 39 percent of them resided with a partner. But relative to fathers and brothers, as well as sisters and mothers, for the minority of caretakers who live with them, partners play a much larger support role, especially in the provision of instrumental support. Friends, however, play a more important role than partners for emotional and informational support.
Only 54 percent of the caretakers of all races in this study reported having fathers with whom they have regular contact and even fewer were residing with partners -- 32 percent of the caretakers of all races reported partners, but 46 percent of white caretakers, 45 percent of Hispanics, and only 22 percent of African Americans reported living with partners. These findings are consistent with what is known about recent changes in family formation among low income populations, especially African-Americans. In a 30-year longitudinal study following a cohort of teen parents and their children, Furstenberg (2001) observed a generational decline in the propensity to marry. In the 1960s the great majority of adolescent mothers married - usually the child's father. More than half of the older generation married by their early twenties, and by their mid-forties, three-fourths had wed. However, observing the next generation of teen mothers, Furstenberg noted that only 14 percent of the younger generation had married by their early twenties, and only 4 percent of those who were not mothers had wed.
Other observers of marriage and childbearing trends have also noted the steady increase in the formation of single-parent (usually female-headed) households, especially among low-income African-American women, over the last several decades (Garfinkel and McLanahan, 1986; Cherlin, 1992). However, the reason for this shift in household structure is not clearly understood. Wilson (1987) has argued that the trend has coincided with other structural shifts such as the decline in the proportion of African-American men who have access to steady work and the simultaneous rise in the incarceration and mortality rates of those men. Others have suggested that cultural changes in attitudes toward the institution of marriage have contributed to the formation of the single-parent family for all Americans and that this societal-wide change has been exacerbated by economic restructuring that hit African-American communities particularly hard (Cherlin, 1992).
Whether the decline in marriage is attributable to structural shifts in the economy, or cultural shifts in attitudes toward marriage, Stack's (1974) research on family support systems in impoverished communities found that African-Americans rely more on extended family members for support of all kinds rather than depending on marriage as the primary source of support (Cherlin, 1992). Certainly, with regard to the composition of their support networks, families in this study fit this characterization.
6.2 Caretakers' Levels of Support
In order to get a rough estimate of the levels of support that might be available to caretakers, we created composite scores for each of the three kinds of support. These composite scores are the products of whether each instance of emotional, instrumental, or informational support available from each supporter (scored 0 - 1) and the frequency of contact with the supporter (scored 1 - 4), summed across supporters. (100) A total support score was computed by simply summing the three (emotional, instrumental, and informational) composite scores for each supporter. Using this scoring scheme, the maximum level of total support available from any one supporter is twelve. For instance, a friend who gave the maximum amount of total support would provide all three types of support and have daily contact with the caretakers. For any single type of support, the maximum level of support available from any one supporter is four.
Table 6-2 summarizes the levels of support that was available to caretakers at the initial interview in several ways. The upper portion of the state tables describe the average total, emotional, instrumental, and informational supports that caretakers reported were available to them from all members of their support networks. The lower portion of the tables summarize the levels of support that were available to caretakers from each of six groups of supporters -- partners, mothers, fathers, sisters, brothers, and friends. The levels of support that were available from family and friends are presented in two ways. The first set of columns provides the average support that was available to caretakers from each group of supporters. The average mean support, presented in the next to the last column, takes into account that caretakers could report several brothers, sisters, and friends, but only one mother, father, and partner. Whereas average support summarizes the contribution to total support from each supporter group, average mean support is the total support available averaged across members of a supporter group. Again, the maximum amount of support available from any one supporter is twelve.
Looking first at the upper portion of the table, we note that the average levels of support across the three types are very similar. In fact, the three types of supports are highly correlated -- emotional-informational, r = .93, emotional-instrumental, r = .80, and informational-instrumental, r = .80 -- so that if a caretaker had available one type of support, he or she usually had the other types available as well. However, the average levels of emotional and informational supports were somewhat greater than was the availability of instrumental support from all supporters, suggesting that members of the caretakers' support networks may have been better able to assist in ways that did not require their labor or strain their material resources.
| New Jersey | ||||||
|---|---|---|---|---|---|---|
| Support at Initial Interview | Average Support | Average Mean Support | ||||
| N | M | p | M | p | ||
| Total | C | 131 | 34.9 | |||
| E | 197 | 34.3 | ||||
| Emotional | C | 131 | 13.0 | |||
| E | 197 | 12.7 | ||||
| Instrumental | C | 131 | 9.4 | |||
| E | 197 | 9.1 | ||||
| Informational | C | 131 | 12.5 | |||
| E | 197 | 12.4 | ||||
| Partner | C | 56 | 9.9 | 9.9 | ||
| E | 69 | 9.0 | 9.0 | |||
| Mother | C | 93 | 5.2 | 5.2 | ||
| E | 116 | 5.0 | 5.0 | |||
| Father | C | 61 | 4.1 | 4.1 | ||
| E | 89 | 3.3 | 3.3 | |||
| Sisters | C | 103 | 8.4 | 3.5 | ||
| E | 161 | 7.9 | 3.7 | |||
| Brothers | C | 87 | 6.3 | 2.3 | ||
| E | 145 | 5.7 | 2.5 | |||
| Friends | C | 96 | 19.5 | 7.9 | ||
| E | 160 | 19.8 | 7.6 | |||
| Kentucky | ||||||
|---|---|---|---|---|---|---|
| Support at Initial Interview | Average Support | Average Mean Support | ||||
| N | M | p | M | p | ||
| Total | C | 155 | 41.1 | |||
| E | 155 | 36.3 | .05 | |||
| Emotional | C | 155 | 15.1 | |||
| E | 155 | 13.6 | ||||
| Instrumental | C | 155 | 11.5 | |||
| E | 155 | 9.9 | .05 | |||
| Informational | C | 155 | 14.5 | |||
| E | 155 | 12.8 | .04 | |||
| Partner | C | 56 | 9.5 | 9.5 | ||
| E | 51 | 10.8 | .02 | 10.8 | .02 | |
| Mother | C | 132 | 5.9 | 5.9 | ||
| E | 124 | 5.2 | 5.2 | |||
| Father | C | 92 | 3.8 | 3.8 | ||
| E | 97 | 3.6 | 3.6 | |||
| Sisters | C | 117 | 8.9 | 4.5 | ||
| E | 100 | 6.8 | 3.5 | |||
| Brothers | C | 112 | 5.6 | 3.5 | ||
| E | 109 | 4.9 | 3.6 | |||
| Friends | C | 144 | 21.1 | 8.0 | ||
| E | 142 | 20.2 | 7.8 | |||
| Tennessee | ||||||
|---|---|---|---|---|---|---|
| Support at Initial Interview | Average Support | Average Mean Support | ||||
| N | M | p | M | p | ||
| Total | C | 37 | 39.5 | |||
| E | 79 | 44.2 | ||||
| Emotional | C | 37 | 13.8 | |||
| E | 79 | 15.3 | ||||
| Instrumental | C | 37 | 13.1 | |||
| E | 79 | 14.3 | ||||
| Informational | C | 37 | 12.7 | |||
| E | 79 | 14.6 | ||||
| Partner | C | 7 | 9.7 | 9.7 | ||
| E | 28 | 10.7 | 10.7 | |||
| Mother | C | 24 | 8.4 | 8.4 | ||
| E | 58 | 6.5 | 6.5 | |||
| Father | C | 23 | 3.8 | 3.8 | ||
| E | 39 | 4.2 | 4.2 | |||
| Sisters | C | 28 | 10.8 | 4.8 | ||
| E | 58 | 13.4 | 5.8 | |||
| Brothers | C | 30 | 9.3 | 4.3 | ||
| E | 57 | 7.7 | 3.7 | |||
| Friends | C | 32 | 16.4 | 8.5 | ||
| E | 70 | 20.5 | 9.1 | |||
| Pennsylvania | ||||||
|---|---|---|---|---|---|---|
| Support at Initial Interview | Average Support | Average Mean Support | ||||
| N | M | p | M | p | ||
| Total | C | 107 | 40.3 | |||
| E | 156 | 35.3 | ||||
| Emotional | C | 107 | 14.5 | |||
| E | 156 | 12.3 | .05 | |||
| Instrumental | C | 107 | 12.5 | |||
| E | 156 | 11.4 | ||||
| Informational | C | 107 | 13.3 | |||
| E | 156 | 11.7 | ||||
| Partner | C | 25 | 9.8 | 9.8 | ||
| E | 34 | 8.7 | 8.7 | |||
| Mother | C | 87 | 5.0 | 5.0 | ||
| E | 120 | 5.0 | 5.0 | |||
| Father | C | 66 | 3.9 | 3.9 | ||
| E | 87 | 3.5 | 3.5 | |||
| Sisters | C | 79 | 10.0 | 4.8 | ||
| E | 117 | 7.4 | 4.9 | |||
| Brothers | C | 82 | 8.2 | 3.9 | ||
| E | 117 | 5.1 | .02 | 3.1 | ||
| Friends | C | 84 | 20.9 | 8.8 | ||
| E | 127 | 20.7 | 127 | 8.8 | ||
The smaller amount of financial and instrumental support that is available from members of low-income individuals' support networks has been documented in other research. For instance, in a study of the supports that were available to former General Assistance recipients, Henly (1994) found that emotional support was provided most often, followed by informational, instrumental, and lastly financial support. Given that, in these studies, the recipients of support had very limited incomes, the relative positions of the various forms of support in the support hierarchy most likely reflect network members' capacity to provide the different kinds of assistance rather than the recipients' particular set of needs.
Examining the average contributions of supporter groups to total support, friends, partners, and sisters were the largest contributors to caretakers' overall support. As groups, brothers, mothers, and fathers contribute somewhat less to total support, but the lower levels of support that is contributed by mothers and fathers is partly attributable to the smaller numbers of supporters in these groups. When support is averaged across members of supporter groups, the positions of mothers and sisters in the supporter hierarchy shifts. Considering the support that was available from individual members of a supporter group, on average, partners, friends, and mothers were perceived to contribute higher levels of support than siblings and fathers.
Comparing the control and experimental groups, at the initial interview there were no differences in the levels of supports between the groups in either New Jersey or Tennessee. But in Pennsylvania, the control group had significantly more emotional support available than the experimental group (p = .05), and more overall brother support (p = .02). In Kentucky, control group members had significantly more instrumental (p = .05), informational (p = .04), and total support (p =.05) available, but the experimental group reported the availability of more support from partners (p = .02). The general similarity in support across the control and experimental groups at the initial interview was expected since randomization should assure that the groups are not different prior to receiving services.
6.3 Effects of Family Preservation on Levels of Support
It is sometimes believed that family preservation programs may strengthen families' informal supports. To examine whether this occurred in the programs we studied, we used multivariate repeated measures analysis to assess the change in the levels of support that caretakers reported were available to them at each of the three interviews (at the beginning of service, four to six weeks after service began, and one year after the beginning of service). For each state, we examined change in the levels of total, emotional, instrumental, and informational support available to caretakers, and change in the levels of support available from family and friends. Table 6-3 summarizes differences between the experimental and control groups at the second and third interviews as well as change between interviews. In the repeated measures analysis, three main null hypotheses are tested. First, that support levels for the experimental group, averaged over the three points in time are equal to those of the control group. Second, that the averages of the groups at each point in time are the same. Third, that there is no interaction between time and group.
Of the three hypotheses, the last is central. A significant interaction between time and group indicates that support in the experimental and control groups changed in different ways. The levels of support that were available to caretakers could either increase or decrease over time in one or both groups, or increase in one and decline in the other group. Generally, we are interested in support increasing over time since more support is presumed to have positive effects on caretaker functioning and family well-being. Furthermore, larger increases in support in the experimental group would suggest that family preservation was helpful in this particular area of service.
By and large, there is not much evidence in any of the states that enhancing the availability of caretakers' informal supports was a strong effect of family preservation efforts. In Tennessee, the average levels of support that were available to caretakers did not change over time. The only significant difference between the control and experimental groups was the change in the level of support that was available from partners. Of caretakers who resided with partners, those in the control group reported that more support was available from their partners at followup than at post-treatment, whereas support from partners in the experimental group had actually decreased slightly over the same period (p = .02).
For New Jersey caretakers, there were no significant differences between the control and experimental groups in the overall average levels of support or in changes in average support from relatives and friends or across any type of support over time.
Relative to the those in the experimental group, caretakers in the Kentucky control group reported having more total, instrumental, and informational support available, and more support available from sisters. (101) However, the level of informational support available over time decreased in the control group and increased in the experimental group (p = .08) so that at the followup interview the level of informational support available to caretakers in the two groups
| Tennessee | ||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Post-treatment | Followup | Multivariate Repeated Measures | ||||||||||||||||
| N | Ma | pb | N | M | p | N | Means | Multivariate ps | Univariate ps-Time | Univariate ps-Grp-time interaction | ||||||||
| Initial | Post | Follow | Grpc | Timed | Time -Grpe | Initial v. laterf | Post v. Followg | Initial v. Later | Post v. Follow | |||||||||
| Total | C | 39 | 42.9 | 36 | 40.6 | 28 | 44.1 | 47.9 | 42.2 | .93 | .38 | .68 | ||||||
| E | 84 | 41.5 | 74 | 43.4 | 61 | 44.4 | 45.1 | 43.6 | ||||||||||
| Emotional | C | 39 | 14.4 | 36 | 13.7 | 28 | 15.5 | 16.2 | 14.5 | .92 | .41 | .89 | ||||||
| E | 84 | 14.2 | 74 | 14.7 | 61 | 15.3 | 15.6 | 14.8 | ||||||||||
| Instrumental | C | 39 | 14.5 | 36 | 14.1 | 28 | 14.4 | 16.4 | 14.4 | .59 | .23 | .47 | ||||||
| E | 84 | 13.6 | 74 | 14.0 | 61 | 14.2 | 14.5 | 14.1 | ||||||||||
| Informational | C | 39 | 14.0 | 36 | 12.9 | 28 | 14.2 | 15.3 | 13.3 | .70 | .56 | .70 | ||||||
| E | 84 | 13.7 | 74 | 14.7 | 61 | 14.8 | 15.0 | 14.8 | ||||||||||
| Partner | C | 6 | 11.0 | 13 | 11.8 | .005 | 4 | 11.5 | 10.5 | 12.0 | .64 | .84 | .06 | .02 | ||||
| E | 28 | 10.9 | 26 | 9.9 | 15 | 10.7 | 11.2 | 10.1 | ||||||||||
| Mother | C | 25 | 8.8 | 24 | 9.2 | 17 | 9.1 | 9.5 | 9.9 | .08 | .32 | 1.0 | ||||||
| E | 60 | 7.8 | 50 | 8.2 | 44 | 7.3 | 7.7 | 8.1 | ||||||||||
| Father | C | 23 | 4.0 | 20 | 3.3 | 16 | 3.6 | 4.1 | 3.3 | .49 | .40 | .94 | ||||||
| E | 44 | 4.2 | 38 | 4.5 | 30 | 4.3 | 4.9 | 4.5 | ||||||||||
| Sister | C | 31 | 13.7 | 31 | 10.7 | 23 | 12.7 | 14.9 | 12.6 | .77 | .43 | .63 | ||||||
| E | 53 | 13.2 | 50 | 12.6 | 40 | 12.2 | 12.9 | 13.0 | ||||||||||
| Brother | C | 29 | 8.6 | 28 | 8.1 | 21 | 11.1 | 10.4 | 10.1 | .13 | .90 | .89 | ||||||
| E | 57 | 7.1 | 56 | 8.6 | 40 | 7.6 | 7.5 | 7.9 | ||||||||||
| Friend | C | 32 | 19.4 | 25 | 18.6 | 17 | 19.1 | 23.0 | 19.3 | .96 | .34 | .54 | ||||||
| E | 70 | 20.4 | 66 | 19.1 | 46 | 21.0 | 21.5 | 19.1 | ||||||||||
| a Means of control and experimental groups b Test of hypothesis of equivalent group means c Test of hypothesis that group means, averaged over time, are equal d Test of hypothesis that means at three points in time, averaged over the groups, are equal e Test of hypothesis of no interaction between group and time, that is, that the pattern of means over time is the same for both groups f Test of hypothesis that time one is equal to average of time two and time three g Test of hypothesis that time two is equal to time three | ||||||||||||||||||
| New Jersey | ||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Post-treatment | Followup | Multivariate Repeated Measures | ||||||||||||||||
| N | Ma | pb | N | M | p | N | Means | Multivariate ps | Univariate ps-Time | Univariate ps-Grp-time interaction | ||||||||
| Initial | Post | Follow | Grpc | Timed | Time -Grpe | Initial v. laterf | Post v. Followg | Initial v. Later | Post v. Follow | |||||||||
| Total | C | 141 | 34.3 | 107 | 35.5 | 84 | 36.3 | 37.1 | 36.9 | .46 | .64 | .35 | ||||||
| E | 221 | 32.9 | 167 | 36.0 | 130 | 35.2 | 33.0 | 35.7 | ||||||||||
| Emotional | C | 141 | 12.6 | 107 | 13.2 | 84 | 13.5 | 13.7 | 13.7 | .58 | .64 | .51 | ||||||
| E | 221 | 12.4 | 167 | 13.5 | 130 | 13.2 | 12.5 | 13.5 | ||||||||||
| Instrumental | C | 141 | 9.2 | 107 | 9.4 | 84 | 10.1 | 9.9 | 9.9 | .26 | .40 | .64 | ||||||
| E | 221 | 8.4 | 167 | 9.4 | 130 | 9.2 | 8.3 | 9.2 | ||||||||||
| Informational | C | 141 | 12.4 | 107 | 12.8 | 84 | 12.6 | 13.5 | 13.3 | .65 | .73 | .24 | ||||||
| E | 221 | 12.1 | 167 | 13.1 | 130 | 12.9 | 12.1 | 13.0 | ||||||||||
| Partner | C | 57 | 9.8 | 53 | 9.4 | 36 | 9.6 | 10.0 | 9.5 | .69 | .62 | .66 | ||||||
| E | 77 | 9.6 | 64 | 9.8 | 36 | 9.1 | 9.3 | 9.7 | ||||||||||
| Mother | C | 95 | 5.7 | 67 | 6.9 | 53 | 6.6 | 6.2 | 6.8 | .23 | .21 | .51 | ||||||
| E | 120 | 5.4 | 101 | 6.1 | 67 | 5.3 | 5.5 | 6.1 | ||||||||||
| Father | C | 69 | 4.2 | 49 | 4.1 | .018 | 36 | 4.8 | 4.8 | 4.7 | .67 | .19 | .12 | |||||
| E | 98 | 3.7 | 63 | 6.1 | 47 | 3.8 | 3.8 | 5.6 | ||||||||||
| Sister | C | 105 | 8.6 | 84 | 8.3 | 60 | 8.4 | 9.5 | 8.7 | .64 | .54 | .73 | ||||||
| E | 168 | 8.2 | 130 | 8.3 | 96 | 8.3 | 8.5 | 7.8 | ||||||||||
| Brother | C | 90 | 5.0 | 73 | 6.1 | 48 | 6.5 | 6.0 | 6.7 | .76 | .29 | .95 | ||||||
| E | 151 | 5.5 | 119 | 6.2 | 90 | 6.2 | 5.3 | 6.3 | ||||||||||
| Friend | C | 107 | 19.6 | 83 | 18.0 | 52 | 18.9 | 20.4 | 17.8 | .47 | .19 | .20 | ||||||
| E | 171 | 19.2 | 144 | 17.7 | 90 | 21.5 | 19.7 | 19.1 | ||||||||||
| a Means of control and experimental groups b Test of hypothesis of equivalent group means c Test of hypothesis that group means, averaged over time, are equal d Test of hypothesis that means at three points in time, averaged over the groups, are equal e Test of hypothesis of no interaction between group and time, that is, that the pattern of means over time is the same for both groups f Test of hypothesis that time one is equal to average of time two and time three g Test of hypothesis that time two is equal to time three | ||||||||||||||||||
| Kentucky | ||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Post-treatment | Followup | Multivariate Repeated Measures | ||||||||||||||||
| N | Ma | pb | N | M | p | N | Means | Multivariate ps | Univariate ps-Time | Univariate ps-Grp-time interaction | ||||||||
| Initial | Post | Follow | Grpc | Timed | Time -Grpe | Initial v. later f | Post v. Followg | Initial v. Later | Post v. Follow | |||||||||
| Total | C | 150 | 39.5 | 119 | 41.0 | 108 | 43.2 | 42.1 | 40.8 | .05 | .82 | .16 | ||||||
| E | 152 | 35.0 | 130 | 39.8 | 117 | 36.5 | 36.4 | 38.8 | ||||||||||
| Emotional | C | 150 | 14.8 | 119 | 15.0 | 108 | 15.8 | 15.6 | 14.8 | .12 | .97 | .16 | ||||||
| E | 152 | 13.4 | 130 | 14.8 | 117 | 13.8 | 14.0 | 14.6 | ||||||||||
| Instrumental | C | 150 | 10.6 | 119 | 11.4 | 108 | 12.1 | 11.5 | 11.3 | .03 | .37 | .65 | ||||||
| E | 152 | 9.0 | 130 | 10.3 | 117 | 9.9 | 9.4 | 9.9 | ||||||||||
| Informational | C | 150 | 14.1 | 119 | 14.7 | 108 | 15.2 | 15.0 | 14.6 | .07 | .51 | .08 | ||||||
| E | 152 | 12.6 | 130 | 14.7 | 117 | 12.8 | 13.0 | 14.4 | ||||||||||
| Partner | C | 50 | 9.1 | .067 | 37 | 9.7 | 32 | 10.1 | 9.7 | 9.6 | .16 | .37 | .43 | |||||
| E | 48 | 10.3 | 49 | 10.2 | 33 | 10.8 | 11.0 | 10.1 | ||||||||||
| Mother | C | 118 | 6.6 | 100 | 6.3 | 85 | 6.2 | 6.6 | 6.0 | .56 | .18 | .22 | ||||||
| E | 118 | 5.5 | 101 | 6.4 | 91 | 5.4 | 6.0 | 6.3 | ||||||||||
| Father | C | 86 | 4.2 | 66 | 3.9 | 61 | 4.0 | 4.4 | 3.7 | .82 | .71 | .31 | ||||||
| E | 94 | 3.5 | 83 | 4.2 | 71 | 3.7 | 3.7 | 4.1 | ||||||||||
| Sister | C | 110 | 9.6 | .031 | 88 | 9.5 | 75 | 9.6 | 10.2 | 10.2 | .04 | .75 | .57 | |||||
| E | 95 | 6.6 | 84 | 7.2 | 69 | 6.7 | 6.3 | 7.2 | ||||||||||
| Brother | C | 112 | 5.7 | 87 | 6.7 | 72 | 6.1 | 6.6 | 6.9 | .36 | .11 | .39 | ||||||
| E | 101 | 5.1 | 93 | 6.4 | 78 | 4.9 | 4.8 | 6.5 | ||||||||||
| Friend | C | 137 | 19.2 | 111 | 20.0 | 93 | 22.6 | 20.3 | 20.3 | .32 | .03 | .44 | .01 | |||||
| E | 139 | 19.5 | 121 | 20.5 | 99 | 20.6 | 19.4 | 20.2 | ||||||||||
| a Means of control and experimental groups b Test of hypothesis of equivalent group means c Test of hypothesis that group means, averaged over time, are equal d Test of hypothesis that means at three points in time, averaged over the groups, are equal e Test of hypothesis of no interaction between group and time, that is, that the pattern of means over time is the same for both groups f Test of hypothesis that time one is equal to average of time two and time three g Test of hypothesis that time two is equal to time three | ||||||||||||||||||
| Pennsylvania | ||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Post-treatment | Followup | Multivariate Repeated Measures | ||||||||||||||||
| N | Ma | pb | N | M | p | N | Means | Multivariate ps | Univariate ps-Time | Univariate ps-Grp-time interaction | ||||||||
| Initial | Post | Follow | Grpc | Timed | Time -Grpe | Initial v. laterf | Post v. Followg | Initial v. Later | Post v. Follow | |||||||||
| Total | C | 115 | 39.7 | 90 | 45.7 | 70 | 39.1 | 41.5 | 48.0 | .23 | .01 | .79 | .01 | .01 | ||||
| E | 161 | 35.6 | 135 | 41.3 | 102 | 36.5 | 38.1 | 42.9 | ||||||||||
| Emotional | C | 115 | 14.4 | .060 | 90 | 16.8 | 70 | 14.1 | 15.3 | 17.9 | .09 | .01 | .57 | .01 | .01 | |||
| E | 161 | 12.4 | 135 | 14.7 | 102 | 12.8 | 13.5 | 15.1 | ||||||||||
| Instrumental | C | 115 | 11.7 | 90 | 13.8 | 70 | 12.3 | 12.1 | 14.5 | .74 | .01 | .82 | .01 | .01 | ||||
| E | 161 | 11.5 | 135 | 13.4 | 102 | 11.7 | 12.1 | 13.9 | ||||||||||
| Informational | C | 115 | 13.5 | .063 | 90 | 15.1 | 70 | 12.7 | 14.1 | 15.6 | .25 | .01 | .66 | .01 | .01 | |||
| E | 161 | 11.6 | 135 | 13.3 | 102 | 12.0 | 12.5 | 13.9 | ||||||||||
| Partner | C | 23 | 10.7 | .014 | 22 | 10.6 | 11 | 10.3 | 11.0 | 10.7 | .50 | .33 | .87 | |||||
| E | 36 | 8.3 | 27 | 10.1 | 12 | 9.3 | 10.1 | 10.4 | ||||||||||
| Mother | C | 87 | 7.2 | 72 | 7.4 | 57 | 7.6 | 7.8 | 7.6 | .94 | .34 | .56 | ||||||
| E | 111 | 7.5 | 104 | 7.5 | 74 | 7.1 | 7.7 | 7.9 | ||||||||||
| Father | C | 63 | 4.4 | .061 | 52 | 5.2 | 40 | 4.0 | 4.8 | 5.2 | .20 | .19 | .23 | |||||
| E | 89 | 3.0 | 78 | 3.8 | 54 | 3.6 | 3.1 | 4.1 | ||||||||||
| Sister | C | 80 | 11.0 | 70 | 12.0 | 50 | 10.4 | 11.8 | 12.7 | .06 | .01 | .83 | .01 | |||||
| E | 116 | 9.6 | 107 | 10.6 | 78 | 7.1 | 9.0 | 10.4 | ||||||||||
| Brother | C | 91 | 10.0 | .019 | 69 | 10.4 | 49 | 9.0 | 10.8 | 11.4 | .01 | .05 | .48 | .03 | ||||
| E | 104 | 6.2 | 97 | 7.6 | 74 | 5.3 | 5.4 | 7.0 | ||||||||||
| Friend | C | 91 | 17.8 | 76 | 20.1 | 42 | 20.9 | 18.0 | 23.9 | .51 | .02 | .09 | .01 | .04 | ||||
| E | 126 | 20.5 | 115 | 20.5 | 74 | 22.6 | 21.8 | 22.4 | ||||||||||
| a Means of control and experimental groups b Test of hypothesis of equivalent group means c Test of hypothesis that group means, averaged over time, are equal d Test of hypothesis that means at three points in time, averaged over the groups, are equal e Test of hypothesis of no interaction between group and time, that is, that the pattern of means over time is the same for both groups f Test of hypothesis that time one is equal to average of time two and time three g Test of hypothesis that time two is equal to time three | ||||||||||||||||||
was essentially equal. The reported levels of friend support declined between the initial and later interviews (p = .01) in both the control and experimental groups.
Pennsylvania caretakers reported higher levels of total, emotional, instrumental, and informational support over time in both the control and experimental groups. With regard to relative and friend support, caretakers in the control group reported more sister and brother support, but the levels of sibling support increased over time in both groups. In addition, the availability of friend support increased over time in the control but not the experimental group (p = .04).
6.4 Conclusions
This analysis examined the kinds of informal supports that were available to caretakers in the study, the composition of the caretakers' support networks, the levels of support that were available to the caretakers from their support networks, and the effect of family preservation services on their levels of support. Over all of the states, the great majority of caretakers identified at least one person that they could rely on for support. Indeed, less than four percent of the caretakers said that they had nobody to count on for any type of support. It is important to recognize that this small group of caretakers may be likely to benefit from services aimed at strengthening their informal support systems. However, most of the caretakers in this study cannot be characterized as being socially isolated since, on average, they reported having over nine friends and relatives in their social networks. This is not to suggest that these caretakers could not use additional informal support, they might, but we cannot assess that with the data at hand.
In terms of the composition of the caretakers' support networks, our findings are consistent with those of other research that has examined family formation trends. Caretakers rely primarily on kinship ties and friends rather than marital bonds for all kinds of support. Overall, only 54 percent of the caretakers in this study reported having fathers with whom they
have regular contact and even fewer were residing with partners (32%). It may be partly a result of the scarcity of fathers and partners and the predominance of women (77 percent of the caretakers' friends were female) in their support networks, that caretakers rely more on females than males for support. However, caretakers who have bothers and fathers are generally less likely to receive support from them than from their mothers, sisters, and friends. Also, brothers and fathers tend to provide lower levels of support. This difference between male and female supporters may be related to the traditional gender division of labor that assigns the bulk of household tasks, family care, and emotional work to women.
Partners who reside with caretakers are more likely to provide support and to provide more support than either male or female extended family members. Hence, becoming involved with a partner might increase the amount of support -- particularly instrumental -- that is available to caretakers. However, the marriage prospects for many single mothers may be quite limited unless the socio-economic conditions that inhibit the formation of two-parent families improve substantially.
Lastly, we examined the effects of family preservation services on caretakers' levels of support and found little impact. It must be emphasized, however, that we do not know the extent to which family preservation workers focused on issues of informal social support. It is possible that this was a relatively unimportant component of these services, and that it is unrealistic to expect to see effects of family preservation services in this area.
Cherlin, A. J. (1992). Marriage, Divorce, Remarriage. Cambridge, MA: Harvard University Press.
Furstenberg, F. F. (2001). The Fading Dream: Prospects for Marriage in the Inner City. In E. Anderson and D. S. Massey (Eds.), Problem of the Century: Racial Stratification in the United States (pp. 224-246). New York, NY: Russell Sage Foundation.
Garfinkel, I. S. and McLanahan, S. S. (1986). Single Mothers and Their Children: A New American Dilemma. Washington, D.C.: The Urban Institute Press.
Henly, J. (1994). Surviving without welfare: The informal support networks of former general assistance recipients. Unpublished doctoral dissertation, University of Michigan.
McLanahan, S., Garfinkel, I., and Mincy, R. B. (2001). Fragile Families, Welfare Reform, and Marriage (Policy Brief No. 10). Washington, DC: The Brookings Institution.
Stack, C. B. (1974). All Our Kin: Strategies for Survival in the Black Community. New York, NY: Harper and Row.
Wilson, W. J. (1987). The Truly Disadvantaged: The Inner City, The Underclass, and Public Policy. Chicago, IL: University of Chicago Press.
99. Repeated measures analysis of this subsample yielded no significant differences between the control and experimental groups' change in the levels of support across the three interviews.
100. Each of the summary measures assumes that caretakers had more support available from supporters with whom they had greater contact. Therefore, they only approximate the caretakers' levels of support.
101. In the secondary analysis, Kentucky experimental group caretakers reported having more support available from their partners (averaged across time). There was, however, no change in the average levels of support over time in either group.
7. Interviews with Investigating or Intake Workers
7.1 Investigating Worker Interviews in Kentucky, New Jersey, and Tennessee
Investigating workers were asked to complete a self-administered questionnaire as quickly as possible after random assignment. (102) Workers were asked a series of questions about the allegations, the investigation, and the results of the investigation. In New Jersey and Kentucky, cases came into the study through ongoing workers as well as investigation units, whereas in Tennessee all cases came from investigation units. Table 7-1 shows the number of completed questionnaires and response rates for cases coming from investigating workers (75% in Kentucky, 50% in New Jersey, and 70% in Tennessee). The content of the questionnaire filled out by Philadelphia intake workers was considerably different; therefore those data are presented in a separate section.
| Site | Number of cases randomly assigned | Percent referred from investigation units (%) | Responses from workers in investigation units | |
|---|---|---|---|---|
| N | % | |||
| Kentucky | 358 | 62 | 164 | 75 |
| New Jersey | 470 | 53 | 126 | 50 |
| Tennessee | 153 | 100 | 107 | 70 |
Table 7-2 provides some data on the timelines and frequency of contact in the investigation of the alleged maltreatment. Depending on the state, an average of one to three days passed between the date the complaint was received and the date the worker first contacted the family. There was greater variation in the number of days between the date the complaint was received and the date the investigation was completed; an average of about 40 days in Kentucky (75% completed in 56 days), 12½ days in New Jersey (75% completed in 13 days), and just under 29 days in Tennessee (75% completed in 27 days). (103) On average, investigating workers in each of the three states reported making about 3 visits to the home where they met with one or
| Kentucky | New Jersey | Tennessee | |||||||
|---|---|---|---|---|---|---|---|---|---|
| N | Mean | s.d. | N | Mean | s.d. | N | Mean | s.d. | |
| Number of days from complaint to first talking with the family | 147 | 1.43 | 3.05 | 119 | 3.34 | 10.34a | 104 | 1.88 | 3.85 |
| Number of days from complaint to completion of investigation | 108 | 43.37 | 46.56 | 102 | 12.52 | 20.86 | 32 | 28.75 | 37.11 |
| Number of visits to the home where worker met with one or more members of the household | 148 | 3.20 | 2.25 | 119 | 2.97 | 1.85 | 104 | 2.85 | 3.32 |
| a The large standard deviation here is primarily due to 3 cases where the first contact with the family occurred 57, 63, and 72 days after the complaint was received. Excluding those three cases, the mean for NJ is 1.77 days (s.d. = 3.21). | |||||||||
more members of the household (the average number of visits was slightly higher in Kentucky and lower in Tennessee).
Table 7-3 provides responses by state for each of the questions asked in the investigating worker interview. When asked who filed the recent complaint, investigating workers in all three states frequently cited school personnel, medical or public health personnel, and law enforcement personnel. Other frequent sources of complaints in Kentucky included social service agency personnel, relatives, and neighbors. In New Jersey, other frequent sources of complaints included social service agency personnel and self-reports by the caretaker. In Tennessee, other frequent sources of complaints were relatives and a category labeled "someone else" which included juvenile court. The source of the complaint was contacted in 83 percent of the investigations in Kentucky, 68 percent of the investigations in New Jersey, and 89 percent of the investigations in Tennessee.
Besides the source of the complaint, investigating workers were asked whether they obtained information from law enforcement, a hospital, clinic or doctor, school, or other agency. In all three states, schools were the most frequent sources of information for investigations (56% in Kentucky, 67% in New Jersey, and 45% in Tennessee). Information was obtained from law enforcement in 32 percent of investigations in Kentucky, 46 percent in New Jersey, and 19 percent in Tennessee. Information was obtained from hospitals, clinics or doctors in 35 to 45 percent of investigations, and from other agencies in about 20 to 30 percent of investigations. (104)
| Kentucky | New Jersey | Tennessee | ||||
|---|---|---|---|---|---|---|
| N | % | N | % | N | % | |
| Who filed the recent complaint (cumulative % > 100) | ||||||
| Medical or public health personnel | 25 | 15 | 23 | 18 | 27 | 25 |
| School personnel | 51 | 31 | 37 | 29 | 22 | 21 |
| Child care personnel | 3 | 2 | 1 | 1 | 1 | 1 |
| Law enforcement personnel | 29 | 18 | 26 | 21 | 10 | 9 |
| Social service agency personnel | 16 | 10 | 15 | 12 | 7 | 7 |
| Caretaker | 13 | 8 | 15 | 12 | 5 | 5 |
| Child victim | 2 | 1 | 5 | 4 | 0 | 0 |
| Other household member | 1 | <1 | 3 | 2 | 1 | 1 |
| Relative(s) not in household | 17 | 10 | 9 | 7 | 14 | 13 |
| Neighbor(s), other non-relative(s) not in home | 19 | 12 | 9 | 7 | 8 | 8 |
| Anonymous person | 4 | 2 | 4 | 3 | 2 | 2 |
| Someone else | 5 | 3 | 2 | 2 | 12 (105) | 11 |
| Don't know | 3 | 2 | 1 | 1 | 1 | 1 |
| Law enforcement took part in the investigation | 44 | 27 | 38 | 30 | 28 | 26 |
| Allegation(s) involved the following harm to a child | ||||||
| Physical harm (other than sex abuse) | 83 | 51 | 72 | 57 | 60 | 56 |
| Sex abuse | 10 | 6 | 3 | 2 | 7 | 7 |
| Drug presence in newborn | 2 | 1 | 2 | 2 | 6 | 6 |
| Alcohol presence in newborn | 0 | 0 | 1 | 1 | 0 | 0 |
| Alleged harm (in 5a) was confirmed by a physician | ||||||
| Physical harm | 3 | 2 | 11 | 9 | 25 | 23 |
| Sex abuse | 1 | <1 | 2 | 2 | 2 | 2 |
| Drug presence | 2 | 1 | 1 | 1 | 6 | 6 |
| Alcohol presence in newborn | 0 | 0 | 1 | 1 | 0 | 0 |
| You or someone else from your agency contacted source of recent complaint | 136 | 83 | 86 | 68 | 95 | 89 |
| You or another worker visited the home | 151 | 92 | 123 | 98 | 100 | 94 |
| Talked to the child's caretaker | 161 | 98 | 125 | 99 | 107 | 100 |
| Found it difficult to gain admission to the home | 15 | 9 | 8 | 6 | 11 | 10 |
| You obtained information from | ||||||
| Law enforcement | 53 | 32 | 58 | 46 | 20 | 19 |
| Hospital, clinic, or doctor | 63 | 38 | 57 | 45 | 36 | 34 |
| School | 92 | 56 | 84 | 67 | 48 | 45 |
| Other agency | 51 | 31 | 27 | 21 | 21 | 20 |
| Total N | 164 | 126 | 107 | |||
| Contact with individuals involved in the investigation: | ||||||
|---|---|---|---|---|---|---|
| The perpetrator | ||||||
| By phone only | 1 | <1 | 0 | 0 | 4 | 4 |
| In person only | 33 | 20 | 37 | 29 | 19 | 18 |
| By phone & in person | 120 | 73 | 63 | 50 | 69 | 65 |
| Did not talk to individual | 5 | 3 | 21 | 17 | 12 | 11 |
| The caretaker | ||||||
| By phone only | 1 | <1 | 1 | 1 | 4 | 4 |
| In person only | 32 | 20 | 39 | 31 | 17 | 16 |
| By phone & in person | 127 | 77 | 77 | 61 | 85 | 79 |
| Did not talk to individual | 1 | <1 | 4 | 3 | 1 | 1 |
| Child(ren) in complaint | ||||||
| By phone only | 0 | 0 | 1 | 1 | 1 | 1 |
| In person only | 113 | 69 | 80 | 64 | 78 | 73 |
| By phone & in person | 34 | 21 | 32 | 25 | 11 | 10 |
| Did not talk to individual | 14 | 9 | 12 | 10 | 16 | 15 |
| Neighbor(s) | ||||||
| By phone only | 13 | 8 | 2 | 2 | 6 | 6 |
| In person only | 15 | 9 | 6 | 5 | 11 | 10 |
| By phone & in person | 11 | 7 | 8 | 6 | 5 | 5 |
| Did not talk to individual | 113 | 69 | 92 | 73 | 74 | 69 |
| Relative(s) | ||||||
| By phone only | 43 | 26 | 11 | 9 | 22 | 21 |
| In person only | 20 | 12 | 34 | 27 | 20 | 19 |
| By phone & in person | 47 | 29 | 21 | 17 | 30 | 28 |
| Did not talk to individual | 50 | 31 | 51 | 41 | 34 | 32 |
| Complainant(s) | ||||||
| By phone only | 52 | 32 | 31 | 25 | 27 | 25 |
| In person only | 19 | 12 | 19 | 15 | 13 | 12 |
| By phone & in person | 57 | 35 | 32 | 25 | 29 | 27 |
| Did not talk to individual | 34 | 21 | 37 | 29 | 35 | 33 |
| Perpetrator denies all or most charges: | 75 | 46 | 64 | 51 | 55 | 51 |
| Worker provided services (other than monitoring): | 120 | 73 | 89 | 71 | 76 | 71 |
| Worker made referrals to services: | 153 | 93 | 119 | 94 | 103 | 96 |
| Actions taken concerning removal of child(ren): | ||||||
| You or another worker removed child(ren) | 16 | 10 | 14 | 11 | 7 | 7 |
| You or another worker asked police or authorities to remove child(ren) | 9 | 6 | 1 | 1 | 2 | 2 |
| Police or other authorities removed child(ren) without being asked to do so by social worker | 12 | 7 | 4 | 3 | 6 | 6 |
| The child(ren) were not removed | 117 | 71 | 97 | 77 | 90 | 84 |
| Total N | 164 | 126 | 107 | |||
In all three states, approximately one quarter to one-third of investigations involved participation of law enforcement.
Investigating workers were asked whether the allegations involved one of four specific types of maltreatment: physical harm, sexual abuse, drug presence in a newborn, and alcohol presence in newborn. In all three states, approximately 50 to 60 percent of the investigations involved allegations of physical harm, whereas less than 10 percent involved sexual abuse, and 1 percent or less involved alcohol presence in a newborn. Drug presence in a newborn was alleged in 6 percent of investigations in Tennessee and 1 to 2 percent of investigations in Kentucky and New Jersey. (106)
For each of the four specified types of alleged harm, investigating workers were asked whether the alleged harm was confirmed by a physician. Investigating workers responded affirmatively in 2 percent or fewer of the cases in Kentucky, regardless of type of harm. In New Jersey, alleged physical harm was confirmed by a physician in 9 percent of the investigations, alleged sexual abuse in 2 percent of investigations, alleged drug presence in 1 percent, and alleged alcohol presence in 1 percent. In Tennessee, 23 percent of investigations involving alleged physical harm were confirmed by a physician. Two percent of Tennessee investigations involving alleged sexual abuse were confirmed by a physician, 6 percent of drug presence in a newborn cases, and in none of the cases did a physician confirm allegations of alcohol presence in a newborn.
When asked about contact and visits to the home, over 90 percent of investigating workers in all three states reported that they or another worker visited the home and that they talked with the child's caretaker. In fewer than 10 percent of the investigations in each state was it reported to be difficult to gain admission to the home. As shown in the table, the perpetrator and caretaker were most frequently contacted both by phone and in person, whereas children were contacted in person in 64 to 73 percent of investigations, by phone and in person in 10 to 25 percent of investigations, and not at all in 9 to 15 percent of investigations.
When asked whether the perpetrator denied all or most of the charges, the investigating worker responded affirmatively in 46 percent of the investigations in Kentucky, 51 percent in New Jersey, and 51 percent in Tennessee.
Workers in all three states reported providing services (other than monitoring) in approximately 70 percent of the investigations. They reported making referrals to services in over 90 percent of investigations.
Finally, investigating workers were asked about actions taken concerning removal of a child or children form the home. Children were removed from the home, either by a worker or by police, in approximately 17 percent of the investigations in Kentucky, 14 percent in New Jersey, and 13 percent in Tennessee. An additional 1 percent of investigations in New Jersey, 2 percent in Tennessee, and 6 percent in Kentucky involved a request by the worker for the police or authorities to remove children from the home. According to the investigating workers, children were not removed from the home in 71 percent of investigations in Kentucky, 77 percent of investigations in New Jersey, and 84 percent of investigations in Tennessee. The number of placements suggested by investigating workers seemed higher than expected given the evidence from analysis of administrative data on placements. Therefore, these cases were reviewed and cross-checked with other data sources.
All 65 cases (19 in NJ, 33 in KY, and 13 in TN) in which an investigating worker indicated either a request was made or action was taken to remove a child from the home were reviewed and cross-checked with available caseworker interviews, caretaker interviews, and administrative data. This additional analysis was done to address two primary concerns: undetected inappropriate referrals (cases in which we were not aware that all identified at-risk children were out of the home at the time of random assignment) and placements that were not captured in the analyses based on administrative data. Table 7-4 provides a breakdown of these cases by experimental group and the status of the findings. In summary, there do not appear to be any inappropriate referrals among these 65 cases. As for cases in which a placement occurred that was not included in any of the placement outcome analyses detailed in the main report, there are at most 9 cases in Kentucky (6 control group (3 of which are relative placements) and 3 FPS (1 relative placement)); and 3 cases in New Jersey (2 control group (both relative placements) and 1 FPS). No such cases were found in Tennessee.
| Placement Status | Kentucky | New Jersey | Tennessee | |||
|---|---|---|---|---|---|---|
| C | E | C | E | C | E | |
| Investigating worker knew about a placement that occurred during the investigation but before the IW form was completed (All of these placements were recorded in the administrative data) | 3 | - | 3 | - | 4 | 5 |
| Administrative data shows a placement occurred after the date that the investigating worker form was completed. (107) | 7 | 2 | 4 | 2 | - | 2 |
| Relative placements (In KY and TN, additional analyses were run that included known relative placements not reflected in the administrative data- the number in parentheses is the number of relative placements identified by investigating workers that were not included in those additional analyses) | 4 (3) | 2 (1) | 4 | - | - | 1(0) |
| No additional confirmation of any placement (a request may have been made without action or children may have been temporarily removed for a short period of time but not officially placed in care) | 5 | 4 | 4 | 2 | - | - |
| A child in the home appears to have a placement. (number in parentheses indicates the number of cases in which no placement is recorded in the administrative data for any member of this case) | 3 (2) | 2 (2) | - | 1 (0) | - | |
| Undetermined / conflicting information (number in parentheses indicates the number of cases in which no placement is recorded in the administrative data for any member of this case) | 1(1) | - | - | 1(1) | - | - |
| Total | 23 | 10 | 13 | 6 | 4 | 8 |
| Note: C = Control Group, E = Experimental Group | ||||||
7.2 Philadelphia Intake Worker Questionnaire
Resisting the use of another survey form solely for the purposes of the evaluation, staff in Philadelphia negotiated the use of a data collection instrument other than the investigating worker survey that was administered in the three Homebuilders sites of this study. Instead, investigating workers completed a brief intake questionnaire focusing on conditions in the home, problems for adults and children in the household, and service referrals that occurred as part of the investigation. This questionnaire was completed for 280 of the 353 randomly assigned cases (79%). Table 7-5 summarizes the findings for all items in the questionnaire.
When asked about the conditions observed during visits to the home, a majority of workers responded favorably on each of the 7 items. However, in 30 percent of the cases workers said they did not find the home in generally good repair; in 32 percent of the cases workers indicated there were not an adequate number of beds and bedding; and in 27 percent of the cases workers reported that they did not feel the neighborhood was safe.
In over 80 percent of the cases, the intake worker responded affirmatively to each of three questions about the relationship between the caretaker and the victim: did the caretaker show affection for the victim, did the caretaker show concern for the victim, and did the victim show signs of attachment to the caretaker.
According to the intake workers, the most frequently noted problems for adults in the household included parenting skills in general (88%), depression (56%), and inadequate supervision of the children (51%). The most frequently noted problems for children in the household included parent child conflict (37%), inadequate supervision of children (37%), and insufficient income for necessities such as food, rent, or clothing (37%).
The overwhelming majority of cases (92%) were referred for family preservation services (evidently, in the remaining 8% someone other than the intake worker made the FPS referral), and approximately a third were also referred to other services. Intake workers reported that in fewer than 10 percent of cases, the court ordered each of the following: family preservation services, foster care placement, other substitute care placement, or other services.
| N | Yes (%) | No (%) | Don't know/ did not observe (%) | |
|---|---|---|---|---|
| When you visited the home, did you find | ||||
| the home in generally good repair | 274 | 69 | 30 | 2 |
| the electricity in working order | 279 | 98 | 1 | 1 |
| the toilet, bath and shower in working order | 279 | 89 | 8 | 3 |
| the refrigerator and stove in working order | 278 | 87 | 10 | 2 |
| adequate number of beds and bedding | 278 | 66 | 32 | 2 |
| the home to be physically safe | 279 | 87 | 11 | 2 |
| the neighborhood to be safe | 275 | 61 | 27 | 12 |
| N | Yes (%) | No (%) | Don't know/ did not observe (%) | |
|---|---|---|---|---|
| Thinking about the relationship between the caretaker and the victim(s) | ||||
| Did the caretaker show affection for the victim(s) | 278 | 82 | 14 | 4 |
| Did the caretaker show concern for the victim(s) | 278 | 89 | 9 | 1 |
| Did the victim(s) show signs of attachment to the caretaker | 277 | 83 | 10 | 6 |
| Adult(s) | Child(ren) | |||||||
|---|---|---|---|---|---|---|---|---|
| N | Yes (%) | No (%) | DK (%) | N | Yes (%) | No (%) | DK (%) | |
| Indicate whether the child(ren) or adult(s) have these common problems that may exist in the household. | ||||||||
| Physical health problems or disabilities | 257 | 32 | 60 | 9 | 268 | 31 | 64 | 5 |
| Depression | 269 | 56 | 25 | 19 | 256 | 22 | 60 | 19 |
| Other mental illness | 266 | 15 | 50 | 35 | 263 | 7 | 68 | 25 |
| Mental retardation | 265 | 5 | 85 | 11 | 267 | 5 | 85 | 11 |
| Alcoholism | 270 | 21 | 63 | 16 | 260 | 0 | 95 | 4 |
| Drug abuse | 275 | 43 | 42 | 15 | 252 | 3 | 93 | 4 |
| Parent child conflict | 269 | 42 | 52 | 6 | 266 | 37 | 57 | 6 |
| Arrests or convictions on criminal charges | 271 | 13 | 54 | 34 | 261 | 2 | 89 | 9 |
| Domestic violence | 273 | 23 | 61 | 17 | 248 | 11 | 77 | 11 |
| Inadequate supervision of children | 267 | 51 | 43 | 5 | 250 | 37 | 58 | 5 |
| Insufficient income for necessities such as food, rent or clothing | 272 | 43 | 51 | 6 | 243 | 37 | 58 | 5 |
| Overly severe discipline measures toward children | 268 | 23 | 73 | 4 | 238 | 17 | 90 | 3 |
| Finding or holding on to a place to live | 269 | 37 | 61 | 2 | 234 | 24 | 74 | 2 |
| Lack of discipline toward children | 267 | 27 | 67 | 6 | 230 | 24 | 72 | 4 |
| Parenting skills in general | 274 | 88 | 11 | 1 | 202 | 45 | 52 | 4 |
| Where was this case referred? | N | (%) |
|---|---|---|
| Family preservation | 280 | 92 |
| Foster care | 280 | 1 |
| Other services | 280 | 31 |
| The case was closed | 280 | 3 |
| Don't know | 280 | 1 |
| N | Yes (%) | No (%) | Don't know (%) | |
|---|---|---|---|---|
| In the course of or as a result of the investigation of the children in this complaint, has the court ordered . . . | ||||
| Family preservation services | 257 | 6 | 84 | 2 |
| Foster care placement | 253 | 1 | 88 | 1 |
| Other substitute care placement | 255 | 3 | 86 | 1 |
| Other services | 255 | 8 | 81 | 2 |
| Note: The maximum N for individual questions ranged from 202 to 280. | ||||
102. In a few cases, these instruments were completed by phone.
103. In Kentucky, some cases were kept open under the investigating worker in order to provide services. It is also possible that these timeframes reflect variations in what is meant by a "complete investigation." It is possible that all but the paperwork was completed in a shorter period of time.
104. Not all cases necessitate contact with these additional sources of information, and the character of the allegation obviously affects whom it is relevant to contact.
105. Ten of these cases were coded as juvenile court
106. The small proportion of cases with drug or alcohol presence is expected given that these family preservation programs avoided serving cases with known drug or alcohol use.
107. The length of time from the date the interview was completed to the date the placement occurred according to the administrative data varied from five days to two years. The placement in the administrative data may or may not be the removal that was referred to by the investigating worker. In any event, a placement is accounted for in the administrative data analysis.
8. The Staff Questionnaire
An important aspect of program implementation and service delivery is the characteristics of the program staff. In their detailed description of the Homebuilders Model, Kinney, Haapala, and Booth (1991) note the difficulty in recruiting and maintaining qualified counselors to staff the family preservation programs. While the authors present their "first choice" applicant as "someone with a masters degree in social services, with a cognitive-behavioral theoretical background, and several years' experience working with families," they also note that "gender, age, race, marital status, parenthood, educational field, and degree have not been correlated with effectiveness on the job." Besides individual characteristics, Kinney, Haapala, and Booth encourage the selection of staff who share similar values, attitudes, and styles, cautioning that "if staff have large differences in how they view clients, it is likely they will differ about other agency policies, procedures, and ways they wish to relate to each other and to the community."
In this study, caseworkers in both public and family preservation agencies completed a self-administered questionnaire in which they were asked about a variety of job-related items, including their experience, qualifications, training, job satisfaction, preparation for dealing with casework issues, and attitudes toward clients and services.
Table 8-1 shows the number of completed questionnaires for each state and agency, separated by whether or not the respondent had a case in the study and by whether or not the respondent was considered to be case level staff. A respondent was considered case level staff if he or she reported carrying cases. (108) This summary focuses only on those case-level staff who carried a case in this study, with response rates ranging from 75 percent in Tennessee and 76 percent in New Jersey to 91 percent in Kentucky. Data on staff in Philadelphia are presented in a separate section as both family preservation and traditional services were provided by private agency staff and some workers carried cases from both random assignment groups.
|
Type of position & agency |
Does this respondent have a case in the study? | |||||||
|---|---|---|---|---|---|---|---|---|
| Kentucky | New Jersey | Tennessee | Philadelphiaa | |||||
| No | Yes | No | Yes | No | Yes | No | Yes | |
| Case level staff | ||||||||
| Public agency | 39 | 159 | 136 | 199 | 6 | 37 | 35 | 139 |
| FPS agency | 4 | 18 | -- | 29 | 1 | 17 | В | |
| "Other" staff | ||||||||
| Public agency | 34 | 19 | 99 | 29 | 5 | 2 | 25 | 12 |
| FPS agency | 3 | -- | 2 | 6 | 1 | 2 | В | |
| Number of completed questionnaires | 80 | 196 | 237 | 263 | 13 | 61 | 60 | 151 |
| Number of questionnaires mailed out to staff who have a case in the study | В | 215 | В | 344 | В | 81 | В | 334 |
| Response rate for staff with a case in the study | В | 91% | В | 76% | В | 75% | В | 63% |
| a Philadelphia respondents are not categorized according to FPS or Public Agency status. Both FPS and traditional (SCOH) services were provided by private agency staff and some workers were responsible for both types of cases. | ||||||||
8.1 Kentucky, New Jersey, and Tennessee Staff
Table 8-2 provides a summary of responses by staff with a case in the study to most of the questionnaire items, separated by state. Within each state, responses from Public and FPS staff are reported separately.
| Kentucky | New Jersey | Tennessee | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Public | FPS | p | Public | FPS | p | Public | FPS | p | |||||||
| N | % | N | % | N | % | N | % | N | % | N | % | ||||
| Respondent is male | 15 | 20 | 18 | 22 | 195 | 22 | 29 | 35 | 36 | 11 | 16 | 25 | |||
| Race/Ethnicity | 15 | 18 | 185 | 28 | 34 | 15 | .0 | ||||||||
| Black (not Hispanic) | 25 | 22 | 43 | 25 | 74 | 53 | |||||||||
| White (not Hispanic) | 73 | 78 | 42 | 54 | 27 | 33 | |||||||||
| Hispanic | 1 | 0 | 11 | 18 | |||||||||||
| Other | 2 | 0 | 4 | 4 | 13 | ||||||||||
| Participated in on the job training in past 12 months | 15 | 92 | 17 | 94 | 196 | 91 | 29 | 100 | 37 | 76 | 17 | 88 | |||
| Attended seminar or workshop in past 12 months | 15 | 91 | 17 | 100 | 196 | 90 | 29 | 93 | 37 | 97 | 17 | 100 | |||
| Taken for-credit courses in a degree program in past 12 | 15 | 18 | 17 | 47 | .01 | 188 | 23 | 28 | 36 | 37 | 16 | 16 | 31 | ||
| Taken non-credit course on a work-related topic in past 12 | 15 | 23 | 17 | 18 | 190 | 43 | 28 | 39 | 36 | 14 | 17 | 29 | |||
| Participated in other in-service training in past 12 months | 15 | 82 | 17 | 94 | 193 | 83 | 29 | 86 | 37 | 89 | 17 | 82 | |||
| Field of study | 15 | 17 | 199 | 29 | 37 | 17 | |||||||||
| Education | 5 | 6 | 10 | 3 | 16 | 17 | |||||||||
| Law/criminal justice | 10 | 6 | 10 | 8 | |||||||||||
| Math/science | 1 | 3 | |||||||||||||
| Mgmt/administration | 3 | 4 | 3 | 16 | |||||||||||
| Other humanities | 6 | 8 | 7 | 5 | 6 | ||||||||||
| Social services | 49 | 65 | 33 | 28 | 32 | 47 | |||||||||
| Social work | 25 | 29 | 26 | 38 | 19 | 41 | |||||||||
| No code/NA/missing | 3 | 2 | 7 | 6 | |||||||||||
| Level of education | 15 | 18 | .001 | 197 | 29 | .001 | 37 | 17 | .0 | ||||||
| AA degree, high school diploma, GED, or less | 7 | 3 | |||||||||||||
| Bachelors degree | 58 | 55 | 31 | 62 | 6 | ||||||||||
| Some graduate study | 23 | 11 | 24 | 17 | 30 | 30 | |||||||||
| Masters degree or beyond | 18 | 89 | 14 | 52 | 5 | 65 | |||||||||
| Future employment plans | 15 | 16 | 183 | 28 | .03 | 34 | 16 | ||||||||
| Do not plan to look for other employment or would like | 83 | 88 | 85 | 100 | 74 | 81 | |||||||||
| Looking for other employment but have not found | 17 | 12 | 15 | 0 | 27 | 19 | |||||||||
| Kentucky | New Jersey | Tennessee | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Public | FPS | Public | FPS | Public | FPS | ||||||||||
| N | Mean | N | Mean | p | N | Mean | N | Mean | p | N | Mean | N | Mean | p | |
| Usual caseload size | 15 | 19.5 | 17 | 2.2 | .00 | 19 | 22.8 | 29 | 3.1 | .00 | 34 | 49.0 | 17 | 2.1 | .00 |
| Largest # of families in caseload | 151 | 23.9 | 18 | 1.9 | .001 | 190 | 28.9 | 27 | 3.0 | .001 | 35 | 58.8 | 17 | 3.1 | .001 |
| % of time spent investigating abuse/neglect complaints | 158 | 38.23 | 18 | 2.5 | .001 | 179 | 38.8 | 26 | .03 | .001 | 36 | 58.2 | 17 | 7.1 | .001 |
| % time spent on family preservation services | 156 | 14.4 | 18 | 91.6 | .001 | 178 | 13.4 | 29 | 93.9 | .001 | 35 | 17.3 | 17 | 94.7 | .001 |
| % time spent on foster care placement and/or supervision | 156 | 15.4 | 18 | 1.7 | .001 | 180 | 16.0 | 26 | 0.0 | .001 | 35 | 5.8 | 17 | 0.3 | .002 |
| % time spent on followup services for abuse/neglect families | 157 | 33.3 | 18 | 3.1 | .001 | 181 | 36.5 | 26 | 3.4 | .001 | 35 | 18.1 | 17 | 1.5 | .001 |
| How satisfied are you with various aspects of your job? | |||||||||||||||
| Salary | 158 | 4.5 | 18 | 3.7 | .02 | 184 | 3.3 | 28 | 3.6 | 36 | 4.2 | 17 | 3.6 | .07 | |
| Workload | 158 | 3.4 | 18 | 2.1 | .001 | 183 | 3.5 | 28 | 2.0 | .001 | 36 | 3.9 | 17 | 2.0 | .001 |
| Supervision | 157 | 2.1 | 18 | 1.9 | 178 | 2.5 | 28 | 1.8 | .002 | 34 | 2.7 | 16 | 1.8 | .01 | |
| Work difficulty | 157 | 2.6 | 18 | 2.0 | .001 | 182 | 3.1 | 27 | 2.4 | .001 | 36 | 3.3 | 16 | 2.5 | .01 |
| Chances of promotion | 157 | 3.2 | 18 | 3.6 | 177 | 3.9 | 27 | 3.9 | 33 | 4.1 | 17 | 4.1 | |||
| Fringe benefits | 157 | 2.6 | 18 | 2.5 | 183 | 2.9 | 28 | 3.0 | 35 | 3.1 | 17 | 3.2 | |||
| Overall job satisfaction (average for all 6 items) | 158 | 3.0 | 18 | 2.6 | .004 | 183 | 3.2 | 28 | 2.8 | .002 | 36 | 3.5 | 17 | 2.9 | .001 |
| Indicate how much you agree/disagree with the following statements: | |||||||||||||||
| Abuse and neglect are social problemsservices cannot do much to save | 155 | 3.1 | 18 | 3.5 | .03 | 185 | 2.8 | 29 | 3.7 | .00 | 37 | 3.6 | 17 | 3.7 | |
| No matter how bad a natural family is, foster care is usually worse | 158 | 3.3 | 18 | 3.1 | 187 | 3.2 | 29 | 3.2 | 37 | 3.4 | 17 | 3.1 | |||
| There are many cases in which children remain at home and would have | 158 | 2.2 | 18 | 2.3 | 186 | 2.3 | 29 | 2.5 | 37 | 2.4 | 17 | 2.3 | |||
| It is never justified to take chances with the lives of children | 158 | 1.4 | 18 | 1.4 | 185 | 1.3 | 29 | 1.4 | 37 | 2.1 | 17 | 1.5 | |||
| If a child I left at home were seriously injured due to maltreatment, I | 158 | 1.9 | 18 | 1.9 | 187 | 1.9 | 29 | 2.3 | .05 | 37 | 2.0 | 17 | 1.9 | ||
| Removing a child from his or her parents can be so deep a trauma to the child that it is almost always worth taking the risk to leave the child | 158 | 3.3 | 18 | 3.1 | 186 | 3.3 | 29 | 3.0 | .04 | 37 | 3.4 | 17 | 2.9 | ||
| Families who deny the truth of a validated allegation of abuse or neglect | 154 | 2.7 | 18 | 3.0 | 185 | 2.6 | 29 | 3.1 | .01 | 37 | 2.9 | 17 | 3.1 | ||
| Most families with records of several past complaints should not be given | 156 | 3.0 | 18 | 3.2 | 187 | 3.0 | 29 | 3.2 | 37 | 3.1 | 17 | 3.5 | |||
| Placement prevention should be primary goal of family preservation pgrms | 156 | 2.0 | 18 | 2.4 | 186 | 1.7 | 29 | 2.1 | .06 | 37 | 2.0 | 17 | 2.0 | ||
| Only families with a child at imminent risk of placement should be referred to intensive family preservation services | 157 | 2.7 | 18 | 1.5 | .001 | 187 | 2.8 | 29 | 2.2 | .001 | 37 | 3.1 | 17 | 2.3 | .08 |
| Kentucky | New Jersey | Tennessee | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Public | FPS | Public | FPS | Public | FPS | ||||||||||
| N | Mean | N | Mean | p | N | Mean | N | Mean | p | N | Mean | N | Mean | p | |
| Indicate how much you agree/disagree with the following statements about your reactions to work with clients: | |||||||||||||||
| I feel I treat some of my clients as impersonal objects | 158 | 3.6 | 18 | 3.8 | .06 | 198 | 3.8 | 27 | 3.9 | 37 | 3.7 | 17 | |||
| I deal very effectively with the problems of my clients | 158 | 1.8 | 18 | 1.5 | .06 | 198 | 1.6 | 29 | 1.6 | 37 | 1.7 | 17 | 1.6 | ||
| I have become more callous toward people since being | 156 | 2.9 | 18 | 3.5 | .009 | 196 | 3.3 | 29 | 3.8 | .002 | 37 | 3.1 | 17 | 3.7 | .01 |
| Many clients cannot be helped no matter what I do | 158 | 2.7 | 18 | 3.4 | .005 | 198 | 2.8 | 29 | 3.4 | .001 | 37 | 2.8 | 17 | 3.5 | .007 |
| I think clients often blame me for their problems | 158 | 2.3 | 18 | 3.6 | .001 | 198 | 2.6 | 29 | 3.8 | .001 | 37 | 2.9 | 17 | 3.3 | |
| I have accomplished much that is worthwhile in this job | 158 | 1.8 | 18 | 1.4 | .09 | 198 | 1.7 | 28 | 1.5 | 37 | 1.7 | 17 | 1.5 | ||
| I feel burned out from my work | 158 | 2.6 | 18 | 3.0 | 197 | 2.5 | 29 | 3.3 | .001 | 37 | 2.1 | 17 | 2.4 | ||
| How prepared do you feel to deal with each of the following | |||||||||||||||
| Assessing problems | 158 | 1.8 | 18 | 1.4 | .05 | 196 | 1.5 | 29 | 1.6 | 37 | 1.6 | 17 | 1.4 | ||
| Assessing risk | 158 | 1.8 | 18 | 1.6 | 197 | 1.6 | 29 | 1.7 | 36 | 1.7 | 17 | 1.5 | |||
| Case planning | 154 | 2.1 | 18 | 1.7 | .09 | 196 | 1.7 | 29 | 1.8 | 35 | 1.8 | 17 | 1.8 | ||
| Assessing family functioning | 155 | 2.0 | 18 | 1.7 | 196 | 1.8 | 29 | 1.6 | 36 | 2.2 | 17 | 1.6 | .03 | ||
| Assessing child functioning | 155 | 2.1 | 18 | 1.5 | .001 | 196 | 1.8 | 29 | 1.8 | 36 | 2.1 | 17 | 1.8 | ||
| Family systems | 153 | 2.1 | 18 | 1.7 | .04 | 193 | 1.9 | 29 | 1.9 | 35 | 2.1 | 17 | 1.9 | ||
| Building client relationships | 153 | 1.6 | 18 | 1.2 | .02 | 196 | 1.5 | 29 | 1.3 | .02 | 35 | 1.5 | 17 | 1.3 | |
| Counseling families | 149 | 2.3 | 18 | 1.5 | .001 | 187 | 2.0 | 29 | 1.6 | .02 | 34 | 2.1 | 17 | 1.5 | .009 |
| Permanency planning | 147 | 2.4 | 12 | 2.1 | .10 | 162 | 2.4 | 24 | 2.2 | 29 | 2.6 | 13 | 2.4 | ||
| Knowing when to terminate a case | 148 | 2.3 | 18 | 1.7 | .02 | 196 | 2.0 | 29 | 1.8 | 37 | 2.4 | 17 | 1.7 | .02 | |
| Combined measure of how prepared respondent feels | 155 | 2.1 | 18 | 1.6 | .001 | 196 | 1.8 | 29 | 1.7 | 35 | 2.0 | 17 | 1.7 | . | |
| How many years of experience have you had in the following knids of work: | |||||||||||||||
| Social work in general | 137 | 5.4 | 16 | 3.9 | 195 | 12.2 | 28 | 6.2 | .001 | 35 | 6.9 | 17 | 7.9 | ||
| Child welfare social work | 130 | 4.0 | 14 | 2.9 | 176 | 9.5 | 20 | 4.6 | .001 | 33 | 5.0 | 10 | 4.2 | ||
| Supervising others in social work | 120 | 0.7 | 12 | 0.2 | 162 | 1.5 | 21 | 1.1 | 32 | 0.7 | 9 | 1.7 | .10 | ||
| Delivering family preservation services | 117 | 1.4 | 13 | 2.5 | 164 | 3.1 | 24 | 2.4 | 32 | 4.4 | 15 | 4.2 | |||
| # of years working for this agency | 147 | 3.7 | 17 | 2.0 | .01 | 189 | 8.1 | 28 | 2.2 | .001 | 37 | 4.9 | 14 | 2.1 | .003 |
| # of years since appointed to this position | 141 | 1.4 | 17 | 1.3 | 182 | 5.3 | 29 | 1.7 | .001 | 35 | 3.9 | 14 | 1.9 | .08 | |
8.1.1 Gender and Race
A majority of the staff in each state were female, with the proportion of male staff ranging from 11 percent in the Tennessee public agencies to 35 percent in the New Jersey family preservation agencies. Within each state, there were no significant differences between public and FPS agencies with respect to gender.
With respect to racial and ethnic composition, respondents from the New Jersey agencies, both public and private, had the greatest diversity. In New Jersey, Whites made up 42 percent of public agency staff and 54 percent of FPS agency staff, in comparison, Black made up 43 percent of public agency respondents and 25 percent of FPS agency respondents. New Jersey had the highest representation of Hispanic respondents (18% among FPS agencies and 11% among public agencies), and at least a small percentage of respondents were represented in all four race/ethnicity categories. A majority (73-78%) of the respondents in both public and FPS agencies in Kentucky were White, with the remainder primarily Black (22-25%) and just a few public agency workers reporting their ethnicity as Hispanic (1%) or "Other" (2%). In Tennessee, approximately three-quarters of respondents in public agencies were Black with the remainder being White, and approximately one-half of FPS agency respondents were Black, one-third were White, none Hispanic, and 13 percent reported their race/ethnicity as other than Black, White, or Hispanic.
8.1.2 Education
In all three of the states, FPS staff had higher levels of education (all significant at p £ .001). In all three states, comparing public agency and FPS staff, a smaller proportion of public agency staff held a degree at the Masters level or higher (18% vs. 89% in Kentucky, 14% vs. 52% in New Jersey, and 5% vs. 65% in Tennessee). A small percentage (7%; n = 14) of public agency staff in New Jersey and one public agency respondent in Tennessee reported an education level less than a Bachelor's degree. Of the fourteen respondents in New Jersey with less than a Bachelors degree, approximately half reported their positions as case manager. The other half of those New Jersey respondents as well as the one Tennessee respondent with less than a Bachelors degree reported their positions as caseworker.
There was a fair amount of variability in workers' responses to questions about the field of study for the highest degree they obtained - particularly among public agency staff. The majority of staff (ranging from 51% in Tennessee public agencies to 94% in Kentucky FPS agencies) indicated that their degree was in social work or social services. Other frequently named fields of study included education, law or criminal justice, and management or administration.
8.1.3 Caseloads and Time Allocation
As anticipated, when asked about usual caseload, public agency staff reported significantly higher numbers than FPS staff across all three states (see Table 8-2). Along the same lines, relative to public agency staff, FPS staff in each of the three states reported spending a significantly greater proportion of their time on family preservation services and a significantly smaller proportion of their time on investigations, placements, and followup services (again, see Table 8-2 for average proportions; all comparisons significant at p<= .001).
8.1.4 Job Satisfaction
Caseworkers were asked about their level of satisfaction with various aspects of their jobs, including salary, workload, supervision, work difficulty, chances of promotion, and fringe benefits. Responses to these items were also combined for an average measure of job satisfaction. In all three states, public agency workers were significantly less satisfied than FPS staff with respect to job satisfaction overall and with their workload and work difficulty in particular. Furthermore, in Kentucky, public agency staff were also significantly less satisfied than FPS staff when it came to salary; in New Jersey, public agency staff were also significantly less satisfied than FPS staff with supervision; and in Tennessee, public agency staff were also significantly less satisfied with supervision and differences were marginally significant with respect to salary (p = .07).
8.1.5 Staff Views on Child Welfare and Family Service Issues
Caseworkers were asked to indicate their level of agreement or disagreement in response to a series of 10 statements expressing views on a number of child welfare and family service issues. For many of the statements, respondents' average levels of agreement were near the mid-point of the scale (2.5, where 1 = strong agreement and 4 = strong disagreement) indicating neither strong agreement nor disagreement on average. For two of the statements - both relating to risk - the average responses for both FPS and public agency staff in all three states were skewed more towards strong agreement. Those statements were: "It is never justified to take chances with the lives of children" and "If a child I left at home were seriously injured due to maltreatment, I would find it hard to forgive myself." In New Jersey, on the second of these statements, there was a significant difference between average responses of FPS and public agency staff. Although both groups indicated agreement, public agency staff indicated stronger agreement with this statement than did FPS staff (1.9 vs. 2.3; p = .05).
In two of the states, there was stronger disagreement by FPS staff (relative to public agency staff) in response to the statement "Child abuse and neglect are social problems driven by strong social forces to the extent that social work services cannot do much to save children from danger." Differences between FPS and public agency staff in response to this statement were significant in Kentucky (3.5 vs. 3.1; p = .05) and New Jersey (3.7 vs. 2.8; p = .001), with small, non-significant differences in Tennessee (3.7 vs. 3.6).
On the issue of referrals to family preservation services, FPS staff in all three states indicated a greater degree of agreement than public agency staff with the statement: "Only families with a child at imminent risk of placement should be referred to intensive family preservation services." These differences were statistically significant in Kentucky (1.5 for FPS and 2.7 for public staff; p = .001) and in New Jersey (2.2 for FPS and 2.8 for public staff; p = .001), and differences were in the same direction but not statistically significant in Tennessee (2.3 for FPS and 3.1 for public staff; p = .08).
In New Jersey, there were two other statements for which there were significant differences in average responses for FPS and public agency staff. Public agency staff indicated stronger disagreement than FPS staff with the statement "Removing a child from his or her parents can be so deep a trauma to the child that it is almost always worth taking the risk to leave the child with his or her parents" (3.3 for public staff and 3.0 for FPS; p = .04). On the other hand, FPS staff indicated stronger disagreement on average than public staff with "Families who deny the truth of a validated allegation of abuse or neglect are such poor prospects for service that placement is usually justified" (3.1 for FPS and 2.6 for public staff; p = .01).
8.1.6 Reactions to Work with Clients
Caseworkers were given seven statements, six of which were drawn from the Maslach Burnout Inventory (Maslach and Jackson, 1986), an instrument designed to measure the following three dimensions of the psychological syndrome of burnout: emotional exhaustion, lack of a sense of personal accomplishment, and depersonalization of clients. For each statement, workers were asked to indicate their agreement on a 4 point scale (1 = strongly agree, 4 = strongly disagree, 2.5 = midpoint).
Three of the statements were intended to assess workers' depersonalization of clients: "I feel I treat some of my clients as impersonal objects," "I have become more callous toward people since being on this job," and "I think clients often blame me for their problems." On average, family preservation workers in all three states disagreed or strongly disagreed with these statements, with mean scores on individual items ranging from 3.5 to 3.8 in Kentucky, 3.8 to 3.9 in New Jersey, and 3.3 to 3.9 in Tennessee. Public agency workers also tended to disagree with these statements, however, their average scores were closer to the midpoint of the scale, particularly on the item "I think clients often blame me for their problems" where mean scores were 2.3 in Kentucky, 2.6 in New Jersey, and 2.9 in Tennessee. Comparing the FPS and public staff responses, FPS workers in all three states indicated significantly stronger disagreement on the item "I have become more callous toward people since being on this job."
Two of the statements were intended to assess workers' sense of personal accomplishment: "I deal very effectively with the problems of my clients," and "I have accomplished much that is worthwhile in this job." Both FPS and public agency workers in all three states indicated some level of agreement with each of these statements as average item scores ranged from 1.4 to 1.8. Differences between FPS and public agency staff in their level of agreement were marginally significant in Kentucky, with FPS workers indicating stronger agreement.
Only one item assessing emotional exhaustion was included in this questionnaire, and that item was a direct statement of burnout: "I feel burned out from my work." In Kentucky and New Jersey, both FPS and public agency workers, on average, indicated disagreement with this statement (although the average score for public agency workers in New Jersey was exactly at the midpoint). In New Jersey, FPS workers indicated significantly stronger disagreement than public agency workers (3.3 vs. 2.5; p = .001). In Tennessee, the average score on this item for public agency workers indicated moderate agreement while the average score for FPS workers was close to the midpoint, a non-significant difference (2.1 vs. 2.4).
The final item regarding workers' reactions to work with clients - one not drawn from the Maslach Burnout Inventory - was "Many clients cannot be helped no matter what I do." While all workers, on average, disagreed with this statement, FPS workers indicated significantly stronger disagreement than public agency workers in all three states.
8.1.7 Preparation in Dealing with Casework Issues
Workers were asked how prepared they felt to deal with 10 specific casework issues: assessing problems, assessing risk, case planning, assessing family functioning, assessing child functioning, family systems, building client relationships, counseling families, permanency planning, and knowing when to terminate a case. For each statement, workers were asked to indicate their agreement on a 5 point scale (1 = very well prepared, 5 = poorly prepared, 3 = midpoint). Within each of the states and groupings for type of staff, of all activities, preparation for permanency planning was thought to be worst - although, on average, all staff reported their preparation level as better than the mid-point of the scale. Within each state and type of staff (FPS or public), respondents thought themselves best prepared for "building client relationships." On average, Kentucky FPS staff reported being significantly better prepared than public agency staff on six of the ten casework issues (assessing problems, assessing child functioning, family systems, building client relationships, counseling families, and knowing when to terminate a case). In New Jersey, FPS staff reported being significantly better prepared than public agency staff on two of the ten casework issues (building client relationships and counseling families). In Tennessee, FPS staff reported being significantly better prepared than public agency staff on three of the ten casework issues (assessing family functioning, counseling families, and knowing when to terminate a case). On none of the items did public agency staff report feeling more prepared than FPS staff. When responses to all ten items were combined for an overall measure of how prepared workers felt, average scores were better than the mid-point of the scale, with FPS staff feeling significantly more prepared than public agency staff in Kentucky (2.1 vs. 1.6; p = .001) and marginally significant differences in Tennessee (1.7 for FPS staff and 2.0 for public agency staff; p = .06).
8.1.8 Experience, Tenure, and Future Employment Plans
When asked how many years of experience workers had in various kinds of social work, there was a considerable amount of variability both between states and between FPS and public agency staff. Looking at social work in general, Kentucky staff averaged around 4-5 years with no significant differences between FPS and public agency staff; New Jersey FPS staff averaged 6.2 years and New Jersey public agency staff averaged 12.2 years - a statistically significant difference (p = .001); Tennessee staff averaged 7-8 years of experience with no significant differences between FPS and public agency staff.
The average amount of experience in child welfare social work was 3 to 5 years for Kentucky and Tennessee staff, with no significant differences between FPS and public agency workers. In New Jersey, FPS workers averaged 4.6 years experience in child welfare and public agency staff averaged 9.5 years - again, a statistically significant difference (p = .001).
On average, staff reported being in their current position for over a year, with public agency staff having spent a greater amount of time in their current positions than FPS staff in both New Jersey (5.3 vs. 1.7; p = .001) and Tennessee (3.9 vs. 1.9; p = .08). Particularly among public agency staff, the average number of years working in the agency was consistently higher than the average number of years working in their current positions (3.7 vs. 1.4 in Kentucky; 8.1 vs. 5.3 in New Jersey; 4.9 vs. 3.9 in Tennessee). This may be indicative of the amount of turnover that occurs among positions but within the public agencies.
Although no effort was made to track the number of workers who left the agency during the timeframe of this study, workers were asked about their future employment plans. In each of the three states, a higher proportion of public agency staff reported that they were either "looking for other employment" or "have definite arrangements to take another job" (17% vs. 12% in Kentucky; 15% vs. 0% in New Jersey; p = .03 (Fisher's exact); and 27% vs. 19% in Tennessee).
8.1.9 Summary for Kentucky, New Jersey, and Tennessee Staff
In summary, the results of this questionnaire confirm anticipated differences between FPS and public agency staff in caseload size and allocation of time. There is also evidence of some significant differences between FPS and public agency staff on level of education, job satisfaction, how prepared they feel, amount of experience, and future employment plans. However, despite the fact that the FPS staff appear to have higher qualifications in some areas and are more satisfied with their jobs, no differences in staff qualifications and attitudes translate into differences in practice and thus differentially affect client outcomes.
There are a number of possible explanations for the apparent lack of a direct link between staff qualifications and attitudes and client outcomes. Inadequate measurement of the outcomes is one possibility. However, many of the outcome measures detected change over time, just not differential change fore the FPS and regular service groups. Availability of recommended services is another possible problem. FPS staff may develop more individualized or comprehensive case plans that help clients achieve desired outcomes, however, those case plans may not be implemented if the services are not available at that time. Another disconnect between staff characteristics and outcomes may occur if case plans are not implemented as a result of different philosophies or service approaches taken by the FPS worker and the public agency caseworker who resumes management of the case at the end of the family preservation program. Lastly, the brevity of the family preservation program may cancel out any advantages due to superior preparation, attitudes, or job satisfaction of FPS workers.
8.2 Philadelphia Staff
All respondents from Philadelphia (n = 151) were employed by private agencies, and some workers were responsible for providing services to both FPS and non-FPS cases. Therefore, respondents were not categorized according to FPS or public agency status. Results are summarized in Table 8-3.
8.2.1 Gender and Race
Relative to the other three states, a fairly large proportion of the Philadelphia staff were male (40%). A majority of the staff were Black (61%) with another third White and a small percentage Hispanic (4%) or Other (3%).
8.2.2 Education
A majority of the staff (56%) held Bachelors degrees, slightly less than a fifth had completed some graduate study, and approximately one-quarter had a degree at the Masters level or higher. The most common areas for the field of study were social services (29%), social work (25%), and law or criminal justice (13%).
| N | % | |
|---|---|---|
| Respondent is male | 139 | 40 |
| Race/Ethnicity | 132 | |
| Black | 61 | |
| White | 33 | |
| Hispanic | 4 | |
| Other | 3 | |
| Participated in on the job training in past 12 months | 136 | 82 |
| Attended seminar or workshop in past 12 months | 134 | 96 |
| Taken for-credit courses in a degree program in past 12 months | 130 | 22 |
| Taken non-credit course on a work-related topic in past 12 months | 129 | 41 |
| Participated in other in-service training in past 12 months | 133 | 87 |
| Field of Study | 139 | |
| Education | 10 | |
| Law/criminal justice | 13 | |
| Math/science | 3 | |
| Mgmt/administration | 3 | |
| Other humanities | 10 | |
| Social services | 29 | |
| Social work | 25 | |
| No code/NA/missing | 8 | |
| Level of education | 137 | |
| AA degree, high school diploma, GED, or less | 1 | |
| Bachelors degree | 56 | |
| Some graduate study | 18 | |
| Masters degree or beyond (doctoral) | 26 | |
| Future employment plans | 129 | |
| Do not plan to look for other employment | 63 | |
| Would like to change jobs but not actively looking | 24 | |
| Looking for other employment but have not found anything | 10 | |
| Have definite arrangements to take another job | 3 | |
| N | Mean | |
| Usual caseload size | 136 | 11.3 |
| Largest # of families in caseload | 138 | 15.2 |
| % of time spent investigating abuse/neglect complaints | 122 | 45.1 |
| % time spent on family preservation services | 117 | 25.6 |
| % time spent on foster care placement and/or supervision | 115 | 7.8 |
| % time spent on followup services for abuse/neglect families | 121 | 25.5 |
| How satisfied are you with various aspects of your job? (1 = highly satisfied, 5 = highly dissatisfied) | ||
|---|---|---|
| Salary | 130 | 3.2 |
| Workload | 130 | 3.0 |
| Supervision | 128 | 2.5 |
| Work difficulty | 130 | 2.8 |
| Chances of promotion | 129 | 3.1 |
| Fringe benefits | 130 | 2.7 |
| Overall job satisfaction (all 6 items) | 131 | 2.9 |
| Indicate how much you agree/disagree with the following statements: [1 =strongly agree, 4=strongly disagree] | ||
| Abuse and neglect are social problemsservices cannot do much to save children from danger | 139 | 3.7 |
| No matter how bad a natural family is, foster care is usually worse | 139 | 3.7 |
| There are many cases in which children remain at home and would have been better off in a good foster home | 139 | 2.8 |
| It is never justified to take chances with the lives of children | 139 | 2.1 |
| If a child I left at home were seriously injured due to maltreatment, I would find it hard to forgive myself | 139 | 2.5 |
| Removing a child from his or her parents can be so deep a trauma to the child that it is almost always worth taking the risk to leave the child with his or her parents | 139 | 3.7 |
| Families who deny the truth of a validated allegation of abuse or neglect are such poor prospects for service that placement is usually justified | 139 | 3.4 |
| Most families with records of several past complaints should not be given any more chances to change | 139 | 3.5 |
| Placement prevention should be the primary goal of family preservation programs | 139 | 2.4 |
| Only families with a child at imminent risk of placement should be referred to intensive family preservation services | 139 | 3.2 |
| N | Mean | |
|---|---|---|
| Indicate how much you agree/disagree with the following statements about your reactions to work with clients: (1 = strongly agree, 4 = strongly disagree) | ||
| I feel I treat some of my clients as impersonal objects | 139 | 3.8 |
| I deal very effectively with the problems of my clients | 139 | 1.6 |
| I have become more callous toward people since being on this job | 139 | 3.4 |
| Many clients cannot be helped no matter what I do | 139 | 3.0 |
| I think clients often blame me for their problems | 139 | 3.1 |
| I have accomplished much that is worthwhile in this job | 139 | 1.8 |
| I feel burned out from my work | 139 | 2.8 |
| How prepared do you feel to deal with each of the following casework issues (1 = very well prepared, 5 = poorly prepared) | ||
| Assessing problems | 137 | 1.7 |
| Assessing risk | 137 | 1.8 |
| Case planning | 135 | 2.1 |
| Assessing family functioning | 136 | 1.9 |
| Assessing child functioning | 137 | 1.9 |
| Family systems | 134 | 2.1 |
| Building client relationships | 138 | 1.7 |
| Counseling families | 136 | 1.9 |
| Permanency planning | 126 | 2.5 |
| Knowing when to terminate a case | 130 | 2.1 |
| Combined measure of how prepared respondent feels (all 10 items) | 137 | 2.0 |
| How many years of experience have you had in the following kinds of work: | ||
| Social work in general | 133 | 7.4 |
| Child welfare social work | 120 | 4.9 |
| Supervising others in social work | 107 | 1.2 |
| Delivering family preservation services | 106 | 1.1 |
| # of years working for this agency | 93 | 2.9 |
| # of years since appointed to this position | 90 | 2.0 |
8.2.3 Caseload and Time Allocation
On average, workers reported a usual caseload size of 11.3 cases, with a large portion of their time (45%) spent investigating abuse or neglect complaints. An additional quarter of their time was spent on each of the following tasks: family preservation services and followup services, and a small proportion of their time (7.8%) was spent on foster care placement or supervision.
8.2.4 Job Satisfaction
Caseworkers were asked about their level of satisfaction with various aspects of their jobs, including salary, workload, supervision, work difficulty, chances of promotion, and fringe benefits. Combining these items for an average measure of job satisfaction, Philadelphia staff responses came out almost exactly at the midpoint of the scale (mean = 2.9 on a scale of 1 to 5). Staff were slightly more satisfied with supervision (mean = 2.5), fringe benefits (mean = 2.7), and work difficulty (mean = 2.8), and they were slightly less satisfied with chances of promotion (mean = 3.1) and salary (mean = 3.2).
8.2.5 Views on Child Welfare and Family Service Issues
In response to 10 statements expressing views on child welfare and family service issues, responses from Philadelphia staff were neutral or in disagreement with all but one of the statements. There was slight agreement (mean = 2.1) with the statement "It is never justified to take changes with the lives of children." Responses were relatively neutral on statements of whether workers would forgive themselves if a child were injured (mean = 2.5) and whether placement prevention should be the primary goal of FPS programs (mean = 2.4). Respondents expressed relatively strong disagreement on four statements containing negative views of services or families (see Table 8-3).
8.2.6 Reactions to Work with Clients
On average, workers expressed disagreement with all three statements assessing depersonalization: treating clients as impersonal objects (mean = 3.8), becoming callous toward people (mean = 3.4), and thinking clients blame them for problems (mean = 3.1). On the items measuring workers' sense of personal accomplishment (dealing effectively with clients, and accomplishing much that is worthwhile), average responses were between neutral and agreement. Staff in Philadelphia disagreed only slightly, on average, with the statement "I feel burned out from my work" (mean = 2.8).
8.2.7 Preparation in Dealing with Casework Issues
Similar to results in Kentucky, New Jersey, and Tennessee, staff in Philadelphia reported feeling worst prepared for dealing with issues of permanency planning. On the other hand, staff in Philadelphia reported feeling best prepared to assess problems and build client relationships. An overall measure of the 10 items yielded an average response of 2.0 - just slightly on the "prepared" side of the scale.
8.2.8 Experience, Tenure, and Future Employment Plans
When asked about their experience in various kinds of social work, Philadelphia staff indicated an average of 7.4 years experience in social work in general, 4.9 years in child welfare social work, 1.2 years in supervising others in social work, and just over one year of experience delivering family preservation services.
On average, workers reported working for this agency for just under three years and being in their current positions for two years. Only a small proportion of respondents (3%) indicated that they had definite arrangements to take another job, and an additional ten percent said they were looking for other employment.
Kinney, J., Haapala, D. & Booth, C. (1991). Keeping Families Together: The Homebuilders Model. New York, NY: Aldine De Gruyter.
Maslach, C. & Jackson, S.E. (1986). Maslach Burnout Inventory (2nd ed.). Palo Alto, CA: Consulting Psychologists Press.
108. If respondents reported that they both carry cases and have supervisory responsibilities, they were only considered case level staff if they reported that they spend no more than 49 percent of their time in supervision.
109. Respondents were considered "case level staff" if they reported that they carry cases. If they reported that they carry cases and have supervisory responsibilities then they were only considered case level staff if they reported that they spend no more than 49 percent of their time in supervision.
Note: p-values are only reported when they are less than or equal to .10
110. Respondents were considered "case level staff" if they reported that they carry cases. If they reported that they carry cases and have supervisory responsibilities then they were only considered case level staff if they reported that they spend no more than 49 percent of their time in supervision.
9. Conclusions
In the late 1980s and early 1990s, family preservation programs became a popular response of states to rising rates of foster care placement of children. It was commonly assumed that many children were unnecessarily removed from their parents and that intensive services could prevent those placements while protecting children from harm. Early evaluations suggested these programs had considerable promise but these studies were criticized for flaws in research design. Later, more rigorously designed studies began to cast doubt on the extensive claims of success. The largest of these studies were in California, New Jersey, and Illinois. No placement prevention effects were found in California and Illinois, while the study in New Jersey found short term effects that dissipated with time. (111) However, these studies were also criticized, most notably for not having examined those programs thought to be most likely to be effective.
This evaluation of family preservation programs was designed to overcome shortcomings of previous studies. It assessed the extent to which key goals of the programs are being met: the goals of reduction of foster care placement, maintaining the safety of children, and improving family functioning. (112) It studied the Homebuilders model of service, thought by many to be the most promising, and it looked at a program in Philadelphia that used a different approach and focused on a particular population, families in which substance abuse was present.
The design for this evaluation was an experiment in which families were randomly assigned to either a family preservation program (the experimental group) or to other, "regular" services of the child welfare system (the control group). Families were followed for over a year after random assignment. Data collection involved multiple interviews with caretakers and caseworkers and examination of administrative data on placements, reports of maltreatment, and case openings and closings.
9.1 Outcomes
9.1.1 Placement
We are unable to conclude that the family preservation programs in these states achieve the objective of reducing placement of children in foster care. (113) A summary of various analyses of placement rates at various points in time following random assignment is shown in Table 9-1. In three of the sites (Kentucky, Tennessee, and Philadelphia) there were no significant differences in placement rates over time for the samples as they were originally randomly assigned (the "primary" analysis). In New Jersey, placement rates were significantly higher in the experimental group. Since some of the families in the control group were actually provided family preservation services ("violations") and some of the families in the experimental group did not receive services or received only minimal services ("minimal service" cases), we also conducted analyses in which we dropped those cases ("secondary" analyses). Results of the secondary analyses were quite similar to the primary analyses.
It was thought that the samples in Kentucky, New Jersey, and Tennessee included families that did not fit the conception of cases best suited for the program model, that is, cases in which there is an imminent risk of placement. (114) Hence, we attempted to identify subgroups that might better fit criteria for referral. This selection was based on the idea that the service is most useful for families in crisis. Hence, we focused on cases referred in the course of an investigation of abuse or neglect and cases with recent substantiated allegations of maltreatment, on the grounds that these groups of cases might reflect families in crisis. These "refined groups" analyses also failed to show differences between the experimental and control groups on placement rates over time.
In Kentucky and Tennessee, we obtained data from case records and caseworkers on placements with relatives that were not recorded in the administrative data. Adding those data to our analyses, there were again no differences between experimental groups. Although not statistically significant, some of the differences between groups appear to be fairly substantial, particularly at the one-year point. However, there is no consistent pattern to these differences, sometimes the experimental group percentage is higher, sometimes it is the other way around.
| Kentucky | 1 month | 6 months | 12 months | 18 months | ||||
|---|---|---|---|---|---|---|---|---|
| E % | C % | E % | C % | E % | C % | E % | C % | |
| Primary | 6 | 5 | 18 | 18 | 25 | 24 | 27 | 27 |
| Secondary | 4 | 4 | 12 | 18 | 20 | 23 | 24 | 25 |
| Refined analyses | ||||||||
| Investigative | 8 | 5 | 15 | 14 | 26 | 15 | 28 | 20 |
| Recent substant. | 6 | 2 | 20 | 11 | 29 | 13 | 32 | 18 |
| Petition cases | 6 | 9 | 16 | 14 | 22 | 29 | 25 | 32 |
| New Jersey | 1 month | 6 months | 12 months | 18 months | ||||
|---|---|---|---|---|---|---|---|---|
| E % | C % | E % | C % | E % | C % | E % | C % | |
| Primary | 5 | 6 | 19 | 17 | 29 | 22 | 35 | 26 |
| Secondary | 3 | 6 | 17 | 17 | 27 | 23 | 34 | 27 |
| Refined analyses | ||||||||
| Investigative | 3 | 5 | 16 | 12 | 25 | 15 | 32 | 19 |
| Recent substant. | 8 | 5 | 19 | 12 | 25 | 14 | 33 | 21 |
| Tennessee | 1 month | 6 months | 12 months | |||
|---|---|---|---|---|---|---|
| E % | C % | E % | C % | E % | C % | |
| Administrative data, primary analysis | 11 | 11 | 22 | 19 | 23 | 19 |
| Administrative data, secondary analysis | 7 | 12 | 18 | 19 | 19 | 19 |
| Including relatives, primary | 11 | 11 | 26 | 21 | 28 | 23 |
| Including relatives, secondary | 7 | 12 | 20 | 19 | 23 | 21 |
| Refined analyses | ||||||
| Recent investigation, CORS | 7 | 12 | 15 | 15 | 17 | 15 |
| Recent investigation, includes Relative | 7 | 12 | 18 | 18 | 22 | 21 |
| Philadelphia | 1 month | 6 months | 12 months | 18 months | ||||
|---|---|---|---|---|---|---|---|---|
| E % | C % | E % | C % | E % | C % | E % | C % | |
| Primary | 1 | 1 | 10 | 12 | 18 | 15 | 24 | 20 |
| Secondary | 1 | 1 | 9 | 13 | 15 | 16 | 21 | 19 |
| Note: C = Control Group, E = Experimental Group | ||||||||
As indicated above, the target group for the services in Kentucky, New Jersey, and Tennessee was families in which at least one child was "in imminent risk of placement." We found that, by and large, the families served were not in that target group. This is shown by the placement rate within a short period of time in the control group, indicating the placement experience in the absence of family preservation services. In all three states, the placement rate in the control group within one month (a liberal definition of "imminent") was quite low. It would, therefore, have been virtually impossible for the programs to be effective in preventing imminent placement, since very few families would have experienced placement within a month without family preservation services. (115) It should be noted, however, that the rates of eventual placement in the control group were higher, about one-fifth to one-fourth within one year. Hence, it would have been possible for family preservation to have shown effects on placement over time, but those effects were not observed.
There was one group that it seemed might represent better targeting, the "petition" cases in Kentucky. Prior to random assignment, workers submitted petitions to the court for placement or some other court ordered intervention on 67 families. It might be supposed that this group would be more likely to have children placed. Although more of the control group families in this group experienced the placement of a child within one month than other subgroups in Kentucky, that proportion was still quite low (10%), suggesting that focusing on groups such as this (cases with court involvement) would not resolve the targeting problem. (116)
9.1.2 Child Safety
In general, the rates of substantiated allegations of abuse or neglect were quite low. In most of our analyses, there was little difference between the family preservation and control groups in the incidence of reports of maltreatment subsequent to random assignment. An exception was the group of cases in Tennessee with prior allegations of harm within 30 days before random assignment. For this set of families, the control group had a significantly higher rate of subsequent substantiated allegations.
The findings of little difference between the experimental and control group can be read in two ways. It indicates that families served by family preservation were no more likely than families not receiving the service to be subjects of allegations of harm. In this sense, children were, by and large, kept safely at home while receiving family preservation services. However, children in both groups were primarily in their homes, and family preservation did not result in lower incidence of maltreatment compared with children in the control group.
9.1.3 Subgroups
In Kentucky, New Jersey, and Philadelphia we examined a number of subgroups of families to determine whether we could detect differences between experimental and control groups on placement and substantiated allegations subsequent to random assignment within each subgroup. Most of the subgroups were defined in terms of problems of the family, for example, substance abuse, financial difficulties, and depression. The number of cases in Tennessee was not sufficient to support subgroup analysis. No subgroups were found in which there were positive effects of family preservation services on placement or subsequent maltreatment. Hence, the effort to find subgroups for which family preservation service was successful in reducing placement was not successful.
9.1.4 Case Closing and Subsequent Reopening
There were no significant differences in case closing rates between the experimental and control group in New Jersey or Tennessee. There was a significant difference in Kentucky. Experimental group cases closed significantly quicker than control group cases. Regarding case reopenings, there were no significant differences between the experimental and control groups in Kentucky or New Jersey. In Tennessee, significantly more of the closed control group cases reopened (9 of 30 or 30%, compared to 8 of 66 or 12% of the experimental group).
9.1.5 Family and Child Functioning
We interviewed caretakers at three points in time, shortly after the beginning of service (the "initial" interview), four to six weeks later (at the end of service for families receiving family preservation services, called the "post-treatment interview"), and again a year after services began (the "followup interview"). Caseworkers for both experimental and control group families were interviewed at the first two of these points in time. In these interviews, we examined a number of areas of family and child functioning that might have been affected by family preservation services. We looked at both levels of functioning at post treatment and followup and changes over time in levels of functioning. We examined responses to some of the individual items in the interviews, and we combined responses into various scales measuring dimensions of functioning. The following are the areas examined.
Caretaker interview:
- Life events. An inventory of recent positive and negative life events was used to construct three scales: positive life events, negative life events, and depression.
- Problems. Nine items, examined individually.
- Economic functioning. Four items on difficulty in paying for rent, electricity and heat, food, and clothes were examined individually and combined in a scale.
- Household condition. Ten items, examined individually and combined in a scale.
- Child care practices. Fifteen items, examined individually and in three scales: positive child care practices, negative child care practices, and punishment.
- Caretaker depression. Scores on the SCL-90 depression scale.
- Child behavior. Thirty-five questions comprising scales for aggression, school problems, positive child behaviors, and negative child behaviors.
- Overall assessment of improvement. A single question.
Caseworker interview:
- Caretaker functioning. Nine five-point scale questions, examined individually and averaged.
- Household condition. Thirteen questions combined in a scale.
- Caretaker problems. Twenty-one questions combined in a scale.
- Child problems. Twelve questions combined in a scale.
The results of the measures of functioning are summarized in Tables 9-2 and 9-3. In a few of these areas of functioning, in one or the other of the states, families in the experimental group appeared to be doing better post-treatment. There were very few differences at the year followup and in changes over time. Those differences that did appear (primarily at post-
| Area | Post treatment | Followup | Change over time |
|---|---|---|---|
| Life events | |||
| Positive life events | KY: Ø NJ: Ø TN: Ø PA: fewer experimentals experienced positive life events | KY: Ø NJ: Ø TN: Ø PA: Ø | KY: Ø NJ: Ø TN: Ø PA: Ø |
| Negative life events | KY: Ø NJ: Ø TN: Ø PA: Ø | KY: Ø NJ: Ø TN: Ø PA: Ø | KY: Ø NJ: Ø TN: Ø PA: Ø |
| Depression | KY: Ø NJ: Ø TN: Ø PA: Ø | KY: Ø NJ: Ø TN: Ø PA: Ø | KY: Ø NJ: Ø TN: Ø PA: Ø |
| Family problems, individual items | KY: Ø NJ: fewer experimentals not enough money for food, rent, or clothing TN: fewer experimentals had few or no friends PA: Ø | KY: Ø NJ: Ø TN: Ø PA: Ø | N/A |
| Economic functioning | |||
| Individual items | KY: Ø NJ: fewer experimentals difficulty paying rent and buying clothes TN: Ø PA: Ø | KY: Ø NJ: Ø TN: fewer experimentals difficulty paying rent PA: more experimentals having difficulty buying food and clothes | N/A |
| Scale | KY: Ø NJ: experimental average lower (better) TN: Ø PA: Ø | KY: Ø NJ: Ø TN: Ø PA: Ø | KY: Ø NJ: Ø TN: Ø PA: Ø |
| Ø denotes that differences between groups were not significant at p <= .05; N/A denotes not applicable | |||
| Area | Post treatment | Followup (one year after beginning of treatment) | Change over time |
|---|---|---|---|
| Household condition | |||
| Individual items | KY: experimentals fewer broken windows or doors NJ: Ø TN: more experimentals in unsafe building because of illegal acts PA: Ø | KY: Ø NJ: Ø TN: Ø PA: more experimentals reporting not enough basic necessities | N/A |
| Scale | KY: Ø NJ: Ø TN: Ø PA: Ø | KY: no significant differences NJ: no significant differences TN: no significant differences PA: experimental group reporting more problems in household condition | KY: Ø NJ: Ø TN: Ø PA: Ø |
| Child care practices | |||
| Individual items | KY: fewer experimentals used punishment for not finishing food NJ: experimentals less often got out of control when punishing child and more often encouraged child to read a book TN: more experimentals went to amusement park, pool, or picnic PA: Ø | KY: Ø NJ: Ø TN: Ø PA: Ø | |
| Positive scale | KY: Ø NJ: Ø TN: Ø PA: Ø | KY: Ø NJ: Ø TN: Ø PA: Ø | KY: Ø NJ: Ø TN: Ø PA: Ø |
| Negative scale | KY: Ø NJ: experimentals lower (better) TN: Ø PA: Ø | KY: Ø NJ: Ø TN: Ø PA: Ø | KY: Ø NJ: Ø TN: Ø PA: Ø |
| Punishment | KY: Ø NJ: experimentals lower (better) TN: Ø PA: Ø | KY: Ø NJ: Ø TN: Ø PA: Ø | KY: Ø NJ: Ø TN: Ø PA: Ø |
| Ø denotes that differences between groups were not significant at p <= .05; N/A denotes not applicable | |||
| Area | Post treatment | Followup (one year after beginning of treatment) | Change over time |
|---|---|---|---|
| Caretaker depression | KY: Ø NJ: Ø TN: Ø PA: Ø | KY: Ø NJ: Ø TN: Ø PA: Ø | KY: Ø NJ: Ø TN: Ø PA: Ø |
| Child behavior | |||
| Aggression | KY: Ø NJ: Ø TN: Ø PA: Ø | KY: Ø NJ: Ø TN: Ø PA: Ø | KY: Ø NJ: Ø TN: Ø PA: Ø |
| School problems | KY: Ø NJ: Ø TN: Ø PA: Ø | KY: Ø NJ: Ø TN: Ø PA: Ø | KY: Ø NJ: Ø TN: Ø PA: Ø |
| Positive child behaviors | KY: Ø NJ: Ø TN: Ø PA: Ø | KY: Ø NJ: Ø TN: Ø PA: Ø | KY: Ø NJ: Ø TN: Ø PA: Ø |
| Negative child behaviors | KY: Ø NJ: experimental group lower (better) TN: Ø PA: Ø | KY: Ø NJ: Ø TN: Ø PA: Ø | KY: Ø NJ: Ø TN: Ø PA: Ø |
| Overall assessment of improvement | KY: experimentals, greater improvement NJ: experimentals, greater improvement TN: Ø PA: Ø | KY: Ø NJ: Ø TN: Ø PA: Ø | N/A |
| Ø denotes that differences between groups were not significant at p<= .05; N/A denotes not applicable | |||
| Area | Post treatment | Change over time |
|---|---|---|
| Caseworker report of caretaker functioning (9 items) | ||
| Individual items | KY: Ø NJ: control group higher (better) in ability in giving affection and providing learning opportunities TN: experimental group higher (better) on five items PA: Ø | KY: respecting child's opinions: experimental group declined, control group increased NJ: control group had more positive change in respecting child's opinions TN: experimental group more positive change on setting firm and consistent limits PA: Ø |
| Scale | KY: Ø NJ: Ø TN: experimental group higher (better) PA: Ø | KY: Ø NJ: Ø TN: Ø PA: Ø |
| Caseworker report of household condition Scale (13 items) | KY: control group better NJ: control group better TN: Ø PA: control group worse | KY: Ø NJ: Ø TN: Ø PA: Ø |
| Caseworker report of caretaker problems Scale (21 items) | KY: experimentals more problems NJ: Ø TN: Ø PA: Ø | KY: Ø NJ: Ø TN: experimentals declined more PA: Ø |
| Caseworker report of child problems Scale (12 items) | KY: Ø NJ: Ø TN: Ø PA: Ø | KY: Ø NJ: Ø TN: Ø PA: Ø |
| Ø denotes that differences between groups were not significant at p <= .05. | ||
treatment) were not consistent across states and were not maintained. At best, it can be said that family preservation services may have small, apparently short-term, effects on some areas of functioning. There was one item with some consistency, the overall assessment of improvement by caretakers. At post treatment, in Kentucky and New Jersey, a significantly larger proportion of experimental group caretakers generally thought there was "great improvement" in their lives. This difference was significant in both the primary and secondary analyses. In the Tennessee secondary analysis, results tended in the same direction, though not significantly (p = .09). At followup, differences between the groups in Kentucky and New Jersey had nearly disappeared. In Tennessee at followup, control group respondents more often thought there was "great improvement" (p = .055).
9.2 Targeting
The findings of no effects of family preservation programs on placement rates and of problems in targeting these programs are not new, they have been observed in a number of rigorously designed experiments. (117) Partially as a result of these previous findings, efforts were made in this project to improve targeting. In New Jersey and Kentucky a screening instrument developed by the evaluators was employed to encourage referral of cases with a risk of imminent placement and to discourage referral of cases not at risk of placement. It is evident that this effort did not work; evidently, the screening instrument was a weak "intervention" in the problem of targeting.
Clearly, referring agents sent families to the programs that did not fit the criterion of imminent risk of placement. Our interviews with referring workers, discussed in earlier chapters, reveal some of the reasons. Workers acknowledged that they often did not refer cases that were at risk of placement, rather they used the programs for families that they thought could benefit from them. Evidently, they believed that in cases where placement was needed, family preservation services were not appropriate, contrary to the assumptions of the designers of these programs. But the programs were valued, and they were used to help families in the context of a generally service-poor child welfare system.
There are other possible explanations for the low placement rate in the control group. It is possible that in cases assigned to the control group, workers on those cases exerted efforts to prevent placement of the child. Placement prevention as a central value may pervade the system (perhaps more during the time we were collecting these data than now, it is possible that the Adoption and Safe Families Act has shifted emphasis away from this value). Of course, in this regard the philosophy of family preservation seems to have been widely adopted, even though rigorous evaluations have not shown placement prevention effects of its services.
But there are still other aspects of the targeting problem. Homebuilders has developed into a quite generalist program, used in a wide variety of cases. In Kentucky and New Jersey there is considerable heterogeneity in the cases referred to these services, in both characteristics and problems of families and in where the case is in the child welfare system. Families come from both the investigative and on-going phases of cases. (118) It seems likely that many of those referred from on-going caseloads are not referred because of likelihood of placement but because the case is not going well and everything else has been tried. (119) Families do not always appear to be in crisis, another important criterion for referral. Furthermore, a number of cases do not involve abuse or neglect, but rather are cases of child dependency or of parent-adolescent conflict. And the cases involve a wide range of ages of children at risk. It could be argued that this variation is detrimental to the development of programs. No one program can expect to be successful in all cases. Having such variation inevitably results in a lack of focus and prevents the development of specialized expertise in handling particular cases. The lack of focus and expertise is likely to affect the outcomes that can be expected. Furthermore, the variation in the character of cases must contribute to variations in outcomes.
A natural response to this state of affairs is that we must tighten up the targeting, demanding strict adherence to referral criteria. Our attempts to assist states to do this were clearly unsuccessful. We suggest that it will be extremely difficult to achieve the goal of better targeting. There are a number of reasons for this skepticism. Referring workers acknowledged that they often referred families that were not at risk of placement, at least not those at imminent risk of placement. We cannot fully explain why workers did not follow the rules for referrals, but we can propose some conjectures. Workers believe that they remove children from the home only when that is absolutely necessary, when no service can prevent placement. In this sense, one might conclude that family preservation values have come to pervade the system, there are few unnecessary placements, leaving few placements to be prevented with intensive services. (120) However, these services are valued by referring workers, they are responses to the needs of families (families other than those with children about to be placed), and services to meet those needs are scarce. Hence, family preservation programs are used for very real needs of families in the child welfare system.
Beyond this dynamic, there is the general tendency to expand the benefits of a good program. If a program is believed to be beneficial, it is often assumed that it will be useful for an ever-expanding range of cases. Evidently, this occurred in the states we studied. Expansion of the target group is aided by the fact that target group definitions usually have one or more vague terms that allow for the expression of discretion (e.g., most people's problems can be conceptualized as "crises").
Finally, our efforts to identify particular groups of families for which the programs are successful at preventing placement were mostly unfruitful. Hence we are unable to satisfy the demands of policy makers and practitioners for guidance on specific groups that might be targeted.
These circumstances, together with the fact that referrals to family preservation programs involve judgments that cannot be completely systematized or circumscribed, lead to our skepticism about the likelihood of improving targeting of these programs. Furthermore, it is possible that the programs are, by and large, being used in those circumstances for which they are best suited. (121)
9.3 Possible Alternative Explanations of the Findings
Positive findings of experimental evaluations provide evidence for the validity of a theory of intervention and confirm the effectiveness of a particular implementation of that theory. Null findings are more ambiguous, they do not necessarily disprove an intervention theory and may not even be evidence of ineffectiveness of implementation. One cannot be sure whether the results are due to problematic program conceptions, inadequate program implementation, unique contextual problems, or flawed evaluation procedures. The findings of this study will be questioned, as have those of the previous studies, for various supposed methodological and implementation shortcomings. We consider here some of the factors that might have affected the findings, beginning with problems in the implementation of the evaluation.
Violations of Experimental Assignment. In all three states, there were violations of experimental group assignment, that is, families assigned to the control group that were given family preservation services. This was particularly a problem in New Jersey, where 14 percent of the control group families received family preservation. The dictates of rigorous analysis required that we retain these cases in the control group (we also conducted "secondary" analyses in which we dropped these cases from analysis and there were few differences between our primary and secondary analyses). Violation cases could significantly affect the findings. For example, they could represent cases that would have experienced placement in the absence of the service. To the extent this was the case, the placement rate in the control group would be underestimated. This could affect the conclusions about both the effective targeting rate and experimental-control group differences in placement.
We attempted to examine the extent to which violations might have affected the results in New Jersey (there were too few violations in Kentucky and Tennessee to have significant effects). Even if all of the violations had been placed early on, the proportion of families in the control group experiencing placement would not have reached levels that one would consider close to adequate targeting. Sensitivity analysis in which all violations are assumed to be placed early suggests that under this extreme assumption there would have been differences in placement rates favoring the family preservation group early on but these differences dissipate over time. (122) Hence, at the very least, violations could not affect a conclusion that family preservation does not appear to prevent long placements of a year or more.
Inclusion of Minimal Service Cases in the Analysis. Some families in the experimental group did not receive family preservation services or received only small amounts of service. These cases were included in the primary analysis and it might be argued that this reduced the apparent effects of the service and that we should have eliminated these cases from analysis to produce a fair estimate of effects. We did drop these cases from our "secondary" analysis, and found few differences compared to the primary analysis. In addition, it should be observed that programs will always have minimal service cases, cases in which the family cannot be found, declines service, or otherwise refuses to cooperate. Retaining them in the analysis is appropriate in determining the average effects of the service over a group of cases thought to need the service. Theoretically, one might be able to reduce the size of the minimal service group through better targeting, but in practice, it is likely to be difficult to identify a substantial proportion of these cases prior to referral.
The "John Henry" Effect. The John Henry effect is reputed to be present in some experimental evaluations. This is the situation in which workers in control group cases exert special efforts on behalf of families, providing them far more service than would have been provided in normal circumstances (so the control group is not a "regular service" group). There are a couple of possible reasons this might occur. A worker might be unhappy with the experiment in general and with the assignment of this particular case to the control group in particular, and exert special effort in response. Alternatively, workers might feel the families assigned to the control group really need the experimental service, the prevention of placement is very important, so efforts are made to emulate Homebuilders. (This may be a special case of experimental leakage.)
In Kentucky and New Jersey, there is no evidence in the data on services to suggest this happened. Families in the experimental group did receive much more service than the control group. It is possible that the control group received more than "regular services." We cannot determine that. So it is possible that there is a threshold of services that has placement prevention effects and that was reached by the control group. If this were the case, it would indicate that the desired results can be obtained without intensive family preservation services.
In Tennessee, there is some evidence that families in the control group may have received as much, or perhaps more, service than the experimental group. This is seen in a specific set of questions asked of the caretakers about services received, and is not confirmed in other evidence regarding services provided to the two groups. Nonetheless, we cannot be as confident in Tennessee that experimental group families received much more service than the control group. Since the outcomes of the two groups were similar, this could again be taken as an indication that the results could be obtained without the family preservation services we studied.
Effects of the Experiment on the Nature of the Referred Group. It is possible that instituting the experiment caused a change in the character of cases referred to the program. In particular, agencies and workers were required to refer more cases in order to fill the control group as well as the experimental group. This resulted in dipping further into the pool of cases, perhaps taking "less severe" cases, those with less risk of placement. Anticipating this problem, we endeavored to select sites for the experiment in which demand considerably exceeded supply, however, we cannot be sure that we succeeded in this regard. It is also possible that workers referred different cases because of the chance that they would be assigned to the control group and not receive family preservation services. Or they may have changed referral practices to sabotage the research.
We cannot be sure that these factors were not present in referrals of families to the experiment, but we have no strong evidence that they were a strong influence. Operating against such dynamics were the desires of workers to provide significant services to families.
The Program Implementation was Flawed. The family preservation programs in Kentucky, New Jersey, and Tennessee claimed adherence to the Homebuilders model of service. However, it is possible that the implementation did not adequately follow that model, with the result that this evaluation was not a fair test of the model. We attempted to measure certain aspects of model adherence and found some variation from the prescribed ideal. One cannot expect any implementation of a model to adhere totally to it, adaptations must be made to local conditions, the character of individual cases, and to the styles of individual workers. Models of social service do not provide for the same response in all cases nor can they be used to prescribe exactly what should be done in each case. Even for the best specified model, judgment abounds in its application, such that there might be legitimate disagreements as to whether it was applied in a particular case. In fact, one might hope that a model would be "robust" for at least small violations of it, having benefit even when it is not applied in an ideal way.
In the end, it is a matter of judgment as to whether the model was adequately adhered to in these three states. The fact that we have three states with similar findings, that is, similar degrees of adherence to the model, is again relevant. Was the model violated in all three states? Possibly, but that would then suggest the difficulty, perhaps the unlikelihood, of adequately implementing it elsewhere.
Contextual Factors Caused the Model to Fail. It is possible that a variety of contextual factors caused the outcomes that we observed. There are a multitude of possible such factors: the political and economic climate, the climate in the agencies, administrative barriers, approaches of judges, competence of workers, availability of other services, etc. These influences would weigh on both the experimental and control groups, presumably in equivalent ways, but they could prevent any new approach from having effects different from usual treatment. While we cannot exclude such factors as explanations for our results, again the fact that we have three states with similar results is relevant. Multiple sites make it less likely that the same contextual factors are explanations of the findings. Furthermore, social programs must operate in less than ideal contexts, to be effective, their conceptualizations must take into account these circumstances.
One set of contextual factors may have prevented positive effects of family preservation services: broad social problems of poverty, racism, inadequate housing, inadequate education, and substance abuse. Perhaps it is unrealistic to expect a short term program to solve such serious problems.
The Program Conceptualization is Flawed. It is always possible that findings such as ours are the result of program design that is flawed. Obviously, this is the interpretation that is most difficult for program advocates to contemplate. But it is possible that the intervention activities of family preservation programs, even if carried out in an ideal way, are inadequate to achieve their goals. We note here one specific aspect of these programs that is often criticized and blamed for perceived failures: their brevity. It is often suggested that a program only four weeks in length, even if it is very intense, cannot expect to have significant effects on very serious individual and family problems, which are often of long duration, therefore requiring much longer interventions. Going even further, it is possible that the available intervention technology is simply inadequate in the face of the problems it is expected to solve.
9.4 What to Make of These Findings
The findings of this study are not new. As in this investigation, a number of previous evaluations with relatively rigorous designs have failed to produce evidence that family preservation programs have placement prevention effects or have more than minimal benefits in improved family or child functioning. The work reported here may be thought of as four independent evaluations, in four states, adding to the set of previous studies with similar results. While the findings of this study can be questioned (as have those of the previous studies), the accumulation of like findings from a number of studies in several states, with varying measures of outcome, is compelling.
The results do not indicate that family preservation services are detrimental to families. Generally, families in these programs did not do worse than those in the control groups. Nor should the findings be taken as showing that these programs serve no useful purpose in the child welfare system. The findings can be seen as a challenge to keep trying, to find new ways to deal with the problems of families in the child welfare system. The findings indicate the grave difficulties facing those who devise approaches to those problems, failure in such undertakings should not be surprising, and those who risk trying to find solutions should not be punished when evaluations such as this indicate they may have come up short.
The accumulation of findings suggests that the functions, target group, and characteristics of services in programs such as this need to be rethought. Obviously, function, target group, and services are closely intertwined. We discuss below some of the issues that should be considered in rethinking these programs.
The foremost of these issues concerns the objectives of the programs. A number of observers have suggested that placement prevention be abandoned as the central objective in intensive family preservation services in favor of other objectives, notably the improvement of family and child functioning. We have suggested above that targeting these services on families at risk of placement is unlikely to be successful, so if these services are to continue, they will continue to serve "in-home" cases, families in which there has been a substantiated allegation of abuse or neglect or serious conflicts between parents and children but in which children remain in the home. Although the focus of concern in child welfare policy has long been on foster care, in most jurisdictions there are far more cases opened for in-home services than for foster care (a relatively small proportion of indicated reports of maltreatment eventuate in removal of the child from the home and even fewer result in long term placement). Many, if not most, of these "intact" families need help. Relatively intensive and relatively short-term services such as those provided by family preservation programs are one source of such help. In this respect, family preservation programs can be thought of as an important part of the continuum of child welfare services.
Another question that program designers must address is that of specialization. We did not find subgroups for which the programs were successful, but as indicated above, these programs are quite generalist in character, and thus may sacrifice some of the benefits of specialization. Among those benefits are a clearer focus of services, tighter target group definition, specification of service characteristics such as length and intensity based on needs of the target group, and the development of more specific competencies on the part of workers. Specialization could be in terms of problems (e.g., substance abuse) or characteristics of clients (young, isolated mothers). There are clear drawbacks to specialization, including the tendency to define problems in terms of the service one offers. Furthermore, limiting target groups inherently limits the impact of programs. Nonetheless, it may be better to mount a series of small programs rather than putting all of one's resources into large, undifferentiated efforts.
Another issue that program planners must address is that of length and intensity. These aspects of services are generally considered to be inversely related. Because of cost, long-term services cannot be as intensive as short-term efforts. The Homebuilders model pushes the combination of intensity and short term to what seems to be the limit: no more than two cases per worker at a time, 10 to 20 hours of work on a case per week for one month, a period of time much shorter than the planned service period in traditional social services. This is a bold departure from the usual way of doing things. It is based on ideas of crisis intervention. At the time of crisis, people are ready to change and ready to make use of intensive help to change. While crises can happen at any time, child welfare clients are thought to be most likely to be "in crisis" at the time of, or shortly after, an investigation of child maltreatment. Hence, the prototypical family preservation case is a family referred by an investigative worker.
The extent to which the intensive-short-term-crisis approach fits the needs of child welfare clients needs to be reexamined. Families encountering the child welfare system have often been there before and have usually been involved with other public or private service programs, so that being investigated and threatened with removal of a child is more an element of on-going experiences than a crisis. Furthermore, the lives of these families are often full of difficulties--externally imposed and internally generated--such that their problems are better characterized as chronic, rather than crises.
Families with chronic difficulties can no doubt benefit from short-term, intensive services, but those services are unlikely to solve, or make much of a dent in the underlying problems. As an example, substance addiction is a chronic problem in many child welfare families, one that cannot usually be successfully treated in a month's time, however intensive the treatment. Of course, the hope is that family preservation programs will be able to connect families with on-going services to treat more chronic problems, but that appears to happen far less than needed. The central point here is that we need a range of service lengths and service intensities to meet the needs of child welfare clients.
Perhaps the best summary of the status of family preservation programs was provided by McGowan in 1990:
Family preservation services must not be viewed as a panacea. These are categorical programs able to help only one segment of the total range of families and children in need of support and are organized to provide limited types of case services. They cannot address the socioeconomic forces that contribute to tensions and inadequacies in family functioning nor can they provide the long-term assistance and/or specialized treatment required by some parents and children. Thus it is essential to maintain realistic expectations of what these programs can and cannot do. (123)
Endnotes
111. J. Littell and J. Schuerman. (1995). A Synthesis of Research on Family Preservation and Family Reunification. http: //aspe.hhs.gov/hsp/cyp/fplitrev.htm .
112. We did not assess the extent to which reducing placement was an appropriate goal in particular cases; preventing placement and preserving families whenever possible is a well accepted value of the child welfare system.
113. The language we use here is carefully chosen. Technically, we cannot conclude that the programs had no effect.
114. This concern was less present in Philadelphia, since that site did not expressly target cases at imminent risk of placement.
115. It would be unreasonable to expect that targeting would be perfect, that is, that all cases referred for services were at imminent risk of placement. But how high should the targeting rate be? The answer to that question depends on the impact of the program, its costs, and the cost of placement. If the impact of the program is large (that is, it substantially reduces the rate of placement in those cases in which placement would have occurred) or if it is relatively inexpensive relative to the cost of placement, the targeting rate can be lower. Some algebra indicates that the ratio of cost of FPS to placement cost averted (per case served) must be less than the proportion of cases in which placement was averted. For example, if the targeting rate was .5 and the success rate was .4, then the proportion of cases served that result in placement avoidance will be .2 (the product of .5 and .4). The ratio of the cost of FPS to the cost of placement must then be less than .2 for FPS to be cost effective.
116. This group also showed the largest difference between the experimental and control groups in percentages of families experiencing placement at one year, a difference of 15 percent favoring the experimental group. However, the difference is not significant. Furthermore, there are other differences in the table almost as large, some favoring the control group.
117. J. Littell and J. Schuerman. (1995). A Synthesis of Research on Family Preservation and Family Reunification.
http: //aspe.hhs.gov/hsp/cyp/fplitrev.htm .
118. A number of cases in Kentucky and New Jersey were referred to family preservation to assist in the return home of children from foster care. These reunification cases were excluded from the experiment, but they may have contributed to the diffusion of the program.
119. The fact that the case is not going well and that everything else has been tried may or may not mean that placement is likely. Note that "everything else has been tried" is sometimes specified as a criterion for Homebuilders referral. As we noted earlier, this criterion conflicts with the objective of immediate response to crisis.
120. However, it is clear that there is great variation among jurisdictions, workers, judges, and other decision makers in the circumstances in which children are removed from their homes (J. Schuerman, P. Rossi, and S. Budde. (1999). Decisions on Placement and Family Preservation. Evaluation Review 23:599-618).
121. We note one effort to solve the targeting problem in the family preservation program in Detroit. As part of an experimental evaluation of the Families First program, judges were asked to identify cases in which they intended to remove a child from the family, but which they deemed could be diverted to family preservation. After screening by project personnel, a group of such cases was randomly assigned to family preservation or to other services, presumably placement. Results of the study have not been published to date. Our understanding is that the group selected for random assignment was a relatively small portion of all families designated for placement. Furthermore, although the procedure was very promising from the standpoint of tightening up the evaluation, it is unlikely that it could be implemented widely or consistently to solve the targeting problem.
122. Under the assumption that all violations would have been placed in the first month, 27 percent of the control group would have been placed in the first six months, compared to 19 percent of the experimental group. At one year, the proportions would have been 29 percent in the control group and 28 percent in the experimental group.
123. Brenda McGowan. (1990). Family-based services and public policy: Context and implications. In J. Whittaker, J. Kinney, E. Tracy, and C. Booth (eds.). Reaching High-Risk Families: Intensive Family Preservation in Human Services. (pp. 81-82) New York: Aldine de Gruyter.