- Background
- The Homebuilders Model
- Evaluation Design
- Site Descriptions
- The Families
- Service Provision
- Findings
- Implications
- Endnotes
Background
This is a report of an evaluation of programs intended to prevent the placement of children in foster care when it can be avoided.(1) This report focuses on programs in three states, using a particular approach to family preservation, Homebuilders, thought by many to be the most promising approach.
Society has accepted a measure of responsibility for the well being of children, so government can intervene in family life when that well being is severely threatened by abuse or neglect, dependency due to death or disability of parents, or family conflict. Governmental intervention includes removing children from their homes when that is necessary. However, it has long been thought that children should remain in their parent's care whenever possible, consistent with their safety. The tension between assuring the safety of children and maintaining the integrity of families has been a perennial source of debate in the child welfare field and in our society more generally.
In 1980, Congress passed the Adoption Assistance and Child Welfare Act of 1980 (P.L. 96-272) which required states to make "reasonable efforts" to prevent children from entering foster care and to return children who are in foster care to their families. Part of the response of states to that Act was the development of family preservation programs. The emphasis on family preservation was further codified in the 1993 Omnibus Budget Reconciliation Act, which established a 5-year capped entitlement program to encourage the development of family preservation and family support programs. This program was revised and extended by the 1997 Adoption and Safe Families Act.
There have been a number of other evaluations of family preservation programs. Early evaluations suggested these programs had considerable promise but these studies were criticized for flaws in research design. Later, more rigorously designed studies began to cast doubt on the extensive claims of success. The largest of these studies were in California, New Jersey, and Illinois. No placement prevention effects were found in California and Illinois, while the study in New Jersey found short term effects that dissipated with time.(2) However, these studies were also criticized, most notably for not having examined programs thought to be most effective, those based on the Homebuilders approach.
The evaluation reported here was mandated by Congress in the 1993 legislation and was intended, in part, to provide information for deliberations on reauthorization of the funding. In using this report in discussions of reauthorization, it should be kept in mind that a relatively small portion of the Adoption and Safe Families Act funds is used for family preservation.(3) It should also be noted that this report concerns only family preservation programs of a certain kind in three states. However, the three states were chosen because it was believed that they had implemented these programs effectively and because it was thought that they would represent the experience with the Homebuilders approach to service. It is hoped that the evaluation will also be useful to the states in making decisions about child welfare programs and to program planners and practitioners in developing responses to significant social problems.
The evaluation was designed to overcome shortcomings of previous studies of family preservation programs. It studied the Homebuilders model of service, the approach to family preservation that many observers believe to be the most effective. The evaluation also examined a number of outcomes. Placement prevention is a major goal of these programs, but family preservation is expected to achieve that goal while assuring the safety of children. A further important goal of these programs is improvement in functioning of parents, families, and children. Finally, it is expected that these programs will enable child welfare agencies to close cases more quickly, ending their involvement with families. Hence, besides placement prevention, the evaluation assessed the safety of children, changes in child and family functioning, and rates of case closing.
An additional issue raised in the earlier evaluations of family preservation concerned the targeting of these programs. It was found that the families served by these programs often were not those for whom they were intended: cases in which it was likely that at least one child would be placed in foster care without special intervention. The evaluation sought to throw light on this issue as well.
[ Go to Contents ]
The Homebuilders Model
Homebuilders, a foster care placement prevention program developed in 1974 in Tacoma, Washington, calls for short-term, time-limited services provided to the entire family in the home.(4) The program is based, in part, on crisis intervention theory. This theory holds that families experiencing a crisis - that is, about to have a child placed in foster care - will be more amenable to receiving services and learning new behaviors. Social learning theory also plays a part in defining the Homebuilders model. Social learning theory rejects the belief that changes in thinking and feeling must precede changes in behavior. Instead, behavior, beliefs, and expectations influence each other in a reciprocal manner. Key program characteristics include: contact with the family within 24 hours of the crisis, caseload sizes of one or two families per worker, service duration of four to six weeks, provision of both concrete services and counseling, and up to 20 hours of service per family per week.
[ Go to Contents ]
Evaluation Design
The design for this evaluation was an experiment in which families were randomly assigned to either a Homebuilders family preservation program (the experimental group) or to other, "regular," services of the child welfare system (the control group). This report concerns programs in Louisville, Kentucky; seven counties in New Jersey; and Memphis, Tennessee. Information was collected through interviews with caseworkers and caretakers to examine caretakers' parenting practices, interaction with children, discipline, social networks, economic functioning, housing, abuse and neglect, psychological functioning, child well-being, and caseworker/caretaker interactions. These interviews were conducted with:
- The investigating worker, caseworker, and caretaker of each family at the start of services;
- The caseworker and the caretaker at the conclusion of family preservation services and at a comparable point in time for families in the control group; and
- Caretakers one year after entry into the experiment.
After each in-person contact with families, experimental and control caseworkers completed a one-page form describing the services provided during the contact. Administrative data provided information on children's placements, reentries, and subsequent abuse and neglect allegations up to 18 months after entry into the experiment. Staff attitudes and characteristics were collected through a self-administered questionnaire. Throughout the project, discussions were held with personnel of the public agency and service provider agency to gather information about agency services, policies, staffing, training, and the context of services.
[ Go to Contents ]
Site Descriptions
While data collection efforts were the same across sites, the sites varied in their approach to identifying families for services, the populations served, and the type of services provided (see Table 1).
- Kentucky has a statewide program that uses the Homebuilders model. A state office coordinator is responsible for developing uniform selection criteria, training, contracting with family preservation providers, and overseeing the state program. The evaluation was conducted in Louisville, where there is a single family preservation provider agency, and child abuse and neglect cases are referred from intake or ongoing workers. A public agency screener reviewed all cases referred for family preservation services. Her role was to ensure that cases were appropriate for the service. There was no age limitation on the children included in the experiment. In Kentucky there were 174 cases in the experimental group and 175 in the control group.
- New Jersey had a statewide program using the Homebuilders model at the time of the study. During the data collection, a state office coordinator was responsible for developing uniform selection criteria, training, contracting with providers, and program oversight. The study was conducted in seven counties: Bergen, Burlington, Camden, Essex, Monmouth, Ocean, and Passaic. Each county had a separate family preservation provider agency. The study population included Division of Youth and Family Service child abuse and neglect and family problem cases (primarily adolescent-parent conflict cases) referred from intake or ongoing workers. Each county had a screener to review cases referred for family preservation. Their major role was to review the appropriateness of the referrals and to make sure there were openings in the program. When the study began, the state was trying to refocus delivery of family preservation services to families with younger children. Not all counties conformed to this expectation, so all children under 18 were included in the experiment. In New Jersey there were 275 cases in the experimental group and 167 in the control group.
- Tennessee had a statewide program using the Homebuilders model during the study period. It also had a state office coordinator responsible for developing uniform selection criteria, training, contracting with providers, and program oversight. The evaluation was conducted in Memphis and focused on families with children under 13 years old referred from the Department of Children's Services. Cases were referred only from intake workers. Prior to the study, workers referred cases directly to the family preservation program. For the study, cases were referred to a screener rather than directly to the program. In Tennessee there were 98 cases in the experimental group and 49 in the control group.
| Program Description | Kentucky | New Jersey | Tennessee |
|---|---|---|---|
| Program Attributes | |||
| Location of evaluation | Jefferson County (Louisville) Fayette County (Lexington) | Bergen, Burlington, Camden, Essex, Monmouth, Ocean, and Passaic counties. | Shelby County (Memphis) |
| Program type | Statewide FP program | Statewide FP program | Statewide FP program |
| Program model | Homebuilders model | Homebuilders model | Homebuilders model |
| Responsibility for: | |||
| State office coordinator | State office coordinator | State office coordinator |
| State office coordinator | State office coordinator | State office coordinator |
| State office coordinator | State office coordinator | State office coordinator |
| Providers | Single FPS provider in study location. | Single FPS provider in each county location. | Single FPS provider in study. |
| Screener | Targeted cases were at high risk and should have entered care without FP. High risk family court cases where a petition was filed were reviewed for placement in the study. Public agency screener reviewed all cases referred to FPS for appropriateness. | Targeted cases were at high risk and should have entered care without FP. Each county had a screener to review cases referred for FP and make sure there were openings in the program. | Targeted cases were at high risk and would have entered care without FP. For the study, the screener referred cases to the FP program (prior to the study workers referred cases directly to program) |
| Population Attributes | |||
| Population criteria | FP cases referred from intake and ongoing units. | FP cases referred from intake and ongoing cases. | FP cases referred from intake only. |
| Child age limit | Children under 18 years of age. At time of study state the state was trying to refocus delivery of FP to younger children. | All children under 18. At the time of study the state was trying to refocus delivery of FP to younger children but not all counties modified targeting. | 1 child in the family had to be under 13 years of age. |
[ Go to Contents ]
The Families
Most families in the study had birth mothers as the primary caretakers. About half of these women had not graduated from high school. Half of the households in Tennessee were headed by a single-birth mother, compared to 43 percent in Kentucky, and 34 percent in New Jersey (see Table 2).
| Kentucky | New Jersey | Tennessee | |||||
|---|---|---|---|---|---|---|---|
| N | % | N | % | N | % | ||
| Gender of Caretaker/Respondent | 311 | 328 | 117 | ||||
| Male | 7 | 12 | 7 | ||||
| Female | 93 | 88 | 93 | ||||
| Race of Caretaker/Respondent | 310 | 327 | 116 | ||||
| African American (not Hispanic) | 43 | 42 | 83 | ||||
| Caucasian (not Hispanic) | 55 | 47 | 15 | ||||
| Hispanic | 1 | 9 | 1 | ||||
| Other | 1 | 2 | 0 | ||||
| Respondent's education level | 311 | 325 | 116 | ||||
| Elementary school or less | 9 | 9 | 9 | ||||
| Some high school | 44 | 40 | 46 | ||||
| High school graduate or obtained GED | 32 | 26 | 18 | ||||
| College | 14 | 20 | 22 | ||||
| Special education or vocational schooling | 1 | 4 | 4 | ||||
| Respondent's marital status | 310 | 328 | 117 | ||||
| Married | 24 | 30 | 17 | ||||
| Divorced | 19 | 23 | 13 | ||||
| Separated | 21 | 11 | 14 | ||||
| Widowed | 3 | 6 | 3 | ||||
| Never Married | 33 | 30 | 54 | ||||
| Respondent's relationship to youngest child | 292 | 326 | 117 | ||||
| Birth Mother | 85 | 71 | 84 | ||||
| Biological Father | 7 | 10 | 6 | ||||
| Grandmother | 6 | 11 | 4 | ||||
| Other Relative | 2 | 8 | 5 | ||||
| Household composition | 311 | 328 | 117 | ||||
| Birth mother, no other adults | 43 | 34 | 50 | ||||
| Birth mother & 1 male adult | 24 | 27 | 21 | ||||
| Birth mother & extended family * | 9 | 8 | 14 | ||||
| Biological father * | 6 | 9 | 6 | ||||
| Other relative caretaker * | 7 | 17 | 9 | ||||
| Other ** | 10 | 4 | 1 | ||||
| Summary Statistics: | N | Mean | N | Mean | N | Mean | |
| Age of respondent | 306 | 33 | 324 | 39 | 116 | 33 | |
| Age of youngest | 311 | 5 | 328 | 7 | 117 | 4 | |
| Age of oldest child | 311 | 10 | 328 | 13 | 117 | 11 | |
| Number of kids | 311 | 3 | 328 | 3 | 117 | 3 | |
| Number of adults | 311 | 2 | 328 | 2 | 117 | 2 | |
| * These categories may also include other non-related adults in the home ** Includes: non-relative caretaker, adoptive or step-parent, birth mother & non-related females, or birth mother, and more than one non-related male. | |||||||
At the time of referral to the Family Preservation program, families were experiencing a range of problems, some quite severe, others much less so (see Table 3). Examples included one case with children ages 10 and 12 who were not enrolled in school for nearly a month and were at risk of being removed from their home due to truancy and neglect. Another family was living in a home with no electricity, no heat, no food, no working appliances, a non-working toilet which was full of feces, and all 4 children slept in one bed. And yet another involved children who were sexually abused and who displayed extremely violent, uncontrollable and sexually inappropriate behavior at home and school. Although there was considerable diversity of problems, parental mental health and problematic child behavior were common issues.
At the time of the first interview, approximately half of the caretakers self-reported feelings of depression or stress. In Kentucky and New Jersey, approximately half of the caretakers answered affirmatively to each of three questions about emotional difficulties: "feeling blue or depressed," "feeling nervous or tense," and "feeling overwhelmed with work or family responsibility." Caretakers in Tennessee reported these difficulties at an even higher rate. Substantial proportions of caretakers reported behavioral problems in children. Between 60 and 74 percent said at least one of their children got upset easily, and two-thirds to four-fifths indicated that the children threw tantrums. Many said their children fight a lot with other kids (18% to 40%) and were very aggressive with their parents (18% to 43%). A number had problems in school, between 30 and 42 percent had children who had been suspended from school while 9 to 16 percent had children who had been expelled.
| Item | Kentucky | New Jersey | Tennessee |
|---|---|---|---|
| Caretaker Problems | |||
| Felt blue or depressed | 55 | 58 | 62 |
| Felt nervous or tense | 56 | 52 | 53 |
| Just wanted to give up | 31 | 33 | 28 |
| Overwhelmed with work or family responsibility | 47 | 56 | 46 |
| Not enough money for food, rent, or clothing | 49 | 52 | 56 |
| Participation in AFDC, food stamps, WIC, social security disability, or housing vouchers | 82 | 68 | 80 |
| Child Problems (%s of cases for which the question was relevant) | |||
| Child doesn't show much interest in what is going on | 84 | 20 | 29 |
| Child Get(s) upset easily | 69 | 74 | 60 |
| Throw(s) tantrums | 83 | 79 | 67 |
| Fight(s) a lot with other kids | 33 | 40 | 18 |
| Has/Have language problems | 30 | 26 | 25 |
| Is/Are very aggressive toward you | 43 | 56 | 18 |
| Hangs with friends you don't like | 28 | 49 | 44 |
| Been absent from school a lot | 38 | 42 | 27 |
| Run away from home overnight | 10 | 26 | 21 |
| Been temporarily suspended from school | 30 | 32 | 42 |
| Been expelled from school | 11 | 9 | 16 |
| Took something that didn't belong to him or her | 34 | 42 | 27 |
| Absent from school for no good reason | 30 | 27 | 18 |
| Failed any classes | 27 | 41 | 38 |
Half or more of the respondents in all three states indicated that they did not have enough money for food, rent, or clothing. About two-thirds of the respondents in New Jersey and Tennessee reported they participated in at least 1 of the 5 income support programs: AFDC, food stamps, WIC, social security disability, and housing vouchers. In Kentucky, over 80 percent participated in one of these programs.
A number of families had previous involvement with the child welfare system. In Tennessee, 41 percent had previous substantiated allegations of abuse or neglect compared to 47 percent in Kentucky and 53 percent in New Jersey. In Kentucky and New Jersey, a fifth of the families had children who had previously been in foster care placement. In Tennessee, only 4 families had children who had previously been placed.
It might be noted that no mention is made here of substance abuse problems, thought by many to be a major issue in many families involved with the child welfare system. Very few caretakers admitted to alcohol or substance abuse in our initial interviews (fewer than five percent said they had either alcohol or drug problems except in Tennessee, where 8% said they "used drugs several times a week"). These are likely underestimates of the extent of substance misuse in the samples. However, states had policies regarding referrals to family preservation that may have limited the number of families with these problems. For example, New Jersey believed that family preservation should be used cautiously for substance abuse problems. Its FPS policy manual suggested that it is unlikely that a substance abuse problem can be resolved in a 5-6 week period. In Kentucky, families in which a drug dependent adult was not in active treatment were excluded from the program.
[ Go to Contents ]
Service Provision
In all three states, the caretaker interview, the caseworker interview, and the contacts data generally confirmed the expectation that the experimental group would receive more services and more intensive services than the control group (see Table 4). In all three states, the number of experimental group caseworker activities reported by caretakers was greater than that reported by control group respondents, and this was also true of "helpful" caseworker activities. As for specific caseworker activities, experimental group workers in all three states were more likely to provide transportation, talk about discipline, and talk about how to handle anger.
The most common subject of counseling, interaction with children and in particular their discipline, reflects central problems in these families, problems of paramount concern to the child protective system. Experimental group caseworkers in all three states were more often reported to have talked about difficult issues, to have helped the caretaker to see her/his good qualities and problems, and to have "understood your situation."
Insofar as there are differences between groups, it can be assumed that the experimental conditions held since the experimental group received substantially more services than the control group. As is to be expected in real life implementations of models, the programs did not adhere completely to the Homebuilders approach as described above. In addition to other critical elements of family preservation, the Homebuilders model specifies that workers should provide an in-home contact within 72 hours of referral, and family preservation workers should be available 7 days per week. Substantial contact should take place within the first week; the model's developers suggest that the typical case receive 11 hours of service in that time. Concrete services are also an important component of service, particularly early in the case. Based on caseworker reports, families did not always receive contact within 72 hours, fewer than expected contacts occurred in the first week of the program, and few contacts occurred on weekends. There was relatively little provision of concrete services early on.
| Caseworker Activities: (Proportion of affirmative answers by caretakers to yes/no questions) | |||||||||
| Kentucky | New Jersey | Tennessee | |||||||
|---|---|---|---|---|---|---|---|---|---|
| C % | E % | p | C % | E % | p | C % | E % | p | |
| Is Caseworker still working with family | 79 | 64 | 0.006 | 75 | 31 | 0.001 | 57 | 34 | 0.02 |
| Caseworker helped with money for rent, electricity, phone | 3 | 17 | 0.001 | 5 | 4 | 5 | 10 | ||
| Caseworker helped with money for other things | 9 | 35 | 0.001 | 10 | 14 | 11 | 19 | ||
| Caseworker provided transportation | 16 | 42 | 0.001 | 12 | 25 | 0.003 | 19 | 34 | 0.10 |
| Caseworker discussed proper feeding of child | 14 | 20 | 5 | 11 | 0.06 | 16 | 28 | ||
| Caseworker talked with you about discipline | 35 | 55 | 0.001 | 39 | 60 | 0.001 | 46 | 70 | 0.01 |
| Caseworker talked with you on relationship with spouse | 16 | 18 | 8 | 14 | 0.09 | 11 | 34 | 0.01 | |
| Caseworker talked with you about how to handle anger | 28 | 43 | 0.005 | 29 | 53 | 0.001 | 42 | 70 | 0.004 |
| Caseworker told you about other agencies | 38 | 43 | 42 | 56 | 0.01 | 19 | 33 | ||
| Caseworker advised on job training programs | 9 | 19 | 0.009 | 7 | 10 | 8 | 16 | ||
| Caseworker talked about how to get paying job | 6 | 17 | 0.004 | 5 | 8 | 11 | 18 | ||
| Caseworker advised on how to continue school | 9 | 18 | 0.04 | 5 | 8 | 14 | 23 | ||
| Caseworker talked about uneasy issues | 27 | 34 | 29 | 44 | 0.008 | 22 | 51 | 0.003 | |
| Caseworker helped you see good qualities | 67 | 79 | 0.03 | 47 | 70 | 0.001 | 53 | 82 | 0.001 |
| Caseworker helped you see your problem | 66 | 76 | 0.10 | 52 | 72 | 0.001 | 50 | 82 | 0.001 |
| Caseworker understood your situation | 75 | 90 | 0.002 | 62 | 79 | 0.001 | 64 | 79 | 0.08 |
| C Mean | E Mean | p | C Mean | E Mean | p | C Mean | E Mean | p | |
| CT report of # of Caseworker activities | 2.18 | 3.90 | 0.0001 | 2.31 | 3.25 | 0.001 | 2.89 | 4.60 | 0.02 |
| CT report of # of "helpful" Caseworker activities | 1.04 | 1.68 | 0.0001 | 1.11 | 1.97 | 0.0001 | 0.83 | 1.33 | 0.04 |
| NOTE: C = Control Group, E = Experimental Group This table only includes items with a primary p-value less than .05 in at least one of the states; p-values greater than .10 are not reported. Items in bold indicate significant findings in favor of the experimental group whereas italicized items indicate significant findings in favor of the control group. | |||||||||
| Services Provided: (Proportion of affirmative answers by caretakers to yes/no questions) | |||||||||
| Kentucky | New Jersey | Tennessee | |||||||
|---|---|---|---|---|---|---|---|---|---|
| C % | E % | p | C % | E % | p | C % | E % | p | |
| Anyone been in job training program | 3 | 8 | 0.09 | 2 | 3 | 3 | 4 | ||
| Anyone been in WIC | 32 | 45 | 0.02 | 22 | 20 | 51 | 41 | ||
| Been in a marriage counseling program | 0 | 7 | 0.006 | 2 | 2 | 0 | 1 | ||
| Anyone receive daycare | 5 | 19 | 0.001 | 10 | 7 | 26 | 26 | ||
| Anyone receive transportation | 7 | 16 | 0.02 | 14 | 12 | 17 | 19 | ||
| Anyone receiving parent education/training classes | 13 | 19 | 6 | 10 | 20 | 8 | 0.06 | ||
| Anyone receive counseling | 35 | 52 | 0.003 | 50 | 56 | 9 | 17 | ||
| Anyone receive help finding a place to live | 1 | 4 | 5 | 2 | 17 | 5 | 0.04 | ||
| Anyone stay at an emergency shelter | 1 | 1 | 2 | 1 | 6 | 0 | 0.03 | ||
| Anyone receive medical or dental care | 8 | 15 | 0.07 | 36 | 42 | 34 | 16 | 0.03 | |
| Anyone receive homemaker services | 1 | 3 | 6 | 3 | 14 | 3 | 0.02 | ||
| Were any needed services not gotten | 27 | 19 | 56 | 42 | 0.01 | 39 | 24 | 0.10 | |
| C Mean | E Mean | p | C Mean | E Mean | p | C Mean | E Mean | p | |
| Caseworker report of # of services provided | 3.16 | 4.99 | 0.001 | 2.31 | 3.17 | 0.001 | 1.58 | 3.19 | 0.0002 |
| NOTE: C = Control Group, E = Experimental Group This table only includes items with a primary p-value less than .05 in at least one of the states; p-values greater than .10 are not reported. Items in bold indicate significant findings in favor of the experimental group whereas italicized items indicate significant findings in favor of the control group. | |||||||||
[ Go to Contents ]
Findings
This evaluation of family preservation programs was designed to assess the extent to which key goals of the programs are being met: the goals of reducing foster care placement, maintaining the safety of children, and improving family functioning.
The assessment of effects on placement and safety of children was based on administrative data which were available on families for at least one year after the beginning of service. Family functioning was assessed through interviews with caretakers at the beginning of service, one month later (at the end of service for the family preservation group), and a year after the beginning of service. Interviews with caseworkers were also conducted at the beginning and one month points.
No significant differences were found between the experimental and control groups on family level rates of placement, case closings, or subsequent maltreatment. There were a few child and family functioning items in which the experimental group displayed better outcomes than the control group in at least one of the states. However, these results did not occur in more than one state. It was found that family preservation programs in two states resulted in higher assessments by clients of the extent to which goals have been accomplished and of overall improvement in their families' lives.
Reducing foster care placement. In none of the three states were there statistically significant differences between the experimental and control groups on family level rates of placement or case closings (see Table 5). In Kentucky, placement rates at the end of one year were 23 and 24 percent for the experimental and control groups, respectively. In New Jersey and Tennessee, the percents were about 28 and 22 percent.
As to be expected with any program, some of the families assigned to family preservation programs did not receive the services or received a minimal dosage of the services. Also, a small number of the families in the control group were actually provided family preservation services. To address these issues, analyses were conducted in which these cases were dropped (secondary analysis). Results of the secondary analyses were quite similar to the primary analyses, also showing no significant differences between the groups in rates of placement.(5)
The ideal family preservation case is one in which there has been a recent significant crisis in the family, resulting in the maltreatment that triggers the possibility of removal of the child from the home. Subsamples of cases that approached this ideal were examined. Again in these analyses there were no statistically significant differences between the experimental and control groups in placement rates over time.
In addition to placement rates at various points in time, placement was examined in terms of proportion of time in substitute care subsequent to random assignment. No significant differences were found in care days for the families in any of the three states. In Kentucky both the experimental and control group children spent an average of 6 percent of the days subsequent to random assignment in care. In New Jersey, experimental group children spent an average of 6 percent of that time in placement compared to 4 percent for the control group children. In Tennessee, experimental group children spent an average of 10 percent of that time in placement, compared to 5 percent for the control group children.
| Kentucky | One month (%) | 6 months (%) | One year (%) | |||
|---|---|---|---|---|---|---|
| E | C | E | C | E | C | |
| Primary | 6 | 5 | 18 | 18 | 23 | 24 |
| Secondary | 4 | 4 | 13 | 17 | 20 | 24 |
| Refined Analyses: | ||||||
| 8 | 5 | 16 | 14 | 26 | 15 |
| 6 | 3 | 17 | 12 | 29 | 16 |
| 7 | 10 | 14 | 26 | 18 | 33 |
| Including relative placement, primary | 8 | 9 | 21 | 25 | 27 | 32 |
| Including relative placement, secondary | 5 | 9 | 14 | 25 | 22 | 32 |
| New Jersey | One month (%) | 6 months (%) | One year (%) | |||
| E | C | E | C | E | C | |
| Primary | 4 | 6 | 19 | 16 | 28 | 22 |
| Secondary | 3 | 6 | 18 | 16 | 26 | 22 |
| Refined Analyses: | ||||||
| 3 | 5 | 18 | 13 | 25 | 16 |
| 8 | 5 | 20 | 12 | 27 | 15 |
| Tennessee | One month (%) | 6 months (%) | One year (%) | |||
| E | C | E | C | E | C | |
| CORS, primary | 11 | 11 | 22 | 19 | 23 | 19 |
| CORS, secondary | 7 | 12 | 18 | 19 | 19 | 19 |
| Including relative placement, primary | 11 | 11 | 26 | 21 | 28 | 23 |
| Including relative placement, secondary | 7 | 12 | 20 | 19 | 23 | 21 |
| Refined Analyses: | ||||||
| 7 | 12 | 15 | 15 | 17 | 15 |
| 7 | 12 | 18 | 18 | 22 | 21 |
| NOTE: Primary analyses included all cases randomly assigned, except for cases that were determined to be inappropriate referrals. Secondary analyses dropped two categories of cases: families in the control group that were actually provided family preservation services ("violations") and families in the experimental group that received no or little service ("minimal service cases"). "Refined" analyses were limited to subgroups that were thought to represent better targeted cases. Most of the analyses above are of records of placements in administrative data. In Kentucky and Tennessee data were also available from case records on placements with relatives that were not recorded in the administrative data. Those data are included in the rows labeled "including relative placement." | ||||||
Targeting. Since these programs were intended to prevent the placement of children, the target group for the services was families in which at least one child was "in imminent risk of placement." As in previous studies, it was found that most of the families served were not in that target group. This is shown by the placement rate within a short period of time in the control group, indicating the placement experience in the absence of family preservation services. In all three states, the placement rate in the control group within one month was quite low. It would, therefore, have been virtually impossible for the programs to be effective in preventing imminent placement, since very few families would have experienced placement within a month without family preservation services.
A number of subgroups that were thought to represent better targeting were examined. These included cases coming directly from the investigation of an allegation of abuse or neglect, cases with recent substantiated allegations of abuse or neglect, and, in Kentucky, a subgroup of cases in which workers had submitted petitions to the court for placement or some other court-ordered intervention. In none of these subgroups did placement rates in the control group within one month exceed 12 percent. Hence, even in these more refined (from the standpoint of targeting) subgroups, the intended target group was not in evidence.
It should be noted that the results found here occurred despite efforts in this project to improve targeting. In Kentucky and New Jersey a special screening form, developed by the evaluation team, was employed to rate the risk to children with the intent that cases with intermediate risk would be referred to the program. In Kentucky efforts were made to divert to family preservation cases that had been referred to the court. In Tennessee, special training efforts were instituted to address concerns about targeting.
Child Safety. Maltreatment subsequent to the beginning of service was generally not related to experimental group membership, except for one subgroup in Tennessee. Subsequent maltreatment was measured by the occurrence or nonoccurrence of a substantiated allegation of abuse or neglect following an investigation of such an allegation. The rate of subsequent maltreatment was relatively low, about 18 percent of the families in Kentucky had a substantiated allegation within one year of random assignment; in New Jersey the rate was 12 percent and in Tennessee, 25 percent. In Tennessee, in those families with an allegation within 30 days prior to random assignment, the experimental group children experienced fewer substantiated allegations than children in the control group did.
The findings of little difference between the experimental and control groups in subsequent maltreatment can be read in two ways. It indicates that families served by family preservation were no more likely than families not receiving the services to be subjects of allegations of harm. In this sense, children were, by and large, kept safely at home while receiving family preservation services. However, children in both groups were primarily in their homes, and family preservation did not result in lower incidence of maltreatment compared with children in the control group.
Subgroups. In an effort to identify groups of cases for which family preservation is effective, subgroups of Kentucky and New Jersey cases were examined.(6) Subgroups were defined in terms of problems of the family (e.g., substance abuse, financial difficulties, and depression) and family structure. Within these subgroups, experimental and control groups were compared on placement and substantiated allegations subsequent to random assignment. Only one significant difference was found. Among single mothers in New Jersey, those in the experimental group were less likely to have a subsequent substantiated allegation than those in the control group. No subgroups were found in which there were effects on placement in either state.
Family functioning. In a few areas of family functioning, in one or the other of the states, families in the experimental group appeared to be doing better at the end of services (see Table 6). There were very few differences at the year follow-up and in changes over time. Those differences that did appear (primarily at the end of services) were not consistent across states and were not maintained. Family functioning was assessed through caretaker and caregiver interviews at three points in time, shortly after the beginning of services, four to six week later (at the end of services for the Homebuilders group), and again a year after services began. Areas assessed included life events, economic functioning, household condition, child care practices, caretaker depression, child behavior, and caretaker functioning. It can be said that family preservation services may have small, apparently short-term, effects on some areas of functioning. There was one item with some consistency across sites, the overall assessment of improvement by caretakers. At post treatment, a significantly larger proportion of experimental group caretakers in Kentucky and New Jersey generally thought there was "great improvement" in their lives. In Tennessee, although not significant, results tended in the same direction.
| Area | Post treatment | Follow up (one year after beginning of treatment) | Change over time |
|---|---|---|---|
| Life events | |||
| Positive life events | No significant differences | No significant differences | No significant differences |
| Negative life events | No significant differences | No significant differences | No significant differences |
| Depression | No significant differences | No significant differences | No significant differences |
| Family problems, individual items | KY: no significant differences NJ: fewer experimentals not enough money for food, rent, or clothing TN: fewer experimentals had few or no friends | No significant differences | |
| Economic functioning | |||
| Individual items | KY: no significant differences NJ: fewer experimentals difficulty paying rent and buying clothes TN: no significant differences | KY: no significant differences NJ: no significant differences TN: fewer experimentals difficulty paying rent | |
| Scale | KY: no significant difference NJ: experimental average lower (better) TN: no significant difference | No significant differences | No significant differences |
| Household condition | |||
| Individual items | KY: experimentals fewer broken windows or doors NJ: no significant differences TN: more experimentals in unsafe building because of illegal acts | No significant differences | |
| Scale | No significant differences | No significant differences | No significant differences |
| Area | Post treatment | Follow up (one year after beginning of treatment) | Change over time |
| Child care practices | |||
| Individual items | KY: fewer experimentals used punishment for not finishing food NJ: experimentals less often got out of control when punishing child and more often encouraged child to read a book TN: more experimentals went to amusement park, pool, or picnic | No significant differences | |
| Positive scale | No significant differences | No significant differences | No significant differences |
| Negative scale | KY: no significant difference NJ: experimentals lower (better) TN: no significant difference | No significant differences | No significant differences |
| Punishment | KY: no significant difference NJ: experimentals lower (better) TN: no significant difference | No significant differences | No significant differences |
| Caretaker depression | No significant differences | No significant differences | No significant differences |
| Child behavior | |||
| Aggression | No significant differences | No significant differences | No significant differences |
| School problems | No significant differences | No significant differences | No significant differences |
| Positive child behaviors | No significant differences | No significant differences | No significant differences |
| Negative child behaviors | KY: no significant differences NJ: experimental group lower (better) TN: no significant differences | No significant differences | No significant differences |
| Overall assessment of improvement | KY: experimentals, greater improvement NJ: experimentals, greater improvement TN: no significant difference | No significant differences | |
| Caseworker report of caretaker functioning | |||
| Individual items | KY: no significant difference NJ: control group higher (better) in ability in giving affection and providing learning opportunities TN: experimental group higher (better) on five items | KY: respecting child's opinions: experimental group declined, control group increased NJ: control group had more positive change in respecting child's opinions TN: experimental group more positive change on setting firm and consistent limits | |
| Scale | KY: no significant difference NJ: no significant difference TN: experimental group higher (better) | No significant differences | |
| Caseworker report of household condition | KY: control group better NJ: control group better TN: no significant difference | No significant differences | |
| Caseworker report of caretaker problems | KY: experimentals more problems NJ: no significant difference TN: no significant difference | KY: no significant difference NJ: no significant difference TN: experimentals declined more | |
| Caseworker report of child problems | No significant differences | No significant differences | |
[ Go to Contents ]
Implications
The findings of this study are not new. A number of previous evaluations with relatively rigorous designs have failed to produce evidence that family preservation programs with varying approaches to service have placement prevention effects or have more than minimal benefits in improved family or child functioning. The work reported here may be thought of as three independent evaluations, in three states, adding to the set of previous studies with similar results, this time focusing on Homebuilders programs. The accumulation of the findings from a number of studies in several states, with varying measures of outcome, is compelling.
The findings should not be taken as showing that these programs serve no useful purpose in the child welfare system. The results can be seen as a challenge to keep trying, to find new ways to deal with the problems of families in the child welfare system. The findings indicate the grave difficulties facing those who devise approaches to these problems, failure in such undertakings should not be surprising, and those who risk trying to find solutions should not be punished when evaluations such as this indicate they may have come up short.
The accumulation of findings suggests that the functions, target group, and characteristics of services in programs such as this need to be rethought. Obviously, function, target group, and services are closely intertwined. The foremost of these issues concerns the objectives of the programs. A number of observers have suggested that placement prevention be abandoned as the central objective in intensive family preservation services in favor of other objectives, notably the improvement of family and child functioning. Targeting these services on families at risk of placement is unlikely to be successful, so if these services are to continue, they will continue to serve "in-home" cases, families in which there has been a substantiated allegation of abuse or neglect or serious conflicts between parents and children but in which children remain in the home. Many, if not most, of these "intact" families need help. Relatively intensive and relatively short-term services such as those provided by family preservation programs are one source of such help. In this respect, family preservation programs can be thought of as an important part of the continuum of child welfare services.
There are some positives in the findings of this study. Services provided to Homebuilders clients were considerably more extensive and intensive than those provided to control group families. This translated into more positive assessments by caretakers of the relationships they had with workers (Homebuilders clients also rated their overall improvement during the service period as greater). Unfortunately, this apparently better relationship did not translate into observable effects on placement rates or changes in functioning. The challenge for programs is to make use of better relationships to bring about changes in functioning.
Another question that program designers must address is that of specialization. Subgroups for which the program was successful were not found, but these programs are quite generalist in character, and thus may sacrifice some of the benefits of specialization. Among those benefits are a clearer focus of services, tighter target group definition, specification of service characteristics such as length and intensity based on needs of the target group, and the development of more specific competencies on the part of workers. Specialization could be in terms of problems (e.g., substance abuse) or characteristics of clients (young, isolated mothers). There are clear drawbacks to specialization, including the tendency to define problems in terms of the service one offers. Furthermore, limiting target groups inherently limits the impact of programs. Nonetheless, it may be better to mount a series of small programs rather than putting all of one's resources into large, undifferentiated efforts.
Program planners must also address the issue of length and intensity. The extent to which the intensive-short-term-crisis approach of these services fits the needs of child welfare clients needs to be reexamined. The lives of these families are often full of difficulties — externally imposed and internally generated — such that their problems are better characterized as chronic, rather than crisis. Families with chronic difficulties can no doubt benefit from short-term, intensive services, but those services are unlikely to solve, or make much of a dent in the underlying problems. Of course, the hope is that family preservation programs will be able to connect families with on-gong services to treat more chronic problems, but that appears to happen far less than needed. The central point here is that we need a range of service lengths and service intensities to meet the needs of child welfare clients. It is essential that policy makers, planners, and program providers maintain realistic expectations of the effects of short-term family preservation programs.
[ Go to Contents ]
Endnotes
1. This is one of a series of reports from this evaluation. A previous report, The Evaluation of the New York City HomeRebuilders Demonstration reported on a program designed to facilitate the reunification of children in foster care with their families. A final report on the family preservation aspects of this project is forthcoming and will include data on a fourth site, Philadelphia, as well as further analysis of data from the three sites considered here.
2. J. Littell and J. Schuerman. (1995). A Synthesis of Research on Family Preservation and Family Reunification. http://aspe.hhs.gov/hsp/cyp/fplitrev.htm.
3. James Bell Associates. Family Preservation and Family Support (FP/FS) Services Implementation Study, Interim Report. March 1999. Prepared for U.S. Department of Health and Human Services, Administration for Children and Families. Contract # 105-94-8103.
4. Jill Kinney, David Haapala, and Charlotte Booth. (1991). Keeping families together: The Homebuilders model. New York: Aldine de Gruyter.
5. It should be noted that the most rigorous approach to analysis requires that cases be maintained in the groups to which they were randomly assigned. Random assignment is used to assure that the groups are as similar as possible at the outset of service. Removing cases from the groups or switching cases from one group to another threatens group equality and allows for the possibility that post-treatment differences could be explained by factors other than service. In particular, it is likely that violations and minimal service cases differ in systematic ways from other cases. Hence, the secondary analyses should be viewed with caution.
6. The number of cases in Tennessee was too small to allow subgroup analysis.