Contents
Background
This report presents an evaluation of family preservation programs. Family preservation programs are intended to prevent the placement of children in foster care when it can be avoided.(1) This report focuses on programs in four states. Three of the sites employ the Homebuilders model of family preservation, thought by many to be the most promising approach. The fourth site employs a broader, home-based, family preservation service model.
An interim evaluation report was released in October 2000. The interim report presented description, service, and outcome analyses on the Homebuilders study sites. This report expands on the interim report by including description, service, and outcome analyses of the non-Homebuilders site. Additionally, analyses on sample attrition, social support, investigating worker questionnaires, staff questionnaires, and secondary analyses are included in this report.(2)
Society has accepted a measure of responsibility for the well-being of children. These measures allow government to intervene in family life when a child is severely threatened by abuse or neglect, dependency due to death or disability of parents, or family conflict. Governmental intervention includes removing children from their homes when that is necessary. However, it has long been thought that children should remain in their parent's care whenever possible, consistent with their safety. The tension between assuring the safety of children and maintaining the integrity of families has been a perennial source of debate in the child welfare field and in our society more generally.
Legislation
In 1980, Congress passed the Adoption Assistance and Child Welfare Act of 1980 (P.L. 96-272). This Act required states to make "reasonable efforts" to prevent children from entering foster care and to return children who are in foster care to their families. Part of the response of states to that Act was the development of family preservation programs. The emphasis on family preservation was further codified in the 1993 Omnibus Budget Reconciliation Act, which established a 5-year capped entitlement program to encourage the development of family preservation and family support programs.
This program was revised and extended by P.L. 105-89, the 1997 Adoption and Safe Families Act (ASFA). The Adoption and Safe Families Act changed and clarified a number of policies established in the 1980 Act with a renewed emphasis on safety, permanency, and adoption. ASFA placed Federal family preservation initiatives under the rubric of "Promoting Safe and Stable Families" and extended funding for FY 2001. The law made safety of children the paramount concern in service delivery and increased the need to understand how family preservation services strengthen families and prevent foster care placement and subsequent abuse and neglect allegations.
Public Law 107-133, the "Promoting Safe and Stable Families Amendments of 2001" was signed into law in January 2002. This legislation reauthorized family preservation services through 2006. Additionally, the legislation authorized the Court Improvement Program, and offered states flexibility in defining family preservation services to allow states to support infant safe haven programs and strengthen parental relationships and promote healthy marriages.
Evaluations
There have been a number of other evaluations of family preservation programs. Early evaluations suggested these programs had considerable promise but these studies were criticized for flaws in research design. Later, more rigorously designed studies began to cast doubt on the extensive claims of success. The largest of these studies were in California, New Jersey, and Illinois. No placement prevention effects were found in California and Illinois, while the study in New Jersey found short-term effects that dissipated with time.(3) However, these studies were also criticized, most notably for not having examined programs thought to be most effective, those based on the Homebuilders approach.
The evaluation reported here was mandated by Congress in the 1993 legislation and was intended, in part, to provide information for deliberations on reauthorization of the funding. It is hoped that the evaluation will also be useful to the states in making decisions about child welfare programs and to program planners and practitioners in developing responses to significant social problems.
The evaluation was designed to overcome shortcomings of previous studies of family preservation programs. It studied the Homebuilders model of service in the states of Kentucky, New Jersey, and Tennessee. The Homebuilders model is the approach to family preservation that many observers believe to be the most effective. The evaluation also studied a program model somewhat less intensive than Homebuilders, in Philadelphia. The evaluation examined a number of outcomes. Placement prevention is a major goal of these programs, but family preservation is expected to achieve that goal while assuring the safety of children. A further important goal of these programs is improvement in functioning of parents, children, and families. Finally, it is expected that these programs will enable child welfare agencies to close cases more quickly, ending their involvement with families. Hence, besides placement prevention, the evaluation assessed the safety of children, changes in child and family functioning, and rates of case closure.
An additional issue raised in the earlier evaluations of family preservation concerned the targeting of these programs. It was found that the families served by these programs often were not those for whom they were intended: cases in which it was likely that at least one child would be placed in foster care without special intervention. The evaluation sought to throw light on this issue for the Homebuilders models as well.
The Homebuilders Model
Homebuilders, a foster care placement prevention program developed in 1974 in Tacoma, Washington, calls for short-term, time-limited services provided to the entire family in the home.(4) The program is based, in part, on crisis intervention theory. This theory holds that families experiencing a crisis that is, about to have a child placed in foster care will be more amenable to receiving services and learning new behaviors. Social learning theory also plays a part in defining the Homebuilders model. Social learning theory rejects the belief that changes in thinking and feeling must precede changes in behavior. Instead, behavior, beliefs, and expectations influence each other in a reciprocal manner. Key program characteristics include:
- contact with the family within 24 hours of the crisis
- caseload sizes of one or two families per worker
- service duration of four to six weeks
- provision of concrete services and counseling
- the family receiving up to 20 hours of service per week.
Broader Home-based Family Preservation Service Model
The broader home-based model focuses on the behavior of the family overall, and attempts to change the way in which the family functions as a whole and within the community. Aside from a primary goal of placement prevention, the model also seeks to improve functioning of parents, families, and children. Programs using the home-base model stress longer-term interventions based on family systems theory. One study site, the Philadelphia Family Preservation Services (FPS), used a broader home-based model. FPS tailored home-based services to build upon the Pennsylvania Free substance abuse services provided in the 1980s. Key characteristics of the Philadelphia FPS program included: 12 weeks of service to families, focus on drug and alcohol abuse in families, caseload sizes of five families per worker, and provision of both concrete services and counseling.
[Go To Contents]
Evaluation Design
The design for this evaluation was an experiment in which families were randomly assigned either to a family preservation program (the experimental group) or to other, "regular," services of the child welfare system (the control group). This report concerns programs in Louisville, Kentucky; seven counties in New Jersey; Memphis, Tennessee; and Philadelphia, Pennsylvania. Information was collected through interviews with caseworkers and caretakers to examine caretakers' parenting practices, interaction with children, discipline, social networks, economic functioning, housing, abuse and neglect, psychological functioning, child well-being, and caseworker/caretaker interactions. These interviews were conducted with:
- The investigating worker, caseworker, and caretaker of each family at the start of services;
- The caseworker and the caretaker at the conclusion of family preservation services and at a comparable point in time for families in the control group; and
- The caretaker one year after entry into the experiment.
After each in-person contact with families, experimental and control caseworkers completed a one-page form describing the services provided during the contact. Administrative data provided information on children's placements, reentries, and subsequent abuse and neglect allegations up to 18 months after entry into the experiment. Staff attitudes and characteristics were collected through a self-administered questionnaire. Throughout the project, discussions were held with personnel of the public agency and service provider agency to gather information about agency services, policies, staffing, training, and the context of services.
Site Descriptions
While data collection efforts were the same across sites, the sites varied in their approach to identifying families for services, the populations served, and the type of services provided (Table 1).
| Program Description | Kentucky | New Jersey | Tennessee | Philadelphia |
|---|---|---|---|---|
| Program Attributes | ||||
| Location of evaluation | Jefferson County (Louisville) Fayette County (Lexington) | Bergen, Burlington, Camden, Essex, Monmouth, Ocean, and Passaic counties. | Shelby County (Memphis) | Philadelphia County |
| Program type | Statewide FP program | Statewide FP program | Statewide FP program | County FP program |
| Program model | Homebuilders model | Homebuilders model | Homebuilders model | Specialized program model |
| Responsibility for: | ||||
| Selection criteria | State office coordinator | State office coordinator | State office coordinator | Public specialized FPS section |
| Training | State office coordinator | State office coordinator | State office coordinator | State DHS office |
| FP provider oversight | State office coordinator | State office coordinator | State office coordinator | State DHS office |
| Providers | Single FPS provider in study location. | Single FPS provider in each county location. | Single FPS provider in study. | Three private FPS providers in study |
| Screener | Targeted cases were at high risk and should have entered foster care without FP. High-risk family court cases where a petition was filed were reviewed for placement in the study. Public agency screener reviewed all cases referred to FPS for appropriateness. | Targeted cases were at high risk and should have entered foster care without FP. Each county had a screener to review cases referred for FP and make sure there were openings in the program. | Targeted cases were at high risk and would have entered foster care without FP. For the study, the screener referred cases to the FP program (prior to the study workers referred cases directly to program) | Targeted cases were at intermediate risk of removal from home. DHS FPS supervisor screened cases to the FPS program and determined if there was an opening in the program. |
| Population Attributes | ||||
| Population criteria | FP cases referred from intake and ongoing units. | FP cases referred from intake and ongoing cases. | FP cases referred from intake only. | FP cases were referred from CPS intake only |
| Child age limit | Children under 18 years of age. At time of study, the state was trying to refocus delivery of FP to younger children. | All children under 18. At the time of study, the state was trying to refocus delivery of FP to younger children but not all counties modified targeting. | 1 child in the family had to be under 13 years of age. | All children under 18 The program originally focused on young children but progressed to serving families with older children |
- Kentucky had a statewide program that uses the Homebuilders model. A state office coordinator was responsible for developing uniform selection criteria, training, contracting with family preservation providers, and overseeing the state program. The evaluation was conducted in Louisville. This location provided a single-family preservation provider agency. Child abuse and neglect cases in Louisville were referred from intake or ongoing workers. A public agency screener reviewed all cases referred for family preservation services. Her role was to ensure that cases were appropriate for the service. There was no age limitation on the children included in the experiment. In Kentucky, there were 174 cases in the experimental group and 175 in the control group.
- New Jersey had a statewide program using the Homebuilders model at the time of the study. During the data collection, a state office coordinator was responsible for developing uniform selection criteria, training, contracting with providers, and program oversight. The study was conducted in seven counties: Bergen, Burlington, Camden, Essex, Monmouth, Ocean, and Passaic. Each county had a separate family preservation provider agency. The study population included Division of Youth and Family Service child abuse and neglect and family problem cases (primarily adolescent-parent conflict cases) referred from intake or ongoing workers. Each county had a screener to review cases referred for family preservation. Their major role was to review the appropriateness of the referrals and to make sure there were openings in the program. When the study began, the state was trying to refocus delivery of family preservation services to families with younger children. Not all counties conformed to this expectation, so all children under 18 were included in the experiment. In New Jersey, there were 275 cases in the experimental group and 167 in the control group.
- Tennessee had a statewide program using the Homebuilders model during the study period. It also had a state office coordinator responsible for developing uniform selection criteria, training, contracting with providers, and program oversight. The evaluation was conducted in Memphis and focused on families with children under 13 years old referred from the Department of Children's Services. Cases were referred only from intake workers. Prior to the study, workers referred cases directly to the family preservation program. For the study, cases were referred to a screener rather than directly to the program. In Tennessee, there were 98 cases in the experimental group and 49 in the control group.
- Philadelphia had a family preservation program that used a broader service model than the traditional Homebuilders model during the study period. The state office was responsible for training and program oversight. The agency-specialized FPS section developed selection criteria for referral. FPS were provided by private agencies in a public-private collaboration. The evaluation included three private agencies Abraxas Foundation, Tabor Children's Services and Youth Service, Inc. FPS were provided by Abraxas Foundation and Tabor Children's Services. All three agencies provided non-FPS Services to Children in their Own Home (SCOH) services to families. Cases were referred only from intake workers. Referrals came through a public supervisor who screened cases for FPS. In Philadelphia, there were 209 cases in the experimental group and 144 in the control group.
[Go To Contents]
The Families
Most families in the study had birth mothers as the primary caretakers. In Kentucky, New Jersey, and Tennessee about half of these women had not graduated from high school. In Philadelphia, 65 percent of the women had not graduated from high school. Half of the households in Tennessee and Philadelphia were headed by a single-birth mother, compared to 43 percent in Kentucky, and 34 percent in New Jersey (Table 2).
| Kentucky | New Jersey | Tennessee | Philadelphia | |||||
|---|---|---|---|---|---|---|---|---|
| N | % | N | % | N | % | N | % | |
| Gender of caretaker/respondent | 311 | 328 | 117 | 263 | ||||
| Male | 7 | 12 | 7 | 5 | ||||
| Female | 93 | 88 | 93 | 95 | ||||
| Race of caretaker/respondent | 310 | 327 | 116 | 263 | ||||
| African American (not Hispanic) | 43 | 42 | 83 | 80 | ||||
| Caucasian (not Hispanic) | 55 | 47 | 15 | 15 | ||||
| Hispanic | 1 | 9 | 1 | 2 | ||||
| Other | 1 | 2 | 0 | 2 | ||||
| Respondent's education level | 311 | 325 | 116 | 263 | ||||
| Elementary school or less | 9 | 9.4 | 9 | 4 | ||||
| Some high school | 44 | 40 | 46 | 61 | ||||
| High school graduate or obtained GED | 32 | 26 | 18 | 19 | ||||
| College | 14 | 20 | 22 | 11 | ||||
| Special education or vocational schooling | 1 | 4.0 | 4 | 4 | ||||
| Respondent's marital status | 310 | 328 | 117 | 263 | ||||
| Married | 24 | 30 | 17 | 10 | ||||
| Divorced | 19 | 23 | 13 | 7 | ||||
| Separated | 21 | 11 | 14 | 11 | ||||
| Widowed | 3 | 6 | 3 | 3 | ||||
| Never married | 33 | 30 | 54 | 69 | ||||
| Respondent's relationship to youngest child | 292 | 326 | 117 | 263 | ||||
| Birth mother | 85 | 69 | 85 | 91 | ||||
| Biological father | 67 | 10 | 6 | 5 | ||||
| Grandmother | 6 | 12 | 3 | 3 | ||||
| Other relative | 2 | 9 | 6 | 2 | ||||
| Household composition | 311 | 328 | 117 | 263 | ||||
| Birth mother, no other adults | 43 | 34 | 49 | 50 | ||||
| Birth mother & 1 male adult | 24 | 27 | 20 | 20 | ||||
| Birth mother & extended family* | 9 | 8 | 14 | 19 | ||||
| Biological father* | 6 | 10 | 6 | 5 | ||||
| Other relative caretaker* | 7 | 18 | 9 | 5 | ||||
| Other** | 10 | 4 | 3 | 3 | ||||
| N | Mean | N | Mean | N | Mean | N | Mean | |
| Age of respondent | 306 | 32 | 324 | 39 | 116 | 33 | 260 | 33 |
| Age of youngest child | 311 | 5 | 328 | 7 | 117 | 4 | 263 | 4 |
| Age of oldest child | 311 | 10 | 328 | 13 | 117 | 11 | 263 | 11 |
| Number of kids | 311 | 3 | 328 | 3 | 117 | 3 | 263 | 3 |
| Number of adults | 311 | 2 | 328 | 2 | 117 | 2 | 263 | 2 |
| * These categories may also include other non-related adults in the home. ** Includes: non-relative caretaker, adoptive or step-parent, birth mother & non-related females, or birth mother, and more than one non-related male. | ||||||||
At the time of referral to the Family Preservation program, families were experiencing a range of problems, some quite severe, others much less so (Table 3). Examples included one case with children ages 10 and 12 who were not enrolled in school for nearly a month and who were at risk of being removed from their home due to truancy and neglect. Another family was living in a home with no electricity, no heat, no food, no working appliances, a non-working toilet which was full of feces, and all four children slept in one bed. Yet another involved children who were sexually abused and who displayed extremely violent, uncontrollable and sexually inappropriate behavior at home and school. Although there was considerable diversity of problems, parental mental health and problematic child behavior were common issues.
| Item | Kentucky % | New Jersey % | Tennessee % | Philadelphia % |
|---|---|---|---|---|
| Caretaker Problems | ||||
| Felt blue or depressed | 55 | 58 | 62 | 62 |
| Felt nervous or tense | 56 | 52 | 53 | 53 |
| Just wanted to give up | 31 | 33 | 28 | 33 |
| Overwhelmed with work or family responsibility | 47 | 56 | 46 | 52 |
| Not enough money for food, rent, or clothing | 49 | 52 | 56 | 56 |
| Participation in AFDC, food stamps, WIC, social security disability, or housing vouchers | 82 | 68 | 80 | 90 |
| Child Problems (% of cases for which the question was relevant) | ||||
| Child doesn't show much interest in what is going on | 84 | 20 | 29 | 17 |
| Child get(s) upset easily | 69 | 74 | 60 | 59 |
| Throw(s) tantrums | 83 | 79 | 67 | 70 |
| Fight(s) a lot with other kids | 33 | 40 | 18 | 31 |
| Has/Have language problems | 30 | 26 | 25 | 18 |
| Is/Are very aggressive toward you | 43 | 56 | 18 | 33 |
| Hangs with friends you don't like | 28 | 49 | 44 | 25 |
| Been absent from school a lot | 38 | 42 | 27 | 19 |
| Run away from home overnight | 10 | 26 | 21 | 5 |
| Been temporarily suspended from school | 30 | 32 | 42 | 22 |
| Been expelled from school | 11 | 9 | 16 | 4 |
| Took something that didn't belong to him or her | 34 | 42 | 27 | 24 |
| Absent from school for no good reason | 30 | 27 | 18 | 9 |
| Failed any classes | 27 | 41 | 38 | 25 |
At the time of the first interview, approximately half of the caretakers self-reported feelings of depression or stress. In Kentucky and New Jersey, approximately half of the caretakers answered affirmatively to each of three questions about emotional difficulties: "feeling blue or depressed," "feeling nervous or tense," and "feeling overwhelmed with work or family responsibility." Caretakers in Tennessee and Philadelphia reported these difficulties at an even higher rate. Substantial proportions of caretakers reported behavioral problems in children. Between 59 and 74 percent said at least one of their children got upset easily, and two-thirds to four-fifths indicated that the children threw tantrums. Many said their children fight a lot with other kids (18% to 40%) and were very aggressive with their parents (18% to 56%). A number had problems in school, between 22 and 42 percent had children who had been suspended from school while 4 to 16 percent had children who had been expelled.
Half or more of the respondents in all four states indicated that they did not have enough money for food, rent, or clothing. About two-thirds of the respondents in New Jersey reported they participated in at least one of the five income-support programs: AFDC, food stamps, WIC, social security disability, and housing vouchers. In Kentucky and Tennessee, about 80 percent participated in one of these programs, and in Philadelphia participation was at 90 percent.
A number of families had previous involvement with the child welfare system. In Tennessee, 41 percent had previous substantiated allegations of abuse or neglect compared to 47 percent in Kentucky, 53 percent in New Jersey, and 81 percent in Philadelphia. In Kentucky and New Jersey, a fifth of the families had children who had previously been in foster care. The rate was slightly lower in Philadelphia at 17 percent. In Tennessee, only a few families had children who had previously been placed.
It might be noted that no mention is made here of substance abuse problems, thought by many to be a major issue in many families involved with the child welfare system. Very few caretakers admitted to alcohol or substance abuse in our initial interviews; fewer than five percent said they had either alcohol or drug problems. The exception was in Philadelphia and Tennessee, where 9 percent and 8 percent respectively said they "used drugs several times a week." These are likely underestimates of the extent of substance misuse in the samples particularly in Philadelphia since FPS service providers in the Philadelphia study site focused on serving families with substance abuse problems. However, other states had policies regarding referrals to family preservation that may have limited the number of families with these problems. For example, New Jersey believed that family preservation should be used cautiously for substance abuse problems. Its FPS policy manual suggested that it is unlikely that a substance abuse problem can be resolved in a 5-6 week period. In Kentucky, families in which a drug-dependent adult was not in active treatment were excluded from the program.
[Go To Contents]
Service Provision
In all sites, the caretaker interview, the caseworker interview, and the contacts data generally confirmed the expectation that the experimental group would receive more services and more intensive services than the control group (Table 4). In all four states, the number of experimental group caseworker activities reported by caretakers was greater than that reported by control group respondents, and this was also true of "helpful" caseworker activities. As for specific caseworker activities, experimental group workers in all four states were more likely to provide transportation, and talk about discipline.
| Caseworker Activities: | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Proportion of affirmative answers by caretakers to yes/no questions | Kentucky | New Jersey | Tennessee | Philadelphia | ||||||||
| C % | E % | p | C % | E % | p | C % | E % | p | C % | E % | p | |
| Is caseworker still working with family | 79 | 64 | 0.006 | 75 | 31 | 0.001 | 57 | 34 | 0.02 | |||
| Caseworker helped with money for rent, electricity, phone | 3 | 17 | 0.001 | 5 | 4 | 5 | 10 | 3 | 4 | |||
| Caseworker helped with money for other things | 9 | 35 | 0.001 | 10 | 14 | 11 | 19 | 5 | 22 | .001 | ||
| Caseworker provided transportation | 16 | 42 | 0.001 | 12 | 25 | 0.003 | 19 | 34 | 0.10 | 35 | 50 | .03 |
| Caseworker discussed proper feeding of child | 14 | 20 | 5 | 11 | 0.06 | 16 | 28 | 22 | 28 | |||
| Caseworker talked with you about discipline | 35 | 55 | 0.001 | 39 | 60 | 0.001 | 46 | 70 | 0.01 | 32 | 53 | .002 |
| Caseworker talked with you on relationship with spouse | 16 | 18 | 8 | 14 | 0.09 | 11 | 34 | 0.01 | 13 | 20 | ||
| Caseworker talked with you about how to handle anger | 28 | 43 | 0.005 | 29 | 53 | 0.001 | 42 | 70 | 0.004 | 31 | 37 | |
| Caseworker told you about other agencies | 38 | 43 | 42 | 56 | 0.01 | 19 | 33 | 0.13 | 39 | 47 | ||
| Caseworker advised on job training programs | 9 | 19 | 0.009 | 7 | 10 | 8 | 16 | 23 | 36 | .04 | ||
| Caseworker talked about how to get paying job | 6 | 17 | 0.004 | 5 | 8 | 11 | 18 | 19 | 33 | .02 | ||
| Caseworker advised on how to continue school | 9 | 18 | 0.04 | 5 | 8 | 14 | 23 | 21 | 34 | .03 | ||
| Caseworker talked about uneasy issues | 27 | 34 | 29 | 44 | 0.008 | 22 | 51 | 0.003 | 27 | 36 | ||
| Caseworker helped you see good qualities | 37 | 79 | 0.03 | 47 | 70 | 0.001 | 53 | 82 | 0.001 | 68 | 82 | .01 |
| Caseworker helped you see your problem | 66 | 76 | 0.10 | 52 | 72 | 0.001 | 50 | 82 | 0.001 | 74 | 76 | |
| Caseworker understood your situation | 75 | 90 | 0.002 | 62 | 79 | 0.001 | 64 | 79 | 0.08 | 82 | 82 | |
| NOTE: C = Control Group, E = Experimental Group | ||||||||||||
| Kentucky | New Jersey | Tennessee | Philadelphia | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| C Mean | E Mean | p | C Mean | E Mean | p | C Mean | E Mean | p | C Mean | E Mean | p | |
| CT report of # of caseworker activities | 2.18 | 3.90 | 0.0001 | 2.31 | 3.25 | 0.001 | 2.89 | 4.60 | 0.02 | 2.9 | 4.6 | .0001 |
| CT report of # of "helpful" caseworker activities | 1.04 | 1.68 | 0.0001 | 1.11 | 1.97 | 0.0001 | 0.83 | 1.33 | 0.04 | 1.5 | 2.2 | .02 |
| Services Provided: | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Proportion of affirmative answers by caretakers to yes/no questions | ||||||||||||
| Kentucky | New Jersey | Tennessee | Philadelphia | |||||||||
| C % | E % | p | C % | E % | p | C % | E % | p | C % | E % | p | |
| Anyone been in job training program | 3 | 8 | 0.09 | 2 | 3 | 3 | 4 | 20 | 26 | |||
| Anyone been in WIC | 32 | 45 | 0.02 | 22 | 20 | 51 | 41 | 40 | 44 | |||
| Been in a marriage counseling program | 0 | 7 | 0.006 | 2 | 2 | 0 | 1 | 2 | 2 | |||
| Anyone receive daycare | 5 | 19 | 0.001 | 10 | 7 | 26 | 26 | 11 | 15 | |||
| Anyone receive transportation | 7 | 16 | 0.02 | 14 | 12 | 17 | 19 | 25 | 39 | .02 | ||
| Anyone receiving parent education/training | 13 | 19 | 6 | 10 | 20 | 8 | 0.06 | 16 | 37 | |||
| Anyone receive counseling | 35 | 52 | 0.003 | 50 | 56 | >9 | 17 | 21 | 26 | |||
| Anyone receive help finding a place to live | 1 | 4 | 5 | 2 | 17 | 5 | 0.04 | 9 | 9 | |||
| Anyone stay at an emergency shelter | 1 | 1 | 2 | 1 | 6 | 0 | 0.03 | 4 | 3 | |||
| Anyone receive medical or dental care | 8 | 15 | 0.07 | 36 | 42 | 34 | 16 | 0.03 | 33 | 39 | ||
| Anyone receive homemaker services | 1 | 3 | 6 | 3 | 14 | 3 | 0.02 | 1 | 1 | |||
| Were any needed services not gotten | 27 | 19 | 56 | 42 | 0.01 | 39 | 24 | 0.10 | 24 | 19 | ||
| C Mean | E Mean | p | C Mean | E Mean | p | C Mean | E Mean | p | C Mean | E Mean | p | |
| Caseworker report of # of services provided | 3.16 | 4.99 | 0.001 | 2.31 | 3.17 | 0.001 | 1.58 | 3.19 | 0.0002 | 3.4 | 4.9 | .0004 |
| NOTE: C = Control Group, E = Experimental Group Table only includes items with a primary p-value less than .05 in at least one of the states; p-values greater than .20 are not reported. Items in bold indicate significant findings in favor of the experimental group; italicized items indicate significant findings in favor of the control group. | ||||||||||||
Central casework activities with families included counseling families, handling anger, and child discipline. These activities reflect common problems with families that are of paramount concern to the child protective system. Experimental group caseworkers in the Homebuilders states were more often reported to have talked about difficult issues, to have helped the caretaker to see her/his good qualities and problems, and to have understood the parent's situation. In Philadelphia, caretakers reported much the same.
Insofar as there are differences between groups, it can be assumed that the experimental conditions held since the experimental group received substantially more services than the control group. As is to be expected in real life implementations of models, the programs did not adhere completely to the Homebuilders approach as described above. In addition to other critical elements of family preservation, the Homebuilders model specifies that workers should provide an in-home contact within 72 hours of referral, and family preservation workers should be available 7 days per week. Substantial contact should take place within the first week; the model's developers suggest that the typical case receive 11 hours of service in that time. Concrete services are also an important component of service, particularly early in the case. Based on caseworker reports, families did not always receive contact within 72 hours, fewer than expected contacts occurred in the first week of the program, and few contacts occurred on weekends. There was relatively little provision of concrete services early on.
[Go To Contents]
Findings
This evaluation of family preservation programs was designed to assess the extent to which key goals of the programs are being met: the goals of reducing foster care placement, maintaining the safety of children, and improving family functioning. The assessment of effects on placement and safety of children was based on administrative data, which were available on families for at least one year after the beginning of service. Family functioning was assessed through interviews with caretakers at the beginning of service, one month later (at the end of service for the family preservation group), and a year after the beginning of service. Interviews with caseworkers were also conducted at the beginning and one month points.
No significant differences were found between the experimental and control groups on family level rates of placement, case closings, or subsequent maltreatment. There were a few child and family functioning items in which the experimental group displayed better outcomes than the control group in at least one of the states. However, these results did not occur in more than one state. It was found that family preservation programs in two states resulted in higher assessments by clients of the extent to which goals have been accomplished and of overall improvement in their families' lives.
Reducing Foster Care Placement. In none of the four states were there statistically significant differences between the experimental and control groups on family level rates of placement or case closings (Table 5). In Kentucky, placement rates at the end of one year were 25 and 24 percent for the experimental and control groups, respectively. In New Jersey, the percents were 29 and 22 percent. The rates in Tennessee were 23 and 19 percent. In Philadelphia, placement rates were 18 and 15 percent at the end of one year.
| Kentucky | 1 month | 6 months | 12 months | 18 months | ||||
|---|---|---|---|---|---|---|---|---|
| E | C | E | C | E | C | E | C | |
| Primary analyses | 6 | 5 | 18 | 18 | 25 | 24 | 27 | 27 |
| Secondary analyses | 4 | 4 | 12 | 18 | 20 | 23 | 24 | 25 |
| Refined analyses | ||||||||
| Investigative | 8 | 5 | 15 | 14 | 26 | 15 | 28 | 20 |
| Recent substantiation | 6 | 2 | 20 | 11 | 29 | 13 | 32 | 18 |
| Petition cases | 6 | 9 | 16 | 14 | 22 | 29 | 25 | 32 |
| New Jersey | 1 month | 6 months | 12 months | 18 months | ||||
|---|---|---|---|---|---|---|---|---|
| E | C | E | C | E | C | E | C | |
| Primary analyses | 5 | 6 | 19 | 17 | 29 | 22 | 35 | 26 |
| Secondary analyses | 3 | 6 | 17 | 17 | 27 | 23 | 34 | 27 |
| Refined analyses | ||||||||
| Investigative | 3 | 5 | 16 | 12 | 25 | 15 | 32 | 19 |
| Recent substantiation | 8 | 5 | 19 | 12 | 25 | 14 | 33 | 21 |
| Tennessee | 1 month | 6 months | 12 months | |||
|---|---|---|---|---|---|---|
| E | C | E | C | E | C | |
| Administrative data, primary analysis | 11 | 11 | 22 | 19 | 23 | 19 |
| Administrative data, secondary analysis | 7 | 12 | 18 | 19 | 19 | 19 |
| Including relatives, primary | 11 | 11 | 26 | 21 | 28 | 23 |
| Including relatives, secondary | 7 | 12 | 20 | 19 | 23 | 21 |
| Refined analyses | ||||||
| Recent investigation, CORS* | 7 | 12 | 15 | 15 | 17 | 15 |
| Recent investigation, includes Relative | 7 | 12 | 18 | 18 | 22 | 21 |
| Philadelphia | 1 month | 6 months | 12 months | 18 months | ||||
|---|---|---|---|---|---|---|---|---|
| E | C | E | C | E | C | E | C | |
| Primary analyses | 1 | 1 | 10 | 12 | 18 | 15 | 24 | 20 |
| Secondary analyses | 1 | 1 | 9 | 13 | 15 | 16 | 21 | 19 |
| * Client Operation and Review System | ||||||||
As to be expected with any program, some of the families assigned to family preservation programs did not receive the services or received a minimal dosage of the services. In addition, a small number of the families in the control group were actually provided family preservation services. To address these issues, analyses were conducted in which these cases were dropped (secondary analysis). Results of the secondary analyses were quite similar to the primary analyses, also showing no significant differences between the groups in rates of placement.(5)
The ideal family preservation case is one in which there has been a recent significant crisis in the family, resulting in the maltreatment that triggers the possibility of removal of the child from the home. Subsamples of cases that approached this ideal were examined. Again in these analyses, there were no statistically significant differences between the experimental and control groups in placement rates over time.
In addition to placement rates at various points in time, placement was examined in terms of proportion of time in substitute care after random assignment. No significant differences were found in care days for the families in any of the four states. In Kentucky, both the experimental and control group children spent an average of 6 percent of the days after random assignment in care. In New Jersey and Philadelphia, experimental group children spent an average of 6 percent of that time in placement compared to 4 percent for the control group children. In Tennessee, experimental group children spent an average of 10 percent of that time in placement, compared to 5 percent for the control group children.
Targeting. Since these programs were intended to prevent the placement of children, the target group for the Homebuilders program services was families in which at least one child was "in imminent risk of placement."(6)
As in previous studies, it was found that most of the families served were not in that target group. This is shown by the placement rate within a short period in the control group, indicating the placement experience in the absence of family preservation services. In all three states, the placement rate in the control group within one month was quite low. It would, therefore, have been virtually impossible for the programs to be effective in preventing imminent placement, since very few families would have experienced placement within a month without family preservation services.
A number of subgroups that were thought to represent better targeting were examined. These included:
- cases coming directly from the investigation of an allegation of abuse or neglect,
- cases with recent substantiated allegations of abuse or neglect,
- cases in a Kentucky subgroup in which workers had submitted petitions to the court for placement or some other court-ordered intervention.
In none of these subgroups did placement rates in the control group within one month exceed 12 percent. Hence, even in these more refined (from the standpoint of targeting) subgroups the intended target group, children in imminent risk of placement, was not in evidence.
It should be noted that the results found here occurred despite efforts in this project to improve targeting. In Kentucky and New Jersey, a special screening form, developed by the evaluation team, was employed to rate the risk to children with the intent that cases with intermediate risk would be referred to the program. In Kentucky, efforts were made to divert to family preservation cases that had been referred to the court. In Tennessee, special training efforts were instituted to address concerns about targeting.
Child Safety. Maltreatment after the beginning of service was generally not related to experimental group membership, except for one subgroup in Tennessee. Subsequent maltreatment was measured by the occurrence or nonoccurrence of a substantiated allegation of abuse or neglect following an investigation of such an allegation. The rate of subsequent maltreatment was relatively low, about 18 percent of the families in Kentucky had a substantiated allegation within one year of random assignment; in New Jersey the rate was 12 percent and in Tennessee, 25 percent. In Tennessee, in those families with an allegation within 30 days prior to random assignment, the experimental group children experienced fewer substantiated allegations than children in the control group did.
The findings of little difference between the experimental and control groups in subsequent maltreatment can be read in two ways. It indicates that families served by family preservation were no more likely than families not receiving the services to be subjects of allegations of harm. In this sense, children were, largely, kept safely at home while receiving family preservation services. However, children in both groups were primarily in their homes, and family preservation did not result in lower incidence of maltreatment compared with children in the control group.
Subgroups. In an effort to identify groups of cases for which family preservation is effective, subgroups of Kentucky, New Jersey, and Philadelphia cases were examined.(7) Subgroups were defined in terms of problems of the family (e.g., substance abuse, financial difficulties, and depression) and family structure. Within these subgroups, experimental and control groups were compared on placement and substantiated allegations after random assignment. Two significant differences were found. Among single mothers in New Jersey, those in the experimental group were less likely to have a subsequent substantiated allegation than those in the control group. Among families in Philadelphia who identified a child having problems with school, those in the experimental group were more likely to have a substantial allegation than those in the control group. No subgroups were found in which there were effects on placement in any state.
Family Functioning. In a few areas of family functioning, across states, families in the experimental group appeared to be doing better at the end of services. There were very few differences at the year follow-up and in changes over time. Those differences that did appear (primarily at the end of services) were not consistent across states and were not maintained. Family functioning was assessed through caretaker and caregiver interviews at three points in time shortly after the beginning of services, four to six week later (at the end of services for the Homebuilders group), and again a year after services began. Differences between groups at post treatment, follow-up, and change over time are presented in Table 6.
| Area | Post treatment | Follow-up (1 year after start of treatment) | Change over time |
|---|---|---|---|
| Life events | |||
| Positive life events | KY: Ø NJ: Ø TN: Ø PA: Fewer experimentals experienced positive life events | KY: Ø NJ: Ø TN: Ø PA: Ø | KY: Ø NJ: Ø TN: Ø PA: Ø |
| Negative life events | KY: Ø NJ: Ø TN: Ø PA: Ø | KY: Ø NJ: Ø TN: Ø PA: Ø | KY: Ø NJ: Ø TN: Ø PA: Ø |
| Depression | KY: Ø NJ: Ø TN: Ø PA: Ø | KY: Ø NJ: Ø TN: Ø PA: Ø | KY: Ø NJ: Ø TN: Ø PA: Ø |
| Family problems, individual items | KY: Ø NJ: fewer experimentals not enough money for food, rent, or clothing TN: fewer experimentals had few or no friends PA: Ø | KY: Ø NJ: Ø TN: Ø PA: Ø | N/A |
| Economic functioning | |||
| Individual items | KY: Ø NJ: fewer experimentals difficulty paying rent and buying clothes TN: Ø PA: Ø | KY: Ø NJ: Ø TN: fewer experimentals having difficulty paying rent PA: more experimentals having difficulty buying food and clothes | N/A |
| Scale | KY: Ø NJ: experimental average lower (better) TN: Ø PA: Ø | KY: Ø NJ: Ø TN: Ø PA: Ø | KY: Ø NJ: Ø TN: Ø PA: Ø |
| Household condition | |||
| Individual items | KY: experimentals had fewer broken windows or doors NJ: Ø TN: more experimentals in unsafe building because of illegal acts PA: Ø | KY: Ø NJ: Ø TN: Ø PA: more experimentals reporting not enough basic necessities | N/A |
| Scale | KY: Ø NJ: Ø TN: Ø PA: Ø | KY: Ø NJ: Ø TN: Ø PA: Experimental group reporting more problems in household condition | KY: Ø NJ: Ø TN: Ø PA: Ø |
| Child care practices | |||
| Individual items | KY: fewer experimentals used punishment for not finishing food NJ: experimentals less often got out of control when punishing child and more often encouraged child to read a book TN: more experimentals went to amusement park, pool, or picnic PA: Ø | KY: Ø NJ: Ø TN: Ø PA: Ø | |
| Positive scale | KY: Ø NJ: Ø TN: Ø PA: Ø | KY: Ø NJ: Ø TN: Ø PA: Ø | KY: Ø NJ: Ø TN: Ø PA: Ø |
| Negative scale | KY: Ø NJ: experimentals lower (better) TN: Ø PA: Ø | KY: Ø NJ: Ø TN: Ø PA: Ø | KY: Ø NJ: Ø TN: Ø PA: Ø |
| Punishment | KY: Ø NJ: experimentals lower (better) TN: Ø PA: Ø | KY: Ø NJ: Ø TN: Ø PA: Ø | KY: Ø NJ: Ø TN: Ø PA: Ø |
| Caretaker depression | KY: Ø NJ: Ø TN: Ø PA: Ø | KY: Ø NJ: Ø TN: Ø PA: Ø | KY: Ø NJ: Ø TN: Ø PA: Ø |
| Child behavior | |||
| Aggression | KY: Ø NJ: Ø TN: Ø PA: Ø | KY: Ø NJ: Ø TN: Ø PA: Ø | KY: Ø NJ: Ø TN: Ø PA: Ø |
| School problems | KY: Ø NJ: Ø TN: Ø PA: Ø | KY: Ø NJ: Ø TN: Ø PA: Ø | KY: Ø NJ: Ø TN: Ø PA: Ø |
| Positive child behaviors | KY: Ø NJ: Ø TN: Ø PA: Ø | KY: Ø NJ: Ø TN: Ø PA: Ø | KY: Ø NJ: Ø TN: Ø PA: Ø |
| Negative child behaviors | KY: Ø NJ: experimental group lower (better) TN: Ø PA: Ø | KY: Ø NJ: Ø TN: Ø PA: Ø | KY: Ø NJ: Ø TN: Ø PA: Ø |
| Overall assessment of improvement | KY: experimentals, greater improvement NJ: experimentals, greater improvement TN: Ø PA: Ø | KY: Ø NJ: Ø TN: Ø PA: Ø | N/A |
| Caseworker report of caretaker functioning | |||
| Individual items | KY: Ø NJ: control group higher (better) in ability in giving affection and providing learning opportunities TN: experimental group higher (better) on five items PA: Ø | N/A | KY: control group had more positive change in respecting child's opinions NJ: control group had more positive change in respecting child's opinions TN: experimental group more positive change on setting firm and consistent limits PA: Ø |
| Scale | KY: Ø NJ: Ø TN: experimental group higher (better) PA: Ø | N/A | KY: Ø NJ: Ø TN: Ø PA: Ø |
| Caseworker report of household condition | KY: control group better NJ: control group better TN: Ø PA: control group worse | N/A | KY: Ø NJ: Ø TN: Ø PA: Ø |
| Caseworker report of caretaker problems | KY: experimentals more problems NJ: Ø TN: Ø PA: Ø | N/A | KY: Ø NJ: Ø TN: experimentals declined more PA: Ø |
| Caseworker report of child problems | KY: Ø NJ: Ø TN: Ø PA: Ø | N/A | KY: Ø NJ: Ø TN: Ø PA: Ø |
| Ø denotes that differences between groups were not significant at p =.05; N/A denotes not applicable. | |||
Areas assessed included life events, economic functioning, household condition, child care practices, caretaker depression, child behavior, and caretaker functioning. It can be said that family preservation services may have small, apparently short-term, effects on some areas of functioning. There was one item with some consistency across sites, the overall assessment of improvement by caretakers. At post treatment, a significantly larger proportion of experimental group caretakers in Kentucky and New Jersey generally thought there was "great improvement" in their lives. In Tennessee and Philadelphia, although not significant, results tended in the same direction.
Implications
The findings of this study are not new. A number of previous evaluations with relatively rigorous designs have failed to produce evidence that family preservation programs with varying approaches to service have placement prevention effects or have more than minimal benefits in improved family or child functioning. The work reported here may be thought of as four independent evaluations in four states, adding to the set of previous studies with similar results, this time focusing on Homebuilders programs. The accumulation of the findings from a number of studies in several states, with varying measures of outcome, is compelling.
The findings should not be taken as showing that these programs serve no useful purpose in the child welfare system. The results can be seen as a challenge to keep trying, to find new ways to deal with the problems of families in the child welfare system. The findings indicate the grave difficulties facing those who devise approaches to these problems. Failure in such undertakings should not be surprising and those who risk trying to find solutions should not be punished when evaluations such as this indicate they may have come up short.
The accumulation of findings suggests that the functions, target group, and characteristics of services in programs such as this need to be rethought. Obviously, function, target group, and services are closely intertwined. The foremost of these issues concerns the objectives of the programs. A number of observers have suggested that placement prevention be abandoned as the central objective in intensive, family preservation services in favor of other objectives, notably the improvement of family and child functioning. Targeting these services on families at risk of placement is unlikely to be successful. So if these services are to continue, they will continue to serve "in-home" cases and families in which there has been a substantiated allegation of abuse or neglect or serious conflicts between parents and children where children remain in the home. Many, if not most, of these "intact" families need help. Relatively intensive and relatively short-term services such as those provided by family preservation programs are one source of such help. In this respect, family preservation programs can be thought of as an important part of the continuum of child welfare services.
Another question that program designers must address is that of specialization. Subgroups for which the program was successful were not found, but these programs are quite general in character, and thus may sacrifice some of the benefits of specialization. Those benefits are a clearer focus of services, a tighter target group definition, specification of service characteristics (such as length and intensity based on needs of the target group), and the development of more specific competencies on the part of workers. Specialization could be in terms of problems (e.g., substance abuse) or characteristics of clients (young, isolated mothers). There are clear drawbacks to specialization, including the tendency to define problems in terms of the service one offers. Furthermore, limiting target groups inherently limits the impact of programs. Nonetheless, it may be better to mount a series of small programs rather than putting all of one's resources into large, undifferentiated efforts.
Program planners must also address the issue of length and intensity. The extent to which the intensive, short-term, crisis approach fits the needs of child welfare clients should be reexamined. The lives of these families are often full of difficulties--externally imposed and internally generated--such that their problems are better characterized as chronic, rather than crisis. Short-term, intensive services may be useful for families with chronic difficulties, but those services are unlikely to solve, or make much of a dent in the underlying problems. Of course, the hope is family preservation programs will be able to connect families with on-going services to treat more chronic problems. But, that appears to happen far less than needed. The central point here is that we need a range of service lengths and service intensities to meet the needs of child welfare clients. It is essential that policy makers, planners, and program providers maintain realistic expectations of the effects of short-term family preservation programs.
[Go To Contents]
Endnotes
1. This is one of two reports completed for the evaluation. A previous report, The Evaluation of the New York City HomeRebuilders Demonstration reported on a program designed to facilitate the reunification of children in foster care with their families.
2. As to be expected with any program, some of the families assigned to family preservation programs did not receive the services or received a minimal dosage of the services. In addition, a small number of the families in the control group were actually provided family preservation services. To address these issues, analyses were conducted in which these cases were dropped (secondary analysis).
3. J. Littell and J. Schuerman. (1995). A Synthesis of Research on Family Preservation and Family Reunification. http://aspe.hhs.gov/hsp/cyp/fplitrev.htm.
4. Jill Kinney, David Haapala, and Charlotte Booth. (1991). Keeping Families Together: The Homebuilders Model. New York: Aldine de Gruyter.
5. It should be noted that the most rigorous approach to analysis requires that cases be maintained in the groups to which they were randomly assigned. Random assignment is used to assure that the groups are as similar as possible at the outset of service. Removing cases from the groups or switching cases from one group to another threatens group equality and allows for the possibility that post-treatment differences could be explained by factors other than service. In particular, it is likely that violations and minimal service cases differ in systematic ways from other cases. Hence, the secondary analyses should be viewed with caution.
6. The Philadelphia FPS program did not target imminent-risk children.
7. The number of cases in Tennessee was too small to allow subgroup analysis.