Evaluating Two Approaches to Case Management: Implementation, Participation Patterns, Costs, and Three-Year Impacts of the Columbus Welfare-to-Work Program

Contents

1. The Findings in Brief
2. Historical Context of Integrated and Traditional Case Management
3. Integrated and Traditional Case Management in Columbus
4. The Evaluation in Columbus
   
   1. Research Design
   2. Characteristics of Sample Members
   3. Environment in Columbus
5. Program Services and Messages
6. Program Costs
7. Program Impacts on Receipt of Education redentials, Employment, Welfare, and Income
8. Discussion and Implications of the Findings
9. Future Research

Endnotes

Welfare program case management is usually organized in one of two ways. Under traditional case management, welfare recipients interact with two separate workers: one who deals with welfare eligibility and payment issues, often called income maintenance, and one who deals with employment and training issues. Under integrated case management, welfare recipients work with only one staff member who handles both the income maintenance and employment and training aspects of their case. Although both strategies have certain advantages  for example, the traditional structure allows staff members to specialize in one particular role, and the integrated structure allows staff members to quickly emphasize the importance of employment and eliminates failures in communication between staff members  little information exists on the effects of the two approaches.

This report presents the results of a random assignment study designed to evaluate the two case management approaches, and thus it addresses some longstanding issues in the management of welfare programs. The study was conducted in Columbus (Franklin County), Ohio, as part of the National Evaluation of Welfare-to-Work Strategies (NEWWS Evaluation), a large-scale evaluation of 11 welfare-to-work programs in seven sites across the nation. The evaluation is being conducted by the Manpower Demonstration Research Corporation (MDRC), under contract to the U.S. Department of Health and Human Services, with support from the U.S. Department of Education.⁽¹⁾ For the study, Columbus operated two separate welfare-to-work programs: one that used integrated case management, referred to in this report as the integrated program, and one that used traditional case management, referred to as the traditional program. Apart from the case management difference, the welfare-to-work programs were the same: They required welfare recipients to participate in activities to build their skills and eventually move into the labor market; provided child care and other services to support this participation; and penalized those who did not follow program rules by reducing their cash grant. Participants in the programs were also subject to the same public assistance eligibility and payment system.

This report provides information on how the integrated and traditional programs were implemented, how they affected participation in employment-related activities, and the costs of providing employment-related services in the two programs. It also discusses program effects, measured three years after sample members' entry into the study, on employment, earnings, and welfare receipt. (The final report in the NEWWS Evaluation will present program effects measured five years after study entry.) To facilitate this assessment, from 1992 to 1994 over 7,000 single-parent welfare applicants and recipients, who were determined to be mandatory for the Columbus welfare-to-work program, were randomly assigned for the evaluation. The study's rigorous research design allows researchers to determine the effects of each program as well as the relative effects of the programs, thus providing two types of information.

First, the report describes and evaluates the effects of two mandatory welfare-to-work programs relative to the effects of no special welfare-to-work program. In contrast to many previously studied programs that emphasized skills-building, which engaged most participants in basic education classes, the Columbus programs engaged many people in basic education but also engaged many others in post-secondary education, primarily at two-year colleges.

Second, this report compares the effectiveness of a welfare-to-work program that used integrated case management with the effectiveness of one that used traditional case management. Because other program features were the same, these comparisons indicate the relative effectiveness of the two case management approaches.⁽²⁾

Columbus's integrated and traditional programs were operated under the Family Support Act (FSA) of 1988. The FSA required states to provide education, employment, and support services to Aid to Families with Dependent Children (AFDC) recipients, who were, in turn, required to participate in the Job Opportunities and Basic Skills Training (JOBS) program created by the act to equip them for work. In 1996, the Personal Responsibility and Work Opportunity Reconciliation Act replaced AFDC with a block grant program, Temporary Assistance for Needy Families (TANF). The law limits most families to five years of federal assistance, offers states financial incentives to run mandatory, work-focused welfare-to-work programs, and requires states to meet relatively high work participation rates or face reductions in their block gran t. The 1996 law's overarching goal is similar to the FSA's: to foster the economic self-sufficiency of welfare recipients through increased employment and decreased welfare receipt. Columbus began operating its TANF program in October 1997, after the follow-up period covered in this report.⁽³⁾

[Go To Contents]

I. The Findings in Brief

Key findings from this report include the following:

• Integrated case managers provided more personalized attention than traditional case managers and more closely monitored participation in program activities. More integrated staff than traditional staff said that they tried to learn in depth about the recipients they worked with and provided positive reinforcement to them. Integrated staff received more timely attendance information from service providers and more quickly contacted participants about attendance problems.
• The integrated program engaged more people in welfare-to-work activities than the traditional program. A higher proportion of recipients in the integrated program attended a JOBS orientation and participated in JOBS activities. This probably reflects integrated staff members' better participation monitoring and follow-up. Also, recipients in the integrated program may have taken the threat of cash grant reductions for noncompliance more seriously than recipients in the traditional program because integrated case managers could reduce grants themselves rather than relying on an income maintenance worker to do so.
• Sanctioning rates in the programs were similar and very high. The rate of "initiating" a sanction, however, was higher in the traditional program. Over a third of sample members in each program had their cash grant reduced because of noncompliance with program rules. More recipients in the traditional program, however, had a sanction initiated (the case manager decided that a sanction should be imposed), which means that fewer of those for whom a sanction was initiated in the traditional program actually had their grant reduced. This probably reflects the fact that traditional case managers had to rely on another staff member to impose sanctions, and, because the case managers did not deal with the eligibility aspects of cases, that they probably initiated sanctions for some people who were no longer receiving cash assistance or were no longer mandatory for JOBS.
• The integrated program had somewhat higher two-year costs for employment-related services than the traditional program. This difference reflects higher expenditures for vocational training and case management. A future benefit-cost analysis will include estimates of the cost of income maintenance services and thus will provide the bottom line on the relative costs of the programs.
• The Columbus programs increased earnings. Over three years, the integrated and traditional programs boosted average earnings by about $1,000, or 10 percent, relative to the control group average.
• Both programs reduced welfare receipt and payments, but the effects of the integrated program were somewhat larger. Over the three-year follow-up period, the integrated program reduced time on cash assistance by about 2 1/2 months and reduced three-year welfare expenditures by 15 percent. The traditional program, in comparison, reduced welfare receipt by about 1 2/3 months and reduced expenditures by 11 percent. Integrated case managers more quickly closed cash assistance cases and were better able to detect individuals who should not be receiving welfare than traditional case managers.
• Neither program increased sample members' average combined income from earnings, cash assistance, and Food Stamps. Earnings gains did not exceed public assistance decreases (both programs decreased Food Stamp payments).
• For sample members who did not have a high school diploma or GED when they entered the study, the integrated program produced larger earnings gains and welfare reductions than the traditional program. It is unclear why the integrated program produced larger effects for this subgroup. It may be that the closer monitoring and higher level of personalized attention and encouragement of the integrated approach especially benefited these more disadvantaged recipients.

Overall, the results show that Columbus ran two moderately effective welfare-to-work programs. Both engaged many welfare recipients in education and training, and, over three years, increased their earnings and decreased their welfare receipt. Additional follow-up, to be presented in the evaluation's final report, will show whether these effects continue in the fourth and fifth years following study entry.

The results also provide evidence that an integrated case management approach can yield additional effects beyond those of a traditional approach  namely, higher participation rates and somewhat larger welfare reductions. It is important to note that Columbus had sufficient program services and an uncommon degree of administrative and clerical support. Integrated case managers found balancing employment services with income maintenance to be demanding even with these supports; without them, they may have found the work to be overwhelming.

The remainder of the summary presents the findings in more detail. Following a brief discussion of the history of the two case management approaches and the operation of the approaches in Columbus, the summary provides some detail about the evaluation. Then it discusses the programs' services, messages, and costs, and their effects on an array of outcomes, including employment, welfare, and income. The summary concludes with a discussion of the findings.

[Go To Contents]

II. Historical Context of Integrated and Traditional Case Management

The idea of administering income maintenance together with employment services and other social services is not new.⁽⁴⁾ After the Social Security Amendments of 1962, which increased federal compensation to state welfare agencies for administrative costs related to social services, most states adopted what was called a casework model. Welfare departments hired caseworkers to review applications for welfare and to attempt to "rehabilitate" recipients so that they would become self-supporting. Supporters of the casework model believed that it would allow welfare staff to show concern for recipients during the course of income maintenance discussions and respond to problems, and make it easier for recipients to request services.

Some people, however, criticized the casework model. In many states, staff members hired to perform casework were not professionally trained and did not know what to look for or how to confront recipients about the problems they observed. Few "hard" services, such as job training, placement assistance, or substance abuse treatment, were provided. Professional social workers argued that "the money function disables or overwhelms the social services."⁽⁵⁾ Conservative lawmakers in Congress feared that liberal caseworkers authorized benefits to which individuals were not entitled. Welfare rights and civil rights groups objected to the assumption that welfare recipients needed rehabilitation and attacked the home visits as an invasion of privacy. Responding to these criticisms, in 1967 the U.S. Department of Health, Education, and Welfare (HEW) issued a directive that urged states to reorganize the administration of their welfare programs by creating separate line agencies to determine welfare eligibility and provide social services.

The 1967 Work Incentive (WIN) program directed some AFDC recipients to participate in employment-related activities and provided funding for these activities. The WIN program was jointly administered by HEW and the U.S. Department of Labor, which fostered the separation of income maintenance and employment services. This administrative structure was replicated at the state and local levels in most states, resulting in a system in which welfare staff generally referred recipients outside the income maintenance office to employment security agencies for WIN assessment and services.

By the 1970s, the separation of social and employment services from income maintenance left most welfare offices focused on determining eligibility, authorizing welfare grants, and distributing welfare checks. Many agencies that once recruited college graduates to do casework downgraded the income maintenance role to a clerical level. A goal of minimizing AFDC payment errors replaced the previous decade's goal of "rehabilitating" welfare recipients.

The FSA of 1988, which created the JOBS program, made state welfare agencies directly accountable for enrolling welfare recipients in education and work-related services. As under WIN, most states continued to separate income maintenance and employment-related services, although a 1992 survey found that 17 states operated programs in which JOBS case managers performed an integrated income maintenance and JOBS role. (As this report was being prepared, information was not yet available on how many states have combined income maintenance and employment services under the 1996 federal welfare reform law.)

Both case management approaches can be argued to have certain advantages and disadvantages. The separation of income maintenance from employment and training tasks allows each staff member to specialize in a particular role. It can also allow the employment services case managers to develop a distinct and often more prestigious professional identity. Common criticisms of this model are that a lack of coordination between income maintenance and employment and training services may prevent the quick enrollment of welfare recipients in work activities or may hinder the imposition of penalties on individuals who do not comply with work participation requirements.

By combining the roles of income maintenance and employment and training in one position, the integrated approach eliminates communication breakdowns between different staff members. Integration also allows staff to quickly emphasize the importance of employment. Two prominent welfare scholars suggested that integration may change the "eligibility-compliance culture" of the average welfare office to a "self-sufficiency culture"  that is, one that structures "interactions and expectations around work and preparation for work, with most of the attention of clients and workers devoted to moving off welfare rather than to validating the credential for staying on it."⁽⁶⁾ A common criticism of integrated case management, however, is that the two functions may overwhelm staff members, and, because they must deal with welfare payments each month, this may lead them to pay less attention to employment and training.

[Go To Contents]

III. Integrated and Traditional Case Management in Columbus

Table 1 summarizes the primary duties of line staff in the integrated and traditional programs in Columbus. In the traditional program, income maintenance (IM) workers determined eligibility for and authorized public assistance benefits provided by the welfare department, including cash assistance, Food Stamps, and Medicaid. They also made changes in benefit amounts as family composition changed or as recipients found work, and they imposed financial sanctions at the request of JOBS case managers. JOBS case managers conducted JOBS orientation sessions, assessed recipients' skills and support service needs, assigned them to JOBS activities, monitored their attendance and progress, and initiated sanctions for those who did not comply with program requirements. In the integrated program, integrated case managers performed all these duties.

Evaluation designers had planned that integrated case managers would carry relatively small caseloads so they could work closely with each case. Various factors caused integrated caseloads to be somewhat larger than intended. As Table 1 shows, caseloads averaged about 260 for both IM workers and JOBS case managers and 140 for integrated case managers (rather than 100 as Evaluation designers originally planned). In other words, on average, every two staff members in the traditional program worked with about 260 recipients, and every two staff members in the integrated program worked with about 280 recipients. Therefore, the evaluation in Columbus is comparing the effectiveness of integrated and traditional case management approaches with similar recipient-to-staff ratios. Any differences that exist between the programs' outcomes can be attributed to the case management approach.

[Go To Contents]

IV. The Evaluation in Columbus

A. Research Design

During the period studied, Columbus required all welfare recipients whose youngest child was at least 3 years old and who did not meet federal exemption criteria to participate in the JOBS welfare-to-work program. (Exemption reasons included working 30 hours or more per week, being ill or incapacitated or caring for an ill or incapacitated household member, being of advanced age, being in at least the second trimester of pregnancy, or living in a remote area that made program activities inaccessible.) For the evaluation, between September 1992 and July 1994, 7,242 JOBS-mandatory, single-parent welfare applicants and recipients were assigned, at random, to one of three groups:

• the integrated group, whose members were required to participate in the integrated JOBS program or face a reduction in their cash grant (a financial sanction);
• the traditional group, whose members were required to participate in the traditional JOBS program or face a financial sanction; or
• the control group, whose members were neither required nor eligible to participate in any special welfare-to-work program. (Control group members received income maintenance services. They could seek out employment-related services available in the community and, if they did, could receive child care assistance from the welfare department.)

Because people were assigned to one of the three groups through a random process, any differences that emerge over time between the groups' outcomes -- for example, in average earnings or average welfare payments  can reliably be attributed to the programs.

The three-way design allows researchers to make two types of rigorous comparisons. First, estimates of the net effects of each program can be made by comparing outcomes of the integrated group with outcomes of the control group, and by comparing outcomes of the traditional group with outcomes of the control group. (The integrated and traditional groups are also referred to as program groups in this report.) Second, estimates of the differential effects of the programs can be made by comparing outcomes of the integrated group with outcomes of the traditional group; because the income maintenance and employment services in the two programs were the same, the differential effects represent the relative effectiveness of the two case management approaches. All these differences in outcomes are referred to as program effects or impacts.

The fact that random assignment occurred at the welfare office when people were referred to JOBS affects how the report's results should be interpreted. First, the impacts reflect the effects not only of JOBS services and mandates, but also of the referral to JOBS and any related follow-up, such as sanctioning for people who did not attend an orientation session. Second, outcomes for sample members who did not attend a JOBS orientation, and thus did not receive any program services, are averaged together with those of orientation attendees; this may dilute the estimate of the effects of the welfare-to-work program services and mandates.

B. Characteristics of Sample Members

Information on various characteristics of sample members was collected at random assignment. Most sample members were women, and roughly half were white and half were African-American. Typical sample members had limited experience in the labor market: Fewer than half reported that they had ever worked full time for six months or longer for one employer, and fewer than a third reported that they had worked for pay in the year before random assignment. Nearly three-fifths had received a high school diploma or GED certificate.

C. Environment in Columbus

Between 1992 and 1997, the period covered in this report, Columbus was a growing metropolitan area with a population of close to 1 million. The labor market was robust, with a low unemployment rate that decreased throughout the period (to 2.7 percent in 1997), and substantial employment growth. Over the follow-up period, the county welfare caseload decreased by almost a third.

[Go To Contents]

V. Program Services and Messages

This report presents findings on the implementation of the integrated and traditional programs, based primarily on interviews and surveys of line staff and supervisors and observations of program activities. Information on recipients' participation in employment-related activities was obtained from reviews of program case files and a survey of sample members administered two years after random assignment. Following are highlights of the implementation and participation findings.

The Columbus programs had plentiful resources, good facilities, and extensive administrative support.

Welfare administrators in Columbus placed a high priority on the JOBS program; they considered it the centerpiece of an agency-wide mission to make welfare temporary and employment-focused. Unavailability of program services was rarely, if ever, a problem. The JOBS center, physically separate from the welfare office, housed the employment and training staff for the integrated and traditional programs. The center also provided spacious classrooms for basic education and job search classes; offices for state employment services staff, and county alcohol, substance abuse, and mental health workers; and a child care facility for children between ages 2 1/2 and 5.

The programs provided line staff with an unusual level of administrative support. Columbus had a child care unit that connected parents with child care providers and a resource unit that collected JOBS activity attendance information and provided it to case managers. Columbus used an automated case record information system that contained information on individuals' past public assistance benefits, JOBS activity assignments, and sanctions for noncompliance. The system guided staff through the welfare eligibility determination process and the JOBS assessment.

Despite larger-than-intended caseloads, integrated case managers, aided by various program supports, successfully performed both their income maintenance and employment and training duties.

For integrated workers whose caseloads are too large, their income maintenance role may overshadow their employment and training role, particularly if management emphasizes income maintenance. Although caseloads in Columbus were larger than planned, and perhaps larger than ideal, integrated staff spent much of their time focused on employment and training issues.

As noted above, program administrators in Columbus emphasized the importance of employment and training, and staff received substantial administrative support. In addition, before starting work integrated case managers received four weeks of training on income maintenance procedures and the automated case management system and one week of training on JOBS procedures (traditional JOBS case managers also received JOBS training). Additional training (for all staff) was provided over time, as part of an agency-wide effort to improve staff performance.

Integrated case managers provided more personalized attention than traditional case managers and more closely monitored participation in program activities.

More integrated case managers than traditional JOBS case managers said that they tried to learn in depth about the recipients they worked with and provided positive reinforcement to them. Program participants corroborated this difference: More recipients in the integrated program than the traditional program said that their case manager knew a lot about them and their family and believed that program staff would help them resolve problems affecting their participation in activities. Integrated staff also more quickly received attendance information from service providers and contacted participants about attendance problems.

The integrated and traditional programs emphasized skills-building prior to entry into the labor market rather than immediate employment.

This emphasis was based on the belief that an initial investment in the skills levels of welfare recipients would allow them to eventually obtain higher-paying and more secure jobs. The programs did not have a specific prescribed activity sequence, but staff strongly encouraged people who did not have a high school diploma or GED certificate to attend basic education classes and earn a diploma or GED, and they encouraged many of those who already had a such a credential to attend vocational training classes or post-secondary education or to participate in work experience before actively seeking a job. Staff referred only the most employable recipients to job search services  typically those who had a high school diploma or GED, some work experience, and no serious problems, such as substance abuse, that might interfere with working.

The programs substantially increased participation in employment-related activities.

Many welfare recipients take part in employment-related activities without the intervention of a welfare-to-work program. For a program to make a difference, it must engage more people than would have volunteered for activities available in the community. For recipients who did not have a high school diploma or GED at random assignment, survey responses show that the programs produced large increases in participation in basic education. For sample members with one of these credentials, the programs substantially increased participation in post-secondary education (most commonly, courses at a two-year college), job search activities, and unpaid work experience. The increases in post-secondary education are large compared with increases found for other programs.

The integrated case management approach engaged more people in the welfare-to-work program than the traditional approach.

As shown in Figure 1, reviews of program case files indicated that a larger proportion of sample members in the integrated program than in the traditional program attended a JOBS orientation, the gateway to program activities. (The asterisks indicate that the difference between the programs' participation levels is statistically significant, that is, not due to chance.) In addition, more integrated group members participated in post-orientation activities, including job search, education, training, life skills workshops, and unpaid work experience.

Figure 1 Two year Participation and Sanction Rates, by Program

These differences probably reflect integrated case managers' closer monitoring of participation and quicker follow-up regarding attendance problems. Integrated group members may also have taken the threat of financial sanction for program noncompliance more seriously than traditional group members because integrated case managers could impose sanctions themselves. The orientation attendance rate may also have been higher because integrated case managers called people in to orientation more quickly than did traditional case managers.

Sanctioning rates in the programs were similar and very high. The rate of "initiating" a sanction, however, was higher in the traditional program than in the integrated program.

Staff in both programs believed that those who receive welfare have an obligation to take part in welfare-to-work activities. They strongly emphasized the program participation mandate and freely used financial sanctions (grant reductions) as a response to recipients' noncompliance with program requirements. As Figure 1 shows, more than a third of those in each program were sanctioned at some point during the follow-up period. (In Columbus, a sanction reduced an average grant by about one-fifth.)

Before a sanction could be imposed, the recipient's case manager had to decide that a financial penalty was in order; in this report, the decision to sanction is referred to as "initiating" a sanction. In both the integrated and traditional programs, some people for whom a sanction was initiated demonstrated good cause for not participating and were not sanctioned. However, more recipients in the traditional group than in the integrated group had a sanction initiated, which means that a smaller proportion of those for whom a sanction was initiated were actually sanctioned in the traditional program than in the integrated program. This difference probably occurred because traditional JOBS case managers could not impose sanctions themselves. In addition, since they did not deal with the income maintenance aspects of cases, they probably initiated a sanction for some people who had not attended a program activity either because they were no longer receiving cash assistance or were no longer mandatory for the JOBS program (and thus could not or should not be sanctioned).

[Go To Contents]

VI. Program Costs

The cost of the employment-related services in the integrated and traditional programs was determined using various sources, including state and county fiscal reports, support service payment records, case file participation records, and sample members' survey responses. As in the other cost analyses in the NEWWS Evaluation, the cost estimates in this report consist of all costs associated with providing employment services and related support services to sample members. They do not include, for either the integrated or traditional program, the costs of authorizing and processing welfare payments (that is, they do not reflect the cost of the IM workers in the traditional program or the cost of the eligibility tasks of the integrated case managers). The five-year benefit-cost analysis, to be presented in the evaluation's final report, will estimate the costs of both employment-related and eligibility services, and thus will provide the bottom line on the differential costs of integrated and traditional case management in Columbus. Key findings on the programs' two-year employment-related costs are presented below.

The integrated program had somewhat higher two-year costs per program group member for employment-related services than the traditional program.

The gross cost per program group member during the two-year follow-up period consists of costs paid by the welfare department and non-welfare agencies for employment-related services while sample members were enrolled in the Columbus programs, as well as for employment and support services after they exited the programs and, in some cases, left welfare. Table 2 shows that this cost was $3,018 in the integrated program and $2,589 in the traditional program.

The programs' net cost is the gross cost minus what would have been spent in the absence of a mandatory welfare-to-work program, as measured by the cost per control group member. Control group members were not eligible to take part in program activities, but could enroll on their own in other employment-related activities in the community and, if they did, were eligible for activity- and employment-related welfare department support services. Thus, control group costs include expenditures for all of the nonprogram activities and support services used by control group members during the follow-up period. Table 2 shows that the two-year net cost per person was $2,149 for the integrated program and $1,720 for the traditional program.

The integrated program had somewhat higher employment-related costs for two main reasons. First, vocational training participants in the integrated group tended to use more expensive services than participants in the traditional group (proprietary schools rather than less expensive nonprofit agencies). Second, integrated employment-related case management costs were somewhat higher. (This does not indicate that the eligibility-related case management costs or total case management costs were higher for the integrated program than for the traditional program.) As noted earlier, caseloads for the integrated case managers were larger than had been planned. If integrated case management had been operated with substantially smaller caseloads, it is very likely that it would have been more expensive.

Similar to the findings for the full sample, the integrated program had somewhat higher employment-related costs than the traditional program for both educational attainment subgroups (those who entered the study with a high school diploma or GED and those who entered the study without such a credential).

[Go To Contents]

VII. Program Impacts on Receipt of Education redentials,Employment, Welfare, and Income

The programs' effects on receipt of education credentials were measured using sample members' responses to a survey administered two years after study entry. Effects on employment and welfare were estimated using automated state unemployment insurance (UI) records and AFDC administrative records data. Following are highlights of the impact findings.

For sample members who did not have a high school diploma or GED at random assignment (nongraduates), the traditional program increased the proportion who received such a credential within two years of entering the study; the integrated program did not.

About 4 percent of nongraduate control group members reported that they had received a high school diploma or GED at some point during the two years following entry into the study. In the traditional group, 13 percent of nongraduates reported that they received such a credential after entering the evaluation. In the integrated group, 9 percent of nongraduates reported receiving a diploma or GED, but the 5 percentage point increase is not statistically significant. Like most welfare-to-work programs studied, neither program in Columbus (despite substantial increases in participation in post-secondary education) increased receipt of a trade certificate, an associate's degree, or a bachelor's degree. Additional sample members may have received a credential after the two-year point, but these effects will not be measured in the evaluation.

The programs raised employment rates and earnings by about the same amount over the three-year follow-up period. They increased earnings primarily because program group members worked at better jobs than their control group counterparts.

Table 3 shows the two programs' effects on employment and earnings. The first set of columns shows effects of the integrated program (integrated-control comparison), and the second set shows effects of the traditional program (traditional-control comparison). The last column shows the difference between outcomes of the integrated and traditional programs (integrated-traditional difference).

During the study period, which was characterized by a very robust labor market, employment rates were high in Columbus, even without the programs' intervention: 78.5 percent of control group members were employed at some point during the three years after entry into the study, and they worked an average of 5.46 quarters (just over 16 months). As the table indicates, both programs produced small increases in employment rates and in the average length of time worked.

Control group members earned an average of $12,027 over the three years (this average includes zeros for people with no earnings). The integrated program boosted three-year earnings by an average of $1,181, or 10 percent, and the traditional program boosted earnings by $1,000, or 8 percent. (The $181 difference between the two program groups' average earnings is not statistically significant.) The Columbus programs increased average earnings primarily because integrated and traditional group members worked for more quarters and earned more per quarter of employment than control group members. This implies that, on average, integrated and traditional group members who worked held better jobs than control group members who worked.

Average quarterly earnings are plotted in the upper panel of Figure 2. As is often found for programs that emphasize building skills prior to finding a job, neither program increased earnings during the first year of follow-up, but did during the second year. (Impacts are illustrated by the distance between the lines on the figure.) By the end of the third year of follow-up, the integrated program's impacts on earnings had decreased, but remained statistically significant. The traditional program's impacts, in contrast, were less consistent during the third year. These patterns suggest that the integrated program will continue to increase employment and earnings during the fourth year of follow-up, but the traditional program may not.

Figure 2: Average Quarterly Earnings and AFDC Payments Over a Three-Year Follow-Up Period

The higher rate of participation in the integrated program did not translate into larger employment and earnings impacts (although quarterly patterns suggest that the integrated program may have more positive results than the traditional program during the fourth year of follow-up). A recently published analysis of participation in welfare-to-work programs found that although a minimum level of participation is necessary to produce employment and earnings impacts, above that threshold there is no linear relationship between participation levels and impacts.⁽⁷⁾ One should not expect, then, that higher participation rates would necessarily yield larger increases in employment and earnings.

Both programs reduced cash assistance receipt and payments. The integrated program's reductions were somewhat larger, probably because integrated staff responded more quickly to changes in sample members' employment and welfare eligibility status and had more knowledge about these changes.

Table 3 shows that control group members received cash assistance for an average of about 21 1/2 months during the three-year follow-up period. The integrated program reduced welfare receipt by more than 2 1/2 months, a decrease of 12 percent relative to the control group average. The traditional program reduced receipt by about 1 2/3 months, or 8 percent.

Control group members received an average of $7,151 in welfare payments during the three-year period. Integrated group members received, on average, 15 percent less in welfare payments than the control group, and traditional group members received 11 percent less. Most of the reduction occurred because integrated and traditional group members spent less time on welfare than their control group counterparts (rather than receiving lower grant amounts).

Average quarterly welfare payments for the research groups are plotted in the lower panel of Figure 2. The programs reduced payments during each year of the follow-up period. The effects grew over time and remained substantial at the end of the three years, which suggests that the reductions in both programs are very likely to persist during the fourth year of follow-up.

The integrated program's somewhat greater success in reducing welfare receipt and payments, without corresponding larger increases in employment and earnings, indicates that the integrated case management structure engendered more effective eligibility case management and facilitated case closures. Specifically, integrated case managers closed cases more quickly, on average, than traditional staff. They also closed cases that would have remained open in the traditional program, probably because  through closer and more frequent contact  they were better able to detect individuals who should not be receiving welfare.

Neither program increased average combined income from earnings, cash assistance, and Food Stamps.

One way to measure a program's effect on sample members' economic self-sufficiency is to examine their combined income from earnings, cash assistance, and Food Stamps. (This income measure does not include estimates of the Earned Income Credit, a credit against federal income taxes for low-income taxpayers.) During the three years following random assignment, control group members received, on average, $25,490 from earnings, cash assistance, and Food Stamps. Integrated group members received $24,895 over the same period, 2 percent less than the control group average, and traditional group members received $25,192, 1 percent less. (In addition to reducing cash assistance payments, both programs reduced Food Stamp payments over the three-year period: the integrated program by $697 and the traditional program by $483.) The small decreases in average combined income are not statistically significant.

 

Table 3.
Program Impacts on Employment and Welfare Over a Three-Year Follow-Up period

Outcome	Integrated-Control Comparison				Traditional-Control Comparison
	Integrated Group	Control Group	Difference (Impact)	Percentage Change	Traditional Group	Control Group	Difference (Impact)	Percentage Change	Integrated- Traditional Difference (Impact)
Ever employed, years 1-3 (%)	81.1	78.5	2.6**	3.3	80.7	78.5	2.2**	2.8	0.4
Year 1	60.0	60.1	-0.1	-0.2	59.9	60.1	-0.1	-0.2	0.0
Year 2	65.2	62.9	2.3*	3.7	64.5	62.9	1.6	2.6	0.7
Year 3	68.9	65.3	3.6***	5.5	67.9	65.3	2.6**	3.9	1.0
Quarters employed, years 1-3	5.75	5.46	0.29***	5.3	5.69	5.46	0.23**	4.1	0.06
Year 1	1.64	1.62	0.02	1.0	1.66	1.62	0.04	2.7	-0.03
Year 2	1.97	1.82	0.15***	8.5	1.94	1.82	0.13***	7.0	0.03
Year 3	2.14	2.02	0.12**	5.8	2.08	2.02	0.06	2.8	0.06
Earnings, years 1-3 ($)	13,208	12,027	1,181***	9.8	13,027	12,027	1,000**	8.3	181
Year 1	2,994	2,914	80	2.8	3,099	2,914	185	6.4	-105
Year 2	4,578	3,982	596***	15.0	4,472	3,982	490***	12.3	106
Year 3	5,635	5,131	505***	9.8	5,456	5,131	325*	6.3	180
Ever received AFDC, years 1-3 (%)	96.4	96.9	-0.5	-0.6	96.3	96.9	-0.6	-0.7	0.1
Year 1	95.8	96.6	-0.8	-0.8	96.0	96.6	-0.6	-0.6	-0.2
Year 2	65.1	69.1	-4.0***	-5.7	65.9	69.1	-3.2**	-4.6	-0.8
Year 3	47.0	54.4	-7.4***	-13.6	49.0	54.4	-5.4***	-10.0	-2.0
Months received AFDC, years 1-3	18.87	21.48	-2.61***	-12.2	19.77	21.48	-1.71***	-8.0	-0.90***
Year 1	8.91	9.62	-0.71***	-7.3	9.16	9.62	-0.46***	-4.8	-0.25**
Year 2	5.91	6.79	-0.87***	-12.9	6.22	6.79	-0.57***	-8.4	-0.30**
Year 3	4.04	5.08	-1.03***	-20.4	4.39	5.08	-0.68***	-13.5	-0.35**
AFDC amount, years 1-3 ($)	6,071	7,151	-1,079***	-15.1	6,335	7,151	-816***	-11.4	-264**
Year 1	2,880	3,199	-318***	-10.0	2,950	3,199	-249***	-7.8	-70*
Year 2	1,895	2,270	-375***	-16.5	1,989	2,270	-281***	-12.4	-95**
Year 3	1,297	1,682	-386***	-22.9	1,396	1,682	-286***	-17.0	-99**
Sample size (total=7,242)	2,513	2,159			2,570	2,159
Notes:  A two-tailed t-test was applied to differences between outcomse for the program and control groups and to differences between outcomes for the integrated and traditional program groups. Statical significance levels are indicated as * = 10 percent, ** = 5 percent, and *** = 1 percent.

For sample members who did not have a high school diploma or GED when they entered the study (nongraduates), the integrated program produced larger earnings gains and welfare reductions than the traditional program. For graduates, however, the programs had similar effects.

Table 4 presents a summary of the programs' effects on employment and welfare for the graduate and nongraduate subgroups. The traditional program increased graduates' average earnings over the three-year period by $1,105, or 7 percent; the $633 increase for the integrated program is not statistically significant. (The $473 difference between the integrated and traditional groups' average earnings is not statistically significant.) Both programs decreased the time that graduate sample members received welfare benefits and reduced their average welfare payments. As mentioned earlier, the programs increased graduates' participation in post-secondary education (as well as in job search and unpaid work experience). The final report of the evaluation will track sample members for five years and show whether the participation increases lead to earnings gains later in the follow-up period.

The lower panel of Table 4 shows that the integrated program boosted nongraduates' three-year earnings by $1,730, or 21 percent, compared with an increase of $734, or 9 percent (not statistically significant), for the traditional program. The integrated program decreased months of welfare receipt by 14 percent, compared with 7 percent for the traditional program, and reduced welfare payments by $1,404, compared with $874.

It is unclear why the integrated program produced larger effects among nongraduates than the traditional program. Participation patterns for nongraduates were similar in the two programs, with many attending basic education classes. It may be that the closer monitoring and more personalized attention and encouragement that the integrated approach provided to recipients made more of a difference with a more disadvantaged subgroup.

For graduates and nongraduates, as for the full sample, both programs slightly reduced average combined income from earnings, cash assistance, and Food Stamps, but the reductions were not statistically significant.

[Go To Contents]

VIII. Discussion and Implications of the Findings

Overall, this report shows that both welfare-to-work programs in Columbus were moderately effective. On average, the programs increased sample members' participation in employment-related activities and their earnings and reduced their welfare receipt.

It is important to emphasize, however, that, on average, sample members gained about the same amount in earnings as they lost in public assistance, resulting in no net increase in income from these sources. Thus, the programs' main financial effect for participants, as is true for many welfare-to-work programs, was to replace some welfare dollars with dollars from work. Furthermore, a substantial proportion of integrated and traditional group members were still receiving cash assistance benefits at the end of the three years studied in the report (about one-third of each program group received benefits in the last quarter of follow-up).

This report also shows that, as operated in Columbus, integrated case management can yield some additional effects beyond those of a traditional approach. First, integrated case management generated higher rates of participation in program activities. As discussed earlier in the summary, this probably reflects better monitoring and follow-up; and recipients may have taken the threat of financial sanction more seriously when it came from a staff member who could impose the sanction herself. A higher participation rate is consequential for at least two reasons. Engaging more welfare recipients in a welfare-to-work program helps enforce the social contract idea that people receiving welfare should, in turn, take part in employment-focused services. In addition, under the 1996 federal welfare law, states must meet relatively high work participation rates or face reductions in their TANF block grant; integrated case management may help them do so.

Second, through more effective eligibility case management, the integrated approach generated somewhat larger decreases in time on welfare over the three-year period, and thus somewhat larger welfare savings. Integrated staff were able to close cases more quickly than traditional staff, and, through closer and more frequent contact, were better able to detect individuals who should not have been receiving welfare. In an environment of time-limited welfare receipt, reducing months of welfare receipt is an especially important goal.

Overall, however, the integrated program did not increase earnings more than the traditional program (and neither program increased combined income from earnings, cash assistance, and Food Stamps). Thus, whether the Columbus results suggest that an integrated case management structure is preferable, compared with a separated, traditional structure, depends on the primary goals of a program. Specifically, this study shows that integrated case management may be more effective in increasing participation rates and decreasing welfare receipt, but offers no evidence that it is more (or less) effective in increasing participants' earnings or income.

When interpreting the Columbus results, however, it is important to remember that the program effects occurred in a specific context. First, and most important, the case management models were operated as part of a well-funded, well-run welfare-to-work program. As discussed, staff in Columbus had extensive administrative support, including a sophisticated case record information system, a child care referral unit, and a clerical unit that tracked recipients' attendance in program activities. Program administrators placed a high priority on the employment services aspect of the program, services were plentiful, and staff training was adequate. Integrated case managers found their job to be demanding with these resources and supports; without them, they may have found the work to be overwhelming, and the effects of the integrated program may have been diminished.

Second, the recipient-to-staff ratio in the integrated and traditional programs was approximately the same. The evaluation results do not indicate how the program effects would have differed if the integrated case managers had worked with fewer recipients, as was originally intended. (A previous MDRC evaluation, however, provided some evidence that smaller caseloads do not yield larger program effects.)⁽⁸⁾

IX. Future Research

A future NEWWS Evaluation report will track Columbus sample members for five years. The additional follow-up will show whether the earnings increases and welfare reductions continue and whether the differences between the programs' outcomes remain for the full sample and for nongraduates. It will also indicate whether the substantial increases in post-secondary education among graduates lead to additional increases in earnings. The report will examine the total costs of each program  eligibility-related costs as well as employment-related costs  and compare the financial benefits and costs of the two programs, both for the government and for individuals in the programs. In addition, the report will compare the results for Columbus with the results for the other programs in the NEWWS Evaluation.

Endnotes

1.  Child Trends, as a subcontractor, is conducting analyses of outcomes for young children in three of the sites. Columbus is not included in this substudy.

2.  This report draws on an earlier paper prepared as part of the NEWWS Evaluation: Thomas Brock and Kristen Harknett, "Welfare-to-Work Case Management: A Comparison of Two Models" (paper prepared by MDRC as part of the National Evaluation of Welfare-to-Work Strategies, 1998). A revised version of this paper was published in Social Service Review (December 1998): Thomas Brock and Kristen Harknett, "A Comparison of Two Welfare-to-Work Case Management Models."

3.  Ohio's TANF program, Ohio Works First, shifted the focus from building welfare recipients' skills through education to quickly engaging them in jobs. The program limits recipients to three years of cash benefits, with up to two additional years of benefits available under certain circumstances.

4.  This section is modified from Brock and Harknett, December 1998. It uses information from the following sources: Winifred Bell, Contemporary Social Welfare (New York: Macmillan, 1983); Mary Jo Bane and David T. Ellwood, Welfare Realities: From Rhetoric to Reform (Cambridge, MA: Harvard University Press, 1994); Gordon Hamilton, "Editor's Page" (Social Work 7, no. 1, January 1962); Demetra Smith Nightingale and Lynn C. Burbridge, The Status of State Work-Welfare Programs in 1986: Implications for Welfare Reform (Washington, DC: Urban Institute, 1987); American Public Welfare Association, Status Report on JOBS Case Management Practices (Washington, DC: American Public Welfare Association, 1992).

5.  Hamilton, 1962, p. 128.

6.  Bane and Ellwood, 1994, p. 7.

7.  Gayle Hamilton and Susan Scrivener, Promoting Participation: How to Increase Involvement in Welfare-to-Work Activities (New York: Manpower Demonstration Research Corporation, 1999).

8.  James Riccio, Daniel Friedlander, and Stephen Freedman, GAIN: Benefits, Costs, and Three-Year Impacts of a Welfare-to-Work Program (New York: Manpower Demonstration Research Corporation, 1994).