Performance Improvement 2004

Foreword

This report, Performance Improvement 2004: Evaluation Activities of the U.S. Department of Health and Human Services, complies with Section 241(b) of the Public Health Service Act, as amended by the Preventive Health Amendments of 1993.

The Act directs the Department to “prepare and submit ... a report summarizing the findings of the evaluations conducted under subsection (a)” to the “Committee on Health, Education, Labor, and Pensions of the Senate and the Committee on Energy and Commerce of the House of Representatives”. Section 241(a) of the Act authorizes that a portion of funds appropriated for programs under the act may be set aside for evaluating the “implementation and effectiveness of such programs.”

This is the 10th annual report to Congress. Performance Improvement 2004 provides results- oriented findings regarding the Department’s programs, policies, and strategies. The full text of the report, text of the individual abstracts it contains, and links to the full text of final reports cited herein are available through the Office of the Assistant Secretary for Planning and Evaluation Policy Information Center web site (http://aspe.hhs.gov/pic). Additional information may be obtained by contacting the Federal Contact, evaluation Performer, or Sponsoring Agency listed.

Chapter 1 contains a discussion of evaluation activities and responsibilities; Chapter 2 provides abstracts of studies completed during the preceding year; and the four appendices provide additional useful information.

We hope the readers of this report will find it useful and informative.

Chapter I - Program Evaluation at the Department of Health and Human Services

This Performance Improvement 2004 report presents to Congress a comprehensive summary of evaluation projects completed by the Department. The Office of the Secretary, and agencies comprising the Department, engage in extensive evaluation activities; some are required by statute, others are determined to be essential by an individual agency, the Department, or the President. Evaluation is a core Federal program management responsibility along with strategic planning, policy and budget development, and program operation (Figure 1, Performance Management System).

Performance Management System 

Figure 1

The Department of Health and Human Services (DHHS) is responsible for more than 300 separate programs costing taxpayers over $500 billion annually for health and social service support payments and approximately $70 billion for discretionary programs. DHHS spends about $2.6 billion for research, demonstration, and evaluation activities. Successful evaluations increase the likelihood of effective delivery of public services through these programs and insure that programs are efficient, targeted to their intended clients, and well managed.

Structure of this Report

Evaluations summarized in this year’s report are organized under the Department’s eight broad Strategic Goals consisting of 40 essential Objectives (Appendix A). As articulated in the Department’s FY 2004- 2009 Strategic Plan, the goals are:

Performance Improvement 2004, the 10th annual report in this series, provides summaries of findings of DHHS evaluation projects completed during Fiscal Year 2003 (October 1, 2002 through September 30, 2003). This Chapter provides an overview of the Department’s evolving evaluation responsibilities. Chapter 2 presents the summaries of the evaluations. In addition to the Department’s Strategic Plan Goals and Objectives found in Appendix A, Appendix B contains a statement of each agency’s Mission and Evaluation Program, Appendix C provides a table of the studies by each agency and the Objective in this report where the study may be found, and Appendix D acknowledges the DHHS officials who contributed to the report.

Role of the Evaluation

Programs need to: work better for the individuals served, assure that tax dollars are wisely spent, and achieve the aims set for them by elected representatives. This report to Congress on Performance Improvement reflects a transition to a more strategic and analytic presentation of evaluation studies. With the gradual implementation of a unified Strategic Plan, as required by the Government Performance and Results Act of 1993, the Department recognizes its responsibility both to evaluate programs and to assure that evaluation funds are targeted to address the core goals and objectives the Congress and Executive branch seek. This re-engineering of management practice underscores the important role of evaluations potential to test, weigh, measure and judge the success of management performance, program outputs, and social outcomes.

Evaluation plays an integral role in carrying out the DHHS mission. Assessing various aspects of agency functioning allows staff to identify means of improving individual program performance. DHHS evaluations directly support: (1) helping government officials and members of the Congress make decisions related to programs, policies, budgets, and strategic planning; (2) enabling managers improve program operations and performance; and (3) disseminating evaluation results and methodological tools useful to the larger health and human services community of state and local officials, researchers, advocates, and practitioners for improving the performance of their programs.

Types of Evaluation

For DHHS, evaluation is the assessment of the performance (efficiency, effectiveness, and responsiveness) of DHHS programs or strategies through the analysis of data or information collected systematically and ethically, and the effective use of resulting information in strategic planning, program or policy decision making and program improvement. Evaluations serve one or more of the following objectives (Figure 2, Types of Evaluation):

Figure 2

Evaluation Resources

Evaluation activities of the various DHHS agencies are largely supported through two funding mechanisms: direct use of program funds and use of special legislative set-aside authorities for evaluation. The first is a common mechanism by which program managers have discretionary authority to use appropriated program funds to support contracts that will design and implement evaluation studies, and analyze evaluation data. In some cases, a program’s legislative authority calls for a specially mandated evaluation, and program funds are used directly to support the evaluation.

The second mechanism for evaluation funding is the legislative set-aside authority which permits the Secretary of DHHS to use a portion of overall program funds for evaluation purposes. The largest of such set-aside authorities at DHHS is Section 241 of the Public Health Service (PHS) Act for evaluations conducted by several HHS agencies:

In addition, two staff components in the Office of the Secretary receive funds under this authority to carry out evaluations:

This authority was originally established in 1970, when the Congress amended the Act to permit the DHHS Secretary to use up to 1 percent of appropriated funds to evaluate authorized programs. Section 241 limited the base from which funds could be reserved for evaluations to programs authorized by the PHS Act. Excluded were funds appropriated for the Food and Drug Administration, the Indian Health Service, and certain other programs that were managed by PHS agencies but not authorized by the Act (e.g., HRSA’s Maternal and Child Health Block Grant and CDC’s National Institute for Occupational Safety and Health)¹. In addition, programs may have other authorized sources of funding for evaluation activities². Section 206 of Division E of the Consolidated Appropriations Act, 2004 increased the amount the Secretary could use for evaluation to 2.2 percent.

¹ FDA programs are principally authorized by the Food, Drug and Cosmetic Act and appropriations are provided by the Appropriations Subcommittee on Agriculture, Rural Development, Food and Drug Administration and Related Agencies. IHS programs are principally authorized by the Indian Health Care Improvement Act and the Indian Self- Determination Act; appropriations are provided by the Appropriations Subcommittee on Interior and Related Agencies.

² The Older Americans Act (OAA) specifies that $1.5 million from Title III, and $1.5 million from Title IV are to be available from its annual appropriations to be used for the evaluation of OAA programs. Since 2000, AoA has used those funds for the Performance Outcome Measures Project and its annual national performance measurement surveys. AoA initiated new evaluation studies of Title III-D Health Promotion and Disease Prevention, and Title III-B Supportive Services in FY 2004 and intends to continue evaluating all OAA titles on a rotating basis in the future.

Evaluation Management

Management of evaluations carried out by DHHS agencies and offices and coordinated by ASPE involves: (1) planning and coordination, (2) project oversight, (3) quality assurance, and (4) dissemination of results (Figure 3, Evaluation Management Cycle). A description of each function follows.

Evaluation Management Cycle

Figure 3

Evaluation Planning and Coordination

DHHS agencies, ASPE, the Office of the Inspector General (OIG), and OPHS develop evaluation plans annually in concert with DHHS’s program planning, legislative development, and budgeting cycles. Plan development is coordinated by ASPE. Each agency or office evaluation plan generally states the evaluation priorities, or projects under consideration for implementation. Typically, DHHS evaluation priorities include: congressionally-mandated program evaluations, evaluations of Secretarial program or policy initiatives, assessments of new programs and ones that are candidates for reauthorization, and evaluations that support program performance management and accountability.

DHHS evaluation planning activities are coordinated with three department-wide planning initiatives. First, DHHS evaluation activities support the Department’s strategic planning and performance management activities in several ways. Completed evaluation studies are used in shaping the specific DHHS strategic goals and objectives. Evaluation findings provide an important source of information or evidence about the success of various DHHS programs or policies that collectively make up the strategies to achieve the goals and objectives. The DHHS Strategic Plan highlights evaluations that document efficacy or effectiveness of strategic programs or policies and lists future evaluations that will benefit strategic planning. DHHS agencies use findings from their evaluations to support GPRA annual performance reporting to Congress and program budget justifications.

Second, Congress requests that DHHS coordinate and report to Congress regarding all of its research, demonstration, and evaluation (RD&E) programs to ensure that the results of these projects address DHHS program goals and objectives. ASPE and the Assistant Secretary for Budget, Technology and Finance work together with DHHS agencies to provide the Congress with a special annual research, demonstration, and evaluation budget plan that coincides with the preparation of the President’s fiscal year budget. The plan outlines DHHS agency research, demonstration, and evaluation priorities as related to the Department’s strategic goals and objectives (Figure 4, Evaluation Reporting Cycle).

Evaluation Reporting Cycle

Figure 4

The Secretarially-created Research Coordination Council (chaired by the ASPE, and containing representatives of DHHS agencies) fosters greater interactions among the research programs. The Council’s work includes streamlining research and evaluating Department-wide research priorities to ensure greater efficiencies in research, demonstration, and evaluation. The Council seeks to strengthen DHHS research coordination and planning around key Departmental priorities and themes.

Third, the Secretary must report to the Congress his plan for using PHS evaluation set-aside funds before implementing the plan (Figure 4). Those agencies and offices that use the PHS evaluation set-aside authority -- ACF, AHRQ, CDC, HRSA, NIH, ASPE, OPHS, SAMHSA -- submit a formal plan to ASPE, which coordinates and develops the individual plans into the DHHS report to the Congress on the use of the PHS authority.

Project Oversight

DHHS agencies, the OIG, and ASPE execute annual evaluation plans which involve developing evaluation contracts and disseminating and applying evaluation results. All agencies and their subunits (centers, institutes, and bureaus) coordinate with one another on research/evaluation project planning and release of final reports that relate to work of other DHHS agencies. While there is some oversight responsibility and execution capability in the Office of the Director or Administrator for each agency, the various agency subunits conduct much of the day-to-day evaluation activity.

The OIG performs independent evaluations through its Office of Evaluations and Inspections (OEI). OEI’s mission is to improve DHHS programs by conducting inspections that provide timely, useful, and reliable information and advice to decision makers. These findings of deficiencies or vulnerabilities and recommendations for corrective action are usually disseminated through inspection reports issued by the Inspector General. A summary of individual inspection reports and other OIG reports can be viewed on the Internet at http://oig.hhs.gov/reports.html. OEI provides technical assistance to DHHS agencies in conducting their evaluations.

Quality Assurance and Improvement

Most evaluation projects are developed at the program or office level. The initial quality review is generally conducted by a committee of agency- or office-level policy and planning staff members. Before a project is approved, it is reviewed for technical quality by a second committee with expertise in evaluation methodology. Technical review committees follow a set of criteria for quality evaluation practice established by each agency. ASPE, for example, has a formalized peer review process in which experienced evaluators on staff review, discuss and approve all proposed research projects before they are submitted for funding. Some DHHS agencies have external evaluation review committees composed of evaluation experts from universities and research centers.

Since DHHS began reporting to Congress in 1995 on completed evaluations through the Performance Improvement report series, the Department has focused attention on improving the quality of evaluation studies performed. An Evaluation Review Panel, convened periodically, has contributed insights to DHHS evaluation officers on the strengths and challenges of ensuring quality evaluation studies. DHHS evaluation officers have had opportunities to discuss these strengths and challenges and identify steps to improve agency evaluation projects. Evaluation Officers meet regularly to collaborate regarding evaluation-related activities and to share information regarding evaluation projects planned or under way.

Dissemination of Evaluation Reports

Maintaining on-line electronic report libraries and distributing information on evaluation results is an important component of DHHS evaluation management. The Department’s information and reports on major evaluations is available centrally through the website of the DHHS Policy Information Center (PIC), located at the following address: http://aspe.hhs.gov/pic. ASPE’s PIC website offers users an opportunity to search – by key word or by selected program or policy topics – the departmental evaluation report database and electronic report library maintained by ASPE. The PIC contains over 8,500 completed and in-progress evaluation and policy research studies conducted by the Department, as well as key studies completed outside of DHHS by the U.S. Government Accountability Office (GAO) and private foundations.

The results of DHHS evaluations are disseminated through targeted distribution of final reports, articles in refereed journals, and presentations at professional meetings and conferences. Although individual DHHS agencies have primary responsibility for disseminating results, the ASPE continues its Department-wide efforts to expand dissemination of evaluation results to the larger research and practice communities through centralized computer communications and publications available on the PIC website.

Chapter II - Summaries with Findings for Completed Evaluations

This Chapter presents brief abstracts summarizing the purposes and findings for each DHHS evaluation completed between October 1, 2002 and September 30, 2003. As discussed in Chapter 1, each study is presented under the Strategic Objective (Appendix A) it most supports. This thematic grouping provides an immediate view of the program and research areas emphasized during the previous year and aids in identifying areas for possible future inquiry. Future evaluation priorities will also be guided by the results of oversight not only by the Congress but also by the Executive Branch through such management mechanisms as the Performance Assessment Reporting Tool (PART) which is already yielding specific recommendations for future evaluative inquiry and program validation.

To find abstracts in this Chapter for studies funded by particular agencies, refer to the Objectives identified in Appendix C. Not all Objectives are represented among the completed studies reported in Chapter 2; a multitude of studies are also already in progress and development of the new fiscal year’s research, demonstration, and evaluation agenda will add other studies to the roster throughout the year. Further information about completed and in-progress reports can be accessed on-line at http://aspe.hhs.gov/pic.

Goal 1 - Reduce the Major Threats to the Health and Well-being of Americans

Objective 1.1 - Reduce Behavioral and Other Factors that Contribute to the Development of Chronic Diseases

Partner Participatory Assessment Tool: Functioning and Operation of the National Diabetes Education Program's Minority Work Groups

This evaluation assessed how well minority work-groups collaborate to deliver National Diabetes Education Program messages. Minority populations, including African Americans, American Indians/Alaska Natives, Hispanics/Latinos, Asian Americans, and Pacific Islanders, are disproportionately affected by diabetes. The National Diabetes Education Program (NDEP), through four minority work groups representing these populations, have created culturally appropriate education messages targeted to these groups. This evaluation assessed activities in the following areas: 1) design and development of culturally appropriate messages and products, 2) delivering the intervention, 3) reaching the intended audiences, and, 4) producing intended changes in awareness, attitudes and reported behavior. The research was managed jointly by the Division of Diabetes Translation (NCCDPHP) and the Division of Health Communication, Office of Communication (OD). The results identified areas of the partnership that were working well and those needing strengthening or improvement. Specific opportunities for improvement regarding management of the work group process that were identified included the communication among work groups, the leadership of the work groups and of NDEP, and the resource allocation. 

PIC ID: 7430
Agency Sponsor: CDC-NCCDPHP, National Center for Chronic Disease Prevention and Health Promotion
Federal Contact: Wong, Faye, 770-448-5037
Performer: Macro International, Inc., Calverton, MD

Tobacco Education Networks in Communities of Color: Identifying Barriers to Success: Final Report

The study examines the development of ethnic-specific Tobacco Education Networks in seven states. Several state tobacco control programs are at various stages of establishing tobacco prevention and control initiatives, such as statewide ethnic-specific Tobacco Education Networks for African-Americans, Hispanics, Asian/Pacific Islanders, American Indians, and other identified groups. These initiatives share a common goal: to assist states in tailoring tobacco control and prevention efforts that are ethnically and culturally relevant and appropriate. The role of each state’s Tobacco Education Networks in developing culturally sensitive tobacco control programs for various U.S. racial and ethnic minority groups was studied. Some of the key findings include the following: (1) funding allocation and distribution has a direct and significant impact on a Network’s ability to build capacity; (2) all state tobacco control programs offer some degree of assistance to support the local Networks in planning, implementing, and evaluating their initiatives, however, most of the Network staff interviewed reported that the state program staff were not equipped to respond to the large variety of queries sent in by the Networks; (3) practices in tobacco control initiatives within communities of color are not based on rigorous social science; and (4) the continual loss of experienced staff severely affects state Networks’ ability to build capacity. Rigorous evaluation research is necessary to provide sound guidelines for tobacco control and prevention initiatives in communities of color. Respondents from six of the seven states identified the lack of adequate and consistent funding to the state’s tobacco control program as the one common element in all the barriers faced by states in implementing Tobacco Education Networks. Consistent funding is necessary to facilitate capacity building and sustainability in the Tobacco Education Networks.

http://aspe.hhs.gov/pic/library/aspe-7815_1.pdf

PIC ID: 7815.1
Agency Sponsor: ASPE-OPPS, Office of Planning and Policy Support
Federal Contact: ChannahSorah, Vijaya, 202-260-3815
Performer: RTI International, Research Triangle Park, NC

Prevention Makes Common Cents

This paper summarizes research findings on the prevalence, effects and costs of some key preventable conditions and highlights several award-winning business prevention programs that make common “cents.” Expenditures for health care in the United States continue to rise and are estimated to reach $1.66 trillion in 2003. Most of the cost can be attributed to the diagnosis and treatment of chronic diseases and conditions such as diabetes, obesity, cardiovascular disease and asthma. There is accumulating evidence that much of the morbidity and mortality associated with these chronic diseases may be preventable; however, a much smaller amount is spent on preventing these conditions. For many Americans, individual behavior and lifestyle choices influence the development and course of these chronic conditions. Public and private efforts and programs are increasingly designed to promote healthy behaviors. This study showed: (1) there is clear evidence that the costs of chronic conditions are enormous, as are the potential savings from preventing them; (2) since a large part of the cause of chronic conditions involves attitudes and behavioral choices, the prospect of reducing their prevalence appears both daunting and promising; (3) attitudes and behaviors may be resistant to change, even when the desire to change is there; (4) people encounter significant barriers in their social and physical environments; (5) the actions that would eliminate much of the morbidity and mortality in our country are clear, and to a large extent there are practical measures persons may take. By changing some of their behaviors, Americans could change their personal health status and thereby alter the health landscape of the Nation dramatically. Americans could save themselves, their employers, and the Nation substantial amounts of money if they took certain modest measures (e.g., eat & drink in moderation, abstain from smoking, exercise regularly, etc.).

http://aspe.hhs.gov/health/prevention/index.shtml

PIC ID: 7971; Agency Sponsor: ASPE-OHP, Office of Health Policy; Federal Contact: Mazanec, Mary, 202-690-6051; Performer: Department of Health and Human Services, Washington, DC

Strategic Plan and Evaluation Strategy for the National Diabetes Control Programs

This task order sought to assist the Division of Diabetes Translation (DDT) to develop its vision for and build the evaluation capacity of the state programs. The assistance included incorporation of the Division’s vision into the grant application review process, develop an evaluation strategy for documenting the impact of the national program, create logic models and establishing the DDT Evaluation Work Group to oversee and assisting evaluation activities.

PIC ID: 7051
Agency Sponsor: CDC-NCCDPHP, National Center for Chronic Disease Prevention and Health Promotion
Federal Contact: Murphy, Dara, 770-488-5046
Performer: Macro International, Inc., Atlanta, GA

Resource Document: National Survey of Physical Activity and Physical Fitness in Youth

This project collected relevant information regarding the design, development and conduct of surveys of physical fitness and physical activity among children and adolescents, and to synthesize and summarize this information into a final report. The design, development, and conduct of a National Youth Physical Fitness and Physical Activity Survey requires a comprehensive working knowledge of large-scale surveys that have measures of physical fitness and/or physical activity among youth that include protocols and measures used, findings, and implications for the design and conduct of future surveys. In addition, it is important to learn how the previous information from youth surveillance studies have been used by schools, communities, researchers and program planners to develop an analytic plan for newly-collected information. The final report includes information on 114 different studies on physical fitness and 171 different studies on physical activity. The information presented in this report will be used to design and develop a nationally representative youth physical fitness and physical activity surveillance system.

PIC ID: 7702
Agency Sponsor: CDC-NCCDPHP, National Center for Chronic Disease Prevention and Health Promotion
Federal Contact: Fulton, Janet, 404-488-5430
Performer: Macro International, Inc., Calverton, MD

Evaluation of the CDC Cardiovascular Health State Health Program: Final Report

This contract called for: creating a logic model to guide activities of cardiovascular health (CVH) state programs; conducting evaluation training; developing an evaluation framework; and conducting a meeting with Comprehensive State program staff to discuss the Evaluation Framework and State evaluation activities. In 1998 Congress made funded a national, state-based cardiovascular disease prevention program starting with eight states and it increased funding for the CVH State Program each subsequent year. The CVH State Program was designed to increase the leadership of State health departments in cardiovascular health promotion, cardiovascular disease prevention and control, and expand and direct efforts to establish a national CVH program. The outcomes indicate that participants felt that the logic model provided them with a clear picture of Federal expectations for the CVH State program and evaluation activities. The participants felt that the program evaluation training was useful and it provided a good introduction to conducting evaluations. The content of the Evaluation Framework was well received by the States and they reported that the content was useful for long-term outcomes, but suggested including more expectations for short-term and intermediate outcomes.

PIC ID: 7703
Agency Sponsor: CDC-NCCDPHP, National Center for Chronic Disease Prevention and Health Promotion
Federal Contact: Williams, Sheree, 770-488-8007
Performer: Macro International, Inc., Calverton, MD

Objective 1.2 - Reduce the Incidence of Sexually Transmitted Diseases and Unintended Pregnancies

Evaluating National Dissemination Strategies for Effective HIV Prevention Programs for Youth

This project evaluated dissemination strategies intended to increase the adoption and implementation of HIV prevention programs for youth in the United States. This Program Operations evaluation focuses on the ongoing Centers for Disease Control and Prevention (CDC) “Programs That Work” project. Specifically, the dissemination of two curricula, “Focus on Kids” and “Safer Choices,” were evaluated. CDC developed the “Programs That Work” project in response to requests from educators for effective programs that reduce health risk behaviors among youth. The purpose of this project is: (1) To identify interventions with credible evidence of effectiveness in reducing risk behaviors. (2) To disseminate “Programs That Work” from the national level to local community and school levels; and (3) encourage state and local education and health agencies, community-based organizations, and national non- governmental organizations to adopt research-based prevention programs and teaching methods. This evaluation adopts both quantitative and qualitative research methods. For each of the curricula, data were collected from two sources: master trainers and local health educators. It was concluded that the dissemination of “Focus on Kids” and “Safer Choices” had been a success: multiple local trainings had been held; information of the curricula had been delivered to a variety of local agencies/groups; and a large number of young people had been taught. However, there are still some barriers to the implementation of the program that can be addressed. The curricula may need to be tailored in order to better suit the needs of varying audiences. In addition, a cost-effectiveness analysis is needed to further determine the success of the programs.

PIC ID: 7869
Agency Sponsor: CDC-NCCDPHP, National Center for Chronic Disease Prevention and Health Promotion
Federal Contact: Banspach, Stephen, 770-488-3257
Performer: Battelle Memorial Institute, MD, VA

Training Manual for Community Organizing Assessment of Needs, and Planning for Teenage Pregnancy Preventioin

This project identified specific lessons learned by 13 demonstration communities funded to implement community-based teen pregnancy prevention programs, utilizing a youth development model; and translated these lessons into a training manual for dissemination to communities throughout the country that are planning to build similar programs. The design and methodology for developing the manual were prepared in connection with Phase I of the Project (1995-1997). Field support exercises were prepared to accompany each chapter in the areas of needs and assets assessment, problem identification, evaluation and sustainability, logic modeling, defining terms and indicators, a data collection checklist for baseline indicators, a program monitoring checklist, and mobilizing the outcome evaluation. The project found that the youth development approach views teens not as problems to be solved but rather as resources to be developed within their own communities.

PIC ID: 7972
Agency Sponsor: CDC, Centers for Disease Control
Federal Contact: Schauer, Mary, 770- 488-6306
Performer: Battelle, Durham, NC

Objective 1.3 - Increase Immunization Rates Among Adults and Children

Evaluation of Standing Orders in U.S. Nursing Homes

This evaluation identified the costs of implementing standing orders programs and their components, compared to other organized immunization programs in long term care (LTC) facilities; and determined the cost-effectiveness of such programs and their components. This evaluation required 2 phases: Phase 1 included the identification of program costs and of effects of program on staff and resident flow at the LTC. Phase 2 included determination of cost effectiveness of the programs related to improvement in vaccination coverage rates in the LTC. The project demonstrated that Quality Improvement Organizations can work with nursing homes to implement quality improvement training and projects.

PIC ID: 7434
Agency Sponsor: CDC-NIP, National Immunization Program
Federal Contact: Edgar, Gary, 404-639-8787
Performer: Research Triangle Institute, Research Triangle Park, NC

Objective 1.4 - Reduce Substance Abuse

Evaluation of Starting Early Starting Smart

This study examined preliminary accomplishments of a program integrating substance abuse prevention and treatment, and mental health services into primary health care and early childhood settings serving children ages zero to five and their families and caregivers. The program, “Starting Early Starting Smart (SESS)” is a national, public-private partnership between the Casey Family Programs and the Substance Abuse and Mental Health Services Administration. The projects target families with very young children at risk for delayed social-emotional, cognitive and physical development due to risk factors such as substance abuse or mental health problems in the home. The SESS program has two overarching objectives: to improve access to and utilization of a comprehensive set of needed services for families/caregivers and young children; and to improve caregiver behavioral health, family functioning, and child social-emotional development and related outcomes. Preliminary findings indicate the SESS program demonstrates some success: (1) Substance abuse and mental health services for caregivers, families, and children were integrated into the daily activities of primary health care and early childhood settings. (2) Access to and use of caregiver, child, and family services increased. (3) On certain measures, the well being of families, and therefore their nurturing and supportive influences on their youngest members grew stronger, and the infants, toddlers, and children benefited in their early development.

PIC ID: 7884
Agency Sponsor: SAMHSA-CSAP, Center for Substance Abuse Prevention
Federal Contact: Basen, Michele M., 240-276-2417
Performer: Evaluation, Management & Training, Inc. (EMT), Encino, CA

Accomplishments in the Persistent Effects of Treatment Studies (PETS)

PETS evaluated the long-term effectiveness (up to thirty-six months) of substance abuse treatment services provided through a series of grants and cooperative agreements; and to conduct special studies and policy analyses that addressed specific drugs of abuse, methods of treatment, populations, or policy issues. Follow-up studies obtained information at selected follow-up periods for persons who had completed the index treatment episode and for whom historical, intake and treatment exit data were available. Major accomplishments included the enhancement of knowledge of the long term outcome course of people who seek treatment in the publicly funded treatment systems, and a substantial contribution to the published literature on longer term treatment outcomes.

PIC ID: 6738
Agency Sponsor: SAMHSA-CSAT, Center for Substance Abuse Treatment
Federal Contact: Kaiser, Javaid, 240-276-2826
Performer: Westat, Inc., Rockville, MD

Evaluation of Opioid Treatment Program Accreditation Project

The goal of the evaluation was to obtain information to guide the full national implementation of a new treatment approach through systematic study of the processes, barriers, costs associated with a change from regulatory to a mixed accreditation-regulatory process, and various other impacts. Between 1996 and 2001, the Center for Substance Abuse Treatment (CSAT) in the Substance Abuse and Mental Health Services Administration (SAMHSA) and other Federal agencies developed a new regulatory framework for monitoring opioid treatment programs (OTPs), which provide opioid treatment for addiction employing such drugs as methadone and levo-alpha-acetyl-methadol. Between 1998 and 2002, SAMHSA conducted and evaluated a phase-in program to obtain information to aid in the full implementation of the new system. Baseline data collection visits were completed to 152 OTPs; of the 144 OTPs that completed follow-up site visits, 104 underwent the accreditation process as experimental sites and 40 were control sites. In the experimental sample, 86% achieved accreditation in initial accreditation surveys. Overall, study participants generally offered positive appraisals of accreditation, with 74% of site directors and staff agreeing that they would rather work in an accredited OTP. Fewer than half (43%) of OTPs in the study said they found the process of accreditation to be burdensome. Average total site costs of preparing for and undergoing accreditation were $48,005 per OTP: site preparation costs accounted for 82% of the total; technical assistance costs (paid by SAMHSA for study participants) accounted for 7% of the total; and accreditation survey fees (also borne by SAMHSA) accounted for 11% of the total. Findings indicated no diminution in treatment capacity in participating OTPs. Sites that underwent accreditation offered significantly more comprehensive services than did sites that did not. Despite the study’s limitations, findings from this study were consistent with the Federal government’s decision to move to the new accreditation-based regulatory model.

PIC ID: 6739
Agency Sponsor: SAMHSA-CSAT, Center for Substance Abuse Treatment
Federal Contact: Lubran, Robert, 240-276-2714
Performer: Research Triangle Institute, Research Triangle Park, NC

Objective 1.6 - Reduce the Incidence and Consequences of Injuries and Violence

Develop a National Research Agenda for Injury Prevention and Control

This project developed a national research agenda for injury prevention and control through an inclusive consensus-building process. “Injury in America,” a 1985 Institute of Medicine report, concluded that injury rates could be reduced by supporting injury prevention research and determining how to persuade people to use successful interventions. The report called on CDC to lead the national development of a research capacity in injury prevention and control with balance across the varied scientific disciplines of the field and to consider all projects that might prove fruitful for reducing injury morbidity, even if the benefits were evident only in the short term. The results of this research agenda-setting process are being used in the National Center for Injury Prevention and Control’s new research initiatives and research program announcements.

PIC ID: 7696
Agency Sponsor: CDC-NCIPC, National Center for Injury Prevention and Control
Federal Contact: Waxweiler, Richard, 770-488-4850
Performer: Macro International, Inc., Atlanta, GA

The Effectiveness of DHHS Interventions on Violence, Injuries, Physical Activity, and Nutrition: Results from the Literature

The study reviewed and summarized evaluation findings. The evaluation reports selected for review were those from DHHS-supported interventions that directly promoted primary prevention of injury and disease, and interventions that directly promoted improvements in nutrition and physical activity. The goal of the Planning Assessment Initiative was to test whether systematic reviews of the scientific literature, guided by logic models, could help the U.S. Department of Health and Human Services (DHHS) assess the impact of its programs, direct resources efficiently, and improve the health of Americans. During the first year of the Planning Assessment Initiative, the Office of the Assistant Secretary for Planning and Evaluation (ASPE) reviewed DHHS-supported interventions in two strategic areas: (1) prevention of violence and unintentional injury, and (2) improvement in diet and physical activity level. Both these areas are critical to the health of Americans, and both correspond to important objectives and strategies in DHHS’s Strategic Plan. The study identified 32 DHHS-supported interventions to reduce violence and unintentional injury. Of these 32 interventions: the literature contained strong evidence that 13 reduced violence and injuries; strong evidence that two were ineffective; seven showed promise, but the evidence to support their effectiveness is limited; little or conflicting information with respect to the effectiveness of six interventions. The study identified 20 interventions intended to promote better nutrition or increased physical activity: the literature contained good evidence that four of the interventions, such as school-based interventions with multiple components, were effective; six interventions showed promise, but the evidence to support their effectiveness is limited.

PIC ID: 7797
Agency Sponsor: ASPE-OPPS, Office of Planning and Policy Support
Federal Contact: ChannahSorah, Vijaya, 202-260-3815
Performer: Mathematica Policy Research, Inc., Plainsboro, NJ

Addressing the Growing Burden of Trauma and Injury in the Developing World

The Fogarty International Center at the National Institutes of Health (NIH) conducted a needs assessment consultation to determine whether to sponsor an international extramural program in the area of Trauma and Injury in Developing Countries. In addition to gathering information to make the decision, FIC expected to identify research and training gaps and models to help develop measurable goals and objectives for the program; and to determine if it was prudent to proceed. FIC conducted a literature review and produced several background working papers on the status of research on injury and trauma in the developing world. FIC convened a panel July 22 to 23, 2003, of 40 experts in trauma and injury from both the United States and developing nations to build a research agenda to create new knowledge, and support the development of new investigators in the field. Each discussant was asked to give a ten-fifteen minute presentation addressing the current status of research, training and delivery of services in their area of expertise in the developing world and to identify in priority order, gaps and needs in the area in the developing world. The participants identified epidemiology and surveillance, basic science, prevention research and policy research as priorities for the investment of NIH funding to improve global health by reducing the impact of trauma and injury in low and middle-income nations.

PIC ID: 8129
Agency Sponsor: NIH-FIC, Fogarty International Center
Federal Contact: Kupfer, Linda, 301-496-3288
Performer: Abt Associates Inc., Cambridge, MA

Evidence-Based Review of Batterer Intervention and Prevention Programs: Final Report

The purpose of this study is to find what is known about, and to make recommendations on efficacious and effective batterer prevention and intervention programs. Intimate partner violence (IPV) is a serious public health issue that disproportionately affects women. IPV is defined as threatened or actual use of physical force (including sexual assault) against an intimate partner that either results in, or has the potential to result in, injury, harm, or death. In the United States, an estimated 1.5 million women a year are physically assaulted by their partner. Unfortunately, very little is known about the effectiveness of various types of intervention and prevention strategies. Much recent attention has been focused on intervention and prevention programs targeted at perpetrators. Early efforts to prevent intimate partner violence focused primarily on the protection of female victims. Relatively little attention was paid to programs that sought to change the behavior of perpetrators. This project utilizes the methodology developed for the “Guide to Community Preventative Services: Systematic Reviews and Evidence-Based Recommendations.” The guide’s methods were chosen for this study to increase the objectivity of the reviews as the goal was to obtain the best available empirical evidence to support decision making. A total of 31 studies were reviewed for this report, 20 focusing on Batterer Intervention Programs (BIP) and 11 focusing on Dating Violence Prevention Programs (DVPP). It was found that: (1) DVPPs attempt to conduct primary intervention by intervening before abusive behavior begins, whereas BIPs intervene with adult men who have already committed abuse. (2) Although both BIPs and DVPPs focus on preventing partner abuse, the literature is quite separate and studies fairly dissimilar.

PIC ID: 7998
Agency Sponsor: CDC-NCIPC, National Center for Injury Prevention and Control
Federal Contact: Williams, Melinda, 770-488-4647
Performer: RTI International, Research Triangle Park, NC

Goal 2 - Enhance the Ability of the Nation's Health Care System to Effectively Respond to Bioterrorism and Other Public Health Challenges

Objective 2.1 - Build the Capacity of the Health Care System to Respond to Public Health Threats, Especially Bioterrorism Threats in a More Timely and Effective Manner

Achieving Performance Standards in Public Health: Early Findings from the National Public Health Performance Standards Program

This study analyzed how pilot-test data from the National Public Health Performance Standards Program could inform public health policy, administration, and practice. The Program was created to develop measurable performance standards for public health systems and a process for encouraging improvements in essential public health services. The study: (1) explored how measures of local public health system performance vary across communities and essential services; (2) examined the institutional, financial, and community characteristics associated with system performance; and (3) explored the relationships between public health system performance and community health outcomes. Findings in this study indicated that, while a majority of systems maintain some level of activity in each of the 10 essential service areas, most give substantially more attention to some essential services than to others. Performance was particularly high for enforcing laws and regulations and informing, educating, and empowering people. The activity composite scores provide a more detailed view of performance in each essential service area by indicating how closely local public health systems align with what is considered to be optimal performance. Finally, estimates from the multivariate regression models indicated that public health performance varies significantly with selected financial and institutional characteristics of the public health system as well as with several community characteristics.

PIC ID: 7995
Agency Sponsor: CDC, Centers for Disease Control
Federal Contact: Shim, Kyumin, 770- 488-8012
Performer: Mathematica Policy Research, Inc., Plainsboro, NJ

Review of the Multilateral Initiative on Malaria

The goal of the review was to provide a perspective to the program leadership and to the Multilateral Initiative on Malaria (MIM) Secretariat on the past five years of activities under MIM and to provide some suggestions for program enhancement over the next five years. An international panel of experts came to the NIH for one week to review the MIM program. The panel was divided into teams according to their area of expertise: planning and administration, partnerships and management, and research and science. The panel heard presentations from all the component programs of MIM and conducted over 40 in- person and telephone interviews with persons supported by or associated with MIM. The panel utilized interview protocols. The review found MIM to be a healthy, growing group of four component organizations--an administrative arm (the Secretariat), a funding arm (MIMTDR), an electronic- communications arm (MIMCOM) and a research material arm (MR4). The component organization’s work was found to have been impressive, especially in bringing African scientists together through improving communication and building science-focused institutional networks.

http://aspe.hhs.gov/pic/pdf/nih-7892.pdf

PIC ID: 7892
Agency Sponsor: NIH-FIC, Fogarty International Center
Federal Contact: Kupfer, Linda, 301-496-3288
Performer: Abt Associates Inc., Cambridge, MA

Objective 2.2 - Improve the Safety of Food, Drugs, Biological Products, and Medical Devices

Valuation of Morbidity Losses: Meta-Analysis of Willingness-to-Pay and Health Status Measures

The study examined the feasibility of strengthening the FDA Center for Food Safety and Nutrition’s ability to assess regulatory alternatives health benefit costs. To conduct regulatory impact analyses, the Center needs reliable, cost-effective benefits assessment methods. The Center doesn’t have resources to conduct original research on the value of all health outcomes affected by its actions. It must make best use of existing research on health valuation to inform its decisions. It relies largely on “benefit transfer” approaches based on an integrated statistical analysis of results from the existing health valuation literature. The study found that by combining the findings from multiple studies, it is feasible to specify and demonstrate a benefit transfer function for acute health effects; it was possible to compare value estimates based on this function with estimates based on separate benefit transfer approaches that the Center has used for regulatory analysis. The study demonstrated that integrated statistical analysis for acute effects lays an important foundation for assessing values for chronic health effects.

PIC ID: 8087
Agency Sponsor: FDA-CFSAN, Center for Food Safety and Applied Nutrition
Federal Contact: Jessup, Amber, 301-426-1689
Performer: RTI International, Research Triangle Park, NC

FY 2003 Evaluation of Cooperative Agreements for Site-Specific Activies

As part of an ongoing, annual process, performance evaluations were conducted during FY 2003 to assess ATSDR’s State Cooperative Agreement Program for Hazardous Waste Site-Specific Activities in the affected States, American Indian communities and Puerto Rico. The evaluation focused on improvements needed as well as highlighting strengths. The evaluation for each state assessed compliance with established agency guidelines. ATSDR’s technical project officers in the Division of Health Assessment and Consultation conducted the evaluation with input from participating agency programs and cooperative agreement states. During FY 2003, ATSDR conducted assessments but took no final actions on these assessments. It worked to define baselines and establish expectations of the states for coming year actions, beginning in FY 2004. (See also 7740, 7740.2)

PIC ID: 7740.1
Agency Sponsor: ATSDR, Agency for Toxic Substances and Disease Registry
Federal Contact: Erlwein, Bobbie, 404-498-0425
Performer: Agency for Toxic Substances and Disease Registry, Atlanta, GA

Goal 3 - Increase the Percentage of the Nation's Children and Adults Who Have Access to Health Care Services, and Expand Consumer Choices

Objective 3.1 - Encourage the Development of New, Affordable Health Insurance Options

Department of Defense Subvention Demonstration Evaluation

This evaluation examined success of a demonstration program in which enrollment in the Department of Defense’s (DoD’s) Senior Prime plan was offered to military retirees over age 65 who: lived within 40 miles of the primary care facilities of one of the six sites, had recently used military health facility services, and were enrolled in Medicare Part B. Medicare made a capitation payment to DoD for each enrollee, where DoD maintained a level of effort for health care services to all retirees who were also Medicare beneficiaries, whether or not they chose to enroll. The evaluation examined issues in four basic areas: (1) enrollment demand, (2) enrollee benefits, (3) cost, and (4) impact on other DoD and Medicare beneficiaries. The study found that the plans experienced significant challenges in the submission of their data to fiscal intermediaries and carriers using established Medicare fee-for-service billing regulations. Programs were provided extensive technical assistance including in-person visits to address these issues.

PIC ID: 7171
Agency Sponsor: CMS, Office of Research, Development & Information
Federal Contact: McVicker, Victor, 410-786-6681
Performer: Rand Corporation, Santa Monica, CA

Objective 3.2 - Strengthen and Expand the Health Care Safety Net

2002 Community Health Center/National Health Service Corp User/Visit Survey

The purpose of this study was to obtain nationally representative data about the users of community health centers (CHCs) and the services they obtain. The data will inform BHPC staff about the users’ care-seeking behaviors, including their sociodemographic characteristics, the reasons for seeking care, health care service utilization, morbidities and co-morbidities, and the content of encounters with medical staff. The data will also be used to help policymakers and Bureau of Primary Health Care (BPHC) staff to assess how well HRSA-supported health care sites are able to meet health care needs, guide planning decisions, complement data that are not routinely collected from other BHPC data sources, and determine the extent to which CHCs fill unmet needs and address any gaps in the health care system. The study also provided a snapshot of users and services of the National Health Service Corp (NHSC) sites. The study consisted of three main components: 1) CHC User Survey, 2) CHC Visit Survey and, 3) an NHSC User Survey. The CHC User and Visit surveys were conducted in a sample of 70 eligible CHCs that had received Section 330 grants, while the NHSC User Survey was conducted in a sample of 15 selected NHSC sites. Data was collected for the User survey by way of computer-assisted personal interviews conducted at the clinics, the sample users’ home, at a public place, or by telephone. That data was then analyzed by SUDAAN software. General demographic findings in the CHC user survey indicated that users are generally younger, more likely to be Black or Hispanic, less likely to be married, more likely to have less than a high school education, poorer, more likely to be unemployed, have some form of public assistance, be uninsured, and more likely to be current smokers. Findings from the CHC visit survey indicated that nearly 10% of all visits by CHC users over a 12-month period were from a referral from another provider. Findings from the NHSC User Survey indicated that a majority of users interviewed were largely Black (40%) or White (40%), with about 14 percent stating they were Hispanic. A majority of these users were male and about half stated they weren’t in the work force, with 60 percent of users with less than a high school education. The study’s overall intent was not to provide in depth analysis of the data, but to draw the readers’ attention to some points of interest for further exploration and analysis.

PIC ID: 6811
Agency Sponsor: HRSA-BPHC, Bureau of Primary Health Care
Federal Contact: Wulu, John, 301-594-3730
Performer: RTI International, Research Triangle Park, NC

Impact of Publicly Funded Programs on Child Health Safety Nets

This study sought to: (a) identify the characteristics of publicly-funded programs that contribute to the survival and financial viability of pediatric safety-net providers; (b) compare the effects Medicaid Managed Care (and the State Children’s Health Insurance Program) has upon pediatric safety-net hospitals versus the its effects on Federally Qualified Health Centers providing pediatric services; (c) investigate institutional and organizational factors among pediatric safety-net providers; and (d) examine these provider’s success confronting changes in their community. Factors indicative of a viable community pediatric safety net included: 1) the presence of community or regional planning for pediatric safety; 2) engaged academic medical centers; 3) effective Medicaid managed care; 4) adaptive health centers; 5) active public health department; and 6) transformed Federally Qualified Health Centers.

PIC ID: 6039
Agency Sponsor: HRSA-BPHC, Bureau of Primary Health Care
Federal Contact: Pandya, Amisha, 301-594-3724
Performer: Northwestern University, Center for Health Services Policy, Evanston IL

Objective 3.3 - Strengthen and Improve Medicare

Longitudinal Study of Ischemic Heart Disease Among Aged Medicare Beneficiaries

This project assessed the use of Medicare covered services among Medicare beneficiaries with ischemic heart disease based on sociodemographic characteristics (e.g., race/ethnicity, sex, age, socioeconomic status). It was one part of a larger CMS and Department of Health and Human Services effort to address health disparities among Medicare beneficiaries. This was done using a longitudinal database that linked Medicare enrollment and claims data with small-area geographic data on income (e.g., U.S. Census data). Due to recent change in the race/ethnic coding in the Medicare enrollment database (EDB), it was not possible to examine health care access, utilization, and outcomes among minority groups. Some key findings indicate that (1) the lower rates of complex diagnostic testing and reperfusion therapies for Blacks compared to Whites. Much less research was published on other minorities besides Blacks and, when available, produced conflicting findings. For example, some studies found Hispanics less likely than Whites to undergo open heart surgery while others found no such differences. (2) Also, rural location and poverty area residence have both been linked to lower use rates among Blacks. Minorities may have less access to costly high-tech procedures because they cannot afford them. (3) Controlling for socioeconomic status, research has shown that Blacks are often treated differently in the same hospital and are less often referred for cardiac catheterization or to a specialist.

PIC ID: 7419
Agency Sponsor: CMS, Centers for Medicare and Medicaid Services
Federal Contact: Meltzer, Arthur, 410-786-9974
Performer: Research Triangle Institute, Research Triangle Park, NC

Evaluation of Vermont's Pharmacy Assistance Programs For Low-Income Medicare Beneficiaries First Round Evaluation: Final Report

In 1996, Vermont Medicaid implemented an 1115 waiver entitled The Vermont Health Access Plan or VHAP. The waiver consisted of a five-year demonstration that introduced a statewide mandatory Medicaid managed care program. One unique component of the demonstration was a program to provide Medicaid prescription drug benefits to the State’s lower income Medicare beneficiaries. Medicare beneficiaries living in households with incomes up to 175 percent of the federal poverty level (FPL) are currently eligible for the full Medicaid outpatient pharmacy benefit. As part of the 1115 waiver, costs incurred under the VHAP Pharmacy program are eligible for federal matching dollars. Vermont’s pharmacy assistance programs provide a useful opportunity for assessing the impact of outpatient prescription drug benefits on the cost and use of drug and non-drug medical services among the elderly and disabled population. This study should also help inform the current national debate over outpatient prescription drug coverage for low- income Medicare beneficiaries at the federal or state levels. Some of the findings of this study include the following: 1) Nearly 16 percent of the 93,332 Medicare beneficiaries who resided in Vermont in 2000 received some form of outpatient prescription drug coverage through the state’s pharmacy assistance programs. 2) Of the 14,659 enrollees in 2000, 66 percent received full drug coverage under the 1115 waiver’s VHAP Pharmacy program for individuals with incomes up to 150 percent of federal poverty level (FPL). An additional 20 percent received coverage for maintenance prescriptions only under the 1115 waiver portion of the VScript program for individuals with incomes between 151 and 175 percent of FP. The remaining 14 percent received maintenance drug coverage through the VScript Expanded program for individuals with incomes between 176 and 225 percent of FPL with state-only dollars.

http://www.cms.hhs.gov/researchers/reports/2003/gilman.pdf

PIC ID: 7953
Agency Sponsor: CMS, Centers for Medicare and Medicaid Services
Federal Contact: Boben, Paul J., 410-786-6629
Performer: Research Triangle Institute, Research Triangle Park, NC

Phase II Evaluation of CNO Demonstration

The Community Nursing Organization (CNO) Demonstration is an innovative approach to the provision of community nursing and ambulatory care services for Medicare beneficiaries. Structured around the two fundamental concepts of nurse case management and capitated payment, CNOs attempted to promote the timely and appropriate use of community health services and to reduce the use of costly acute-care services. This evaluation employed two complimentary approaches: (1) the comparison of average utilization and expenditures exclusively for randomized beneficiaries and (2) the analysis of all treatment group beneficiaries, compared to a reference group drawn from the Medicare population living in the same geographic area. Both analytic approaches found that capitation rates for CNO-covered services resulted in payments for the treatment group that were higher than payments for the control or population reference groups. The analysis of randomized beneficiaries indicated that this was the only source of elevated expenditures for the treatment group, implying that a reduction in capitation rates might make the CNO budget neutral. By contrast, comparison of the treatment group to the population reference group suggested that expenditures for non-CNO services, particularly inpatient hospitalization, were also higher for the treatment group, implying that CNO participation induced higher utilization of non-CNO services.

PIC ID: 6306.1
Agency Sponsor: CMS, Office of Research, Development & Information
Federal Contact: McVicker, Victor, 410-786-6681
Performer: Abt Associates Inc., Cambridge, MA

Medicare Beneficiaries Who Involuntarily Disenroll from their Health Plans

The purpose of this survey was to collect data that will help CMS understand how Medicare beneficiaries are affected by Medicare health plan withdrawals and reductions in service areas. Understanding the difficulties some beneficiaries may have in response to changes in the health care system will help CMS meet its goal of providing all Medicare beneficiaries with adequate health care. The survey included a sample of Medicare beneficiaries who were enrolled in managed care plans that either terminated their risk contracts or reduced their service areas as of January 1, 2002. The survey was conducted as a mail survey with telephone follow-up of non-respondents. The findings show that there are some clear differences between involuntary disenrollees in vulnerable and less vulnerable subgroups regarding their understanding of the options available to them and the implications of plans withdrawing from the Medicare program. Overall, the incidence of potentially negative impacts of plan withdrawals, such as disruption in provider arrangements and reduced access to care, appears to have affected relatively fewer beneficiaries in 2002 than in 2001. However, those in vulnerable subgroups such as the disabled, those in poor health, and those with less education, continued to be more likely to experience negative consequences than other beneficiaries. Furthermore, there was no observed improvement in the level of awareness of options or understanding of what would happen when a plan stopped covering its beneficiaries.

PIC ID: 7421
Agency Sponsor: CMS, Office of Research, Development & Information
Federal Contact: Riley, Gerald, 410-786-6699
Performer: University of Wisconsin, Madison, WI

Analysis of Medicare Beneficiary Knowledge Data Using the Medicare Current Beneficiary Survey (MCBS): Phase 3.2

This report uses the 1998 through 2000 rounds of the Medicare Current Beneficiary Survey (MCBS) to measure progress toward achieving the Centers for Medicare and Medicaid Services (CMS) education campaign goals following the national distribution of the Handbooks Medicare & You 2000 in 1999 and Medicare & You in 2001 in 2000. In addition to descriptive analyses, longitudinal regression modeling is used to measure if the latest Handbook, Medicare & You 2001, was significantly related to selected campaign objective outcomes. The broad policy questions of the Medicare beneficiary education campaign that are examined in this phase include: (1) Do beneficiaries receive (have access to) information and is this access increasing over time? (2) Are beneficiaries aware of Medicare’s features and options and is this awareness increasing over time? (3) Do beneficiaries understand the information they receive, can they use it for making informed choices about their Medicare participation if they so desire, and is this level of understanding increasing over time? (4) Are beneficiaries using the information to make choices and is use of the information increasing over time? From the 2000 MCBS, it was found that almost three-quarters of non-instrumentalized Medicare beneficiaries remembered receiving a copy of the Medicare & You Handbook. These persons were using the Handbook as a reference document; although virtually all kept their copy, most had not read it thoroughly. Approximately half of beneficiaries were aware of the Medicare toll-free telephone number. Among those who had Internet access, only a small percentage had visited the official Web site for Medicare information.

PIC ID: 7930
Agency Sponsor: CMS, Centers for Medicare and Medicaid Services
Federal Contact: Terrell, Sherry, 410-786-6601
Performer: Research Triangle Institute, Research Triangle Park, NC

Evaluation of Qualified Medicare Beneficiaries and Specified Low-Income Medicare Beneficiary Programs

This study was designed to identify the factors, both policy and individual, that influence enrollment in the Medicare Savings Programs, as well as program impacts on beneficiaries. This study focused primarily on Qualified Medicare Beneficiaries (QMBs) and Specified Low-Income Medicare Beneficiaries (SLMBs) who were not otherwise eligible for Medicaid under SSI or medically needy programs. Major findings included: (1) lack of awareness, not motivation was the main reason eligible beneficiaries did not enroll in the programs; (2) personal assistance is the key to enrolling more beneficiaries into Medicare savings programs; (3) enrollment in the QMB/SLMB programs increased utilization of medical care services; (4) most QMB/SLMB enrollees continued to incur some out-of-pocket costs; and (5) the effect on health outcomes was unknown.

http://www.cms.hhs.gov/researchers/reports/2003/haber2.pdf

PIC ID: 7390
Agency Sponsor: CMS, Office of Research, Development & Information
Federal Contact: Rudolph, Noemi, 410-786-6662
Performer: Research Triangle Institute, Waltham, MA

IDS Solutions for Transferring Medication Information Across Patient Care Settings: Volume 1 & 2

An important part of ensuring patient safety is the effective transmission of medication information across care settings. Inaccurate or incomplete information is a leading cause of medical error. This research project focuses on information exchange between ambulatory care and acute care settings. The purpose of this study was to gain an in-depth understanding of the medication information transfer process and identify likely process failures, which can be practically addressed. This qualitative study used various data collection approaches to improve our understanding of the medication information transfer process. A series of focus groups with clinicians at Providence Health System provided the basic foundation of information for creating a generic care process model. The following were key findings from PHS focus groups: (1) Breakdowns in medication information transfer were a problem that concerned clinicians, who believed that these breakdowns were definitely a source of adverse medical events. (2) Medication information transfer was not a single process, but rather a complex set of overlapping processes that engaged multiple caregivers and depended on source of admission and discharge destination. (3) Clinicians expected that patients and their family members would serve as a reliable source of information. (4) Medication information education with patients/family is often limited to new medications (versus a comprehensive review of all prescribed medications). (5) Information technology as a solution for enhancing medication information transfer had mixed success.

PIC ID: 7682
Agency Sponsor: AHRQ, Agency for Healthcare Research and Quality
Federal Contact: Morgan, Kelly, 301-594-1782
Performer: Research Triangle Institute, Research Triangle Park, NC

Using MSIS Data to Analyze Medicaid Eligibility and Managed Care Enrollment Patterns in 1999

In this report, the new MSIS enrollment data are utilized to provide detailed information on Medicaid eligibility patterns and managed care participation in calendar year 1999. A series of 14 tables were constructed for each of the 50 states and the District of Columbia, and then summarized at the national level. A separate File Listing gives the user the ability to go directly to the section or set of tables they are interested in. The data show that Medicaid eligibility and managed care enrollment vary dramatically across states. For example, Tennessee, for example, enrolled its entire Medicaid population in HMOs, while Alaska, Louisiana and Wyoming did not use any type of managed care for Medicaid enrollees in 1999. Overall, about 35 percent were enrolled in comprehensive managed care organizations (HMOs), and another 11 percent were enrolled in primary care case management (PCCM) plans. An additional 9 percent were enrolled in other types of prepaid health plans, such as behavioral health plans (BHPs) or dental plans, bringing the total enrollment in any kind of managed care plan to 55 percent. In terms of eligibility groups, about 69 percent of AFDC children (Section 1931) and 65 percent of AFDC adults (Section 1931) were either in an HMO or a PCCM, whereas only 33 percent of SSI disabled persons and 16 percent of SSI aged persons received care in that form.

http://aspe.hhs.gov/daltcp/reports/msisdata.htm

PIC ID: 8012
Agency Sponsor: ASPE-ODALTCP, Office of Disability, Aging, and Long-Term Care Policy
Federal Contact: Drabek, John, 202-690-6443
Performer: Mathematica Policy Research, Inc., Plainsboro, NJ

IDS Solutions for Transferring Medication Information Across Patient Care Settings: AHRQ Fact Sheet Assessment

This research project focused on information exchange between ambulatory care and acute care settings in order to gain an in-depth understanding of the medication information transfer process and identify likely process failures, which can be addressed practically. An important part of ensuring patient safety is the effective transmission of medication information across care settings. Inaccurate or incomplete information is a leading cause of medical error. This qualitative study used various data collection approaches to improve our understanding of the medication information transfer process. In the study of integrated delivery system solutions for transferring medication data across patient care settings, Research Triangle Institute developed a model showing that use of an electronic medication list at hospital admission and discharge may reduce by 50 percent the risks and errors associated with poor information transfer. Examples of these risks are medication omissions, wrong doses, and allergic reactions. Providence Hospital in Portland, OR, is using the study findings to reduce medication errors.

PIC ID: 7682.2
Agency Sponsor: AHRQ, Agency for Healthcare Research and Quality
Federal Contact: Morgan, Kelly, 301-594-1782
Performer: Research Triangle Institute, Research Triangle Park, NC

Objective 3.4 - Eliminate Racial and Ethnic Health Disparities

Developing a Self-Assessment Tool for Culturally and Linguistically Appropriate Services in Local Public Health Agencies

This project developed a survey instrument that local public health agencies (LPHAs) can use to self- assess the nature and extent of culturally and linguistically appropriate services (CLAS) in their service jurisdictions. As a logical follow-up to the national study of CLAS in Managed Care Organizations (MCOs), the project built upon the survey instrument developed under that study and revised components as appropriate to meet the particular needs and characteristics of LPHAs. This effort was guided by a Project Expert Panel comprised of representatives of LPHAs and non-Federal experts and a Federal Project Advisory Group comprised of representatives of relevant HHS agencies who, together, have expertise in survey research and evaluation, local public health agencies, CLAS and cultural competency, and health care quality. The instrument was pre-tested for appropriateness and accuracy of the instrumentation (format and content), revised accordingly, and disseminated for voluntary use by interested LPHAs. The survey protocols and analysis plan used in the MCO study are included with the LPHA CLAS self- assessment tool as a model for implementing the tool and analyzing results. Recommendations are offered to LPHAs that plan to use the self-assessment tool to monitor quality of services for culturally and linguistically diverse members, and for organizations that plan to conduct future research in this area.

PIC ID: 7712
Agency Sponsor: OPHS-OMH, Office of Minority Health
Federal Contact: Hawks, Betty, 301-443-5084
Performer: Cosmos Corporation, Bethesda, MD

Objective 3.5 - Expand Access to Health Care Services for Targeted Populations with Special Health Care Needs

The Breakthrough Donation Practices Study: Best Practices Final Report

This study is the most recent component of U.S. Secretary of Health and Human Services Tommy G. Thompson’s Gift of Life Donation initiative, which addresses the widening gap between the supply and demand for organs and tissues. The purpose of the study is to generate significant, measurable increases in organ donation by helping the national community of organ procurement organizations (OPOs) and hospitals to quickly identify, learn, adapt, replicate, and celebrate “breakthrough” practices associated with higher donation rates. The Lewin Group pursued a qualitative case study approach to identify the best practices. The study called for an expert panel to identify hospital and OPO practices that were most likely associated with increased conversion. In January 2003, the study was based on new, available data that brought sharper focus on OPOs and hospitals with outstanding organ donor practices; 6 OPOs and 16 hospitals with conversion rates higher than the national mean. Based on these visits, interviews, and face-to-face discussions, the investigators identified a set of principles associated with higher rates of organ donation: 1) Integrating organ donation fully into routine roles and responsibilities, 2) setting high standards for donation performance to reduce the unacceptable shortage of life saving organs, 3) involving OPO and hospital staff in ongoing standards setting and redesign of means to achieve these standards, 4) holding OPOs, hospitals and their staff accountable for achieving these standards and recognizing their staff accordingly, 5) establish, maintain, and revitalize a network of interpersonal relationships and trust involving OPO and hospital staff, donor families, and other key agents, 6) collaborate to meet the range of needs of potential donor families and achieve informed consent to donate, and 7) conduct ongoing data collection and feedback to drive decision making toward performance improvement. The results of this study led to Secretary Thompson’s formation of the Organ Donation Breakthrough Collaborative, whose intention is to spread the practices of large hospitals and OPOs with high donation rates to other hospitals wanting to improve their own donation practices. Successful practices will eventually be expanded to 800 other large U.S. hospitals, where 90 percent of all eligible donors are located.

http://www.organdonor.gov/bestpractice.htm

PIC ID: 7288
Agency Sponsor: HRSA-OSP, Office of Special Programs
Federal Contact: Wagner, Dennis, 301-443-0625
Performer: The Lewin Group, Falls Church, VA

Strategies for Addressing Asthma Within a Coordinated School Health Program

This document is a manual discussing asthma wellness in schools which outlines best practices and policies for an asthma wellness and education program in the context of a school health program. A healthy student is a student ready to learn. Asthma-friendly schools are those that make the effort to create safe and supportive learning environments for students with asthma. They have policies and procedures that allow students to successfully manage their asthma. Chances for success are better when the whole school community takes part--school administrators, teachers, and staff, as well as students and parents. The Centers for Disease Control and Prevention (CDC) has identified six strategies for schools and districts to consider when addressing asthma within a coordinated school health program. These strategies can be effective whether the program is for the entire school district or just one school. 1) Establish management and support systems for asthma-friendly schools. 2) Provide appropriate school health and mental health services for students with asthma. 3) Provide asthma education and awareness programs for students and school staff. 4) Provide a safe and healthy school environment to reduce asthma triggers. 5) Provide safe, enjoyable physical education and activity opportunities for students with asthma. 6) Coordinate school, family, and community efforts to better manage asthma symptoms and reduce school absences among students with asthma.

http://www.cdc.gov/HealthyYouth/asthma/pdf/strategies.pdf

PIC ID: 7700
Agency Sponsor: CDC-NCCDPHP, National Center for Chronic Disease Prevention and Health Promotion
Federal Contact: Wechsler, Howell, 770-488-6100
Performer: Macro International, Inc., Calverton, MD

Maximizing the Cost Effectiveness of Home Health Care (HHC)

This study sought to examine the following hypotheses: (1) volume-outcome relationships are present in HHC for common patient conditions, (2) upper and lower volume thresholds define the range of services most beneficial to patients, and (3) a strengthened physician role and better integration of HHC with other services during an episode of care can optimize patient outcomes while controlling costs. Rapid growth in home health use has occurred despite limited evidence about the necessary volume of HHC needed to achieve optimal patient outcomes, and whether or not it substitutes for more costly institutional care. The following outcomes were found: 1) limitations in activities of daily living (ADLs) were significantly greater for the high-volume groups, these patients had a greater prevalence of chronic conditions, and their institutional utilization within the 14 days prior to admission was less likely to be an acute-care hospital, indicating the more postacute nature of the low-volume groups; 2) the general case mix difference was consistent with the greater use of aide services for high-volume patients; 3) preliminary analyses of outcomes suggested relatively few differences in outcomes by volume, after controlling for condition.

PIC ID: 7179
Agency Sponsor: CMS, Office of Research, Development & Information
Federal Contact: Meadow, Ann, 410-786-6602
Performer: Center for Health Policy Research, Washington, DC

The Future Supply of Long-Term Care Workers in Relation to the Aging Baby Boom Generation: Report to Congress

One of the challenges facing the U.S. in the 21st Century will be to ensure that individuals throughout their life will have the supports they need and will be treated with dignity. For the growing population of the elderly and people with disabilities, ensuring the adequacy and availability of direct care workers is key to meeting this ideal. As this report shows, the aging “baby boomer generation” will be the most significant factor increasing the demand for long-term care services over the next half century. This report finds that the number of individuals using either nursing facilities, alternative residential care, or home care services is expected to increase from 15 million in 2000 to 27 million in 2050. The Bureau of Labor Statistics (BLS) estimates that by 2010, direct care worker jobs in long-term care settings should grow by about 800,000 jobs, or roughly 45 percent. According to estimates developed by HHS’s Office of the Assistant Secretary for Planning and Evaluation (ASPE), after 2010, the demand for direct care workers in long-term care settings becomes even greater as the baby boomers reach age 85, beginning in 2030. These projections indicate that it is critical to retain existing long-term care workers and attract new ones. Some recommendations include (1) Engage employers and employees as well as medical professionals and state and local government, in a dialogue on issues relating to improved pay, benefits, career ladders, and working conditions in long-term care. (2) Explore with faith and community-based organizations their potential roles in addressing shortages in labor imbalances through strengthening relationships with the workforce investment system, and in recruiting volunteers for respite care for family “back-up” services, and home-based support.

http://aspe.hhs.gov/daltcp/reports/ltcwork.htm

PIC ID: 7951
Agency Sponsor: ASPE-ODALTCP, Office of Disability, Aging, and Long-Term Care Policy
Federal Contact: Frank, Andreas, 202-690-6443
Performer: Department of Health and Human Services, Washington, DC

Evaluation of Pre- and Post-Booking Diversion Programs

This study examined nine established SAMHSA-funded state jail diversion programs serving individuals with co-occurring disorders in the criminal justice system. The program seeks new knowledge on improving prevention and treatment of substance abuse and mental illness and effectively applying that knowledge through work with state and local governments, providers, families and consumers. Research Triangle Institute coordinated the research; the National GAINS Center provided technical assistance to participating programs. Research staff interviewed diverted and non-diverted individuals (1,966 interviewees at the start; declining to 1,497 at three months and 1,353 at one year) using a national committee-developed interview protocol. Major findings included: (1) diverted individuals spent less time either hospitalized or incarcerated than did non-diverted subjects in the first year after contact/arrest; (2) both groups reported similar rates of re-arrest during the following year; (3) diverted individuals received significantly more mental health treatment than the non-diverted; and (4) both groups had fewer mental health symptoms over time. Four of the sites participated in a year long jail diversion cost study which addressed the cost differences of jail diversion programs for the average participant versus costs under traditional criminal justice processes. The diverted groups incurred higher community-based treatment costs and the non-diverted groups incurred higher jail costs, resulting in no difference overall in costs in two of the sites, the cost of diversion significantly higher in one site and lower in another.

PIC ID: 7720
Agency Sponsor: SAMHSA-CMHS, Center for Mental Health Services
Federal Contact: Salasin, Susan, 240-276-1908
Performer: Research Triangle Institute, Research Triangle Park, NC

Evaluation of the Effects of the Vermont Parity Act

In 1997, Vermont enacted parity legislation that requires health plans to provide coverage for the treatment of “mental health conditions” and prohibits rates, terms, or conditions that impose a greater financial burden for access to treatment for mental illness than for physical illness. Vermont’s parity law is more comprehensive than most others, and therefore, offers a unique opportunity to learn about the actual effects of a full parity mandate. Some of the questions to be addressed in this evaluation included: What mandates governed MH/SA benefits prior to parity? Were there identifiable public/private cost shifts? Has improved commercial reimbursement provided increased flexibility for use of funds from non- commercial sources (e.g., have dollars been redirected to prevention programming)? Do consumers report improved access and fewer barriers to MH/SA services as a result of the law? This study reflected experiences during the first 2 to 3 years of parity in Vermont; it is possible that a longer study period might yield different results. Further, the study was limited to a single State, and results may not be generalizable to other States in which the mix of providers or services differs. The study shows that increased use of managed care helped make parity affordable in Vermont, but may have reduced access and utilization for some services and beneficiaries. Limited knowledge of the law complicated implementation for employers, providers, and consumers. Vermont stakeholders recommended that more attention be paid to education and other proactive efforts to better prepare for a change of this magnitude.

http://www.mentalhealth.samhsa.gov/publications/allpubs/sma03-3822/default.asp

PIC ID: 7728
Agency Sponsor: SAMHSA-CMHS, Center for Mental Health Services
Federal Contact: Buck, Jeffrey, 240-276-1959
Performer: Mathematica Policy Research, Inc., Plainsboro, NJ

Equality of Opportunity and the Importance of Place

The National Academy of Sciences (NAS) was funded to commission a series of papers to synthesize the available research on the relationship between place-based factors and outcomes in the areas of health, child and youth development, and welfare reform. There is a growing body of research documenting the synergistic effect that the configuration of community resources and values, demographic, geography, and other factors have on health and human service outcomes. The papers commissioned by NAS identified available data to assess these interactions and made recommendations for future research. NAS convened a workshop of researchers and policy makers from federal, state and local government, academic institutions and private organizations to reflect on the papers and develop and refine an integrated research agenda. The papers, as well as a workshop summary including the research agenda, were disseminated and posted on NAS’ website. NAS recommended additional research on the importance of place as it relates to: welfare-to-work, child and adolescent development, and public health.

http://www.nap.edu/books/0309084679/html/

PIC ID: 7552
Agency Sponsor: ASPE-OHSP, Office of Human Services Policy
Federal Contact: Benesch, Brenda, 202-260-0382
Performer: National Academy of Sciences, Washington, DC

Comprehensive Mental Health Services Program

• The Center for Mental Health Services (CMHS) conducted an evaluation of 23 sites of the Comprehensive Community Mental Health Services Program for Children and their Families funded in fiscal years 1997 and 1998. Evaluators hired at each site gathered descriptive data on such measures as demographics, functional status, diagnosis, and risk factors of enrolled children and their families. They collected outcome data in areas such as social functioning, emotional and behavioral strengths, substance abuse, school performance, and juvenile justice involvement in a representative sample of about 280 children per site. To obtain the outcome data, children are assessed at the time of entry (i.e., baseline) and every six months for up to 36 months. External evaluators visit the sites annually to study the development of systems of care according to system-of-care principles. Patterns of service use and their costs are examined using agency management information system data. A study of long-term sustainability is being conducted across all communities. In addition, two systems of care were compared with two usual service delivery systems to study the effectiveness of systems of care and, in two other communities, a study of the effectiveness of evidence-based practices within a community-based systems of care is being conducted.

  PIC ID: 7269.1
  Agency Sponsor: SAMHSA-CMHS, Center for Mental Health Services
  Federal Contact: Fisher, Sylvia K, 240-276-1923
  Performer: ORC Macro, Calverton, MD

Neighborhoods and Health: Building Evidence for Local Policy

This study focused on the development, analysis and use of neighborhood health indicators pertaining to children and youth. Data was provided by five data intermediary organizations (located in large cities) that participated in the Neighborhood Indicators Partnership (NNIP). In the first component of the project, each site compiled and analyzed new neighborhood-level health indicators and used the data to further local health improvement initiatives. In the second component, researchers examined relationships across sites between neighborhood conditions and key health indicators (e.g., teen birthrates; rates of early prenatal care, low-birth-weight births, infant mortality and age-adjusted mortality), and examined changes in these relationships over the past decade. Geospacial analysis and mapping techniques are used. Cross-site study findings indicated that although gaps seem to have diminished in the past decade, health problems of high-poverty neighborhoods remain substantially more serious than those of non-poor neighborhoods in all cities for which there was data. Thus, the findings confirm that neighborhoods do make a difference for health outcomes and health policy.

http://aspe.hhs.gov/hsp/neighborhoods-health03/index.htm

PIC ID: 7954
Agency Sponsor: ASPE-OHSP, Office of Human Services Policy
Federal Contact: Halpern, Peggy, 202-260-0285
Performer: Urban Institute, Washington, DC

Health Care Coverage Among Child Support-Eligible Children

The Child Support Performance and Incentives Act of 1998 (CSPIA) required HHS to establish a joint HHS/DoL Medical Child Support Working Group to suggest improvements to medical child support enforcement. The Working Group’s report to Congress, released in 2000, includes comprehensive recommendations to shift from the noncustodial parent to the custodial parent as the preferred health insurance carrier (though not necessarily the payer). The Working Group also recommended that the child support enforcement program assist children in the their caseload that are without private insurance to gain access to public coverage. This report assesses the potential effects of these recommendations by using the 1999 National Survey of America’s Families in order to determine the health care coverage status of child support-eligible children who live with their mothers and estimate their eligibility for Medicaid and the State Children’s Health Insurance Program (SCHIP). Key findings show that custodial mothers already provide much health care coverage for their children, modest gains in private insurance coverage could be made by enrolling currently uninsured children in private health care coverage that is already available to one of their parents, and many child support-eligible children are also eligible for Medicaid or SCHIP.

http://aspe.hhs.gov/hsp/CSE-health-ben02/index.htm

PIC ID: 8112
Agency Sponsor: ASPE-OHSP, Office of Human Services Policy
Federal Contact: Burnszynski, Jennifer, 202-690-8651
Performer: Urban Institute, Washington, DC

Changes in Elderly Disability Rates and Implications for Health Care Utilization and Cost

This study was undertaken to better understand disability trends and the potential implications for acute health and long-term care use. Recent research has shown that functional disability among older Americans has decreased. There is also evidence that cognitive impairment and physical limitations, such as lifting 10 pounds, walking short distances, and climbing a flight of stairs, which may be precursors to functional disability, may have also declined in recent years. Four panels (1984, 1989, 1994 and 1999) of the National Long Term Care Survey were used to identify older Americans with a chronic disability (defined as a functional limitation lasting at least 3 months). Aggregate trends in disability were examined and decomposed by type of functional limitation, i.e., disability based exclusively on limitations in instrumental activities of daily living (IADLs) such as money management and meal preparation that are associated with the ability to maintain independence at home, and limitations in activities of daily living (ADLs) such as dressing and eating that indicate a higher level of disability or frailty. The major findings include: (1) the prevalence of chronic disability among the elderly declined significantly over the 15-year period, from 22.1 percent in 1984 to 19.7 percent in 1999, (2) the aggregate decline was mostly due to decreases in the percentage of elderly persons receiving help from someone for IADLs, (3) nearly all individual IADLs declined over the period, but the most dramatic change was a 3.7 percentage point drop in help with money management between 1984 and 1989 when Social Security direct deposit became the norm, (4) no individual ADLs significantly declined in prevalence over the period, and (5) the prevalence of institutional residence, the most costly form of long-term care, was about 5 percent of the elderly throughout the period.

http://aspe.hhs.gov/daltcp/reports/hcutlcst.htm

PIC ID: 7554
Agency Sponsor: ASPE-ODALTCP, Office of Disability, Aging, and Long-Term Care Policy
Federal Contact: Marton, William, 202-690-6443
Performer: Urban Institute, Washington, DC

Objective 3.6 - Increase Access to Health Services for American Indians and Alaska Natives (AI/AN)

HIV/AIDS/STD/TB/Hepatitis B and C Case Reporting of Tribally Operated Health Facilities

This second in a series qualitative assessments investigated public health surveillance of a related set of infectious diseases among American Indian and Alaska Native (AI/AN) people. AI/AN people are affected disproportionately by many infectious diseases, including human immunodeficiency virus and acquired immunodeficiency syndrome (HIV/AIDS), sexually transmitted diseases (STDs), hepatitis B and C (HBV and HCV, respectively), and active tuberculosis (TB). In recent years, under the Indian Self-Determination and Education Assistance Act (1975), tribes are increasingly assuming more control over health care resources. More than 50 percent of federally recognized tribes have already participated in some manner in the movement toward managing health care for their members. Serious concerns have been raised regarding the impact on public health surveillance of recent changes in the way health care is delivered to AI/AN communities. Whether and how public health surveillance has been affected by this decentralization of services that were originally provided by IHS has not been studied. The findings illustrated surveillance issues that exist in tribal and urban Indian health facilities. The results of this evaluation suggest that the difficulty and complexity of addressing infectious disease surveillance challenges in tribally operated and urban Indian health facilities.

PIC ID: 7875
Agency Sponsor: CDC, Centers for Disease Control
Federal Contact: Bertolli, Jeanne, 404- 639-8500
Performer: Research Triangle Institute, Research Triangle Park, NC

Goal 4 - Enhance the Capacity and Productivity of the Nation's Health Science Research Enterprise

Objective 4.3 - Strengthen and Diversify the Pool of Qualified Health and Behavioral Science Researchers

Evaluation of the Bridges to the Future Program

This study examined the feasibility of evaluating the Bridges to the Future program. The mission of the Bridges to the Future Program, established in 1992, is to make available to the biomedical science research enterprise and to the nation the intellectual talents of an increasing number of underrepresented minority students. Bridges to the Future accomplishes this mission by supporting programs that facilitate the transition of students from associate- to baccalaureate-degree granting institutions and from master’s- to doctoral-degree granting institutions. Bridges to the Future promotes effective inter-institutional partnerships to improve the quality and quantity of underrepresented minority students being trained as the next generation of scientists. Bridges to the Future aims to increase the number of underrepresented minority biomedical scientists. The conclusion of the feasibility assessment was that, for the results of any evaluation to be meaningful, it is essential that the questions to be addressed by the evaluation are identified clearly prior to initiating data collection. Furthermore, data collection must be much more systematic, intensive, and more thoroughly validated than currently possible using existing data. In view of the need for detailed and high-quality data, a large-scale evaluation effort designed to address general questions regarding what strategies work among the Bridges programs was found infeasible.

PIC ID: 7651
Agency Sponsor: NIH-NIGMS, National Institute of General Medical Sciences
Federal Contact: Onken, James, 301-594-2764
Performer: University of Chicago, Chicago IL 60637

Evaluation of the Public Health Prevention Service (PHPS) Program

This multi-year project sought information regarding disease/illness prevention service program effectiveness and what could be done to improve the program. In 1997 the Public Health Prevention Service (PHPS) Program was established to “improve the nation’s public health practice by preparing entry-level public health professionals to conduct prevention programs that improve the health and prevent injury and to manage emerging public health problems.” A year later CDC initiated a project to develop and implement an evaluation plan for the program. The evaluation process spanned three years (1999-2001) and the results from the first phases of the evaluation were used to develop and refine the PHPS program and the plan for on-going evaluations. Thus, the evaluation focused on recruiting details, participant training, as well as the effects of creating a qualified pool of public health professionals. A variety of evaluation methods were used, including surveys, interviews and focus groups. The final report contains specific conclusions and recommendations about the program. These recommendations focus on a variety of issues including the application process, rotational assignments, field assignments, training aspects of the program and the relationship of curricula to management and leadership.

Recommendations from this evaluation project will be used to refine and improve the PHPS program.

PIC ID: 7038
Agency Sponsor: CDC-EPO, Epidemiology Program Office
Federal Contact: Jarvis, Dennis, 404-639-4087
Performer: Macro International, Inc., Atlanta, GA

Objective 4.4 - Improve the Coordination, Communication, and Application of Health Research Results

Future Directions for Family Planning Research: A Framework for Title X

This study identified research opportunities that both build from the existing knowledge base and expand the availability of useful information that leads to the enhancement of the quality of family planning services in the United States. The final report relied on two primary sources of information: descriptive findings from an examination of the research literature and the opinions of a group of 30 technical experts who convened twice to identify high-priority research issues facing the field and to discuss how future research could be guided and harnessed to provide better information to practitioners about best practices. Over 240 reports and articles on research about family planning were identified and examined. With input expert technical advisors, the authors organized the review around the key issues facing the family planning field today: reaching high-priority populations, strengthening family planning practices and improving the organization and administration of services. The report identifies priority directions for future family planning research and suggestions for enhancing the impact of research investments made by the Office of Family Planning. The report summarizes the research evidence about family planning services and effective practices. This information should be useful to program directors and practitioners as they consider their own service efforts. Finally, the report calls for an enhanced effort to document, track, and evaluate how family planning services can be more effectively delivered. Furthermore,  research is needed to understand how to disseminate information about effective practices to service providers in a more timely and comprehensive manner.

http://opa.osophs.dhhs.gov/pubs/eval/03may/study-overview.html

PIC ID: 7956
Agency Sponsor: OPHS-OPA, Office of Population Affairs
Federal Contact: Kappeler, Evelyn, 301-594-7608
Performer: Urban Institute, Washington, DC

Verfication of Encounter Data: Data Reasonability Analysis

This report presented a summary of Medstat’s evaluation of the data submitted as part of the Encounter Data Validation Project. The completeness, accuracy, timeliness, and overall quality of data submitted by the Medicare Choices Demonstration plans is described. The findings were that all plans ceased participating in the demonstration and either disbanded or were converted to Medicare+Choice (M+C) plans by December 31, 2000.

PIC ID: 7928
Agency Sponsor: CMS, Centers for Medicare and Medicaid Services
Federal Contact: McVicker, Victor, 410-786-6681
Performer: T. Head and Company, Inc., 1090 Vermont Avenue N.W., Washington, DC

Goal 5 - Improve the Quality of Health Care Services

Objective 5.1 - Reduce Medical Errors

Investigation of the Utility of Healthcare Costs and Utilization Project Data for Annual Surveillance of Hospitalized Injuries

PIC ID: 7999
Agency Sponsor: CDC-NCIPC, National Center for Injury Prevention and Control
Federal Contact: Johnson, Renee, 770-488-1479
Performer: Battelle, Durham, NC

Objective 5.2 - Increase the Appropriate Use of Effective Health Care Services by Medical Providers

U.S. Preventive Services Task Force Evaluation

This evaluation sought to: (1) determine how useful the United States Preventive Services Task Force (Task Force) recommendations and products were, to primary care clinicians training programs and how the products could be made more meaningful, useful and relevant; (2) enumerate the ways that Task Force recommendations and other products are being used by training programs. (3) determine the optimal way to format and disseminate Task Force products including exploration of the acceptability of web-based media as a dissemination tool for future products in order to have the greatest impact on training programs. The evaluation consisted of a telephone survey designed to assess the satisfaction of health care teaching professionals with the informational materials developed by the Task Force. In addition to assessing the level of satisfaction with Task Force materials, the survey also obtained information on instructor knowledge and satisfaction with the Healthy People 2010 materials, Guide to Community Preventive Services and the Health Employer Data Information Set measures. The study concluded that: (1) More than 20% of educators contacted had no knowledge of the Task Force or its materials. (2) 93% of the instructors familiar with the Task Force were also familiar with some of the materials. The peers of over 85% of respondents familiar with the Task Force were also familiar with Task Force recommendations. (3) Task Force Materials were considered valuable tools both for  instruction and clinical practice; more than 70% of respondents rated the materials as an “8” or higher on a 10 point scale. This applied to both instructional and clinical uses. (4) 5% of respondents indicated they had decided not to use Task Force materials in their programs. (5) Respondents indicated that the Task Force Web Site was the most effective means of providing information on Task Force recommendations; 30% of respondents chose the Web Site; 25% chose the Guides to Clinical Practice; 20%, the Clinicians Handbook of Preventive Services. (6) 33% of the respondents believed that the Task Force had a substantial impact on training preventive services professionals. Relating to the questions concerning CDC preventive medicine programs the pertinent survey results included: (1) The survey population was considerably more familiar with the Healthy People 2010 initiative than the other 2 programs. More than 86% of respondents were aware of Healthy People 2010 compared to roughly 35% for the Guide to Community Preventive Services, and 39% for the Health Employer Data Information Set (HEDIS) measures. (2) On the whole, respondents viewed the CDC sets as materials as useful for instruction and clinical practice. Of those with knowledge of the materials, more than 86% viewed Healthy People 2010 as useful. The figure for the Community Guide was 92.9%, and for the HEDIS measures 73.2%.

PIC ID: 7673
Agency Sponsor: AHRQ, Agency for Healthcare Research and Quality
Federal Contact: Coopey, Margaret, 301-594-4022
Performer: Washington Consulting Group, Bethesda, MD

Leadership by Example: Coordinating Government Roles in Improving Health Care Quality

The final report, Leadership by Example: Coordinating Government Roles in Improving Health Care,” focuses on six government programs: Medicare, Medicaid, the State Children’s Health Insurance Program, the Department of Defense’s TRICARE and TRICARE for Life Programs (considered as one program), the Veteran’s Health Administration, and the Indian Health Service. The report finds many opportunities for improvement within and across these major health delivery programs. Current quality improvement efforts are hampered by a lack of consistency in performance measurement requirements within and across programs, the burden which the absence of standardized measures puts on providers, the lack of  a conceptual framework for selecting performance measures, a lack of computerized clinical data, the lack of a strong commitment to transparency and the availability of data in the public domain, and the  absence of a systematic approach to assessing the impact of quality improvement efforts. Its recommendations set forth an ambitious agenda which is characterized by a high degree of cross- department coordination. It stresses the importance of using a coordinating body such as the already existing Quality Interagency Task Force as a vehicle for coordination within the Federal Government and for collaboration with private sector groups such as the National Quality Forum, the Leapfrog Group, the National Committee for Quality Assurance, the Joint Commission on Accreditation of Healthcare Organizations, and the Foundation for Accountability.

PIC ID: 7894
Agency Sponsor: ASPE-OHP, Office of Health Policy 
Federal Contact: Taplin, Caroline, 202- 690-7906
Performer: Institute of Medicine, Washington, DC

Impact of the Mammography Quality Standards Act of 1992

The study estimated the economic impact of regulations developed under the MQSA. It evaluated conditions of facilities after implementation of the regulations in order to evaluate whether patient access to quality screening services has been adversely affected. The contractor examined agency data sets as well as other public sources to estimate the effect of regulations on facility closures and patient access. Field contacts were used to discuss impacts with industry experts. Preliminary findings are that the regulations have not adversely affected patient access to high quality mammography screens.

PIC ID: 6080.4
Agency Sponsor: FDA-CDRH, Center for Devices and Radiological Health
Federal Contact: Tucker, Steve, 301-827-5339
Performer: Eastern Research Group, Lexington, MA

Objective 5.4 - Improve Consumer and Patient Protections

Addressing the New Health Care Crisis: Reforming the Medical Litigation System to Improve the Quality of Health Care

PIC ID: 7933
Agency Sponsor: ASPE-ODALTCP, Office of Disability, Aging, and Long-Term Care Policy
Federal Contact: Bergofsky, Linda, 202-690-6443
Performer: Department of Health and Human Services, Washington, DC

Privacy Issues in Mental Health and Substance Abuse Treatment: Information Sharing Between Providers and Managed Care Organizations

Confidentiality is a key element of mental health and substance abuse treatment. In the absence of assured confidentiality, many patients with mental disorders or substance abuse problems might refuse or fail to seek treatment. As the payers of treatment, however, managed care organizations (MCOs) and insurance companies need to know the services for which payment is being requested and whether the treatment is appropriate. The dual needs for confidentiality and disclosure have created tension between providers and payers of services. This report clarifies the sources of tension between providers and payers with regard to what personal information should be shared for patients receiving mental health or substance abuse treatment. It also provides information to support a m ore consistent application of privacy-sensitive approaches to collecting personal health information in the future. It was found that MCOs collect personal information about enrollees receiving mental health and substance abuse services for many reasons. It was found that there was wide variation in the information collected by MCOs for authorizing outpatient treatment, although there are some common elements. It was also found that respondents for inpatient authorizations described them as much more intrusive.

http://aspe.hhs.gov/datacncl/reports/MHPrivacy/index.htm

PIC ID: 7881
Agency Sponsor: ASPE-OSP, Office of Science Policy
Federal Contact: Fanning, John, 202- 690-7100
Performer: Mathematica Policy Research, Inc., Plainsboro, NJ

HCUP Quality Indicators

This proposal extended work that was being conducted on the first two modules of the next generation of Healthcare Cost and Utilization Project (HCUP) Quality Indicators (QI) developed by Stanford University. The development of reliable and valid QI for HCUP is a high priority and this proposal aided in this effort. The current validation proposal was funded to evaluate approximately 45 measures developed by Stanford. The project validated approximately 30 additional measures that are currently being developed by Stanford focusing on complications or adverse medical events. Overall, all users agreed that the tool was easy to use, had lots of good choices, and had a nice layout. According to one user, “The most saving grace of [this tool] is that a lot of us downloaded that info from AHRQ, and it is extremely cumbersome. That’s why I am so thrilled with this.” Users were eager to learn about the availability of the final product, requesting Utah Department of Health staff to inform them as soon as the tool becomes available. In addition to positive feedback, participants suggested a number of creative improvements for data display and tool functionality that would better meet their needs.

PIC ID: 7672
Agency Sponsor: AHRQ, Agency for Healthcare Research and Quality
Federal Contact: Elixhauser, Anne, 301-594-6815
Performer: Research Triangle Institute, Research Triangle Park, NC

Patient safety training & outplacement program

This feasibility study sought to determine the feasibility of establishing, operating, and maintaining an applied patient safety training and outplacement program. The study strongly supports the concept of and need for a patient safety training program; however, key informants did not agree on who should participate, what the program should look like, or how it should be implemented. The study laid out the leading alternatives AHRQ must chose among with respect to participants, educational format and curriculum, desired impact, and implementation.

PIC ID: 7683
Agency Sponsor: AHRQ, Agency for Healthcare Research and Quality
Federal Contact: Keyes, Marge, 301-594-1824
Performer: Research Triangle Institute, Research Triangle Park, NC

Objective 5.5 - Accelerate the Development and use of an electronic health information infrastructure

A guide for public health agencies developing, adopting, or purchasing interactive web-based data dissemination systems

This project developed guidelines to aid agencies in making web-related decisions. The information presented in these guidelines provides a framework for making an informed decision about whether to develop, adopt, or purchase a Web-based data dissemination system. Increasingly, public health agencies are using the Internet to disseminate data. Many of these agencies are in various stages of developing, adopting, or purchasing systems that will better enable them to combine data from different sources, improve the timeliness, accuracy, and quality of the data, and present the data in a way that is useful to their constituents. The increased activity and interest by state and local health agencies have created a need for uniform guidance to address the key organizational, technical, and system design issues  involved in developing, adopting, or purchasing a system for Web-based data dissemination. The guidelines also outline the key tasks and steps within each task required for implementation, with attention to best practices in usability and accessibility of Web-based systems. The guidelines focus on the user interface of a system; they are not intended to address the complete process of designing an entire Web application. By following the steps contained in these guidelines, public health agencies will have: (1) conducted a full analysis of implications and costs associated with their choice of option (2) completed the individual stages of implementing the developed, adopted or purchased system (3) tested and evaluated the system, and (4) developed an ongoing maintenance plan. Incorporating the best practices and procedures addressed in the many references and resources provided throughout the guidelines, and using the hands-on tools provided in the appendices, will help ensure that the developed, adopted, or purchased system is consistent with industry standards, federal guidelines, and the specific needs of the public health agency within the environment and context in which it operates.

http://www.macroint.com/~atl/dphsi/35016-23%20Web-based%20Guide.pdf

PIC ID: 7751.6
Agency Sponsor: CDC-EPO, Epidemiology Program Office
Federal Contact: Schumacher, Patricia, 770-488-8375
Performer: ORC Macro, Calverton, MD

Usability and focus group testing of the NIH senior health web site

PIC ID: 7921
Agency Sponsor: NIH-NIA, National Institute on Aging
Federal Contact: Dailey, Stephanie, 301-496-1754
Performer: UserWorks, Silver Spring, MD

Goal 6 - Improve the economic and social well-being of individuals, families, and communities, especially those most in need

Objective 6.1 - Increase the proportion of low-income individuals and families, including those receiving welfare, who improve their economic condition

Study of the TANF Application Process

This study collected comprehensive information about application process changes under Temporary Assistance for Needy Families (TANF) and State data systems measuring application events and results. The information is being used to understand how those changes and data systems affect the meaning of applications data and their comparability across States and over time. Major findings include: (1) In most study sites, families who decided not to apply for benefits were no worse off, and often better off, than families who filed for and were receiving TANF benefits. (2) States often differ on how they define and count TANF application events. Some of the biggest differences include how states handle individuals returning to TANF after a recent case closing and how states count applicants who withdraw their application before eligibility can be determined. (3) As the types of benefits funded by the TANF block grant have expanded, many states include counts of applications for benefits other than ongoing TANF cash assistance in their data on applications, but not necessarily in their data on caseloads. (4) States use a variety of codes for application denials and do not always use mutually exclusive reason categories. (5) The differences in the definition and measurement of various application events are large enough to compromise the analysis of application data across states. (6) Changes in which events count as “applications approved” and “applications denied” since welfare reform also compromise the analysis of trends within states. (7) The collection of consistent national data for applications would be difficult.

http://www.acf.hhs.gov/programs/ofa/ifr/index.htm

PIC ID: 8150
Agency Sponsor: ACF-OFA, Office of Family Assistance
Federal Contact: Poe, Dennis, 202- 401-4053
Performer: Abt Associates Inc., Cambridge, MA

Marriage and family formation data analysis project

PIC ID: 8151
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Campbell, Nancye, 202-401-5760
Performer: Abt Associates Inc., Cambridge, MA

The fiscal effects of welfare reform

Building on a four-state pilot project, case studies were conducted in 13 states to assess changes in social services spending and policies after welfare reform was implemented. Major findings were: (1) Overall State spending on social services increased (adjusting for inflation and changes in the poverty population); the median study State increased real spending per person in poverty by 7 percent between 1995 and 1999, with spending from both federal and State funds up over the period. (2) Between FY 1995 and 1999, every State in the study decreased assistance spending sharply, consistent with caseload declines; however, almost every State increased non-cash assistance social services spending to offset cash assistance decline. The total median increase was 36 percent for cash assistance and all other programs combined.

http://stateandlocalgateway.rockinst.org/fiscal_pub/other/docs/RIG_Brookings_ACF_Report_final.pdf

PIC ID: 7545
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Dubinsky, Michael, 202-401-3442
Performer: State University of New York at Albany, Albany, NY

Welfare transitions in the 1990s: the economy, welfare policy and the EITC

This study examined the reasons for the sharp caseload declines that occurred in the nation from 1986 to 1999. It used data from Survey of Income and Program Participation to develop caseload models. Unlike most studies of this nature which directly model the caseload, this study separately modeled entry and exit rates. It should be noted that the work of Haider and Klerman has shown that models that directly estimate the caseload are misspecified and give biased results. The models used here included economic, demographic and programmatic variables. Programmatic variables included the implementation of TANF and whether the state had been granted waivers. Economic variables included EITC levels, an area not addressed in most studies of the caseload declines of the 1990s. There were declines in caseloads caused by decreases in entry rates and increases in exit rates. Major findings included the following: (1) reductions in entry rates were most significantly affected by the economy, the decline in the real value of cash assistance, and the expansion of the EITC; (2) increases in exit rates were most significantly affected by the economy and federal welfare reform; (3) federal reform had the greatest affect on exit rates for long-term spells of welfare; and (4) welfare reform and the EITC played the greatest role in caseload reduction and the economy played a smaller role.

PIC ID: 7537
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Sternbach, Leonard, 415-437-7671
Performer: Jeffrey Grogger, Chicago, IL

Ohio works first

This demonstration evaluated the Ohio Works First (OWF) provisions implemented under Temporary Assistance for Needy Families (TANF). OWF provisions include: (1) completion of a self-sufficiency contract, (2) more generous income disregards, (3) a 36-out-of-any-60 month time limit, and (4) whole family sanctions. In Ohio, local jurisdictions have great flexibility in running their TANF programs, and this evaluation looked carefully at implementation efforts in 10 counties. There was great variation in how counties implemented TANF from both an administrative and policy standpoint. There were different degrees of specialization of caseworkers with some counties splitting the eligibility and work related components and others not doing so. In some cases various activities were contracted out while others were handled in-house. Counties offered various approaches to diversion activities and different services and incentives to support persons being diverted. There were differences in how counties handled persons that were approaching the end of their time limits. Some had dedicated workers that worked with these cases. During the 5-year period of the study the caseload dropped dramatically in all counties.  While OWF participation and cash assistance declined, total family income and child well-being remained constant.

http://jfs.ohio.gov/oraa/Final_Report.pdf

PIC ID: 6758
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Sternbach, Leonard, 415-437-7671
Performer: Ohio Department of Human Services, Columbus, OH

The Illinois families study

This project assessed the relationship between welfare use, income, employment and training, and health care coverage, on the one hand, and multiple measures of child well-being within the TANF recipient population on the other. As families transition from welfare to work in the context of welfare reform policies, large variation in these features is inevitable. The focus of the research was the first three (3) years of an ongoing panel study involving 1,500 families who were receiving TANF in late 1998. Annual survey interviews assessed parental reports of each child’s academic performance, behaviors, and health, as well as key risk and protective factors. The survey data were combined with administrative data from the public welfare system, the unemployment insurance system, children’s school records, medical chart reviews of a subset of children from the sample, and Medicaid claims data. Major findings included: in Illinois, the goal of decreasing welfare dependence was clearly met; efforts to increase work were moderately successful, although early success appears to be leveling off; progress was also made towards the family formation goals, although it is unclear to what extent welfare reform was responsible for observed changes in marriage and childbearing.

http://www.northwestern.edu/ipr/publications/papers/ifsyear3.pdf

PIC ID: 7533
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Yaffe, Alan, 202-401-4537
Performer: Northwestern University, Evanston IL

Evaluation of Montana's TANF program: an assessment of welfare reform in a rural setting

PIC ID: 8001
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Dubinsky, Michael, 202-401-3442
Performer: Abt Associates Inc., Cambridge, MA

Minnesota WorkFirst Program (track 2)

Using a quasi-experimental, pre-test/post-test comparison group design consisting of an impact and process study, this study compared the alternative Welfare-to-Work program strategies (WorkFirst and Minnesota Family Investment Plan (MFIP)). WorkFirst uses a rapid labor force attachment strategy versus MFIP, which represents a progressive labor force attachment model. The major finding of the study indicates that the effects of both programs on welfare participation were virtually identical. Clients in both programs had the same levels of employment, earnings, overall well being, family formation, and educational experience for children.

http://www.cortidesignhost.com/maximus/cpss/Evaluation_of_Minnesotas_WorkFIRST- MFIP_Programs.pdf 

PIC ID: 6825
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Dubinsky, Michael, 202-401-3442
Performer: Minnesota Department of Human Services, St. Paul, MN

The indiana welfare reform evaluation: five-year impacts, implementation, costs and benefits

This project evaluated Indiana’s welfare reform program, which included a strong work-first focus combined with a 24-month time limit on adults’ receipt of assistance, a family cap policy, and a number of personal responsibility provisions. The program was evaluated using a statewide random assignment experiment. Major impact findings included the following: For the first-year cohort, Indiana’s program reduced receipt of TANF and food stamps over the 5-year follow-up period, and increased earnings and employment rates in each of the follow-up years, but did not affect families’ total income; welfare reform benefited taxpayers because savings more than offset welfare reform expenditures.

http://www.acf.hhs.gov/programs/opre/indiana5year/indiana_5yr_rpt.pdf

PIC ID: 6869
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Yaffe, Alan, 202-401-4537
Performer: Indiana Family and Social Services Administration, Indianapolis, IN

Achieving change for texans: evaluation

PIC ID: 6765
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Koerper, Karl, 202-401-4535
Performer: State of Texas, Department of Human Services, Austin, TX

Welfare reform in miami: implementation, effects, and experiences of poor families and neighborhoods

This report focuses on the implementation of welfare reform in Miami-Dade County between 1996 and 2002. Florida’s welfare reform went beyond federal law by imposing a relatively short, two- or three-year time limit on benefit receipt and by cutting off all cash assistance when adults failed to comply with work requirements or other rules. However, to encourage work, Florida’s reform also allowed welfare recipients to keep more of their earnings without losing welfare eligibility. Far more families were cut off welfare because of noncompliance with work rules than because of time limits. Miami-Dade’s welfare rolls fell sharply, but how much of the decline was due to welfare reform is unclear. Between 1993 and 2002, Miami-Dade’s welfare caseload dropped by 75 percent as families both left welfare at a faster rate and came onto the rolls more slowly. A review of county welfare records found that these trends began well before 1996 and continued unabated thereafter, making it difficult to discern whether welfare reform had any effect. Welfare reform appears to have led to an increase in employment among welfare recipients. A sample of nearly 600 women who received welfare shortly before the implementation of welfare reform was interviewed in 1998 and again in 2001. Most reported that they had exited welfare, started working, and increased their income. In general, the women earned low wages. Nevertheless, the women tended to be better off financially than when they were on welfare, particularly if they took advantage of the Earned Income Credit. Neighborhood conditions remained stable or improved. Both countywide and in   the poorest neighborhoods, there were substantial declines in the teen birthrate, infant deaths, child abuse and neglect, and violent crime.

http://www.mdrc.org/publications/387/full.pdf

PIC ID: 8103
Agency Sponsor: ASPE-OHSP, Office of Human Services Policy
Federal Contact: Lower- Basch, Elizabeth, 202-690-6808
Performer: MDRC, New York, NY

Welfare time limits

PIC ID: 7621
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Dubinsky, Michael, 202-401-3442
Performer: Manpower Demonstration Research Corporation, New York, NY

Privatization in practice: case studies of contracting for TANF case management

The purpose of this project was to conduct case studies of state and local governments that have privatized case management under the Temporary Assistance for Needy Families (TANF) program, a function traditionally provided by the human services agency. Privatization of welfare services increased significantly and expanded into new services after passage of the Personal Responsibility and Work Opportunity Reconciliation Act of 1996 (PRWORA) gave states additional flexibility to design and operate their TANF programs. Despite its increasing popularity, privatizing welfare services poses significant challenges to the state and local government agencies that are responsible for contracting out. Study sites contracted with a variety of organizations, including both for-profits and non-profits (including faith- based organizations), and used a variety of contract types, including pure pay-for-performance, cost- reimbursement, fixed price, and hybrid contracts. The project described the key decisions and activities undertaken in privatizing TANF case management and documented the lessons learned in six study sites from their experiences in privatizing TANF case management. Key topics included (1) the why, what and to whom of privatizing TANF case management; (2) ensuring a fair, effective and competitive procurement process; (3) designing contract performance measures and payment structures; (4) monitoring TANF contractors to ensure accountability; (5) addressing the challenges of service provision under privatization; and (6) some key lessons learned by public welfare agencies as they took on new responsibilities in privatizing TANF case management.

http://aspe.hhs.gov/hsp/privatization-rpt03/index.htm

PIC ID: 7755
Agency Sponsor: ASPE-OHSP, Office of Human Services Policy
Federal Contact: McCormick, Ann, 202-401-6659
Performer: Mathematica Policy Research, Inc., Washington, DC

families on TANF in illinois: assets and liabilities

PIC ID: 8100
Agency Sponsor: ASPE-OHSP, Office of Human Services Policy
Federal Contact: Tambornino, John, 202-401-6639
Performer: Mathematica Policy Research, Inc., Princeton, NJ

Monitoring outcomes for los angeles county's pre- and post-CalWORKs leavers: How are they faring?

This study examined the post-welfare experiences of two cohorts of welfare leavers: a group who exited welfare in Fall 1996 (pre-CalWORKs) and a group who exited welfare in Fall 1998 (post-CalWORKs). Welfare and other public benefit receipt and employment were tracked using administrative data for approximately 27,000 leavers. To provide detail unavailable through administrative records, a sample of 340 Fall 1998 leavers were selected for in-depth interviews, of which 62 percent completed a mixed-  mode survey. This was one of the 13 studies of welfare leavers funded by ASPE. With respect to demographics, the two cohorts of leavers looked very similar. However, the post-CalWORKs leavers had more employment experience during the year prior to leaving welfare, possibly due to work requirements and expanded earned income disregards. During the year after exit, 52 percent of pre-CalWORKs leavers and 59 percent of the post-CalWORKs leavers worked at least one quarter. Both cohorts experienced low earnings and modest earnings growth during their first year off of welfare. Most were not earning enough to lift their families out of poverty without other sources of income. For the survey sample, where household income is available, the average monthly income was above the poverty level -- $1,466. However, 54 percent were living in households with income below the poverty level. There were substantial increases in post-exit participation in Medicaid. 92 percent of the post-CalWORKs leavers received Medi-Cal, compared with 65 percent of the pre-CalWORKs leavers. Most of this increase was due to gains among those who stayed off welfare, but received Medi-Cal. Gains in Food Stamp participation were more modest. Leavers who did not work after exit, or worked only some quarters, were more likely to report multiple barriers to employment than those who worked steadily. They also reported higher levels of hardship, and were more likely to receive assistance from government agencies or other sources. However, even those who worked steadily reported significant hardships.

http://www.mdrc.org/publications/326/full.pdf

PIC ID: 7754.01
Agency Sponsor: ASPE-OHSP, Office of Human Services Policy
Federal Contact: Lower- Basch, Elizabeth, 202-690-6808
Performer: MDRC, New York, NY

core performance indicators for homeless-serving programs administered by the U.S. department of health and human services

PIC ID: 8050
Agency Sponsor: ASPE-OHSP, Office of Human Services Policy
Federal Contact: Fletcher, Anne, 202-690-5739
Performer: Johns Hopkins University, Baltimore, MD

Indicators of Welfare Dependence: Annual Report to Congress 2003

The Welfare Indicators Act of 1994 requires the Department of Health and Human Services to prepare annual reports to Congress on indicators and predictors of welfare dependence. The “2002 Indicators of Welfare Dependence”, the sixth annual report, provides welfare dependence indicators through 2000, reflecting changes that have taken place since enactment of the Personal Responsibility and Work Opportunity Reconciliation Act in August 1996. This 2003 report uses data from the Current Population Survey (CPS) and administrative data to provide updated measures through 2000 for several dependence indicators. Selected highlights of the report include: 1) In 2000, 3.0 percent of the total population was dependent in the sense of receiving more than half of total family income from TANF, food stamps, and/or SSI. 2) Although the 2001 dependency rate cannot yet be calculated, preliminary data suggest it will remain approximately 3 percent. 3) The drop in dependence parallels the more well-known drop in AFDC/TANF and food stamp caseloads. For example, the percentage of individuals receiving AFDC/TANF fell from 4.6 percent to 1.9 percent between 1996 and 2001. 4) In an average month in 2000, more than half of TANF recipients lived in families with at least one family member in the labor force.

http://aspe.hhs.gov/hsp/indicators03/

PIC ID: 7281.5
Agency Sponsor: ASPE-OHSP, Office of Human Services Policy
Federal Contact: Isaacs, Julia, 202-690-7507
Performer: Office of Program Planning and Evaluation, Bethesda, MD

Objective 6.2 - Increase the proportion of older Americans who stay active and healthy

performance outcome measures project - field testing phase

The Performance Outcomes Measures project (POMP) has been sponsored by the Administration on Aging (AoA) to develop performance-based measures for evaluating aging services. Another goal is to develop a prototype system for recording and reporting agency level information. POMP is a multi-agency collaboration involving state and local agencies on aging, AoA, contractors and consultants. The performance measures developed for this project covered a range of issues such as documenting the services which are targeted to persons with the greatest social and economic needs, as the Older Americans Act requires. It also identified, explored, or monitored important aspects of the services provided on the communities and client populations served by programs, even where no clear link to services can be established and analyzed. AOA recently provided grants to thirteen states for a third year of field testing of the survey instruments developed during the first year of the project. The Performance Outcome Measures Project has resulted in the development of measurement instruments that the Administration on Aging has applied on a national basis in FY 2003 and FY 2004. The resulting national data are key to AoA performance measures under the Government Performance and Results Act and OMB’s Program Assessment Rating Tool. AoA has extended the project to pursue the development of performance measurement instruments that link program impact to program costs.

PIC ID: 6855
Agency Sponsor: AOA-OASA, Office of Assistant Secretary for Aging
Federal Contact: Burns, Frank, 202-357-3516
Performer: Westat, Inc., Rockville, MD

Objective 6.3 - Increase the independence and quality of life of persons with disabilities, including those with long-term care needs

estimates of the risk of long-term care: assisted living and nursing home facilities

This study examined the characteristics of persons entering assisted living and nursing facilities to determine what factors were associated with the risk of moving to these types of settings, and whether the factors associated with assisted living facility transitions are different from those associated with traditional nursing home admission. The time until an elderly person residing in the community entered a nursing home or an assisted living facility was modeled using a hazard function, which gives the probability at any point in time that the individual will experience a transition of a particular type. Analyses supported the hypothesis that assisted living may substitute for nursing homes for some segments of the elderly population. However, there is also evidence that nursing homes are more likely to serve lower income and older populations and those with the most severe disabilities.

http://aspe.hhs.gov/daltcp/Reports/riskest.htm

PIC ID: 8013
Agency Sponsor: ASPE-ODALTCP, Office of Disability, Aging, and Long-Term Care Policy
Federal Contact: Marton, William, 202-690-6443
Performer: Urban Institute, Washington, DC

State Nursing home quality improvement programs: site visit and synthesis report

The purpose of this study is to inform state and federal policymakers about state-initiated quality improvement programs for nursing home quality of care, with the goal of providing information to states that may wish to develop similar programs. Some states are using consultative, collaborative technical assistance (TA) programs in an effort to improve nursing home quality in addition to the traditional regulatory approach embedded in survey and enforcement processes. These TA programs allow the states to provide on-site consultation, training, and/or sharing of best practices in an effort to improve nursing home quality of care. Key stakeholders were contacted in seven states-- Florida, Iowa, Maine, Maryland, Missouri, Texas, and Washington—to gain information about states’ TA programs. The design and focus of TA programs varied across states, but shared several characteristics: the provision of on-site training and consultation with nursing facility staff; an emphasis on collaborative approaches between facilities and TA staff; a non-punitive approach that typically was not shared with the survey and certification agency unless serious violations were observed. The circumstances leading to a state’s decision to use a TA program were unique, yet the catalyst for the decision process was the same---the desire to “try something new” and provide a positive stimulus to quality improvement in addition to the long-term care survey process. The main findings from the study were: (1) that the relationship established between TA and survey program is a critical decision point for these states, (2) non- mandatory TA programs may not reach facilities most in need of help, but a non-mandatory program  may be the only option for some states with budget limitations, and (3) evaluation needs to be part of the initial program’s design, which is particularly difficult since states often implement several statewide programs simultaneously.

http://aspe.hhs.gov/daltcp/Reports/statenh.htm

PIC ID: 8014
Agency Sponsor: ASPE-ODALTCP, Office of Disability, Aging, and Long-Term Care Policy
Federal Contact: Harvell, Jennie, 202-690-6443
Performer: Abt Associates Inc., Cambridge, MA

long term financing

The Long-Term Care Financing Model is a resource which has been extensively used by ASPE to project future long- term care utilization and expenditures and simulate various long term care policy options. These include expansions of public financing, such as those proposed during health care reform, as well as changes to Medicaid and Medicare. The model has also been used extensively to study private sector policy options, such as the impact of further growth on private long-term care insurance and the impacts of changing trends in disability rates on long-term care use and expenditures. Recent data on disability rates, nursing home use, and home care use will be used to update existing portions of the model. In addition, the model will be expanded to include acute care use, thereby increasing the ability to simulate a wide range of policy options. ASPE will use this computer model for projections and a series of policy simulations which will be presented in reports. In addition to its previous long-term care policy uses, the revised model will enable ASPE to address acute care issues, such as the combined burden of acute and long-term care spending on the elderly. Since the model simulates the income and assets of the population, including pension and Social Security payments, the model will also be used to study other aging-related issues, such as the impact of changes in employer-sponsored health insurance for retirees.

PIC ID: 7143
Agency Sponsor: ASPE-ODALTCP, Office of Disability, Aging, and Long-Term Care Policy
Federal Contact: Drabek, John, 202-690-6443
Performer: The Lewin Group, Fairfax, VA

Objective 6.4 - Improve the economic and social development of distressed communities

Welfare reform in Philadelphia: implementation, effects, and experiences of poor families and neighborhoods

This report focuses on the implementation of welfare reform in Philadelphia County between 1996 and 2000. Pennsylvania substantially changed its welfare system under Welfare Reform, and in Philadelphia, implementation of the law was lenient in some respects. During the first two years on welfare, recipients were asked to conduct an eight-week job search but otherwise were not held to a strict work requirement. At the 24-month limit, many parents who were not working were placed in subsidized jobs. In addition, families received extensions to the lifetime limit on benefits if they participated in assigned activities. Between 1992 and 2000, welfare receipt declined and employment increased in Philadelphia. TANF seems to have encouraged long-term recipients to leave the rolls faster, to have increased employment (but mostly unstable employment), and to have raised the likelihood that some families would return quickly to welfare. Because positive trends in welfare receipt and employment began prior  to TANF, it is clear that the economy and other factors also played important roles in these outcomes. A longitudinal survey of welfare mothers living in the city’s poorest neighborhoods suggests that, over time, more worked and fewer received welfare, while household incomes increased. Between 1992 and 2000, social conditions in the city’s poorest neighborhoods generally improved. Despite these improvements, the number of neighborhoods with high concentrations of welfare recipients declined only slightly over time, and Philadelphia’s welfare caseload remains concentrated in neighborhoods with some of the worst social and economic conditions in the city.

http://www.mdrc.org/publications/352/full.pdf

PIC ID: 7754.02
Agency Sponsor: ASPE-OHSP, Office of Human Services Policy
Federal Contact: Lower- Basch, Elizabeth, 202-690-6808
Performer: MDRC, New York, NY

Tribal self-governance demonstration feasibility study

PIC ID: 7798
Agency Sponsor: ASPE-OPPS, Office of Planning and Policy Support
Federal Contact: Rock, Andrew, 202-260-0398
Performer: Office of the Assistant Secretary for Planning and Evaluation, Washington, DC

Evaluation of a new methodology for community participation research: final report

In recent years there has been an increased focus on the importance of community-based participatory research (CBPR)- research that engages members of a community with research partners in all phases of the research process. A variety of entities including the Committee on Public Health, the Institute of Medicine and the Centers for Disease Control and Prevention (CDC) have concluded that communities must address public health issues in a comprehensive, collaborative and participatory manner. The Committee on Public Health has determined that communities must work together with researchers to address their health issues. This project assesses the participatory research methods of the CDC Urban Research Centers located in Detroit, New York City, and Seattle. These projects are in various phases of using participatory action research methods to determine “what works” to improve the quality of life of inner city, impoverished populations. The research questions included: (1) how community-based participatory research has been fostered and developed; (2) how infrastructures have been established to facilitate collaboration across diverse community partners to address urban health issues; and (3) whether and how promising approaches to addressing urban health issues have been developed, evaluated, and disseminated. The assessment of CDC URC’s participatory research methods confirmed that while CBPR yields substantial community involvement and investment in community health, it  remains a time-consuming and resource intensive process. The Cross-Site Evaluation Team determined that participatory processes help build equitable partnerships; teach communities how to examine local health problems; and teach researchers about the perspectives of the people who live and work in communities. In addition, the team determined that funding institutions like the CDC should consider the time required to implement participatory processes when making funding decisions.

PIC ID: 7870
Agency Sponsor: CDC, Centers for Disease Control
Federal Contact: Cheal, Nancy, 404- 639-7222
Performer: Macro International, Inc., Calverton, MD

Goal 7 - improve the stability and healthy development of our nation's children and youth

Objective 7.1 - promote family formation and healthy marriages

the long-term effects of the minnesota family investment program on marriage and divorce among two-parent families

This project examined the effects on marriage over a 7-year follow-up period for the full sample of two- parent families from the Manpower Demonstration Research Corporation’s (MDRC) 2000 evaluation of the Minnesota Family Investment Program (MFIP). The study found that a sample of two-parent recipient families assigned to MFIP were 40 percent more likely to be married at a three-year follow-up period than two-parent recipient families assigned to AFDC. Results show that the positive effects on marriage stability to hold up over time. Specifically, MFIP decreased divorce by 3.5 percentage points, or by about 25 percent, seven years after study entry. Impacts were most pronounced for blacks who were already married and for parents who were cohabiting at study entry and subsequently got married. Effects persisted even after the MFIP pilot program being tested went statewide in 1998, after which all two- parent families (treatment and control) were subject to the new statewide MFIP rules. MFIP had no cumulative effect on divorce among two-parent applicant families.

http://aspe.os.dhhs.gov/hsp/MFIP-2-parent03/index.htm

PIC ID: 7760
Agency Sponsor: ASPE-OHSP, Office of Human Services Policy
Federal Contact: Landey, Alana, 202-401-6636
Performer: Manpower Demonstration Research Corporation, New York, NY

State experience and perspectives on reducing out-of-wedlock births

This study examined states’ experiences related to the goal of reducing nonmarital births, and the factors that helped shape state policies since the passage of the Personal Responsibility and Work Opportunity Reconciliation Act (PRWORA) of 1996. The study provides detailed information about the experiences of a diverse sample of nine states gathered through a series of phone interviews and follow-up calls with representatives from TANF, health and other relevant agencies in these states. The PRWORA replaced  the Aid to Families with Dependent Children (AFDC) entitlement program with the Temporary Assistance for Needy Families (TANF) block grant program. In addition to moving people to self-sufficiency by promoting job preparation, work, and marriage, a major goal of TANF is reducing out-of-wedlock pregnancies. Early discussions of TANF reauthorization in 2002 included a focus of whether TANF was meeting the goals set out in PRWORA, particularly the goals related to family formation. This study determined that: (1) A majority of states have taken advantage of most welfare provisions intended to reduce nonmarital births. (2) State TANF expenditures for pregnancy prevention and two-parent family formation activities have been modest. (3) States generally emphasize programs for teens (rather than adults). (4) Funding for nonmarital birth prevention activities has increased. (5) States face difficulties serving some populations.

http://aspe.hhs.gov/hsp/nonmarital-births03/

PIC ID: 7757
Agency Sponsor: ASPE-OHSP, Office of Human Services Policy
Federal Contact: Kaye, Kelleen, 202-401-6634
Performer: The Lewin Group, Fairfax, VA and The Johns Hopkins University, Baltimore, MD

evaluation of family preservation and reunification programs: final report

The Evaluation of Family Preservation and Reunification Programs, conducted between 1994 and 2002, studied four local programs providing services intended to improve family functioning and reduce unnecessary foster care placements. This traditional, experimental-design evaluation examined outcomes of sites that each used the popular HomeBuilders service model which provides intensive services to families over a period of approximately 6 weeks. The report discusses family outcomes approximately one year after program entry. Key findings include: (1) Families served experienced a range of problems. (2) Participating families received a wider and deeper array of services than did the comparison group who received the community’s “usual services”. (3) Foster care placement was not reduced. (4) Child safety was maintained. (5) Family functioning did not generally improve. (6) All subgroups experienced similar outcomes. (7) Families thought their lives had improved. Programs were evaluated in Louisville, Kentucky; seven counties in New Jersey; Memphis, Tennessee; and Philadelphia, Pennsylvania. Volume 1 of the report includes chapters providing a study overview; study methodology and implementation; site descriptions; and family descriptions. Volume 2 includes information on services; outcomes; the Philadelphia program not described in interim reports; an attrition analysis; data on social supports; information about investigating workers and program staff; and conclusions. Volume 3 includes the study instrumentation.

http://aspe.hhs.gov/hsp/evalfampres94/

PIC ID: 5337.4
Agency Sponsor: ASPE-OHSP, Office of Human Services Policy
Federal Contact: Radel, Laura, 202-690-5938
Performer: Westat, Inc., Rockville, MD

Objective 7.2 - improve the development and learning readiness of preschool children

Child care subsidy policy issues and prospects for evaluation research in states and communities

This project sought to learn about current childcare policy issues and to develop potential research questions to inform the design of a childcare evaluation research initiative. The project was jointly sponsored by the Administration on Children and Families and the Smith Richardson Foundation.  Childcare policy issues have grown over the past decade. Greater numbers of low-income parents need childcare assistance as they make the transition from welfare to work. At the same time, childcare funding has expanded by adding resources to fund many policy and program strategies to address childcare policy issues. Mathematica Policy Research, Inc. spoke with childcare administrators representing 9 of the 10 ACF regions, 19 states and the District of Columbia. Overall, the state and regional childcare administrators expressed keen interest in participating in policy evaluation research  with other states and the ACF. Some of the issues state and regional childcare administrators identified during the project included: 1) how to balance the goal of focusing scarce childcare assistance on families with the least resources while ensuring that families continue to receive assistance during transitions, 2) how to balance the need to transition families to paying full market cost of childcare while keeping co- payments affordable, and 3) how to increase the stability of childcare assistance for eligible, low-income families. For every issue identified, a policy or program issue strategy was developed and described.

PIC ID: 8018
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Jakopic, Richard, 202-205-5930
Performer: Mathematica Policy Research, Inc., Washington, DC

Objective 7.3 - Increase the involvement and financial support of non-custodial parents in the lives of their children

State practices in medical child support cross program coordination

This report documents current policies and practices in Connecticut, Minnesota, and Texas in which the child support enforcement program (IV-D), Medicaid, and the State Children’s Health Insurance Program (SCHIP) coordinate in order to secure and sustain appropriate health care coverage for child support- eligible children. By coordinating their work, child support enforcement, Medicaid, and State Children’s Health Insurance Program (SCHIP) agencies can ensure that children are enrolled in the health care coverage that is most appropriate for them, maximizing private coverage for those children to whom it is available and affordable and reserving public resources for those who do not have access to affordable private coverage. Among the state initiatives documented, medical child support coordination was facilitated by: having common goals and objectives; specific legislation; training; previous collaboration; coordination among top program officials; shared staff responsibilities among the programs; automated data tools; collaborative design of the Medicaid/SCHIP application; cost savings to Medicaid; and a specific funding source. Cross-program coordination was challenged by a preference for the noncustodial parent, rather than the custodial family, to provide private coverage; the current regulatory requirement that any insurance available through a noncustodial parent’s employer is defined as being of “reasonable cost” and therefore must be ordered; a lack of automation; court backlogs and case continuances; lack of a federal medical child support performance incentive for child support enforcement agencies; and stigma issues.

http://aspe.hhs.gov/hsp/CSE-medsupp03/index.htm

PIC ID: 7236
Agency Sponsor: ASPE-OHSP, Office of Human Services Policy
Federal Contact: Burnszynski, Jennifer, 202-690-865
Performer: Urban Institute, Washington, DC

OCSE Responsible fatherhood programs: early implementation lessons

Eight states received Responsible Fatherhood demonstration grants or waivers through the Office of Child Support Enforcement to allow them to test comprehensive approaches to encourage more responsible fathering by non-custodial parents. Each state project was different but they all provided a range of needed services such as job search and training, access and visitation, social services or referral, case management and child support services. The initial implementation report documented the status of the program, services being delivered and child support context. The final report indicated that the responsible fatherhood services resulted in increased employment rates ranging from 8 to 33 percent, especially for those who were unemployed previously; increased child support compliance ranging from 4 to 31 percent primarily for those who had not been paying previously; and increased time spent with children with 27 percent of the fathers reported seeing their children more often after the program. The report also suggests that the child support program should be more sensitive to the limitations of low- income non-custodial parents in the establishment of child support orders. For example, for the very poor, (earnings less than $6000 a year) current child support obligations and arrearages were found to be from two to six times greater than their reported earnings. For those with earnings of $6000 to $12,000 a year, current child support and arrearages were 21 to 61 percent of reported earnings, depending upon the site.

http://www.acf.hhs.gov/programs/cse/rpt/process.htm

PIC ID: 7055
Agency Sponsor: ACF-OCSE, Office of Child Support Enforcement
Federal Contact: Arnaudo, David, 202-401-5364
Performer: The Lewin Group, Fairfax, VA and The Johns Hopkins University, Baltimore, MD

Objective 7.4 - Increase the percentage of children and youth living in a permanent, safe environment

Welfare reform and children: a synthesis of impacts in five states

The Child Outcomes Synthesis Project synthesized the results from the Project on State-Level Child Outcomes, a series of demonstrations in five States that measured the impacts of welfare reform on the well-being of children. The synthesis first looked at adult outcomes that the programs attempted to change (e.g., employment and earnings) and then turned to aspects of young children’s lives—including child care and the home environment—that also may have been changed by the programs. Finally the project focused on how children themselves were affected by the programs. Major findings included the following: There is little evidence that these programs resulted in widespread harm or benefit to young school-age children; positive impacts on children’s functioning appear to be related to increases in family income; the programs increased children’s participation in child care; and most of the programs showed few impacts on the children’s family life, given the number of measures examined. In two of the states, Florida and Minnesota, the programs had the most favorable impacts on young school-age children in more disadvantaged families, such as those with a longer history of welfare receipt or less work experience. Where there were impacts on adolescents’ school performance, they were primarily negative; apart from any program impacts, the children in these families were experiencing multiple stressors, including high levels of economic disadvantage, parental depressive symptoms, and domestic violence.

http://www.acf.hhs.gov/programs/opre/welfare_reform_children/welf_ref_child.pdf

PIC ID: 7527
Agency Sponsor: ACF-OPRE, Office of Planning, Research and Evaluation
Federal Contact: Yaffe, Alan, 202-401-4537
Performer: Child Trends, Inc., Washington, DC

National study of child protective services systems and reform efforts: summary

The purpose of this 2-year study was to describe the current status of the Child Protective Services (CPS) system in the United States and reform efforts underway around the nation to improve its operation. The study focused on the following areas: 1) state and local child protective services mandates and policies; 2)  child protective services agency functions and practices that are conducted in order to meet those mandates and policies; and 3) innovative reform efforts that are being introduced and their impact on agency functions and practices. This paper summarizes key findings on practice and policy, as well as changes being undertaken, which were identified in the national study. Topics include background, screening and triage, investigation, collaboration with law enforcement, alternatives to investigation, collaboration in providing services, and looking toward the future. These findings were discussed at a symposium of persons knowledgeable about child protective services policies and practices and their observations are included in the paper.

http://aspe.hhs.gov/hsp/CPS-status03/index.htm

PIC ID: 8036
Agency Sponsor: ASPE-OHSP, Office of Human Services Policy
Federal Contact: Radel, Laura, 202-690-5938
Performer: Walter R. McDonald & Associates, Inc., Rockville, MD

National study of child protective services and reform efforts: findings on local CPS practices

PIC ID: 8036.1
Agency Sponsor: ASPE-OHSP, Office of Human Services Policy
Federal Contact: Radel, Laura, 202-690-5938
Performer: Walter R. McDonald & Associates, Inc., Rockville, MD

National study of child protective services systems and reform efforts: review of state CPS policy

The purpose of this 2-year study was to describe the current status of the Child Protective Services (CPS) system in the United States and reform efforts underway around the nation to improve its operation. The study focused on the following areas: 1) state and local child protective services mandates and policies; 2) child protective services agency functions and practices that are conducted in order to meet those mandates and policies; and 3) innovative reform efforts that are being introduced and their impact on agency functions and practices. This “Review of State CPS Policy” report presents findings from the analysis of State policies. Chapters in administration, screening and intake, investigation, and alternative response are included. Policy manuals covering these functions were reviewed for all States and the District of Columbia. Forty-eight states participated in confirmation interviews designed to clarify policies identified in written materials. This review determined that although there is considerable variation in policy, there are common functions and features of CPS Policy that seem to reflect the requirements of the Federal Child Abuse Prevention and Treatment Act (CAPTA) and practice principles recommended by professional organizations such as the Council on Accreditation and Child Welfare League of America.

http://aspe.hhs.gov/hsp/CPS-status03/index.htm

PIC ID: 8036.3
Agency Sponsor: ASPE-OHSP, Office of Human Services Policy
Federal Contact: Radel, Laura, 202-690-5938
Performer: Walter R. McDonald & Associates, Inc., Rockville, MD

Assessing the field of post-adoption services: family needs, program models, and evaluation issues summary report

This project employed several analytic methods to explore the service needs of families following the adoption of a child from the public child welfare system and assessed the “state of the art” in the current array of post adoption services. The purpose of the project was to bring together what we know about post adoption services from existing research, what we can learn from the analysis of national and/or multi-state data and visits to existing programs, and to suggest an agenda for future federal research on these issues, particularly as they arise from the Adoption and Safe Families Act. This summary report provides a brief description of each study component (literature review, case studies of post-adoption services programs, evaluation issues, and secondary data analysis, each of which is described in full detail in separate reports) and concludes with discussions of the current status of post-adoption services and strategies to move the field forward.

http://aspe.hhs.gov/hsp/post-adoption01/summary/index.htm

PIC ID: 7952
Agency Sponsor: ASPE-OHSP, Office of Human Services Policy
Federal Contact: Radel, Laura, 202-690-5938
Performer: RTI International, Research Triangle Park, NC

Assessing the field of post-adoption services: family needs, program models, and evaluation issues

The purposes of the project were to bring together what is known about post-adoption services from existing research, what can be learned from the analysis of national and/or multi-state data and visits to existing programs, and to suggest an agenda for future federal research on these issues, particularly as they arise from the Adoption and Safe Families Act. This report focuses on evaluation issues. It reviews existing evaluations of post adoption services, including needs assessments, data on the characteristics of children and families served, client satisfaction assessments, and outcome evaluations. The report describes constraints and incentives for evaluation in this field and suggests strategies and directions for future evaluations. The child welfare field has spent a great deal of time during the last several years focusing on increasing the number of children adopted from the public child welfare system. The increasing number of children adopted has given rise to concerns about the need for improved post- adoption services to support adoptive families who may experience difficulties. Through several analytical methods, this project has explored the service needs of families following the adoption of a child from the public child welfare system and has assessed the “state of the art” in the current array of post adoption services.. Two particular strategies are suggested. The first strategy is the development of fundamental evaluation tools that can reduce the start-up costs of evaluation to programs and increase comparability of evaluations across programs. This would include consistent service classifications, a basic program data set to describe child and family characteristics and functioning, and identification of a core set of “best practices” models with recommended evaluation strategies. The second strategy is the implementation of measures to reduce barriers to evaluation at the federal and state levels. This would include promoting evaluation as a program improvement tool and making evaluation processes more useful to programs and families, earmarking evaluation funds so that evaluation activities are not perceived as impinging on service delivery; and providing technical assistance to post-adoption services programs to assist with design, data collection, and analysis.

http://aspe.hhs.gov/hsp/post-adoption01/

PIC ID: 7952.1
Agency Sponsor: ASPE-OHSP, Office of Human Services Policy
Federal Contact: Radel, Laura, 202-690-5938
Performer: RTI International, Research Triangle Park, NC

Assessing the field of post-adoption services: family needs, program models, and evaluation issues: analysis of secondary data

PIC ID: 7952.3
Agency Sponsor: ASPE-OHSP, Office of Human Services Policy
Federal Contact: Radel, Laura, 202-690-5938
Performer: RTI International, Research Triangle Park, NC

Assessing the field of post-adoption services: family needs, program models, and evaluation issues - case study report

The purpose of the project was to bring together what we know about post adoption services from existing research, what we can learn from the analysis of national and/or multi-state data and visits to existing programs, and to suggest an agenda for future federal research on these issues, particularly as they arise from the Adoption and Safe Families Act. This report provides case studies of post-adoption services models in place in five states: Georgia, Massachusetts, Oregon, Texas and Virginia. The child welfare field has spent a great deal of time during the last several years focusing on increasing the number of children adopted from the public child welfare system. The increasing number of children adopted has given rise to concerns about the need for improved post-adoption services to support adoptive families who may experience difficulties. Through several analytical methods, this project has explored the service needs of families following the adoption of a child from the public child welfare system and has assessed the “state of the art” in the current array of post adoption services. Site visits were conducted to programs in each of the five states during 1991. Sites were selected to include well- regarded programs that varied in structure and services offered. During site visits, study staff interviewed state adoption program managers and post-adoption services providers/coordinators and conducted  focus groups with adoptive parents. The case study was designed to address the following research questions: (1) What are the service needs of families following adoption of a child from the public child welfare system? (2) What are the characteristics of existing post-adoption services programs?, and (3) How are post-adoption services programs monitoring and assessing their performance.

http://aspe.hhs.gov/hsp/post-adoption01/

PIC ID: 7952.4
Agency Sponsor: ASPE-OHSP, Office of Human Services Policy
Federal Contact: Radel, Laura, 202-690-5938
Performer: Research Triangle Institute, Research Triangle Park, NC

Trends in the Well-Being of America's children & youth 2002

PIC ID: 6170.6
Agency Sponsor: ASPE-OHSP, Office of Human Services Policy
Federal Contact: Kelsey, Meredith, 202-690-6652
Performer: Westat, Inc., Rockville, MD

Trends in the Well-Being of America's children & youth 2003

This is the eighth edition of an annual report from the Department of Health and Human Services (HHS) on trends in the well-being of children and youth. The report presents the most recent and estimates on more than 80 indicators of well-being. The indicators have been organized into five broad areas: (1) population, family, and neighborhood; (2) economic security; (3) health conditions and health care; (4) social development, behavioral health; and (5) education and achievement. For each indicator, the report provides graphics to highlight key trends and important population subgroup differences and tables that provide more detailed information. The indicators reported have been collected more than once the past few years so that trends may be shown from the 70s to the year 2002. Several sections were revised for this 2003 edition. Indicators have been combined, while others have been updated with new data sources. Some were removed due to unavailable timely information. Finally, data used at the state and local levels are less plentiful than at the national level. However, even with all of this extensive data, there remain gaps in the Federal statistical system that, when filled, will give a more complete picture of the quality of our children’s lives. For example, positive measures of social development are limited, with the result that the current set of indicators may present a gloomier picture of our children’s overall well- being. The Federal statistical system is positioned to play a significant role in increasing the availability of such data for use at the state and local levels.

PIC ID: 6170.7
Agency Sponsor: ASPE-OHSP, Office of Human Services Policy
Federal Contact: Kelsey, Meredith, 202-690-6652
Performer: Westat, Inc., Rockville, MD

Goal 8 - Achieve excellence in management programs

Objective 8.1 - Create a unified HHS committed to functioning as one department

Identification of strategies for CDC/CMS

The purpose of this project was to assess previous and ongoing intragency collaborations between the Centers for Disease Control and Prevention (CDC) and the Centers for Medicare and Medicaid Services (CMS). In the last ten years, CDC and CMS have collaborated formally and informally on approximately twenty different interagency projects. The prevailing view among staff who have participated in these collaborations has been that in working together the two agencies achieve common goals more effectively and efficiently than they would if each had worked separately. The project has provided CDC and an interagency steering committee with information to guide strategic planning and analysis of options for continuous coordination between CDC and CMS in initiating and conducting interagency collaborations. The final report describes important contextual features of the CDC/CMS collaborations, describes administrative, agency and staff-related factors that affect the success of the collaborations, and provides information about whether a new interagency unit would service to improve collaborations between CDC and CMS.

PIC ID: 8047
Agency Sponsor: CDC, Centers for Disease Control
Federal Contact: McKibben, Linda, 404- 498-1113
Performer: Battelle, Durham, NC

Objective 8.3 - Enhance the efficiency and effectiveness of competitive sourcing

How to differentiate your organization and convey value to your target audiences

Seeking to apply marketing techniques to public goods, this project addressed how to develop a value proposition, define your audience’s attributes and desired outcomes, conduct audience research, and develop audience specific communication strategies. These steps also included in-depth interviews with participants and the development of audience profiles. Successful communications occur when the organization and key audiences are aligned. Brand identity of an organization is a key concept in successful communication in that it differentiates a relevant, enduring, and credible promise of value associated with a product, service, or organization and indicates the source of that value. Findings indicated a good response to the communication strategies that resulted in the development of valued relationships, and building of awareness and buy-in to the research agenda of the brand. However, it was also determined that more work is to be done to maintain the confidence of audience participants, which is directly related to building credibility with audiences as a new, and perhaps unproven, brand.

PIC ID: 8000
Agency Sponsor: CDC, Centers for Disease Control
Federal Contact: Cheal, Nancy, 404- 639-7222
Performer: ORC Macro, Calverton, MD

Objective 8.5 - enhance the use of electronic commerce in service delivery and record keeping

Result of usability testing of national library of medicine website

PIC ID: 7649
Agency Sponsor: NIH-NLM, National Library of Medicine
Federal Contact: Backus, Joyce, 301-496-7732
Performer: Human Factors International, Baltimore, MD

Objective 8.6 - Achieve Integration of budget and performance information

Healthy people 2010 community implementation (microfinancing) grants

The purpose of this report was to evaluate the effectiveness of promoting Healthy People (HP) 2010 objectives through the distribution of small grants to non-profit organizations. Methods: Yale University’s Prevention Research Center awarded “microgrants” of $2,010 to 103 organizations throughout Connecticut. Recipients completed 103 baseline and 84 follow-up surveys related to organizational capacity and knowledge about Healthy People 2010. In addition, sixty-seven control group organizations from Connecticut, Rhode Island and Massachusetts were selected for comparative analysis. Data Analysis: Descriptive statistics were calculated on the baseline and end of project surveys that were administered to the recipients of microgrants and the non-funded control group. Knowledge and utilization of HP 2010 objectives were assessed using a Likert scale. Differences among types of organizations (e.g., health and human service, educational, civic) and the focus areas/Healthy People 2010 objectives selected were also compared. Results: At the start of the study 44% of the grant recipients and 79% of the control group agencies indicated low levels of familiarity with HP 2010 goals and objectives as measured on a 10-point Likert scoring three or less (lowest levels of knowledge). Changes in knowledge of HP 2010 goals for the microgrant group increased significantly from 5.24 + 3.67 to 7.83 + 1.86 (p < .05) by the end of the project. The microgrant and control group had largely similar organizational characteristics (e.g., size, age, population served) however, more of the control group represented Health and Human Service organizations. The top five HP 2010 focus areas targeted by the grant agencies were nutrition and overweight, physical activity, access to quality health services, substance abuse and tobacco use. Conclusions: A significant proportion of agencies working in public health are largely unfamiliar with the nation’s health agenda represented by Healthy People 2010. Microgrants can be an effective way to promote Healthy People 2010 and broaden the scope of non- traditional agencies such as faith-based and civic organizations working to achieve the Healthy People 2010 goals and objectives.

PIC ID: 7831
Agency Sponsor: OPHS-ODPHP, Office of Disease Prevention and Health Promotion
Federal Contact: Martone, Sue, 202-205-5820
Performer: Prevention Research Program, Biometry Branch, Bethesda, MD

Designing an approach to assessing and reporting progress on recommendations from NCI's progress review groups (PRGs)

PIC ID: 7631
Agency Sponsor: NIH-NCI, National Cancer Institute
Federal Contact: Corrigan, James, 301-496-5515
Performer: (unknown)

Assessment of Strategies for engaging key healthy people constituents and potential partners through the health people consortium

This assessment examined the Healthy People Consortium, which includes representatives from State health agencies, as a mechanism for engaging external stakeholders in implementing Healthy People 2010 goals and objectives. Information and input from various Healthy People stakeholders, including representatives of State agencies, was collected during a State Healthy People Coordinators meeting convened in Washington, DC, December 3, 2002. It yielded important information on: (1) what strategies have or have not enhanced the achievement of Healthy People 2010 goals and objectives; (2) how Consortium members use Healthy People 2010, and (3) how strategies can improve the usefulness of Healthy People 2010.

PIC ID: 7835
Agency Sponsor: OPHS-ODPHP, Office of Disease Prevention and Health Promotion
Federal Contact: Blakey, Carter, 240-453-8254
Performer: (unknown)

Feasibility study of NHGRI's educational program

The purpose of the study was to assess the feasibility of conducting an outcome evaluation of the educational components of 50 Years of DNA: From Double Helix to Health that were aimed at high school teachers and students. The National Human Genome Research Institute (NHGRI) brought genomics to the classroom by developing on-line lesson plans, curriculum supplements, and other resources that could be used to teach genetics and the ethical, legal, and social implications of genomic research during the celebration and throughout the school year and by encouraging teachers to participate in a national “DNA Day” on April 25, 2003. Study recommendations were informed by (1) a review of the literature; (2) an assessment of the strengths and weaknesses of alternative methodologies and measures; (3) synthesis of Office of Management and Budget (OMB) requirements; and (4) examination of cost estimates of other federally funded outcome evaluations. Major findings included the following: (1) an outcome evaluation is justified for measuring the extent to which program goals are met and for guiding decisions about future allocation of resources; (2) evidence of program success can be detected by a well-designed outcome evaluation conducted with a sufficiently large sample; and (3) the most appropriate methodology is a mixed methods design that combines both quantitative and qualitative measures to evaluate program effects.

PIC ID: 7960
Agency Sponsor: NIH-NHGRI, National Human Genome Research Institute
Federal Contact: Lanier, Lisa, 301-435-2674
Performer: Clare Von Secker, Ph.D., Chevy Chase, MD

Appendix a - DHHS FY 2004-2009 Strategic Goals and Objectives

Preventing Disease and Illness

GOAL 1: Reduce the major threats to the health and well-being of Americans

Objective 1.1     Reduce behavioral and other factors that contribute to the development of chronic diseases
Objective 1.2     Reduce the incidence of sexually transmitted diseases and unintended pregnancies
Objective 1.3     Increase immunization rates among adults and children
Objective 1.4     Reduce substance abuse
Objective 1.5     Reduce tobacco use, especially among youth
Objective 1.6     Reduce the incidence and consequences of injuries and violence

Protecting our homeland

GOAL 2: Enhance the ability of the Nation’s health care system to effectively respond to bioterrorism and other public health challenges

Objective 2.1        Build the capacity of the health care system to respond to public health threats in a more timely and effective manner, especially bioterrorism threats

Objective 2.2        Improve the safety of food, drugs, biological products, and medical devices

Closing the gaps in health care

GOAL 3: Increase the percentage of the Nation’s children and adults who have access to health care services, and expand consumer choices

Objective 3.1     Encourage the development of new, affordable health insurance options
Objective 3.2     Strengthen and expand the health care safety net
Objective 3.3     Strengthen and improve Medicare
Objective 3.4     Eliminate racial and ethnic health disparities
Objective 3.5     Expand access to health care services for targeted populations with special health care needs
Objective 3.6     Increase access to health services for American Indians and Alaska Natives (AI/AN)

Improving health science

GOAL 4: Enhance the capacity and productivity of the Nation’s health science research enterprise

Objective 4.1     Advance the understanding of basic biomedical and behavioral science and how to prevent, diagnose, and treat disease and disability
Objective 4.2     Accelerate private sector development of new drugs, biologic therapies, and medical technology
Objective 4.3     Strengthen and diversify the pool of qualified health and behavioral science researchers
Objective 4.4     Improve the coordination, communication, and application of health research results
Objective 4.5     Strengthen the mechanisms for ensuring the protection of human subjects and the integrity of the research process

Realizing the possibilities of 21st century health care

Objective 5.1     Reduce medical errors
Objective 5.2     Increase the appropriate use of effective health care services by medical providers
Objective 5.3     Increase consumer and patient use of health care quality information
Objective 5.4     Improve consumer and patient protections
Objective 5.5     Accelerate the development and use of an electronic health information infrastructure

Working toward independence

GOAL 6: Improve the economic and social well-being of individuals, families, and communities, especially those most in need

Objective 6.1     Increase the proportion of low-income individuals and families, including those receiving welfare, who improve their economic condition
Objective 6.2     Increase the proportion of older Americans who stay active and healthy
Objective 6.3     Increase the independence and quality of life of persons with disabilities, including those with long-term care needs
Objective 6.4     Improve the economic and social development of distressed communities
Objective 6.5     Expand community and faith-based partnerships

Leaving no child behind

GOAL 7: Improve the stability and healthy development of our Nation’s children and youth

Objective 7.1     Promote family formation and healthy marriages
Objective 7.2     Improve the development and learning readiness of preschool children
Objective 7.3     Increase the involvement and financial support of non-custodial parents in the lives of their children
Objective 7.4     Increase the percentage of children and youth living in a permanent, safe environment

Improving Department Management

GOAL 8: Achieve excellence in management practices

Objective 8.1     Create a unified HHS committed to functioning as one Department
Objective 8.2     Improve the strategic management of human capital
Objective 8.3     Enhance the efficiency and effectiveness of competitive sourcing
Objective 8.4     Improve financial management
Objective 8.5     Enhance the use of electronic commerce in service delivery and record keeping
Objective 8.6     Achieve integration of budget and performance information
Objective 8.7     Reduce regulatory burden on providers and consumers of HHS services

Appendix B - Agency Mission and Evaluation program statements

Administration for children and families

Mission
To promote the economic and social well-being of families, children, individuals, and communities.

The Administration for Children and Families (ACF) administers a broad range of formula and discretionary programs, including family self-sufficiency (Temporary Assistance for Needy Families); child support; children and family services (Head Start, Child Welfare, Family Preservation and Support, and youth programs); and special programs for targeted populations, such as the developmentally disabled, refugees, and Native Americans.

Administration on aging

Evaluation Program
The Administration on Aging (AoA) is the federal focal point and advocate agency for the concerns of older persons. The agency administers key federal programs mandated under various titles of the Older Americans Act. These programs help vulnerable older persons remain in their own homes by providing supportive services. Other programs offer opportunities for older Americans to enhance their health and to be active contributors to their families, communities, and the Nation through employment and volunteer programs. AoA works closely with its nationwide network of regional offices and State and Area Agencies on Aging to plan, coordinate, and develop community-level systems of services that meet the unique needs of individual older persons and their caregivers. AoA collaborates with other federal agencies, national organizations, and representatives of business to ensure that, whenever possible, their programs and resources are targeted to the elderly and coordinated with those of the network on aging. As the responsibilities of this nationwide network of State and Area Agencies on Aging continue to grow, it is essential that they have the necessary information to meet these responsibilities.

The overall evaluation priorities of the AoA are to support studies that provide information on: (1) successful program implementation in meeting the goals of the Older Americans Act; (2) design and operation of effective programs; and (3) issues relevant to policy development, legislative planning, and program management.

Agency for healthcare research and quality

Mission
To improve the quality, safety, efficiency and effectiveness of health care for all Americans.

Evaluation Program
Evaluation activities within the Agency for Healthcare Research and Quality (AHRQ) provide executive management, program officers and audiences external to the Agency with evaluative findings concerning the Agency’s effectiveness and efficiency in meeting its GPRA, PART and other performance goals. The work is conducted by external, independent evaluators and complies with (OMB) Paperwork Reduction Act requirements. Evaluation components are built into virtually all major AHRQ programmatic or       portfolio activities beginning at the design phase. Among evaluation mechanisms used by the Agency are targeted evaluation studies undertaken through contracts that use a variety of quantitative and qualitative methods and that tend increasingly to provide more real-time monitoring feedback.

Evaluation activities also include satisfaction feedback from AHRQ customers regarding the usefulness of its research findings and dissemination products.

Agency for toxic substances and disease registry

Centers for disease control and prevention

Mission
To promote health and quality of life by preventing and controlling disease, injury, and disability.

Evaluation Program
The Centers for Disease Control and Prevention (CDC) conducts evaluation studies designed to provide essential information about its programs, goals, and priorities. These projects support the assessment of CDC’s strategies, which are to: Protect the health and safety of Americans; Provide credible information to enhance health decisions; and Promote health through strong partnerships.

CDC emphasizes evaluations that answer policy, program, and strategic planning questions related to the goals and objectives of Healthy People 2010. Performance improvement studies, such as those focusing on the development of key performance indicators consistent with the Government Performance and Results Act (GPRA) and the Office of Management and Budget’s Program Assessment Rating Tool (PART) are of particular interest and import to the Agency. In addition, CDC supports a variety of activities to enhance evaluation quality, use, and understanding.

Centers for medicare and medicaid services

Mission
To assure health care financing security for beneficiaries.

Evaluation Program
The research arm of the Centers for Medicare and Medicaid Services (CMS), the Office of Research, Development and Information (ORDI), performs and supports research and evaluations of demonstrations (through intramural studies, contracts and grants) to develop and implement new health care financing policies and provide information on the impact of CMS’ programs. ORDI’s activities embrace all areas of health care: costs, access, quality, service delivery models, and financing. ORDI’s responsibilities include evaluations of ongoing Medicare and Medicaid programs and demonstration projects testing new health care financing and delivery approaches.

Examples of research themes include state program flexibility, the future of Medicare, provider payment and delivery, and vulnerable populations and dual eligibles.

Food and drug administration

Mission
To protect the public health by assuring the safety, efficacy, and security of human and veterinary drugs, biological products, medical devices, our nation’s food supply, cosmetics, and products that emit radiation; to advance public health by helping to speed innovations that make medicines and foods more effective, safer, and affordable; and helping the public get the accurate, science-based information they need to use medicines and foods to improve their health.

Evaluation Program
The Food and Drug Administration (FDA) evaluates programs it is responsible for consistent with the goals established and promulgated by the DHHS strategic performance planning process. FDA uses its own strategic framework to accomplish these goals. This process also satisfies the implementation requirements of the Government Performance and Results Act (GPRA) and the Food and Drug Administration Modernization Act of 1997 (FDAMA). The strategic and performance process is an evolving set of program directions as changes occur in FDA’s dynamic environment. Meeting these challenges, now and in the future, will rest on its ability to leverage its efforts in that environment, which grows increasingly complex and more institutionally networked. The Agency strives to maintain the scientific knowledge base necessary to achieve greater effectiveness in assuring the quality and availability of the products it regulates.

Health Resources and services administration

Mission
To provide national leadership, program resources and services needed to improve access to culturally competent, quality health care.

Evaluation Program
The Health Resources and Services Administration (HRSA) supports a wide array of very different programs and activities that promote access to needed health care for all, including primary health care centers, the National Health Service Corps, HIV/AIDS programs, maternal and child health activities, health professions training, rural health programs, organ donation and transplantation initiatives, and telehealth activities.  To provide underpinning for these efforts, HRSA’s evaluation program is designed to enhance strategic planning, strengthen budget and legislative development, and improve program performance.

Indian health service

National institutes of health

Office of the assistant secretary for planning and evaluation

Office of public health and science

Mission
To provide advice to the Secretary on public health and science; to provide executive direction to program offices within the Office of Public Health and Science, and, at the Secretary’s direction, to coordinate crosscutting public health and science initiatives in the Department.

Evaluation Program
The Office of Public Health and Science (OPHS) provides advice, policy and program coordination, and leadership in the implementation, management, and development of activities related to public health and science, as directed by the Secretary. OPHS helps DHHS conduct broad-based public health assessments to better address and solve public health problems. It assists other components of DHHS in anticipating future public health issues and helps ensure that DHHS designs and implements appropriate approaches, interventions, and evaluations that will maintain, sustain, and improve the health of the Nation. OPHS provides leadership and policy recommendations on population-based public health and science and, at the Secretary’s direction, leads or coordinates initiatives that cut across agencies and operating divisions. In addition, OPHS communicates and interacts, on behalf of the Secretary, with professional and constituency organizations on matters of public health and science. Finally, OPHS’s unique role allows it to use its resources to link important DHHS programs or fill gaps in areas needing better policy formulation and coordination.

OPHS’ evaluation strategy focuses on public health and science issues that cut across multiple interests of the operating divisions and requires a coordinated approach to achieve the most effective results. OPHS evaluations support the Assistant Secretary for Health as the Secretary’s senior advisor for public health and science. OPHS also conducts evaluations specific to the needs of the programs operated from the offices located within OPHS, such as women’s health, minority health, disease prevention and health promotion, and research integrity. Some evaluation funds are made available to the ten DHHS Regional Health Administrators.

Substance abuse and mental health services administration

Mission
To build resilience and facilitate recovery for people with or at risk for substance abuse and mental illness.

Evaluation Program
The Substance Abuse and Mental Health Services Administration (SAMHSA) evaluates the effectiveness of prevention, treatment, and rehabilitation approaches and systems of care used by its overall programs and individual grant projects. SAMHSA conducts evaluations to ensure accountability for federal funds and to measure results toward its programmatic and policy objectives. In compliance with the Government Performance and Results Act (GPRA), SAMHSA is improving performance management and results by identifying annual, long-term and cost-efficiency performance measures to manage its programs.

SAMHSA has an integrated evaluation and planning process. Strategic planning identifies priorities that drive the development of grant programs and evaluations. The formulation of programmatic and evaluation priorities includes consultation with SAMHSA Center Advisory Councils, with other DHHS agencies, and with experts in the fields of evaluation and service delivery. Early and continuous coordination of program planning and evaluation design results in the articulation of program objectives that may be evaluated. Evaluations measure achievement of grant programs overall objectives, and these results are used for program and policy development. The strategic planning and policy development processes then use these results to refine SAMHSA’s priorities and performance objectives.

The specific type of evaluation required depends on the type and purpose of the particular grant  program. To the greatest extent possible, SAMHSA encourages the use of comparable data elements and instruments across its evaluations to implement a comprehensive evaluation system and to minimize respondent burden. Efforts to improve evaluation are continuing and SAMHSA is committed to using systematic approaches in using data to accomplish its mission.

Appendix C - List of Studies by agency

Appendix D - Acknowledgement of HHS officials

The following persons from HHS contributed to preparing information on evaluation projects for Performance Improvement 2004: Evaluation Activities of the U.S. Department of Health and Human Services:

Preparation of this report was managed by the Office of the Assistant Secretary for Planning and Evaluation, Office of Planning and Policy Support. Project Director: Andrew Rock. Technical and Systems Support was provided by Kimberley Berlin. Additional ASPE staff contributing to the development of these materials included: Barbara Greenberg, Alana Landey, Bill Marton, and Lynn Nonnemaker.

Susan Belsinger of Library Associates, Inc. provided database management and support in the development and production of the overall report; Philip Henery of Library Associates, Inc. provided database updating support.