Evaluation of the District of Columbia's Demonstration Program, "Managed Care System for Disabled and Special Needs Children": Year One Report

Robert Coulam, Carol Irvin, Michele Teitelbaum, Laurine Thomas and Tanisha Carino

Abt Associates Inc.

July 28, 1998

This report was prepared under contract #500-96-0003 between the U.S. Department of Health and Human Services (HHS), Office of Disability, Aging and Long-Term Care Policy (DALTCP) and Abt Associates Inc. Additional funding was provided by HHS's Health Care Financing Administration (now the Centers for Medicare and Medicaid Services). For additional information about this subject, you can visit the DALTCP home page at http://aspe.hhs.gov/_/office_specific/daltcp.cfm or contact the office at HHS/ASPE/DALTCP, Room 424E, H.H. Humphrey Building, 200 Independence Avenue, S.W., Washington, D.C. 20201. The e-mail address is: webmaster.DALTCP@hhs.gov. The Project Officer was Gavin Kennedy.

CHAPTER 1: INTRODUCTION AND EXECUTIVE SUMMARY

1.0 Introduction to the Demonstration

The District of Columbia has implemented a demonstration that relies on an innovative design of a Medicaid-financed managed care system for children in the District's Supplemental Security Income (SSI) program. The operational phase of this Demonstration began on December 15, 1995 under the authority of section 1115(a) of the Social Security Act. Under this section, the District of Columbia is allowed to deviate from many standard Medicaid requirements in order to test new system designs of policy merit. In return for this greater flexibility, the District must commit to a policy experiment that can be formally evaluated.

The District's demonstration waiver authority permits them for three years to restructure the system of health care offered to children residing in the District and enrolled in the SSI program. This Demonstration tests a Medicaid payment system that includes risk sharing based on 85% risk corridors (as defined by medical loss ratios) between the District's Medicaid office and the non-profit health plan operating the Demonstration, a capitated payment system for the health plan, and negotiated fee schedules for providers. This Demonstration is offered to eligible children as an alternative to the current Medicaid fee-for-service system found in the District of Columbia. The managed care organization operating the Demonstration, Health Services for Children with Special Needs (HSCSN), offers enhanced Medicaid benefits, including individual plans of treatment, care management services, respite care, van transportation and taxi vouchers, and limited-use phones. Participation in the Demonstration is voluntary and open to all SSI recipients living in the District and under 22 years of age. Exceptions are limited to children who are also eligible for Medicare benefits (the dually-eligible) and services related to transplants.

1.1 Scope of the Evaluation

1.1.1 Summary of Workplan

The Health Care Financing Administration (HCFA) and the Assistant Secretary for Planning and Evaluation (ASPE) in the Office of the Secretary have contracted with Abt Associates Inc. to evaluate the District's demonstration.¹ The research agenda for the evaluation is broad, but comprehensive, and is best characterized as an in-depth, descriptive analysis of the Demonstration. Results of this work will increase general understanding of what factors go into designing, implementing, and maintaining this type of health care system; which children are willing to enter a health care system such as the one designed by this Demonstration; and what kinds of experiences the organizations, providers, and clients can expect to have when a managed care system like the Demonstration is introduced.

The work is necessarily descriptive because the Demonstration is not based on an experimental design, and the evaluation is limited in its ability to construct a comparison group. Therefore, the measurement of demonstration impacts is limited. This limitation, coupled with changing Medicaid and welfare policies and local governmental issues, demands that the evaluation be approached with creativity and use a research agenda that is flexible and responsive to what has been, and will continue to be, a very fluid research environment.

The evaluation considers not only access and cost issues, but places a considerable emphasis on issues related to quality of care and satisfaction. The research design centers on three main activities:

• analyses of the implementation of the Demonstration from which other states and managed care systems might learn how to better serve similar populations;
• analyses of enrollment which will describe which children enroll in this type of system, as well as disenroll; and
• outcomes analyses that document the experiences of the District, plan, providers, and children and their families.

The study of the implementation of the Demonstration involves an in-depth case study based on interviews, a longitudinal series of site visits to the District's Medicaid office and HSCSN, and review of the documentary record. This component of the work seeks to provide:

• A selective, problem-directed review of the context of the Demonstration. As the Demonstration proceeds, unexpected aspects of the environment can be expected to take on importance and impact on the decisions made by the District and HSCSN.

• A precise description of "the Demonstration." It is obviously essential to establish precisely what the Demonstration interventions are and are not, as actually implemented. The Demonstration is also expected to change over time as HSCSN moves from a new business enterprise, to an experienced provider of services for children with special needs, and as it reacts to a changing medical and social service environment.

• Detailed description of demonstration processes and intermediate outcomes, to facilitate understanding of how the Demonstration could give rise to the observed patterns of results. Other components of the evaluation (e.g., analyses of enrollment and outcomes) are likely to identify unexpected or conflicting patterns of results. To interpret these results, an understanding of the underlying processes of the Demonstration is required in order to formulate plausible hypotheses about the sources of the observed patterns. Only then can key policy questions raised by the Demonstration be answered.

Chapter 2 and Chapter 3 describe the results of the first year of work on the case study. However, the results of this work also appear in the other chapters on care management and providers and enrollment. While the context in which this Demonstration operates is extremely complex, and some of this complexity arises from a unique political environment, the context in which this Demonstration operates does not necessarily mean that whatever is learned in this Demonstration is not generalizable. Many aspects of this program are resilient to the environment such as the implications of establishing a new business enterprise to implement a federally funded demonstration. Whenever possible, the evaluators try to draw distinctions between findings that are unique to this Demonstration and those that can be generalized.

Enrollment is the evaluation's second focus because the Demonstration relies on voluntary enrollment, and understanding enrollment patterns will be critical to understanding outcomes. The District's demonstration presents an important opportunity to analyze health plan choice, and the factors important to that choice, within a population with extremely heterogenous and costly health care needs. The enrollment analyses involve two components, an analysis of the Demonstration's enrollment process, as well as determining which eligible children enroll and how they differ from those who do not enroll. Chapter 5 discusses the enrollment experience of the Demonstration as it closes its second full year of operation.

The third focus of the evaluation is to examine outcomes. The Demonstration could have a considerable range of impacts from the client level to the District level. Outcome analyses are therefore, grouped in the following way:

• Organizational Outcomes. The potential organizational outcomes are highly variable. However, the specific question of interest is the adequacy of the risk sharing arrangements between the District, HSCSN, and providers. This assessment can not be definitive, but an attempt is made to draw a distinction between adequate payments to an inefficient organization and inadequate payments to an efficient organization. Efficiency in this context is not be an empirical certainty, but, assessment of various measures does provide some suggestive evidence. In addition, to financial outcomes, the evaluation is also looking at the extent to which this Demonstration is able to integrate with other publicly-funded agencies serving the targeted population (e.g., schools and foster care).

• Provider Outcomes. The impact of the District's demonstration program on providers can be assessed at two levels: satisfaction of participating providers and impacts on the wider provider community. Provider satisfaction focuses on administrative aspects of the program and the provider fee schedule. One important issue will be the referral processes--do providers in HSCSN's network face incentives to promote or hinder the use of needed referrals? Other sources of provider satisfaction are the negotiated fee schedule and HSCSN's approach to assigning enrollees to providers.

• Child and Family-Level Outcomes. Because the Demonstration represents a systematic change, and the health care needs of the targeted population are extremely heterogeneous, child and family level outcomes could be wide-ranging. Therefore, analyses at this level will focus on: access and use of health care services, satisfaction, quality of care, health status, and family caregiving burden. This work relies on data collected from claims, a survey of primary caregivers, a small medical record review, and focus groups of parents.

As noted above, the ability of the evaluation to measure demonstration impacts is limited. The Demonstration does not have a random assignment design and the only comparison group involves eligibles who decide not to participate. In addition, the evaluation did not begin until after the Demonstration became operational resulting in the loss of some baseline information. These and other challenges to the evaluation necessitate outcome analyses that include strong descriptive analysis, as well as modeling and hypothesis testing when appropriate.

1.1.2 Work Completed To-Date

The evaluation has concluded its first year of work. While much of this time has been spent planning and finalizing details of the research design, a considerable amount of qualitative research has been done. However, the work that is presented in the following chapters regarding outcomes is necessarily preliminary. The Demonstration is currently on-going and outcomes observed now may not be resilient over time, or may change as components of the Demonstration change. At the levels of the District, HSCSN, and providers, intermediate outcomes have already been observed as the Demonstration has moved from its start-up phase to being fully operational. Unfortunately, at the time of this report data were not available to analyze the selection effects experienced by the Demonstration. Other than what has been obtainable through focus groups, few outcomes at the level of the child and family have been observed. As the evaluation work continues, selection effects and outcomes will increasingly become the primary focus of the work.

In order to understand the context in which the Demonstration operates and what the Demonstration is, two in-depth site visits to the District and HSCSN have been completed and were followed by interim phone interviews to clarify and confirm information obtained during the visits. During the subsequent years, the evaluation team will conduct other site visits to obtain longitudinal information, follow-up on earlier issues, and to determine organizational outcomes.

In addition to these site visits, the evaluation has begun to develop an understanding of provider level issues. Interviews at HSCSN provided information about the Demonstration's approach to the development of its provider network and care management system. Care managers were interviewed to discuss HSCSN's unique care management system. Numerous interviews with providers have been completed to obtain information on the incentives faced by providers and how they participate in HSCSN's care management system. In succeeding years of the evaluation, this work will continue and will more fully focus on provider outcomes such as satisfaction and changes in practice patterns.

The evaluation has also conducted its first set of focus groups with parents. These groups were composed of five groups of parents with children in the Demonstration and one group who declined demonstration services. These groups discussed parent reaction to the Demonstration's marketing and outreach efforts and the elements that went into their enrollment decision. During these groups the evaluators heard some reaction to demonstration services. However, parent satisfaction and changes in family caregiving burden will be explored more fully in subsequent focus groups.

During the first year of the evaluation, quantitative analyses were limited. Secondary data sources are large and complex and the process of obtaining these data is time consuming. The primary data collection efforts, the medical record abstraction and the survey of primary caregivers, were only in the planning stages during the first year. It is anticipated that these data collection efforts will begin in the next year. Despite these limitations, the evaluation has been able to document demonstration enrollment patterns and conduct preliminary sketches of children entering the Demonstration.

1.2 Executive Summary

The following section highlights some of the interim findings that the evaluators found during the first year of work. During this first year the objectives of the work were to determine the context in which the Demonstration operates and what the Demonstration is, as opposed to what was planned or described in documents. The findings of this work are presented more fully in Chapter 2, Chapter 3, and Chapter 4. Knowing what the Demonstration is as implemented provides substantial background for Chapter 5 which presents the enrollment experience of the Demonstration after 22 months of operation.

1.2.1 What The Evaluation Knows So Far

Most demonstrations are developed and operate in circumstances that are unusual or atypical. What is striking in the District is how fundamental some of these circumstances were and continue to be. The planning and start-up phases of the Demonstration began in a context that had the following features:

• Severe costs problems in the Medicaid program. The DC Medicaid program was facing severe cost problems in the early 1990s. It was the fastest growing health program in a city facing fundamental fiscal crisis. By 1994, as the Hospital for Sick Children (HSC) was reviewing the possibilities for managed care for children with special needs in the District, the District Medicaid program was making early policy commitments to capitated solutions for a series of high-cost Medicaid services, thought to be at the heart of the cost problem: long-term care, substance abuse, and mental health/mental retardation. When the HSC proposal for a demonstration came, the Medicaid program (Commission on Health Care Financing, or CHCF) was in many ways prepared for it.

• The targeted population was thought to be poorly served. One Medicaid official we interviewed suggested that the District has "awful" statistics on preventive, chronic, and long-term care. The organization and financing of District human services was thought to be partly responsible. A proposal to integrate services for children with special needs thus offered help in an area that the District was believed to need help.

• Some of the principal agencies did not have close working relationship before the Demonstration. The key players in putting the Demonstration together--principally, the Commission on Health Care Finance of the District of Columbia (CHCF), the Hospital for Sick Children (HSC), and the Health Care Financing Administration's Office of Research and Demonstrations (HCFA/ORD)--had not worked closely together prior to this initiative.² This is not unusual for local demonstrations operating under waivers approved by HCFA. However, it increases the potential for misunderstandings and unexpected delays, which were an important factor here.

• Major changes in the surrounding context occurred. As waiver negotiations began in earnest, DC finances worsened. The federal price for resolving this continuing crisis was a substantial reduction in home rule powers. The increasing powers of the outside Control Board imposed by Congress threatened to change the politics and administration of the District government in large and small ways. Meanwhile, somewhat independent of that upheaval, the Medicaid program itself was about to change: a relatively small, optional managed care program for mainstream Medicaid enrollees was about to become mandatory for all mainstream enrollees in 1997. While children with special needs were carved out of the mandate, the commitment of the Medicaid program to managed care had spread substantially from the early days of planning for the Demonstration.

From this context came a unique waiver application for a demonstration; a new business enterprise would be established and it would operate a managed care system for children living in the District and participating in the federally-funded SSI program. The waiver application process was long and tense, one result of this process was an enrollment process that underwent several design changes. In addition, because the community at-large was not involved during the early design stages, community opposition made the application process more difficult. The evaluators can at least offer the following tentative implications for other states and entities considering this kind of model:

• The waiver approval process has undergone many modifications in recent years, including new provisions contained in the Balanced Budget Act of 1997. The difficulties of the waiver negotiation process for this Demonstration should not be generalized too quickly as a result. But at least a mild suggestion is obvious: no state or local entity that must receive HCFA approval for reform or demonstration initiatives should assume approval or a timeline for an approval, absent discussion with HCFA.

• That conservatism is especially important when the review process must resolve some fairly substantial issues, issues that in this case were evident almost as soon as the application was submitted. The original design proposed in this Demonstration contained unusual restrictions on beneficiary choice: mandatory enrollment in a single health plan that, although related to a substantial health provider for this population, had no experience doing what it proposed. These restrictions were for care of a notably vulnerable indigent population with substantial service needs. This proposal posed a particularly difficult form of the restriction, especially in view of the requirements in the Americans with Disabilities Act (ADA). That difficulty was reflected in the complexity of the discussions that followed.

• This Demonstration faced political and community opposition that could have been reduced if the District and HSCSN had followed fairly rudimentary forms of consultation. The District itself concedes that it should have done more, earlier, to consult the community--and parents, in particular. As it was, parents were uninformed, and that made them more vulnerable and naturally worried about rumors (some rumors apparently orchestrated by providers) concerning the new Demonstration. Many parents who were opponents later became proponents (two now sit on HSCSN's board), but only after extended efforts to explain the Demonstration. That the parents could be persuaded suggested how valuable it would have been to involve them up front. For obvious reasons, HCFA paid close attention to parents' complaints--and those could have been derailed by earlier contact with parents, in the opinion of District demonstration staff.

Managed care systems can be implemented in many different ways, particularly where, as here, the venture is without extensive precedents, long-established protocols, or settled standards of monitoring and review. To understand the Demonstration and its effects, it is obviously essential to establish precisely what the Demonstration interventions are and are not, as actually implemented. The Demonstration plans are clear enough, but many different versions of the Demonstration could follow from those plans. The intervention must be mapped with care to establish a foundation for the implementation analyses, as well as to document processes of the Demonstration to support inferences and interpretations of other data (e.g., survey results and utilization analyses).

The Demonstration intervention can be divided into the following three different areas:

1. Risk sharing and the concept of managed care for children with special needs. The risk sharing arrangements specify that the organization operating the Demonstration, HSCSN, receives from DC Medicaid a per member per month capitated payment. Individual providers are not paid a capitated rate, but from a fee schedule that HSCSN has separately negotiated with each provider. Any profits and or losses experienced by HSCSN are shared between HSCSN and DC Medicaid based on 85% risk corridors. The basic premise implied by this risk sharing arrangement is that the organization operating the Demonstration has the incentive to manage care actively, in a way that no public agency or fee-for-service provider would be able to do. While Medicaid and other systems imposed limits on utilization and coverage in various ways (e.g., prior authorization requirements), no one in those systems was empowered to take responsibility for the active management of individual cases--e.g., to assist in scheduling appointments and arranging transportation, to coordinate different treatments, to eliminate critical barriers to care (such as physical impediments in housing), and to refine and adapt overall programs of treatment. HSCSN uses care managers to play this active integrating, coordinating role. The key challenges to the care management system are to augment services in cost-effective ways, to integrate health and social support systems, to establish linkages across different types of demonstration and non-demonstration services, and to facilitate access where key barriers to care impeded cost-effective use of preventive and other services.

2. The Benefit Package. In this Demonstration, HSCSN is required to provide the full panoply of mandatory and optional Medicaid benefits available in the District, including inpatient hospital, nursing facility, intermediate care facility for the mentally retarded, outpatient, physician, family planning, dental, prescription drug, home health, vision care, transportation, mental health/substance abuse, hospice, and other services. But HSCSN enhances this package. They include 24-hour care management services which are perhaps the defining characteristic of this Demonstration. The implied premises of the Demonstration are that this is the most important benefit of all, since it is the process of obtaining care that has been the primary frustration of parents of children with special needs. With prior authorization HSCSN also provides: limited-use telephones, transportation to appointments, home modifications, feeding and nutritional programs, respite care, and expansion of home health care, medical supplies and equipment, dental services, and physical, occupational, and speech therapy consistent with habilitative and rehabilitative needs.

3. The Organization. This Demonstration involves the introduction of a new, non-profit business enterprise into the District's health care system. As a new entity, HSCSN has seen not only growth in staff, but a restructuring as it moved through the phases of design, start-up, and being fully operational. Had the Demonstration relied on a previously established managed care organization, management structure might have been more stable and the objectives of the entity operating the Demonstration might have been clearer. In the case of this Demonstration, the parent company invested a substantial amount of funds during the planning and start-up phases. It is not necessarily clear to what extent this funding was provided as a strong and benevolent commitment to deliver managed care benefits to children with special needs in the District or that the parent company (and now HSCSN) viewed the Demonstration as a strategic business decision to insure its survival in a new competitive environment. Both are likely organizational objectives. Regardless of the precise nature of organizational objectives, HSCSN has developed an information management system that is based on systems for the following five operational areas: payment, quality management, credentialing, internal networking (e.g., email), and care management. The first four were relatively easy and relied on offthe-shelf products. The fifth system, critical to HSCSN's role as an integrator, has been difficult and as of this report, was in its final planning and design phase. The absence of any adaptable system on the market suggests that the kind of care management envisioned is indeed different. That HSCSN sought to develop a more sophisticated technical infrastructure for care management suggests a longer-term view of the Demonstration and a commitment to putting in place a new care management technology for service well beyond this Demonstration.

Because this Demonstration does not make use of traditional management techniques such as capitated payments to providers, physician gatekeepers, or penalty co-payments for the inappropriate use of services such as emergency rooms, one focus of the evaluators has been to determine how care is managed in this Demonstration. We find that care is managed by a system that relies heavily on HSCSN's care management system and its network of providers.

• HSCSN's Care Management System. HSCSN care management employs an innovative staffing model that is designed to accommodate children with a variety of needs and levels of severity. The three care management teams are each lead by a senior, experienced health professional: one nurse, one clinical social worker, and one occupational therapist. Under each team leader, there are care managers who are experienced professionals, including nurses, social workers, and therapists--a mix of professions and experiences on each team. Under each team leader, there are also care manager associates, who have a B.A. and a variety of work experiences, including administrative jobs in health-related organizations. There are currently three team leaders, seven care managers and twelve care manager associates. All three types of personnel are referred to as "care managers" by HSCSN staff, providers, and members' families.

  Care managers at all levels--even those with very specialized training--emphasize the multiple non-medical needs of most members' families and the importance of addressing these so that the child can benefit from a stable home environment. They spend much of their time educating families about, and helping them use, appropriate sources of health care. Care managers believe that the children who can benefit most from the HSCSN care management program are not necessarily the most medically complex, but those who have multiple needs, including behavior problems.

  Assessments of new members are reviewed at team meetings, during which each new member is assigned for care management. The criteria for assignment are level of medical acuity, and type and complexity of service needs. The most complex are assigned to the team leaders: the most medically complex members to the nurse, the most serious mental health problems to the social worker, and children with the most serious functional problems or rehabilitative needs to the therapist. The team leaders, who handle the most complex needs, have the smallest caseloads, ranging from 35-40. A child in the mid- range in terms of acuity would be assigned to a care manager, and, again, the specialty of the care manager would be matched to the needs of the child: for example, a child who uses a feeding tube would be assigned to a nurse, and a child with behavior problems would be assigned to a social worker. Caseloads at this level range from 50-80. A child with less severe problems would be assigned to a care manager associate. Caseloads at this level range from 90-140.

  Care managers must develop a plan of treatment (PoT) based on an assessment, but, recognizing the limitations of the assessment process, they consider the PoT to be a living document from the start, to be revised when more information is learned by the care manager and when the PCP has examined the child. A completed PoT includes goals, prescribed services and planned care management activities for: preventive care; nutrition; functional skills; education/vocation; and support/education for member, family, and caregivers. PCPs may receive PoTs in advance of medical records. When evaluators interviewed PCPs, they were found to be less bothered by the weaknesses of the assessment and PoT development process than the care managers. PCPs generally said that the PoTs were a good start and provided more information than they often had about a child.

  There was notable agreement among care managers and providers that the speed of the HSCSN authorization process had an especially favorable impact on the timely provision of durable medical equipment (DME). One DME provider pointed out that braces and other equipment have to be customized to fit growing children. The authorization process for conventional Medicaid coverage could take so long that in the time elapsed the child could grow sufficiently to require a new fitting for the needed equipment. That the HSCSN authorization process for equipment under $2,500 could be completed in a couple of days was considered by many to be an important outcome of the program.

• Reactions to the Care Management System. The general trend found in our interviews was: the more frequent the contact between provider (or provider's employee) and care managers, the more favorably impressed the provider would be with the care management system and HSCSN services overall. We know little about the reactions of families to this system, but what we have heard is favorable.

• The Provider Network. In recruiting providers, a strong selling point was the care management program. Some physicians had been reluctant to serve Medicaid patients and many had been unwilling to increase their Medicaid caseloads. They were concerned that the time required for physicians' office personnel to contact families and follow up on appointments to increase compliance, and to address urgent non-medical needs that interfered with their medical treatment, could not possibly be covered by Medicaid reimbursement. HSCSN offered them a care manager to address these needs.

  In most cases, HSCSN offered better rates than conventional Medicaid, although some smaller specialty providers that we spoke to claim that this was not the case. Some complained that rates were still too low, even though they were higher than those of conventional Medicaid. HSCSN also had intended to reimburse PCPs for some interactions with care managers, but there appears to be a lot of confusion about this, and it is not clear if many PCPs are sufficiently well-informed to know how to bill for this payment. Evaluators have not found a description of this policy, or the billing procedure in the HSCSN Physician Manual. HSCSN also promised timely payment. Most providers we spoke to found that they delivered on this promise. Others commented that they had not had a serious problem with timeliness of conventional Medicaid payments, but that the regular Medicaid program often made mistakes in payments (resulting from inaccurate entering of codes) and these could take months to correct. No provider reported that problem with HSCSN.

• Provider Experiences. In general, PCPs in the network are more satisfied than physician specialists, for several reasons. First, providers who have more contact with the care managers, and with HSCSN in general, tend to be more satisfied. The care management process requires frequent contact between the care manager and the PCP, or at least with the PCP's office. In most cases, this seems sensible because the PCP has much more contact with, and a broader range of responsibilities for, the child. However, some children also see a specialist regularly, and the formal process does not require more interaction with the care manager for these specialists. In addition to having less communication with care managers, specialists claim to be less frequently contacted by HSCSN, and sometimes they claim to have negotiated less favorable rates and working arrangements with HSCSN than the PCPs.

  Another reason for less enthusiastic responses of specialists is that HSCSN follows the by-now customary managed care strategy of encouraging prevention and primary care while looking for opportunities to decrease--or at least control more closely--the use of expensive specialty services. A typical example is the psychiatrist who maintains that he is only used for prescriptions while less expensive mental health practitioners provide ongoing care, and some behavioral issues that he might want to address are handled by the PCP.

  The perceptions of specialists are clearly different from those of PCPs, as some specialists quickly conclude that PCPs have "more to gain" from the HSCSN approach. That some PCPs refer to the HSCSN program as "the only game in town" if one wants to serve this population, reinforces the perceptions of some specialists that PCPs are anxious to please HSCSN, and therefore some of their care decisions may be suspect. These perceptions are not conducive to team building, and there do not seem to be any system requirements or incentives for team interaction beyond care manager/PCP contact.

  When providers were interviewed, if they had experience with conventional Medicaid, they were asked to compare care delivered in the conventional Medicaid program with care delivered by HSCSN. Most providers believed that HSCSN delivered more primary care and did a better job of screening children. If providers interviewed had experience with other managed care organizations, they were asked to compare care delivered by those organizations with care delivered by HSCSN. In most cases, HSCSN was said to be comparable or better. In view of the hundreds of children served by these providers--and many of the providers have been with HSCSN since the beginning of the Demonstration-- there were surprisingly few instances in which requests for a service had been challenged, much less denied. One denial of service that was described--involving a request for cosmetic surgery to address self-esteem issues--had left both a PCP and a specialist with a negative impression of HSCSN.

  Problems surfaced at two major Washington D.C. institutions that are in the HSCSN provider network. At the provider level, HSCSN appears to have a good working relationship with a D.C. General clinic (which is not a public health clinic) that provides services to children with special needs. But at the management level there are issues. HSCSN attempted to include the public health clinics operated by D.C. General, but ended this effort when the unionized providers in them refused to allow their credentials to be examined.

  Children's National Medical Center is the other institution at which problems surfaced. Initially, we interviewed at the management level at Children's, where we found general satisfaction. We also interviewed employees of two Children's physician specialty practices, who had extensive contact with HSCSN care managers, and we heard positive comments from them. Finally, we talked to the physicians who facilitate the primary care services provided to HSCSN by Children's, and there we heard about a number of problems.

  Children's insists on assigning PCPs to the enrollees so that they can distribute the workload reasonably. Also, primary care physicians at Children's are said to have less nursing support and fewer administrative personnel than the specialists at Children's. They therefore have to communicate personally with the HSCSN care manager, and they believe that HSCSN should provide at least a part-time on-site care manager at Children's. This may or may not be a good solution for HSCSN, but the Children's PCPs feel strongly that the possibility has to be seriously entertained. They would also like all authorizations to be made by the PCPs, not by the HSCSN care managers. The physicians we spoke to did not address the issue of sharing risk, nor have they discussed this with Children's management, which is likely to be more sensitive to the issue of linking authorization decisions to the assumption of risk. The Children's PCPs also have noticed that some HSCSN care managers want to authorize visits to specialists (often outside of Children's) that members visited prior to enrolling in HSCSN. They believe that many of these conditions could be monitored successfully by PCPs at Children's. In effect, the Children's PCPs believe they have most necessary services under their own roof, and they want to function in a manner that would approximate a staff-model HMO; while HSCSN is asking them to be part of a larger network and to follow rules designed for a dispersed network of providers.

As outlined above, the Demonstration involves the creation of a new business entity and the introduction of a partial risk sharing system that relies heavily on an innovative care management approach. What has been the reaction of the targeted population? In February 1996, the first eight children began receiving services from HSCSN. As of November 1, 1997, monthly enrollment stood at 1,901 children, representing about 71% of the eligible children that have been located. During the 22 months of operation, the Demonstration has served a total of 2,089 different children and has experienced a total of 26,286 member months.

At 71% of located children and little disenrollment due to dis-satisfaction, demonstration enrollment appears to be very successful. However, achieving this success has not been easy. The most important problem faced by the District and HSCSN has been identifying and locating a large proportion of the population believed to be eligible for program services. In addition, the initial enrollment materials have not been particularly effective in eliciting enrollment, consequently during the planning phase, the District and HSCSN had not fully anticipated the amount of time and resources necessary to market the program. Now during a period of stable enrollment, HSCSN views the enrollment process as manageable, but unexpectedly problematic. Lastly, many families appear not to realize that enrollment in the Demonstration is voluntary. This lack of awareness may result from an enrollment process that is ostensibly operated by HSCSN and enrollment materials that inaccurately state that if the family does not respond to the enrollment mailing, then the child will be automatically enrolled in the Demonstration.

1.2.2 Future Areas of Investigation

Our work on the context and origins of the Demonstration is substantially done. One principal task remains:

Selective data collection from all demonstration partners, as new issues arise: As the Demonstration proceeds and our evaluation expands, unexpected aspects of the environment will take on additional importance, and the evaluation must be alerted to collect data intensively in these areas. Thus, for example, utilization patterns may exhibit pre/post variations that make an understanding of pre-demonstration patterns more important to document; or difficulties in collaborating with potential partners (e.g., the schools) may increase the importance of understanding the early decisions made by the CHCF, HSCSN, and the District schools. We will do data collection as needed on these issues.

More generally, however, the focus of our future work will be not on the origins and context of the Demonstration, but rather on its implementation. Key areas of data collection and analysis in this area will focus on seven general areas of interest.

1. The evolution of HSCSN as an organization, including staffing, priorities, and financial performance.

2. The effects of personnel changes at HSCSN and CHCF. These changes may influence the relationship between these two entities. To the extent that the collaboration between CHCF and HSCSN is due to the novelty of the initiative, we might see the development of more formal standards of monitoring as the protocols or operation become better understood. In any event, if any problems are revealed in the performance of the contractor, it will be important to reexamine the CHCF-HSCSN relationship to understand whether anything in that relationship contributed to the problems.

3. The developing understandings between HSCSN and related service systems, including the Commission on Mental Health and foster care. One of the most important related service systems is the school system. The school system may have reasons for avoiding collaboration with the Demonstration that are critical to the Demonstration. Such information could change our understanding of how CHCF and HSCSN went about the work of coordination with other service systems.

4. The financial performance of HSCSN, including documentation of risk-sharing calculations for all years, audited results for HSCSN for years after 1996, and areas of major cost savings against baseline data. If HSCSN does not continue to break even on a cash basis, our tentative confirmation of the practicality of the principal assumptions of the Demonstration will have to be reconsidered.

5. The implementation of any new care management system, with a particular emphasis on documenting how the system changes care management staffing, procedures, and accomplishments. The basic care management system appears to be stable and well- designed for this Demonstration. It could ultimately have an impact on care, but it is not as apparent that it will have an impact on reducing costs. The current belief at HSCSN appears to be that an emphasis on primary care, coordination of services, and supportive services, with close review of the most expensive services, will be cost effective in the long term. If that approach works, how will it work? Among the possible explanations are:

   • Children who formerly received no regular primary care will make fewer visits to the emergency room and have fewer acute episodes.
   • Coordination of care will avoid duplication of services and the provision of unnecessary services.
   • Improved knowledge of a member's needs, gained from monitoring, will lead to the provision of more appropriate services.
   • Provision of supportive services, and resolution of non-medical issues, will enable the member to get the maximum benefit from medical care.

   Any change in philosophy at HSCSN should be evident in the care management program and in the ways that care managers interact with providers. For example, a more explicit focus on cost containment might immediately impact on the way that care managers spend their time and on the services they authorize.

6. The provider network and provider relationships with HSCSN management and care management system. Relative to the care management system, provider arrangements are not as stable as one might expect at the moment. Issues of dis-satisfaction among the PCPs at Children's could have serious consequences if they are not resolved, given the importance of that institution in the Demonstration. Outside of Children's, specialty provider issues noted are not unusual in a managed care program. They are probably not threatening to the program, but could be addressed. In general, communication with providers, and how providers communicate and interact with care managers can be worked on. Arguably the change that could make the most difference would be if HSCSN initiated risk-sharing arrangements with providers. This would address concerns about incentivizing providers, but decision making strategies might have to be reexamined. Whether or not the PCPs have "really" been making the medical decisions all along, would become an active issue, as would any inclination on their part to make different decisions when their incentives are changed.

7. The estimation of the frequency and context of utilization of certain enhanced benefits in this Demonstration (e.g., telephones, residential modifications, and home care) to explore how much, if any, change in utilization is due to the expanded continuum of benefits being provided. If our utilization analyses failed to confirm the changes in utilization being used to explain demonstration savings, we would need to reexamine how HSCSN was able to save money.

Beyond these research issues on the implementation of the Demonstration, the evaluation must now begin to focus more closely on outcomes, and to understand these outcomes we must begin to understand the extent and direction of any selection effects experienced by the Demonstration. Understanding which children have entered the Demonstration and which remain in the fee-for-service system will influence how HSCSN and the District adjust features of the Demonstration, and how providers and children and their families experience and benefit from demonstration services.

NOTES

1. HCFA Contract Number 500-96-0003, T.O. #3.

2. HSC was a Medicaid provider and thus had routine exchanges with the Medicaid intermediary, CHCF, and HCFA prior to the Demonstration. CHCF was the administrator of the Medicaid program for the District and thus had frequent contact with HCFA and providers in that role. Those activities provided some foundation for the Demonstration activities, but District and HSC/HSCSN respondents generally seemed unfamiliar with HCFA staff or waiver negotiations.