U.S. Department of Health and Human Services

A Pilot Study of the Adequacy of Post-Hospital Community Care for the Elderly: Final Report

Executive Summary

Barbara R. Phillips

Mathematica Policy Research, Inc.

September 13, 1989

This report was prepared under contract #500-86-0017 with Mathematica Policy Research, Inc. Funding was provided by the U.S. Department of Health and Human Services (HHS), Office of Social Services Policy (now the Office of Disability, Aging and Long-Term Care Policy) and the Health Care Financing Administration. For additional information about the study, you may visit the DALTCP home page at http://aspe.hhs.gov/daltcp/home.shtml or contact the office at HHS/ASPE/DALTCP, Room 424E, H.H. Humphrey Building, 200 Independence Avenue, SW, Washington, DC 20201. The e-mail address is: webmaster.DALTCP@hhs.gov. The DALTCP Project Officer was Robert Clark.

Most of the work of Mathematica Policy Research on this project was completed under a subcontract to System Sciences, Inc., 4330 East West Highway, Suite 1117, Bethesda, Maryland 20814. Cyrus Baghelai was project director for System Sciences, Inc. for most of the period of the subcontract.

One prominent response of hospitals to the necessity of controlling costs under the Prospective Payment System (PPS) has been to reduce lengths of stay. Since the introduction of PPS, patients are now more ill on average when they are discharged from hospitals, and thus probably require more health and personal care services upon discharge.

Since the introduction of PPS, extensive concern has been expressed--by both the public and the Secretary of Health and Human Services and other senior officials of the Department--about the adequacy of post-hospital care for elderly patients discharged to the community. Moreover, Congress has mandated that information on the quality of post-hospital care by included in the Department's annual reports to Congress on PPS.

The current evidence on the adequacy of post-hospital community care is largely impressionistic. A systematic assessment is required to develop objective evidence on the extent of the problems associated with post-hospital care and whether they lead to adverse health outcomes for patients.

Unfortunately, the methodology available up to now has not been adequate to support a systematic assessment of national scope. The primary limitation of the available methodology is that adequacy-of-care assessments have required a review of individual cases by physicians, which is difficult to implement in a national study and which has been found to be unreliable. This report describes a new methodology developed to overcome these limitations and to guide a systematic, national assessment of both the adequacy of community post-hospital care for elderly Medicare patients and whether inadequate care leads to adverse health outcomes. Central to this methodology is a series of "guidelines"which specify the amount of care that is minimally adequate to prevent adverse health outcomes for patients who exhibit a wide variety of conditions that commonly require post-hospital community care.

This methodology has been implemented in a pilot study. Based on an analysis of pilot study data, this report assesses the validity, effectiveness, and feasibility of the new methodology. The conclusion of this assessment is that the methodology is generally valid, feasible, and effective, although some refinements are required. With respect to the guidelines, conclusion is that (taken as a group) the guidelines provide a reasonable definition of minimally adequate post-hospital community care.

The pilot study results must be considered preliminary. The pilot study was limited only to nine hospitals in two States. In addition, the pilot study encountered a substantial amount of missing data, and some of the estimates are sensitive to the assumptions made about missing data.

Figures 1 and 2 depict the pilot study results on access to adequate post-hospital community care. The results indicate that most 74-80 percent with a midpoint of 77 percent) of the post-hospital community care needs under the guidelines were met, although a substantial minority, or even a majority, of patients (43-55 percent with a midpoint of 49 percent) experienced care that did not meet one or more of the guidelines applicable to them.1 The great majority (87 percent) of care needs that were not met involved skilled care, and about half of the patients who experienced care that did not meet the guidelines were not seen in the two weeks after hospital discharge by the type of health care professional appropriate to their care needs.

Figure 3 depicts the pilot study results on adverse outcomes. A minority of Medicare patients (13-23 percent with a midpoint of 18 percent) suffered adverse outcomes that may be associated with inadequate post-hospital community care. Most (80 percent) of these outcomes involved morbidities rather than an unexpected use of health care services. While not trivial, outcomes that do not involve unexpected service use are generally less serious than those that do involve such use.

A. THE METHODOLOGY

As indicated above, the guidelines are central to the new methodology for measuring the adequacy of post-hospital community care for the elderly. A number of clinicians with extensive experience in post-hospital community care helped develop the guidelines. Draft guidelines developed in conjunction with the staff of the Geriatrics Section of Boston University Medical Center were reviewed by a consensus panel of distinguished clinicians. The guidelines were revised on the basis of the panel's comments and pretested for a sample of patients recently discharged from the hospital. They were again revised on the basis of the results of the pretest, and were reviewed by the consensus panel prior to their application in the pilot study.

Guidelines were developed for 45 distinct conditions (many of which encompass subconditions). Information on medical condition, procedures performed in the hospital, functioning, the availability of informal caregivers (typically family members), and instruction provided in the hospital was used to determine which of the conditions covered by the guidelines were applicable to a given patient. (Multiple guidelines may apply to the same patient.) The guidelines cover both skilled care (primarily nursing and therapy) and semi/unskilled care (primarily personal care) that are needed in the immediate post-discharge period (defined as the two weeks following discharge) by elderly Medicare patients discharged to the community. Care provided in patients' homes and in physicians' offices and clinics is included. The guidelines for skilled care specify both the minimum number of professional visits necessary in the two weeks after discharge to prevent adverse outcomes and the latest acceptable day (relative to discharge) of the initial professional visit. The guidelines for semi/unskilled care typically specify the frequency with which care must be provided (e.g., the number of times a day) to prevent adverse outcomes. The guidelines for each condition includes a list of adverse health outcomes that are clinically associated with inadequate care for that condition. (The guidelines appear as Appendix A to this report.)

In addition to the guidelines, the methodology also includes screening procedures to identify patients with one or more of the conditions covered by the guidelines, as well as risk classification procedures to identify patients at high risk of experiencing care that does not meet the guidelines and of suffering adverse outcomes. The screening procedures rely chiefly on information on medical condition, procedures performed in the hospital, and functioning. The risk classification procedures rely on information on the patient's living arrangements, the availability of formal and informal care, the exhaustion of informal caregivers, the receipt of discharge planning, the severity of impairment, the severity of illness, and age. High-risk patients are oversampled relative to their proportion in the population, so as to obtain a sufficient sample of patients who actually experience care that does not meet the guidelines and who suffer adverse outcomes to permit investigating their relationship.

An extensive data collection effort was required to obtain the data necessary for applying the screening and risk classification procedures and the guidelines. The data were derived from two primary sources: (1) hospital medical records; and (2) interviews with patients and their caregivers conducted two and six weeks after hospital discharge. Information for screening and risk classification was obtained from the summary sheet of medical records and from screening interviews conducted two weeks after discharge. The detailed information on the patient's condition necessary for determining which guidelines applied to him or her was abstracted from the full hospital medical record. Information on service receipt in the two weeks after discharge was collected in the full interview at two weeks, and information on adverse outcomes was collected in full interviews at two and six weeks. Services received were compared with the guideline specifications to determine whether the patient experienced care that did not meet guidelines.

B. TESTING AND REFINING THIS METHODOLOGY

The purpose of pilot study was to test the validity of the guidelines, the effectiveness of the screening and risk classification procedures, and the feasibility of the data collection strategy. The pilot study was also designed to identify any necessary refinements to the methodology to be implemented before a national study is to be undertaken.

1. The Guidelines

Three analyses were undertaken to test the validity of the guidelines and to identify refinements to them:

• a comparison of care ordered at the time of hospital discharge with care specified under the guidelines

• a clinical review of 100 pilot study cases

• a statistical analysis of the likelihood of adverse outcomes when care met and did not meet the guidelines.

In the remainder of this section, we summarize the results of each of these analyses.

The hypothesis underlying the analysis of the likelihood of adverse outcomes is that adverse outcomes will be substantially more likely when care does not meet the guidelines than when if does, if the guidelines (taken as a group) provide valid specifications of minimally adequate care. This is what we observed. When we corrected a measurement problem involving two outcomes, we estimated large differences in the likelihood of adverse outcomes when care meets and does not meet the guidelines. We estimated that adverse outcomes are four and one-half time as likely when care does not meet the semi/unskilled guidelines and twice as likely when care does not meet the skilled care guidelines (relative to the likelihood when care meets the guidelines). For all the guidelines combined, we estimated that adverse outcomes are about three times as likely when care meets the guidelines than when it does not. Figure 4 depicts these results. The estimated differences in the likelihood of adverse outcomes are statistically significant for all guidelines and the skilled care guidelines, and approach significance for all guidelines and the skilled care guidelines, and approach significance for the semi/unskilled guidelines. Further, when we relaxed or tightened the specifications of the guidelines, we obtained less reasonable results: we estimated that the effect of inadequate care was to reduce adverse outcomes. This finding is encouraging, in that it appears to suggest that the guideline standards are neither too relaxed nor too tight. However, these results also indicate that changes in the standards for one guideline (the guideline on medication supervision) can have a dramatic effect on estimates of the likelihood of adverse outcomes.

Two hypotheses underlie the comparison of care ordered at the time of hospital discharge with the care specified under the guidelines. First, if the guidelines cover all types of care commonly provided to elderly patients in the community, we will observe relatively few patients with orders for care to whom no guidelines are applicable. Second, if the guidelines specify minimally adequate care, the amount of care specified under the guidelines will not exceed the amount ordered. Both hypotheses were confirmed. At least one skilled guideline was applicable to about 89 percent of the patients with orders for nursing or therapy, and to about 80 percent of the patients with orders for follow-up physician care in the two weeks following discharge. The amount of care specified under the guidelines never exceeded the amount ordered. A review of the cases in which care was ordered but no guidelines was applicable indicates that several existing guidelines must be defined, but major revisions are unnecessary.

Because the primary purpose of the clinical review was to identify cases that needed refining, the sample of cases chosen for review was selected deliberately to include cases in which the guidelines were most likely to be problematic (e.g., cases in which the patient experienced care that met the guidelines but suffered an adverse outcome). Despite the nature of the sample, the clinicians who conducted the review concluded that, in general, the guideline standards were clinically sound. However, they did suggest refinements to the guidelines and to the procedures for applying them.

While the analysis of the guidelines supports their validity, it also suggests a number of refinements to them. Many of the suggested refinements are relatively trivial (e.g., the inclusion of additional adverse outcomes); others call for subdividing the existing guidelines so that different care may be specified for the subdivisions. For example, one suggested refinement entails subdividing the guideline on diabetic care for patients with and without informal caregivers, with more care specified for the patients with informal caregivers. We recommend that the refinements be reviewed by a clinical consensus panel.

2. The Effectiveness of Screening and Risk Classification Procedures

The analysis of the effectiveness of the screening procedures compared the need for care according to these procedures with the need for care according the guidelines to determine the extent to which patients to whom at least one guideline applied were incorrectly screened out and, conversely, the extent to which patients to whom no guidelines applied were incorrectly screened in. The guidelines cover a broad scope of care, and at least one guideline was applicable to the vast majority (94 percent) of elderly Medicare patients discharged from the hospital to the community. The screening procedures correctly identified almost all of the patients (97 percent) to whom at least one guideline was applicable. However, the screening procedures were not effective at identifying patients who did not need care under the guidelines. At least one guideline was applicable to about 60 percent of a small sample of cases that had been screened out as not needing care under the guidelines. While these cases represent only a small portion (about 6 percent) of the patients who needed care, refinements to the screening procedures would be desirable. A review of the cases that were incorrectly screened out suggests that the effectiveness of the screening procedures could be improved substantially by adding items on laboratory testing, follow-up physician care, and specific personal care activities. With these additional items, we estimate that the guidelines would be applicable to fewer than a quarter of the patients screened out as not needing care, and that the patients who were incorrectly screened out as not needing care, and that the patients who were incorrectly screened out would comprise no more than 2 percent of the Medicare patients who need post-hospital community care.

The analysis of the risk classification procedures compared the incidence of care that did not meet the guidelines and adverse outcomes for patients at high risk and patients not at high risk to determine whether patients at high risk exhibited a higher incidence of care that did not meet the guidelines and of adverse outcomes relative to those not at high risk. We observed such a difference. About 29 percent of the patients in the high-risk group experienced care that did not meet the guidelines and suffered adverse outcomes, compared with about 12 percent of the patients not in the high-risk group. When we considered care and outcomes separately, we found a large and statistically significant difference in the incidence of adverse outcomes but not in the incidence of care that did not meet the guidelines. However, the observed difference in the incidence of care that did not meet the guidelines for patients at high risk and not at high risk was attenuated in the pilot study sample due to an artifact of the sample associated with a revision to the risk classification procedures that was implemented after data collection had begun. The evidence suggests that we would have found a larger (and statistically significant) difference in the incidence of inadequate care in the two risk groups had the revised risk classification procedures been in place at the beginning of data collection.

3. The Feasibility of the Data Collection Strategy

The experience of the pilot study indicates that the data collection strategy is generally feasible and deals successfully with a number of operational issues. The cooperation of hospitals was satisfactory; over 80 percent of the hospitals that were approached agreed to participate. However, for-profit hospitals were the most reluctant to participate, suggesting that non-participating hospitals may differ systematically, and that it may be desirable to adjust the results of a national study for the nonresponse of hospitals. Patients and their caregivers were very cooperative; the completion rates for the interviews ranged from 88-99 percent. Selecting the sample of discharged patients, obtaining the information on condition and procedure codes for screening purposes, and processing this information in a timely way did not present intractable problems. However, by using hospital discharge disposition codes to identify patients discharged to the community, we failed to include a small minority of eligible patients who were coded as discharged to an institution, but who were actually discharged to the community.

The major problem with the data collection strategy is the extent of missing data. Missing data precluded determining whether or not a patient had a given condition in about 14 percent of all the potential observations on condition, and missing data precluded determining whether care met the guidelines or adverse outcomes were suffered in about 23 percent of the cases in which the condition could be determined. The actual number of observations lost to analysis due to missing data on condition, care, or outcomes is unknown, but lies between 23 percent and 37 percent. The major cause of the missing data was inconsistency between the hospital medical records and the interview reports. Because medical records were not available for abstraction until quite some time after a patient was discharged and because abstraction is a very time-consuming process, the patient's report of his or her condition was used to determine which interview questions on care and outcomes were applicable. If the information in the hospital medical record indicated that the patient had a given condition, but the patient did not report having that condition when interviewed, the guideline for that condition could have been applicable based on the medical records information, but data on care and adverse outcomes would not have been available from the interview. Another major cause of missing data was a failure to find information in the medical record, either because it did not exist or because it was overlooked.

While one cannot hope to eliminate missing data entirely, it is very important that the data collection strategy be revised in a national study to reduce the amount of missing data. We recommend revisions to the data collection strategy to reduce missing data that involve changing the interviews to ask as many questions as possible regardless of the condition reported by the patient, and expanding the "callback" procedures so that patients are recontacted to resolve inconsistencies between the hospital records data and the interview data and to provide missing data.

The other major problem with the data collection strategy involves the procedures for abstracting medical records and applying the guidelines automatically. The clinical reviewers felt that the guidelines had been applied incorrectly in a substantial number of the 100 cases that they reviewed. Further investigation indicates that the major factors accounting for the incorrect application of the guidelines were different decision rules that were applied under the automated procedures and in the clinical review, different decisions that were applied in ambiguous cases, and difficult-to-locate information that was overlooked during abstraction.

Most of the cases in which decision rules differed involved cases in which information on functioning in the medical record and the interview was inconsistent. To eliminate these inconsistencies, we recommend that the interview be used as the primary data source on functioning. (Presently, the medical record is the primary source.) We also recommend refining the medical records abstraction and the automated procedures for applying the guidelines to deal with ambiguous cases. Finally, we recommend increasing the time devoted to medical records abstraction.

NOTES

1. In the figures, the midpoints of these percentage ranges are used.