U.S. Department of Health and Human Services

Costs and Benefits of Health Information Technology

Executive Summary

Paul G. Shekelle, M.D., Ph.D., Sally C. Morton, Ph.D., and Emmett B. Keeler, Ph.D.

Southern California Evidence-based Practice Center

April 2006

This report was prepared under contract #290-02-0003 between U.S. Department of Health and Human Services (HHS), Office of Disability, Aging and Long-Term Care Policy (DALTCP) and the RAND Corporation. Additional funding was provided by HHS's Agency for Healthcare Research and Quality. For additional information about this subject, you can visit the DALTCP home page at http://aspe.hhs.gov/_/office_specific/daltcp.cfm or contact the ASPE Project Officer, Jennie Harvell, at HHS/ASPE/DALTCP, Room 424E, H.H. Humphrey Building, 200 Independence Avenue, S.W., Washington, D.C. 20201. Her e-mail address is: Jennie.Harvell@hhs.gov.

The opinions and views expressed in this report are those of the authors. They do not necessarily reflect the views of the Department of Health and Human Services, the contractor or any other funding organization.

INTRODUCTION

The United States health care system is at risk due to increasing demand, spiraling costs, inconsistent and poor quality of care, and inefficient, poorly coordinated care systems. Some evidence suggests that health information technology (HIT) can improve the efficiency, cost-effectiveness, quality, and safety of medical care delivery by making best practice guidelines and evidence databases immediately available to clinicians, and by making computerized patient records available throughout a health care network. However, much of the evidence is based on a small number of systems developed at academic medical centers, and little is known about the organizational changes, costs, and time required for community practices to successfully implement off-the-shelf systems.

An analysis of the usefulness of implementing HIT must take into consideration several factors:

• The potential of this technology to improve health care quality, safety, and patient satisfaction--and how this potential has been demonstrated.
• The cost-effectiveness of the technology--the business case for adoption of the technology--including the total costs of implementation (both financial and in terms of resources) and any cost savings that accrue. Concerns exist that those who bear the greatest share of such costs are not able to recoup those costs.
• The ability to generalize the effects of an HIT intervention on costs and benefits in existing systems (using published experience with or research on these systems) to the technology's use by other health care organizations.

The Leap Frog Group and a number of components of the U.S. Department of Health and Human Services (HHS)--the Centers for Medicare & Medicaid Services (CMS), the Office of the Assistant Secretary for Planning and Evaluation (ASPE), the Office of Disease Prevention and Health Promotion (ODPHP), and the Agency for Healthcare Research and Quality (AHRQ)--requested a review of the research on HIT to compile and evaluate the evidence regarding the value of discrete HIT functions and systems in various health care settings. This Evidence-based Practice Report on the costs and benefits of health information technology systems, along with an accompanying interactive database that catalogs and assesses the existing evidence was prepared by the Southern California Evidence-based Practice Center (EPC). This report systematically reviews the literature on the implementation of HIT systems in all care settings and assesses the evidence in four specific circumstances:

1. The costs and benefits of HIT for pediatric care.
2. The ability of one aspect of HIT, the electronic health record (EHR), to improve the quality of care in ambulatory care settings.
3. The costs and cost-effectiveness of implementing HER.
4. The effect of HIT on making care more patient-centered.

METHODS

An electronic search of PubMed, the Cochrane Controlled Clinical Trials Register, and the Cochrane Database of Reviews of Effectiveness (DARE) was conducted for articles published from 1995 to January 2004. Additional references were obtained by reviewing the references in several major reports prepared by private industry and by RAND Health. Two reviewers, each trained in the critical analysis of scientific literature, independently reviewed each study and resolved disagreements by consensus. The principal investigator resolved any disagreements that remained unresolved after discussions between the reviewers.

Studies selected for review had to be either:

• A meta-analysis.
• A systematic review.
• Original research that tested a hypothesis (that is, a report that compared data between groups or across time periods, assessing a specific question and using statistical tests to assess differences).
• Original research that conducted predictive analyses (a report that used modeling techniques and simulations to predict the effects of an HIT implementation).

Of 855 articles screened, 256 were accepted for review. Descriptive studies of HIT implementations were identified and classified according to the categories listed below, but were not reviewed in more detail.

The contents of each selected article or report were abstracted using electronic data-abstraction forms prepared especially for this analysis. Abstracted data included the system's capabilities, interventions used, study design, implementation processes, evaluation methods, outcomes, costs, and barriers to implementation. A structured abstract was created for each report; these abstracts can be accessed in an online, interactive database created for this evidence report. (This database can be accessed at http://healthit.ahrq.gov/tools/rand.)

RESULTS

Overall Results

Of the 256 studies reviewed, 156 were about decision support, 84 assessed the electronic medical record, and 30 were about computerized physician order entry (CPOE). One hundred twenty-four of the studies assessed the effect of the HIT system in the outpatient or ambulatory setting, while 82 assessed its use in the hospital or inpatient setting. Ninety-seven studies used a randomized design. There were 11 controlled clinical trials, 33 studies that used a pre/post design, 20 studies that did a time series, and another 17 that were case studies with a concurrent control. Among the 211 hypothesis-testing studies, 81 contained at least some cost data.

Many of the studies concerned HIT systems developed and evaluated by academic and institutional leaders in HIT.

• Regenstrief Institute in Indianapolis, IN (18 studies)
• Partners/Brigham and Women's Hospital in Boston, MA (19 studies)
• Intermountain Health in Salt Lake City, UT (11 studies)
• Kaiser Permanente health care system (5 studies)
• Vanderbilt University in Nashville, TN (2 studies)
• U.S. Department of Veterans Affairs (VA) health care system (15 studies)

Studies from these institutions have contributed greatly to our knowledge about the usefulness of particular HIT functionalities (such as CPOE or computerized electronic alerts), and are examples of what can be realized by the implementation of broadly functional HIT at these specific institutions. But these studies also have limitations, in terms of their usefulness to inform decisions about the adoption of HIT elsewhere. The primary limitation is that these HIT systems were developed over the course of many years by technology champions at these institutions and, in a process of co-evolution, were adapted particularly to the working environment and culture of their respective institutions. Consequently, the "intervention" at these sites consists not only of the HIT system but also the local champions, who were often also the evaluators in published studies. Furthermore, it is challenging to calculate the cost of the development of the HIT system as a whole, since this process occurred over many years at each institution. In addition, these systems are not commercially available from a vendor--and vendors supply most HIT systems in use in the U.S.

We were able to identify only 15 studies that used a randomized or controlled clinical (RCT or CCT) design, included cost data, and assessed HIT systems that were not from one of the leading academic and institutional HIT institutions or the United Kingdom (another setting that has limited generalizability to U.S. health care institutions). When these 15 studies were examined for their HIT functionality using the classification system developed by the Institute of Medicine, 4 of them concerned only decision support and 4 assessed HIT systems with decision support and administrative processes. The remaining seven studies addressed other single functionalities or combinations of up to three functionalities. We were not able to find a single study that used a randomized or controlled clinical trial design, that did not report data from one of the leading academic or institutional HIT systems or the U.K., that reported cost outcomes and that assessed an HIT system including at least four of the eight IOM categories of functionality.

For the 103 hypothesis-testing studies that used a design other than a randomized or controlled clinical trial, 45 reported cost data. Of these 45 studies, 23 assessed systems that were not one of the leading academic or institutional HIT systems or that came from the U.K. An examination of these 23 studies for their functionalities showed, as in the studies using an RCT or CCT design, that most studies did not evaluate systems with a broad level of functionality. Five studies assessed only decision support, and three studies each assessed only administrative processes or order entry management. Three studies assessed HIT systems with two functionalities, order entry management and decision support. The remaining nine studies assessed various combinations of two or three functionalities. No study evaluated an HIT system with at least four of the eight categories of functionality.

The literature is even sparser regarding information about the organizational context of an HIT implementation. Of the hypothesis-testing studies, we identified only 3 studies that provided information about the financial context of the organization, such as the degree of managed care/capitation penetration; 6 studies with information about system penetration; 2 studies about facilitators to implementation; 1 study explicitly discussing sustainability of the HIT intervention; 12 studies reporting extrinsic factors in valuing costs and benefits such as the health care market competitiveness; and 6 and 9 studies, respectively, reporting the initial costs of the HIT system and costs of implementation.

In summary, we identified no study or collection of studies--outside of those from a handful of HIT leadership institutions--that would allow a reader to make a determination whether the study's reported benefit was generalizable. Besides these studies from HIT leaders, no other research assessed HIT systems that had comprehensive functionality while including data on costs, relevant information on organizational context and process change, and data on implementation. This limitation in generalizable knowledge is not only a matter of study design and internal validity. Even if further randomized, controlled trials are performed, the generalizability of the evidence would remain low unless additional systematic, comprehensive, and relevant descriptions and measurements are made regarding how the technology is utilized, the individuals using it, and the environment it is used in.

The Costs and Benefits of HIT in Pediatric Settings

Early evidence shows that stand-alone clinical decision-support systems (CDSS) (such as drug dosing calculators) can reduce medication dosing errors, and CPOE plus CDSS can reduce the incidence of harmful medication errors in the inpatient pediatric and neonatal intensive care settings. Other HIT systems, such as electronic medication administration records, pharmacy-based robots, smart infusion pumps/devices, and medication bar-coding, are predicted to reduce medication errors, but need further study.

The use of CPOE plus CDSS has been demonstrated, in separate studies, to (1) reduce the frequency or duration of antibiotic use for common pediatric illnesses such as pharyngitis and otitis media, and (2) improve completeness and reduce variation in clinical documentation. In the ambulatory setting, a single study showed that an appointment reminder system is cost-effective and significantly reduces missed appointments, while in the neonatal intensive care unit, another study showed that CPOE can reduce medication and radiology turnaround times. Therefore, the evidence for HIT cost-savings in pediatrics is limited, but appears optimistic.

Electronic Health Records and the Quality of Ambulatory Care

Adoption of EHR systems is widely believed to be critical to the delivery of consistent, high-quality health care, although the current use of EHRs is limited. Seven studies were identified on the use of EHR in four ambulatory care settings (three in the United States and one in the Netherlands). The findings reported in all of these studies were primarily related to the implementation processes and to changes in clinical processes.

With the exception of one study that examined the effects of incorporating HIV care guidelines and alerts on quality of care for HIV-positive patients, all the studies assessed the effects of adding various types of information related to laboratory test and prescription ordering to EHR ordering screens. In general, these studies showed that providing laboratory test guidelines and related information on test-ordering screens was associated with a decrease in orders for overused tests and an increase in orders for underused tests; provision of formulary guidance was associated with increased adherence to a formulary for at least one class of medication; and addition of HIV care guidelines and alerts was associated with improved quality of care.

The Economic Value of an EHR System

While EHR systems may be essential for improving efficiency and quality of health care, implementation of an EHR system requires substantial capital investments and organizational change. Consequently, many health care organizations are seeking evidence from previously implemented systems about the costs and benefits of EHR adoption in order to better inform decisions about the optimal timing and strategy for implementation.

Not all of the costs and benefits reported when implementing new systems or making changes to existing systems were financial. EHRs were associated with improvements in service and other resource utilization, provider productivity, care efficiency, documentation quality, clinical decisionmaking, guideline compliance, and costs of care.

Despite considerable variation among the few studies that modeled financial costs and benefits, all predicted substantial cost savings from EHR implementation. However, these studies each made a number of assumptions, and the predicted break-even points ranged from as short a time as 3 years to as long as 13 years.

HIT and Patient-Centered Care

The evidence is sparse for the ability of HIT systems to make health care more patient-centered. The best evidence of such a change is the beneficial effect on preventive care of using computerized reminders to patients. Telemedicine and consumer health informatics also have limited evidence of benefit in specific contexts. The evidence is much more limited about the health effects of more general, interactive health information technologies such as the Internet or e-mail, or the effect on patient trust and satisfaction of implementing HIT systems such as the electronic health record.

Barriers to HIT Implementation

Studies identified a large number of barriers to the implementation of HIT. These barriers can be classified as situational barriers (including time and financial concerns), cognitive and/or physical barriers (including users' physical disabilities and insufficient computer skills), liability barriers (including confidentiality concerns), and knowledge and attitudinal barriers. Cutting across all of these categories, however, may be the need for a major structural and ideological reorganization of clinical medicine as it is now practiced in the majority of settings to be able to integrate itself with and enjoy the benefits of HIT.

CONCLUSIONS

Limitations of the Review

• The primary limitation of this review is the quality, quantity, and generalizability of the available (published) studies. Substantially more information regarding implementation may have been obtained by contacting leading HIT implementers and conducting structured interviews with them.
• Many of the costs and financial benefits of EHR will change over the years, because they depend on the changing price of such factors as hardware, software, labor, and pharmaceuticals and medical devices. Consequently, the costs reported in some of the older articles are of limited relevance.

General Conclusions

• Predictions based on statistical models suggest that HIT has the potential to assist in dramatically transforming the delivery of health care, making it safer, more effective, and more efficient. However, the experimental evidence supporting benefits from HIT is more limited.
• A number of large health care organizations have realized some of these major gains through the implementation of multifunctional, interoperable HIT systems built around an electronic health record.
• The impact of HIT implementation on the cost and quality of care is not going to be consistent across institutions, independent of context. However, the specific context within which HIT is implemented, including the setting, the clinical issues, and the patient populations, greatly influences its use and effects.
• More widespread implementation of HIT is limited by the lack of generalizable knowledge about what types of HIT and methods for its implementation will prove most useful for specific health organizations, especially for small practices and small hospitals.
• The reporting of HIT developments and implementations needs to be improved, with greater attention given to descriptions of both the intervention and the organizational/economic environment in which the technology is implemented.
• A high priority must be placed on establishing standards for the information that needs to be measured and reported in studies of HIT implementation, similar to the CONSORT standards developed for reporting clinical trials of therapeutics.
• Using existing published evidence, it is not possible to draw firm conclusions about which HIT functionalities are most likely to achieve certain health benefits--and the assessment of costs is even more uncertain.
• Existing evidence is not sufficient to clearly define "who pays for" and "who benefits from" HIT implementation in any health care organization--except those, such as Kaiser and the VA, that are responsible for paying for and delivering all the care for the defined population.
• Statistical models can be built to estimate the costs and benefits of interoperable HIT systems within and across health care provider settings, payers/purchasers, and cumulatively across the health care continuum, but these models are based on many untested assumptions.
• Implementation of HIT faces many barriers, including institutional barriers, cognitive and/or physical barriers, liability barriers, and knowledge and attitudinal barriers.