Comparative Effectiveness Research (CER) Inventory

STATUS: Completed Project

BACKGROUND

The purpose of PCORTF’s specific contribution to the Comparative Effectiveness Research (CER) Inventory was initially to maintain, update, transition, and test the CER Inventory. The original plan was that the project would transition the operations of the inventory to Health & Human Services (HHS) staff at the end of the year. During the course of a year, it became evident that inherited factors were inhibiting the user-friendliness and utility of the inventory. As a result, the project was modified to conduct a benchmarking exercise that examined how the inventory performed, compared to other databases (i.e., Google Scholar, PubMed.gov, and ClinicalTrials.gov).

PROJECT PURPOSE & GOALS

The overall CER Inventory project was to design and implement a system for the categorization and cataloguing of CER activities through a web-based tool for inventorying Federal and non-Federal outputs and activity. This work had grown out of a larger project supported by the American Recovery and Reinvestment Act (ARRA) of 2009. ARRA had established the Federal Coordinating Council for CER (FCC) to foster optimum coordination of CER conducted and supported by Federal departments and agencies. The FCC submitted a report to the President and Congress defining CER and its purpose; describing the then-current Federal CER infrastructure and a partial inventory of CER activities; and making recommendations for investment. A simultaneous report of the Institute of Medicine assisted in catalyzing the creation of an inventory of existing and ongoing CER activity and data infrastructure.

Project Objective:

• Accurately and comprehensively inventory Federal and non-Federal CER activities (within both for-profit and not-for-profit organizations), utilizing the definition for CER adopted by the FCC for CER.

PROJECT ACHIEVEMENTS & HIGHLIGHTS

• The outcome of the comparisons showed that Google Scholar, PubMed, and ClinicalTrials were relatively comparable with the CER Inventory when CER Inventory experts were involved in the tests. Otherwise, the CER Inventory did not perform as well. Other challenges to the Inventory’s utility were identified. They included the evolving definition of CER, the broad scope of the inventory, and the lack of a defined audience, among others. In addition, other limitations included the retrospective algorithm, and the search-engine approach for handling the evolving definition of CER, and the ability accommodate manual submissions methods, such as those adopted by ClinicalTrials.gov. The project submitted the Lessons Learned Report along with final technical documentation and database copy of the Inventory.

• Due to the rapidly evolving technologies supporting web-based search engines, and the improved methods for identification of more recent CER studies, the development of the CER Inventory (as a web-based search engine using a retrospective algorithm) was determined to have been superseded by existing search engine tools available. The benchmarking exercise found the Inventory to be comparable to but less user-friendly and algorithm challenged than other common public search engines. Taken together, the advantages of addressing the challenges to the CER Inventory were outweighed by the cost involved. After much deliberation, the decisions were made not to pursue the necessary and sustained investment and to retire the CER Inventory.

PUBLICATIONS, PRESENTATIONS, AND OTHER PUBLICALLY AVAILABLE RESOURCES

Resources:

• The Agency for Healthcare Research and Quality (AHRQ) now hosts a Web-library for patient-centered outcomes research (PCOR), the Library of PCOR Resources. This resource provides up-to-date information; it was last reviewed in October of 2016. The library can be found here: https://www.ahrq.gov/pcor/library-of-resources/index.html

• The Lewin Group. Comparative Effectiveness Research Inventory: Final Report Task 6 Lessons Learned. (2013, September 27). Available upon request. Contact OSPCORTF@hhs.gov

RELATED PROJECTS

Below is a list of ASPE-funded PCORTF projects that are related to this project

Beta Testing the Multi-Payer Claims Database (MPCD) – The goal of the Multi-Payer Claims Database (MPCD) was to evaluate the benefits of consolidating longitudinal health care claims data from public and private payers to facilitate comparative effectiveness research (CER). As part of its development, the MPCD underwent a beta test to evaluate its utility by surveying the experiences of federal and contracted researchers who requested and used customized data extracts for CER studies. The purpose of this project was to conduct a beta test of the Multi-Payer Claims Database (MPCD). During the beta test, the contractor supplied data extracts to researchers accessing the database. Beta test participants included: 14 Federal researchers from FDA, AHRQ, NIH, ASPE HRSA, and SAMHSA; and 12 contracted researchers from eight universities.

Maintenance and Support of the Chronic Conditions Warehouse (CCW) for Comparative Effectiveness Research (CER) – The Centers for Medicare and Medicaid CCW provides researchers with Medicare and Medicaid beneficiary claims, and assessment data. The CCW data is linked by a unique beneficiary key, permitting researchers to analyze data across the continuum of care in Medicare and Medicaid administrative data. The PCORTF project provided maintenance and support for the CCW database and analytic files as well as provided one year funding to support 15 PCOR focused research studies conducted via the CMS Virtual Research Data Center (VRDC). Approved researchers using Medicare and Medicaid data may be permitted to link clinical and other data for their research projects.

Expanding Data Collection for the National Program of Cancer Registries (NPCR) for Comparative Effectiveness Research (CER) – Enhanced data collection and data linkages between central cancer registries and other health related datasets can provide researchers with essential real-world and population-based data for CER. This project built upon previous funding provided to central cancer registries operated by 10 states—part of the National Program of Cancer Registries (NPCR)—to collect detailed treatment and outcomes data from electronic data sources. The PCORTF funding extended longitudinal data collection for a subset of cancer patients in these 10 states, enabling patient-level CER of cancer outcomes, and to improve the tools and timeliness of cancer reporting to the Centers for Disease Control.