Measuring Long-Term Care Work: A Guide to Selected Instruments to Examine Direct Care Worker Experiences and Outcomes

Measuring Long-Term Care Work: A Guide to Selected Instruments to Examine Direct Care Worker Experiences and Outcomes

Executive Summary

Kristen M. Kiefer, MPP
Lauren Harris-Kojetin, PhD
Diane Brannon, PhD
Teta Barry, PhD
Joseph Vasey, PhD
Michael Lepore, PhD Candidate

Institute for the Future of Aging Services

April 2005

This report was prepared under contract #HHS-100-01-0025 between the U.S. Department of Health and Human Services, Office of the Assistant Secretary for Planning and Evaluation, Office of Disability, Aging and Long-Term Care Policy and the Institute for the Future of Aging Services. Additional funding was provided by the Office of the Assistant Secretary for Policy, U.S. Department of Labor. For additional information about this subject, you can visit the DALTCP home page at http://aspe.hhs.gov/daltcp/home.shtml or contact the ASPE Project Officer, Emily Rosenoff, at HHS/ASPE/DALTCP, Room 424E, H.H. Humphrey Building, 200 Independence Avenue, S.W., Washington, D.C. 20201. Her e-mail address is: Emily.Rosenoff@hhs.gov.

Additional information can also be found at the OASP home page at http://www.dol.gov/asp/welcome.html or contact the OASP Project Officer, Stephanie Swirsky, at DOL/OASP, Suite S-2312, 200 Constitution Avenue, N.W. Washington, D.C. 20210. Her e-mail address is: Swirsky.Stephanie@DOL.GOV.

This report was prepared under contract #HHS-100-01-0025 between the U.S. Department of Health and Human Services, the U.S. Department of Labor, and the Institute for the Future of Aging Services (IFAS). The views expressed are those of the authors and should not be attributed to the Federal Government, to IFAS or its funders.

ACKNOWLEDGMENTS

Several IFAS staff contributed significantly to the content, format, and production of this Guide: Trish Hampton, Executive Assistant; Debra Lipson, MHSA, Deputy Director, Better Jobs Better Care; Nancy Mosely, Administrative Assistant, Better Jobs Better Care; Kristen Santaromita, MHA, Research Assistant; and Robyn I. Stone, DrPH, Executive Director.

IFAS project staff would like to thank our project officers, Andreas Frank (Office of the Assistant Secretary for Planning and Evaluation, U.S. Department of Health and Human Services) and Stephanie Swirsky (Office of Policy, U.S. Department of Labor) for their careful, thoughtful, and valuable review and comments on this Guide.

IFAS staff would also like to thank the Key Informants and Technical Expert Panel (TEP) members for their insightful comments and suggestions throughout this process. Appendix H contains a list of Key Informants and TEP members.

Special thanks to Judith Braun, RN, PhD, Director of Affiliate Services, Kendal Corporation; Farida Ejaz, PhD, LISW, Senior Research Associate, Margaret Blenkner Research Institute; Elsie Norton, MBA, Vice President, Health Services Administration, ACTS Retirement-Life Communities; and Linda Noelker, PhD, Senior Vice President, Planning and Organizational Resources, Benjamin Rose and Editor-in-Chief, The Gerontologist, for the feedback they provided throughout the Guide’s development.

IFAS staff would like to recognize representatives from provider organizations who shared their work with measuring employee experiences and outcomes in this Guide: Judith Passerini, CNHA, CAS, Deputy Secretary and Chief Operating Officer, Catholic Health Care Services; Jan Roth, SPHR, Director of Human Resources, Christian Living Campus; Julie Secviar, Senior Vice President of Strategic Resources, Franciscan Sisters of Chicago Service Corporation; and Renae Spohn, RHIA, CPHQ, Quality Improvement Department Director, Good Samaritan Society.

EXECUTIVE SUMMARY

Long-term care (LTC) providers face enormous challenges each day trying to provide high quality care to clients. One of the biggest challenges is staff retention among direct care workers (DCWs) -- the nursing assistants, personal care attendants and home health aides who provide hands-on care to clients.

High turnover rates among DCWs are costly. Both the direct costs (recruiting, training new employees, hiring temporary staff) and indirect costs (reduced productivity, deterioration in organizational culture and morale) associated with turnover can compromise the quality and continuity of residents’ care.¹

While doing nothing about turnover can be costly, doing something that does not address the real causes of turnover in an organization can also be expensive and frustrating. Surveys and research show that employees’ feelings about various aspects of their jobs affect their commitment, overall job satisfaction, and the likelihood that they will remain with their employer (Kuokkanen & Katajisto; 2003; Laschinger, Finegan, & Shamian, 2001; Burke, 2003).

Employee surveys can help pinpoint what may improve staff satisfaction. They can help identify the key drivers of staff satisfaction, which can differ in each organization. They can quickly tell managers whether it is best to focus on supervision, skill development, or advancement opportunities. Quantitative findings from surveys or records-based data are a nice complement to qualitative data organizations often collect through focus groups or in-depth interviews with employees.

While there are some standard questions that organizations may regularly ask employees, most organizations have unique cultures or goals that influence the types of questions that should be asked of employees. Each organization’s workforce goals, such as improved retention or enhanced skills in providing care, may determine which survey instruments are best.

This Guide was developed to help providers devise appropriate surveys for measuring DCWs’ opinions about their jobs. This Guide can help organizations:

• Understand the importance of accurate measurement to guiding effective DCW retention efforts
• Develop a measurement plan to target DCW retention strategies
• Become a more informed user of survey-based and records-based data for monitoring and improving the work environment.

Benefits of the Guide

While this Guide may be helpful to many audiences -- providers, state agencies, workforce development groups, worker groups and researchers -- it is intended for providers in institutional, home care and other residential settings. Many different types of providers may find this Guide useful. Some may already be surveying employees using an in-house research center or an outside data collection vendor, but wish to enhance or supplement them in a number of areas. Others may not be conducting employee opinion surveys yet and want to know more before jumping in.

For providers already using measurement

If organizations are already conducting surveys or measuring turnover or retention rates in a systematic way, this Guide can provide additional ways to supplement workforce measurement efforts. This Guide provides a wealth of measures in 12 topic areas that have proven reliability and validity and are free of charge. Reliability is the degree to which an instrument can produce consistent results on different occasions. Validity is the degree to which an instrument measures what it is supposed to measure (CDC, 2002).

Chapter 3 provides definitions of the topic areas included in this Guide. It includes measures for each topic area that organizations may use to enhance the effort and resources already dedicated to using worker outcomes and experiences to inform organizational decisions.

For providers interested in measurement who would like more information

If organizations are not yet conducting surveys but are interested in learning more about how to do it, this Guide can help them understand the many ways that investing in measurement of outcomes could be beneficial. Chapter 2 provides examples of how other LTC providers use information collected from measurement instruments in a meaningful way. Measurement can help organizations make informed decisions about things that may specifically help given particular circumstances. For example, if an organization has noticed a lag in the energy levels of direct care staff, management may want to understand the cause. Do they feel monotony in their daily tasks? Do they feel their workload is too heavy?

For those concerned that they don’t have the knowledge or skills to measure staff outcomes or survey employees, this Guide might make it easier to specify needs and concerns if an organization decides to engage local researchers or become a consumer of vendor services. It will also give basic tools to help administer surveys and/or participate in the data collection process with their guidance. Appendix C provides detailed information on issues to think about and discuss with researchers or consultants when planning and implementing a data collection and analysis process. Chapter 3 provides descriptions of the topic areas, measures and subscales organizations may consider using to address the issues most relevant to them.

Uses of the Guide

Employee opinion and outcome measurement can be done in different ways, depending on the purpose of the survey. Organizations might choose to use this Guide for certain purposes:

• Measure a single topic of interest using one of the instruments in Chapter 3
• Construct a multi-topic survey instrument, either with or without assistance of researchers/consultants, using several of the instruments in Chapter 3
• Gain access to existing survey instruments that encompass many topics in Appendix F

Measure a single topic of interest

For organizations that would like to understand how employees feel about a specific part of their job (e.g., the organizational culture, or perceptions of their job design, or their relationship with clients/residents), the use of a single measure might best meet this need. For example, if an organization recently implemented a participatory team approach where CNAs have input into a resident’s care plan, it can measure CNAs’ perceptions of the way their jobs are designed and find out if they have improved. Before implementation, a survey can establish a “baseline” of CNAs’ feelings and subsequent surveys can be conducted after implementation at a specific time (e.g., 6 months or 12 months after implementation). In this case, the topic area titled “Job Design” in this Guide may help the organization identify measures that could capture CNAs’ feelings. In Appendix A, a scenario is provided of how a nursing home may use this Guide to measure a single topic of interest (based on its organizational needs).

Construct a multi-topic survey instrument

Many organizations would find it more efficient to survey employees about numerous topics all at one time. In this case, the development of a survey instrument relevant to the organizational goals is more involved than simply using a one-dimensional measure or its subscales. A first key step is to select the topics and related measures or subscales from Chapter 3 that are most consistent with organizational goals. Next, the organization would likely opt to construct and pretest the questionnaire, develop strategies for administering it and discuss how the results will be analyzed, communicated back to staff and addressed. Many organizations have found it helpful to work with a consultant or researcher during this process. In Appendix A, a scenario of a continuing care retirement community (CCRC) that constructed its own multi-topic survey instrument is provided as an example.

Gain access to existing multi-topic survey instruments

Some organizations might prefer access to existing survey instruments that measure multiple topics. Appendix F includes two instruments that have already been developed for specific purposes and have not been tested for reliability and validity themselves, however.

Other Tools Available in the Guide

• sample scenarios for selecting and developing survey instruments
• overview charts of all measures and their properties
• discussion around data collection and analysis issues
• templates of letters to use when surveying employees
• copies of survey instruments ready for use
• additional workforce instruments that are not the focus of this Guide but may be useful
• names and affiliations of Key Informants and Technical Expert Panel members who helped develop the Guide

NOTES

1. Some estimates have shown that it costs nursing homes $3,000 to $4,000 to replace a nursing assistant who resigns or is fired (Noelker and Ejaz, 2001). These costs are likely underestiamted because they are generally based on direct care costs and do not account for indirect costs, which are more difficult to quantify (Seavie, 2004).