Special communication
Response Shift Theory: Important Implications for Measuring Quality of Life in People With Disability

Presented to the Expert Panel on Health Status Measurement, Oregon Health & Science University Rehabilitation Research and Training Center, November 7, 2005, Houston, TX.
https://doi.org/10.1016/j.apmr.2006.12.032Get rights and content

Abstract

Schwartz CE, Andresen EM, Nosek MA, Krahn GL, and the RRTC Expert Panel on Health Status Management. Response shift theory: important implications for measuring quality of life in people with disability.

Measurement of health-related quality of life (HRQOL) in people with disability can be problematic. Ambiguous or paradoxical findings can occur because of differences among people or changes within people regarding internal standards, values, or conceptualization of HRQOL. These “response shifts” can affect standard psychometric indices, such as reliability and validity. Attending to appraisal processes and response shift theory can inform development of HRQOL measures for people with disability that do not confound function and health and that consider important causal indicators such as environment. By design, most HRQOL measures equate function with health, necessarily leading to a lower measured HRQOL in people with functional impairments regardless of their level of self-perceived health. In this article, we present theoretical and conceptual distinctions building on response shift theory and other current developments in HRQOL research. We then submit a set of suggested directions for future measurement development in populations with disabilities that consider these distinctions and extend their use in future measurement developments.

Section snippets

Definitions and Distinctions

In this article, we use the World Health Organization’s definition of health as a state of complete physical, mental, and social well-being and not merely the absence of disease or infirmity.2 We are in accord with McHorney,3 who in 1999 wrote that HRQOL should be regarded as equivalent to health status, and we use HRQOL to refer to a perceived state of health of an individual, group, or population that reflects the degree to which a person is able to participate physically, emotionally, and

Response Shift in HRQOL Data

The growth in the field of HRQOL research and, more broadly speaking, self-reported health outcomes, has resulted in a substantial number of generic and condition-specific tools for measuring HRQOL. Although this development has promoted the voice of the research participant in outcomes research, this voice may be muffled because of inherent ambiguity in self-report measurement items. To examine this issue of item ambiguity further, consider an item that has been used in many HRQOL instruments:

Identifying Response Shift in HRQOL Measurement

We and others suggest that beneath the surface of HRQOL measurement lie response shift phenomena. Response shift refers to a change in the meaning of one’s self-evaluation of the target construct QOL as a result of (1) a change in the respondent’s internal standards of measurement (ie, recalibration), (2) a change in the respondent’s values (ie, reprioritization), or (3) a redefinition of the construct by the respondent (ie, reconceptualization).8, 9 Originally explored in the contexts of

How Can Response Shift Theory Improve the Measurement of HRQOL for People With Disabilities?

Figure 1 provides an annotated version of the Sprangers and Schwartz9 theoretical model of response shift as a starting point for discussion. This model serves to predict changes over time in perceived HRQOL as a result of the interaction of catalysts, antecedents, mechanisms, and response shifts. Catalysts refer to health states or changes in health states, as well as other health-related events, treatment interventions, the vicarious experience of such events, and other life events

Effect Versus Causal Indicator Distinction

In conceptual and statistical modeling, an important distinction is whether the outcome construct is operationalized with effect indicators or with causal indicators. This distinction is shown in figure 2 for the construct of HRQOL in the context of disability. Effect indicators are those variables that are regarded as reflecting the latent variable. In this example, measures of pain, social participation, and sense of well-being are regarded as reflections of the latent variable

Focusing on Evaluation-Based Ratings to Capture Response Shifts

A second important distinction relates to the 3 types of measurement items commonly used in QOL measures in determining which is most appropriate for detecting response shifts. QOL items are generally either performance based, perception based, or evaluation based.18 Performance-based items involve a measurement process that is independent of judgment (eg, “How long does it take Juanita to prepare dinner?”), and discrepancies between expected (based on statistical modeling) and observed scores

Evaluation-Based Ratings and the Appraisal Process

Evaluation-based items are key portholes into a person’s appraisal process. This appraisal process is seen as comprising 4 parameters: (1) induction of a frame of reference, (2) recall and sampling of salient experiences, (3) use of standards of comparison to appraise experiences, and (4) application of a subjective algorithm to prioritize and combine appraisals to arrive at a QOL rating.19 In our example, Juanita’s frame of reference changed dramatically when she changed from being an

How Does Response Shift Affect the Psychometrics of Measurement?

The standard criteria for measurement validation are that the measure must be reliable and valid. Good reliability is defined as high internal consistency, convergence among raters, and stability over time in the absence of change. Good validity is defined as correlating very highly with other instruments that measure the same construct (ie, criterion validity), correlating somewhat highly with other instruments that assess related constructs (ie, construct validity), and distinguishing among

How Does Response Shift Affect Clinical Decision Making?

In addition to the noted caveats in using ambiguous HRQOL data in the context of research, there are clear costs to using ambiguous HRQOL measures in clinical practice. Detecting HRQOL problems and responding appropriately to them may be more difficult if response shifts are prevalent and unrecognized. Furthermore, ignoring this ambiguity may result in missed opportunities to learn about the role of meaning on the process of response shift, particularly as it relates to negative shift, and the

First Steps in Understanding the Role of Response Shift and Environment in Health Assessment for Persons With Disabilities

We believe that response shift theory has the potential to help in understanding the so-called disability paradox. Emerging methods aimed at detecting and exploring response shifts in HRQOL data can provide useful tools for developing a measure of HRQOL for people with and without disabilities that does not penalize them for impairment. As a starting point for such measurement development, we suggest the following first steps. First, the item-development process for any new measure of HRQOL

Conclusions

We believe that response shift phenomena are salient in health, HRQOL, and disability measurement and that considering these phenomena will enhance the conceptual clarity of relevant psychometric models as well as facilitate research and clinical measurements. We believe that health and disability measurement would benefit from focused attention to appraisal processes, comparison of measurement and structural models, disability neutral item identification, and important distinctions between

Acknowledgments

Members of the RRTC Expert Panel on Health Status Measurement are: Elena Andresen, PhD, University of Florida; Vincent Campbell, PhD, Centers for Disease Control; Brad Cardinal, PhD, Oregon State University; Charles Drum, JD, PhD, Oregon Health & Science University; Glenn Fujiura, PhD, University of Illinois at Chicago; Trevor Hall, PhD, Oregon Health & Science University; Laura Hammond, MPH, Oregon Health & Science University; Willi Horner-Johnson, PhD, Oregon Health & Science University;

References (52)

  • C.E. Schwartz et al.

    Learning from unreliability: the importance of inconsistency in coping dynamics

    Soc Sci Med

    (1999)
  • G. Whiteneck et al.

    Environmental factors and their role in participation and life satisfaction after spinal cord injury

    Arch Phys Med Rehabil

    (2004)
  • World Health Organization. Preamble to the Constitution of the World Health Organization as adopted by the...
  • C.A. McHorney

    Health status assessment methods for adults: past accomplishments and future challenges

    Annu Rev Public Health

    (1999)
  • C.S. Burckhardt et al.

    The Quality of Life Scale (QOLS): reliability, validity, and utilization

    Health Qual Life Outcomes

    (2003)
  • J.A. Smith

    The idea of health: a philosophical inquiry

    ANS Adv Nurs Sci

    (1981)
  • International classification of functioning, disability and health

    (2001)
  • A. Garratt et al.

    Quality of life measurement: bibliographic study of patient assessed health outcome measures

    BMJ

    (2002)
  • A.A. Armenakis

    A review of research on the change typology

    Res Org Change Dev

    (1988)
  • R.T. Golembiewski et al.

    Measuring change and persistence in human affairs: types of change generated by OD designs

    J Appl Behav Sci

    (1975)
  • G.S. Howard et al.

    Internal invalidity in pretest-posttest self-report evaluations and a re-evaluation of retrospective pretests

    Appl Pychol Meas

    (1979)
  • C.E. Schwartz et al.

    An evaluation of a psychosocial intervention for survivors of childhood cancer: paradoxical effects of response shift over time

    Psychooncology

    (1999)
  • F.J. Oort et al.

    Response shift detection in the measurement of health status of patients with bone metastasis [abstract]

    Qual Life Res

    (2005)
  • P. Fayers et al.

    Factor analysis, causal indicators and quality of life

    Qual Life Res

    (1997)
  • P.M. Fayers et al.

    Causal variables, indicator variables and measurement scales: an example from quality of life

    J R Stat Soc A

    (2002)
  • C.E. Schwartz et al.

    Reconsidering the psychometrics of quality of life assessment in light of response shift and appraisal

    Health Qual Life Outcomes

    (2004)
  • Cited by (280)

    View all citing articles on Scopus

    Supported in part by the National Institute of Disability and Rehabilitation Research (grant no. H133 B040034) and the Department of Veterans Affairs Medical Center, Gainesville, FL.

    No commercial party having a direct financial interest in the results of the research supporting this article has or will confer a benefit upon the author(s) or upon any organization with which the author(s) is/are associated.

    View full text