Evaluation of Vocational Rehabilitation Services

This book presents the latest advances in models and data for evaluating the efficacy of vocational rehabilitation (VR)services provided to individuals with disabilities. For the first time, the VR-ROI (return on investment) model is used tosimultaneously compare short- and long-term labor market outcomes across multiple state agencies and four distinct disability groups. For each disability group, the book provides information about the return on investment, as measured bythe rate of return, for VR services. By offering this broad and in-depth evaluation in concert with intuitive explanations of the model and the estimation methodology, the book helps to bridge the gap between research and practice and to equip stakeholders with data-driven insights to enhance vocational rehabilitation programs for individuals with disabilities.

Return on investment (ROI) analysis of state vocational rehabilitation (VR) agencies is a way to evaluate the efficacy of a VR program. Several different formulas can be used to make this comparison, but all compare program benefits with costs in some manner. For every dollar spent on services provided to a VR client, the ROI reports how many extra dollars (in present value terms) the client earns as a result. Although an ROI measure is straightforward to calculate given its components, credibly estimating program benefits and costs from available data can be difficult (King & O’Shea, 2003; Clapp et al., 2019). In this chapter, we review the empirical literature on VR program ROI. To do so, we first provide an overview of the basic conceptual issues involved in estimating VR program benefits and costs and, ultimately, the ROI of VR programs. Our aim is to highlight some of the key issues in ROI evaluations and how the approaches used in the VR literature have evolved over time, not to provide an exhaustive how-to guide. McGuire-Kuletz and Tomlinson (2015) and articles in the special issue introduced by Schmidt et al. (2019b) provide a more detailed guide to ROI analysis of VR programs.

The goal of our work is to measure the effectiveness of vocational rehabilitation (VR) services by comparing benefits and costs. Relative to earlier work, we add the following important features:

In this chapter, we describe the model and data used to estimate the effectiveness of VR services and compute the return on investment of the VR program.

The goal of this chapter is to consider some of the costs and benefitsassociated with using national data instead of state agency data (as was discussed in Chap. 3). We provide information ona list of national datasets available for research and discuss the costs and benefits of using national datasets. We discuss the implied differences in modeling between using national datasets and state agency datasets. The discussion implies that using agency data is the preferred approach.

This chapter presents results from the most comprehensive and expansive application of the structural VR-Return on Investment (VR-ROI) model to date, which leverages longitudinal quarterly employment and earnings data to assess the impact of VR across eight distinct state agencies, four distinct disability groups, and seven to nine distinct VR services. For the first time, the VR-ROI model is employed to concurrently compare short- and long-run labor market outcomes across a broad spectrum of agencies and disability groups, which yields unparalleled insights into the diverse effects of vocational rehabilitation services. This analysis sets the stage for comprehensive understanding of VR’s efficacy and provides a detailed picture that informs stakeholders about the nuanced benefits and challenges within these critical programs.

Although the VR-ROI model provides a state-of-the-art approach for ROI analysis of VR, the model’s complexity can render interpreting and assessing the benefits of VR challenging. Moreover, the models are difficult to estimate and require advanced computational methods, statistical knowledge, programming skills, and computing resources. This complexity can make it prohibitive for VR agency staff to estimate and use such models to evaluate the ROI of VR programs in other states and time periods.

Given these practical concerns, a critical issue is determining whether a simplified model and estimator can provide credible agency-specific ROI estimates. Focusing on the North Carolina program as a case study, we estimate the benefits and net present value (NPV) of VR from simpler models that are relatively easy to understand and can be estimated using standard statistical software packages on a laptop computer.

In this concluding chapter, we briefly summarize some of the key themes from this book and then discuss directions for future research and refinements of the model.