Rival perspectives in health technology assessment and other economic evaluations for investing in global and national health. Who decides? Who pays?

Introduction

It seems a desirable thing to set international quality standards for conducting health technology assessments and other economic evaluations (EEs) of the impact and desirability of investments that promote health¹. Such standards are, however, not easy to define, let alone to get agreement on². What is easy is to fall into a trap of small-town thinking, by setting standards whose applicability is arbitrarily and quite needlessly restricted to: (a) particular cultural and political contexts, (b) within any such context, a set of choices bounded by further arbitrary value judgments, (c) a presumption that EE methods are designed for use only in the public sector – and, moreover, only when the object is “the public interest”. Parochialism has two unattractive features. First, it denies the possible usefulness of EEs outside the public sector and to non-governmental agencies, like trade unions, for-profit companies and non-profit organisations like charities. Second, it imports value judgments to the effect that (a) EE can be conducted in a context-free way without attention being given to cultural and political values that may be particular to the application, which may not be readily transferable from jurisdiction to jurisdiction and (b) that the use of patients’ (or the public’s) preferences, in constructing outcome measure such as the quality-adjusted life year (QALY) or disability-adjusted life year (DALY) via estimated or expressed consumer values, rather than using, say, judgments made by parliamentary committees, expert groups or citizens’ juries, provides an appropriate and relevant index of value. We seek to avoid such restraints by proposing that questions of perspective be determined in context, where the job of EE analysts is to help decision makers to articulate a perspective that is appropriate to the circumstances rather than to prescribe a perspective independently of its context of use (being “context free”).

First, we review the recent history of “perspective”.

What is perspective and why does it matter?

Guides to best practice in cost-effectiveness analysis (CEA) can be traced back at least as far as 1980, with the publication, at the initiative of Senate Committee on Labor and Human Resources and commissioned by the then Office of Technology Assessment (OTA), of The Implications of Cost-Effectiveness Analysis of Medical Technology³. Other significant landmarks appeared in 1994 by the Canadian Coordinating Office for Health Technology Assessment⁴, in 1996 by Gold et al. (the “Washington Panel”)⁵, in 1997 by Drummond et al.⁶, in 2003 by the World Health Organization⁷, in 2004 by the National Institute for Clinical Excellence (NICE)⁸ and in 2016 by the second Panel⁹ and the International Decision Support Initiative (IDSI)¹⁰.

The Washington Panel coined the term “Reference Case” to describe the characteristics of a high quality economic evaluation, thereby establishing both a lasting piece of terminology and launching a stream of literature (see original publication here: https://www.ncbi.nlm.nih.gov/pubmed/8861994 and over 170 references in PubMed: https://www.ncbi.nlm.nih.gov/pubmed?LinkName=pubmed_pubmed_citedin&from_uid=8861994). The Panel’s work absorbed much of that by Drummond et al⁶. One of the characteristics of a high-quality EE was the explicit stating of a study’s “perspective”, described by the panel as “the study’s point of view”, which “determines which health outcomes and costs are relevant and plays a part as well in how they should be valued”. We wholly agree with this principle and it seems to have formed a part of all subsequent reference cases, including one developed by the Drummond et al. team and us, through the Gates-sponsored iDSI Reference Case (http://www.idsihealth.org/resource-items/idsi-reference-case-for-economic-evaluation/). The case for it is made not merely on grounds of transparency, which matters because it enables other researchers to check that the scope and measures of costs and benefits were indeed fit for purpose, but also on grounds of accountability: what to include and how to measure it are essential political and social value judgments that ought to be made by duly accountable decision makers (not EE analysts!). A good study will be, therefore, explicit about its perspective (or perspectives, if more than one is to be explored).

The Washington Panel recognised that a perspective must reflect the purpose of a study. It is not always appreciated that the perspective will usually be context-dependent, with the context determining the appropriate perspective. For example, a study might appraise the costs and benefits of a workplace intervention to reduce back strain on workers through redesigned work stations from the points of view of managers on the one hand and trade unions on the other, with a view to anticipating opposition and identifying compensatory and regulatory measures for successful implementation. In such a case, there are three perspectives to be considered – those of management, workers and government. The perspectives to be used are defined by the purpose of the study which in turn is specified by its sponsor: in this example, management, workers or the government (or a consortium of all three).

When it comes to perspectives, is broader better?

The First Washington Panel’s Reference Case recommended that a societal perspective be adopted, offering decision makers as wide as possible an understanding of the costs and consequences of alternative actions both in the health care sector and beyond it into the private health care sector and non-health sectors, public or private. Its reasoning is plain: only if the societal approach is adopted can decision makers be confronted with a full information set of the costs and consequences of alternative actions. Anything less comprehensive will necessarily be subject to omitted variable bias, probably of unknown sign and size, causing either over- or under-investment in both old and new technologies. It is hard to raise an objection of principle to this. Pragmatically, however, to seek inclusion of every consequence, no matter what the costs are of tracking it down and quantifying it, seems completely over the top: the limits of inclusion ought more rationally to be determined by a sensitivity analysis of whether inclusion or exclusion makes a difference of significant concern to any relevant outcome. At any rate, reasonably full information might be conceded as a desirable thing when the objective is to produce a decision that is in the public interest. What constitutes “reasonable” in any given context would necessarily be a matter for local judgment (decision makers and advisers). If the objective were less comprehensive than societal, however, to collect full information would be to collect much that was irrelevant to the context of that decision question. The argument that “more information is always better” is therefore not an argument for adopting the societal perspective; it is an argument for EEs that can reflect many perspectives, including the “societal” one, but only up to the point at the value of further information is judged to be less than the cost of collecting and incorporating it.

What “societal” means is, even in its most comprehensive version stated in the literature, commonly rather more restricted in terms of the nature of the consequences considered than may be thought. Compassionate externalities^11–13 are usually not included. Stresses and welfare losses resulting from changes in delivery pathways, provider identity, loss of employment (temporary or permanent), limited involvement in or being informed about changes, confusion and misunderstanding of the purposes of changes – such effects characteristically accompany both investment decisions and changes in the ways in which decisions are made but none, however, feature in the scope of “societal” as commonly stated. “Societal” is thereby significantly short of being as comprehensive as may appear.

In any event, in practice, not all have followed the Panel’s recommendation. Table 1 provides an illustrative variety of approaches. As would be expected, perspectives offered by an academic do not necessarily reflect the expressed views of public agencies. Developed countries like England, Australia and Canada rely on EE to inform allocation decisions and tend to be narrower in their scope of analysis whereas global Reference Cases tend to adopt a wider perspective. Some low-income and middle-income countries (LMICs), where out-of-pocket costs are significant select a societal perspective though very few have a track record of using EE systematically.

The limits of feasibility are much more tightly drawn in LMICs than in rich countries, where a closer approximation to “full information” may be both possible and desirable. In LMICs, what is possible is much more limited due to poor or absent data, lack of technical skills to conduct and use EEs, and lack of political understanding, leadership and support^14,15. Table 2 shows that a healthcare payer’s perspective is the most popular recommendation in the national guidelines of high-income countries where EE is commonly used to inform reimbursement of health services. Only a third of the guidelines from both high-income countries and LMICs recommend a societal perspective.

It would also be rash to suppose that what is regarded as the public interest is always and everywhere the same. Just such a belief underlies the World Health Organization (WHO) advocacy of DALYs as a kind of universal outcome metric with weights attached to dimensions of disability that are invariant with respect to culture and context. This is advanced as an advantage of DALYs over QALYs, since QALYs typically use customized country weights. A similar insensitivity can be seen in the WHO’s abandoned attempts to recommend thresholds¹⁶. A threshold is an aggregate expression of collective willingness to pay for treatments in national health insurance. It represents a marginal preference for public expenditure on health care relative to the many other objects of public policy – poverty reduction, education and training, housing, law and order, civil and national defence, and so on. There can be no presumption that one size fits all, regardless of political stability, stage of economic development, culture, religion, or history. We should not encourage a similar insensitivity to enter an authoritative methodological guide to best practice in EE.

Societal perspective: Whose values and whose judgments count?

A problem with the use of a societal perspective is that it raises the possibility of reintroducing into health policy decisions many of the very biases that most health care systems have been designed to avoid. A critically important one concerns the implications of a perspective for the role of valuation and willingness to pay for health gains. It is part of the conventional wisdom of health economics that the health care market is characterised by more market failures – and serious ones at that – than almost any other market. Many of these impinge on the reasonableness of using individual willingness to pay as a priority-setting criterion and, more generally, of using the Pareto criterion (or the potential Pareto criterion) to identify socially beneficial changes¹⁷. These are standard aspects of the textbook treatments of heath economics: we merely list those that are of chief concern (Table 3). It is not satisfactory, as in Pauly (1995)¹⁸ and others who advocate the use of consumer preferences as a basis for public priority setting, to glide over these issues as if they were empirically or politically unimportant. The problems of reliance on consumer preferences for healthcare relate not only to measures of individual (as distinct from collective) willingness to pay but to all expressions of patient preferences. They raise the difficult question of the right balance to be struck between individual expressions of value and collective ones.

This is not the place to argue the cases for or against individual willingness to pay as a basis for making health investments or of locating the patient as a decision maker in a healthcare system. Our purpose is only to argue that a societal perspective need not necessarily require its source of value, whether or not monetised, to be individual patients. The welfare of patients remains a central concern but avoiding the distortions of the real world is necessary if patients are to be responsibly protected.

What, therefore, is the appropriate source for social valuations of the kind used in EE? And whose willingness to pay for the treatments offered in public insurance benefit packages of the sort aspired to, especially in LMICs, should be embodied in policy? We are not concerned here with individual choices of clinical providers and treatments, though the bulleted issues below do have implications for information provision, clinical audit and health care quality assurance. The questions for EE concern instead the value content of a set of key collective judgments that usually have to be made. The answer to all of the following will be shaped by the perspective:

The collective answers to these questions require no knowledge of individual willingnesses to pay for health care. Instead they require decisions taken at the societal level by decision makers accountable through some locally legitimate political process, advised where appropriate by expert opinion and supported by the best available evidence. Individual preferences/values might be considered relevant in selecting outcome measures, their dimensions (physical, mental functioning, etc.), the weights attached to various dimensions that enable them to be added up to create an index of outcome, and the weights (commonly unity) to attach to individuals when individual levels of health or disability are aggregated to community level. Decision makers need to exercise judgment regarding the extent to which individual members of a community are to be involved in these judgments. The societal judgments that are required are themselves choices that should be context-dependent and essentially locally accountable.

“Societal” does not necessarily imply “everyone”. Decision makers may have their own views about whom to consult and when. They will have their own views as to what exposures to risk and distributions of cost and benefit are fair. They may have general across-the-board financial rules for public sector investments that stipulate discount rates and time horizons. They will address at a higher level what can be afforded for health care in the light of pressing needs for education, housing, poverty relief, law and order, etc. They will decide whether their constitutional preference is to narrow the idea of health to whatever impact health care provision has on it. They will decide whether to consider non-health consequences of health care investments.

Who are the stakeholders? A tentative list

Stakeholders all have perspectives, some of which may be shared, but most of which will need to be accommodated in some way in the methods of EE and the processes through which it is conducted. The extent to which individual ideas about perspective can be effectively suppressed in the interest of having a shared and publicly stated perspective has not yet, so far as we are aware, been explored empirically. EE analysts can give guidance. For example, a suggested list of possible stakeholders whose roles are for decision makers to consider (which has to be context-sensitive) might be as follows:

Selection of stakeholders and possible roles they might play in determining the perspective are matters on which EE analysts may advise, for example, by listing options, sharing experience elsewhere, indicating likely consequences, and building models.

On the role of the analyst (and how we differ from others)

Our recommendation is, then, that analysts should approach the issue of perspective at two levels of specificity: the universal or general (context-free) on the one hand, and the context-dependent on the other. A fundamental universal context-free principle is that the perspective always be stated, be consistently adhered to throughout an EE study and appear in reports of the study. Another is that the reasons for the perspective should be given, any departures from previous or common practice elsewhere noted and explained.

These two principles will also shape much of the process in conducting an EE; for example, identifying partners and stakeholders, the extent of consultation, the need for new context-specific research, time-lines and reporting accountability. Their roles will depend upon the context-specific perspective actually adopted. For example, most perspectives will doubtless need to include physicians and patients in consultation meetings, but the scope of the issues discussed will differ according to the specific perspectives: in some studies they might advise on the probable accuracy of the outcome measure and the likely cost consequences of adopting a new technology in the social care sector, in others, they may be concerned only with the outcome measure. The national payer may specify that no additional resources will be available, which will encourage a relatively narrow perspective. Alternatively, the national payer may want an EE performed in order to inform a review of public expenditure across all sectors, in which case a broader public-sector perspective is implied. A health advocacy group, by contrast, may wish a societal perspective to be adopted to establish a case for increasing the health budget. A public payer committed to health maximisation is likely to adopt a perspective that focuses narrowly on health as an outcome while being much broader in its consideration of cost-effective means of promoting health by embracing nutrition, public education, housing and other determinants not normally within the direct control of the Ministry of Health. In each case the perspective is determined by the interests of the client for the analysis. A donor committed to universal health care might adopt any of a number of positions regarding perspective, depending both on its fundamental strategic purposes (perhaps with a single disease perspective) and on the specific context of application (perhaps having specific regard to local capacities to manage complex delivery requirements).

Conclusions

Our conclusions can be simply stated: