Statistical Decision Theory and Bayesian Analysis

In evaluating the consequences of possible actions, two major problems are encountered. The first is that the values of the consequences may not have any obvious scale of measurement. For example, prestige, customer goodwill, and reputation are important to many businesses, but it is not clear how to evaluate their importance in a concrete way. A typical problem of this nature arises when a relatively exclusive company is considering marketing its “name” product in discount stores. The immediate profit which would accrue from increased sales is relatively easy to estimate, but the longterm effect of a decrease in prestige is much harder to deal with.

This chapter differs from later chapters in scope, because Bayesian analysis is an essentially self-contained paradigm for statistics. (Later chapters will, for the most part, deal with special topics within frequentist decision theory.) In order to provide a satisfactory perspective on Bayesian analysis, we will discuss Bayesian inference along with Bayesian decision theory. Before beginning the study, however, we briefly discuss the seven major arguments that can be given in support of Bayesian analysis. (Later chapters will similarly begin with a discussion of justifications.) Some of these arguments will not be completely understandable initially, but are best placed together for reference purposes.

This chapter is devoted to the implementation and evaluation of decision-theoretic analysis based on the minimax principle introduced in Section 1.5. We began Chapter 4 with a discussion of axioms of rational behavior, and observed that they lead to a justification of Bayesian analysis. It would be nice to be able to say something similar about minimax analysis, but the unfortunate fact is that minimax analysis is not consistent with such sets of axioms. We are left in the uncomfortable position of asking why this chapter is of any interest. (Indeed many Bayesians will deny that it is of any interest.) It thus behooves us to start with a discussion of when minimax analysis can be useful.

The invariance principle is an intuitively appealing decision principle which is frequently used, even in classical statistics. It is interesting not only in its own right, but also because of its strong relationship with several other proposed approaches to statistics, including the fiducial inference of Fisher (1935), the structural inference of Fraser (1968, 1979), and the use of noninformative priors. Unfortunately, space precludes discussion of fiducial inference and structural inference. Many of the key ideas in these approaches will, however, be brought out in the discussion of invariance and its relationship to the use of noninformative priors. The basic idea of invariance is best conveyed through an example.

We have previously observed that it is unwise to repeatedly use an inadmis­sible decision rule. (The possible exception is when an inadmissible rule is very simple and easy to use, and is only slightly inadmissible.) It is, therefore, of interest to find, for a given problem, the class of acceptable (usually admissible) decision rules. Such a class is often much easier to work with, say in finding a sequential Bayes, minimax or a Γ-minimax decision rule, than is the class of all decision rules. In this chapter, we discuss several of the most important situations in which simple reduced classes of decision rules have been obtained. Unfortunately, the subject tends to be quite difficult mathematically, and so we will be able to give only a cursory introduction to some of the more profound results.