A note about model selection and tests for non-nested contingent valuation models

doi:10.1016/S0165-1765(01)00566-3

Economics Letters

Volume 74, Issue 3, February 2002, Pages 363-370

https://doi.org/10.1016/S0165-1765(01)00566-3 Get rights and content

Abstract

The different approaches (Vuong’s model selection test and Cox’s non-nested hypothesis test) for the evaluation of non-nested, discrete choice models are analyzed, and their performance compared with Akaike’s selection criterion for application to simulated contingent valuation data.

Introduction

Non-market goods can be evaluated by means of contingent valuation models, where the parameters of the underlying discrete choice model are estimated by maximum likelihood, and the value of the mean, or median, willingness to pay (WTP) is calculated as a function of the estimated parameters. It is well known that the parameter estimates, and hence the estimates of mean and median WTP, depend crucially on distributional assumptions, and therefore model specification should play an important role before policy implications are derived from any estimated model.

When comparing two different non-nested models for the willingness to pay, an applied researcher might follow one of two distinct paths: apply model selection criteria to come up with a ‘better’ model, in the sense that it fits the data better, or apply a non-nested test to see whether, given an alternative model and the data, there is evidence against the model of the null hypothesis (Gourieroux and Monfort, 1995). Although the purpose of carrying out a non-nested hypothesis test is not to come up with a ‘better’ model, the applied researcher might well decide to use either one or none of the models for further inference based on the results of the test. In contingent valuation analyses, non-nested competing models are generally assessed by means of selection criteria. For example, McFadden (1994) used the highest likelihood and the Akaike information criteria to choose among models. While some doubts as to the effectiveness of such methods have been raised (Reiser and Shechter, 1999), we are not aware of any alternative approach in this field; and, it might be added, that testing procedures for non-nested models are few in discrete data modeling in general. As reported by Pesaran and Weeks (2000), the extra computational difficulties that implementing the Cox test entail may explain why this path has been so neglected. However, the probabilistic framework suggested by Vuong (1989) for model selection has also not been much used, notwithstanding its computational simplicity.

The purpose of the present paper is to provide some guidance to the applied researcher as to what might be the outcome of the two different paths —Vuong’s testing approach to model selection and the Cox test for non-nested hypothesis — as well as of the widely used Akaike’s criterion. The experiment will involve the application of standard models for contingent valuation (and for discrete models in general) to simulated data.

The structure of the paper is as follows. Section 2 gives a brief background about the different procedures. Section 3 describes the experimental setting. The results of the simulation exercise are presented in Section 4. Section 5 contains our conclusions.

Section snippets

Methods

We are interested in comparing pairs of competing parametric families of conditional densities of Y_i given X_i, given by $H_{f} : f(y_{i} ∣x_{i};β), β∈B⊂ R^{F},$ $H_{g} : g(y_{i} ∣x_{i};γ), γ∈Γ⊂ R^{G} .$ A wide variety of criteria are available in the model selection literature to select the model that best fits the data. For instance, based on the Kullback–Leibler Information Criterion (KLIC), Akaike (1973) suggested comparing the values of the log-likelihood penalized by the number of parameters and selecting the model that yields the

Experimental design

We adopt the censored regression model proposed by Cameron and James (1987) and Cameron (1988) where the latent variable (WTP) is a known function of the regressors and an additive error term and what we observe is a YES–NO answer to a hypothetical payment question. For our experiment, we considered different DGPs for the WTP, obtained from a linear functional form for the deterministic part of the model, which is common to all experiments, and an additive error term that is varied across

Results

Table 1 reports the results of the pairwise model comparisons for each of the three DGPs (Normal, Extreme Value, translated Lognormal) and the three candidate models: probit, logit and weibit. The upper section of the table reports the percentage of times that a given conclusion is reached for each one of the three approaches, while the lower section reports which model yields the lowest MSE for the estimated mean and median WTP. The purpose of this study is to assess the three approaches in

Conclusions

When the selection of a model is based on model selection criteria or Vuong’s test, one should bear in mind that the ‘best fit’ model could still give a very poor fit if both models are heavily misspecified. Of course, it could be possible that, from some prior knowledge, the researcher is able to narrow down the pool of candidate models to those that are somehow ‘close’ to the truth: for an approach in this sense, see Carson and Jeon (2000).

From an operative point of view, in many

Acknowledgements

We would like to thank Karim Abadir for helpful comments on a previous draft of this paper.

References (14)

T.A. Cameron
A new paradigm for valuing non-market goods using referendum data: maximum likelihood estimation by censored logistic regression
Journal of Environmental Economics and Management
(1988)
C.Y. Sin et al.
Information criteria for selecting possibly misspecified parametric models
Journal of Econometrics
(1996)
H. Akaike
Information theory and an extension of the maximum likelihood principle
A. Aznar Grasa
Econometric Model Selection: A New Approach
(1989)
T.A. Cameron et al.
Efficient estimation methods for closed-ended contingent valuation surveys
The Review of Economics and Statistics
(1987)
R.T. Carson et al.
On overcoming informational deficiencies in estimating willingness to pay distributions
C. Gourieroux et al.
Statistics and Econometric Models
(1995)

There are more references available in the full text version of this article.

Cited by (18)

Learning from inferred foregone payoffs
2015, Journal of Economic Dynamics and Control
A player׳s knowledge of her own actions and the corresponding payoffs may enable her to infer or form beliefs about what the payoffs would have been if she had played differently. For quantitative learning models employed in studies of low information environments, players׳ ex-post inferences and beliefs have been largely ignored. For games with large strategy spaces, this omission can seriously weaken the predictive power of a learning model. We propose a novel method of using players׳ ex-post inferences and assessments to impute foregone payoffs for unplayed strategies in low-information environments. We then use the resulting learning model to explain the pricing and learning behavior observed in a Bertrand market experiment. Maximum likelihood estimation shows that the extended model organizes the data remarkably well at both the aggregate and individual levels.
A drift correction optimization technique for the reduction of the inter-measurement dispersion of isotope ratios measured using a multi-collector plasma mass spectrometer
2014, Spectrochimica Acta - Part B Atomic Spectroscopy
Citation Excerpt :
The reason for this difference among the Pb ratios is not known. In principle it would be possible to determine the probability density function of S [19–22,25] and then re-derive the necessary MLE equations for a range dependent merit function. The low number of data points (a minimum of N ≥ 7, spread out, as uniformly as possible, over the run time) makes it very likely that a statistical hypothesis test of an = 0 will prove inconclusive, with the recommendation that more data points be acquired [25].
The effects of polynomial interpolation and internal standardization drift corrections on the inter-measurement dispersion (statistical) of isotope ratios measured with a multi-collector plasma mass spectrometer were investigated using the (analyte, internal standard) isotope systems of (Ni, Cu), (Cu, Ni), (Zn, Cu), (Zn, Ga), (Sm, Eu), (Hf, Re) and (Pb, Tl). The performance of five different correction factors was compared using a (statistical) range based merit function ω_m which measures the accuracy and inter-measurement range of the instrument calibration. The frequency distribution of optimal correction factors over two hundred data sets uniformly favored three particular correction factors while the remaining two correction factors accounted for a small but still significant contribution to the reduction of the inter-measurement dispersion.
Application of the merit function is demonstrated using the detection of Cu and Ni isotopic fractionation in laboratory and geologic-scale chemical reactor systems. Solvent extraction (diphenylthiocarbazone (Cu, Pb) and dimethylglyoxime (Ni)) was used to either isotopically fractionate the metal during extraction using the method of competition or to isolate the Cu and Ni from the sample (sulfides and associated silicates). In the best case, differences in isotopic composition of ± 3 in the fifth significant figure could be routinely and reliably detected for Cu65/63 and Ni61/62.
One of the internal standardization drift correction factors uses a least squares estimator to obtain a linear functional relationship between the measured analyte and internal standard isotope ratios. Graphical analysis demonstrates that the points on these graphs are defined by highly non-linear parametric curves and not two linearly correlated quantities which is the usual interpretation of these graphs. The success of this particular internal standardization correction factor was found in some cases to be due to a fortuitous, scale dependent, parametric curve effect.
A path-size weibit stochastic user equilibrium model
2013, Transportation Research Part B: Methodological
The aim of this paper is to develop a path-size weibit (PSW) route choice model with an equivalent mathematical programming (MP) formulation under the stochastic user equilibrium (SUE) principle that can account for both route overlapping and route-specific perception variance problems. Specifically, the Weibull distributed random error term handles the identically distributed assumption such that the perception variance with respect to different trip lengths can be distinguished, and a path-size factor term is introduced to resolve the route overlapping issue by adjusting the choice probabilities for routes with strong couplings with other routes. A multiplicative Beckmann’s transformation (MBec) combined with an entropy term are used to develop the MP formulation for the PSW-SUE model. A path-based algorithm based on the partial linearization method is adopted for solving the PSW-SUE model. Numerical examples are also provided to illustrate features of the PSW-SUE model and its differences compared to some existing SUE models as well as its applicability on a real-size network.
Spatially and temporally varying associations between temporary outmigration and natural resource availability in resource-dependent rural communities in South Africa: A modeling framework
2012, Applied Geography
Migration-environment models tend to be aspatial within chosen study regions, although associations between temporary outmigration and environmental explanatory variables likely vary across the study space. This research extends current approaches by developing migration models considering spatial non-stationarity and temporal variation – through examination of the migration-environment association at nested geographic scales (i.e. whole-population, village, and subvillage) within a specific study site. Demographic survey data from rural South Africa, combined with indicators of natural resource availability from satellite imagery, are employed in a nested modeling approach that brings out distinct patterns of spatial variation in model associations derived at finer geographic scales. Given recent heightened public and policy concern with the human migratory implications of climate change, we argue that consideration of spatial variability adds important nuance to scientific understanding of the migration-environment association.
Value relevance of comprehensive income and its components in front of the net income
2011, Revista de Contabilidad-Spanish Accounting Review
Este trabajo tiene como objeto verificar si el resultado global presenta mayor relevancia valorativa respecto al resultado tradicional, y si es así cuáles son los componentes del resultado global que explican las diferencias. El estudio muestra que el resultado global es una variable relevante tanto para explicar el valor intrínseco de la empresa como para explicar su rentabilidad. Sin embargo, sólo al explicar la rentabilidad el resultado global y su incremento aumentan la relevancia valorativa del resultado neto y su incremento. De los componentes analizados del resultado global, tan sólo los resultados por diferencias de conversión de la moneda extranjera a la moneda de presentación añaden poder explicativo adicional al del resultado neto, aunque sólo en los modelos de rentabilidad. Ni los resultados por valoración de instrumentos financieros a valor razonable con cambios en patrimonio ni los resultados por coberturas de flujos de efectivo aportan un mayor poder explicativo al del resultado neto.
This study analyses if comprehensive income is more value relevant than the net income and, if that is the case, what are the elements of the comprehensive income explaining the difference. Our study shows that comprehensive income is a relevant variable both to explain the value of the company and also to explain its return. However, comprehensive income and its variation show to be more relevant than the net income and its variation only when explaining return. As for the components of the comprehensive income, foreign currency adjustments increase usefulness in the return models but cash flow hedges and adjustments due to fair value valuation do not add value relevance to the net income.
Can unbiased be tighter? Assessment of methods to reduce the bias-variance trade-off in WTP estimation
2011, Resource and Energy Economics
Citation Excerpt :
Clarke (2007) demonstrated that the distribution-free test is more efficient than the Vuong test if the empirical distribution of the individual log-likelihood ratios is leptokurtic. The low power of the Vuong test was shown in an experimental setting by Genius and Strazzera (2002); therefore we applied both tests to select between the Gaussian copula with extreme value marginals and the Bivariate Normal model. The empirical distribution of the likelihood ratios is leptokurtic, and, in line with Clarke's findings, the Vuong test does not lead to a strong rejection of the null hypothesis that the two models are equivalent (chi-square = 1.62, p-value = 0.105); while the Clarke test indicates the Gaussian with extreme value marginals (B6 = 233, n = 376, p-value = 0.000) as the better model between the two.
This paper aims at verifying the claim, appeared in recent literature, that it is possible to control for response bias associated to the double bound elicitation method, while keeping gains in efficiency of the WTP estimates. Results from a Monte Carlo analysis lead, in general, to reject the claim; but when initial bids are not correctly chosen, the gains in efficiency are confirmed. An empirical application dealing with WTP estimation for drinking water quality improvements illustrates a case where a flexible modeling approach based on Copula distributions allows relevant gains with respect to the Single Bound estimator.

View all citing articles on Scopus

View full text

A note about model selection and tests for non-nested contingent valuation models

Abstract

Introduction

Section snippets

Methods

Experimental design

Results

Conclusions

Acknowledgements

Journal of Environmental Economics and Management

Journal of Econometrics

Information theory and an extension of the maximum likelihood principle

Econometric Model Selection: A New Approach

Efficient estimation methods for closed-ended contingent valuation surveys

The Review of Economics and Statistics

On overcoming informational deficiencies in estimating willingness to pay distributions

Statistics and Econometric Models