Skip to main content
SpringerLink
Log in

Bayesian inference for finite mixtures of generalized linear models with random effects

  • Published:
Psychometrika Aims and scope Submit manuscript

Abstract

We present an hierarchical Bayes approach to modeling parameter heterogeneity in generalized linear models. The model assumes that there are relevant subpopulations and that within each subpopulation the individual-level regression coefficients have a multivariate normal distribution. However, class membership is not known a priori, so the heterogeneity in the regression coefficients becomes a finite mixture of normal distributions. This approach combines the flexibility of semiparametric, latent class models that assume common parameters for each sub-population and the parsimony of random effects models that assume normal distributions for the regression parameters. The number of subpopulations is selected to maximize the posterior probability of the model being true. Simulations are presented which document the performance of the methodology for synthetic data with known heterogeneity and number of sub-populations. An application is presented concerning preferences for various aspects of personal computers.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Akaike, H. (1973). Information theory and an extension of the maximum likelihood principle. In N. Petrov & F. Csadki (Eds.),Proceedings of the Second International Symposium of Information Theory (pp. 267–281). Budapest: Akademiai Kiado.

    Google Scholar 

  • Albert, J. H., & Chib, S. (1993). Bayesian analysis of binary and polychotomous response data.Journal of American Statistical Association, 88, 669–679.

    Google Scholar 

  • Allenby, G. M., Arora, N., & Ginter, J. L. (1998). On the Heterogeneity of Demand.Journal of Marketing Research, 37(November), 384–389.

    Google Scholar 

  • Bozdogan, H. (1987). Model selection and Akaike's information criterion (AIC): The general theory and its analytical extension.Psychometrika, 52, 345–370.

    Google Scholar 

  • Breslow, N. E., & Clayton, D. G. (1993). Approximate inference in generalized linear mixed models.Journal of American Statistical Association, 88, 8–25.

    Google Scholar 

  • Carlin, B. P., & Chib, S. (1995). Bayesian model choice via Markov chain Monte Carlo methods.Journal of the Royal Statistical Society, Series B, 57, 473–484.

    Google Scholar 

  • Chib, S. (1995). Marginal likelihood from the Gibbs output.Journal of American Statistical Association, 90, 1313–1321.

    Google Scholar 

  • Chib, S., & Greenberg, E. (1995). Understanding the Metropolis-Hastings algorithm.The American Statistician, 49, 327–335.

    Google Scholar 

  • Damien, P., Wakefield, J. C., & Walker, S. (1999). Gibbs sampling for Bayesian nonconjugate and hierarchical models using auxiliary variables.Journal of the Royal Statistical Society, Series B, 61, 331–334.

    Google Scholar 

  • Diebolt, J., & Robert, C. P. (1994). Estimation of finite mixture distributions through Bayesian sampling.Journal of the Royal Statistical Society, Series B, 56, 362–375.

    Google Scholar 

  • Dempster, A. P., Laird, N. M., & Rubin, R. B. (1977). Maximum likelihood from incomplete data via the EM-algorithm.Journal of the Royal Statistical Society, Series B, 39, 1–38.

    Google Scholar 

  • DeSarbo, W. S., & Cron W. L. (1988). A maximum likelihood methodology for clusterwise linear regression.Journal of Classification, 5, 248–282.

    Google Scholar 

  • DeSarbo, W. S., Wedel, M., Vriens, M., & Ramaswamy, V. (1992). Latent class metric conjoint analysis.Marketing Letters, 3, 273–288.

    Google Scholar 

  • DeSarbo, W. S., Ramaswamy, V., Reibstein, D. J., & Robinson, W. T. (1993). A latent pooling methodology for regression analysis with limited time series of cross sections: A PIMS data application.Marketing Science, 12, 103–124.

    Google Scholar 

  • De Soete, G., & DeSarbo, W. S. (1991). A latent class probit model for analyzing pick any/n data.Journal of Classification, 8, 45–63.

    Google Scholar 

  • Everitt, B. S., & Hand, D. J. (1981).Finite mixture distributions. London: Chapman and Hall.

    Google Scholar 

  • Gelfand, A. E., & Dey, D. K. (1994). Bayesian model choice: asymptotics and exact calculations.Journal of the Royal Statistical Society, Series B,56, 501–514.

    Google Scholar 

  • Gelfand, A. E., & Smith, A. F. M. (1990). Sampling-based approaches to calculating marginal densities.Journal of American Statistical Association, 85, 398–409.

    Google Scholar 

  • Gelman, A., Roberts, G. O., & Gilks, W. R. (1996). Efficient metropolis jumping rules. In J. M. Bernardo, J. O. Berger, A. P. Dawid, & A. M. F. Smith (Eds.),Bayesian statistics 5 (pp. 599–607). Cambridge: Oxford University Press.

    Google Scholar 

  • Gelman, A., & Rubin, D. B. (1992). Iterative simulation using single and multiple sequences.Statistical Science, 7, 457–511.

    Google Scholar 

  • Geyer, C. J. (1992). Practical Markov chain Monte Carlo.Statistical Science, 7, 473–511.

    Google Scholar 

  • Goodman, L. A. (1974). Exploratory latent structure analysis using both identified and unidentified models.Biometrika, 61, 215–231.

    Google Scholar 

  • Hastings, W. K (1970). Monte Carlo sampling methods using Markov chains and their applications.Biometrika, 57, 97–109.

    Google Scholar 

  • Hosmer, D. W. (1974). Maximum likelihood estimates of the parameters of a mixture of two regression lines.Communications in Statistics, Series B,3, 995–1006.

    Google Scholar 

  • Jeffreys, H. (1961).Theory of probability (3rd. ed.). Cambridge: Oxford University Press.

    Google Scholar 

  • Jones, P. N., & McLachlan, G. J. (1992). Fitting finite mixture models in a regression context.Australian Journal of Statistics, 43, 233–240.

    Google Scholar 

  • Kamakura, W. A. (1991). Estimating flexible distributions of ideal-points with external analysis of preference.Psychometrika, 56, 419–448.

    Google Scholar 

  • Kamakura, W. A., & Russell, G. J. (1989). A probabilistic choice model for market segmentation and elasticity structure.Journal of Marketing Research, 26, 379–390.

    Google Scholar 

  • Kass, R. E., & Raftery, A. E. (1995). Bayes factors.Journal of American Statistical Association, 90, 773–795.

    Google Scholar 

  • Lenk, P. J., DeSarbo, W. S., Green, P. E., & Young, M. R. (1996). Hierarchical Bayes conjoint analysis: Recovery of partworth heterogeneity from reduced experimental designs.Marketing Science, 15(2), 173–191.

    Google Scholar 

  • Lewis, S. M. & Raftery, A. E. (1997). Estimating Bayes factors via posterior simulation with the Laplace—Metropolis estimator.Journal of American Statistical Association, 92, 648–655.

    Google Scholar 

  • Luce, R. D. (1959).Individual choice behavior: A theoretical analysis. New York: John Wiley & Sons.

    Google Scholar 

  • Lwin, T., & Martin, P. J. (1989). Probits of mixtures.Biometrics, 45, 721–732.

    Google Scholar 

  • McCullagh, P., & Nelder, J. A. (1983).Generalized linear models. New York: Chapman and Hall.

    Google Scholar 

  • McFadden, D. (1974). Conditional logit analysis of qualitative choice behavior. In P. Zarembda (Ed.),Frontiers in econometrics (pp. 105–142). New York: Academic Press.

    Google Scholar 

  • Newcomb, S. (1886). A generalized theory of the combination of observations so as to obtain the best result.American Journal of Mathematics, Series B,8, 343–366.

    Google Scholar 

  • Pearson, K. (1894). Contribution to the mathematical theory of evolution.Philosophical Transactions, Series A,185, 71–110.

    Google Scholar 

  • Polson, Nicholas G. (1996). Convergence of Markov chain Monte Carlo algorithms. In J. M. Bernardo, J. O. Berger, A. P. Dawid, & A. M. F. Smith (Eds.),Bayesian statistics 5 (pp. 297–312). Cambridge: Oxford University Press.

    Google Scholar 

  • Quandt, R. E. (1972). A new approach to estimating switching regressions.Journal of American Statistical Association, 67, 306–310.

    Google Scholar 

  • Quandt, R. E., & Ramsey, J. B. (1978). Estimating mixtures of normal distributions and switching regression.Journal of American Statistical Association, 73, 730–738.

    Google Scholar 

  • Roberts, G. O., & Polson, N. G. (1994). On the geometric convergence of the Gibbs sampler.Journal of the Royal Statistical Society, Series B,56, 377–384.

    Google Scholar 

  • Seber, G. A. F. (1984).Multivariate observations. New York: John Wiley & Sons.

    Google Scholar 

  • Schwarz, G. (1978). Estimating the dimension of a model.Annals of Statistics, 6, 461–464.

    Google Scholar 

  • Smith, A. F. M., & Roberts, G. O. (1993). Bayesian computation via the Gibbs sampler and related Markov chain Monte Carlo methods.Journal of the Royal Statistical Society, Series B,55, 3–23.

    Google Scholar 

  • Tanner, M. A. (1993).Tools for statistical inference (Lecture Notes in Statistics 67). New York: Springer-Verlag.

    Google Scholar 

  • Titterington, D. M., Smith, A. F. M., & Makov, U. E. (1985).Statistical analysis of finite mixture distributions. New York: John Wiley & Sons.

    Google Scholar 

  • Verdinelli, I., & Wasserman, L. (1995). Computing Bayes factors using a generalization of the Savage-Dickey density ratio.Journal of American Statistical Association, 90, 614–618.

    Google Scholar 

  • Wang, P. M., Cockburn, I. M., & Puterman, M. L. (1998). ”Analysis of Patent Data—A Mixed Poisson Regression Model.Journal of Business and Economic Statistics, 16, 27–41.

    Google Scholar 

  • Wang, P. M., & Puterman, M. L. (in press). Mixed logistic regression models.Journal of Agricultural, Biological, and Environmental Statistics.

  • Wang, P. M., Puterman, M. L., Le, N., & Cockburn, I. (1996). Mixed Poisson regression with covariate dependent rates.Biometrics, 52, 381–400.

    Google Scholar 

  • Wedel, M., & DeSarbo, W. S. (1993). A latent class binomial logit methodology for the analysis of paired comparison data: An application reinvestigating the determinants of perceived risk.Decision Science, 24, 1157–1170.

    Google Scholar 

  • Wedel, M., & DeSarbo, W. S. (1994). A Review of recent developments in latent structure regression models. In R. Bagozzi (Ed.),Advanced methods of marketing research (pp. 352–388), London: Blackwell Publishing.

    Google Scholar 

  • Wedel, M., & DeSarbo, W. S. (1995). A mixture likelihood approach for generalized linear models.Journal of Classification, 12, 21–55.

    Google Scholar 

  • Wedel, M., DeSarbo, W. S., Bult, J. R., & Ramaswamy, V. (1993). A latent class Poisson regression model for heterogeneous count data with an application to direct mail.Journal of Applied Econometrics, 8, 397–411.

    Google Scholar 

  • Zeger, S. L., & Karim, M. R. (1991). Generalized linear models with random effects: A Gibbs sampling approach.Journal of American Statistical Association, 86, 79–679.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Michigan Business School, 701 Tappan Street, 48109-1234, Ann Arbor, MI

    Peter J. Lenk

  2. Pennsylvania State University, USA

    Wayne S. DeSarbo

Authors
  1. Peter J. Lenk
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wayne S. DeSarbo
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Lenk, P.J., DeSarbo, W.S. Bayesian inference for finite mixtures of generalized linear models with random effects. Psychometrika 65, 93–119 (2000). https://doi.org/10.1007/BF02294188

Download citation

  • Received:

  • Revised:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF02294188

Key words

Search

Navigation