Skip to main content
Erschienen in: Quality & Quantity 5/2023

29.10.2022

Simultaneous raise regression: a novel approach to combating collinearity in linear regression models

verfasst von: Jinse Jacob, R. Varadharajan

Erschienen in: Quality & Quantity | Ausgabe 5/2023

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

The effect of multicollinearity deems the estimation and interpretation of regression coefficients by the method of Ordinary Least Squares (OLS) as burdensome. Even if there are alternate techniques like Ridge Regression (RR) and Liu Regression (LR) for providing a model, the statistical inference is impaired. In this study, we present the Simultaneous Raise Regression (SRR) approach, which is based on QR decomposition and reduces multicollinearity while preserving statistical inference. In contrast to the classical raise approach, the proposed strategy attains the number of variables to be raised and their corresponding raise parameter with a single step. We have designed Sequential Variation Inflation Factor for this purpose (SVIF). The performance of SRR has been compared with OLS, Ridge Regression and Liu Regression through Predictive Root Mean Square Error (PRMSE), Predictive Mean Absolute Error (PMAE) and Standard Error of the coefficients. To validate the suggested approach, a simulation study and a real-life example are presented; both outcomes imply that our proposed method outperforms the competition.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
Zurück zum Zitat Belsley, D.A., Kuh, E., Welsch, R.E.: The condition number. Regression diagnostics: Identifying influential data and sources of collinearity 100, 104 (1980) Belsley, D.A., Kuh, E., Welsch, R.E.: The condition number. Regression diagnostics: Identifying influential data and sources of collinearity 100, 104 (1980)
Zurück zum Zitat Belsley, D.A.: A guide to using the collinearity diagnostics. Comput. Sci. Econ. Manag. 4(1), 33–50 (1991)CrossRef Belsley, D.A.: A guide to using the collinearity diagnostics. Comput. Sci. Econ. Manag. 4(1), 33–50 (1991)CrossRef
Zurück zum Zitat Chatterjee, S., Price, B.: Regression Diagnostics. New York (1991) Chatterjee, S., Price, B.: Regression Diagnostics. New York (1991)
Zurück zum Zitat Cule, E., Moritz, S., Frankowski, D.: ridge: ridge regression with automatic selection of the penalty parameter. R Package Version 3, 3 (2022) Cule, E., Moritz, S., Frankowski, D.: ridge: ridge regression with automatic selection of the penalty parameter. R Package Version 3, 3 (2022)
Zurück zum Zitat Druilhet, P., Mom, A.: Shrinkage structure in biased regression. J. Multivar. Anal. 99(2), 232–244 (2008)CrossRef Druilhet, P., Mom, A.: Shrinkage structure in biased regression. J. Multivar. Anal. 99(2), 232–244 (2008)CrossRef
Zurück zum Zitat Efron, B., Hastie, T., Johnstone, I., Tibshirani, R.: Least angle regression. Ann. Stat. 32(2), 407–499 (2004)CrossRef Efron, B., Hastie, T., Johnstone, I., Tibshirani, R.: Least angle regression. Ann. Stat. 32(2), 407–499 (2004)CrossRef
Zurück zum Zitat Frisch, R.: Statistical confluence analysis by means of complete regression systems, volume 5. Universitetets Økonomiske Instituut (1934) Frisch, R.: Statistical confluence analysis by means of complete regression systems, volume 5. Universitetets Økonomiske Instituut (1934)
Zurück zum Zitat Garcia, C.G., Pérez, J.G., Liria, J.S.: The raise method. An alternative procedure to estimate the parameters in presence of collinearity. Qual. Quant. 45(2), 403–423 (2011)CrossRef Garcia, C.G., Pérez, J.G., Liria, J.S.: The raise method. An alternative procedure to estimate the parameters in presence of collinearity. Qual. Quant. 45(2), 403–423 (2011)CrossRef
Zurück zum Zitat García, C.B., Salmerón, R., García, C., García, J.: Residualization: justification, properties and application. J. Appl. Stat. 47(11), 1990–2010 (2020)CrossRef García, C.B., Salmerón, R., García, C., García, J.: Residualization: justification, properties and application. J. Appl. Stat. 47(11), 1990–2010 (2020)CrossRef
Zurück zum Zitat Giacalone, M., Panarello, D., Mattera, R.: Multicollinearity in regression: an efficiency comparison between LP-norm and least squares estimators. Qual. Quant. 52(4), 1831–1859 (2018)CrossRef Giacalone, M., Panarello, D., Mattera, R.: Multicollinearity in regression: an efficiency comparison between LP-norm and least squares estimators. Qual. Quant. 52(4), 1831–1859 (2018)CrossRef
Zurück zum Zitat Gómez, R.S., Sánchez, A.R., García, C.G., Pérez, J.G., et al.: The VIF and MSE in raise regression. Mathematics 8(4), 1–28 (2020) Gómez, R.S., Sánchez, A.R., García, C.G., Pérez, J.G., et al.: The VIF and MSE in raise regression. Mathematics 8(4), 1–28 (2020)
Zurück zum Zitat Halawa, A., El Bassiouni, M.: Tests of regression coefficients under ridge regression models. J. Stat. Comput. Simul. 65(1–4), 341–356 (2000)CrossRef Halawa, A., El Bassiouni, M.: Tests of regression coefficients under ridge regression models. J. Stat. Comput. Simul. 65(1–4), 341–356 (2000)CrossRef
Zurück zum Zitat Hoerl, A.E., Kennard, R.W.: Ridge regression: applications to nonorthogonal problems. Technometrics 12(1), 69–82 (1970)CrossRef Hoerl, A.E., Kennard, R.W.: Ridge regression: applications to nonorthogonal problems. Technometrics 12(1), 69–82 (1970)CrossRef
Zurück zum Zitat Hoerl, A.E., Kennard, R.W.: Ridge regression: biased estimation for nonorthogonal problems. Technometrics 12(1), 55–67 (1970)CrossRef Hoerl, A.E., Kennard, R.W.: Ridge regression: biased estimation for nonorthogonal problems. Technometrics 12(1), 55–67 (1970)CrossRef
Zurück zum Zitat Imdad, M.U., Aslam, M.: liureg: Liu regression with Liu biasing parameters and statistics. R Package Version 1(1), 2 (2018) Imdad, M.U., Aslam, M.: liureg: Liu regression with Liu biasing parameters and statistics. R Package Version 1(1), 2 (2018)
Zurück zum Zitat Imdadullah, M., Aslam, M.: liureg: A comprehensive r package for the Liu estimation of linear regression model with collinear regressors. R J. 9(2), 232–247 (2017)CrossRef Imdadullah, M., Aslam, M.: liureg: A comprehensive r package for the Liu estimation of linear regression model with collinear regressors. R J. 9(2), 232–247 (2017)CrossRef
Zurück zum Zitat Kejian, L.: A new class of blased estimate in linear regression. Commun. Stat. Theory Methods 22(2), 393–402 (1993)CrossRef Kejian, L.: A new class of blased estimate in linear regression. Commun. Stat. Theory Methods 22(2), 393–402 (1993)CrossRef
Zurück zum Zitat Kendall, M.G.: A Course in Multivariate Analysis: London. Charles Griffin & Co, London (1957) Kendall, M.G.: A Course in Multivariate Analysis: London. Charles Griffin & Co, London (1957)
Zurück zum Zitat Klein, L.R., Nakamura, M.: Singularity in the equation systems of econometrics: some aspects of the problem of multicollinearity. Int. Econ. Rev. 3(3), 274–299 (1962)CrossRef Klein, L.R., Nakamura, M.: Singularity in the equation systems of econometrics: some aspects of the problem of multicollinearity. Int. Econ. Rev. 3(3), 274–299 (1962)CrossRef
Zurück zum Zitat Kovács, P., Petres, T., Tóth, L.: A new measure of multicollinearity in linear regression models. Int. Stat. Rev. 73(3), 405–412 (2005)CrossRef Kovács, P., Petres, T., Tóth, L.: A new measure of multicollinearity in linear regression models. Int. Stat. Rev. 73(3), 405–412 (2005)CrossRef
Zurück zum Zitat Kutner, M.H., Nachtsheim, C.J., Neter, J., Wasserman, W.: Applied linear regression models, volume 4. McGraw-Hill/Irwin New York (2004) Kutner, M.H., Nachtsheim, C.J., Neter, J., Wasserman, W.: Applied linear regression models, volume 4. McGraw-Hill/Irwin New York (2004)
Zurück zum Zitat Lin, F.-J.: Solving multicollinearity in the process of fitting regression model using the nested estimate procedure. Qual. Quant. 42(3), 417–426 (2008)CrossRef Lin, F.-J.: Solving multicollinearity in the process of fitting regression model using the nested estimate procedure. Qual. Quant. 42(3), 417–426 (2008)CrossRef
Zurück zum Zitat Liu, X.-Q.: Improved liu estimator in a linear regression model. J. Stat. Plann. Inference 141(1), 189–196 (2011)CrossRef Liu, X.-Q.: Improved liu estimator in a linear regression model. J. Stat. Plann. Inference 141(1), 189–196 (2011)CrossRef
Zurück zum Zitat Marquardt, D.W.: Comment: You should standardize the predictor variables in your regression models. J. Am. Stat. Assoc. 75(369), 87–91 (1980) Marquardt, D.W.: Comment: You should standardize the predictor variables in your regression models. J. Am. Stat. Assoc. 75(369), 87–91 (1980)
Zurück zum Zitat Marquardt, D.W., Snee, R.D.: Ridge regression in practice. Am. Stat. 29(1), 3–20 (1975) Marquardt, D.W., Snee, R.D.: Ridge regression in practice. Am. Stat. 29(1), 3–20 (1975)
Zurück zum Zitat Midi, H., Bagheri, A.: Robust multicollinearity diagnostic measure in collinear data set. In: Proceedings of the 4th International Conference on Applied Mathematics, Simulation, Modeling (pp. 138–142).: World Scientific and Engineering Academy and Society (WSEAS) (2010) Midi, H., Bagheri, A.: Robust multicollinearity diagnostic measure in collinear data set. In: Proceedings of the 4th International Conference on Applied Mathematics, Simulation, Modeling (pp. 138–142).: World Scientific and Engineering Academy and Society (WSEAS) (2010)
Zurück zum Zitat Midi, H., Bagheri, A.: Robust multicollinearity diagnostic measures based on minimum covariance determination approach. Econ. Comput. Econ. Cybern. Stud. Res. 47(4), 1–15 (2013) Midi, H., Bagheri, A.: Robust multicollinearity diagnostic measures based on minimum covariance determination approach. Econ. Comput. Econ. Cybern. Stud. Res. 47(4), 1–15 (2013)
Zurück zum Zitat Obite, C., Olewuezi, N., Ugwuanyim, G., Bartholomew, D.: Multicollinearity effect in regression analysis: a feed forward artificial neural network approach. Asian J. Probab. Stat. 6(1), 22–33 (2020)CrossRef Obite, C., Olewuezi, N., Ugwuanyim, G., Bartholomew, D.: Multicollinearity effect in regression analysis: a feed forward artificial neural network approach. Asian J. Probab. Stat. 6(1), 22–33 (2020)CrossRef
Zurück zum Zitat Özkale, M.R., Kaçıranlar, S.: A prediction-oriented criterion for choosing the biasing parameter in Liu estimation. Commun. Stat. Theory Methods 36(10), 1889–1903 (2007)CrossRef Özkale, M.R., Kaçıranlar, S.: A prediction-oriented criterion for choosing the biasing parameter in Liu estimation. Commun. Stat. Theory Methods 36(10), 1889–1903 (2007)CrossRef
Zurück zum Zitat Perez-Melo, S., Kibria, B.: On some test statistics for testing the regression coefficients in presence of multicollinearity: a simulation study. Stats 3(1), 40–55 (2020)CrossRef Perez-Melo, S., Kibria, B.: On some test statistics for testing the regression coefficients in presence of multicollinearity: a simulation study. Stats 3(1), 40–55 (2020)CrossRef
Zurück zum Zitat Salmeron, R., Garcia, C., Garcia, J., Lopez, M.D.: The raise estimator estimation, inference, and properties. Commun. Stat. Theory Methods 46(13), 6446–6462 (2017) Salmeron, R., Garcia, C., Garcia, J., Lopez, M.D.: The raise estimator estimation, inference, and properties. Commun. Stat. Theory Methods 46(13), 6446–6462 (2017)
Zurück zum Zitat Salmerón, R., García, C., García, J.: Overcoming the inconsistences of the variance inflation factor: a redefined VIF and a test to detect statistical troubling multicollinearity. Econometrica (2020) Salmerón, R., García, C., García, J.: Overcoming the inconsistences of the variance inflation factor: a redefined VIF and a test to detect statistical troubling multicollinearity. Econometrica (2020)
Zurück zum Zitat Saville, D.J., Wood, G.R.: Statistical Methods: The Geometric Approach. Springer, Berlin (2012) Saville, D.J., Wood, G.R.: Statistical Methods: The Geometric Approach. Springer, Berlin (2012)
Zurück zum Zitat Sharma, S.: Applied Multivariate Techniques. John Wiley & Sons Inc, Hoboken (1996) Sharma, S.: Applied Multivariate Techniques. John Wiley & Sons Inc, Hoboken (1996)
Zurück zum Zitat Stewart, G.W.: Collinearity and least squares regression. Stat. Sci. 2(1), 68–84 (1987) Stewart, G.W.: Collinearity and least squares regression. Stat. Sci. 2(1), 68–84 (1987)
Zurück zum Zitat Stine, R.A.: Graphical interpretation of variance inflation factors. Am. Stat. 49(1), 53–56 (1995) Stine, R.A.: Graphical interpretation of variance inflation factors. Am. Stat. 49(1), 53–56 (1995)
Zurück zum Zitat Strang, G.: Introduction to linear algebra. 5th edn Cambridge. United Kingdom: Wellesley-Cambridge Press (2016) Strang, G.: Introduction to linear algebra. 5th edn Cambridge. United Kingdom: Wellesley-Cambridge Press (2016)
Zurück zum Zitat Venables, W.N., Ripley, B.D.: Modern Applied Statistics with S. New York: Springer, fourth edition (2002) Venables, W.N., Ripley, B.D.: Modern Applied Statistics with S. New York: Springer, fourth edition (2002)
Zurück zum Zitat Watkins, D.S.: Fundamentals of matrix computations. Math. Comput. 59, 299 (1992)CrossRef Watkins, D.S.: Fundamentals of matrix computations. Math. Comput. 59, 299 (1992)CrossRef
Metadaten
Titel
Simultaneous raise regression: a novel approach to combating collinearity in linear regression models
verfasst von
Jinse Jacob
R. Varadharajan
Publikationsdatum
29.10.2022
Verlag
Springer Netherlands
Erschienen in
Quality & Quantity / Ausgabe 5/2023
Print ISSN: 0033-5177
Elektronische ISSN: 1573-7845
DOI
https://doi.org/10.1007/s11135-022-01557-9

Weitere Artikel der Ausgabe 5/2023

Quality & Quantity 5/2023 Zur Ausgabe

Premium Partner