Skip to main content

2019 | OriginalPaper | Buchkapitel

12. Analysis of Big Data Using GLM

verfasst von : Md. Rezaul Karim, M. Ataharul Islam

Erschienen in: Reliability and Survival Analysis

Verlag: Springer Singapore

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

The application of the generalized linear models to big data is discussed in this chapter using the divide and recombine (D&R) framework. In this chapter, the exponential family of distributions for binary, count, normal, and multinomial outcome variables and the corresponding sufficient statistics for parameters are shown to have great potential in analyzing big data where traditional statistical methods cannot be used for the entire data set.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Literatur
Zurück zum Zitat Buhlmann P, Petros D, Michael K, van der Mark L (2016) Handbook of big data. Routledge, LondonCrossRef Buhlmann P, Petros D, Michael K, van der Mark L (2016) Handbook of big data. Routledge, LondonCrossRef
Zurück zum Zitat Chen Y, Dong G, Han J, Pei J, Wah BW, Wang J (2006) Regression cubes with lossless compression and aggregation. IEEE Trans Knowl Data Eng 18:1–15CrossRef Chen Y, Dong G, Han J, Pei J, Wah BW, Wang J (2006) Regression cubes with lossless compression and aggregation. IEEE Trans Knowl Data Eng 18:1–15CrossRef
Zurück zum Zitat Chen X, Xie M (2014) A split-and-conquer approach for analysis of extraordinarily large data. Stat Sinica 24:1655–1684MathSciNetMATH Chen X, Xie M (2014) A split-and-conquer approach for analysis of extraordinarily large data. Stat Sinica 24:1655–1684MathSciNetMATH
Zurück zum Zitat Cleveland S, Hafen R (2014) Divide and recombine (D&R): data science for large complex data. Stat Anal Data Min 7:425–433MathSciNetCrossRef Cleveland S, Hafen R (2014) Divide and recombine (D&R): data science for large complex data. Stat Anal Data Min 7:425–433MathSciNetCrossRef
Zurück zum Zitat Dobson AJ, Barnett AG (2018) An introduction to generalized linear models, 4th edn. CRC Press, Boca RatonMATH Dobson AJ, Barnett AG (2018) An introduction to generalized linear models, 4th edn. CRC Press, Boca RatonMATH
Zurück zum Zitat Donoho D (2015) 50 Years of data science. Presentation at the Tukey Centennial Workshop, Princeton, New Jersey, Sep 2015 Donoho D (2015) 50 Years of data science. Presentation at the Tukey Centennial Workshop, Princeton, New Jersey, Sep 2015
Zurück zum Zitat Einav L, Levin J (2014) Economics in the age of big data. Science 346:1243089-1, -5CrossRef Einav L, Levin J (2014) Economics in the age of big data. Science 346:1243089-1, -5CrossRef
Zurück zum Zitat Fahrmeir L, Tutz G (2001) Multivariate statistical modelling based on generalized linear models, 2nd edn. Springer, New YorkCrossRef Fahrmeir L, Tutz G (2001) Multivariate statistical modelling based on generalized linear models, 2nd edn. Springer, New YorkCrossRef
Zurück zum Zitat Fisher RA (1920) A mathematical examination of the method of determining the accuracy of an observation by the mean error and by the mean square error, M.N.R. Astron Soc 80(8):758–770CrossRef Fisher RA (1920) A mathematical examination of the method of determining the accuracy of an observation by the mean error and by the mean square error, M.N.R. Astron Soc 80(8):758–770CrossRef
Zurück zum Zitat Fisher RA (1922) On the mathematical foundations of theoretical statistics. Philos Trans R Soc Lond A 222:309–368CrossRef Fisher RA (1922) On the mathematical foundations of theoretical statistics. Philos Trans R Soc Lond A 222:309–368CrossRef
Zurück zum Zitat Fisher RA (1925) Theory of statistical estimation. Proc Camb Philos Soc 22:700–725CrossRef Fisher RA (1925) Theory of statistical estimation. Proc Camb Philos Soc 22:700–725CrossRef
Zurück zum Zitat Guha S, Hafen R, Rounds J, Xia J, Li J, Xi B, Cleveland WS (2012) Large complex data: divide and recombine (D&R) with RHIPE. Stat 1(1):53–67CrossRef Guha S, Hafen R, Rounds J, Xia J, Li J, Xi B, Cleveland WS (2012) Large complex data: divide and recombine (D&R) with RHIPE. Stat 1(1):53–67CrossRef
Zurück zum Zitat Hafen R (2016) Divide and recombine: approach for detailed analysis and visualization of large complex data. Handbook of big data. Chapman and Hall, Boca Raton Hafen R (2016) Divide and recombine: approach for detailed analysis and visualization of large complex data. Handbook of big data. Chapman and Hall, Boca Raton
Zurück zum Zitat Halmos PR, Savage LJ (1949) Application of the radon-nikodym theorem to the theory of sufficient statistics. Ann Math Stat 20:225–241MathSciNetCrossRef Halmos PR, Savage LJ (1949) Application of the radon-nikodym theorem to the theory of sufficient statistics. Ann Math Stat 20:225–241MathSciNetCrossRef
Zurück zum Zitat Härdle WK, Lu HHS, Shen X (eds) (2018) Handbook of big data analytics. Springer Härdle WK, Lu HHS, Shen X (eds) (2018) Handbook of big data analytics. Springer
Zurück zum Zitat Lee JYL, Brown JJ, Ryan MM (2017) Sufficiency revisited: rethinking statistical algorithms in the big data era. Am Stat 71(3):202–208MathSciNetCrossRef Lee JYL, Brown JJ, Ryan MM (2017) Sufficiency revisited: rethinking statistical algorithms in the big data era. Am Stat 71(3):202–208MathSciNetCrossRef
Zurück zum Zitat Lehmann EL (1959) Theory of hypothesis testing. Wiley, New York Lehmann EL (1959) Theory of hypothesis testing. Wiley, New York
Zurück zum Zitat Liu W, Li Y (2018) A new stochastic restricted Liu estimator for the logistic regression model. Open J Stat 8:25–37CrossRef Liu W, Li Y (2018) A new stochastic restricted Liu estimator for the logistic regression model. Open J Stat 8:25–37CrossRef
Zurück zum Zitat Pitman EJG (1936) Sufficient statistics and intrinsic accuracy. Proc Camb Philos Soc 32:567–579CrossRef Pitman EJG (1936) Sufficient statistics and intrinsic accuracy. Proc Camb Philos Soc 32:567–579CrossRef
Zurück zum Zitat Xi R, Lin N, Chen Y (2008) Compression and aggregation for logistic regression analysis in data cubes. IEEE Trans Knowl Data Eng 1(1):1–14 Xi R, Lin N, Chen Y (2008) Compression and aggregation for logistic regression analysis in data cubes. IEEE Trans Knowl Data Eng 1(1):1–14
Zurück zum Zitat Zomaya AY, Sakr S (eds) (2017) Handbook of big data technologies. Springer Zomaya AY, Sakr S (eds) (2017) Handbook of big data technologies. Springer
Zurück zum Zitat ZuoW Li Y (2018) A new stochastic restricted Liu estimator for the logistic regression model. Open J Stat 8:25–37CrossRef ZuoW Li Y (2018) A new stochastic restricted Liu estimator for the logistic regression model. Open J Stat 8:25–37CrossRef
Metadaten
Titel
Analysis of Big Data Using GLM
verfasst von
Md. Rezaul Karim
M. Ataharul Islam
Copyright-Jahr
2019
Verlag
Springer Singapore
DOI
https://doi.org/10.1007/978-981-13-9776-9_12