Skip to main content
Top

2024 | OriginalPaper | Chapter

Fitting Big Data Using Structural Measurement Error Model

Authors : Saja Alzubai, Amjad D. Al-Nasser

Published in: Mathematical Analysis and Numerical Methods

Publisher: Springer Nature Singapore

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

The chapter delves into the challenges of analyzing big data, particularly the volume and complexity that exceed conventional database systems. It introduces the structural measurement error model (SMEM) and the subsampling algorithm as a solution to efficiently process big data. The author demonstrates how the sub-sampling algorithm can reduce computational time and improve estimator accuracy through hypothetical examples and Monte Carlo experiments. The chapter also highlights the robustness of the method under various data conditions, including the presence of outliers and non-normal distributions. By comparing AIC and BIC values, the author shows that the sub-sampling approach provides nearly identical statistical inferences to using the full dataset, but with significantly improved computational efficiency.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
1.
2.
go back to reference Al-Nasser, A.D.: Entropy type estimator to simple linear measurement error models. Austrian J. Stat. 34(3), 283–294 (2005)CrossRef Al-Nasser, A.D.: Entropy type estimator to simple linear measurement error models. Austrian J. Stat. 34(3), 283–294 (2005)CrossRef
3.
go back to reference Al-Nasser, A.D.: An information-theoretic approach to the measurement error model. Stat. Transit.—New Ser. 11(1), 9–24 (2010) Al-Nasser, A.D.: An information-theoretic approach to the measurement error model. Stat. Transit.—New Ser. 11(1), 9–24 (2010)
4.
go back to reference Cavanaugh, J.E., Neath, A.A.: The Akaike information criterion: Background, derivation, properties, application, interpretation, and refinements. Wiley Interdiscipl. Rev.: Comput. Statist. 11(3), e1460 (2019)MathSciNetCrossRef Cavanaugh, J.E., Neath, A.A.: The Akaike information criterion: Background, derivation, properties, application, interpretation, and refinements. Wiley Interdiscipl. Rev.: Comput. Statist. 11(3), e1460 (2019)MathSciNetCrossRef
5.
go back to reference Cheng, C.-L., Van Ness, J.W.: Statistical Regression with Measurement Error. Arnold, London (1999) Cheng, C.-L., Van Ness, J.W.: Statistical Regression with Measurement Error. Arnold, London (1999)
6.
go back to reference Dumbill, E.: Planning for Big Data. Sebastopoli, Sonoma County, CA, USA (2012) Dumbill, E.: Planning for Big Data. Sebastopoli, Sonoma County, CA, USA (2012)
8.
go back to reference Erl, T., Khattak, W., Buhler, P.: Big Data Fundamentals: Concepts, Drivers & Techniques, 186–194. Prentice Hall Press (2016) Erl, T., Khattak, W., Buhler, P.: Big Data Fundamentals: Concepts, Drivers & Techniques, 186–194. Prentice Hall Press (2016)
9.
go back to reference Fuller, W.A.: Measurement Error Models. John Wiley, New York (1987) Fuller, W.A.: Measurement Error Models. John Wiley, New York (1987)
10.
go back to reference Hurwitz, J., Nugent, A., Halper, F., Kaufman, M.: Big Data for Dummies, Vol. 336. John Wiley & Sons, Hoboken, NJ (2013) Hurwitz, J., Nugent, A., Halper, F., Kaufman, M.: Big Data for Dummies, Vol. 336. John Wiley & Sons, Hoboken, NJ (2013)
12.
go back to reference Wang, H.: More efficient estimation for logistic regression with optimal subsamples. J. Mach. Learn. Res. 20(132), 1–59 (2019)MathSciNet Wang, H.: More efficient estimation for logistic regression with optimal subsamples. J. Mach. Learn. Res. 20(132), 1–59 (2019)MathSciNet
13.
go back to reference Wang, H., Yang, M., Stufken, J.: Information-based optimal sub-data selection for big data linear regression. J. Am. Stat. Assoc. 114(525), 393–405 (2019)CrossRef Wang, H., Yang, M., Stufken, J.: Information-based optimal sub-data selection for big data linear regression. J. Am. Stat. Assoc. 114(525), 393–405 (2019)CrossRef
Metadata
Title
Fitting Big Data Using Structural Measurement Error Model
Authors
Saja Alzubai
Amjad D. Al-Nasser
Copyright Year
2024
Publisher
Springer Nature Singapore
DOI
https://doi.org/10.1007/978-981-97-4876-1_30

Premium Partner