Skip to main content
Top

2020 | OriginalPaper | Chapter

Algorithm Selection and Model Evaluation in Application Design Using Machine Learning

Authors : Srikanth Bethu, B. Sankara Babu, K. Madhavi, P. Gopala Krishna

Published in: Machine Learning for Networking

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

AI has turned into a focal piece of our life – as buyers, clients, and, ideally, as scientists and professionals! Regardless of whether we are applying prescient displaying systems to our examination or business issues, accept we make them thing in like manner: We need to make “great” forecasts! Fitting a model to our preparation information would one say one is a thing, however how would we realize that it sums up well to concealed information? How would we realize that it does not only retain the information we sustained it and neglects to make high forecasts on future examples, tests that it has not seen previously? Additionally, how would we select an appropriate model in any case? Perhaps an alternate learning calculation could be more qualified for the current issue? The right utilization of model assessment, model choice, and calculation choice systems is indispensable in scholarly AI examine just as in numerous mechanical settings. This article audits various systems that can be utilized for every one of these three subtasks and talks about the primary focal points and drawbacks of every method with references to theoretical and observational investigations. Further, suggestions are given to empower best yet plausible practices in research and uses of AI. In this article, we have used applications like Drowsiness detection, Oil price prediction, Election result evaluation as examples to explain algorithm selection and model evaluation.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Alpaydin, E.: Combined 5x2cv F test for comparing supervised classification learning algorithms. Neural Comput. 11(8), 1885–1892 (1999)CrossRef Alpaydin, E.: Combined 5x2cv F test for comparing supervised classification learning algorithms. Neural Comput. 11(8), 1885–1892 (1999)CrossRef
2.
go back to reference Bengio, Y., Grandvalet, Y.: No unbiased estimator of the variance of k-fold cross-validation. J. Mach. Learn. Res. 5(Sep), 1089–1105 (2004)MathSciNetMATH Bengio, Y., Grandvalet, Y.: No unbiased estimator of the variance of k-fold cross-validation. J. Mach. Learn. Res. 5(Sep), 1089–1105 (2004)MathSciNetMATH
3.
go back to reference Bonferroni, C.: Teoria statistica delle classi e calcolo delle probabilita. Pubblicazioni del R Istituto Superiore di Scienze Economiche e Commericiali di Firenze 8, 3–62 Bonferroni, C.: Teoria statistica delle classi e calcolo delle probabilita. Pubblicazioni del R Istituto Superiore di Scienze Economiche e Commericiali di Firenze 8, 3–62
4.
go back to reference Breiman, L., Friedman, J., Stone, C.J., Olshen, R.A.: Classification and Regression Trees. CRC Press, Boca Raton (2004)MATH Breiman, L., Friedman, J., Stone, C.J., Olshen, R.A.: Classification and Regression Trees. CRC Press, Boca Raton (2004)MATH
6.
go back to reference Jung, C.R., Kleber, C.R.: A lane departure warning system based on a linear-parabolic lane model. In: Proceedings of the IEEE Intelligent Vehicles Symposium, pp. 891–895 (2004) Jung, C.R., Kleber, C.R.: A lane departure warning system based on a linear-parabolic lane model. In: Proceedings of the IEEE Intelligent Vehicles Symposium, pp. 891–895 (2004)
7.
go back to reference Dietterich, T.G.: Approximate statistical tests for comparing supervised classification learning algorithms. Neural Comput. 10(7), 1895–1923 (2008)CrossRef Dietterich, T.G.: Approximate statistical tests for comparing supervised classification learning algorithms. Neural Comput. 10(7), 1895–1923 (2008)CrossRef
9.
go back to reference Efron, B., Tibshirani, R.J.: An Introduction to the Bootstrap. CRC Press, Boca Raton (1994)MATH Efron, B., Tibshirani, R.J.: An Introduction to the Bootstrap. CRC Press, Boca Raton (1994)MATH
10.
go back to reference Edwards, A.L.: Note on the “correction for continuity” in testing the significance of the difference between correlated proportions. Psychometrika 13(3), 185–187 (1948)CrossRef Edwards, A.L.: Note on the “correction for continuity” in testing the significance of the difference between correlated proportions. Psychometrika 13(3), 185–187 (1948)CrossRef
12.
go back to reference Efron, B.: Estimating the error rate of a prediction rule: improvement on cross-validation. J. Am. Stat. Assoc. 78(382), 316–331 (1983)MathSciNetMATHCrossRef Efron, B.: Estimating the error rate of a prediction rule: improvement on cross-validation. J. Am. Stat. Assoc. 78(382), 316–331 (1983)MathSciNetMATHCrossRef
14.
go back to reference Efron, B., Tibshirani, R.: Improvements on cross-validation: the 632+ bootstrap method. J. Am. Stat. Assoc. 92(438), 548–560 (1997)MathSciNetMATH Efron, B., Tibshirani, R.: Improvements on cross-validation: the 632+ bootstrap method. J. Am. Stat. Assoc. 92(438), 548–560 (1997)MathSciNetMATH
15.
go back to reference Fleiss, J.L., Levin, B., Paik, M.C.: Statistical Methods for Rates and Proportions. Wiley, Hoboken (2013)MATH Fleiss, J.L., Levin, B., Paik, M.C.: Statistical Methods for Rates and Proportions. Wiley, Hoboken (2013)MATH
17.
go back to reference Hawkins, D.M., Basak, S.C., Mills, D.: Assessing model fit by cross-validation. J. Chem. Inf. Comput. Sci. 43(2), 579–586 (2003)CrossRef Hawkins, D.M., Basak, S.C., Mills, D.: Assessing model fit by cross-validation. J. Chem. Inf. Comput. Sci. 43(2), 579–586 (2003)CrossRef
18.
go back to reference Batista, J.: A drowsiness and point of attendance monitoring system for driver vigilance. In: Proceedings of the IEEE Intelligent Transportation Systems Conference, pp. 702–708 (2007) Batista, J.: A drowsiness and point of attendance monitoring system for driver vigilance. In: Proceedings of the IEEE Intelligent Transportation Systems Conference, pp. 702–708 (2007)
19.
go back to reference Clanton, J.M., Bevly, D.M., Hodel, A.S.: A low-cost solution for an integrated multi-sensor lane departure warning system. IEEE Trans. Intell. Transp. Syst. 10(1), 47–59 (2009)CrossRef Clanton, J.M., Bevly, D.M., Hodel, A.S.: A low-cost solution for an integrated multi-sensor lane departure warning system. IEEE Trans. Intell. Transp. Syst. 10(1), 47–59 (2009)CrossRef
20.
go back to reference Kohavi, R.: A study of cross-validation and bootstrap for accuracy estimation and model selection. In: IJCAI, vol. 14, no. 2, pp. 1137–1145 (1995) Kohavi, R.: A study of cross-validation and bootstrap for accuracy estimation and model selection. In: IJCAI, vol. 14, no. 2, pp. 1137–1145 (1995)
21.
go back to reference Bergasa, L.M., Member, A., Nuevo, J., Sotelo, M.A., Barea, R., Lopez, M.E.: Real-time system for monitoring driver vigilance. IEEE Trans. Intell. Transp. Syst. 7(1), 63–77 (2016)CrossRef Bergasa, L.M., Member, A., Nuevo, J., Sotelo, M.A., Barea, R., Lopez, M.E.: Real-time system for monitoring driver vigilance. IEEE Trans. Intell. Transp. Syst. 7(1), 63–77 (2016)CrossRef
22.
go back to reference Flores, M.J., Armingol, J.M., de la Escalera, A.: Driver drowsiness detection system under infrared illumination for an intelligent vehicle. IET Intell. Transp. Syst. 5(4), 241–251 (2011)CrossRef Flores, M.J., Armingol, J.M., de la Escalera, A.: Driver drowsiness detection system under infrared illumination for an intelligent vehicle. IET Intell. Transp. Syst. 5(4), 241–251 (2011)CrossRef
23.
go back to reference Nickerson, D., Rogers, T.: Do you have a voting plan? Implementation intentions, voter turnout, and organic plan making. Psychol. Sci. 21(2), 194–199 (2010)CrossRef Nickerson, D., Rogers, T.: Do you have a voting plan? Implementation intentions, voter turnout, and organic plan making. Psychol. Sci. 21(2), 194–199 (2010)CrossRef
24.
go back to reference Nickerson, D.W.: Partisan mobilization using volunteer phone banks and door hangers. Ann. Am. Acad. Polit. Soc. Sci. 601, 10–27 (2015)CrossRef Nickerson, D.W.: Partisan mobilization using volunteer phone banks and door hangers. Ann. Am. Acad. Polit. Soc. Sci. 601, 10–27 (2015)CrossRef
25.
go back to reference Nickerson, D.W., Friedrichs, R.F., King, D.C.: Partisan mobilization experiments in the field: results from a statewide turnout experiment in Michigan. Polit. Res. Q. 34(1), 271–292 (2016) Nickerson, D.W., Friedrichs, R.F., King, D.C.: Partisan mobilization experiments in the field: results from a statewide turnout experiment in Michigan. Polit. Res. Q. 34(1), 271–292 (2016)
26.
go back to reference Nickerson, D.W.: Quality is job one: volunteer and professional phone calls. Am. J. Polit. Sci. 51(2), 269–282 (2017)CrossRef Nickerson, D.W.: Quality is job one: volunteer and professional phone calls. Am. J. Polit. Sci. 51(2), 269–282 (2017)CrossRef
27.
go back to reference Kulkarni, S., Haidar, I.: Forecasting model for crude oil price using artificial neural networks and commodity futures prices. Int. J. Comput. Sci. Inf. Secur. 2(1) (2009) Kulkarni, S., Haidar, I.: Forecasting model for crude oil price using artificial neural networks and commodity futures prices. Int. J. Comput. Sci. Inf. Secur. 2(1) (2009)
28.
go back to reference Abdullah, S.N., Zeng, X.: Machine learning approach for crude oil price prediction with Artificial Neural Networks-Quantitative (ANN-Q) model. In: Proceedings of the International Joint Conference on Neural Networks, pp. 1–8 (2010) Abdullah, S.N., Zeng, X.: Machine learning approach for crude oil price prediction with Artificial Neural Networks-Quantitative (ANN-Q) model. In: Proceedings of the International Joint Conference on Neural Networks, pp. 1–8 (2010)
29.
go back to reference Shin, H., Hou, T., Park, K., Park, C.K., Choi, S.: Prediction of movement direction in crude oil prices based on semi-supervised learning. Decis. Support Syst. 55, 348–358 (2013)CrossRef Shin, H., Hou, T., Park, K., Park, C.K., Choi, S.: Prediction of movement direction in crude oil prices based on semi-supervised learning. Decis. Support Syst. 55, 348–358 (2013)CrossRef
Metadata
Title
Algorithm Selection and Model Evaluation in Application Design Using Machine Learning
Authors
Srikanth Bethu
B. Sankara Babu
K. Madhavi
P. Gopala Krishna
Copyright Year
2020
DOI
https://doi.org/10.1007/978-3-030-45778-5_12

Premium Partner