Skip to main content

Advertisement

SpringerLink
Log in

Comparative performance of six supervised learning methods for the development of models of hard rock pillar stability prediction

  • Original Paper
  • Published:
Natural Hazards Aims and scope Submit manuscript

Abstract

The prediction of pillar stability (PS) in hard rock mines is a crucial task for which many techniques and methods have been proposed in the literature including machine learning classification. In order to make the best use of the large variety of statistical and machine learning classification methods available, it is necessary to assess their performance before selecting a classifier and suggesting improvement. The objective of this paper is to compare different classification techniques for PS detection in hard rock mines. The data of this study consist of six features, namely pillar width, pillar height, the ratio of pillar width to its height, uniaxial compressive strength of the rock, pillar strength, and pillar stress. A total of 251 pillar cases between 1972 and 2011 are analyzed. Six supervised learning algorithms, including linear discriminant analysis, multinomial logistic regression, multilayer perceptron neural networks, support vector machine (SVM), random forest (RF), and gradient boosting machine, are evaluated for their ability to learn for PS based on different input parameter combinations. In this study, the available data set is randomly split into two parts: training set (70 %) and test set (30 %). A repeated tenfold cross-validation procedure (ten repeats) is applied to determine the optimal parameter values during modeling, and an external testing set is employed to validate the prediction performance of models. Two performance measures, namely classification accuracy rate and Cohen’s kappa, are employed. The analysis of the accuracy together with kappa for the PS data set demonstrates that SVM and RF achieve comparable median classification accuracy rate and Cohen’s kappa values. All models are fitted by “R” programs with the libraries and functions described in this study.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  • Berrueta LA, Alonso-Salces RM, Heberger K (2007) Supervised pattern recognition in food analysis. J Chromatogr A 1158(1–2):196–214

    Article  Google Scholar 

  • Bieniawski ZT (1968) The effects of specimen size on the compressive strength of coal. Int J Rock Mech Min Sci Geomech Abstr 5(4):325–335

  • Bieniawski ZT, Van Heerden WL (1975) The significance of in situ tests on large rock specimens. Int J Rock Mech Min Sci Geomech Abstr 12(4):101–113.

  • Bishop CM (1995) Neural networks for pattern recognition. Oxford University Press, New York

    Google Scholar 

  • Brady B, Brown ET (2003) Rock mechanics for underground mining, 2nd edn. Chapman and Hall, London

    Google Scholar 

  • Breiman L (2001) Random forests. Mach Learn 45:5–32

    Article  Google Scholar 

  • Breiman L, Cutler A (2007) Randomforests—classification description: randomforests. http://stat-www.berkeley.edu/users/breiman/RandomForests/cc_home.htm. Accessed 15 Jan 2014

  • Cauvin M, Verde T, Salmon R (2009) Modeling uncertainties in mining pillar stability analysis. Risk Anal 29(10):1371–1380

    Article  Google Scholar 

  • Clark M (2013) An introduction to machine learning: with applications in R. http://www3.nd.edu/~mclark19/learn/ML.pdf

  • Congalton RG, Green K (2009) Assessing the accuracy of remotely sensed data: principles and practices, 2nd edn. Lewis, Boca Raton

    Google Scholar 

  • Deng J, Yue ZQ, Tham LG, Zhu HH (2003) Pillar design by combining finite element methods, neural networks and reliability: a case study of the Feng Huangshan copper mine, China. Int J Rock Mech Min Sci 40(4):585–599

    Article  Google Scholar 

  • Elmo D, Stead D (2010) An integrated numerical modelling–discrete fracture network approach applied to the characterisation of rock mass strength of naturally fractured pillars. Rock Mech Rock Eng 43(1):3–19

    Article  Google Scholar 

  • Esterhuizen GS (1993) Variability considerations in hard rock pillar design. In: Proceedings of the SANGORM symposium: rock engineering problems related to hard rock mining at shallow to intermediate depth, Rustenburg, South Africa

  • Esterhuizen GS, Dolinar DR, Ellenberger JL (2011) Pillar strength in underground stone mines in the United States. Int J Rock Mech Min Sci 48(1):42–50

    Article  Google Scholar 

  • Fisher DH (1936) The use of multiple measurements in taxonomic problems. Ann Eugen 7(7):179–188

    Article  Google Scholar 

  • Friedman JH (2001) Greedy function approximation: a gradient boosting machine. Ann Stat 29(5):1189–1232

    Article  Google Scholar 

  • Garzón MB, Blazek R, Neteler M, de Dios RS, Ollero HS, Furlanello C (2006) Predicting habitat suitability with machine learning models: the potential area of Pinus sylvestris L. in the Iberian Peninsula. Ecol Model 97:383–393

    Article  Google Scholar 

  • Ghasemi E, Shahriar K (2012) A new coal pillars design method in order to enhance safety of retreat mining in room and pillar mines. Saf Sci 50:579–585

    Article  Google Scholar 

  • Ghasemi E, Shahriar K, Sharifzadeh M, Hashemolhosseini H (2010) Quantifying the uncertainty of pillar safety factor by Monte Carlo simulation—a case study. Arch Min Sci 55:623–635

    Google Scholar 

  • Ghasemi E, Ataei M, Shahriar K (2014a) An intelligent approach to predict pillar sizing in designing room and pillar coal mines. Int J Rock Mech Min Sci 65:86–95

    Google Scholar 

  • Ghasemi E, Ataei M, Shahriar K (2014b) Prediction of global stability in room and pillar coal mines. Nat Hazards 1–18

  • González-Nicieza C, Alvarez-Fernandez MI, Menéndez-Díaz A, Alvarez-Vigil AE (2006) A comparative analysis of pillar design methods and its application to marble mines. Rock Mech Rock Eng 39(5):421–444

    Article  Google Scholar 

  • González-Rufino E, Carrión P, Cernadas E, Fernández-Delgado M, Domínguez-Petit R (2013) Exhaustive comparison of colour texture features and classification methods to discriminate cells categories in histological images of fish ovary. Pattern Recogn 46(9):2391–2407

    Article  Google Scholar 

  • Griffiths DV, Fenton GA, Lemons CB (2007) The random finite element method (RFEM) in mine pillar stability analysis. Probabilist Methods Geotech Eng 491:271–294

  • Griffiths DV, Fenton GA, Lemons CB (2002) Probabilistic analysis of underground pillar stability. Int J Numer Anal Meth Geomech 26(8):775–791

    Article  Google Scholar 

  • Guelman L (2012) Gradient boosting trees for auto insurance loss cost modeling and prediction. Expert Syst Appl 39:3659–3667

    Article  Google Scholar 

  • Hastie T, Tibshirani R, Friedman J (2009) The elements of statistical learning: data mining, inference, and prediction, 2nd edn. Springer, New York

    Book  Google Scholar 

  • Haykin S (1998) Neural networks: a comprehensive foundation, 2nd edn. Prentice Hall, New Jersey

    Google Scholar 

  • Hedley DGF, Grant F (1972) Stope-and-pillar design for the Elliot Lake Uranium Mines. Bull Can Inst Min Metall 65:37–44

    Google Scholar 

  • Hoek E, Brown ET (1980) Underground excavation in rock. The Institute of Mining and Metallurgy, London

    Google Scholar 

  • Hudyma MR (1988) Rib pillar design in open stope mining. MASc thesis, University of BC

  • Hustrulid WA (1976) A review of coal pillar strength formulas. Rock Mech 8(2):115–145

    Article  Google Scholar 

  • Hutchinson DJ, Phillips C, Cascante G (2002) Risk considerations for crown pillar stability assessment for mine closure planning. Geotech Geol Eng 20(1):41–63

    Article  Google Scholar 

  • Jaiswal A, Shrivastva BK (2009) Numerical simulation of coal pillar strength. Int J Rock Mech Min Sci 46(4):779–788

    Article  Google Scholar 

  • Jaiswal A, Sharma SK, Shrivastva BK (2004) Numerical modeling study of asymmetry in the induced stresses over coal mine pillars with advancement of the goaf line. Int J Rock Mech Min Sci 41(5):859–864

    Article  Google Scholar 

  • Jawed M, Sinha RK, Sengupta S (2013) Chronological development in coal pillar design for bord and pillar workings: a critical appraisal. J Geol Min Res 5(1):1–11

    Article  Google Scholar 

  • Karatzoglou A, Smola A, Hornik K, Zeileis A (2004) Kernlab—an S4 package for kernel methods in R. J Stat Softw 11(9):1–20

    Google Scholar 

  • Kohavi R (1995) A study of cross-validation and bootstrap for accuracy estimation and model selection. In: IJCAI’95 Proceedings of the 14th international joint conference on artificial intelligence, vol 2. Morgan Kaufmann Publishers Inc., San Francisco, pp 1137–1143

  • Kordon AK (2010) Applying computational intelligence: How to create value. The Dow Chemical Company, Freeport, p 459

    Book  Google Scholar 

  • Krishnapuram B, Carin L, Figueiredo MA, Hartemink AJ (2005) Sparse multinomial logistic regression: fast algorithms and generalization bounds. Pattern Anal Mach Intell IEEE Trans 27(6):957–968

    Article  Google Scholar 

  • Kuhn M (2008) Building predictive models in R using the caret package. J Stat Softw 28(5):1–26

    Google Scholar 

  • Kuhn M (2012) “Caret” package (R Package Version 5.15-023). R Foundation for Statistical Computing, Vienna, Austria

  • Kuhn M, Johnson K (2013) Applied predictive modeling. Springer, New York

    Book  Google Scholar 

  • Landis J, Koch G (1977) The measurement of observer agreement for categorical data. Biometrics 33(1):159–174

    Article  Google Scholar 

  • Li XB, Li DY, Guo L, Ye ZY (2007) Study on mechanical response of highly-stressed pillars in deep mining under dynamic disturbance. Chin J Rock Mech Eng 26(5):922–928

    Google Scholar 

  • Li XB, Li DY, Liu ZX, Zhao GY, Wang WH (2013) Determination of the minimum thickness of crown pillar for safe exploitation of a subsea gold mine based on numerical modelling. Int J Rock Mech Min Sci 57:42–56

    Google Scholar 

  • Liaw A, Wiener M (2002) Classification and regression by randomforest. R News 2:18–22

    Google Scholar 

  • Liu XZ, Zhai DY (2000) The reliability design of pillar. Chin J Rock Mech Eng 18(6):85–88

    Google Scholar 

  • Liu ZB, Shao JF, Xu WY, Meng YD (2013) Prediction of rock burst classification using the technique of cloud models with attribution weight. Nat Hazards 68(2):549–568

    Article  Google Scholar 

  • Liu ZB, Shao JF, Xu WY, Chen HJ, Zhang Y (2014) An extreme learning machine approach for slope stability evaluation and prediction. Nat Hazards 1–18

  • Lunder PJ (1994) Hard rock pillar strength estimation an applied empirical approach. University of British Columbia, Vancouver (MASc thesis)

  • Lunder PJ, Pakalnis R (1997) Determination of the strength of hard rock mine pillars. Bull Can Inst Min Metall 90(1013):51–55

    Google Scholar 

  • Mark C (2006) The evolution of intelligent coal pillar design: 1981–2006. In: Proceedings of the 25th international conference on ground control in mining. West Virginia University, Morgantown, pp 325–334

  • Mark C, Barton TM (1997) Pillar design and coal strength. In: Proceedings of the New Technology for Ground Control in Retreat Mining. US Department of Health and Human Services, Public Health Service, Centers for Disease Control and Prevention, National Institute for Occupational Safety and Health, Pittsburgh, PA, pp 49–59, DHHS (NIOSH) Publication (No. 97–122)

  • Martin CD, Maybee WG (2000) The strength of hard-rock pillars. Int J Rock Mech Min Sci 37:1239–1246

    Article  Google Scholar 

  • Mitri HS (2007) Assessment of horizontal pillar burst in deep hard rock mines. Int J Risk Assess Manag 7(5):695–707

    Article  Google Scholar 

  • Molinaro AM, Simon R, Pfeiffer RM (2005) Prediction error estimation: a comparison of resampling methods. Bioinformatics 21(15):3301–3307

    Article  Google Scholar 

  • Monjezi M, Hesami SM, Khandelwal M (2011) Superiority of neural networks for pillar stress prediction in bord and pillar method. Arab J Geosci 4(5–6):845–853

    Article  Google Scholar 

  • Mortazavi A, Hassani FP, Shabani M (2009) A numerical investigation of rock pillar failure mechanism in underground openings. Comput Geotech 36(5):691–697

    Article  Google Scholar 

  • Pandya DH, Upadhyay SH, Harsha SP (2014) Fault diagnosis of rolling element bearing by using multinomial logistic regression and wavelet packet transform. Soft Comput 18(2):255–266

    Article  Google Scholar 

  • Pino-Mejías R, Carrasco-Mairena M, Pascual-Acosta A, Cubiles-De-La-Vega M, Joaquín Muñoz-García J (2008) A comparison of classification models to identify the Fragile X Syndrome. J Appl Stat 35(3):233–244

    Article  Google Scholar 

  • Pino-Mejías R, Cubiles-de-la-Vega MD, Anaya-Romero M, Pascual-Acosta A, Jordán-López A, Bellinfante-Crocci N (2010) Predicting the potential habitat of oaks with data mining models and the R system. Environ Model Softw 25(7):826–836

    Article  Google Scholar 

  • Potvin Y, Hudyma M, Miller HDS (1989) Rib pillar design in open stope mining. Bull Can Inst Min Metall 82(927):31–36

    Google Scholar 

  • Pozdnoukhov A, Foresti L, Kanevski M (2009) Data-driven topo-climatic mapping with machine learning methods. Nat Hazards 50(3):497–518

    Article  Google Scholar 

  • R Development Core Team (2013) R: a language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. ISBN 3-900051-07-0. http://www.R-project.org/

  • Ridgeway G (2007) Generalized boosted models: a guide to the gbm package. http://cran.r-project.org/web/packages/gbm/index.html

  • Ripley B (2009) Nnet: feed-forward neural networks and multinomial log-linear models. http://cran.r-project.org/web/packages/nnet/index.html

  • Sadat-Hashemi SM, Kazemnejad A, Lucas C, Badie K (2005) Predicting the type of pregnancy using artificial neural networks and multinomial logistic regression: a comparison study. Neural Comput Appl 14(3):198–202

    Article  Google Scholar 

  • Sakiyama Y, Yuki H, Moriya T, Hattori K, Suzuki M, Shimada K, Honma T (2008) Predicting human liver microsomal stability with machine learning techniques. J Mol Graph Model 26:907–915

    Article  Google Scholar 

  • Salamon MDG (1970) Stability, instability and design of coal pillar workings. Int J Rock Mech Min Sci Geomech Abstr 7(6):613–631

    Article  Google Scholar 

  • Salamon MDG, Munro AH (1967) A study of the strength of coal pillars. J South Afr Inst Min Metall 68:55–67

  • Sheng JH, Liao WJ, Li WM (2010) Analysis of pillar safety factor in Gaoshan gypsum mine. Metal Mine (suppl.):791–793

  • Sjoberg JS (1992) Failure modes and pillar behaviour in the Zinkgruvan mine. In: Tillerson JA, Wawersik WR (eds) Proceedings of the 33rd U.S. rock mechanics symposium. A.A. Balkema, Sante Fe. Rotterdam, pp 491–500

  • Tawadrous AS, Katsabanis PD (2007) Prediction of surface crown pillar stability using artificial neural networks. Int J Numer Anal Meth Geomech 31(7):917–931

    Article  Google Scholar 

  • Tesfamariam S, Liu Z (2010) Earthquake induced damage classification for reinforced concrete buildings. Struct Saf 32(2):154–164

    Article  Google Scholar 

  • Vapnik VN (1995) The nature of statistical learning theory. Springer, New York

    Book  Google Scholar 

  • Venables WN, Ripley BD (2002) Modern applied statistics with S. Springer, New York

  • Von Kimmelman MR, Hyde B, Madgwick RJ (1984) The use of computer applications at BCL Limited in planning pillar extraction and design of mining layouts. In: Proceedings of the ISRM symposium: design and performance of underground excavations. British Geotechnical Society, London, p 53J63

  • Wattimena RK (2014) Predicting the stability of hard rock pillars using multinomial logistic regression. Int J Rock Mech Min Sci 71:33–40

    Google Scholar 

  • Wattimena RK, Kramadibrata S, Sidi ID, Azizi MA (2013) Developing coal pillar stability chart using logistic regression. Int J Rock Mech Min Sci 58:55–60

    Google Scholar 

  • York G (1998) Numerical modelling of the yielding of a stabilizing pillar/foundation system and a new design consideration for stabilizing pillar foundations. J. S.A. Inst Min Metall 98:281–293

    Google Scholar 

  • Zhou J, Shi XZ, Dong L, Hu HY, Wang HY (2010) Fisher discriminant analysis model and its application for prediction of classification of rockburst in deep-buried long tunnel. J Coal Sci Eng 16(2):144–149

    Article  Google Scholar 

  • Zhou J, Li XB, Shi XZ, Wei W, Wu BB (2011) Predicting pillar stability for underground mine using Fisher discriminant analysis and SVM methods. Trans Nonferrous Metals Soc China 21(12):2734–2743

    Article  Google Scholar 

  • Zhou J, Li XB, Shi XZ (2012) Long-term prediction model of rockburst in underground openings using heuristic algorithms and support vector machines. Saf Sci 50(4):629–644

    Article  Google Scholar 

  • Zhou J, Li XB, Mitri HS, Wang SM, Wei W (2013) Identification of large-scale goaf instability in underground mine using particle swarm optimization and support vector machine. Int J Min Sci Technology 23(5):701–707

    Article  Google Scholar 

Download references

Acknowledgments

This research was partially supported bythe National Natural Science Foundation Project (Grant Nos. 11472311 and 41272304) of China, the Graduated Students’ Research, Innovation Fund Project (Grant No. CX2011B119) of Hunan Province of China, Project (Grant No. 1343-76140000022) supported by the Scholarship Award for Excellent Doctoral Student of Ministry of Education of China and the Valuable Equipment Open Sharing Fund of Central South University. The authors would like to express thanks to these foundations. The first author would like to thank the Chinese Scholarship Council for financial support to the joint PhD at McGill University, Canada. We also would like to thank the three anonymous referees and editors for their valuable comments and suggestions which improved a previous version of this manuscript.

Author information

Authors and Affiliations

  1. School of Resources and Safety Engineering, Central South University, Changsha, 410083, China

    Jian Zhou & Xibing Li

  2. Department of Mining and Materials Engineering, McGill University, Montreal, QC, H3A 2A7, Canada

    Jian Zhou & Hani S. Mitri

  3. School of Civil Engineering, Henan Polytechnic University, Jiaozuo, 454150, China

    Hani S. Mitri

Authors
  1. Jian Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xibing Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hani S. Mitri
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jian Zhou.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhou, J., Li, X. & Mitri, H.S. Comparative performance of six supervised learning methods for the development of models of hard rock pillar stability prediction. Nat Hazards 79, 291–316 (2015). https://doi.org/10.1007/s11069-015-1842-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11069-015-1842-3

Keywords

Search

Navigation