Skip to main content

2018 | OriginalPaper | Buchkapitel

Big Data: Issues, Challenges, and Techniques in Business Intelligence

verfasst von : Mudasir Ahmad Wani, Suraiya Jabin

Erschienen in: Big Data Analytics

Verlag: Springer Singapore

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

During the last decade, the most challenging problem the world envisaged was big data problem. The big data problem means that data is growing at a much faster rate than computational speeds. And it is the result of the fact that storage cost is getting cheaper day by day, so people as well as almost all business or scientific organizations are storing more and more data. Social activities, scientific experiments, biological explorations along with the sensor devices are great big data contributors. Big data is beneficial to the society and business but at the same time, it brings challenges to the scientific communities. The existing traditional tools, machine learning algorithms, and techniques are not capable of handling, managing, and analyzing big data, although various scalable machine learning algorithms, techniques, and tools (e.g., Hadoop and Apache Spark open source platforms) are prevalent. In this paper, we have identified the most pertinent issues and challenges related to big data and point out a comprehensive comparison of various techniques for handling big data problem.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Kaisler, S., Armour, F., Espinosa, J.A., Money, W.: Big data: issues and challenges moving forward. In: 2013 46th Hawaii International Conference on System Sciences (HICSS), pp. 995–1004. IEEE (2013) Kaisler, S., Armour, F., Espinosa, J.A., Money, W.: Big data: issues and challenges moving forward. In: 2013 46th Hawaii International Conference on System Sciences (HICSS), pp. 995–1004. IEEE (2013)
2.
Zurück zum Zitat Katal, A., Wazid, M., Goudar, R.: Big data: issues, challenges, tools and good practices. In: 2013 Sixth International Conference on Contemporary Computing (IC3), pp. 404–409. IEEE (2013) Katal, A., Wazid, M., Goudar, R.: Big data: issues, challenges, tools and good practices. In: 2013 Sixth International Conference on Contemporary Computing (IC3), pp. 404–409. IEEE (2013)
3.
Zurück zum Zitat Fan, J., Han, F., Liu, H.: Challenges of big data analysis. Natl. Sci. Rev. 1(2), 293–314 (2014)CrossRef Fan, J., Han, F., Liu, H.: Challenges of big data analysis. Natl. Sci. Rev. 1(2), 293–314 (2014)CrossRef
4.
Zurück zum Zitat Beyer, M.A., Laney, D.: The Importance of Big Data: A Definition. Gartner, Stamford (2012) Beyer, M.A., Laney, D.: The Importance of Big Data: A Definition. Gartner, Stamford (2012)
5.
Zurück zum Zitat Laney, D.: 3d data management: controlling data volume, velocity and variety. META Group Research Note, 6, 70 (2001) Laney, D.: 3d data management: controlling data volume, velocity and variety. META Group Research Note, 6, 70 (2001)
6.
Zurück zum Zitat Minelli, M., Chambers, M., Dhiraj, A.: Big Data, Big Analytics: Emerging Business Intelligence and Analytic Trends for Today’s Businesses. Wiley, New York (2012) Minelli, M., Chambers, M., Dhiraj, A.: Big Data, Big Analytics: Emerging Business Intelligence and Analytic Trends for Today’s Businesses. Wiley, New York (2012)
7.
Zurück zum Zitat Vossen, G.: Big data as the new enabler in business and other intelligence. Vietnam J. Comput. Sci. 1(1), 3–14 (2014)CrossRef Vossen, G.: Big data as the new enabler in business and other intelligence. Vietnam J. Comput. Sci. 1(1), 3–14 (2014)CrossRef
8.
Zurück zum Zitat Laney, D.: 3D data management: controlling data volume, velocity and variety. META Group Research Note, 6, 70 (2001) Laney, D.: 3D data management: controlling data volume, velocity and variety. META Group Research Note, 6, 70 (2001)
9.
Zurück zum Zitat Han, J., Kamber, M.: Data Mining: Concepts and Techniques. Morgan Kaumann Publishers, United States of America (2001)MATH Han, J., Kamber, M.: Data Mining: Concepts and Techniques. Morgan Kaumann Publishers, United States of America (2001)MATH
10.
Zurück zum Zitat Jabin, S., Zareen, F.J.: Biometric signature verification. Int. J. Biom. 7(2), 97–118 (2015)CrossRef Jabin, S., Zareen, F.J.: Biometric signature verification. Int. J. Biom. 7(2), 97–118 (2015)CrossRef
11.
Zurück zum Zitat Jabin, S.: Stock market prediction using feed-forward artificial neural network. Int. J. Comput. Appl. 99(9), 4–8 (2014) Jabin, S.: Stock market prediction using feed-forward artificial neural network. Int. J. Comput. Appl. 99(9), 4–8 (2014)
12.
Zurück zum Zitat Jabin, S.: Learning classifier systems approach for automated discovery of hierarchical censored production rules. In: Information and Communication Technologies, pp. 68-77. Springer, Berlin (2010) Jabin, S.: Learning classifier systems approach for automated discovery of hierarchical censored production rules. In: Information and Communication Technologies, pp. 68-77. Springer, Berlin (2010)
13.
Zurück zum Zitat Xiong, H.Y., Alipanahi, B., Lee, L.J., Bretschneider, H., Merico, D., Yuen, R.K., Hua, Y., Gueroussov, S., Najafabadi, H.S., Hughes, T.R., et al.: The human splicing code reveals new insights into the genetic determinants of disease. Science 347(6218), 1254806 (2015)CrossRef Xiong, H.Y., Alipanahi, B., Lee, L.J., Bretschneider, H., Merico, D., Yuen, R.K., Hua, Y., Gueroussov, S., Najafabadi, H.S., Hughes, T.R., et al.: The human splicing code reveals new insights into the genetic determinants of disease. Science 347(6218), 1254806 (2015)CrossRef
14.
Zurück zum Zitat Kellis, M., Wold, B., Snyder, M.P., Bernstein, B.E., Kundaje, A., Marinov, G.K., Ward, L.D., Birney, E., Crawford, G.E., Dekker, J., et al.: Defining functional DNA elements in the human genome. Proc. Natl. Acad. Sci. 111(17), 6131–6138 (2014)CrossRef Kellis, M., Wold, B., Snyder, M.P., Bernstein, B.E., Kundaje, A., Marinov, G.K., Ward, L.D., Birney, E., Crawford, G.E., Dekker, J., et al.: Defining functional DNA elements in the human genome. Proc. Natl. Acad. Sci. 111(17), 6131–6138 (2014)CrossRef
15.
Zurück zum Zitat Tsikerdekis, M., Zeadally, S.: Multiple account identity deception detection in social media using nonverbal behavior. IEEE Trans. Inf. Forensics Secur. 9(8), 1311–1321 (2014)CrossRef Tsikerdekis, M., Zeadally, S.: Multiple account identity deception detection in social media using nonverbal behavior. IEEE Trans. Inf. Forensics Secur. 9(8), 1311–1321 (2014)CrossRef
16.
Zurück zum Zitat Schön, D.A., Argyris, C.: Organizational learning: a theory of action perspective. Reis: Revista española de investigaciones sociológicas 77, 345–350 (1997) Schön, D.A., Argyris, C.: Organizational learning: a theory of action perspective. Reis: Revista española de investigaciones sociológicas 77, 345–350 (1997)
17.
Zurück zum Zitat Ebner, K., Buhnen, T., Urbach, N.: Think big with big data: identifying suitable big data strategies in corporate environments. In: 2014 47th Hawaii International Conference on System Sciences (HICSS), pp. 3748–3757. IEEE (2014) Ebner, K., Buhnen, T., Urbach, N.: Think big with big data: identifying suitable big data strategies in corporate environments. In: 2014 47th Hawaii International Conference on System Sciences (HICSS), pp. 3748–3757. IEEE (2014)
19.
Zurück zum Zitat Che, D., Safran, M., Peng, Z.: From big data to big data mining: challenges, issues, and opportunities. In: Database Systems for Advanced Applications, pp. 1–15. Springer, Berlin (2013) Che, D., Safran, M., Peng, Z.: From big data to big data mining: challenges, issues, and opportunities. In: Database Systems for Advanced Applications, pp. 1–15. Springer, Berlin (2013)
20.
Zurück zum Zitat Hashem, I.A.T., Yaqoob, I., Anuar, N.B., Mokhtar, S., Gani, A., Khan, S.U.: The rise of “big data” on cloud computing: review and open research issues. Inf. Syst. 47, 98–115 (2015) Hashem, I.A.T., Yaqoob, I., Anuar, N.B., Mokhtar, S., Gani, A., Khan, S.U.: The rise of “big data” on cloud computing: review and open research issues. Inf. Syst. 47, 98–115 (2015)
22.
Zurück zum Zitat Wu, X., Zhu, X., Wu, G.Q., Ding, W.: Data mining with big data. IEEE Trans. Knowl. Data Eng. 26(1), 97–107 (2014)CrossRef Wu, X., Zhu, X., Wu, G.Q., Ding, W.: Data mining with big data. IEEE Trans. Knowl. Data Eng. 26(1), 97–107 (2014)CrossRef
23.
Zurück zum Zitat Buhl, H.U., Röglinger, M., Moser, D.K.F., Heidemann, J.: Big data. Bus. Inf. Syst. Eng. 5(2), 65–69 (2013)CrossRef Buhl, H.U., Röglinger, M., Moser, D.K.F., Heidemann, J.: Big data. Bus. Inf. Syst. Eng. 5(2), 65–69 (2013)CrossRef
24.
Zurück zum Zitat Dean, J., Ghemawat, S.: MapReduce: simplified data processing on large clusters. Commun. ACM 51(1), 107–113 (2008)CrossRef Dean, J., Ghemawat, S.: MapReduce: simplified data processing on large clusters. Commun. ACM 51(1), 107–113 (2008)CrossRef
25.
Zurück zum Zitat Wu, K.: Fastbit: an efficient indexing technology for accelerating data-intensive science. In: Journal of Physics: Conference Series, vol. 16, p. 556. IOP Publishing (2005) Wu, K.: Fastbit: an efficient indexing technology for accelerating data-intensive science. In: Journal of Physics: Conference Series, vol. 16, p. 556. IOP Publishing (2005)
26.
Zurück zum Zitat Dittrich, J., Quiane-Ruiz, J.A.: Efficient big data processing in hadoop MapReduce. Proc. VLDB Endowment 5(12), 2014–2015 (2012)CrossRef Dittrich, J., Quiane-Ruiz, J.A.: Efficient big data processing in hadoop MapReduce. Proc. VLDB Endowment 5(12), 2014–2015 (2012)CrossRef
27.
Zurück zum Zitat Triguero, I., Peralta, D., Bacardit, J., Garc a, S., Herrera, F.: MRPR: a MapReduce solution for prototype reduction in big data classification. Neurocomputing 150, 331–345 (2015) Triguero, I., Peralta, D., Bacardit, J., Garc a, S., Herrera, F.: MRPR: a MapReduce solution for prototype reduction in big data classification. Neurocomputing 150, 331–345 (2015)
28.
Zurück zum Zitat Neumeyer, L., Robbins, B., Nair, A., Kesari, A.: S4: distributed stream computing platform. In: 2010 IEEE International Conference on Data Mining Workshops (ICDMW), pp. 170–177 (2010) Neumeyer, L., Robbins, B., Nair, A., Kesari, A.: S4: distributed stream computing platform. In: 2010 IEEE International Conference on Data Mining Workshops (ICDMW), pp. 170–177 (2010)
29.
Zurück zum Zitat Chen, C.P., Zhang, C.Y.: Data-intensive applications, challenges, techniques and technologies: a survey on big data. In: Information Conference on Cloud System and Big Data Engineering, pp. 404–409. IEEE (2013) Chen, C.P., Zhang, C.Y.: Data-intensive applications, challenges, techniques and technologies: a survey on big data. In: Information Conference on Cloud System and Big Data Engineering, pp. 404–409. IEEE (2013)
30.
Zurück zum Zitat Manyika, J., Chui, M., Brown, B., Bughin, J., Dobbs, R., Roxburgh, C., Byers, A.H.: Big data: the next frontier for innovation, competition, and productivity (2011) Manyika, J., Chui, M., Brown, B., Bughin, J., Dobbs, R., Roxburgh, C., Byers, A.H.: Big data: the next frontier for innovation, competition, and productivity (2011)
31.
Zurück zum Zitat Kim, G.H., Trimi, S., Chung, J.H.: Big-data applications in the government sector. Commun. ACM 57(3), 78–85 (2014)CrossRef Kim, G.H., Trimi, S., Chung, J.H.: Big-data applications in the government sector. Commun. ACM 57(3), 78–85 (2014)CrossRef
32.
Zurück zum Zitat Stonebraker, M., Hong, J.: Researchers ‘big data crisis; understanding design and functionality. Commun ACM 55(2), 10–11 (2012)CrossRef Stonebraker, M., Hong, J.: Researchers ‘big data crisis; understanding design and functionality. Commun ACM 55(2), 10–11 (2012)CrossRef
33.
Zurück zum Zitat Zaharia, M., Chowdhury, M., Franklin, M.J., Shenker, S., Stoica, I.: Spark: cluster computing with working sets. In: Proceedings of the 2nd USENIX Conference on Hot Topics in Cloud Computing, vol. 10, p. 10 (2010) Zaharia, M., Chowdhury, M., Franklin, M.J., Shenker, S., Stoica, I.: Spark: cluster computing with working sets. In: Proceedings of the 2nd USENIX Conference on Hot Topics in Cloud Computing, vol. 10, p. 10 (2010)
34.
Zurück zum Zitat Driscoll, A.O., Daugelaite, J., Sleator, R.D.: Big data, Hadoop and cloud computing in genomics. J. Biomed. Inf. 46(5), 774–781 (2013) Driscoll, A.O., Daugelaite, J., Sleator, R.D.: Big data, Hadoop and cloud computing in genomics. J. Biomed. Inf. 46(5), 774–781 (2013)
35.
Zurück zum Zitat Simoff, S., Bohlen, M.H., Mazeika, A.: Visual Data Mining: Theory, Techniques and Tools for Visual Analytics, vol. 4404. Springer Science & Business Media (2008) Simoff, S., Bohlen, M.H., Mazeika, A.: Visual Data Mining: Theory, Techniques and Tools for Visual Analytics, vol. 4404. Springer Science & Business Media (2008)
36.
Zurück zum Zitat Sawant, N., Shah, H.: Big data visualization patterns. In: Big Data Application Architecture Q&A, pp. 79–90 (2013) Sawant, N., Shah, H.: Big data visualization patterns. In: Big Data Application Architecture Q&A, pp. 79–90 (2013)
40.
Zurück zum Zitat Gu, L., Li, H.: Memory or time: performance evaluation for iterative operation on hadoop and spark. In: 2013 IEEE 10th International Conference on High Performance Computing and Communications & 2013 IEEE International Conference on Embedded and Ubiquitous Computing (HPCC EUC), pp. 721–727. IEEE (2013) Gu, L., Li, H.: Memory or time: performance evaluation for iterative operation on hadoop and spark. In: 2013 IEEE 10th International Conference on High Performance Computing and Communications & 2013 IEEE International Conference on Embedded and Ubiquitous Computing (HPCC EUC), pp. 721–727. IEEE (2013)
41.
Zurück zum Zitat Zheng, X., Zeng, Z., Chen, Z., Yu, Y., Rong, C.: Detecting spammers on social networks. Neurocomputing 159, 27–34 (2015)CrossRef Zheng, X., Zeng, Z., Chen, Z., Yu, Y., Rong, C.: Detecting spammers on social networks. Neurocomputing 159, 27–34 (2015)CrossRef
42.
Zurück zum Zitat Conti, M., Poovendran, R., Secchiero, M.: Fakebook: detecting fake profiles in on-line social networks. In: Proceedings of the 2012 International Conference on Advances in Social Networks Analysis and Mining (ASONAM 2012), pp. 1071–1078. IEEE Computer Society (2012) Conti, M., Poovendran, R., Secchiero, M.: Fakebook: detecting fake profiles in on-line social networks. In: Proceedings of the 2012 International Conference on Advances in Social Networks Analysis and Mining (ASONAM 2012), pp. 1071–1078. IEEE Computer Society (2012)
43.
Zurück zum Zitat Anwar, T., Abulaish, M.: Ranking radically influential web forum users. IEEE Trans. Inf. Forensics Secur. 10(6), 1289–1298 (2015)CrossRef Anwar, T., Abulaish, M.: Ranking radically influential web forum users. IEEE Trans. Inf. Forensics Secur. 10(6), 1289–1298 (2015)CrossRef
44.
Zurück zum Zitat McKenna, A., Hanna, M., Banks, E., Sivachenko, A., Cibulskis, K., Kernytsky, A., Garimella, K., Altshuler, D., Gabriel, S., Daly, M., et al.: The genome analysis toolkit: a MapReduce framework for analyzing next-generation DNA sequencing data. Genome Res. 20(9), 1297–1303 (2010)CrossRef McKenna, A., Hanna, M., Banks, E., Sivachenko, A., Cibulskis, K., Kernytsky, A., Garimella, K., Altshuler, D., Gabriel, S., Daly, M., et al.: The genome analysis toolkit: a MapReduce framework for analyzing next-generation DNA sequencing data. Genome Res. 20(9), 1297–1303 (2010)CrossRef
47.
Zurück zum Zitat Marx, V.: Biology: the big challenges of big data. Nature 498(7453), 255–260 (2013)CrossRef Marx, V.: Biology: the big challenges of big data. Nature 498(7453), 255–260 (2013)CrossRef
48.
Zurück zum Zitat Assuncao, M.D., Calheiros, R.N., Bianchi, S., Netto, M.A., Buyya, R.: Big data computing and clouds: trends and future directions. J. Parallel Distrib. Comput. 79, 3–15 (2015)CrossRef Assuncao, M.D., Calheiros, R.N., Bianchi, S., Netto, M.A., Buyya, R.: Big data computing and clouds: trends and future directions. J. Parallel Distrib. Comput. 79, 3–15 (2015)CrossRef
49.
Zurück zum Zitat Kambatla, K., Kollias, G., Kumar, V., Grama, A.: Trends in big data analytics. J. Parallel Distrib. Comput. 74(7), 2561–2573 (2014)CrossRef Kambatla, K., Kollias, G., Kumar, V., Grama, A.: Trends in big data analytics. J. Parallel Distrib. Comput. 74(7), 2561–2573 (2014)CrossRef
50.
Zurück zum Zitat Shanahan, J.G., Dai, L.: Large scale distributed data science using apache spark. In: Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 2323–2324. ACM (2015) Shanahan, J.G., Dai, L.: Large scale distributed data science using apache spark. In: Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 2323–2324. ACM (2015)
51.
Zurück zum Zitat Stich, V., Jordan, F., Birkmeier, M., Oazgil, K., Reschke, J., Diews, A.: Big data technology for resilient failure management in production systems. In: Advances in Production Management Systems: Innovative Production Management Towards Sustainable Growth, pp. 447–454. Springer, Berlin (2015) Stich, V., Jordan, F., Birkmeier, M., Oazgil, K., Reschke, J., Diews, A.: Big data technology for resilient failure management in production systems. In: Advances in Production Management Systems: Innovative Production Management Towards Sustainable Growth, pp. 447–454. Springer, Berlin (2015)
52.
Zurück zum Zitat Agrawal, D., Das, S., El Abbadi, A.: Big data and cloud computing: current state and future opportunities. In: Proceedings of the 14th International Conference on Extending Database Technology, pp. 530–533. ACM (2011) Agrawal, D., Das, S., El Abbadi, A.: Big data and cloud computing: current state and future opportunities. In: Proceedings of the 14th International Conference on Extending Database Technology, pp. 530–533. ACM (2011)
53.
Zurück zum Zitat McDaniel, M.A.: Big-brained people are smarter: a meta-analysis of the relation-ship between in vivo brain volume and intelligence. Intelligence 33(4), 337–346 (2005)CrossRef McDaniel, M.A.: Big-brained people are smarter: a meta-analysis of the relation-ship between in vivo brain volume and intelligence. Intelligence 33(4), 337–346 (2005)CrossRef
54.
Zurück zum Zitat Tan, K.H., Zhan, Y., Ji, G., Ye, F., Chang, C.: Harvesting big data to enhance supply chain innovation capabilities: an analytic infrastructure based on deduction graph. Int. J. Prod. Econ. 165, 223–233 (2015)CrossRef Tan, K.H., Zhan, Y., Ji, G., Ye, F., Chang, C.: Harvesting big data to enhance supply chain innovation capabilities: an analytic infrastructure based on deduction graph. Int. J. Prod. Econ. 165, 223–233 (2015)CrossRef
55.
Zurück zum Zitat Chen, M., Mao, S., Liu, Y.: Big data: a survey. Mobile Netw. Appl. 19(2), 171–209 (2014)CrossRef Chen, M., Mao, S., Liu, Y.: Big data: a survey. Mobile Netw. Appl. 19(2), 171–209 (2014)CrossRef
Metadaten
Titel
Big Data: Issues, Challenges, and Techniques in Business Intelligence
verfasst von
Mudasir Ahmad Wani
Suraiya Jabin
Copyright-Jahr
2018
Verlag
Springer Singapore
DOI
https://doi.org/10.1007/978-981-10-6620-7_59

Premium Partner