Skip to main content
Erschienen in: Soft Computing 11/2020

22.05.2019 | Focus

On construction of the air pollution monitoring service with a hybrid database converter

verfasst von: Chao-Tung Yang, Shuo-Tsung Chen, Jung-Chun Liu, Pei-Lun Sun, Neil Y. Yen

Erschienen in: Soft Computing | Ausgabe 11/2020

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Air pollution has a severe impact on human health, pollution harms human health, and people start to pay attention to how to use the monitoring system in real-time recording for analysis. To maintain smooth monitoring and analysis, we have to manage the historical data separated from the incoming data. Historical data is used when we need to analyze the data, while real-time incoming data is used to visualize the real condition. To achieve this objective, we need to collect real-time data from environmental protection open data resource. However, the data might grow faster and become huge; in this case, the relational database was not designed to process a large amount of data. Therefore, we require a database technology that can handle massive volume data, that is not only Structured Query Language (NoSQL). This method raises an important point regarding how to dump the data to NoSQL without change relational database (RDB) system. Accordingly, this paper proposed an air pollution monitoring system combines Hadoop cluster to dump data from RDB to NoSQL and data backup. By this way, it will not only reduce the performance of RDB loading but also keep the service status. Dump data to NoSQL need to process without affecting the real-time monitoring of air pollution monitoring system. In this part, we focus on without interruption web service, and it can be up to 60%, through optimizing it with dump method and backup data service, MapReduce can restart the service and distribute the database when RDB is impairing. Besides that, through three different types of conversion mode, we can get the best data conversion in our system. Finally, air pollution monitoring service provides a variant of air pollution factor as an essential basis of environment detection and analysis to serve people living in a more comfortable environment.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Literatur
Zurück zum Zitat Ashraf Q, Hadaebi M (2015) Autonomic schemes for threat mitigation in internet of things. J Netw Comput Appl 49:112–127CrossRef Ashraf Q, Hadaebi M (2015) Autonomic schemes for threat mitigation in internet of things. J Netw Comput Appl 49:112–127CrossRef
Zurück zum Zitat Gadiraju KK, Verma M, Davis KC, Talaga PG (2016) Benchmarking performance for migrating a relational application to a parallel implementation. Future Gener Comput Syst 63:148CrossRef Gadiraju KK, Verma M, Davis KC, Talaga PG (2016) Benchmarking performance for migrating a relational application to a parallel implementation. Future Gener Comput Syst 63:148CrossRef
Zurück zum Zitat Gallagher E (2014) Nosql benchmark study release Gallagher E (2014) Nosql benchmark study release
Zurück zum Zitat Gil D, Song I-Y (2016) Modeling and management of big data: challenges and opportunities. Future Gener Comput Syst 63:96–99CrossRef Gil D, Song I-Y (2016) Modeling and management of big data: challenges and opportunities. Future Gener Comput Syst 63:96–99CrossRef
Zurück zum Zitat Grolinger K, Higashino WA, Tiwari A, Capretz MA (2013) Data management in cloud environments: nosql and newsql data stores. J Cloud Comput 2:22CrossRef Grolinger K, Higashino WA, Tiwari A, Capretz MA (2013) Data management in cloud environments: nosql and newsql data stores. J Cloud Comput 2:22CrossRef
Zurück zum Zitat Jain A, Bhatnagar V (2015) Crime data analysis using pig with hadoop. Proc Comput Sci 78:571–578CrossRef Jain A, Bhatnagar V (2015) Crime data analysis using pig with hadoop. Proc Comput Sci 78:571–578CrossRef
Zurück zum Zitat Kamal S, Ripon SH, Dey N, Ashour AS, Santhid V (2016) A MapReduce approach to diminish imbalance parameters for big deoxyribonucleic acid dataset. Comput Methods Progr Biomed 131:191–206CrossRef Kamal S, Ripon SH, Dey N, Ashour AS, Santhid V (2016) A MapReduce approach to diminish imbalance parameters for big deoxyribonucleic acid dataset. Comput Methods Progr Biomed 131:191–206CrossRef
Zurück zum Zitat Li C (2010) Transforming relational database into hbase: a case study. In: International conference on software engineering and service sciences, pp 683–687 Li C (2010) Transforming relational database into hbase: a case study. In: International conference on software engineering and service sciences, pp 683–687
Zurück zum Zitat Liao Y-T, Zhou J, Chen S-C, Hsu C-H, Chen W, Jiang M-F, Chung Y-C (2016) Data adapter for querying and transformation between SQL and NoSQL database. Future Gener Comput Syst 65:111CrossRef Liao Y-T, Zhou J, Chen S-C, Hsu C-H, Chen W, Jiang M-F, Chung Y-C (2016) Data adapter for querying and transformation between SQL and NoSQL database. Future Gener Comput Syst 65:111CrossRef
Zurück zum Zitat Liu J, Huang X, Liu JK (2014) Secure sharing of personal health records in cloud computing: ciphertext policy attribute based signcryption. Future Gener Comput Syst 52:67–76CrossRef Liu J, Huang X, Liu JK (2014) Secure sharing of personal health records in cloud computing: ciphertext policy attribute based signcryption. Future Gener Comput Syst 52:67–76CrossRef
Zurück zum Zitat Luo Y, Luo S, Guan J, Zhou S (2013) A ramcloud storage system based on HDFS: architecture implementation and evaluation. J Syst Softw 86:744–750CrossRef Luo Y, Luo S, Guan J, Zhou S (2013) A ramcloud storage system based on HDFS: architecture implementation and evaluation. J Syst Softw 86:744–750CrossRef
Zurück zum Zitat Maheshwari N, Nanduri R, Varma V (2012) Dynamic energy efficient data placement and cluster reconfiguration algorithm for mapreduce framework. Future Gener Comput Syst 28:119–127CrossRef Maheshwari N, Nanduri R, Varma V (2012) Dynamic energy efficient data placement and cluster reconfiguration algorithm for mapreduce framework. Future Gener Comput Syst 28:119–127CrossRef
Zurück zum Zitat Merino J, Caballero I, Rivas B, Serrano M, Piattini M (2016) A data quality in use model for big data. Future Gener Comput Syst 63:123–130CrossRef Merino J, Caballero I, Rivas B, Serrano M, Piattini M (2016) A data quality in use model for big data. Future Gener Comput Syst 63:123–130CrossRef
Zurück zum Zitat Niedermayer H, Holz R, Pahl M-O, Carle G (2009) On using home networks and cloud computing for a future internet of things. Future Internet 6152:70–80 Niedermayer H, Holz R, Pahl M-O, Carle G (2009) On using home networks and cloud computing for a future internet of things. Future Internet 6152:70–80
Zurück zum Zitat Rees R (2010) NoSQL, no problem-an introduction to NoSQL databases Rees R (2010) NoSQL, no problem-an introduction to NoSQL databases
Zurück zum Zitat Shahzad Farrukh (2014) State-of-the-art survey on cloud computing security challenges. Proc Comput Sci 37:357–362CrossRef Shahzad Farrukh (2014) State-of-the-art survey on cloud computing security challenges. Proc Comput Sci 37:357–362CrossRef
Zurück zum Zitat Sultan N (2014) Discovering the potential of cloud computing in accelerating the search for curing serious illnesses. Int J Inf Manag 34:221–225CrossRef Sultan N (2014) Discovering the potential of cloud computing in accelerating the search for curing serious illnesses. Int J Inf Manag 34:221–225CrossRef
Zurück zum Zitat Tummalapalli S, Rao Machavarapu V (2016) Managing mysql cluster data using cloudera impala. Proc Comput Sci 85:463–474CrossRef Tummalapalli S, Rao Machavarapu V (2016) Managing mysql cluster data using cloudera impala. Proc Comput Sci 85:463–474CrossRef
Zurück zum Zitat Wang H, Xua Z, Fujita H, Liud S (2016) Towards felicitous decision making: an overview on challenges and trends of big data. Inf Sci 367–368:747–765CrossRef Wang H, Xua Z, Fujita H, Liud S (2016) Towards felicitous decision making: an overview on challenges and trends of big data. Inf Sci 367–368:747–765CrossRef
Zurück zum Zitat Yang CT, Chen ST, Walter D, Wang YT, Kristiani E (2018) Implementation of an intelligent indoor environmental monitoring and management system in cloud. Future Gener Comput Syst 96:731CrossRef Yang CT, Chen ST, Walter D, Wang YT, Kristiani E (2018) Implementation of an intelligent indoor environmental monitoring and management system in cloud. Future Gener Comput Syst 96:731CrossRef
Zurück zum Zitat Zhang F, Cao J, Khan SU, Li K, Hwang K (2015) A task-level adaptive mapreduce framework for real-time streaming data in healthcare application. Future Gener Comput Syst 43:149–160CrossRef Zhang F, Cao J, Khan SU, Li K, Hwang K (2015) A task-level adaptive mapreduce framework for real-time streaming data in healthcare application. Future Gener Comput Syst 43:149–160CrossRef
Metadaten
Titel
On construction of the air pollution monitoring service with a hybrid database converter
verfasst von
Chao-Tung Yang
Shuo-Tsung Chen
Jung-Chun Liu
Pei-Lun Sun
Neil Y. Yen
Publikationsdatum
22.05.2019
Verlag
Springer Berlin Heidelberg
Erschienen in
Soft Computing / Ausgabe 11/2020
Print ISSN: 1432-7643
Elektronische ISSN: 1433-7479
DOI
https://doi.org/10.1007/s00500-019-04079-z

Weitere Artikel der Ausgabe 11/2020

Soft Computing 11/2020 Zur Ausgabe