Skip to main content
Top
Published in: Soft Computing 11/2020

22-05-2019 | Focus

On construction of the air pollution monitoring service with a hybrid database converter

Authors: Chao-Tung Yang, Shuo-Tsung Chen, Jung-Chun Liu, Pei-Lun Sun, Neil Y. Yen

Published in: Soft Computing | Issue 11/2020

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Air pollution has a severe impact on human health, pollution harms human health, and people start to pay attention to how to use the monitoring system in real-time recording for analysis. To maintain smooth monitoring and analysis, we have to manage the historical data separated from the incoming data. Historical data is used when we need to analyze the data, while real-time incoming data is used to visualize the real condition. To achieve this objective, we need to collect real-time data from environmental protection open data resource. However, the data might grow faster and become huge; in this case, the relational database was not designed to process a large amount of data. Therefore, we require a database technology that can handle massive volume data, that is not only Structured Query Language (NoSQL). This method raises an important point regarding how to dump the data to NoSQL without change relational database (RDB) system. Accordingly, this paper proposed an air pollution monitoring system combines Hadoop cluster to dump data from RDB to NoSQL and data backup. By this way, it will not only reduce the performance of RDB loading but also keep the service status. Dump data to NoSQL need to process without affecting the real-time monitoring of air pollution monitoring system. In this part, we focus on without interruption web service, and it can be up to 60%, through optimizing it with dump method and backup data service, MapReduce can restart the service and distribute the database when RDB is impairing. Besides that, through three different types of conversion mode, we can get the best data conversion in our system. Finally, air pollution monitoring service provides a variant of air pollution factor as an essential basis of environment detection and analysis to serve people living in a more comfortable environment.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Literature
go back to reference Ashraf Q, Hadaebi M (2015) Autonomic schemes for threat mitigation in internet of things. J Netw Comput Appl 49:112–127CrossRef Ashraf Q, Hadaebi M (2015) Autonomic schemes for threat mitigation in internet of things. J Netw Comput Appl 49:112–127CrossRef
go back to reference Gadiraju KK, Verma M, Davis KC, Talaga PG (2016) Benchmarking performance for migrating a relational application to a parallel implementation. Future Gener Comput Syst 63:148CrossRef Gadiraju KK, Verma M, Davis KC, Talaga PG (2016) Benchmarking performance for migrating a relational application to a parallel implementation. Future Gener Comput Syst 63:148CrossRef
go back to reference Gallagher E (2014) Nosql benchmark study release Gallagher E (2014) Nosql benchmark study release
go back to reference Gil D, Song I-Y (2016) Modeling and management of big data: challenges and opportunities. Future Gener Comput Syst 63:96–99CrossRef Gil D, Song I-Y (2016) Modeling and management of big data: challenges and opportunities. Future Gener Comput Syst 63:96–99CrossRef
go back to reference Grolinger K, Higashino WA, Tiwari A, Capretz MA (2013) Data management in cloud environments: nosql and newsql data stores. J Cloud Comput 2:22CrossRef Grolinger K, Higashino WA, Tiwari A, Capretz MA (2013) Data management in cloud environments: nosql and newsql data stores. J Cloud Comput 2:22CrossRef
go back to reference Jain A, Bhatnagar V (2015) Crime data analysis using pig with hadoop. Proc Comput Sci 78:571–578CrossRef Jain A, Bhatnagar V (2015) Crime data analysis using pig with hadoop. Proc Comput Sci 78:571–578CrossRef
go back to reference Kamal S, Ripon SH, Dey N, Ashour AS, Santhid V (2016) A MapReduce approach to diminish imbalance parameters for big deoxyribonucleic acid dataset. Comput Methods Progr Biomed 131:191–206CrossRef Kamal S, Ripon SH, Dey N, Ashour AS, Santhid V (2016) A MapReduce approach to diminish imbalance parameters for big deoxyribonucleic acid dataset. Comput Methods Progr Biomed 131:191–206CrossRef
go back to reference Li C (2010) Transforming relational database into hbase: a case study. In: International conference on software engineering and service sciences, pp 683–687 Li C (2010) Transforming relational database into hbase: a case study. In: International conference on software engineering and service sciences, pp 683–687
go back to reference Liao Y-T, Zhou J, Chen S-C, Hsu C-H, Chen W, Jiang M-F, Chung Y-C (2016) Data adapter for querying and transformation between SQL and NoSQL database. Future Gener Comput Syst 65:111CrossRef Liao Y-T, Zhou J, Chen S-C, Hsu C-H, Chen W, Jiang M-F, Chung Y-C (2016) Data adapter for querying and transformation between SQL and NoSQL database. Future Gener Comput Syst 65:111CrossRef
go back to reference Liu J, Huang X, Liu JK (2014) Secure sharing of personal health records in cloud computing: ciphertext policy attribute based signcryption. Future Gener Comput Syst 52:67–76CrossRef Liu J, Huang X, Liu JK (2014) Secure sharing of personal health records in cloud computing: ciphertext policy attribute based signcryption. Future Gener Comput Syst 52:67–76CrossRef
go back to reference Luo Y, Luo S, Guan J, Zhou S (2013) A ramcloud storage system based on HDFS: architecture implementation and evaluation. J Syst Softw 86:744–750CrossRef Luo Y, Luo S, Guan J, Zhou S (2013) A ramcloud storage system based on HDFS: architecture implementation and evaluation. J Syst Softw 86:744–750CrossRef
go back to reference Maheshwari N, Nanduri R, Varma V (2012) Dynamic energy efficient data placement and cluster reconfiguration algorithm for mapreduce framework. Future Gener Comput Syst 28:119–127CrossRef Maheshwari N, Nanduri R, Varma V (2012) Dynamic energy efficient data placement and cluster reconfiguration algorithm for mapreduce framework. Future Gener Comput Syst 28:119–127CrossRef
go back to reference Merino J, Caballero I, Rivas B, Serrano M, Piattini M (2016) A data quality in use model for big data. Future Gener Comput Syst 63:123–130CrossRef Merino J, Caballero I, Rivas B, Serrano M, Piattini M (2016) A data quality in use model for big data. Future Gener Comput Syst 63:123–130CrossRef
go back to reference Niedermayer H, Holz R, Pahl M-O, Carle G (2009) On using home networks and cloud computing for a future internet of things. Future Internet 6152:70–80 Niedermayer H, Holz R, Pahl M-O, Carle G (2009) On using home networks and cloud computing for a future internet of things. Future Internet 6152:70–80
go back to reference Rees R (2010) NoSQL, no problem-an introduction to NoSQL databases Rees R (2010) NoSQL, no problem-an introduction to NoSQL databases
go back to reference Shahzad Farrukh (2014) State-of-the-art survey on cloud computing security challenges. Proc Comput Sci 37:357–362CrossRef Shahzad Farrukh (2014) State-of-the-art survey on cloud computing security challenges. Proc Comput Sci 37:357–362CrossRef
go back to reference Sultan N (2014) Discovering the potential of cloud computing in accelerating the search for curing serious illnesses. Int J Inf Manag 34:221–225CrossRef Sultan N (2014) Discovering the potential of cloud computing in accelerating the search for curing serious illnesses. Int J Inf Manag 34:221–225CrossRef
go back to reference Tummalapalli S, Rao Machavarapu V (2016) Managing mysql cluster data using cloudera impala. Proc Comput Sci 85:463–474CrossRef Tummalapalli S, Rao Machavarapu V (2016) Managing mysql cluster data using cloudera impala. Proc Comput Sci 85:463–474CrossRef
go back to reference Wang H, Xua Z, Fujita H, Liud S (2016) Towards felicitous decision making: an overview on challenges and trends of big data. Inf Sci 367–368:747–765CrossRef Wang H, Xua Z, Fujita H, Liud S (2016) Towards felicitous decision making: an overview on challenges and trends of big data. Inf Sci 367–368:747–765CrossRef
go back to reference Yang CT, Chen ST, Walter D, Wang YT, Kristiani E (2018) Implementation of an intelligent indoor environmental monitoring and management system in cloud. Future Gener Comput Syst 96:731CrossRef Yang CT, Chen ST, Walter D, Wang YT, Kristiani E (2018) Implementation of an intelligent indoor environmental monitoring and management system in cloud. Future Gener Comput Syst 96:731CrossRef
go back to reference Zhang F, Cao J, Khan SU, Li K, Hwang K (2015) A task-level adaptive mapreduce framework for real-time streaming data in healthcare application. Future Gener Comput Syst 43:149–160CrossRef Zhang F, Cao J, Khan SU, Li K, Hwang K (2015) A task-level adaptive mapreduce framework for real-time streaming data in healthcare application. Future Gener Comput Syst 43:149–160CrossRef
Metadata
Title
On construction of the air pollution monitoring service with a hybrid database converter
Authors
Chao-Tung Yang
Shuo-Tsung Chen
Jung-Chun Liu
Pei-Lun Sun
Neil Y. Yen
Publication date
22-05-2019
Publisher
Springer Berlin Heidelberg
Published in
Soft Computing / Issue 11/2020
Print ISSN: 1432-7643
Electronic ISSN: 1433-7479
DOI
https://doi.org/10.1007/s00500-019-04079-z

Other articles of this Issue 11/2020

Soft Computing 11/2020 Go to the issue

Premium Partner