Skip to main content
Top

2020 | OriginalPaper | Chapter

Coordinated Placement of Meteorological Workflows and Data with Privacy Conflict Protection

Authors : Tao Huang, Shengjun Xue, Yumei Hu, Qing Yang, Yachong Tian, Dan Zeng

Published in: Cloud Computing, Smart Grid and Innovative Frontiers in Telecommunications

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Cloud computing is cited by various industries for its powerful computing power to solve complex calculations in the industry. The massive data of meteorological department has typical big data characteristics. Therefore, cloud computing has been gradually applied to deal with a large number of meteorological -services. Cloud computing increases the computational speed of meteorological services, but data transmission between nodes also generates additional data transmission time. At the same time, based on cloud computing technology, a large number of computing tasks are cooperatively processed by multiple nodes, so improving the resource utilization of each node is also an important evaluation indicator. In addition, with the increase of data confidentiality, there are some data conflicts between some data, so the conflicting data should be avoided being placed on the same node. To cope with this challenge, the meteorological application is modeled and a collaborative placement method for tasks and data based on Differential Evolution algorithm (CPDE) is proposed. The Non-dominated Sorting Differential Evolution (NSDE) algorithm is used to jointly optimize the average data access time, the average resource utilization of nodes and the data conflict degree. Finally, a large number of experimental evaluations and comparative analyses verify the efficiency of our proposed CPDE method.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
2.
go back to reference Lin, B., et al.: A time-driven data placement strategy for a scientific workflow combining edge computing and cloud computing. IEEE Trans. Ind. Inf. 15(7), 4254–4265 (2019)CrossRef Lin, B., et al.: A time-driven data placement strategy for a scientific workflow combining edge computing and cloud computing. IEEE Trans. Ind. Inf. 15(7), 4254–4265 (2019)CrossRef
3.
go back to reference Tang, J., Tang, X., Yuan, J.: Traffic-optimized data placement for social media. IEEE Trans. Multimedia 20, 1008–1023 (2017)CrossRef Tang, J., Tang, X., Yuan, J.: Traffic-optimized data placement for social media. IEEE Trans. Multimedia 20, 1008–1023 (2017)CrossRef
4.
go back to reference Li, X., et al.: A novel workflow-level data placement strategy for data-sharing scientific cloud workflows. IEEE Trans. Serv. Comput. 12, 370–383 (2019)CrossRef Li, X., et al.: A novel workflow-level data placement strategy for data-sharing scientific cloud workflows. IEEE Trans. Serv. Comput. 12, 370–383 (2019)CrossRef
5.
go back to reference Dong, Y., Yang, Y., Liu, X., Chen, J.: A data placement strategy in scientific cloud workflows. Future Gener. Comput. Syst. 26(8), 1200–1214 (2010)CrossRef Dong, Y., Yang, Y., Liu, X., Chen, J.: A data placement strategy in scientific cloud workflows. Future Gener. Comput. Syst. 26(8), 1200–1214 (2010)CrossRef
6.
go back to reference Deng, K., Kong, L., Song, J., Ren, K., Dong, Y.: A weighted k-means clustering based co-scheduling strategy towards efficient execution of scientific workflows in collaborative cloud environments. In: IEEE Ninth International Conference on Dependable, Autonomic and Secure Computing, DASC 2011, 12–14 December 2011, Sydney, Australia, pp. 547–554 (2011) Deng, K., Kong, L., Song, J., Ren, K., Dong, Y.: A weighted k-means clustering based co-scheduling strategy towards efficient execution of scientific workflows in collaborative cloud environments. In: IEEE Ninth International Conference on Dependable, Autonomic and Secure Computing, DASC 2011, 12–14 December 2011, Sydney, Australia, pp. 547–554 (2011)
7.
go back to reference Kim, H., Kim, Y.: An adaptive data placement strategy in scientific workflows over cloud computing environments. In: NOMS 2018–2018 IEEE/IFIP Network Operations and Management Symposium, pp. 1–5 (2018) Kim, H., Kim, Y.: An adaptive data placement strategy in scientific workflows over cloud computing environments. In: NOMS 2018–2018 IEEE/IFIP Network Operations and Management Symposium, pp. 1–5 (2018)
8.
go back to reference Kashlev, A., Lu, S., Ebrahimi, M., Mohan, A.: BDAP: a big data placement strategy for cloud-based scientific workflows. In: 2015 IEEE First International Conference on Big Data Computing Service and Applications, pp. 813–820 (2015) Kashlev, A., Lu, S., Ebrahimi, M., Mohan, A.: BDAP: a big data placement strategy for cloud-based scientific workflows. In: 2015 IEEE First International Conference on Big Data Computing Service and Applications, pp. 813–820 (2015)
9.
go back to reference Liao, Z., Yu, B., Liu, K., Wang, J.: Learning-based adaptive data placement for low latency in data center networks. In: IEEE 43rd Conference on Local Computer Networks (2018) Liao, Z., Yu, B., Liu, K., Wang, J.: Learning-based adaptive data placement for low latency in data center networks. In: IEEE 43rd Conference on Local Computer Networks (2018)
10.
go back to reference Whaiduzzaman, M., Gani, A., Naveed, A. PEFC: performance enhancement framework for cloudlet in mobile cloud computing. In: IEEE-ROMA-2014, pp. 224–229 (2014) Whaiduzzaman, M., Gani, A., Naveed, A. PEFC: performance enhancement framework for cloudlet in mobile cloud computing. In: IEEE-ROMA-2014, pp. 224–229 (2014)
11.
go back to reference Xu, X. et al.: A multi-objective data placement method for IoT applications over big data using NSGA-II. In: IEEE International Conference on Internet of Things (iThings) and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber, Physical and Social Computing (CPSCom) and IEEE Smart Data (SmartData), pp. 503–509 (2018) Xu, X. et al.: A multi-objective data placement method for IoT applications over big data using NSGA-II. In: IEEE International Conference on Internet of Things (iThings) and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber, Physical and Social Computing (CPSCom) and IEEE Smart Data (SmartData), pp. 503–509 (2018)
13.
go back to reference Cui, L., Zhang, J., Yue, L., Shi, Y., Li, H., Yuan, D.: A genetic algorithm based data replica placement strategy for scientific applications in clouds. IEEE Trans. Serv. Comput. 11(4), 727–739 (2015)CrossRef Cui, L., Zhang, J., Yue, L., Shi, Y., Li, H., Yuan, D.: A genetic algorithm based data replica placement strategy for scientific applications in clouds. IEEE Trans. Serv. Comput. 11(4), 727–739 (2015)CrossRef
14.
go back to reference Kang, S., Veeravalli, B., Aung, K.M.M.: A security-aware data placement mechanism for big data cloud storage systems. In: IEEE 2nd International Conference on Big Data Security on Cloud (BigDataSecurity), IEEE International Conference on High Performance and Smart Computing (HPSC) and IEEE International Conference on Intelligent Data and Security (IDS), pp. 327–332 (2016) Kang, S., Veeravalli, B., Aung, K.M.M.: A security-aware data placement mechanism for big data cloud storage systems. In: IEEE 2nd International Conference on Big Data Security on Cloud (BigDataSecurity), IEEE International Conference on High Performance and Smart Computing (HPSC) and IEEE International Conference on Intelligent Data and Security (IDS), pp. 327–332 (2016)
15.
go back to reference Wang, R., Yiwen, L., Zhu, K., Hao, J., Wang, P., Cao, Y.: An optimal task placement strategy in geo-distributed data centers involving renewable energy. IEEE Access 6, 61948–61958 (2018)CrossRef Wang, R., Yiwen, L., Zhu, K., Hao, J., Wang, P., Cao, Y.: An optimal task placement strategy in geo-distributed data centers involving renewable energy. IEEE Access 6, 61948–61958 (2018)CrossRef
16.
go back to reference Liu, L., Song, J., Wang, H.: BRPS: a big data placement strategy for data intensive applications. In: IEEE 16th International Conference on Data Mining Workshops (ICDMW), pp. 813–820 (2016) Liu, L., Song, J., Wang, H.: BRPS: a big data placement strategy for data intensive applications. In: IEEE 16th International Conference on Data Mining Workshops (ICDMW), pp. 813–820 (2016)
17.
go back to reference Shu, J., Liu, X., Jia, X., Yang, K., Deng, R.H.: Anonymous privacy-preserving task matching in crowdsourcing. IEEE Internet Things J. 5(4), 3068–3078 (2018)CrossRef Shu, J., Liu, X., Jia, X., Yang, K., Deng, R.H.: Anonymous privacy-preserving task matching in crowdsourcing. IEEE Internet Things J. 5(4), 3068–3078 (2018)CrossRef
18.
go back to reference Chi, Z., Wang, Y., Huang, Y., Tong, X.: The novel location privacy-preserving CKD for mobile systems. IEEE Access 6, 5678–5687 (2018)CrossRef Chi, Z., Wang, Y., Huang, Y., Tong, X.: The novel location privacy-preserving CKD for mobile systems. IEEE Access 6, 5678–5687 (2018)CrossRef
Metadata
Title
Coordinated Placement of Meteorological Workflows and Data with Privacy Conflict Protection
Authors
Tao Huang
Shengjun Xue
Yumei Hu
Qing Yang
Yachong Tian
Dan Zeng
Copyright Year
2020
DOI
https://doi.org/10.1007/978-3-030-48513-9_44

Premium Partner