Skip to main content
Top
Published in: Neural Computing and Applications 9/2019

25-04-2018 | S.I. : Emergence in Human-like Intelligence towards Cyber-Physical Systems

Application research of multi-objective Artificial Bee Colony optimization algorithm for parameters calibration of hydrological model

Authors: Jiuyuan Huo, Liqun Liu

Published in: Neural Computing and Applications | Issue 9/2019

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Parameter optimization methods for hydrological models have an important impact for the hydrological forecasting. To achieve the parameters’ optimization and calibration for the distributed, conceptual watershed Xinanjiang model effectively and accurately, a multi-objective Artificial Bee Colony algorithm named RMOABC which adopts the mechanisms of regulation operator and Adaptive Grid is introduced in the paper. In the evolution of the algorithm, the regulation operator mechanism can balance the weights of local search and global search, and the Adaptive Grid mechanism is utilized to evaluate and maintain the Pareto solutions in the external archive. In the experiments, three commonly used multi-objective optimization algorithms, the NSGA-II, the ε-MOEA and the SMPSO, with the RMOABC algorithm were applied in Heihe River Basin, and the parameter optimization problem of Xinanjiang hydrological model was taken as the application case for long-term runoff prediction to validate and compare their performance. The experiments results showed the RMOABC algorithm can provide more comprehensive and reliable parameters sets for practical hydrological forecasting in the study area with lower execution time.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference James W (2000) Simulation modeling for watershed management. Springer, New York James W (2000) Simulation modeling for watershed management. Springer, New York
2.
go back to reference Ye J, Ding Y (2018) Controllable keyword search scheme supporting multiple users. Future Gener Comput Syst 81:433–442CrossRef Ye J, Ding Y (2018) Controllable keyword search scheme supporting multiple users. Future Gener Comput Syst 81:433–442CrossRef
3.
go back to reference Carpenter TM, Georgakakos KP (2004) Impacts of parametric and radar rainfall uncertainty on the ensemble streamflow simulations of a distributed hydrologic model. J Hydrol 298:27–60CrossRef Carpenter TM, Georgakakos KP (2004) Impacts of parametric and radar rainfall uncertainty on the ensemble streamflow simulations of a distributed hydrologic model. J Hydrol 298:27–60CrossRef
4.
go back to reference Yang BB, Wang WC (2010) Comparison between multi-objective evolutionary algorithms for calibration of Xinanjiang model. J China Hydrol 30(3):38–42 Yang BB, Wang WC (2010) Comparison between multi-objective evolutionary algorithms for calibration of Xinanjiang model. J China Hydrol 30(3):38–42
5.
go back to reference Tavares LG, Lopes HS, Lima CRE (2009) A study of topology in insular parallel genetic algorithms. In: World congress on nature and biologically inspired computing Tavares LG, Lopes HS, Lima CRE (2009) A study of topology in insular parallel genetic algorithms. In: World congress on nature and biologically inspired computing
6.
go back to reference Kennedy J, Eberhart R (1995) Particle swarm optimization. In: Proceedings of IEEE international conference on neural networks, pp 1942–1948 Kennedy J, Eberhart R (1995) Particle swarm optimization. In: Proceedings of IEEE international conference on neural networks, pp 1942–1948
7.
go back to reference Fan QQ, Yan XF (2015) Self-adaptive differential evolution algorithm with discrete mutation control parameters. Expert Syst Appl 42:1551–1572CrossRef Fan QQ, Yan XF (2015) Self-adaptive differential evolution algorithm with discrete mutation control parameters. Expert Syst Appl 42:1551–1572CrossRef
8.
go back to reference Karaboga D (2005) An idea based on honey bee swarm for numerical optimization. Technical Report TR06, Computer Engineering, Department, Erciyes University, Turkey Karaboga D (2005) An idea based on honey bee swarm for numerical optimization. Technical Report TR06, Computer Engineering, Department, Erciyes University, Turkey
10.
go back to reference Gozde H, Taplamacioglu MC (2001) Comparative performance analysis of artificial bee colony algorithm for automatic voltage regulator (AVR) system. J Frankl Inst Eng Appl Math 348(8):1927–1946CrossRef Gozde H, Taplamacioglu MC (2001) Comparative performance analysis of artificial bee colony algorithm for automatic voltage regulator (AVR) system. J Frankl Inst Eng Appl Math 348(8):1927–1946CrossRef
11.
go back to reference Sonmez M (2011) Artificial bee colony algorithm for optimization of truss structures. Appl Soft Comput 11(2):2406–2418CrossRef Sonmez M (2011) Artificial bee colony algorithm for optimization of truss structures. Appl Soft Comput 11(2):2406–2418CrossRef
12.
go back to reference Li ZJ, Xin PL, Tang JH (2011) Study of the Xinanjiang model parameter calibration. J Hydrol Eng 18:1513–1521 Li ZJ, Xin PL, Tang JH (2011) Study of the Xinanjiang model parameter calibration. J Hydrol Eng 18:1513–1521
13.
go back to reference Huo JY, Zhang YN, Zhao HX (2014) Improved artificial bee colony algorithm and its application in parameter estimation. Comput Eng 40:166–171 Huo JY, Zhang YN, Zhao HX (2014) Improved artificial bee colony algorithm and its application in parameter estimation. Comput Eng 40:166–171
14.
go back to reference Li ZJ, Zhang H, Cheng Y, Kan GY (2013) Application of coupling global optimization of single-objective algorithm with multi-objective algorithm to calibration of Xinanjiang model parameters. J Hydroelectr Eng 32(5):6–12 Li ZJ, Zhang H, Cheng Y, Kan GY (2013) Application of coupling global optimization of single-objective algorithm with multi-objective algorithm to calibration of Xinanjiang model parameters. J Hydroelectr Eng 32(5):6–12
15.
go back to reference Huo JY, Liu LQ, Zhang YN (2016) Comparative research of optimization algorithms for parameters calibration of watershed hydrological model. J Comput Methods Sci Eng 16(3):653–669 Huo JY, Liu LQ, Zhang YN (2016) Comparative research of optimization algorithms for parameters calibration of watershed hydrological model. J Comput Methods Sci Eng 16(3):653–669
16.
go back to reference Deb K (2001) Multi-objective optimization using evolutionary algorithms. Wiley, ChichesterMATH Deb K (2001) Multi-objective optimization using evolutionary algorithms. Wiley, ChichesterMATH
17.
go back to reference Pareto V (1968) The rise and fall of the elites. N. J., Bedminster Press, Totowa Pareto V (1968) The rise and fall of the elites. N. J., Bedminster Press, Totowa
18.
go back to reference Bekele EG, Nicklow JW (2007) Multi-objective optimal control Model for watershed management using SWAT and NSGA-II. In: World environmental and water resources congress, pp 1–10 Bekele EG, Nicklow JW (2007) Multi-objective optimal control Model for watershed management using SWAT and NSGA-II. In: World environmental and water resources congress, pp 1–10
19.
go back to reference Huo JY, Liu LQ (2017) An improved multi-objective artificial bee colony optimization algorithm with regulation operators. Information 8:18CrossRef Huo JY, Liu LQ (2017) An improved multi-objective artificial bee colony optimization algorithm with regulation operators. Information 8:18CrossRef
20.
go back to reference Deb K, Pratap A, Agarwal S, Meyarivan T (2002) A fast and elitist multi-objective genetic algorithm: NSGA-II. IEEE Trans Evol Comput 6:182–197CrossRef Deb K, Pratap A, Agarwal S, Meyarivan T (2002) A fast and elitist multi-objective genetic algorithm: NSGA-II. IEEE Trans Evol Comput 6:182–197CrossRef
21.
go back to reference Deb K, Mohan M, Mishra S (2003) A fast multi-objective evolutionary algorithm for finding well-spread pareto-optimal solutions. KanGAL Report No 2003002 Deb K, Mohan M, Mishra S (2003) A fast multi-objective evolutionary algorithm for finding well-spread pareto-optimal solutions. KanGAL Report No 2003002
22.
go back to reference Nebro AJ, Durillo JJ, Garcia-Nieto J, Coello Coello CA, Luna F, Alba E (2009) SMPSO: A new PSO-based metaheuristic for multi-objective optimization. In: 2009 IEEE symposium on computational intelligence in multicriteria decision-making, pp 66-73 Nebro AJ, Durillo JJ, Garcia-Nieto J, Coello Coello CA, Luna F, Alba E (2009) SMPSO: A new PSO-based metaheuristic for multi-objective optimization. In: 2009 IEEE symposium on computational intelligence in multicriteria decision-making, pp 66-73
23.
go back to reference Carlos A, Coello C, Gregorio TP, Maximino SL (2004) Handling multiple objectives with particle swarm optimization. IEEE Trans Evol Comput 8:256–279CrossRef Carlos A, Coello C, Gregorio TP, Maximino SL (2004) Handling multiple objectives with particle swarm optimization. IEEE Trans Evol Comput 8:256–279CrossRef
24.
go back to reference Yapo PO, Gupta HV, Sorooshian S (1998) Multi-objective global optimization for hydrologic models. J Hydrol 204:83–97CrossRef Yapo PO, Gupta HV, Sorooshian S (1998) Multi-objective global optimization for hydrologic models. J Hydrol 204:83–97CrossRef
25.
go back to reference Zhang L, Cui GB (2009) Automatic calibration of a hydrological model using multi-objective particle swarm optimization and TOPSIS. In: Computer science and information engineering, 2009 WRI world congress, vol 4, pp 617–621 Zhang L, Cui GB (2009) Automatic calibration of a hydrological model using multi-objective particle swarm optimization and TOPSIS. In: Computer science and information engineering, 2009 WRI world congress, vol 4, pp 617–621
26.
go back to reference Amir MSII, Khan MMK, Sharma MGRRH, Akram F (2013) Automatic multi-objective calibration of a rainfall runoff model for the Fitzroy Basin, Queensland, Australia. Int J Environ Sci Dev 4:311–315CrossRef Amir MSII, Khan MMK, Sharma MGRRH, Akram F (2013) Automatic multi-objective calibration of a rainfall runoff model for the Fitzroy Basin, Queensland, Australia. Int J Environ Sci Dev 4:311–315CrossRef
28.
go back to reference Wang Y, Brubaker K (2015) Multi-objective model auto-calibration and reduced parameterization: exploiting gradient-based optimization tool for a hydrologic model. Environ Model Softw 70:1–15CrossRef Wang Y, Brubaker K (2015) Multi-objective model auto-calibration and reduced parameterization: exploiting gradient-based optimization tool for a hydrologic model. Environ Model Softw 70:1–15CrossRef
29.
go back to reference Liao X, Zhou J, Ouyang S et al (2014) Multi-objective artificial bee colony algorithm for long-term scheduling of hydropower system: a case study of china. Water Util J 7:13–23 Liao X, Zhou J, Ouyang S et al (2014) Multi-objective artificial bee colony algorithm for long-term scheduling of hydropower system: a case study of china. Water Util J 7:13–23
30.
go back to reference Pérez CJ, Vega-Rodríguez MA, Reder K, Flörke M (2017) A Multi-Objective Artificial Bee Colony-based optimization approach to design water quality monitoring networks in river basins. J Clean Prod 166:579–589CrossRef Pérez CJ, Vega-Rodríguez MA, Reder K, Flörke M (2017) A Multi-Objective Artificial Bee Colony-based optimization approach to design water quality monitoring networks in river basins. J Clean Prod 166:579–589CrossRef
31.
go back to reference Tang Y, Reed P, Wagener T (2006) How effective and efficient are multi-objective evolutionary algorithms at hydrologic model calibration. Hydrol Earth Syst Sci 10:289–307CrossRef Tang Y, Reed P, Wagener T (2006) How effective and efficient are multi-objective evolutionary algorithms at hydrologic model calibration. Hydrol Earth Syst Sci 10:289–307CrossRef
32.
go back to reference Efstratiadis A, Koutsoyiannis D (2010) One decade of multi-objective calibration approaches in hydrological modelling: a review. Hydrol Sci J 55:58–78CrossRef Efstratiadis A, Koutsoyiannis D (2010) One decade of multi-objective calibration approaches in hydrological modelling: a review. Hydrol Sci J 55:58–78CrossRef
33.
go back to reference Chen GD, Li X, Wang SG (2009) Heihe River Basin: integrated management research of water. In: Ecological, economic systems. Science Press, Beijing, pp 336–408 Chen GD, Li X, Wang SG (2009) Heihe River Basin: integrated management research of water. In: Ecological, economic systems. Science Press, Beijing, pp 336–408
34.
go back to reference Huo JY, Liu LQ, Zhang YN (2018) An improved multi-cores parallel artificial Bee colony optimization algorithm for parameters calibration of hydrological model. Future Gener Comput Syst 81C:492–504CrossRef Huo JY, Liu LQ, Zhang YN (2018) An improved multi-cores parallel artificial Bee colony optimization algorithm for parameters calibration of hydrological model. Future Gener Comput Syst 81C:492–504CrossRef
35.
go back to reference Wang SG (2010) Studies on parameter estimation methods for hydrological model and associated uncertainties. Yellow River Water Conservancy Press, Henan, pp 5–142 Wang SG (2010) Studies on parameter estimation methods for hydrological model and associated uncertainties. Yellow River Water Conservancy Press, Henan, pp 5–142
36.
go back to reference Zhao RJ (1992) Watershed hydrological model—Xinanjiang model and Northern Shanxi model. China Water Power Press, Beijing, pp 126–127 Zhao RJ (1992) Watershed hydrological model—Xinanjiang model and Northern Shanxi model. China Water Power Press, Beijing, pp 126–127
37.
go back to reference Lu HS, Hou T, Horton R (2013) The streamflow estimation using the Xinanjiang rainfall runoff model and dual state-parameter estimation method. J Hydrol 480:102–114CrossRef Lu HS, Hou T, Horton R (2013) The streamflow estimation using the Xinanjiang rainfall runoff model and dual state-parameter estimation method. J Hydrol 480:102–114CrossRef
38.
go back to reference Horn J (1997) Multicriterion decision making. In: Bäck T, Fogel D, Michalewicz Z (eds) Handbook of evolutionary computation, vol 1. Oxford University Press, London, pp F1.9:1–F1.9:15 Horn J (1997) Multicriterion decision making. In: Bäck T, Fogel D, Michalewicz Z (eds) Handbook of evolutionary computation, vol 1. Oxford University Press, London, pp F1.9:1–F1.9:15
39.
go back to reference Raquel CR, Naval PC (2005) An effective use of crowding distance in multiobjective particle swarm optimization. In: Genetic and evolutionary computation conference, GECCO 2005, Washington DC, USA, pp 257–264 Raquel CR, Naval PC (2005) An effective use of crowding distance in multiobjective particle swarm optimization. In: Genetic and evolutionary computation conference, GECCO 2005, Washington DC, USA, pp 257–264
40.
go back to reference Bastos-Filho CJA, Figueiredo EMN, Martins-Filho JF, Chaves DAR, Segatto MEV, Cani S, Pontes MJ (2011) Design of distributed optical-fiber Raman amplifiers using multi-objective particle swarm optimization. J Microw Optoelectron Electromagn Appl 10(2):323–336CrossRef Bastos-Filho CJA, Figueiredo EMN, Martins-Filho JF, Chaves DAR, Segatto MEV, Cani S, Pontes MJ (2011) Design of distributed optical-fiber Raman amplifiers using multi-objective particle swarm optimization. J Microw Optoelectron Electromagn Appl 10(2):323–336CrossRef
41.
go back to reference da Silva Maximiano M, Vega-Rodríguez MA, Gómez-Pulido JA, Sánchez-Pérez JM (2013) A new multiobjective artificial bee colony algorithm to solve a real-world frequency assignment problem. Neural Comput Appl 22(7–8):1447–1459CrossRef da Silva Maximiano M, Vega-Rodríguez MA, Gómez-Pulido JA, Sánchez-Pérez JM (2013) A new multiobjective artificial bee colony algorithm to solve a real-world frequency assignment problem. Neural Comput Appl 22(7–8):1447–1459CrossRef
42.
go back to reference Knowles JD, Corne DW (2000) Approximating the non-dominated front using the Pareto archived evolution strategy. Evolut Comput 8:149–172CrossRef Knowles JD, Corne DW (2000) Approximating the non-dominated front using the Pareto archived evolution strategy. Evolut Comput 8:149–172CrossRef
43.
go back to reference Zhu GP, Kwong S (2010) Gbest-guided artificial bee colony algorithm for numerical function optimization. Appl Math Comput 217:3166–3173MathSciNetMATH Zhu GP, Kwong S (2010) Gbest-guided artificial bee colony algorithm for numerical function optimization. Appl Math Comput 217:3166–3173MathSciNetMATH
44.
go back to reference de Vos NJ, Rientjes THM (2007) Multi-objective performance comparison of an artificial neural network and a conceptual rainfall-runoff model. Hydrol Sci J 52(3):397–413CrossRef de Vos NJ, Rientjes THM (2007) Multi-objective performance comparison of an artificial neural network and a conceptual rainfall-runoff model. Hydrol Sci J 52(3):397–413CrossRef
45.
go back to reference Shafii M, De Smedt F (2009) Multi-objective calibration of a distributed hydrological model (WetSpa) using a genetic algorithm. Hydrol Earth Syst Sci 13:2137–2149CrossRef Shafii M, De Smedt F (2009) Multi-objective calibration of a distributed hydrological model (WetSpa) using a genetic algorithm. Hydrol Earth Syst Sci 13:2137–2149CrossRef
46.
go back to reference Nebro AJ, Durill JJ, Coello CAC (2013) Analysis of leader selection strategies in a multi-objective particle swarm optimizer. In: Evolutionary computation, IEEE, pp 3153–3160 Nebro AJ, Durill JJ, Coello CAC (2013) Analysis of leader selection strategies in a multi-objective particle swarm optimizer. In: Evolutionary computation, IEEE, pp 3153–3160
48.
go back to reference Xu Z, Liu Y, Zhang H, Luo X, Mei L, Hu C (2017) Building the multi-modal storytelling of urban emergency events based on Crowdsensing of social media analytics. MONET 22(2):218–227 Xu Z, Liu Y, Zhang H, Luo X, Mei L, Hu C (2017) Building the multi-modal storytelling of urban emergency events based on Crowdsensing of social media analytics. MONET 22(2):218–227
49.
go back to reference Karaboga D, Akay B, Ozturk C (2007) Artificial Bee Colony (ABC) optimization algorithm for training feed-forward neural networks. In: Modeling decisions for artificial intelligence, vol 4617/2007, No 0302-9743, pp 318–329 Karaboga D, Akay B, Ozturk C (2007) Artificial Bee Colony (ABC) optimization algorithm for training feed-forward neural networks. In: Modeling decisions for artificial intelligence, vol 4617/2007, No 0302-9743, pp 318–329
50.
go back to reference Guo J, Zhou JZ, Zou Q, Song LX, Zhang YC (2011) Study on multi-objective calibration of hydrological model and effect of objective functions combination on optimization results. J Sichuan Univ (Eng Sci Ed) 43(6):58–63 Guo J, Zhou JZ, Zou Q, Song LX, Zhang YC (2011) Study on multi-objective calibration of hydrological model and effect of objective functions combination on optimization results. J Sichuan Univ (Eng Sci Ed) 43(6):58–63
51.
go back to reference Mohammadi SAR, Feizi Derakhshi MR, Akbari R (2013) An adaptive multi-objective artificial bee colony with crowding distance mechanism. IJST Trans Electr Eng 37(E1):79–92 Mohammadi SAR, Feizi Derakhshi MR, Akbari R (2013) An adaptive multi-objective artificial bee colony with crowding distance mechanism. IJST Trans Electr Eng 37(E1):79–92
52.
go back to reference Deb K, Mohan M, Mishra S (2005) Evaluating the epsilon-domination based multi-objective evolutionary algorithm for a quick computation of Pareto-optimal solutions. Evol Comput 13(4):501–525CrossRef Deb K, Mohan M, Mishra S (2005) Evaluating the epsilon-domination based multi-objective evolutionary algorithm for a quick computation of Pareto-optimal solutions. Evol Comput 13(4):501–525CrossRef
53.
go back to reference Coello Coello CA, Lamont GB, Van Veldhuizen DA (2007) Evolutionary algorithms for solving multi-objective problems, 2nd edn. Springer, New YorkMATH Coello Coello CA, Lamont GB, Van Veldhuizen DA (2007) Evolutionary algorithms for solving multi-objective problems, 2nd edn. Springer, New YorkMATH
54.
go back to reference Madsen H (2003) Parameter estimation in distributed hydrological catchment modelling using automatic calibration with multiple objectives. Adv Water Resour 26:205–216CrossRef Madsen H (2003) Parameter estimation in distributed hydrological catchment modelling using automatic calibration with multiple objectives. Adv Water Resour 26:205–216CrossRef
55.
go back to reference Nash JE, Sutcliffe JV (1970) River flow forecasting through conceptual models. J Hydrol 10:282–290CrossRef Nash JE, Sutcliffe JV (1970) River flow forecasting through conceptual models. J Hydrol 10:282–290CrossRef
56.
go back to reference Werkhoven KV, Wagener T, Reed P, Yong T (2009) Sensitivity-guided reduction of parametric dimensionality for multi-objective calibration of watershed models. Adv Water Resour 32(8):1154–1169CrossRef Werkhoven KV, Wagener T, Reed P, Yong T (2009) Sensitivity-guided reduction of parametric dimensionality for multi-objective calibration of watershed models. Adv Water Resour 32(8):1154–1169CrossRef
57.
go back to reference Iman RL, Conover WJ (1980) Small sample sensitivity analysis techniques for computer models, with an application to risk assessment. Commun Stat A9(17):1749–1842MathSciNetCrossRef Iman RL, Conover WJ (1980) Small sample sensitivity analysis techniques for computer models, with an application to risk assessment. Commun Stat A9(17):1749–1842MathSciNetCrossRef
58.
go back to reference Gupta HV, Beven KJ, Wagener T (2005) Model calibration and uncertainty estimation, encyclopedia of hydrological sciences, Part 11, Rainfall-runoff modeling. Wiley, New York Gupta HV, Beven KJ, Wagener T (2005) Model calibration and uncertainty estimation, encyclopedia of hydrological sciences, Part 11, Rainfall-runoff modeling. Wiley, New York
59.
go back to reference Pagano RR (2012) Understanding statistics in the behavioral sciences. Cengage Learning, Boston Pagano RR (2012) Understanding statistics in the behavioral sciences. Cengage Learning, Boston
Metadata
Title
Application research of multi-objective Artificial Bee Colony optimization algorithm for parameters calibration of hydrological model
Authors
Jiuyuan Huo
Liqun Liu
Publication date
25-04-2018
Publisher
Springer London
Published in
Neural Computing and Applications / Issue 9/2019
Print ISSN: 0941-0643
Electronic ISSN: 1433-3058
DOI
https://doi.org/10.1007/s00521-018-3483-4

Other articles of this Issue 9/2019

Neural Computing and Applications 9/2019 Go to the issue

S.I. : Emergence in Human-like Intelligence towards Cyber-Physical Systems

Forest fire forecasting using ensemble learning approaches

Premium Partner