Skip to main content
Top
Published in: Soft Computing 14/2019

24-04-2018 | Methodologies and Application

A Tabu Search implementation for adaptive localization in ensemble-based methods

Authors: Elias D. Nino-Ruiz, Luis E. Morales-Retat

Published in: Soft Computing | Issue 14/2019

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

In this paper, we propose an efficient and practical implementation of the ensemble Kalman filter based on adaptive localization via Tabu search. The proposed method works as follows: during assimilation steps, observed components from the model state are split into two groups: the training set and the validation set, after which analysis states are obtained by using the training data, while posterior errors are estimated by means of the validation set. These steps are repeated for a fixed number of iterations and based on a Tabu search implementation, for each model component an optimal radius of influence is estimated. Experimental tests are performed by using the Lorenz 96 model which mimics the chaotic behaviour of the atmosphere. We assess the accuracy of the proposed method by contrasting its numerical results with those obtained by reference filters from the specialized literature such as the local ensemble transform Kalman filter and the ensemble Kalman filter based on modified Cholesky decomposition. Besides, numerical simulations are enriched by using different ensemble sizes, radii of influences (where appropriate), and inflation factors. The results reveal that, for all configurations, the proposed adaptive localization-based filter can improve the accuracy as well as the convergence of ensemble-based methods in the context of sequential data assimilation methods.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Literature
go back to reference Anderson E, Bai Z, Dongarra J, Greenbaum A, McKenney A, Du Croz J, Hammerling S, Demmel J, Bischof C, Sorensen D (1990) Lapack: a portable linear algebra library for high-performance computers. In: Proceedings of the 1990 ACM/IEEE conference on supercomputing, IEEE Computer Society Press, pp 2–11 Anderson E, Bai Z, Dongarra J, Greenbaum A, McKenney A, Du Croz J, Hammerling S, Demmel J, Bischof C, Sorensen D (1990) Lapack: a portable linear algebra library for high-performance computers. In: Proceedings of the 1990 ACM/IEEE conference on supercomputing, IEEE Computer Society Press, pp 2–11
go back to reference Askarzadeh A, dos Santos Coelho L, Klein CE, Mariani VC (2016) A population-based simulated annealing algorithm for global optimization. In: 2016 IEEE international conference on systems, man, and cybernetics (SMC), IEEE, pp 004626–004633 Askarzadeh A, dos Santos Coelho L, Klein CE, Mariani VC (2016) A population-based simulated annealing algorithm for global optimization. In: 2016 IEEE international conference on systems, man, and cybernetics (SMC), IEEE, pp 004626–004633
go back to reference Bell N, Garland M (2009) Implementing sparse matrix-vector multiplication on throughput-oriented processors. In: Proceedings of the conference on high performance computing networking, storage and analysis, ACM, p 18 Bell N, Garland M (2009) Implementing sparse matrix-vector multiplication on throughput-oriented processors. In: Proceedings of the conference on high performance computing networking, storage and analysis, ACM, p 18
go back to reference Bishop CH, Hodyss D (2011) Adaptive ensemble covariance localization in ensemble 4d-var state estimation. Mon Weather Rev 139(4):1241–1255CrossRef Bishop CH, Hodyss D (2011) Adaptive ensemble covariance localization in ensemble 4d-var state estimation. Mon Weather Rev 139(4):1241–1255CrossRef
go back to reference Bishop CH, Toth Z (1999) Ensemble transformation and adaptive observations. J Atmos Sci 56(11):1748–1765CrossRef Bishop CH, Toth Z (1999) Ensemble transformation and adaptive observations. J Atmos Sci 56(11):1748–1765CrossRef
go back to reference Bouttier C, Babando O, Gadat S, Gerchinovitz S, Laporte S, Nicol F (2017) Adaptive simulated annealing with homogenization for aircraft trajectory optimization. In: Operations research proceedings 2015, Springer, pp 569–574 Bouttier C, Babando O, Gadat S, Gerchinovitz S, Laporte S, Nicol F (2017) Adaptive simulated annealing with homogenization for aircraft trajectory optimization. In: Operations research proceedings 2015, Springer, pp 569–574
go back to reference Ceberio J, Mendiburu A, Lozano JA (2016) A note on the boltzmann distribution and the linear ordering problem. In: Conference of the Spanish association for artificial intelligence, Springer, pp 441–446 Ceberio J, Mendiburu A, Lozano JA (2016) A note on the boltzmann distribution and the linear ordering problem. In: Conference of the Spanish association for artificial intelligence, Springer, pp 441–446
go back to reference Chang S-Y, Saha A (2016) Application of 3d var Kalman filter in a three-dimensional subsurface contaminant transport model for a continuous pollutant source. In: Proceedings of the 2013 national conference on advances in environmental science and technology, Springer, pp 97–104 Chang S-Y, Saha A (2016) Application of 3d var Kalman filter in a three-dimensional subsurface contaminant transport model for a continuous pollutant source. In: Proceedings of the 2013 national conference on advances in environmental science and technology, Springer, pp 97–104
go back to reference Evensen G (2003) The ensemble Kalman filter: theoretical formulation and practical implementation. Ocean Dyn 53(4):343–367CrossRef Evensen G (2003) The ensemble Kalman filter: theoretical formulation and practical implementation. Ocean Dyn 53(4):343–367CrossRef
go back to reference Evensen G (2006) Data assimilation: the ensemble Kalman filter. Springer, New YorkMATH Evensen G (2006) Data assimilation: the ensemble Kalman filter. Springer, New YorkMATH
go back to reference Fertig EJ, Harlim J, Hunt BR (2007) A comparative study of 4d-var and a 4d ensemble Kalman filter: perfect model simulations with lorenz-96. Tellus A 59(1):96–100CrossRef Fertig EJ, Harlim J, Hunt BR (2007) A comparative study of 4d-var and a 4d ensemble Kalman filter: perfect model simulations with lorenz-96. Tellus A 59(1):96–100CrossRef
go back to reference Fu H, Wu X, Li W, Xie Y, Han G, Zhang S (2016) Reconstruction of typhoon structure using 3-dimensional doppler radar radial velocity data with the multigrid analysis: a case study in an idealized simulation context. Adv Meteorol 2016:1–10CrossRef Fu H, Wu X, Li W, Xie Y, Han G, Zhang S (2016) Reconstruction of typhoon structure using 3-dimensional doppler radar radial velocity data with the multigrid analysis: a case study in an idealized simulation context. Adv Meteorol 2016:1–10CrossRef
go back to reference Furrer R, Genton MG, Nychka D (2006) Covariance tapering for interpolation of large spatial datasets. J Comput Gr Stat 15(3):502–523MathSciNetCrossRef Furrer R, Genton MG, Nychka D (2006) Covariance tapering for interpolation of large spatial datasets. J Comput Gr Stat 15(3):502–523MathSciNetCrossRef
go back to reference Gillijns S, Mendoza OB, Chandrasekar J, De Moor BLR, Bernstein DS, Ridley A (2006) What is the ensemble Kalman filter and how well does it work? In: American control conference, 2006, pages 6 pp –, June 2006 Gillijns S, Mendoza OB, Chandrasekar J, De Moor BLR, Bernstein DS, Ridley A (2006) What is the ensemble Kalman filter and how well does it work? In: American control conference, 2006, pages 6 pp –, June 2006
go back to reference Golnari PA, Malik S (2017) Evaluating matrix representations for error-tolerant computing. In: 2017 design, automation and test in Europe conference and exhibition (DATE), IEEE, pp 1659–1662 Golnari PA, Malik S (2017) Evaluating matrix representations for error-tolerant computing. In: 2017 design, automation and test in Europe conference and exhibition (DATE), IEEE, pp 1659–1662
go back to reference Grewal MS (2011) Kalman filtering. In: International Encyclopedia of statistical science, Springer, pp 705–708 Grewal MS (2011) Kalman filtering. In: International Encyclopedia of statistical science, Springer, pp 705–708
go back to reference Greybush SJ, Kalnay E, Miyoshi T, Ide K, Hunt BR (2011) Balance and ensemble Kalman filter localization techniques. Mon Weather Rev 139(2):511–522CrossRef Greybush SJ, Kalnay E, Miyoshi T, Ide K, Hunt BR (2011) Balance and ensemble Kalman filter localization techniques. Mon Weather Rev 139(2):511–522CrossRef
go back to reference Gu Y, Wylie BK, Boyte SP, Picotte J, Howard DM, Smith K, Nelson KJ (2016) An optimal sample data usage strategy to minimize overfitting and underfitting effects in regression tree models based on remotely-sensed data. Remote Sens 8(11):943CrossRef Gu Y, Wylie BK, Boyte SP, Picotte J, Howard DM, Smith K, Nelson KJ (2016) An optimal sample data usage strategy to minimize overfitting and underfitting effects in regression tree models based on remotely-sensed data. Remote Sens 8(11):943CrossRef
go back to reference Hamill TM, Whitaker JS, Snyder C (2001) Distance-dependent filtering of background error covariance estimates in an ensemble Kalman filter. Mon Weather Rev 129(11):2776–2790CrossRef Hamill TM, Whitaker JS, Snyder C (2001) Distance-dependent filtering of background error covariance estimates in an ensemble Kalman filter. Mon Weather Rev 129(11):2776–2790CrossRef
go back to reference Houtekamer PL, Mitchell HL (2005) Ensemble Kalman filtering. Q J R Meteorol Soc 131(613):3269–3289CrossRef Houtekamer PL, Mitchell HL (2005) Ensemble Kalman filtering. Q J R Meteorol Soc 131(613):3269–3289CrossRef
go back to reference Hunt BR, Kostelich EJ, Szunyogh I (2007) Efficient data assimilation for spatiotemporal chaos: a local ensemble transform Kalman filter. Physica D 230(1):112–126MathSciNetCrossRefMATH Hunt BR, Kostelich EJ, Szunyogh I (2007) Efficient data assimilation for spatiotemporal chaos: a local ensemble transform Kalman filter. Physica D 230(1):112–126MathSciNetCrossRefMATH
go back to reference Kang J-S, Kalnay E, Liu J, Fung I, Miyoshi T, Ide K (2011) Variable localization in an ensemble Kalman filter: application to the carbon cycle data assimilation. J Geophys Res Atmos 116(D9):D09110 Kang J-S, Kalnay E, Liu J, Fung I, Miyoshi T, Ide K (2011) Variable localization in an ensemble Kalman filter: application to the carbon cycle data assimilation. J Geophys Res Atmos 116(D9):D09110
go back to reference Karimi A, Paul MR (2010) Extensive chaos in the lorenz-96 model. Chaos Interdiscip J Nonlinear Sci 20(4):043105CrossRef Karimi A, Paul MR (2010) Extensive chaos in the lorenz-96 model. Chaos Interdiscip J Nonlinear Sci 20(4):043105CrossRef
go back to reference Kaufman CG, Schervish MJ, Nychka DW (2008) Covariance tapering for likelihood-based estimation in large spatial data sets. J Am Stat Assoc 103(484):1545–1555MathSciNetCrossRefMATH Kaufman CG, Schervish MJ, Nychka DW (2008) Covariance tapering for likelihood-based estimation in large spatial data sets. J Am Stat Assoc 103(484):1545–1555MathSciNetCrossRefMATH
go back to reference Lou Z, Reinitz J (2016) Parallel simulated annealing using an adaptive resampling interval. Parallel Comput 53:23–31MathSciNetCrossRef Lou Z, Reinitz J (2016) Parallel simulated annealing using an adaptive resampling interval. Parallel Comput 53:23–31MathSciNetCrossRef
go back to reference Luo X, Bhakta T, Nædal G et al (2017) Data driven adaptive localization with applications to ensemble-based 4d seismic history matching. In: SPE Bergen one day seminar. Society of Petroleum Engineers Luo X, Bhakta T, Nædal G et al (2017) Data driven adaptive localization with applications to ensemble-based 4d seismic history matching. In: SPE Bergen one day seminar. Society of Petroleum Engineers
go back to reference Mattern JP, Song H, Edwards CA, Moore AM, Fiechter J (2017) Data assimilation of physical and chlorophyll a observations in the california current system using two biogeochemical models. Ocean Model 109:55–71CrossRef Mattern JP, Song H, Edwards CA, Moore AM, Fiechter J (2017) Data assimilation of physical and chlorophyll a observations in the california current system using two biogeochemical models. Ocean Model 109:55–71CrossRef
go back to reference Merrill D, Garland M (2016) Merge-based sparse matrix-vector multiplication (spmv) using the CSR storage format. In: Proceedings of the 21st ACM SIGPLAN symposium on principles and practice of parallel programming, ACM, p 43 Merrill D, Garland M (2016) Merge-based sparse matrix-vector multiplication (spmv) using the CSR storage format. In: Proceedings of the 21st ACM SIGPLAN symposium on principles and practice of parallel programming, ACM, p 43
go back to reference Miyoshi T (2011) The Gaussian approach to adaptive covariance inflation and its implementation with the local ensemble transform Kalman filter. Mon Weather Rev 139(5):1519–1535CrossRef Miyoshi T (2011) The Gaussian approach to adaptive covariance inflation and its implementation with the local ensemble transform Kalman filter. Mon Weather Rev 139(5):1519–1535CrossRef
go back to reference Nerger L, Schulte S, Bunse-Gerstner A (2014) On the influence of model nonlinearity and localization on ensemble Kalman smoothing. Q J R Meteorol Soc 140(684):2249–2259CrossRef Nerger L, Schulte S, Bunse-Gerstner A (2014) On the influence of model nonlinearity and localization on ensemble Kalman smoothing. Q J R Meteorol Soc 140(684):2249–2259CrossRef
go back to reference Nino Ruiz ED, Sandu A (2016) A derivative-free trust region framework for variational data assimilation. J Comput Appl Math 293:164–179MathSciNetCrossRefMATH Nino Ruiz ED, Sandu A (2016) A derivative-free trust region framework for variational data assimilation. J Comput Appl Math 293:164–179MathSciNetCrossRefMATH
go back to reference Nino Ruiz ED, Sandu A, Anderson J (2015) An efficient implementation of the ensemble Kalman filter based on an iterative Sherman–Morrison formula. Stat Comput 25(3):561–577MathSciNetCrossRefMATH Nino Ruiz ED, Sandu A, Anderson J (2015) An efficient implementation of the ensemble Kalman filter based on an iterative Sherman–Morrison formula. Stat Comput 25(3):561–577MathSciNetCrossRefMATH
go back to reference Niño ED (2012) Samods and sagamods: novel algorithms based on the automata theory for the multiobjective optimization of combinatorial problems. Int J Artif Intell 8(S12):147–165 Niño ED (2012) Samods and sagamods: novel algorithms based on the automata theory for the multiobjective optimization of combinatorial problems. Int J Artif Intell 8(S12):147–165
go back to reference Niño ED, Ardila CJ, Barrios A, Donoso Y (2011) Mods: a novel metaheuristic of deterministic swapping for the multi-objective optimization of combinatorials problems. Constraints 9:1 Niño ED, Ardila CJ, Barrios A, Donoso Y (2011) Mods: a novel metaheuristic of deterministic swapping for the multi-objective optimization of combinatorials problems. Constraints 9:1
go back to reference Nino-Ruiz ED, Ardila C, Capacho R (2017) Local search methods for the solution of implicit inverse problems. Soft Comput, 1–14 Nino-Ruiz ED, Ardila C, Capacho R (2017) Local search methods for the solution of implicit inverse problems. Soft Comput, 1–14
go back to reference Nino-Ruiz ED, Sandu A, Deng X (2015) A parallel ensemble Kalman filter implementation based on modified Cholesky decomposition. In: Proceedings of the 6th workshop on latest advances in scalable algorithms for large-scale systems, ACM, p 4 Nino-Ruiz ED, Sandu A, Deng X (2015) A parallel ensemble Kalman filter implementation based on modified Cholesky decomposition. In: Proceedings of the 6th workshop on latest advances in scalable algorithms for large-scale systems, ACM, p 4
go back to reference Nino-Ruiz ED, Sandu A, Deng X (2017) A parallel implementation of the ensemble Kalman filter based on modified Cholesky decomposition. J Comput Sci Nino-Ruiz ED, Sandu A, Deng X (2017) A parallel implementation of the ensemble Kalman filter based on modified Cholesky decomposition. J Comput Sci
go back to reference Obrien RM (2007) A caution regarding rules of thumb for variance inflation factors. Qual Quant 41(5):673–690CrossRef Obrien RM (2007) A caution regarding rules of thumb for variance inflation factors. Qual Quant 41(5):673–690CrossRef
go back to reference Ott E, Hunt BR, Szunyogh I, Zimin AV, Kostelich EJ, Corazza M, Kalnay E, Patil DJ, Yorke JA (2004) A local ensemble Kalman filter for atmospheric data assimilation. Tellus A 56(5):415–428CrossRef Ott E, Hunt BR, Szunyogh I, Zimin AV, Kostelich EJ, Corazza M, Kalnay E, Patil DJ, Yorke JA (2004) A local ensemble Kalman filter for atmospheric data assimilation. Tellus A 56(5):415–428CrossRef
go back to reference Ott E, Hunt B, Szunyogh I, Zimin AV, Kostelich EJ, Corazza M, Kalnay E, Patil DJ, Yorke JA (2008) A local ensemble transform Kalman filter data assimilation system for the NCEP global model. Tellus A 60(1):113–130CrossRef Ott E, Hunt B, Szunyogh I, Zimin AV, Kostelich EJ, Corazza M, Kalnay E, Patil DJ, Yorke JA (2008) A local ensemble transform Kalman filter data assimilation system for the NCEP global model. Tellus A 60(1):113–130CrossRef
go back to reference Petra CG, Zavala VM, Nino-Ruiz ED, Anitescu M (2016) A high-performance computing framework for analyzing the economic impacts of wind correlation. Electr Power Syst Res 141:372–380CrossRef Petra CG, Zavala VM, Nino-Ruiz ED, Anitescu M (2016) A high-performance computing framework for analyzing the economic impacts of wind correlation. Electr Power Syst Res 141:372–380CrossRef
go back to reference Prasad VS, Johny CJ, Sodhi JS (2016) Impact of 3d Var GSI-ENKF hybrid data assimilation system. J Earth Syst Sci 125(8):1509–1521CrossRef Prasad VS, Johny CJ, Sodhi JS (2016) Impact of 3d Var GSI-ENKF hybrid data assimilation system. J Earth Syst Sci 125(8):1509–1521CrossRef
go back to reference Refaeilzadeh P, Tang L, Liu H (2009) Cross-validation. In: Encyclopedia of database systems, Springer, pp 532–538 Refaeilzadeh P, Tang L, Liu H (2009) Cross-validation. In: Encyclopedia of database systems, Springer, pp 532–538
go back to reference Shahvari O, Logendran R (2017) An enhanced tabu search algorithm to minimize a bi-criteria objective in batching and scheduling problems on unrelated-parallel machines with desired lower bounds on batch sizes. Comput Oper Res 77:154–176MathSciNetCrossRefMATH Shahvari O, Logendran R (2017) An enhanced tabu search algorithm to minimize a bi-criteria objective in batching and scheduling problems on unrelated-parallel machines with desired lower bounds on batch sizes. Comput Oper Res 77:154–176MathSciNetCrossRefMATH
go back to reference Soto M, Sevaux M, Reinholz A, Rossi A (2017) Multiple neighborhood search, tabu search and ejection chains for the multi-depot open vehicle routing problem. Comput Ind Eng 107:211–222CrossRef Soto M, Sevaux M, Reinholz A, Rossi A (2017) Multiple neighborhood search, tabu search and ejection chains for the multi-depot open vehicle routing problem. Comput Ind Eng 107:211–222CrossRef
go back to reference Tippett MK, Anderson JL, Bishop CH, Hamill TM, Whitaker JS (2003) Ensemble square root filters. Mon Weather Rev 131(7):1485–1490CrossRef Tippett MK, Anderson JL, Bishop CH, Hamill TM, Whitaker JS (2003) Ensemble square root filters. Mon Weather Rev 131(7):1485–1490CrossRef
go back to reference Yu W, Yin P, Liu T, Yang L, Liu X (2016) Study on the optimal mode of foreign investment based on simulated annealing algorithm. J Comput Theor Nanosci 13(5):3019–3023CrossRef Yu W, Yin P, Liu T, Yang L, Liu X (2016) Study on the optimal mode of foreign investment based on simulated annealing algorithm. J Comput Theor Nanosci 13(5):3019–3023CrossRef
go back to reference Zupanski M (2009) Theoretical and practical issues of ensemble data assimilation in weather and climate. In: Park SK, Xu L (eds) Data assimilation for atmospheric, oceanic and hydrologic applications. Springer, Berlin, pp 67–84CrossRef Zupanski M (2009) Theoretical and practical issues of ensemble data assimilation in weather and climate. In: Park SK, Xu L (eds) Data assimilation for atmospheric, oceanic and hydrologic applications. Springer, Berlin, pp 67–84CrossRef
Metadata
Title
A Tabu Search implementation for adaptive localization in ensemble-based methods
Authors
Elias D. Nino-Ruiz
Luis E. Morales-Retat
Publication date
24-04-2018
Publisher
Springer Berlin Heidelberg
Published in
Soft Computing / Issue 14/2019
Print ISSN: 1432-7643
Electronic ISSN: 1433-7479
DOI
https://doi.org/10.1007/s00500-018-3210-1

Other articles of this Issue 14/2019

Soft Computing 14/2019 Go to the issue

Premium Partner