Top

Autonomous Robots

Published in:

28-01-2019

Gaussian process decentralized data fusion meets transfer learning in large-scale distributed cooperative perception

Authors: Ruofei Ouyang, Bryan Kian Hsiang Low

Published in: Autonomous Robots | Issue 3-4/2020

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

This paper presents novel Gaussian process decentralized data fusion algorithms exploiting the notion of agent-centric support sets for distributed cooperative perception of large-scale environmental phenomena. To overcome the limitations of scale in existing works, our proposed algorithms allow every mobile sensing agent to utilize a different support set and dynamically switch to another during execution for encapsulating its own data into a local summary that, perhaps surprisingly, can still be assimilated with the other agents’ local summaries (i.e., based on their current support sets) into a globally consistent summary to be used for predicting the phenomenon. To achieve this, we propose a novel transfer learning mechanism for a team of agents capable of sharing and transferring information encapsulated in a summary based on a support set to that utilizing a different support set with some loss that can be theoretically bounded and analyzed. To alleviate the issue of information loss accumulating over multiple instances of transfer learning, we propose a new information sharing mechanism to be incorporated into our algorithms in order to achieve memory-efficient lazy transfer learning. Empirical evaluation on three real-world datasets for up to 128 agents show that our algorithms outperform the state-of-the-art methods.

previous article Cooperative visual-inertial sensor fusion: fundamental equations and state determination in closed-form

next article SOUL: data sharing for robot swarms

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Available only for authorised users

PITC generalizes the Bayesian Committee Machine (BCM) (Schwaighofer and Tresp 2002), the latter of which assumes the support set to be the set of unobserved input locations whose measurements are to be predicted (Quiñonero-Candela and Rasmussen 2005). As a result, BCM does not scale well with a large set of such unobserved input locations.

An exception is the work of Park et al. (2011) that overcomes this boundary effect by imposing continuity constraints along the boundaries in a centralized manner.

The conditional independence of \(Y_{\mathcal {D}_1},\ldots ,Y_{\mathcal {D}_N}\) given \(Y_{\mathcal {S}}\) assumed by PITC and PIC (hence, GP-DDF and GP-\(\hbox {DDF}^+\)) improves their scalability over the GP model (Sect. 2) at the cost of poorer predictive performance. To potentially reduce the degree of violation of this assumption, an informative support set can be \(\hbox {selected}^6\). Furthermore, the experimental results in Chen et al. (2015) show that GP-DDF and GP-\(\hbox {DDF}^+\) can achieve predictive performance comparable to that of the GP model while enjoying lower computational cost over it. The predictive performance of GP-DDF and GP-\(\hbox {DDF}^+\) can be improved by increasing the size of \(\mathcal {S}\) at the expense of greater time and communication overhead.

Naively, an agent can delay transfer learning by simply storing a separate local summary based on the support set for every previously visited local area, which is not memory-efficient.

Multiple backups of the local summary and support set for the same local area may exist if agents leave this area at the same time, which rarely happens. In this case, agent i should retrieve (and remove) all these backups from the agents storing them.

Alternatively, active learning can be used to select an informative support set a priori for each local area (Chen et al. 2015). Empirically, this yields little performance improvement due to a sufficiently dense (yet small) support set uniformly distributed over the local area and slightly beyond its boundary by \(10\%\) of its width.

Local GPs result from a sparse block-diagonal \(\varSigma _{\mathcal {D}\mathcal {D}}\) (2).

The predictive performance of centralized PITC corresponds to that of GP-DDF, as discussed in Sect. 2.2. Hence, the RMSE of centralized PITC coincides exactly with that of GP-DDF in Fig. 8.

The incurred time of centralized PITC is slightly less than that of GP-DDF (Fig. 8) increased by a factor of the total number of agents. This agrees with the analysis of the time complexity of PITC versus GP-DDF in Sect. 2.2. This can also be observed in Fig. 9 where the incurred time of GP-DDF increases by nearly two fold when the number of agents is halved.

If the subset sizes differ, then “virtual” locations are added to each subset to make all subsets to be of the same size as \(T\triangleq \arg \max _{s\in \mathcal {S}} |\mathcal {D}_{is}|\) (\(T'\triangleq \arg \max _{s\in \mathcal {S}} |\mathcal {S}'_{s}|\)). The virtual locations added to \(\mathcal {D}_{is}\) (\(\mathcal {S}'_{s}\)) are chosen as \(s\in \mathcal {S}\) so that they do not induce additional errors but will loosen the bound.

Cao, N., Low, K. H., & Dolan, J. M. (2013). Multi-robot informative path planning for active sensing of environmental phenomena: A tale of two algorithms. In Proceedings of AAMAS.

Chen, J., Cao, N., Low, K. H., Ouyang, R., Tan, C. K. Y., & Jaillet, P. (2013a). Parallel Gaussian process regression with low-rank covariance matrix approximations. In Proceedings of UAI (pp. 152–161).

Chen, J., Low, K. H., Jaillet, P., & Yao, Y. (2015). Gaussian process decentralized data fusion and active sensing for spatiotemporal traffic modeling and prediction in mobility-on-demand systems. IEEE Transactions on Automation Science and Engineering, 12, 901–921.CrossRef

Chen, J., Low, K. H., Tan, C. K. Y., Oran, A., Jaillet, P., Dolan, J. M., & Sukhatme, G. S. (2012). Decentralized data fusion and active sensing with mobile sensors for modeling and predicting spatiotemporal traffic phenomena. In Proceedings of UAI (pp. 163–173).

Chen, J., Low, K. H., & Tan, C. K. Y. (2013b). Gaussian process-based decentralized data fusion and active sensing for mobility-on-demand system. In Proceedings of robotics: science and systems conference.

Choudhury, A., Nair, P. B., & Keane, A. J. (2002). A data parallel approach for large-scale Gaussian process modeling. In Proceedings of SDM (pp. 95–111).

Chung, T. H., Gupta, V., Burdick, J. W., & Murray, R. M. (2004). On a decentralized active sensing strategy using mobile sensor platforms in a network. In Proceedings of CDC (pp. 1914–1919).

Coates, M. (2004). Distributed particle filters for sensor networks. In Proceedings of IPSN (pp. 99–107).

Cortes, J. (2009). Distributed kriged Kalman filter for spatial estimation. IEEE Transactions on Automatic Control, 54(12), 2816–2827.MathSciNetCrossRef

Das, J., Harvey, J. B. J., Py, F., Vathsangam, H., Graham, R., Rajan, K., & Sukhatme, G. S. (2013). Hierarchical probabilistic regression for AUV-based adaptive sampling of marine phenomena. In Proceedings of IEEE ICRA (pp. 5571–5578).

Das, K., & Srivastava, A. N. (2010). Block-GP: Scalable Gaussian process regression for multimodal data. In Proceedings of ICDM (pp. 791–796).

Daxberger, E., & Low, K. H. (2017). Distributed batch Gaussian process optimization. In Proceedings of ICML (pp. 951–960).

Dolan, J. M., Podnar, G., Stancliff, S., Low, K. H., Elfes, A., Higinbotham, J., Hosler, J. C., Moisan, T. A., & Moisan, J. (2009). Cooperative aquatic sensing using the telesupervised adaptive ocean sensor fleet. In Proceedings of SPIE conference on remote sensing of the ocean, sea ice, and large water regions Vol. 7473.

Guestrin, C., Bodik, P., Thibaus, R., Paskin, M., & Madden, S. (2004). Distributed regression: An efficient framework for modeling sensor network data. In Proceedings of IPSN (pp. 1–10).

Hoang, T. N., Hoang, Q. M., & Low, K. H. (2016). A distributed variational inference framework for unifying parallel sparse Gaussian process regression models. In Proceedings of ICML (pp. 382–391).

Hoang, Q. M., Hoang, T. N., & Low, K. H. (2017). A generalized stochastic variational Bayesian hyperparameter learning framework for sparse spectrum Gaussian process regression. In Proceedings of AAAI (pp. 2007–2014).

Hoang, T. N., Hoang, Q. M., & Low, K. H. (2018). Decentralized high-dimensional Bayesian optimization with factor graphs. In Proceedings of AAAI (pp. 3231–3238).

Hoang, T. N., Hoang, Q. M., Low, K. H., & How, J. P. (2019). Collective online learning of Gaussian processes in massive multi-agent systems. In Proceedings of AAAI.

Hoang, T. N., Low, K. H., Jaillet, P., & Kankanhalli, M. (2014). Nonmyopic \(\epsilon \)-Bayes-optimal active learning of Gaussian processes. In Proceedings of ICML (pp. 739–747).

Kim, Y., & Shell, D. (2014). Distributed robotic sampling of non-homogeneous spatiotemporal fields via recursive geometric sub-division. In Proceedings of IEEE ICRA (pp. 557–562).

Krause, A., Singh, A., & Guestrin, C. (2008). Near-optimal sensor placements in Gaussian processes: Theory, efficient algorithms and empirical studies. JMLR, 9, 235–284.MATH

Leonard, N. E., Palley, D. A., Lekien, F., Sepulchre, R., Fratantoni, D. M., & Davis, R. E. (2007). Collective motion, sensor networks, and ocean sampling. Proceedings of the IEEE, 95(1), 48–74.CrossRef

Ling, C. K., Low, K. H., & Jaillet, P. (2016). Gaussian process planning with Lipschitz continuous reward functions: Towards unifying Bayesian optimization, active learning, and beyond. In Proceedings of AAAI (pp. 1860–1866).

Low, K. H., Chen, J., Dolan, J. M., Chien, S., & Thompson, D. R. (2012). Decentralized active robotic exploration and mapping for probabilistic field classification in environmental sensing. In Proceedings of AAMAS (pp. 105–112).

Low, K. H., Chen, J., Hoang, T. N., Xu, N., & Jaillet, P. (2015a). Recent advances in scaling up Gaussian process predictive models for large spatiotemporal data. In S. Ravela, A. Sandu (Eds.), Proceedings of dynamic data-driven environmental systems science conference (DyDESS’14), LNCS 8964, Springer.

Low, K. H., Dolan, J. M., & Khosla, P. (2008). Adaptive multi-robot wide-area exploration and mapping. In Proceedings of AAMAS (pp. 23–30).

Low, K. H., Dolan, J. M., & Khosla, P. (2009). Information-theoretic approach to efficient adaptive path planning for mobile robotic environmental sensing. In Proceedings of ICAPS.

Low, K. H., Dolan, J. M., & Khosla, P. (2011). Active Markov information-theoretic path planning for robotic environmental sensing. In Proceedings of AAMAS (pp. 753–760).

Low, K. H., Gordon, G. J., Dolan, J. M., & Khosla, P. (2007). Adaptive sampling for multi-robot wide-area exploration. In Proceedings of IEEE ICRA (pp. 755–760).

Low, K. H., Yu, J., Chen, J., & Jaillet, P. (2015b). Parallel Gaussian process regression for big data: Low-rank representation meets Markov approximation. In Proceedings of AAAI (pp. 2821–2827).

Min, W., & Wynter, L. (2011). Real-time road traffic prediction with spatio-temporal correlations. Transportation Research Part C: Emerging, 19(4), 606–616.CrossRef

Ouyang, R., & Low, K. H. (2018). Gaussian process decentralized data fusion meets transfer learning in large-scale distributed cooperative perception. In Proceedings of AAAI (pp. 3876–3883).

Ouyang, R., Low, K. H., Chen, J., & Jaillet, P. (2014). Multi-robot active sensing of non-stationary Gaussian process-based environmental phenomena. In Proceedings of AAMAS (pp. 573–580).

Park, C., Huang, J. Z., & Ding, Y. (2011). Domain decomposition approach for fast Gaussian process regression of large spatial data sets. JMLR, 12, 1697–1728.MathSciNetMATH

Paskin, M. A., Guestrin, C. (2004). Robust probabilistic inference in distributed systems. In Proceedings of UAI (pp. 436–445).

Podnar, G., Dolan, J. M., Low, K. H., & Elfes, A. (2010). Telesupervised remote surface water quality sensing. In Proceedings of IEEE aerospace conference.

Quiñonero-Candela, J., & Rasmussen, C. E. (2005). A unifying view of sparse approximate Gaussian process regression. JMLR, 6, 1939–1959.MathSciNetMATH

Rasmussen, C. E., & Williams, C. K. I. (2006). Gaussian processes for machine learning. Cambridge: MIT Press.MATH

Rosencrantz, M., Gordon, G., & Thrun, S. (2003). Decentralized sensor fusion with distributed particle filters. In Proceedings of UAI (pp. 493–500).

Schwaighofer, A., & Tresp, V. (2002). Transductive and inductive methods for approximate Gaussian process regression. In Proceedings of NIPS (pp. 953–960).

Singh, A., Krause, A., Guestrin, C., & Kaiser, W. J. (2009). Efficient informative sensing using multiple robots. Journal of Artificial Intelligence Research, 34, 707–755.MathSciNetCrossRef

Snelson, E. L., & Ghahramani, Z. (2007). Local and global sparse Gaussian process approximation. In Proceedings of AISTATS.

Sukkarieh, S., Nettleton, E., Kim, J., Ridley, M., Goktogan, A., & Durrant-Whyte, H. (2003). The ANSER project: Data fusion across multiple uninhabited air vehicles. IJRR, 22(7–8), 505–539.

Sun, S., Zhao, J., & Zhu, J. (2015). A review of Nyström methods for large-scale machine learning. Information Fusion, 26, 36–48.CrossRef

Thompson, D. R., Cabrol, N., Furlong, M., Hardgrove, C., Low, K. H., Moersch, J., & Wettergreen, D. (2013). Adaptive sampling of time series with application to remote exploration. In Proceedings of IEEE ICRA (pp. 3463–3468).

Wang, Y., & Papageorgiou, M. (2005). Real-time freeway traffic state estimation based on extended Kalman filter: A general approach. Transportation Research Part B: Methodological, 39(2), 141–167.CrossRef

Work, D. B., Blandin, S., Tossavainen, O., & Piccoli, B. (2010). Bayen A (2010) A traffic model for velocity data assimilation. AMRX, 1, 1–35.MATH

Xu, N., Low, K. H., Chen, J., Lim, K. K., & Özgül, E.B. (2014). GP-Localize: Persistent mobile robot localization using online sparse Gaussian process observation model. In Proceedings of AAAI (pp. 2585–2592).

Zhang, K., Tsang, I. W., & Kwok, J. T. (2008). Improved Nyström low-rank approximation and error analysis. In Proceedings of ICML (pp. 1232–1239).

Zhang, Y., Hoang, T. N., Low, K. H., & Kankanhalli, M. (2016). Near-optimal active learning of multi-output Gaussian processes. In Proceedings of AAAI.

Title: Gaussian process decentralized data fusion meets transfer learning in large-scale distributed cooperative perception
Authors: Ruofei Ouyang
Bryan Kian Hsiang Low
Publication date: 28-01-2019
Publisher: Springer US
Published in: Autonomous Robots / Issue 3-4/2020
Print ISSN: 0929-5593
Electronic ISSN: 1573-7527
DOI: https://doi.org/10.1007/s10514-018-09826-z

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Other articles of this Issue 3-4/2020

Cooperative visual-inertial sensor fusion: fundamental equations and state determination in closed-form

Competition and cooperation in a community of autonomous agents

A robust localization system for multi-robot formations based on an extension of a Gaussian mixture probability hypothesis density filter

SOUL: data sharing for robot swarms

Average case constant factor time and distance optimal multi-robot path planning in well-connected environments

Adaptive target tracking with a mixed team of static and mobile guards: deployment and activation strategies