Skip to main content
Top

Hint

Swipe to navigate through the chapters of this book

2020 | OriginalPaper | Chapter

Seq2VAR: Multivariate Time Series Representation with Relational Neural Networks and Linear Autoregressive Model

Authors : Edouard Pineau, Sébastien Razakarivony, Thomas Bonald

Published in: Advanced Analytics and Learning on Temporal Data

Publisher: Springer International Publishing

share
SHARE

Abstract

Finding understandable and meaningful feature representation of multivariate time series (MTS) is a difficult task, since information is entangled both in temporal and spatial dimensions. In particular, MTS can be seen as the observation of simultaneous causal interactions between dynamical variables. Standard way to model these interactions is the vector linear autoregression (VAR). The parameters of VAR models can be used as MTS feature representation. Yet, VAR cannot generalize on new samples, hence independent VAR models must be trained to represent different MTS. In this paper, we propose to use the inference capacity of neural networks to overpass this limit. We propose to associate a relational neural network to a VAR generative model to form an encoder-decoder of MTS. The model is denoted Seq2VAR for Sequence-to-VAR. We use recent advances in relational neural network to build our MTS encoder by explicitly modeling interactions between variables of MTS samples. We also propose to leverage reparametrization tricks for binomial sampling in neural networks in order to build a sparse version of Seq2VAR and find back the notion of Granger causality defined in sparse VAR models. We illustrate the interest of our approach through experiments on synthetic datasets.
Appendix
Available only for authorised users
Literature
1.
go back to reference Davis, R.A., Zang, P., Zheng, T.: Sparse vector autoregressive modeling. J. Comput. Graph. Stat. 25(4), 1077–1096 (2016) MathSciNetCrossRef Davis, R.A., Zang, P., Zheng, T.: Sparse vector autoregressive modeling. J. Comput. Graph. Stat. 25(4), 1077–1096 (2016) MathSciNetCrossRef
3.
go back to reference Esling, P., Agon, C.: Time-series data mining. ACM Comput. Surv. (CSUR) 45(1), 12 (2012) CrossRef Esling, P., Agon, C.: Time-series data mining. ACM Comput. Surv. (CSUR) 45(1), 12 (2012) CrossRef
4.
go back to reference Gong, M., Zhang, K., Schoelkopf, B., Tao, D., Geiger, P.: Discovering temporal causal relations from subsampled data. In: International Conference on Machine Learning, pp. 1898–1906 (2015) Gong, M., Zhang, K., Schoelkopf, B., Tao, D., Geiger, P.: Discovering temporal causal relations from subsampled data. In: International Conference on Machine Learning, pp. 1898–1906 (2015)
5.
go back to reference Haufe, S., Müller, K.-R., Nolte, G., Krämer, N., Sparse causal discovery in multivariate time series. In: Causality: Objectives and Assessment, pp. 97–106 (2010) Haufe, S., Müller, K.-R., Nolte, G., Krämer, N., Sparse causal discovery in multivariate time series. In: Causality: Objectives and Assessment, pp. 97–106 (2010)
9.
go back to reference Kipf, T., Fetaya, E., Wang, K.-C., Welling, M., Zemel, R.: Neural relational inference for interacting systems. arXiv preprint arXiv:​1802.​04687 (2018) Kipf, T., Fetaya, E., Wang, K.-C., Welling, M., Zemel, R.: Neural relational inference for interacting systems. arXiv preprint arXiv:​1802.​04687 (2018)
10.
go back to reference Larochelle, H., Murray, I.: The neural autoregressive distribution estimator. In: Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, pp. 29–37 (2011) Larochelle, H., Murray, I.: The neural autoregressive distribution estimator. In: Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, pp. 29–37 (2011)
12.
go back to reference Locatello, F., Bauer, S., Lucic, M., Gelly, S., Schölkopf, B., Bachem, O.: Challenging common assumptions in the unsupervised learning of disentangled representations. arXiv preprint arXiv:​1811.​12359 (2018) Locatello, F., Bauer, S., Lucic, M., Gelly, S., Schölkopf, B., Bachem, O.: Challenging common assumptions in the unsupervised learning of disentangled representations. arXiv preprint arXiv:​1811.​12359 (2018)
13.
14.
15.
go back to reference Maddison, C.J., Mnih, A., Teh, Y.W.: The concrete distribution: a continuous relaxation of discrete random variables. arXiv preprint arXiv:​1611.​00712 (2016) Maddison, C.J., Mnih, A., Teh, Y.W.: The concrete distribution: a continuous relaxation of discrete random variables. arXiv preprint arXiv:​1611.​00712 (2016)
17.
go back to reference Sak, H., Senior, A., Beaufays, F.: Long short-term memory recurrent neural network architectures for large scale acoustic modeling. In: Fifteenth Annual Conference of the International Speech Communication Association (2014) Sak, H., Senior, A., Beaufays, F.: Long short-term memory recurrent neural network architectures for large scale acoustic modeling. In: Fifteenth Annual Conference of the International Speech Communication Association (2014)
18.
go back to reference Santoro, A., et al.: A simple neural network module for relational reasoning. In: Advances in Neural Information Processing Systems, pp. 4967–4976 (2017) Santoro, A., et al.: A simple neural network module for relational reasoning. In: Advances in Neural Information Processing Systems, pp. 4967–4976 (2017)
19.
go back to reference Seabold, S., Perktold, J.: Statsmodels: econometric and statistical modeling with python. In: Proceedings of the 9th Python in Science Conference, vol. 57, p. 61. Scipy (2010) Seabold, S., Perktold, J.: Statsmodels: econometric and statistical modeling with python. In: Proceedings of the 9th Python in Science Conference, vol. 57, p. 61. Scipy (2010)
20.
go back to reference Shen, X., Su, H., Niu, S., Demberg, V.: Improving variational encoder-decoders in dialogue generation. In: Thirty-Second AAAI Conference on Artificial Intelligence (2018) Shen, X., Su, H., Niu, S., Demberg, V.: Improving variational encoder-decoders in dialogue generation. In: Thirty-Second AAAI Conference on Artificial Intelligence (2018)
21.
go back to reference Toda, H.Y., Phillips, P.C.B.: Vector autoregression and causality: a theoretical overview and simulation study. Econ. Rev. 13(2), 259–285 (1994) MathSciNetCrossRef Toda, H.Y., Phillips, P.C.B.: Vector autoregression and causality: a theoretical overview and simulation study. Econ. Rev. 13(2), 259–285 (1994) MathSciNetCrossRef
22.
23.
go back to reference Yi, S., Pavlovic, V.: Sparse granger causality graphs for human action classification. In: Proceedings of the 21st International Conference on Pattern Recognition (ICPR 2012), pp. 3374–3377. IEEE (2012) Yi, S., Pavlovic, V.: Sparse granger causality graphs for human action classification. In: Proceedings of the 21st International Conference on Pattern Recognition (ICPR 2012), pp. 3374–3377. IEEE (2012)
Metadata
Title
Seq2VAR: Multivariate Time Series Representation with Relational Neural Networks and Linear Autoregressive Model
Authors
Edouard Pineau
Sébastien Razakarivony
Thomas Bonald
Copyright Year
2020
DOI
https://doi.org/10.1007/978-3-030-39098-3_10

Premium Partner