A sequence-to-sequence based multi-scale deep learning model for satellite cloud image prediction

Satellite cloud images can help meteorologists characterize the weather patterns, such as identifying tropical cyclone (TC) intensity, detecting climate anomaly regions and predicting rain effects, which makes satellite cloud image forecasting become an important task. In recent years, an increasing number of deep learning models have demonstrated their ability to predict spatiotemporal types of data. However, applying these methods directly to predict satellite cloud images does not consider the chaotic nature of the atmosphere and the satellite images of large scale observation areas, especially in extreme climate events, such as tropical cyclones. Hence, we propose both a novel deep learning model called “Satellite Cloud Spatio-Temporal sequence” (SCSTque) and a dataset benchmark called “Tropical Cyclone Cloudage Map Dataset” (TCCMD) for tropical cyclone satellite images to address these problems. Specifically, the SCSTque model is a sequence-to-sequence (Seq2Seq) autoencoder architecture to use the previous images to predict the next few images, which can fully extract spatial appearance features and temporal dynamics. The TCCMD dataset is a real-world large-scale dataset, which contains satellite image sequences for each corresponding tropical cyclone event from 2010 to 2020. The novel method is validated using the TCCMD dataset and the Moving MNIST dataset. The experimental results proved the SCSTque method outperforms the baseline Seq2Seq-based methods, including a ConvLSTM model, a PredRNN model and a FCLSTM model.