Self-Organizing Deep Learning Architectures For Scalable Data Analysis In Environmental Climate Prediction Applications

Abstract

Environmental climate predictions through deep learning architectures such as Convolutional Recurrent Networks or even large Transformer-based weather forecasters are often engineered based on an a priori decided upon architecture and capacity of the model, thereby necessitating a manual re-designing of the network whenever there is an increase in size, geographical coverage, and complexity of the input data, a problem that frequently occurs as climate observation networks evolve from a few local networks to full-scale continental or even global networks. In this paper, we introduce the Self-Organizing Deep Architecture (SODA) model where a competitive, Kohonen-type self-organizing topology layer assigns the incoming spatiotemporal patterns of the climate into an increasing number of regional experts, creating new experts as soon as the quantization error-based complexity measure indicates the inadequacy of the existing topology. Every expert has a deep spatiotemporal prediction backbone, and an aggregated map-reduce style layer merges the predictions of all experts into a global prediction, whereas a reorganization cycle periodically changes the architecture whenever the climate regime changes due to events like ENSO phase changes. The performance of this framework is measured for multivariate climate data modeled based on ERA5 reanalysis data variables for four different data-scale scenarios: single-region, multi-region, continental-scale, and global-scale with extreme events. With respect to a CNN-LSTM baseline model and a Transformer baseline model with a fixed architecture, SODA framework decreases temperature prediction RMSE from 2.87°C and 2.31°C to 1.39°C in the global-scale scenario, and also achieves higher training throughput, which suggests that self-organizing growth of architecture results in better performance and scalability with increasing climate data complexity.