Runoff Prediction Using State-Cross-Attention LSTM Model with Comparisons to Transformer

Accurate runoff prediction plays a key role in water resources management. It is challenging due to its high nonlinearity and complexity. In recent years, long short-term memory (LSTM)s and Transformers have achieved high prediction accuracies. These two sorts have their pros and cons in encoding time series information and connecting time points. In this paper, we propose a State-Cross-Attention LSTM (SCA-LSTM) model for one-step runoff predictions, which can encode time series information naturally due to its recurrent structure and can connect the state of last LSTM cell with that of other cells flexibly with its SCA module. Additionally, the SCA module considers the relation between the last state and other states directly and efficiently, rather than redundant relations among global states. Thus, this model has advantages of both LSTM and Transformer, i.e., it utilizes time series information of dynamic inputs and focuses on more important states. We employ the CAMELS-AUS dataset (i.e., the Australia version of CAMELS) and test the performance on 172 catchments, in which the proposed SCA-LSTM model outperforms two Transformer models and three LSTM models in prediction accuracy and transfer ability. Specifically, the median of Nash-Sutcliffe efficiency for 172 catchments provided by SCA-LSTM in one-step prediction and k-fold testing are 0.8150 and 0.7423, in which those provided by the best benchmarks are 0.7984 and 0.7233, respectively.