Interpretable Transformer Neural Network Prediction of Diverse Environmental Time Series Using Weather Forecasts

Transformer Neural Networks (TNNs) have caused a paradigm shift in deep learning domains like natural language processing and gathered immense interest due to their versatility in other fields such as time series forecasting (TSF). Most current TSF applications of TNNs use only historic observations to predict future events, ignoring information available in weather forecasts to inform better predictions, and with little attention given to the interpretability of the model’s use of environmental input factors. This work explores the potential for TNNs to perform TSF across multiple environmental variables (streamflow, stage, water temperature, and salinity) in two ecologically important regions: the Peace River watershed (Florida) and the northern Gulf of Mexico (Louisiana). The TNN was tested (and uncertainty quantified) for each response variable from one- to fourteen-day-ahead forecasts using past observations and spatially distributed weather forecasts. Additionally, a sensitivity analysis was performed on the trained TNNs’ attention weights to identify the relative influence of each input variable on each response variable’s prediction. Overall model performance ranged from good to very good (0.78