LSTM Neural Networks: Input to State Stability and Probabilistic Safety Verification

The goal of this paper is to analyze Long Short Term Memory (LSTM) neural networks from a dynamical system perspective. The classical recursive equations describing the evolution of LSTM can be recast in state space form, resulting in a time invariant nonlinear dynamical system. In this work, a sufficient condition guaranteeing the Input-to-State (ISS) stability property of this system are provided. Then, a discussion on the verification of LSTM networks is provided; in particular, a dedicated approach based on the scenario algorithm is devised. The proposed method is eventually tested on a pH neutralization process.