Combining Real-Time Segmentation and Classification of Rehabilitation Exercises with LSTM Networks and Pointwise Boosting

Abstract Autonomous biofeedback tools in support of rehabilitation patients are commonly built as multi-tier pipelines, where a segmentation algorithm is first responsible for isolating motion primitives, and then classification can be performed on each primitive. In this paper, we present a novel segmentation technique that integrates on-the-fly qualitative classification of physical movements in the process. We adopt Long Short-Term Memory (LSTM) networks to model the temporal patterns of a streaming multivariate time series, obtained by sampling acceleration and angular velocity of the limb in motion, and then we aggregate the pointwise predictions of each isolated movement using different boosting methods. We tested our technique against a dataset composed of four common lower-limb rehabilitation exercises, collected from heterogeneous populations (clinical and healthy). Experimental results are promising and show that combining segmentation and classification of orthopaedic movements is a valid method with many potential real-world applications.