Reinforcement learning-driven parametric curve fitting for snake robot gait design

Snake-inspired robots demonstrate exceptional versatility through challenging terrains such as sand, rubble, and ice. However, their high-dimensional continuous action spaces make analytical gait design challenging. Early works by Hirose (1994) showed that gait parameterization over low-dimensional spatially and temporally varying sine waves can serve as basis functions for the shape-space or central pattern generators (CPGs). Recent approaches to designing CPGs have combined annealed chain-fitting, which solves for joint angles that fit a snake robot to a desired backbone curve, and keyframe extraction, which then fits analytic shape functions to the resulting optimized joint angles. However, the non-convex optimization associated with these methods is fraught with local optima exacerbated by constraints such as actuator limits. Reinforcement Learning has emerged as a promising alternative for searching over such spaces. However, end-to-end RL approaches trained purely in simulation are vulnerable to reality distribution shifts, lack safety guarantees, and don’t yield an intuitive representation of the learned gait. We propose a method that translates a gait found via policy search into a parametric representation of its component sinusoidal equations thus leveraging the strengths of both learning-based and classical approaches. Simulation and hardware experiments show that the proposed pipeline can generate parametric gaits where classical curve fitting-based approaches fail.