Learning End-to-end Multimodal Sensor Policies for Autonomous Navigation

We proposed a multimodal end-to-end policy based on deep reinforcement learning (DRL) that leverages sensor fusion to reduced performance drops in noisy environment from 50% to 10% compared with the baseline and makes the policy functional even in the face of partial sensor failure by using a novel stochastic technique called Sensor Dropout to reduce sensitivity to any sensor subset, and a new auxiliary loss on policy network along with standard DRL loss that reduces the action variations.