Action Anticipation with RBF Kernelized Feature Mapping RNN

We introduce a novel Recurrent Neural Network-based algorithm for future video feature generation and action anticipation called ame. Our novel RNN architecture builds upon three effective principles of machine learning, 1. parameter sharing, 2. Radial basis function kernels and 3. Adversarial training. Using only a fraction of earliest frames of a video, we are able to generate accurate future features thanks to the generelization capacity of our novel RNN. Using a simple two layered MLP facilitated with a RBF kernel layer, we classify generated future features for the action anticipation. In our experiments, we obtain 18% improvement on JHMDB-21 dataset, 6% on UCF101-24 and 13% improvement on UT-Interaction datasets over prior state-of-the-art for action anticipation.