Skip to content
conftrace_
2018 CVPR CVPR 2018

Demo2Vec: Reasoning Object Affordances From Online Videos

Abstract

Watching expert demonstrations is an important way for humans and robots to reason about affordances of unseen objects. In this paper, we consider the problem of reasoning object affordances through the feature embedding of demonstration videos. We design the Demo2Vec model which learns to extract embedded vectors of demonstration videos and predicts the interaction region and the action label on a target image of the same object. We introduce the Online Product Review dataset for Affordance (OPRA) by collecting and labeling diverse YouTube product review videos. Our Demo2Vec model outperforms various recurrent neural network baselines on the collected dataset.

🌉 Interdisciplinary Bridge - Artificial Intelligence and Computer Vision and Robotics
🐣 Hot Topic Early Bird - visual reasoning

Authors

Kuan Fang, Te-Lin Wu, Daniel Yang, Silvio Savarese, Joseph J. Lim

Topics

Machine Learning > Core Methods > Representation Learning Machine Learning > Core Methods > Embedding Learning Computer Vision > Analysis > Object Detection Deep Learning > Learning Types > Representation Learning Robotics > Applications > Robotics

Keywords

action recognition feature embedding video understanding visual reasoning demonstration learning recurrent neural network convolutional network video embedding affordance prediction object affordance
Download PDF

More from CVPR 2018

All CVPR 2018 papers →
Multi-Level Factorisation Net for Person Re-Identification
Attention-Aware Compositional Network for Person Re-Identification
Look at Boundary: A Boundary-Aware Face Alignment Algorithm
Monocular 3D Pose and Shape Estimation of Multiple People in Natural Scenes - The Importance of Multiple Scene Constraints
3D Human Sensing, Action and Emotion Recognition in Robot Assisted Therapy of Children With Autism
Facial Expression Recognition by De-Expression Residue Learning