2022
COLING
COLING 2022
Visual Recipe Flow: A Dataset for Learning Visual State Changes of Objects with Recipe Flows
Abstract
AbstractWe present a new multimodal dataset called Visual Recipe Flow, which enables us to learn a cooking action result for each object in a recipe text. The dataset consists of object state changes and the workflow of the recipe text. The state change is represented as an image pair, while the workflow is represented as a recipe flow graph. We developed a web interface to reduce human annotation costs. The dataset allows us to try various applications, including multimodal information retrieval.
🌉
Interdisciplinary Bridge
- Artificial Intelligence and Computer Vision and Natural Language Processing
🐣
Hot Topic Early Bird
- multimodal dataset
Authors
Topics
Artificial Intelligence > Core AI > Multimodal Learning
Computer Vision > Analysis > Scene Understanding
Computer Vision > Generation > Image Captioning
Natural Language Processing > Applications > Information Retrieval
Natural Language Processing > Resources & Methods > Multilingual NLP
Machine Learning > Learning Types > Multi-Task Learning
Computer Vision > Core AI > Multimodal Learning
Machine Learning > Learning Types > Multimodal Learning