Multimodal NLP

ADAPT: Vision-Language Navigation With Modality-Aligned Action Prompts CVPR 2022 Bootstrapping a high quality multilingual multimodal dataset for Bletchley ACML 2022 Bridging between Cognitive Processing Signals and Linguistic Features via a Unified Attentional Network AAAI 2022 CogBERT: Cognition-Guided Pre-trained Language Models COLING 2022 Focus! Relevant and Sufficient Context Selection for News Image Captioning EMNLP 2022 LAVT: Language-Aware Vision Transformer for Referring Image Segmentation CVPR 2022 Multi-Grained Vision Language Pre-Training: Aligning Texts with Visual Concepts ICML 2022 RaP: Redundancy-aware Video-language Pre-training for Text-Video Retrieval EMNLP 2022 Revisiting the "Video" in Video-Language Understanding CVPR 2022 Separate What You Describe: Language-Queried Audio Source Separation Interspeech 2022 TaiSu: A 166M Large-scale High-Quality Dataset for Chinese Vision-Language Pre-training NeurIPS 2022 UniTranSeR: A Unified Transformer Semantic Representation Framework for Multimodal Task-Oriented Dialog System ACL 2022 Unsupervised Vision-and-Language Pre-Training via Retrieval-Based Multi-Granular Alignment CVPR 2022 Visual Grounding of Inter-lingual Word-Embeddings EMNLP 2022 Why is Winoground Hard? Investigating Failures in Visuolinguistic Compositionality EMNLP 2022 XYLayoutLM: Towards Layout-Aware Multimodal Networks for Visually-Rich Document Understanding CVPR 2022 Audio-Oriented Multimodal Machine Comprehension via Dynamic Inter- and Intra-modality Attention AAAI 2021 DocFormer: End-to-End Transformer for Document Understanding ICCV 2021 Effect of Visual Extensions on Natural Language Understanding in Vision-and-Language Models EMNLP 2021 LAMS: A Location-aware Approach for Multimodal Summarization (Student Abstract) AAAI 2021 M3P: Learning Universal Representations via Multitask Multilingual Multimodal Pre-Training CVPR 2021 The World of an Octopus: How Reporting Bias Influences a Language Model’s Perception of Color EMNLP 2021 Transductive Visual Verb Sense Disambiguation WACV 2021 Vision-and-Language or Vision-for-Language? On Cross-Modal Influence in Multimodal Transformers EMNLP 2021 Visual Goal-Step Inference using wikiHow EMNLP 2021