Multimodal Learning

Leveraging Visual Knowledge in Language Tasks: An Empirical Study on Intermediate Pre-training for Cross-Modal Knowledge Transfer ACL 2022 Lexi: Self-Supervised Learning of the UI Language EMNLP 2022 Linguistic Frameworks Go Toe-to-Toe at Neuro-Symbolic Language Modeling NAACL 2022 Linking Transformer to Hawkes Process for Information Cascade Prediction (Student Abstract) AAAI 2022 Lite-MDETR: A Lightweight Multi-Modal Detector CVPR 2022 LiteVL: Efficient Video-Language Learning with Enhanced Spatial-Temporal Modeling EMNLP 2022 LiT: Zero-Shot Transfer With Locked-Image Text Tuning CVPR 2022 Low-resource Neural Machine Translation with Cross-modal Alignment EMNLP 2022 LVP-M3: Language-aware Visual Prompt for Multilingual Multimodal Machine Translation EMNLP 2022 M3L: Language-Based Video Editing via Multi-Modal Multi-Level Transformers CVPR 2022 M4Singer: A Multi-Style, Multi-Singer and Musical Score Provided Mandarin Singing Corpus NeurIPS 2022 M5Product: Self-Harmonized Contrastive Learning for E-Commercial Multi-Modal Pretraining CVPR 2022 MACK: Multimodal Aligned Conceptual Knowledge for Unpaired Image-text Matching NeurIPS 2022 MAD: A Scalable Dataset for Language Grounding in Videos From Movie Audio Descriptions CVPR 2022 MAESTRO: Matched Speech Text Representations through Modality Matching Interspeech 2022 MCSE: Multimodal Contrastive Learning of Sentence Embeddings NAACL 2022 MedCLIP: Contrastive Learning from Unpaired Medical Images and Text EMNLP 2022 Memotion Analysis through the Lens of Joint Embedding (Student Abstract) AAAI 2022 MERLOT Reserve: Neural Script Knowledge Through Vision and Language and Sound CVPR 2022 MGDoc: Pre-training with Multi-granular Hierarchy for Document Image Understanding EMNLP 2022 Micro and Macro Level Graph Modeling for Graph Variational Auto-Encoders NeurIPS 2022 Mind Reader: Reconstructing complex images from brain activities NeurIPS 2022 Mind the Gap: Cross-Lingual Information Retrieval with Hierarchical Knowledge Enhancement AAAI 2022 Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learning NeurIPS 2022 MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge NeurIPS 2022