Multimodal Learning

SpeechForensics: Audio-Visual Speech Representation Learning for Face Forgery Detection NeurIPS 2024 SpeechGuard: Exploring the Adversarial Robustness of Multi-modal Large Language Models ACL 2024 Speech Translation with Speech Foundation Models and Large Language Models: What is There and What is Missing? ACL 2024 Speechworthy Instruction-tuned Language Models EMNLP 2024 Split to Merge: Unifying Separated Modalities for Unsupervised Domain Adaptation CVPR 2024 SRTube: Video-Language Pre-Training with Action-Centric Video Tube Features and Semantic Role Labeling CVPR 2024 STDiff: Spatio-Temporal Diffusion for Continuous Stochastic Video Prediction AAAI 2024 Step Differences in Instructional Video CVPR 2024 STICKERCONV: Generating Multimodal Empathetic Responses from Scratch ACL 2024 STimage-1K4M: A histopathology image-gene expression dataset for spatial transcriptomics NeurIPS 2024 Stitching Segments and Sentences towards Generalization in Video-Text Pre-training AAAI 2024 StreamingFlow: Streaming Occupancy Forecasting with Asynchronous Multi-modal Data Streams via Neural Ordinary Differential Equation CVPR 2024 Structural Information Guided Multimodal Pre-training for Vehicle-Centric Perception AAAI 2024 Structure-CLIP: Towards Scene Graph Knowledge to Enhance Multi-Modal Structured Representations AAAI 2024 Summarize the Past to Predict the Future: Natural Language Descriptions of Context Boost Multimodal Object Interaction Anticipation CVPR 2024 SURf: Teaching Large Vision-Language Models to Selectively Utilize Retrieved Information EMNLP 2024 SWAG: Storytelling With Action Guidance EMNLP 2024 SyncMask: Synchronized Attentional Masking for Fashion-centric Vision-Language Pretraining CVPR 2024 Synergetic Event Understanding: A Collaborative Approach to Cross-Document Event Coreference Resolution with Large Language Models ACL 2024 Synthetic Multimodal Question Generation EMNLP 2024 Tables as Texts or Images: Evaluating the Table Reasoning Ability of LLMs and MLLMs ACL 2024 Tactile-Augmented Radiance Fields CVPR 2024 Tag-grounded Visual Instruction Tuning with Retrieval Augmentation EMNLP 2024 TAP4LLM: Table Provider on Sampling, Augmenting, and Packing Semi-structured Data for Large Language Model Reasoning EMNLP 2024 Task-Driven Exploration: Decoupling and Inter-Task Feedback for Joint Moment Retrieval and Highlight Detection CVPR 2024