Yonatan Bisk

MM-SeR: Multimodal Self-Refinement for Lightweight Image Captioning CVPR 2026 Unsupervised Discovery of Long-Term Spatiotemporal Periodic Workflows in Human Activities WACV 2026 CASPER: Inferring Diverse Intents for Assistive Teleoperation with Vision Language Models CoRL 2025 Direct Preference Optimization of Video Large Multimodal Models from Language Model Reward NAACL 2025 Energy Considerations of Large Language Model Inference and Efficiency Optimizations ACL 2025 Explore Theory of Mind: program-guided adversarial data generation for theory of mind reasoning ICLR 2025 Language Models Need Inductive Biases to Count Inductively ICLR 2025 MolErr2Fix: Benchmarking LLM Trustworthiness in Chemistry via Modular Error Detection, Localization, Explanation, and Correction EMNLP 2025 Position: You Can’t Manufacture a NeRF ICML 2025 Re-thinking Temporal Search for Long-Form Video Understanding CVPR 2025 ANAVI: Audio Noise Awareness using Visual of Indoor environments for NAVIgation CoRL 2024 Diffusion PID: Interpreting Diffusion via Partial Information Decomposition NeurIPS 2024 Gradient Localization Improves Lifelong Pretraining of Language Models EMNLP 2024 How to Train Your Fact Verifier: Knowledge Transfer with Multimodal Open Models EMNLP 2024 OpenEQA: Embodied Question Answering in the Era of Foundation Models CVPR 2024 Situated Instruction Following ECCV 2024 Skews in the Phenomenon Space Hinder Generalization in Text-to-Image Generation ECCV 2024 SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents ICLR 2024 SOTOPIA-π: Interactive Learning of Socially Intelligent Language Agents ACL 2024 Tools Fail: Detecting Silent Errors in Faulty Tools EMNLP 2024 Vid2Robot: End-to-end Video-conditioned Policy Learning with Cross-Attention Transformers RSS 2024 VISREAS: Complex Visual Reasoning with Unanswerable Questions ACL 2024 WebArena: A Realistic Web Environment for Building Autonomous Agents ICLR 2024 Computational Language Acquisition with Theory of Mind ICLR 2023 EXCALIBUR: Encouraging and Evaluating Embodied Exploration CVPR 2023 HomeRobot: Open-Vocabulary Mobile Manipulation CoRL 2023 SLAP: Spatial-Language Attention Policies CoRL 2023 SPAE: Semantic Pyramid AutoEncoder for Multimodal Generation with Frozen LLMs NeurIPS 2023 SPRING: Studying Papers and Reasoning to play Games NeurIPS 2023 The Framework Tax: Disparities Between Inference Efficiency in NLP Research and Deployment EMNLP 2023 A Framework for Learning to Request Rich and Contextually Useful Information from Humans ICML 2022 Don’t Copy the Teacher: Data and Model Challenges in Embodied Dialogue EMNLP 2022 EvEntS ReaLM: Event Reasoning of Entity States via Language Models EMNLP 2022 FILM: Following Instructions in Language with Modular Methods ICLR 2022 KAT: A Knowledge Augmented Transformer for Vision-and-Language NAACL 2022 On Advances in Text Generation from Images Beyond Captioning: A Case Study in Self-Rationalization EMNLP 2022 Symmetric Machine Theory of Mind ICML 2022 Transformers Are Adaptable Task Planners CoRL 2022 WebQA: Multihop and Multimodal QA CVPR 2022 Worst of Both Worlds: Biases Compound in Pre-trained Vision-and-Language Models NAACL 2022 ALFWorld: Aligning Text and Embodied Environments for Interactive Learning ICLR 2021 An Empirical Study on the Generalization Power of Neural Representations Learned via Visual Guessing Games EACL 2021 Dependency Induction Through the Lens of Visual Perception CoNLL 2021 Few-shot Language Coordination by Modeling Theory of Mind ICML 2021 Grounding ‘Grounding’ in NLP ACL 2021 Knowledge-driven Data Construction for Zero-shot Evaluation in Commonsense Question Answering AAAI 2021 Language Grounding with 3D Objects CoRL 2021 TACo: Token-Aware Cascade Contrastive Learning for Video-Text Alignment ICCV 2021 A Benchmark for Structured Procedural Knowledge Extraction from Cooking Videos EMNLP 2020 ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Tasks CVPR 2020 Experience Grounds Language EMNLP 2020 Imagining Grounded Conceptual Representations from Perceptual Information in Situated Guessing Games COLING 2020 PIQA: Reasoning about Physical Commonsense in Natural Language AAAI 2020 RMM: A Recursive Mental Model for Dialogue Navigation EMNLP 2020 Benchmarking Hierarchical Script Knowledge NAACL 2019 Defending Against Neural Fake News NeurIPS 2019 From Recognition to Cognition: Visual Commonsense Reasoning CVPR 2019 HellaSwag: Can a Machine Really Finish Your Sentence? ACL 2019 Proceedings of the Combined Workshop on Spatial Language Understanding (SpLU) and Grounded Communication for Robotics (RoboNLP) NAACL 2019 Robust Navigation with Language Pretraining and Stochastic Sampling EMNLP 2019 Shifting the Baseline: Single Modality Performance on Visual Navigation & QA NAACL 2019 Tactical Rewind: Self-Correction via Backtracking in Vision-And-Language Navigation CVPR 2019 Inducing Grammars with and for Neural Machine Translation ACL 2018 Proceedings of the Workshop on Generalization in the Age of Deep Learning NAACL 2018 SWAG: A Large-Scale Adversarial Dataset for Grounded Commonsense Inference EMNLP 2018 Synthetic and Natural Noise Both Break Neural Machine Translation ICLR 2018 Natural Language Inference from Multiple Premises IJCNLP 2017 Evaluating Induced CCG Parsers on Grounded Semantic Parsing EMNLP 2016 Natural Language Communication with Robots NAACL 2016 Supertagging With LSTMs NAACL 2016 Labeled Grammar Induction with Minimal Supervision ACL 2015 Probing the Linguistic Strengths and Limitations of Unsupervised Grammar Induction ACL 2015 Induction of Linguistic Structure with Combinatory Categorial Grammars NAACL 2012 Normal-form parsing for Combinatory Categorial Grammars with generalized composition and type-raising COLING 2010