Shafiq Joty

Multimodal Large Language Models for Human-AI Interaction: Foundations, Agents, and Inclusive Applications EACL 2026 Aligning Text, Code, and Vision: A Multi-Objective Reinforcement Learning Framework for Text-to-Visualization EACL 2026 Hard2Verify: A Step-Level Verification Benchmark for Open-Ended Frontier Math ACL 2026 DashboardQA: Benchmarking Multimodal Agents for Question Answering on Interactive Dashboards EACL 2026 J4R: Learning to Judge with Equivalent Initial State Group Relative Policy Optimization ACL 2026 DnA-Eval: Enhancing Large Language Model Evaluation through Decomposition and Aggregation COLING 2025 ChartGemma: Visual Instruction-tuning for Chart Reasoning in the Wild COLING 2025 From Charts to Fair Narratives: Uncovering and Mitigating Geo-Economic Biases in Chart-to-Text EMNLP 2025 Why Vision Language Models Struggle with Visual Arithmetic? Towards Enhanced Chart and Geometry Understanding ACL 2025 ChartQAPro: A More Diverse and Challenging Benchmark for Chart Question Answering ACL 2025 Relevant or Random: Can LLMs Truly Perform Analogical Reasoning? ACL 2025 Evaluating Judges as Evaluators: The JETTS Benchmark of LLM-as-Judges as Test-Time Scaling Evaluators ICML 2025 Text2Vis: A Challenging and Diverse Benchmark for Generating Multimodal Visualizations from Text EMNLP 2025 Deploying Tiny LVLM Judges for Real-World Evaluation of Chart Models: Lessons Learned and Best Practices EMNLP 2025 Topic-Guided Reinforcement Learning with LLMs for Enhancing Multi-Document Summarization EMNLP 2025 LLMs Are Biased Towards Output Formats! Systematically Evaluating and Mitigating Output Format Bias of LLMs NAACL 2025 On Positional Bias of Faithfulness for Long-form Summarization NAACL 2025 ReIFE: Re-evaluating Instruction-Following Evaluation NAACL 2025 ParaICL: Towards Parallel In-Context Learning NAACL 2025 Adaptation of Large Language Models NAACL 2025 Multi2: Multi-Agent Test-Time Scalable Framework for Multi-Document Processing EMNLP 2025 Investigating Factuality in Long-Form Text Generation: The Roles of Self-Known and Self-Unknownan EMNLP 2025 FaithEval: Can Your Language Model Stay Faithful to Context, Even If "The Moon is Made of Marshmallows" ICLR 2025 Preference Optimization for Reasoning with Pseudo Feedback ICLR 2025 Direct Judgement Preference Optimization EMNLP 2025 CEMTM: Contextual Embedding-based Multimodal Topic Modeling EMNLP 2025 Beyond In-Context Learning: Aligning Long-form Generation of Large Language Models via Task-Inherent Attribute Guidelines ACL 2025 Demystifying Domain-adaptive Post-training for Financial LLMs EMNLP 2025 What Makes a Good Natural Language Prompt? ACL 2025 Does Context Matter? ContextualJudgeBench for Evaluating LLM-based Judges in Contextual Settings ACL 2025 Can We Further Elicit Reasoning in LLMs? Critic-Guided Planning with Retrieval-Augmentation for Solving Challenging Tasks ACL 2025 Beyond Output Matching: Bidirectional Alignment for Enhanced In-Context Learning ACL 2025 Learning Auxiliary Tasks Improves Reference-Free Hallucination Detection in Open-Domain Long-Form Generation ACL 2025 Judging the Judges: Can Large Vision-Language Models Fairly Evaluate Chart Comprehension and Reasoning? ACL 2025 Democratizing LLMs for Low-Resource Languages by Leveraging their English Dominant Abilities with Linguistically-Diverse Prompts ACL 2024 XCodeEval: An Execution-based Large Scale Multilingual Multitask Benchmark for Code Understanding, Generation, Translation and Retrieval ACL 2024 Learning Planning-based Reasoning by Trajectories Collection and Process Reward Synthesizing EMNLP 2024 Evaluating Psychological Safety of Large Language Models EMNLP 2024 A Systematic Survey and Critical Review on Evaluating Large Language Models: Challenges, Limitations, and Recommendations EMNLP 2024 Diffusion Model Alignment Using Direct Preference Optimization CVPR 2024 DataNarrative: Automated Data-Driven Storytelling with Visualizations and Texts EMNLP 2024 FOLIO: Natural Language Reasoning with First-Order Logic EMNLP 2024 Prompt Leakage effect and mitigation strategies for multi-turn LLM Applications EMNLP 2024 Open-RAG: Enhanced Retrieval Augmented Reasoning with Open-Source Large Language Models EMNLP 2024 Benchmarking Generation and Evaluation Capabilities of Large Language Models for Instruction Controllable Summarization NAACL 2024 AdaPT: A Set of Guidelines for Hyperbolic Multimodal Multilingual NLP NAACL 2024 DIVKNOWQA: Assessing the Reasoning Ability of LLMs via Open-Domain Question Answering over Knowledge Base and Text NAACL 2024 Lifelong Event Detection with Embedding Space Separation and Compaction NAACL 2024 P-FOLIO: Evaluating and Improving Logical Reasoning with Abundant Human-Written Reasoning Chains EMNLP 2024 Explaining Language Model Predictions with High-Impact Concepts EACL 2024 Data Augmentation using LLMs: Data Perspectives, Learning Paradigms and Challenges ACL 2024 ChartInstruct: Instruction Tuning for Chart Comprehension and Reasoning ACL 2024 XL-HeadTags: Leveraging Multimodal Retrieval Augmentation for the Multilingual Generation of News Headlines and Tags ACL 2024 Efficiently Aligned Cross-Lingual Transfer Learning for Conversational Tasks using Prompt-Tuning EACL 2024 "X-InstructBLIP: A Framework for Aligning Image, 3D, Audio, Video to LLMs and its Emergent Cross-modal Reasoning" ECCV 2024 Embrace Divergence for Richer Insights: A Multi-document Summarization Benchmark and a Case Study on Summarizing Diverse Information from News Articles NAACL 2024 Exploring Self-supervised Logic-enhanced Training for Large Language Models NAACL 2024 Modeling Uncertainty and Using Post-fusion as Fallback Improves Retrieval Augmented Generation with LLMs ACL 2024 On Context Utilization in Summarization with Large Language Models ACL 2024 Chain-of-Knowledge: Grounding Large Language Models via Dynamic Knowledge Adapting over Heterogeneous Sources ICLR 2024 CodeChain: Towards Modular Code Generation Through Chain of Self-revisions with Representative Sub-modules ICLR 2024 Retrieving Multimodal Information for Augmented Generation: A Survey EMNLP 2023 Did You Read the Instructions? Rethinking the Effectiveness of Task Definitions in Instruction Learning ACL 2023 Revisiting the Gold Standard: Grounding Summarization Evaluation with Robust Human Evaluation ACL 2023 Towards Robust Low-Resource Fine-Tuning with Multi-View Compressed Representations ACL 2023 Randomized Smoothing with Masked Inference for Adversarially Robust Text Classifications ACL 2023 Verify-and-Edit: A Knowledge-Enhanced Chain-of-Thought Framework ACL 2023 SWiPE: A Dataset for Document-Level Simplification of Wikipedia Pages ACL 2023 Modeling What-to-ask and How-to-ask for Answer-unaware Conversational Question Generation ACL 2023 Is GPT-3 a Good Data Annotator? ACL 2023 Learning to Initialize: Can Meta Learning Improve Cross-task Generalization in Prompt Tuning? ACL 2023 A Systematic Study and Comprehensive Evaluation of ChatGPT on Benchmark Datasets ACL 2023 Unsupervised Summarization Re-ranking ACL 2023 Contrastive Learning with Generated Representations for Inductive Knowledge Graph Embedding ACL 2023 A Data-centric Framework for Improving Domain-specific Machine Reading Comprehension Datasets EACL 2023 Lifelong Sequence Generation with Dynamic Module Expansion and Adaptation EMNLP 2023 Personalized Distillation: Empowering Open-Sourced LLMs with Adaptive Learning for Code Generation EMNLP 2023 Towards Low-Resource Automatic Program Repair with Meta-Learning and Pretrained Language Models EMNLP 2023 SummEdits: Measuring LLM Ability at Factual Reasoning Through The Lens of Summarization EMNLP 2023 UniChart: A Universal Vision-language Pretrained Model for Chart Comprehension and Reasoning EMNLP 2023 Towards Interpretable and Efficient Automatic Reference-Based Summarization Evaluation EMNLP 2023 NLP+Vis: NLP Meets Visualization EMNLP 2023 HPE: Answering Complex Questions over Text by Hybrid Question Parsing and Execution EMNLP 2023 Refining Low-Resource Unsupervised Translation by Language Disentanglement of Multilingual Translation Model NIPS 2022 Continual Few-shot Relation Learning via Embedding Space Regularization and Data Augmentation ACL 2022 GlobalWoZ: Globalizing MultiWoZ to Develop Multilingual Task-Oriented Dialogue Systems ACL 2022 ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning ACL 2022 Rethinking Self-Supervision Objectives for Generalizable Coherence Modeling ACL 2022 Learning Label Modular Prompts for Text Classification in the Wild EMNLP 2022 Enhancing Multilingual Language Model with Massive Multilingual Knowledge Triples EMNLP 2022 Towards Summary Candidates Fusion EMNLP 2022 OpenCQA: Open-ended Question Answering with Charts EMNLP 2022 BotSIM: An End-to-End Bot Simulation Framework for Commercial Task-Oriented Dialog Systems EMNLP 2022 Data Selection Curriculum for Neural Machine Translation EMNLP 2022 SummaReranker: A Multi-Task Mixture-of-Experts Re-ranking Framework for Abstractive Summarization ACL 2022 Chart-to-Text: A Large-Scale Benchmark for Chart Summarization ACL 2022 Contrastive Clustering to Mine Pseudo Parallel Data for Unsupervised Translation ICLR 2022 Alleviating Sparsity of Open Knowledge Graphs with Ternary Contrastive Learning EMNLP 2022 Towards Multi-Sense Cross-Lingual Alignment of Contextual Embeddings COLING 2022 CoHS-CQG: Context and History Selection for Conversational Question Generation COLING 2022 UNISON: Unpaired Cross-Lingual Image Captioning AAAI 2022 LFPT5: A Unified Framework for Lifelong Few-shot Language Learning Based on Prompt Tuning of T5 ICLR 2022 Weakly Supervised Neuro-Symbolic Module Networks for Numerical Reasoning over Text AAAI 2022 AugVic: Exploiting BiText Vicinity for Low-Resource NMT IJCNLP 2021 MulDA: A Multilingual Data Augmentation Framework for Low-Resource Cross-Lingual NER IJCNLP 2021 GeDi: Generative Discriminator Guided Sequence Generation EMNLP 2021 A Conditional Splitting Framework for Efficient Constituency Parsing IJCNLP 2021 Reliability Testing for Natural Language Processing Systems IJCNLP 2021 UXLA: A Robust Unsupervised Data Augmentation Framework for Zero-Resource Cross-Lingual NLP IJCNLP 2021 Addressing the Vulnerability of NMT in Input Perturbations NAACL 2021 Cross-model Back-translated Distillation for Unsupervised Machine Translation ICML 2021 Straight to the Gradient: Learning to Use Novel Tokens for Neural Text Generation ICML 2021 AugVic: Exploiting BiText Vicinity for Low-Resource NMT ACL 2021 MulDA: A Multilingual Data Augmentation Framework for Low-Resource Cross-Lingual NER ACL 2021 A Conditional Splitting Framework for Efficient Constituency Parsing ACL 2021 Reliability Testing for Natural Language Processing Systems ACL 2021 UXLA: A Robust Unsupervised Data Augmentation Framework for Zero-Resource Cross-Lingual NLP ACL 2021 Align before Fuse: Vision and Language Representation Learning with Momentum Distillation NIPS 2021 Effective Fine-Tuning Methods for Cross-lingual Adaptation EMNLP 2021 CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation EMNLP 2021 A Unified Speaker Adaptation Approach for ASR EMNLP 2021 Code-Mixing on Sesame Street: Dawn of the Adversarial Polyglots NAACL 2021 Improving Zero and Few-Shot Abstractive Summarization with Intermediate Fine-tuning and Data Augmentation NAACL 2021 RST Parsing from Scratch NAACL 2021 Rethinking Coherence Modeling: Synthetic vs. Downstream Tasks EACL 2021 Pronoun-Targeted Fine-tuning for NMT with Hybrid Losses EMNLP 2020 Data Diversification: A Simple Strategy For Neural Machine Translation NIPS 2020 Zero-Resource Cross-Lingual Named Entity Recognition AAAI 2020 Explicit Memory Tracker with Coarse-to-Fine Reasoning for Conversational Machine Reading ACL 2020 It’s Morphin’ Time! Combating Linguistic Discrimination with Inflectional Perturbations ACL 2020 Efficient Constituency Parsing by Pointing ACL 2020 Differentiable Window for Dynamic Local Attention ACL 2020 Finding It at Another Side: A Viewpoint-Adapted Matching Encoder for Change Captioning ECCV 2020 Self-Supervised Relationship Probing NIPS 2020 Discern: Discourse-Aware Entailment Reasoning Network for Conversational Machine Reading EMNLP 2020 LNMap: Departures from Isomorphic Assumption in Bilingual Lexicon Induction Through Non-Linear Mapping in Latent Space EMNLP 2020 VD-BERT: A Unified Vision and Dialog Transformer with BERT EMNLP 2020 Mind Your Inflections! Improving NLP for Non-Standard Englishes with Base-Inflection Encoding EMNLP 2020 DAGA: Data Augmentation with a Generation Approach for Low-resource Tagging Tasks EMNLP 2020 Online Conversation Disentanglement with Pointer Networks EMNLP 2020 Response Selection for Multi-Party Conversations with Dynamic Topic Tracking EMNLP 2020 Tree-Structured Attention with Hierarchical Accumulation ICLR 2020 Speech Transformer with Speaker Aware Persistent Memory INTERSPEECH 2020 Universal Speech Transformer INTERSPEECH 2020 Cross Attention with Monotonic Alignment for Speech Transformer INTERSPEECH 2020 Revisiting Adversarial Autoencoder for Unsupervised Word Translation with Cycle Consistency and Improved Training NAACL 2019 Discourse Analysis and Its Applications ACL 2019 Hierarchical Pointer Net Parsing EMNLP 2019 Unpaired Image Captioning via Scene Graph Alignments ICCV 2019 Evaluating Pronominal Anaphora in Machine Translation: An Evaluation Measure and a Test Suite EMNLP 2019 Using Clinical Notes with Time Series Data for ICU Management EMNLP 2019 A Unified Neural Coherence Model EMNLP 2019 Hierarchical Pointer Net Parsing IJCNLP 2019 A Unified Neural Coherence Model IJCNLP 2019 Evaluating Pronominal Anaphora in Machine Translation: An Evaluation Measure and a Test Suite IJCNLP 2019 Using Clinical Notes with Time Series Data for ICU Management IJCNLP 2019 Sentence-Level Evidence Embedding for Claim Verification with Hierarchical Attention Networks ACL 2019 Adversarial Unsupervised Representation Learning for Activity Time-Series AAAI 2019 A Unified Linear-Time Framework for Sentence-Level Discourse Parsing ACL 2019 Adaptation of Hierarchical Structured Models for Speech Act Recognition in Asynchronous Conversation NAACL 2019 Domain Adaptation with Adversarial Training and Graph Embeddings ACL 2018 NLP for Conversations: Sentiment, Summarization, and Group Dynamics COLING 2018 Coherence Modeling of Asynchronous Conversations: A Neural Entity Grid Approach ACL 2018 SegBot: A Generic Neural Text Segmentation Model with Pointer Network IJCAI 2018 Joint Multitask Learning for Community Question Answering Using Task-Specific Embeddings EMNLP 2018 Unpaired Image Captioning by Language Pivoting ECCV 2018 VQA-E: Explaining, Elaborating, and Enhancing Your Answers for Visual Questions ECCV 2018 Cross-language Learning with Adversarial Neural Networks CONLL 2017 A Neural Local Coherence Model ACL 2017 ConvKN at SemEval-2016 Task 3: Answer and Question Selection for Question Answering on Arabic and English Fora SEMEVAL 2016 Joint Learning with Global Inference for Comment Classification in Community Question Answering NAACL 2016 Speech Act Modeling of Written Asynchronous Conversations with Task-Specific Embeddings and Conditional Structured Models ACL 2016 A Deep Fusion Model for Domain Adaptation in Phrase-based MT COLING 2016 An Interactive System for Exploring Community Question Answering Forums COLING 2016 Thread-Level Information for Comment Classification in Community Question Answering IJCNLP 2015 Fine-grained Opinion Mining with Recurrent Neural Networks and Word Embeddings EMNLP 2015 How to Avoid Unwanted Pregnancies: Domain Adaptation using Neural Network Models EMNLP 2015 Global Thread-level Inference for Comment Classification in Community Question Answering EMNLP 2015 Pairwise Neural Machine Translation Evaluation IJCNLP 2015 Thread-Level Information for Comment Classification in Community Question Answering ACL 2015 Pairwise Neural Machine Translation Evaluation ACL 2015 QCRI: Answer Selection for Community Question Answering - Experiments for Arabic and English SEMEVAL 2015 Semantic Kernels for Semantic Parsing EMNLP 2014 A Study of using Syntactic and Semantic Structures for Concept Segmentation and Labeling COLING 2014 Learning to Differentiate Better from Worse Translations EMNLP 2014 Using Discourse Structure Improves Machine Translation Evaluation ACL 2014 Discriminative Reranking of Discourse Parses Using Tree Kernels EMNLP 2014 Towards Topic Labeling with Phrase Entailment and Aggregation NAACL 2013 Combining Intra- and Multi-sentential Rhetorical Parsing for Document-level Discourse Analysis ACL 2013 A Novel Discriminative Framework for Sentence-Level Discourse Analysis CONLL 2012 A Novel Discriminative Framework for Sentence-Level Discourse Analysis EMNLP 2012 Exploiting Conversation Structure in Unsupervised Topic Segmentation for Emails EMNLP 2010 Do Automatic Annotation Techniques Have Any Impact on Supervised Complex Question Answering? IJCNLP 2009 Do Automatic Annotation Techniques Have Any Impact on Supervised Complex Question Answering? ACL 2009 Selecting Sentences for Answering Complex Questions EMNLP 2008 Improving the Performance of the Random Walk Model for Answering Complex Questions ACL 2008