ModusPonens at SemEval-2026 Task 11: Breaking the Plausibility Trap in LLMs via Conflict-Aware Ensembling

AbstractLarge Language Models (LLMs) often struggle to disentangle formal logical validity from real-world plausibility, a phenomenon known as the “belief bias”. This paper describes our submission to SemEval-2026 Task 11. We frame the task as a calibration problem between “System 1” (heuristic) and “System 2” (logical) thinking. Our experiments reveal that standard neuro-symbolic interventions, such as Structural Chain-of-Thought (CoT) and Nonsense Augmentation, degrade performance in low-resource regimes due to an “abstraction penalty”. Instead, we propose a Conflict-Aware Logit Ensemble. We fine-tune two variations of Qwen-2.5-14B: a standard “Believer” model and a bias-hardened “Skeptic” model trained on oversampled conflict data. By ensembling their logits via soft-voting, we achieve a Pareto-optimal balance, reducing the Total Content Effect (TCE) to 3.21 while maintaining an overall accuracy of 94.27%, resulting in a Combined Score of 39.09.