Skip to content
conftrace_
2026 EMNLP EMNLP 2026

PEARL: Perturbation Efficient Alignment Group Relative Reinforcement Learning

AbstractReinforcement learning with verifiable rewards (RLVR) has become a widely used approach for improving the mathematical reasoning ability of large language models (LLMs). However, recent studies show that LLMs may solve a math problem in its original form but fail when the same problem is reworded without changing its meaning. This suggests that the model has not fully acquired the underlying reasoning ability and instead relies on the surface form of the problem. In this paper, we propose PEARL, an RLVR framework that uses different versions of the same math problem as structured training signals. PEARL first groups each original problem with its checked variants to build semantic perturbation clusters, and then uses partially solved clusters as the active RL training set. PEARL further performs cluster-level RL optimization by comparing model outputs across different versions of the same mathematical problem. Experiments on mathematical reasoning benchmarks show that PEARL achieves strong results against competitive RLVR baselines while using a small active training set. These results suggest that semantically equivalent variants can serve as useful training units for improving mathematical reasoning in LLMs.

More from EMNLP 2026

All EMNLP 2026 papers →