On July 6, 2026, the paper "CURE: Critique-Driven Unified Reinforcement Learning for Test-Time Self-Improvement," a collaboration between the Gaoling School of Artificial Intelligence, Renmin University of China, and WeChat, Tencent Inc., received the Outstanding Paper Award at the 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026).

The paper's first three authors - Guirong Chen, Shuqi Ye, and Wenkai Yang - are doctoral students at the Gaoling School of Artificial Intelligence, Renmin University of China, and are co-first authors. The corresponding author is Associate Professor Yankai Lin of the same school.

Research on large language models is shifting from scaling training-time compute to scaling test-time compute. One promising path for test-time scaling is to enable models to iteratively verify and critique their previous outputs during inference, thereby achieving continuous improvement through multi-round self-reflection. Although Reinforcement Learning with Verifiable Rewards (RLVR) can enhance single-turn reasoning, standard methods often lack autonomous improvement mechanisms. Existing critique-guided methods, meanwhile, commonly rely on stronger teacher models, ground-truth labels, or other external feedback, making them difficult to sustain at test time.

To address this limitation, the paper proposes CURE (Critique-Driven Unified Reinforcement Learning), a framework that jointly optimizes a single policy for solving, critiquing, and critique-guided re-exploration. It enables the model to form a closed solve-verify-critique-explore loop and continuously improve its answers without relying on external teachers or test-time labels.

A critical design choice in CURE is to have the model first determine whether the current solution contains errors, then generate high-level strategic hints and reconstruct the reasoning path. During re-exploration, the model discards the complete incorrect solution and retains only the original problem and the hint, reasoning from a fresh context to mitigate anchoring bias and avoid repeating previous errors. During training, CURE also replays successful trajectories discovered through re-exploration, injecting effective learning signals for hard problems.

Figure 1. Unified CURE training framework: solving, critiquing, and critique-guided re-exploration are jointly optimized within a single policy.

The paper presents a systematic evaluation across multiple mathematical reasoning and code generation benchmarks. The results show that CURE maintains competitive single-turn reasoning performance while consistently converting additional test-time compute into accuracy gains.

On Qwen2.5-7B-Instruct, iterative self-improvement raises CURE's average accuracy across mathematics tasks from 42.0% to 44.9%; on AIME25, the cumulative gain over multiple iterations reaches 4.2 percentage points; and on CodeForces, accuracy rises steadily from 10.0% to 13.0%. These results show that CURE can effectively and consistently convert additional test-time compute into downstream performance gains, offering a practical technical path toward test-time scaling and autonomous improvement in LLMs.

The ACL 2026 Outstanding Paper Award ceremony.

The Annual Meeting of the Association for Computational Linguistics (ACL) is one of the world's most influential international academic conferences in computational linguistics and natural language processing and is ranked as a CCF-A conference by the China Computer Federation (CCF). The 64th ACL was held in San Diego, USA, from July 2 to 7, 2026.

Paper page: https://aclanthology.org/2026.acl-long.1321/

Code: https://github.com/RUCBM/CURE

ACL 2026 award information: https://2026.aclweb.org/program/best_papers/