Zone of Proximal Policy Optimization: Teacher in Prompts, Not Gradients

375 · Research Nvidia · Aug. 11, 2026, 7:19 a.m.
Summary
The blog discusses a novel approach called Zone of Proximal Policy Optimization (ZPPO) in reinforcement learning, which uses teacher-student dynamics to improve learning outcomes. Unlike traditional methods that rely heavily on logit imitation, ZPPO incorporates a teacher's responses into prompts instead of gradients, leading to better performance on difficult questions. The method shows significant improvements in student models across various benchmarks.