NVIDIA researchers, with Princeton College and the College of Maryland, have launched PivotOPD, an on-policy distillation technique for multi-turn LLM brokers. PivotOPD on-policy distillation trains an agent to keep away from its most damaging early mistake, and to get better when it occurs anyway. Towards 13 baselines, it posts the most effective common on ALFWorld, WebShop and Search-based QA for Qwen3-1.7B and Qwen3-8B college students. The takeaway: restoration is learnable, and normal OPD not often teaches it.
TL;DR
- Measurement: A coaching technique, not a mannequin. Examined on Qwen3-1.7B and Qwen3-8B college students, plus a Nemotron-3.5-SFT scholar on SWE-Bench Verified.
- Runs on: Skilled on NVIDIA H100 nodes. Provides 0 inference price, so the educated agent runs wherever its base mannequin runs.
- Efficiency: First on all 8 per-benchmark averages in opposition to 13 baselines, throughout 3 seeds.
- Finest: Recovers from 72.7% of replayed pivotal errors, vs 20.3% for traditional OPD.
- Worst: 55.9% on ALFWorld “Look” duties with the 1.7B scholar, vs 83.9% for SOD.
- Backside line: Finest: teaches restoration that outcome-only RL can not attain. Worst: relies on replayable environments and a trainer whose pivots match the oracle in 77.8% of failed rollouts.
What’s a pivotal mistake in a multi-turn agent?
A pivotal mistake is an motion that lengthens the shortest remaining path to ending a job, or makes it unsolvable. ALFWorld’s symbolic oracle measures this at each flip.
Throughout Qwen3-8B, Qwen3-30B-A3B and Qwen3-235B-A22B, 59% of failed rollouts (155 of 262) contained one. The primary pivotal flip arrived early, at a median of flip 8 to 12 out of 30. The brokers then wasted 18 to 21 extra turns with out recovering.
In replays of Qwen3-8B failures, correcting the pivotal flip raised success from 8% to 59%. Leaving the error in place and forcing the proper motion for the following 2 turns nonetheless reached 58%.
Why does normal on-policy distillation miss it?
Normal OPD lowered the held-out failure charge from 79% to 56%. Failures after a pivotal flip solely fell from 51% to 49%. The proper motion stayed under 1% likelihood at each pivotal flip, so 8 rollouts not often pattern it.
Final result-based RL shares the blind spot: if each rollout fails, the group-relative benefit is 0.
How does PivotOPD work?
PivotOPD provides 3 parts to group-based RL, mixed in a single PPO replace.
- Pivot detection: A bigger trainer mannequin reads every rollout and its final result in hindsight. It picks candidate turns and names a gold motion at every. A flip counts as pivotal when the scholar’s motion differs from the gold motion. On ALFWorld, detected pivots land inside 1 flip of the oracle’s pivot in 77.8% of failed rollouts on common.
- Preventive distillation (reverse KL): A frozen copy of the scholar, hinted with the gold motion, re-scores the scholar’s personal response. This pushes the scholar away from the dedicated mistake.
- Restoration distillation (ahead KL): After every pivot, the trainer names a restoration motion for as much as Okay turns. The hinted self-teacher writes restoration responses, and the unhinted scholar trains on them. Ahead KL is mass-covering, so it lifts actions the scholar virtually by no means samples.
The trainer solely names actions. Token-level targets come from the scholar’s personal hinted distribution.
How does PivotOPD carry out on agent benchmarks?
With the 1.7B scholar, PivotOPD averages 73.7% on ALFWorld, 5.5 factors above SDAR. It averages 44.5% on Search-based QA, 5.9 factors above RLSD. On WebShop it beats RLSD by 1.2 in rating however by 14.1 in success charge (76.6%).
With the 8B scholar, it reaches 93.0% on ALFWorld, 47.4% on Search-based QA and 81.9% WebShop success. Margins are smaller, not less than 1.8 factors.
With Qwen3-8B as its personal trainer, PivotOPD nonetheless wins all 3 benchmarks by not less than 1.5 factors, 3.9 on common.
On SWE-Bench Verified, a Nemotron-3.5-SFT scholar taught by Nemotron-3-Tremendous went from 62.8% to 66.0%. Normal OPD reached 63.0%, and the trainer scores 73.0%.
Restoration is the standout. Throughout 72 replayed pivotal errors, PivotOPD recovered 72.7% of the time, vs 8.3% for the bottom mannequin, 20.3% for traditional OPD and 45.8% for preventive-only. It averaged 9.7 turns to get better, in opposition to an optimum 6.2.
<iframe id="mtp-pivotopd-frame" title="PivotOPD interactive explainer" loading="lazy" peak="600" model="peak:600px;" srcdoc="<!DOCTYPE html>
<html lang="en"><head><meta charset="utf-8"><meta identify="viewport" content material="width=device-width,initial-scale=1">
<title>PivotOPD Explainer</title>
<model>
html,physique{margin:0;padding:0;background:#111!vital;}
#mtp-pivotopd{max-width:880px;margin:0 auto;background:#111!vital;shade:#e8e8e8!vital;font-family:Inter,"Segoe UI",system-ui,sans-serif;border:1px stable #2a2a2a!vital;border-radius:14px;overflow:hidden;box-sizing:border-box;}
#mtp-pivotopd *{box-sizing:border-box;}
#mtp-pivotopd hr,#mtp-pivotopd p:empty,#mtp-pivotopd del,#mtp-pivotopd s{show:none!vital;}
#mtp-pivotopd .hd{padding:20px 24px 14px;border-bottom:1px stable #222!vital;background:linear-gradient(135deg,#151a0c,#111)!vital;}
#mtp-pivotopd .kick{font-size:11px;letter-spacing:.14em;text-transform:uppercase;shade:#76B900!vital;font-weight:700;}
#mtp-pivotopd h2{margin:6px 0 4px;font-size:22px;shade:#fff!vital;line-height:1.25;}
#mtp-pivotopd .sub{margin:0;font-size:13.5px;shade:#a8a8a8!vital;}
#mtp-pivotopd .tabs{show:flex;hole:6px;padding:12px 16px;background:#141414!vital;border-bottom:1px stable #222!vital;flex-wrap:wrap;}
#mtp-pivotopd .tab{background:#1c1c1c!vital;shade:#cfcfcf!vital;border:1px stable #2c2c2c!vital;border-radius:999px;padding:7px 14px;font-size:13px;cursor:pointer;font-family:inherit;transition:all .2s;}
#mtp-pivotopd .tab:hover{border-color:#76B900!vital;shade:#fff!vital;}
#mtp-pivotopd .tab.on{background:#76B900!vital;shade:#0b0b0b!vital;border-color:#76B900!vital;font-weight:700;}
#mtp-pivotopd .panel{show:none;padding:20px 24px 22px;animation:mtpfade .35s ease;}
#mtp-pivotopd .panel.on{show:block;}
@keyframes mtpfade{from{opacity:0;remodel:translateY(6px)}to{opacity:1;remodel:none}}
#mtp-pivotopd p{font-size:14px;line-height:1.55;shade:#d6d6d6!vital;margin:0 0 12px;}
#mtp-pivotopd .muted{shade:#8f8f8f!vital;font-size:12px;}
#mtp-pivotopd .turns{show:flex;hole:4px;margin:14px 0 8px;flex-wrap:wrap;}
#mtp-pivotopd .t{width:22px;peak:22px;border-radius:5px;background:#2a2a2a!vital;show:flex;align-items:heart;justify-content:heart;font-size:10px;shade:#777!vital;transition:background .25s,remodel .25s;}
#mtp-pivotopd .t.okay{background:#4a5a7a!vital;shade:#e6ecff!vital;}
#mtp-pivotopd .t.pv{background:#e0603a!vital;shade:#fff!vital;remodel:scale(1.15);}
#mtp-pivotopd .t.wa{background:#3a2a2a!vital;shade:#b88!vital;}
#mtp-pivotopd .t.rc{background:#76B900!vital;shade:#0b0b0b!vital;}
#mtp-pivotopd .legend{show:flex;hole:14px;flex-wrap:wrap;font-size:12px;shade:#aaa!vital;margin-bottom:10px;}
#mtp-pivotopd .legend i{show:inline-block;width:10px;peak:10px;border-radius:3px;margin-right:5px;vertical-align:-1px;}
#mtp-pivotopd .btn{background:clear!vital;shade:#76B900!vital;border:1px stable #76B900!vital;border-radius:8px;padding:7px 14px;font-size:13px;cursor:pointer;font-family:inherit;margin-right:6px;}
#mtp-pivotopd .btn:hover{background:#76B900!vital;shade:#0b0b0b!vital;}
#mtp-pivotopd .stats{show:grid;grid-template-columns:repeat(3,1fr);hole:10px;margin-top:14px;}
#mtp-pivotopd .stat{background:#181818!vital;border:1px stable #262626!vital;border-radius:10px;padding:12px;}
#mtp-pivotopd .stat b{show:block;font-size:24px;shade:#76B900!vital;}
#mtp-pivotopd .stat span{font-size:12px;shade:#aaa!vital;line-height:1.35;show:block;}
#mtp-pivotopd .steps{show:grid;grid-template-columns:repeat(3,1fr);hole:10px;margin:6px 0 14px;}
#mtp-pivotopd .step{background:#181818!vital;border:1px stable #262626!vital;border-radius:10px;padding:12px;cursor:pointer;transition:all .25s;opacity:.55;}
#mtp-pivotopd .step.on{opacity:1;border-color:#76B900!vital;box-shadow:0 0 0 1px #76B900 inset,0 0 18px rgba(118,185,0,.18);}
#mtp-pivotopd .step .n{font-size:11px;shade:#76B900!vital;font-weight:700;letter-spacing:.1em;}
#mtp-pivotopd .step h4{margin:4px 0 4px;font-size:14px;shade:#fff!vital;}
#mtp-pivotopd .step small{font-size:12px;shade:#9a9a9a!vital;}
#mtp-pivotopd .element{background:#151a0c!vital;border:1px stable #2f4200!vital;border-radius:10px;padding:14px;min-height:96px;}
#mtp-pivotopd .element h5{margin:0 0 6px;font-size:14px;shade:#9fe02a!vital;}
#mtp-pivotopd .circulation{show:flex;align-items:heart;hole:6px;margin:10px 0 2px;flex-wrap:wrap;font-size:12px;}
#mtp-pivotopd .chip{padding:4px 9px;border-radius:6px;background:#222!vital;shade:#ddd!vital;border:1px stable #333!vital;}
#mtp-pivotopd .chip.g{background:#76B900!vital;shade:#0b0b0b!vital;border-color:#76B900!vital;font-weight:700;}
#mtp-pivotopd .chip.r{background:#e0603a!vital;shade:#fff!vital;border-color:#e0603a!vital;}
#mtp-pivotopd .arr{shade:#666!vital;}
#mtp-pivotopd .tog{show:inline-flex;border:1px stable #333!vital;border-radius:8px;overflow:hidden;margin-bottom:12px;}
#mtp-pivotopd .tog button{background:#1a1a1a!vital;shade:#bbb!vital;border:0!vital;padding:6px 12px;font-size:12px;cursor:pointer;font-family:inherit;}
#mtp-pivotopd .tog button.on{background:#76B900!vital;shade:#0b0b0b!vital;font-weight:700;}
#mtp-pivotopd .bench{margin-bottom:14px;}
#mtp-pivotopd .bench h6{margin:0 0 6px;font-size:12.5px;shade:#cfcfcf!vital;font-weight:600;}
#mtp-pivotopd .row{show:flex;align-items:heart;hole:8px;margin:4px 0;font-size:12px;}
#mtp-pivotopd .row .lb{width:92px;shade:#aaa!vital;flex:none;}
#mtp-pivotopd .row .tr{show:block;flex:1;peak:16px;background:#1d1d1d!vital;border-radius:4px;overflow:hidden;}
#mtp-pivotopd .row .bar{show:block;peak:100%;width:0;background:#4a5a7a!vital;border-radius:4px;transition:width 1s cubic-bezier(.2,.8,.2,1);}
#mtp-pivotopd .row.me .bar{background:#76B900!vital;}
#mtp-pivotopd .row.me .lb{shade:#76B900!vital;font-weight:700;}
#mtp-pivotopd .row .v{width:40px;text-align:proper;shade:#ddd!vital;flex:none;}
#mtp-pivotopd .case{show:grid;grid-template-columns:1fr 1fr;hole:10px;margin-top:12px;}
#mtp-pivotopd .col{background:#181818!vital;border:1px stable #262626!vital;border-radius:10px;padding:10px;}
#mtp-pivotopd .col h6{margin:0 0 6px;font-size:12px;shade:#fff!vital;}
#mtp-pivotopd .ln{font-size:12px;padding:5px 7px;border-radius:6px;margin:3px 0;shade:#ccc!vital;background:#202020!vital;opacity:0;remodel:translateX(-6px);transition:all .3s;}
#mtp-pivotopd .ln.present{opacity:1;remodel:none;}
#mtp-pivotopd .ln.dangerous{background:#3a1d16!vital;shade:#f3b5a2!vital;}
#mtp-pivotopd .ln.good{background:#1f2c08!vital;shade:#c6f07a!vital;}
#mtp-pivotopd .ft{padding:10px 24px;border-top:1px stable #222!vital;font-size:12px;shade:#777!vital;show:flex;justify-content:space-between;flex-wrap:wrap;hole:6px;background:#0f0f0f!vital;}
#mtp-pivotopd .ft b{shade:#76B900!vital;}
#mtp-pivotopd a{shade:#76B900!vital;text-decoration:none;}
@media (max-width:640px){
#mtp-pivotopd .hd,#mtp-pivotopd .panel{padding-left:16px;padding-right:16px;}
#mtp-pivotopd h2{font-size:19px;}
#mtp-pivotopd .stats,#mtp-pivotopd .steps,#mtp-pivotopd .case{grid-template-columns:1fr;}
#mtp-pivotopd .t{width:18px;peak:18px;font-size:9px;}
#mtp-pivotopd .row .lb{width:74px;}
}
</model></head><physique>
<div id="mtp-pivotopd">
<div class="hd">
<div class="kick">NVIDIA Analysis · Interactive explainer</div>
<h2>PivotOPD: instructing brokers to get better from pivotal errors</h2>
<p class="sub">On-policy distillation that forestalls early agent errors and trains restoration from them.</p>
</div>
<div class="tabs" position="tablist">
<button class="tab on" data-p="0">1. The issue</button>
<button class="tab" data-p="1">2. The way it works</button>
<button class="tab" data-p="2">3. Benchmarks</button>
<button class="tab" data-p="3">4. Restoration</button>
</div>
<div class="panel on" data-p="0">
<p>In ALFWorld, 59% of failed rollouts throughout 3 Qwen3 fashions include a <b model="shade:#e0603a">pivotal mistake</b>: an motion that lengthens the shortest path to ending the duty. It lands early, then the agent wastes the remainder of the 30-turn episode.</p>
<div class="legend"><span><i model="background:#4a5a7a"></i>Appropriate flip</span><span><i model="background:#e0603a"></i>Pivotal mistake</span><span><i model="background:#3a2a2a"></i>Wasted flip</span><span><i model="background:#76B900"></i>Restoration</span></div>
<div class="turns" id="mtp-turns"></div>
<div model="margin:10px 0 2px"><button class="btn" id="mtp-play1">Play failed rollout</button><button class="btn" id="mtp-play2">Play with restoration</button></div>
<p class="muted" id="mtp-cap">Illustrative 30-turn episode. Pivot proven at flip 10, contained in the reported median vary of 8 to 12.</p>
<div class="stats">
<div class="stat"><b data-c="59">0%</b><span>replayed success after fixing the pivotal flip (from 8%)</span></div>
<div class="stat"><b data-c="58">0%</b><span>success when solely the two turns after the error are guided</span></div>
<div class="stat"><b>51% → 49%</b><span>pivotal-turn failures after normal OPD barely transfer</span></div>
</div>
</div>
<div class="panel" data-p="1">
<p>PivotOPD provides 3 components to group-based RL, all utilized in a single PPO replace. Click on a step or use the buttons.</p>
<div class="steps">
<div class="step on" data-s="0"><div class="n">STEP 01</div><h4>Pivot detection</h4><small>Trainer finds the flip and the gold motion</small></div>
<div class="step" data-s="1"><div class="n">STEP 02</div><h4>Preventive distillation</h4><small>Reverse KL, on the scholar's personal response</small></div>
<div class="step" data-s="2"><div class="n">STEP 03</div><h4>Restoration distillation</h4><small>Ahead KL, on self-teacher responses</small></div>
</div>
<div class="element" id="mtp-detail"></div>
<div model="margin-top:12px"><button class="btn" id="mtp-prev">
Prev</button><button class="btn" id="mtp-next">Subsequent 
</button></div>
</div>
<div class="panel" data-p="2">
<div class="tog" id="mtp-tog"><button class="on" data-m="s">Qwen3-1.7B</button><button data-m="l">Qwen3-8B</button></div>
<div id="mtp-bench"></div>
<p class="muted">Averages over 3 seeds from Desk 1 of the paper. In contrast with the strongest baselines. PivotOPD ranks first on all 8 per-benchmark averages in opposition to 13 baselines.</p>
<div class="bench"><h6>SWE-Bench Verified resolve charge (Nemotron-3.5-SFT scholar)</h6><div id="mtp-swe"></div></div>
</div>
<div class="panel" data-p="3">
<p>72 oracle-labeled pivotal errors, replayed 8 instances per coverage. How usually does every coverage end the duty anyway?</p>
<div id="mtp-rec"></div>
<p class="muted">Common turns to get better: PivotOPD 9.7, normal OPD 12.3, base 13.4, optimum 6.2.</p>
<button class="btn" id="mtp-case">Replay case research</button>
<div class="case">
<div class="col"><h6>Base mannequin</h6><div id="mtp-ca"></div></div>
<div class="col"><h6 model="shade:#76B900">PivotOPD mannequin</h6><div id="mtp-cb"></div></div>
</div>
</div>
<div class="ft"><span>Supply: <a href="https://arxiv.org/abs/2609.40285" goal="_blank" rel="noopener">arXiv 2609.40285</a></span><b>© Marktechpost</b></div>
</div>
<script>
(perform(){
var R=doc.getElementById(‘mtp-pivotopd’);
perform $(s){return R.querySelector(s);} perform $$(s){return R.querySelectorAll(s);}
perform resize(){attempt{guardian.postMessage({mtpPivotOPDHeight:R.offsetHeight+40},’*’);}catch(e){}}
var proven={};
$$(‘.tab’).forEach(perform(b){b.onclick=perform(){
$$(‘.tab’).forEach(perform(x){x.classList.take away(‘on’)});b.classList.add(‘on’);
var p=b.getAttribute(‘data-p’);
$$(‘.panel’).forEach(perform(x){x.classList.toggle(‘on’,x.getAttribute(‘data-p’)===p)});
if(p===’2′)bench(mode); if(p===’3′)rec(); setTimeout(resize,50);};});
/* panel 1 */
var T=$(‘#mtp-turns’),cells=[];
for(var i=1;i<=30;i++){var d=doc.createElement(‘div’);d.className=”t”;d.textContent=i;T.appendChild(d);cells.push(d);}
var timer;
perform play(get better){clearInterval(timer);cells.forEach(perform(c){c.className=”t”});var ok=0;
timer=setInterval(perform(){var c=cells[k];var n=ok+1;
if(n<10)c.className=”t okay”;else if(n===10)c.className=”t pv”;
else if(get better){c.className=n<=12?’t rc’:(n<=16?’t okay’:’t’);}
else c.className=”t wa”;
ok++; if(get better&&n===16){clearInterval(timer);$(‘#mtp-cap’).textContent=”Guided restoration after the error, then job accomplished. Replays present 58% success with 2 guided turns.”;}
if(ok>=30){clearInterval(timer);$(‘#mtp-cap’).textContent=”No restoration: the remaining 20 turns are wasted. Actual brokers wasted 18 to 21 activates common.”;}
},90);}
$(‘#mtp-play1’).onclick=perform(){play(false)};$(‘#mtp-play2’).onclick=perform(){play(true)};
perform rely(){ $$(‘.stat b[data-c]’).forEach(perform(b){var t=+b.getAttribute(‘data-c’),v=0;var iv=setInterval(perform(){v+=2;if(v>=t){v=t;clearInterval(iv);}b.textContent=v+’%’;},20);});}
/* panel 2 */
var S=[
{h:’1. Pivot detection’,b:’A larger teacher reads each rollout and its outcome in hindsight, picks candidate turns, and names a gold action. A turn is pivotal when the student’s action differs. Detected pivots land within 1 turn of the oracle’s in 77.8% of failed ALFWorld rollouts on average.’,f:[‘<span class="chip">Rollout + outcome</span>’,'<span class="arr">→</span>’,'<span class="chip">Teacher model</span>’,'<span class="arr">→</span>’,'<span class="chip r">Pivotal turn t</span>’,'<span class="chip g">Gold action</span>’]},
{h:’2. Preventive distillation (reverse KL)’,b:’The frozen scholar, hinted with the gold motion, turns into a privileged self-teacher. It re-scores the response the scholar really wrote, which shifts likelihood away from the error. Weight w_prev is saved small (0.001).’,f:[‘<span class="chip">Student response at t</span>’,'<span class="arr">→</span>’,'<span class="chip">Self-teacher + hint(gold)</span>’,'<span class="arr">→</span>’,'<span class="chip g">Reverse KL update</span>’]},
{h:’3. Restoration distillation (ahead KL)’,b:’After the pivot, the trainer names a restoration motion for every of the following Okay turns. The hinted self-teacher writes the restoration response and the unhinted scholar trains on it. Ahead KL is mass-covering, so it lifts actions the scholar virtually by no means samples. Finest Okay: 2 on ALFWorld, 1 on WebShop and Search QA.’,f:[‘<span class="chip r">Post-mistake state</span>’,'<span class="arr">→</span>’,'<span class="chip">Teacher: recovery action</span>’,'<span class="arr">→</span>’,'<span class="chip">Self-teacher writes response</span>’,'<span class="arr">→</span>’,'<span class="chip g">Forward KL update</span>’]}
];
var si=0;
perform step(i){si=(i+3)%3;$$(‘.step’).forEach(perform(x){x.classList.toggle(‘on’,+x.getAttribute(‘data-s’)===si)});
var s=S[si];$(‘#mtp-detail’).innerHTML='<h5>’+s.h+'</h5><p model="margin:0">’+s.b+'</p><div class="circulation">’+s.f.be part of(”)+'</div>’;setTimeout(resize,30);}
$$(‘.step’).forEach(perform(x){x.onclick=perform(){step(+x.getAttribute(‘data-s’))}});
$(‘#mtp-prev’).onclick=perform(){step(si-1)};$(‘#mtp-next’).onclick=perform(){step(si+1)};step(0);
/* panel 3 */
var B={s:[
[‘ALFWorld avg success (%)’,[[‘PivotOPD’,73.7],[‘SDAR’,68.2],[‘OPID’,61.3],[‘RLSD’,60.7],[‘GRPO’,42.7]]],
[‘Search-based QA avg accuracy (%)’,[[‘PivotOPD’,44.5],[‘RLSD’,38.6],[‘SOD’,38.2],[‘GRPO’,37.7]]],
[‘WebShop success rate (%)’,[[‘PivotOPD’,76.6],[‘OPID’,68.0],[‘RLSD’,62.5],[‘GRPO’,57.0]]]],
l:[
[‘ALFWorld avg success (%)’,[[‘PivotOPD’,93.0],[‘RLSD’,90.8],[‘Skill-SD’,88.6],[‘GRPO’,35.3]]],
[‘Search-based QA avg accuracy (%)’,[[‘PivotOPD’,47.4],[‘OPID’,45.6],[‘OPSD’,43.3],[‘GRPO’,37.5]]],
[‘WebShop success rate (%)’,[[‘PivotOPD’,81.9],[‘SDAR’,79.7],[‘RLSD’,78.9],[‘GRPO’,75.0]]]]};
perform bars(el,rows,max){el.innerHTML=rows.map(perform(r){return ‘<div class="row’+(r[0]===’PivotOPD’?’ me’:”)+’"><span class="lb">’+r[0]+'</span><span class="tr"><span class="bar" data-w="’+(r[1]/max*100)+’"></span></span><span class="v">’+r[1].toFixed(1)+'</span></div>’}).be part of(”);
requestAnimationFrame(perform(){setTimeout(perform(){el.querySelectorAll(‘.bar’).forEach(perform(b){b.model.width=b.getAttribute(‘data-w’)+’%’})},40)});}
var mode=”s”;
perform bench(m){mode=m;var W=$(‘#mtp-bench’);W.innerHTML=”;B[m].forEach(perform(g){var d=doc.createElement(‘div’);d.className=”bench”;d.innerHTML='<h6>’+g[0]+'</h6><div></div>’;W.appendChild(d);bars(d.lastChild,g[1],100);});
bars($(‘#mtp-swe’),[[‘Teacher’,73.0],[‘PivotOPD’,66.0],[‘Std OPD’,63.0],[‘Student’,62.8]],100);setTimeout(resize,60);}
$$(‘#mtp-tog button’).forEach(perform(b){b.onclick=perform(){$$(‘#mtp-tog button’).forEach(perform(x){x.classList.take away(‘on’)});b.classList.add(‘on’);bench(b.getAttribute(‘data-m’));}});
/* panel 4 */
perform rec(){bars($(‘#mtp-rec’),[[‘PivotOPD’,72.7],[‘Prev. only’,45.8],[‘Std OPD’,20.3],[‘Base’,8.3]],100);setTimeout(resize,60);}
var CA=[[‘Turns 1-2: no egg in the fridge’,”],[‘Turn 3: takes tomato from fridge’,’bad’],[‘Turn 4: goes to microwave’,”],[‘Turn 5: moves tomato to microwave’,’bad’],[‘Turns 6-30: never finds the egg’,’bad’],[‘✕ Failure’,’bad’]];
var CB=[[‘Turns 1-2: no egg in the fridge’,”],[‘Turn 3: takes tomato from fridge’,’bad’],[‘Turn 4: recovery begins, goes to table’,’good’],[‘Turn 5: sets the tomato aside’,’good’],[‘Turn 6: takes egg from table’,’good’],[‘Turns 7-11: cools egg, moves it to microwave’,’good’],[‘✓ Success’,’good’]];
perform fill(el,arr){el.innerHTML=arr.map(perform(a){return ‘<div class="ln ‘+a[1]+’">’+a[0]+'</div>’}).be part of(”);}
perform caseplay(){fill($(‘#mtp-ca’),CA);fill($(‘#mtp-cb’),CB);var a=$$(‘#mtp-ca .ln’),b=$$(‘#mtp-cb .ln’);var ok=0;var iv=setInterval(perform(){if(a[k])a[k].classList.add(‘present’);if(b[k])b[k].classList.add(‘present’);ok++;if(ok>7)clearInterval(iv);},380);setTimeout(resize,60);}
$(‘#mtp-case’).onclick=caseplay;
fill($(‘#mtp-ca’),CA);fill($(‘#mtp-cb’),CB);$$(‘#mtp-pivotopd .ln’).forEach(perform(x){x.classList.add(‘present’)});
rely();
window.addEventListener(‘load’,resize);window.addEventListener(‘resize’,resize);setTimeout(resize,300);
})();
</script>
</physique></html>
“>
window.addEventListener(“message”,perform(e){if(e.information&&e.information.mtpPivotOPDHeight){var f=doc.getElementById(“mtp-pivotopd-frame”);if(f&&e.supply===f.contentWindow){f.model.peak=e.information.mtpPivotOPDHeight+”px”;}}});
How does PivotOPD evaluate with different agent distillation strategies?
| Metrics | PivotOPD | OPSD (normal OPD) | OPID | SDAR | RLSD |
|---|---|---|---|---|---|
| Core concept | Stop and get better at teacher-detected pivotal turns | Privileged self-teacher on each response | Hierarchical abilities from on-policy hindsight | OPSD as sigmoid-gated auxiliary loss | Self-distillation units replace measurement, RLVR units route |
| Trains on teacher-written restoration responses | Sure (ahead KL) | No | No | No | No |
| ALFWorld avg, Qwen3-1.7B | 73.7 | 12.4 | 61.3 | 68.2 | 60.7 |
| Search QA avg, Qwen3-1.7B | 44.5 | 36.8 | 37.5 | 37.1 | 38.6 |
| WebShop success, Qwen3-1.7B | 76.6 | 10.1 | 68.0 | 57.0 | 62.5 |
| ALFWorld avg, Qwen3-8B | 93.0 | 46.7 | 83.7 | 83.8 | 90.8 |
| Code | Coming quickly | GitHub | GitHub | Not disclosed | Not disclosed |
Rating supply: PivotOPD Desk 1.
What does it price to coach, and may you run it?
PivotOPD modifications solely coaching, so inference prices nothing further. On ALFWorld with the 1.7B scholar, on 4 H100 GPUs, the overhead over GRPO is 12.4% at Okay = 1. It jumps to 94.2% at Okay = 2, as a result of later restoration turns want atmosphere replay. The chosen price range is Okay = 2 on ALFWorld and Okay = 1 on WebShop and Search-based QA.
Key Takeaways
- Over half of failed agent rollouts hinge on 1 early, recoverable mistake.
- Normal OPD barely touches these failures: 51% to 49%.
- PivotOPD recovers from 72.7% of replayed errors, vs 20.3% for OPD.
- Finest common in opposition to 13 baselines on ALFWorld, WebShop and Search QA.
- 0 inference overhead, however code will not be but launched.
Take a look at the paper and the challenge web page. All credit score goes to the researcher of this challenge. Additionally, be happy to comply with us on Twitter and don’t overlook to hitch our 150k+ML SubReddit and Subscribe to our Publication. Wait! are you on telegram? now you possibly can be part of us on telegram as effectively.
The put up NVIDIA PivotOPD Teaches Multi-Flip AI Brokers to Get better From Pivotal Errors appeared first on MarkTechPost.
![11 social media tendencies each marketer ought to watch in 2026 [new data]](https://blog.aimactgrow.com/wp-content/uploads/2026/09/social20media20trends-1-120x86.png)
