SEED 7/17 (arXiv 2607.14777): self-evolving on-policy distillation lifts agentic RL across ALFWorld / Search-QA / WebShop — Qwen3-1.7B jumps 46.1 → 92.0

arXiv 2607.14777 (indexed 7/17) introduces SEED, a self-evolving on-policy distillation framework for long-horizon agentic RL that converts completed trajectories into hindsight skills and distills them back into the policy, eliminating the sparse-reward credit assignment problem. Two stages: (1) Hindsight Skill SFT fine-tunes the base policy on 1,440 trajectories (180 tasks × K0=8 rollouts) annotated by an external analyzer (GLM-5.2) to extract reusable natural-language skills; (2) Self-Evolving OPD has the current policy snapshot simultaneously act as rollout actor and trajectory analyzer, refreshed each iteration so actor and analyzer co-evolve. The same model plays both roles (actor + analyzer) sharing parameters; confidence-gated token-level distillation via sigmoid(β_opd × Δlog-prob); joint loss L_SEED = L_RL (GRPO with KL regularization) + λ_opd · L_OPD; gradients flow only through the ordinary student branch; at inference, the deployed policy acts from ordinary history alone — no skills, memory, or retrieval required. Reported benchmark numbers (ALFWorld / Search-QA / WebShop score / WebShop succ): Qwen2.5-3B-Instruct — GRPO 75.0 / 36.4 / 79.8 / 63.3 vs Seed 91.8 / 45.7 / 88.5 / 78.9; Qwen2.5-7B-Instruct — GRPO 81.2 / 42.0 / 80.9 / 72.6 vs Seed 96.1 / 48.6 / 89.7 / 78.1; Qwen3-1.7B-Instruct — GRPO 46.1 / 40.8 / 67.3 / 38.3 vs Seed 92.0 / 42.2 / 87.1 / 77.3; sample efficiency: Seed with 60% of training data (ALFWorld 80.7) exceeds full-data GRPO (75.0); cross-domain generalization: +15.3 points on ALFWorld Unseen split (86.2 vs 70.9); multimodal (Qwen2.5-VL-3B) Sokoban 82.0% / EZPoints 100.0%, averaging 91.0% vs GRPO 77.0%; ablations (ALFWorld avg) — removing Hindsight-Skill SFT → 86.0 (-5.8); removing Self-Evolving OPD → 87.0 (-4.8); replacing on-policy skills with static offline skills → 84.4 (-7.4).