Beyond Validation Loss: Coverage and Reliability Dynamics in Robot Policy Pretraining
把机器人训练想成反复练习一套操作题：继续练习，可能让熟悉的题做得更稳，却让难题原本偶尔能成功的解法消失。本文用“同一题多试几次能否成功”检查这种损失，再用离线动作误差分布选择模型，并通过生成训练样本减轻损失。
Think of robot training as repeatedly practicing a set of manipulation problems: more practice can make familiar problems more dependable while erasing the occasional successful solution to harder ones. The paper measures this loss through repeated attempts from the same start, selects models using offline action-error distributions, and mitigates the loss with generated training examples.
研究动机：平均成功率漏掉了什么？
Motivation: what does average success miss?
机器人策略通常先模仿人类演示，再接受 RL、SFT 或价值函数引导。若只按验证损失、平均成功率或最后一次保存选择模型，就可能丢掉后续训练需要的成功行为。论文研究的是预训练过程如何改变模型可改进的空间，而非仅比较最终部署分数。
Robot policies usually imitate human demonstrations before receiving RL, SFT, or value-guided refinement. Selecting by validation loss, average success, or the final saved model can discard successful behaviors that later training needs. The paper asks how pretraining changes a model’s capacity for subsequent improvement, beyond its immediate deployment score.
关键区分是：在一部分初始状态上反复成功，与在更多初始状态上至少保留成功机会，不是同一件事。若后续数据来自策略自身，后一种模型可能产生更广的有效经验；固定目标数据上的 SFT 实验则说明，差异也不只来自数据采集。§§1–3
Repeatedly succeeding from some initial states differs from retaining a chance of success across more initial states. When later data comes from the policy itself, the latter model can provide broader useful experience; SFT with fixed target data shows that the difference is not confined to data collection. §§1–3
术语速查：在本文中具体指什么？
Terms in context: what do they mean here?
| 术语 Term | 本文中的具体含义 Concrete meaning in this paper |
|---|---|
| 策略 / policy Policy | 输入相机图像、机器人自身状态等信息，随机生成一段连续控制动作的模型；主实验采用 Diffusion Policy 或 π0.5。 A model that takes camera images, proprioception, and other conditioning and stochastically generates a chunk of continuous controls; main experiments use Diffusion Policy or π0.5. |
| 初始条件 c Initial condition c | 例如物体初始摆放、机器人初始状态与场景配置；同一条件下重置后重复执行。 For example, object placement, initial robot state, and scene configuration; repeated trials reset to this same condition. |
| 轨迹 / rollout Rollout | 从重置状态到成功或超时的一整次执行，包含反复观测与动作生成；不是单次动作采样。 An entire execution from reset to success or timeout, including repeated observation and action generation; not one action sample. |
| 可靠性 / reliability Reliability | 一次完整执行成功的平均概率，即 pass@1；不是安全认证或最坏情况保证。 Mean probability that one complete rollout succeeds, or pass@1; not a safety certification or worst-case guarantee. |
| 覆盖性 / coverage Coverage | 用较大 k 的 pass@k 衡量：多少初始条件在有限次独立尝试内仍可能成功；不是随意动作多样性。 Measured by large-k pass@k: how broadly initial conditions remain solvable within a finite number of independent attempts; not arbitrary action diversity. |
| 离线分位数 Offline quantiles | 对每个保留观测采样多个动作块，分别与对应专家动作比较，再跨观测平均各个误差分位数。 Sample multiple action chunks at each held-out observation, compare them with its demonstrated action, then average each error quantile across observations. |
核心方法：诊断、选模型、改变训练
Core methods: diagnose, select, and change training
1. 固定起点的 pass@k，把成功率拆开看
1. Fixed-start pass@k separates two kinds of success
设第 τ 个模型在初始条件 c 上一次成功的概率为 qτ(c)。每次重置到同一条件，再独立执行 k 次，至少成功一次的概率为：
Let qτ(c) be the probability that checkpoint τ succeeds once from initial condition c. Reset to that condition and execute k independent rollouts; the probability of at least one success is:
\[P_k(\tau)=\mathbb E_{c\sim p_{\mathrm{eval}}}\left[1-(1-q_\tau(c))^k\right].\]
必须先对每个初始条件计算，再取平均。不能把全局平均成功率代入 1−(1−p)k；那会掩盖困难条件。下面是教学示例，不是论文实验：两种策略平均成功率都是 50%，但能力分布不同。
Compute this within each initial condition before averaging. Substituting the global average success into 1−(1−p)k hides hard conditions. The following is a teaching example, not an experimental result: both policies average 50% success but distribute their ability differently.
| 示例策略 Illustrative policy | 两个等权条件的单次成功率 Success probabilities for two equally weighted conditions | pass@1 pass@1 | pass@4 pass@4 |
|---|---|---|---|
| 两种条件都偶尔成功 Sometimes succeeds in either condition | 0.5, 0.5 | 50% | 93.75% |
| 一种必成，另一种必败 Always succeeds in one; never in the other | 1.0, 0.0 | 50% | 50% |
对 k>1，1−(1−q)k 是凹函数。因此，在平均成功率不变时，把成功概率从困难条件移向容易条件，会降低 pass@k。较大的 k 对低成功率条件更敏感。有限 k 仍是预算相关的指标；在独立试验假设下，k 趋于无穷时才趋于 q(c)>0 的条件占比。Eq. 2.2; App. B
For k>1, 1−(1−q)k is concave. Moving success probability from harder to easier conditions at a fixed mean therefore lowers pass@k. Larger k is more sensitive to low-success conditions. Finite k remains budget dependent; under independent trials, only the infinite-k limit equals the fraction of conditions with q(c)>0. Eq. 2.2; App. B
2. 用动作误差的整个分布选择检查点
2. Select checkpoints using the full action-error distribution
只看 best-of-k 动作 MSE 会选得过早：一个模型可能偶尔生成非常接近专家的动作，但其余采样很差，整条执行仍会失败。平均 MSE 又可能掩盖这些低误差动作的消失。作者同时衡量误差较低的一半与误差较高的一半。
Best-of-k action MSE selects too early: a model may occasionally sample an action very close to the expert while most samples are poor, making complete execution fail. Mean MSE can instead hide the disappearance of those low-error actions. The authors assess both the lower half and upper half of the error-quantile profile.
- 对每个保留观测采样 M 个动作块，计算相对对应专家动作的 MSE；先在每个观测内计算分位数 Qᵢ,τ(u)，再跨观测平均，得到 Q̄τ(u)。 Draw M action chunks per held-out observation and compute MSE against its demonstrated action. Compute quantiles Qᵢ,τ(u) within each observation, then average across observations to obtain Q̄τ(u).
- 对每个分位数，在截至 H 的检查点之间，将 log Q̄ 归一化到 0–1；0 表示此分位数上已观察到的最好模型，1 表示最差。 For each quantile, normalize log Q̄ to 0–1 across checkpoints available through H: 0 is the best observed checkpoint at that quantile, and 1 is the worst.
- 分别平均 u<0.5 与 u≥0.5 的归一化误差，得到覆盖性风险 r_cov 与可靠性风险 r_rel；选择二者中较大值最小的模型。 Average normalized errors separately for u<0.5 and u≥0.5 to obtain coverage risk r_cov and reliability risk r_rel; select the checkpoint minimizing the worse of the two.
\[r_{\tau\mid H}(u)=\frac{\log\bar Q_\tau(u)-\min_{s\le H}\log\bar Q_s(u)}{\max_{s\le H}\log\bar Q_s(u)-\min_{s\le H}\log\bar Q_s(u)},\qquad \tau_H^*=\arg\min_{\tau\le H}\max\{r_{\rm cov\mid H}(\tau),r_{\rm rel\mid H}(\tau)\}.\]
该准则不需要机器人闭环执行、不需要指定部署 k，也不需要调两个风险之间的权重。但它仍需要保留数据、多次动作采样和一组已保存的模型；“离线”并不等于零计算。Threading D2 的 200 条演示实验选择 epoch 350；RoboCasa 选择 20k steps，而最终模型为 100k steps。§4; App. E
The criterion needs no closed-loop robot execution, deployment-specific k, or tuned weight between the risks. It still requires held-out data, repeated action sampling, and saved checkpoints; offline does not mean computation free. It selects epoch 350 for Threading D2 with 200 demonstrations, and 20k steps for RoboCasa versus the final 100k-step model. §4; App. E
3. 为什么会发生：有限演示上的后期过拟合
3. Why it happens: late fitting to finite demonstrations
作者让两个模型各自学习互不重叠的 200 条演示，再在共同的 600 条保留演示上比较。两个模型的预测先趋同、后分离，生成动作也逐渐更接近各自训练集。后期模型的训练去噪损失继续降低，但保留数据在低噪声阶段的损失反而上升，且观测支持稀疏处更明显。
Two models learn disjoint sets of 200 demonstrations and are compared on a shared held-out set of 600. Their predictions first converge and then diverge, while generated actions become increasingly biased toward their respective training sets. Later checkpoints keep reducing training denoising loss, yet held-out loss rises at low noise, especially where observation support is sparse.
更强的干预证据来自替换采样步骤：把 epoch 800 模型的 10 次 DDIM 调用中最后两次换成 epoch 200 权重，恢复了该模型对的大部分 pass@10 差距，同时降低 pass@1。冻结视觉编码器、重新训练去噪器以及使用低维状态的实验也保留该趋势。因此，视觉编码器过拟合不能独自解释结果，但这仍不是普适因果定理。App. D, G
A more direct intervention replaces only the final two of ten DDIM calls in the epoch-800 model with epoch-200 weights. This recovers most of the pass@10 gap for that model pair while reducing pass@1. The pattern also persists with a frozen visual encoder, a newly trained denoiser, and low-dimensional states. Visual-encoder overfitting alone therefore cannot explain the result, though this is not a universal causal theorem. App. D, G
4. JointAug：生成相互匹配的观测与动作
4. JointAug: generate observations and actions together
JointAug 在相同演示上训练联合生成器 G(o,a)，同时生成图像、机器人自身状态和动作块。策略训练时，50% 样本来自真实演示，50% 来自生成器；策略架构和普通行为克隆目标保持相同。联合建模旨在保留观测与动作的对应关系，而不是只给固定图像的动作标签添加任意噪声。
JointAug trains a joint generator G(o,a) on the same demonstrations to generate images, proprioceptive state, and action chunks together. Policy training mixes 50% real demonstrations with 50% generated samples, retaining the policy architecture and ordinary behavior-cloning objective. Joint modeling aims to preserve observation–action correspondence instead of simply adding arbitrary noise to action labels at fixed images.
生成器也会过拟合。作者比较生成动作到训练集与保留集的最近邻距离，在比值首次小于 1 的检查点开始，默认混合它及之后每隔 100 epochs 的四个检查点。这个五检查点窗口是启发式；消融发现邻近单个检查点也保留大部分收益。联合学习鼓励一致性，但并不保证每个合成样本在物理上都正确。
The generator can also overfit. The authors compare nearest-neighbor distances from generated actions to training and held-out actions. Starting at the first checkpoint where the ratio falls below 1, the default mixes that checkpoint with four more spaced 100 epochs apart. This five-checkpoint window is a heuristic; ablations show nearby single checkpoints preserve most gains. Joint learning encourages consistency but does not guarantee every synthetic sample is physically correct.
\[\mathcal L_{\rm aug}=\tfrac12\mathbb E_{(o,a)\sim D}[\mathcal L_{\rm BC}(o,a)]+\tfrac12\mathbb E_{(\tilde o,\tilde a)\sim G_{\rm mix}}[\mathcal L_{\rm BC}(\tilde o,\tilde a)].\]
RoboCasa 的实现不同：由于联合生成成本较高，作者在 π0.5 上采用 action-only 变体。先选 c*，再用该策略对每个训练观测的多次动作采样估计协方差 Σc*(o)，从 c* 恢复训练，并加入均值为零、该协方差的高斯动作扰动。这里没有生成新图像；这些结果不能直接证明大规模联合图像–动作生成已经有效。§5; App. H.3
The RoboCasa implementation differs: because joint generation is expensive, π0.5 uses an action-only variant. First select c*, estimate covariance Σc*(o) from its repeated action samples at each training observation, then resume from c* with zero-mean Gaussian action perturbations of that covariance. No new images are generated here; these results do not directly establish large-scale joint image–action generation. §5; App. H.3
主要实验结果：哪些证据最有说服力？
Experimental results: what does the evidence establish?
评测范围与分母
Evaluation scope and denominators
| 设置 Setting | 训练与评测 Training and evaluation | 能够检验什么 What it tests |
|---|---|---|
| MimicGen | 主实验为 5 个任务，各 200 条演示；每任务 500 个保留初始配置。附录增加 500 条演示设置。 Five main tasks with 200 demonstrations each; 500 held-out initial configurations per task. Appendix adds 500-demonstration settings. | Diffusion Policy 的 BC 动态与后续 residual RL。 Diffusion Policy BC dynamics and subsequent residual RL. |
| RoboCasa365 / π0.5 | 从 pi05_base 开始，在 65 个任务上各用 100 条演示训练；18 个 SFT 目标任务各用 100 条演示；每评测任务 50 个保留配置。 Start from pi05_base and train on 65 tasks with 100 demonstrations each; 18 SFT target tasks use 100 demonstrations each; 50 held-out configurations per evaluated task. | 多任务 VLA 检查点选择及固定目标数据的 SFT。 Multitask VLA checkpoint selection and SFT with fixed target data. |
| 真实机器人 Push-T Real-robot Push-T | 740 条人类演示，其中 680 条训练、60 条验证；20 个固定初始状态，各试 5 次。 740 human demonstrations: 680 training and 60 validation; 20 fixed initial states, five attempts each. | 离线选择是否迁移到真实硬件；每模型 100 次执行。 Whether offline selection transfers to hardware; 100 rollouts per checkpoint. |
“Pretraining” 在这里同时指从头训练单任务策略，以及在已有 pi05_base 上进行 RoboCasa 多任务训练；不应把所有结果解释成从零开始训练通用基础模型。MimicGen 的 residual RL 使用低维状态作为 actor/critic 输入，而基础策略仍使用图像；RL 曲线报告最近 100 个完成 episode 的训练成功率，不能直接当成独立保留测试分数。App. H
“Pretraining” includes both training single-task policies from scratch and RoboCasa multitask training starting from pi05_base; these are not all foundation-model runs from scratch. MimicGen residual RL gives low-dimensional state to the actor/critic while the base policy remains image based. RL curves report training success over the last 100 completed episodes, not an independent held-out test score. App. H
精确报告的对比
Exactly reported comparisons
下表 Selected 指离线准则选出的较早模型，Final 指同一基线训练运行的最终模型。Filtered BC 的数值是用成功 rollout 重新训练后的新策略分数，不是原始检查点分数；两个来源使用相同 rollout 预算。Tables 1–2
In the table, Selected is the earlier checkpoint chosen offline; Final is the last checkpoint of the same baseline run. Filtered BC numbers belong to new policies retrained using successful rollouts, not the source checkpoints; both sources receive the same rollout budget. Tables 1–2
| 实验 / 任务 Experiment / task | 指标 Metric | Selected Selected | Final Final | 差值 Difference |
|---|---|---|---|---|
| Filtered BC · Threading D2 | pass@1 / pass@32 | 0.371 / 0.886 | 0.321 / 0.812 | +5.0 / +7.4 pp |
| Filtered BC · Mug Cleanup D1 | pass@1 / pass@32 | 0.558 / 0.914 | 0.502 / 0.866 | +5.6 / +4.8 pp |
| Critic 排序 · Threading D2 Critic ranking · Threading D2 | 动作对排序准确率 Pairwise action-ranking accuracy | 0.534 | 0.521 | +1.3 pp |
| Critic 排序 · Mug Cleanup D1 Critic ranking · Mug Cleanup D1 | 动作对排序准确率 Pairwise action-ranking accuracy | 0.589 | 0.547 | +4.2 pp |
| 真实 Push-T · epoch 46 vs. 200 Real Push-T · epoch 46 vs. 200 | pass@1 / pass@5 | 0.62 / 0.95 | 0.52 / 0.75 | +10 / +20 pp |
Critic 排序检验中，两种 critic 给同一组 8 个候选动作打分；每个候选的参考价值由 16 次分支模拟执行估计。因此 0.534 是排序准确率，不是 53.4% 的机器人成功率。
For critic ranking, both critics score the same eight candidate actions; reference values are estimated using 16 branched simulator rollouts per candidate. Thus 0.534 is ranking accuracy, not 53.4% robot success.
后续训练、增强与世界模型
Posttraining, augmentation, and world models
主图显示，JointAug 的最终模型在 MimicGen 上比最终基线更适合 residual RL，常达到或超过基线中挑选出的模型；RoboCasa 的 action-only 变体在 SFT 上达到与选定模型相近的表现。更困难的 3-Piece Assembly D2 使用 500 条演示，JointAug 的 pass@10 约为 0.8，基线低于 0.1；后续 residual RL 成功率超过 0.8，而对比基线初始化在测试预算内仍接近零。这是附录报告的近似结果，不是跨任务平均。Figs. 5–6, 18
The main figures show that the final JointAug model is a stronger residual-RL initialization than the final baseline on MimicGen, often matching or exceeding the selected baseline. The RoboCasa action-only variant reaches SFT performance similar to the selected checkpoint. On the harder 3-Piece Assembly D2 task with 500 demonstrations, JointAug achieves roughly 0.8 pass@10 versus below 0.1 for the baseline; subsequent residual RL exceeds 0.8 success while the compared baseline initialization remains near zero within the tested budget. These are approximate appendix results, not an across-task average. Figs. 5–6, 18
证据并不限于冻结基础策略：低维状态输入、约 1.17M 参数的 flow-matching 策略经 OGPO 全策略微调时，选定模型在 5 个任务中的 4 个上更好。另一方面，更早且 pass@1 更低的模型，可能在更长 RL 预算下反超；不存在对所有训练预算都最优的单一排序。App. C
The evidence extends beyond frozen base policies: with low-dimensional inputs and approximately 1.17M-parameter flow-matching policies, selected checkpoints perform better on four of five tasks under OGPO full-policy finetuning. However, even earlier models with lower pass@1 can overtake others given longer RL budgets; no single ranking is optimal for every budget. App. C
真实 Push-T 的世界模型实验把相同演示分别与 200 条 selected 或 final 模型的 rollout 混合，保留成功与失败轨迹；另外从中间检查点收集 100 条验证轨迹。Selected 数据训练的模型，第一段预测 PSNR 为 27.9 对 26.8，LPIPS 为 0.112 对 0.120；第二段自回归预测分别为 25.2 对 24.1、0.136 对 0.145。PSNR 越高越好，LPIPS 越低越好；这些是图像预测指标，不是控制成功率。Table 3
For real Push-T world models, the same demonstrations are combined with 200 selected- or final-checkpoint rollouts, retaining both successes and failures; 100 validation trajectories come from intermediate checkpoints. The selected-data model achieves first-chunk PSNR 27.9 versus 26.8 and LPIPS 0.112 versus 0.120; for the autoregressive second chunk, the values are 25.2 versus 24.1 and 0.136 versus 0.145. Higher PSNR and lower LPIPS are better; these are image-prediction metrics, not control success. Table 3
消融结果：哪些简单解释已被检验？
Ablations: which simple explanations were tested?
采样器从 DDIM 10 步换到 100 步、改变 η 或使用 DDPM，仍观察到后期覆盖下降。改变学习率计划、加长 EMA 记忆，以及把 U-Net 从 24M 扩到 1.0B 参数，都未根除该趋势。增加演示到 500 条通常减弱或推迟下降，但两个数据规模不足以建立 scaling law。PostBC 常保留更多覆盖，却可能损失 pass@1；TMRL 的 context-smoothed 预训练在本设置不及 JointAug，但该实验用从任务演示训练的视觉表征，不能直接推广到原方法的预训练 VLM 表征。App. E–G
The late coverage decline persists when changing DDIM from 10 to 100 steps, varying η, or using DDPM. Changing learning-rate schedules, extending EMA memory, and scaling the U-Net from 24M to 1.0B parameters do not remove it. Increasing demonstrations to 500 generally weakens or delays the decline, but two dataset sizes cannot establish a scaling law. PostBC often preserves more coverage at a possible pass@1 cost. TMRL context-smoothed pretraining trails JointAug here, but the test uses visual representations learned from task demonstrations, so it does not directly generalize to the original method’s pretrained VLM representations. App. E–G
局限与展望：如何评价这项贡献？
Limitations and outlook: how should this contribution be judged?
怀疑者的解读：这可以看作“有限数据过拟合 + 分布感知的早停 + 合成数据正则化”。pass@k 与生成增强本身都不是全新概念。真正新增的具体环节，是将固定起点的闭环成功分布与机器人后续训练联系起来，并给出可直接在保留动作样本上计算的分位数 minimax 选择规则，再用训练干预检验能否改善两类指标。
A skeptical reading: this is finite-data overfitting plus distribution-aware early stopping and synthetic-data regularization. Neither pass@k nor generative augmentation is new by itself. The concrete addition is linking fixed-start closed-loop success distributions to robot posttraining, providing a quantile-minimax selection rule computed from held-out action samples, and testing whether a training intervention improves both metrics.
- 代理指标的边界：与单条专家动作的 MSE 不等于任务价值；同一观测可能存在多种有效动作。低误差分位数不是闭环覆盖的数学保证，归一化也依赖已检查的模型集合与训练范围。 Proxy limits: MSE to one demonstrated action is not task value; multiple actions may be valid at the same observation. Low error quantiles do not mathematically guarantee closed-loop coverage, and normalization depends on the checkpoints and training range examined.
- 不是任意后训练的预测器：大 k 的优势在 pass@1 相近时最有解释力。RL 算法、任务动力学与交互预算会改变排序；覆盖性也不能代替执行稳定性。 No universal posttraining predictor: large-k advantages are most informative when pass@1 is comparable. RL algorithms, task dynamics, and interaction budgets can change the ordering; coverage cannot replace stable execution.
- 真实实验规模有限：只有一个真实 Push-T 任务、20 个初始状态。表中的点估计没有给出该对比的置信区间，五次内成功率也不是实际部署中允许任意重置的承诺。独立起点数量比总 rollout 数更能限制跨场景结论。 Limited hardware evidence: one real Push-T task and 20 initial states. The reported point estimates do not provide confidence intervals for this comparison, and success within five attempts does not imply arbitrary resetting is available in deployment. The number of distinct starts, not merely the rollout count, limits conclusions across conditions.
- 增强仍有成本与偏差：联合生成器需要额外训练、采样与窗口选择；论文没有证明其学到真实总体分布。大规模 VLA 使用了另一种 action-only 方法，尚未验证完整 JointAug 在该规模上的成本收益。 Augmentation adds cost and bias: a joint generator requires extra training, sampling, and window selection; the paper does not prove it learns the true population distribution. Large-scale VLA experiments use a different action-only method, leaving the full JointAug cost–benefit tradeoff at that scale untested.
合理的下一步是同时控制示范数据的覆盖范围与局部密度，并在同一总计算与机器人交互预算下比较完整预训练→后训练流程。本文支持“检查整个流程”这一实验方向；并未证明更多预训练普遍有害，或 JointAug 是所有策略的最佳训练方法。
A useful next step is to control both the breadth and local density of demonstrations and compare complete pretraining→posttraining pipelines under matched total compute and robot-interaction budgets. The paper supports evaluating the full pipeline; it does not establish that more pretraining is universally harmful or that JointAug is the best method for every policy.
如何使用这些结论
Putting the findings to use
- 保留足够多的检查点。在相同保留观测上重复采样动作，跟踪误差分位数，不只记录一个均值；匹配采样配置有助于公平比较。 Retain a checkpoint sequence. Repeatedly sample actions at the same held-out observations and track error quantiles alongside the mean; match sampling settings for fair comparisons.
- 按论文准则选择候选。先逐观测计算分位数再平均，避免把全部动作误差混成一个分布。实现时为零误差取对数与零归一化范围规定数值处理；这是工程补充，不是论文报告的新方法。 Select a candidate using the paper’s criterion. Compute quantiles within observations before averaging rather than pooling all errors. Define numerical handling for log-zero errors and zero normalization ranges; this is an engineering addition, not a new method reported in the paper.
- 用闭环与后训练检查实际收益。若可重置，在同一组起点测 pass@1 与若干较大 k；再在计划使用的 SFT 数据或 RL 预算下比较候选与最终模型。不能只用离线代理宣布成功。 Verify gains in execution and posttraining. Where resets are possible, measure pass@1 and several larger k on matched starts; then compare the candidate and final model using the intended SFT data or RL budget. Offline proxies alone cannot establish success.
评测细节：有限样本 pass@k 与 oracle
Evaluation detail: finite-sample pass@k and the oracle
在某个初始条件上执行 K 次，其中 s 次成功，论文对 k≤K 使用下式，再跨初始条件取平均。它使用全部样本的组合，而非只看前 k 次；不能从聚合成功率恢复该条件级指标。
With K rollouts at one initial condition and s successes, the paper uses the following estimator for k≤K, then averages across conditions. It uses combinations of all samples rather than only the first k attempts; this condition-level metric cannot be recovered from aggregate success alone.
\[\widehat{P}_k(c)=1-\frac{\binom{K-s}{k}}{\binom{K}{k}}.\]
论文所谓 closed-loop oracle，是把各个 k 的 pass@k 在检查点间归一化成距离最优值的差，再最小化最差差值；它不是直接选择后训练成功率最高的模型。因此“接近 oracle”指接近这个平衡规则，不等于证明任意后训练下全局最优。App. E, H
The paper’s closed-loop oracle normalizes each k’s pass@k across checkpoints into a gap from the best value, then minimizes the worst gap. It does not directly select the model with the highest posttraining success. “Near the oracle” therefore means near this balancing rule, not globally optimal for arbitrary posttraining. App. E, H
来源与版本
Sources and version
本研究以 2026-11-01 发布的 arXiv v1 全文及附录为数值依据，项目页和公开代码页提供入口。页面中的解释性两条件示例与批判性分析为本研究整理；图 1–4 提取自论文 HTML，图 5 根据原文精确数值重绘。未运行机器人训练或独立复现实验。
Numerical claims use the full arXiv v1 paper and appendices, released 2026-11-01; the project and public code pages provide access points. The two-condition teaching example and critical interpretation are additions in this study. Figures 1–4 are extracted from the paper HTML; Figure 5 replots exact reported values. Robot training and independent experimental reproduction were not performed.