Error-Conditioned Neural Solvers

研究动机：满足方程不等于接近正确解

Motivation: satisfying the equation versus recovering the solution

神经算子把输入场直接映射为解场，速度快，但推理时通常不会检查自己的预测是否满足方程。PINO 的测试时优化、DiffusionPDE 的梯度引导和 PCFM 的约束投影会做物理修正，却增加了逐实例的优化开销。在病态系统中，这种优化还可能主要降低一个与真正解误差不一致的指标。

Neural operators map input fields directly to solution fields and are fast, but usually do not check their predictions against the equations at inference. PINO test-time optimization, DiffusionPDE gradient guidance, and PCFM constraint projection add physical corrections at the cost of per-instance optimization. In ill-conditioned systems, that optimization can mainly improve a metric that does not track actual solution error.

最直接的例子是线性残差 \(r(u)=Au-f\)。若某个方向被 \(A\) 缩小很多，解沿这个方向偏移很远，方程却只感觉到很小的变化。ENS 的出发点是：保留残差的空间信息作为输入，用真实解误差教网络如何更新。

The simplest example is the linear residual \(r(u)=Au-f\). If \(A\) strongly suppresses one direction, the solution can move far in that direction while barely changing the residual. ENS therefore keeps the residual’s spatial information as input and learns updates using true solution error.

核心方法：让网络读自己的残差场

Method: let the network read its current residual field

术语速查

Concrete terminology

1. 预测、计算残差、再修正

1. Predict, evaluate the residual, and correct

1. 初始化：预测器 \(P_\theta(f,g)\) 产生初始解；这里 \(f,g\) 代表任务提供的输入场与条件。
2. Initialize: the predictor \(P_\theta(f,g)\) produces an initial solution; \(f,g\) denote the task’s input fields and conditions.
3. 检查方程：静态方程使用二阶中心差分；流体方程使用空间谱微分与时间有限差分，在每次修正后重新计算残差。
4. Check the equation: use second-order central differences for static equations, or spectral spatial derivatives and finite differences in time for flows. Recompute the residual after every update.
5. 更新解：将原始条件、当前解、残差场沿通道维拼接，输入共享修正器，再用固定步长缩放输出。推理时无需对残差损失反向传播，也无需显式求解 Jacobian 系统。
6. Update: concatenate the original conditions, current solution, and residual along the channel dimension, feed them to the shared corrector, and scale its output by a fixed step size. Inference needs neither backpropagation through a residual loss nor an explicit Jacobian solve.

\[r^{(k)}=\mathcal F(\hat u^{(k)};f),\qquad \hat u^{(k+1)}=\hat u^{(k)}+\beta C_\phi(\hat u^{(k)},r^{(k)};f,g).\]

2. 用正确答案训练每一步

2. Supervise each step against the solution

训练时展开 5 次修正，用每一步预测相对于数值参考解的均方误差取平均；训练目标中不包含 PDE 残差项。正文说明每个修正步的输入会 detach，因此不跨迭代反向传播。作者称预测器与修正器联合优化，但这不能理解为对整个循环做无截断的梯度传播。

Training unrolls 5 corrections and averages the mean squared error between each prediction and the numerical reference solution. There is no PDE-residual term in the training objective. The main text states that each correction step’s input is detached, so gradients do not propagate across iterations. The authors describe joint optimization of predictor and corrector, which should not be read as unrestricted backpropagation through the full loop.

\[\mathcal L=\frac{1}{5}\sum_{k=1}^{5}\frac{1}{n}\|\hat u^{(k)}-u_{\mathrm{gt}}\|_2^2.\]

静态网络使用 4 层 FNO、64 个隐藏通道、每维 20 个 Fourier modes，输入和输出投影改为 3 层 CNN（Darcy 为 5 层）。流体任务使用 VideoPDE 的 Transformer 骨干。AdamW 学习率与 weight decay 都为 10⁻⁴，batch size 为 20，实验使用单张 RTX 6000 Ada。

Static networks use 4 FNO layers, 64 hidden channels, and 20 Fourier modes per dimension, with 3-layer CNN input and output projections (5 layers for Darcy). Flow tasks use a VideoPDE Transformer backbone. AdamW uses a learning rate and weight decay of 10⁻⁴, with batch size 20 on a single RTX 6000 Ada.

3. 推理步数可增加，但需要选择停止点

3. More inference steps require a stopping choice

每个任务先用少量验证样本观察残差平台，再固定测试步数，并非每个测试样本都使用自动停止判据。Helmholtz 从同分布的 15 步增至波数外推 80 步、超分辨率 120 步；Navier–Stokes 为 5、15、10 步，分别对应同分布、黏度变化、外力变化；Kolmogorov flow 为 12、15、15、90 步，最后一项为超分辨率。

A few validation examples determine a residual plateau and a fixed step count for each task; this is not an automatic stopping criterion applied to each test sample. Helmholtz increases from 15 in-distribution steps to 80 for wavenumber extrapolation and 120 for super-resolution. Navier–Stokes uses 5, 15, and 10 steps for in-distribution, viscosity-shift, and forcing-shift tasks. Kolmogorov flow uses 12, 15, 15, and 90 steps, with the last count for super-resolution.

常规步长 β：大多数静态问题为 0.05，Poisson 为 0.03，Navier–Stokes 为 0.1，Kolmogorov flow 为 0.05。超分辨率与 Darcy 渗透率外推需要在小型保留集上用 line search 选择更小步长。因此“zero-shot”指无需在目标分布微调网络权重，并不表示不调推理超参数。

Default β is 0.05 for most static problems, 0.03 for Poisson, 0.1 for Navier–Stokes, and 0.05 for Kolmogorov flow. Super-resolution and Darcy permeability extrapolation use smaller steps selected by line search on a small held-out set. Thus “zero-shot” means no network-weight fine-tuning on the target distribution, not the absence of inference-hyperparameter tuning.

最关键的一句话：网络学习如何利用残差修正解；它没有把减小残差本身当作训练目标。

The central mechanism: the network learns how to use the residual to correct the solution, without making residual reduction itself the training objective.

主要实验结果与消融

Results and ablations

四类二维 PDE 包括 Helmholtz（线性与非线性）、Poisson、Darcy 与 Navier–Stokes（含 Kolmogorov forcing）。每个方程分别训练模型；静态任务与 Navier–Stokes 各用 1,000 个训练样本，Kolmogorov flow 用 5,000 个，每个数据集有 100 个测试样本。主要任务包括系数外推、128²→256² 超分辨率，以及选定的跨方程迁移。

The four 2D PDE families are Helmholtz (linear and nonlinear), Poisson, Darcy, and Navier–Stokes (including Kolmogorov forcing). Models are trained separately per equation. Static tasks and Navier–Stokes use 1,000 training samples each, Kolmogorov flow uses 5,000, and each dataset has 100 test samples. Tests cover coefficient extrapolation, 128²→256² super-resolution, and selected cross-equation transfers.

表中的 relative L₂ 是预测与参考解之差的范数除以参考解范数，越小越好；PDE residual 是离散残差场的 MSE。残差尺度随方程、系数和离散方式改变，不应把不同 PDE 的原始残差值直接排序。以下为作者报告的结果，未独立重跑。

Relative L₂ is the norm of prediction error divided by the reference-solution norm; lower is better. PDE residual is the mean squared discrete residual. Its scale depends on the equation, coefficients, and discretization, so raw residual values should not be ranked across different PDEs. Results below are author-reported and were not independently rerun.

代表性精度比较

Representative accuracy comparisons

Kolmogorov 外力从 8π 变为 10π 时，ENS 的 relative L₂ 恰为 PINO(TTOP) 的 1/10；这给出一个具体的“10×”比较。这个优势不适用于所有任务：Navier–Stokes 超分辨率中，POSEIDON 的误差约低 4.5 倍，PINO(TTOP) 的 0.0173 也优于 ENS。

For the Kolmogorov forcing change from 8π to 10π, ENS has exactly 1/10 of PINO(TTOP)’s relative L₂ error, giving a concrete “10×” comparison. The advantage is not universal: on Navier–Stokes super-resolution, POSEIDON has about 4.5 times lower error, and PINO(TTOP)’s 0.0173 also beats ENS.

为什么必须同时报告残差和解误差

Why both residual and solution error matter

原表 4 与 A4 中，PINO(TTOP) 的残差更低，解却更差。在其他静态结果中，ENS 的 Poisson 同分布误差为 0.00363（FNO：0.0179），非线性 Helmholtz 外推误差为 0.0631（最佳非 ENS 的 PCFM：0.175）。跨方程结果存在标签不一致，见下方的来源核对。

In Tables 4 and A4, PINO(TTOP) has a lower residual but a worse solution. Other static results include ENS’s Poisson in-distribution error of 0.00363 (FNO: 0.0179) and nonlinear Helmholtz extrapolation error of 0.0631 (best non-ENS comparator PCFM: 0.175). Cross-equation results have a labeling inconsistency, discussed in the source audit below.

速度：明显快于逐实例优化，慢于单次 FNO

Speed: much faster than per-instance optimization, slower than one FNO pass

这些是原表 2–4 的同分布平均推理时间。Helmholtz 与 Navier–Stokes 分别约比 PINO(TTOP) 快 100 倍与 230 倍；ENS 仍比 FNO 慢。外推需要更多迭代，不能直接沿用这些延迟。附录的线性 Helmholtz 数值数据生成时间为 CPU 上 0.04 秒／样本，而 Navier–Stokes 为 GPU 上 23.3 秒／样本；这不是统一硬件、统一精度的数值求解器竞赛，也没有计入神经网络训练成本。

These are in-distribution mean inference times from Tables 2–4. Helmholtz and Navier–Stokes are approximately 100 and 230 times faster than PINO(TTOP), respectively; ENS remains slower than FNO. Extrapolation needs more iterations, so these latencies do not carry over. Appendix data-generation times are 0.04 seconds/sample on CPU for linear Helmholtz and 23.3 seconds/sample on GPU for Navier–Stokes. This is not a matched-hardware, matched-accuracy numerical-solver contest, and neural-network training costs are excluded.

消融支持了什么

What the ablations establish

• 残差输入确实有用：保持网络和步数不变，将残差换成全零场后，修正无法持续降低两种误差（图 7、A8–A9）。
• The residual input matters: replacing it with zeros while holding architecture and step count fixed prevents sustained improvement in both errors (Figures 7, A8–A9).
• 不是只靠更强骨干：同骨干 VideoPDE(TTOP) 在 Kolmogorov 同分布上误差为 0.434，ENS 为 0.00808，约相差 54 倍（表 A9）。
• A stronger backbone alone does not explain the gain: matched-backbone VideoPDE(TTOP) has Kolmogorov in-distribution error 0.434 versus ENS’s 0.00808, a roughly 54-fold difference (Table A9).
• 骨干仍是必要条件：纯 FNO 修正器未收敛；CNN-FNO-CNN 达到 0.00304。能接受残差通道，不代表一定能表示高频残差信息（表 A6）。
• Backbone capacity still matters: the pure FNO corrector fails to converge, whereas CNN-FNO-CNN reaches 0.00304. Accepting a residual channel does not ensure that high-frequency residual information is representable (Table A6).
• 残差场与损失梯度不同：改用物理损失梯度作为输入能更快降低残差，却使重建误差停滞；添加残差训练损失也未改善作者设置中的重建精度（图 A12、正文 §4）。
• A residual field differs from a loss gradient: conditioning on the physics-loss gradient lowers the residual faster but stalls reconstruction; adding a residual training loss also does not improve reconstruction in the reported settings (Figure A12, §4).

局限与批判性解读

Limitations and critical reading

它没有消除残差与重建误差的差距

It does not eliminate the residual–reconstruction gap

图 A1 是关键反例：ENS 的残差趋于平台时，relative L₂ 在经过最低点后会重新增大。初始化鲁棒性也仅是所测情况下残差收敛到相似水平，并不是所有初值收敛到同一个正确解的定理。因此，残差条件输入绕开了“只优化残差”的训练目标，却没有提供全局收敛或解误差认证。

Figure A1 is an important counterexample: as ENS’s residual approaches a plateau, relative L₂ can rise after reaching its minimum. Initialization robustness likewise means similar residual floors in the tested cases, not a theorem that every initialization reaches the same correct solution. Residual conditioning avoids training solely to reduce residuals, but provides neither global convergence nor certified solution accuracy.

贡献的准确边界

The precise contribution

怀疑者可以把 ENS 概括为“用监督学习训练一个循环残差修正器”。学习优化器、Recurrent Inference Machines 与迭代修正早已存在；正文还明确讨论了同样把残差信息引入网络的同期 PRISMA。本文更具体的贡献是：针对前向 PDE 求解，把当前解产生的完整空间残差交给共享修正器，并用解误差监督闭环更新，再展示一系列分布外测试。不能把它概括为首次使用残差信息的神经网络。

A skeptic could summarize ENS as a supervised recurrent residual corrector. Learned optimizers, Recurrent Inference Machines, and iterative correction already exist; the main text also discusses concurrent PRISMA, which puts residual information inside a network. The more specific contribution is a forward-PDE loop that feeds the current solution’s full spatial residual to a shared corrector and supervises its updates with solution error, backed by distribution-shift experiments. It should not be described as the first neural network to use residual information.

目前验证限于相对简单的二维系统，并假定推理时知道控制方程；每个方程单独训练，选定的跨方程迁移还不是一个通用 PDE 基础模型。三维、真实噪声观测、部分未知方程都仍需验证。“可能容忍不精确残差”是作者的假设，不能当作已证明能力。

Validation is limited to relatively simple 2D systems with known governing equations at inference. Separate training per equation and selected cross-equation transfers do not yet constitute a universal PDE foundation model. Three-dimensional systems, real noisy observations, and partially unknown equations remain untested extensions. Greater tolerance of imperfect residuals is an author hypothesis, not an established capability.

来源与复现

Sources and reproduction

主要依据是指定的 arXiv v2 正文与附录：方法见 §4、B.2、B.4–B.5；结果见表 2–4、A3–A9；停止行为见图 A1；理论限定见 A.1。原始图 1、3、6、A1 保留作者图内标记，两种语言共享同一组图片。交互图由表 4 数值重绘。

The primary source is the requested arXiv v2 text and appendix: method in §4, B.2, B.4–B.5; results in Tables 2–4 and A3–A9; stopping behavior in Figure A1; theoretical qualifications in A.1. Original Figures 1, 3, 6, and A1 retain their embedded author labels and are shared by both languages. The interactive plot uses Table 4 values.

公开代码 README 提供按方程划分的训练／评估入口、分布变化配置，以及数据与 checkpoint 链接。本研究读取了论文、项目页和 README，没有执行模型训练或独立复现实验。

The public code README provides equation-specific training and evaluation entry points, shift configurations, and links to data and checkpoints. This study read the paper, project page, and README; it did not train models or independently reproduce experiments.

• Error-Conditioned Neural Solvers · v2
• arXiv · version history
• ENS · project
• ENS · code and README