RigidFormer: Learning Rigid Dynamics using Transformers

研究动机：碰撞局部发生，整件物体一起移动

Motivation: local contact moves the whole object

碰撞可能在很小的表面区域突然改变运动，而自回归预测会不断放大早期误差。基于网格顶点、边或面的消息传递能够表达接触，但需要网格连接信息，并且随几何分辨率增加而变贵。感知系统提供的点云往往没有这些连接信息。

A collision can abruptly change motion at a small surface region, and autoregressive prediction amplifies early errors. Message passing over mesh vertices, edges, or faces can represent contact, but it requires connectivity and becomes more expensive as geometry resolution grows. Perception systems often provide point clouds without that connectivity.

核心想法是将接触几何的读取与刚体运动的输出分开：编码器和邻近点搜索仍然读取表面细节，Transformer 主要在物体之间交流，最终输出受刚体约束。经典物理引擎、PointNet、物体交互模型和刚体配准都已有先例；这里的研究价值在于把它们组合成能长时展开、支持多种步长的点云仿真器。

The central idea separates reading contact geometry from producing rigid motion: the encoder and nearest-neighbor search still read surface detail, the Transformer mainly exchanges information between objects, and the output is constrained to be rigid. Classical engines, PointNet, object interaction models, and rigid alignment all have precedents; the contribution is their integration into a point-cloud simulator supporting long rollouts and multiple step sizes.

核心方法：从点云到受约束的状态更新

Method: from point clouds to constrained state updates

术语速查：每个抽象词具体指什么

Terminology: what the abstractions mean here

1. 用点特征形成物体 token

1. Build object tokens from point features

每个点输入 12 维特征：到另一物体或地面的最近点位移 3 维、两帧位置差 3 维、相对参考帧的位置差 3 维，以及质量、摩擦系数、恢复系数 3 维。两帧位置差是离散速度替代量，并未除以时间间隔。共享的分层 PointNet 在 100%、50%、25%、12.5% 四个层级聚合，形成一个 768 维物体 token。

Each point supplies 12 features: 3 nearest-point displacement components to another object or the ground, 3 position increments between frames, 3 offsets from the reference frame, and 3 scalars for mass, friction, and restitution. The position increment is a discrete velocity surrogate, not divided by elapsed time. A shared hierarchical PointNet aggregates at 100%, 50%, 25%, and 12.5% levels to form one 768-dimensional object token.

因此“mesh-free”表示不需要三角形连接信息，并不表示不需要几何处理、物体分组或跨帧点对应。最近邻搜索仍然在表面点上进行，而且是实测的主要耗时来源。

Thus “mesh-free” means triangle connectivity is unnecessary; it does not remove geometry processing, object grouping, or point correspondence between frames. Nearest-neighbor search still operates on surface points and is the largest measured runtime cost.

2. 物体之间交流，锚点读取局部接触信息

2. Exchange object context and read local contact features

4 层 Transformer 使用 6 个 attention heads，并加入 16 个可学习 register tokens 作为共享工作空间。每个 attention 输出有依赖 query 的 sigmoid 门，削弱不相关的交互；每层用 FiLM 接收步长编码 [s, s²]。没有按物体序号设置的位置编码，因此重排物体只会重排输出。

A 4-layer Transformer uses 6 attention heads and 16 learned register tokens as a shared workspace. A query-dependent sigmoid gate attenuates each attention output, and each layer receives the step code [s, s²] through FiLM. There is no object-index positional encoding, so reordering objects simply reorders their outputs.

每个锚点可以查询场景中所有有效物体，而不只查询自己的物体。附录 G 说明查询并行读取 decoder 的 {0, 1, 2, 4} 层表示，合并浅层几何与深层交互信息。AVP 再补充锚点附近的表面特征：距离越近，权重越大；聚合后投影为 256 维。

Each anchor can query all valid objects in the scene, not just its own object. Appendix G specifies parallel reads from decoder representations {0, 1, 2, 4}, combining shallow geometry with deeper interaction context. AVP additionally supplies nearby surface features: closer points receive larger weights, and the pooled feature is projected to 256 dimensions.

这里 q 是锚点坐标，x 是表面点坐标，f 是编码器点特征，σ 是学习的核带宽。这个归一化求和不依赖点的排列；距离权重在整体刚体变换下不变，但这并不使全部特征或整个网络具有旋转等变性。

Here q is an anchor position, x a surface-point position, f an encoded point feature, and σ the learned kernel bandwidth. The normalized sum is independent of point order. Its distance weights are unchanged by a common rigid transform, but that does not make all features or the whole network rotation-equivariant.

3. 预测锚点加速度，再强制恢复刚体

3. Predict anchor acceleration, then enforce rigidity

网络预测每个锚点的 3 维加速度，用 Verlet 公式得到候选下一帧位置。候选锚点可能彼此拉伸，因此用可微 Kabsch 配准，把参考锚点与候选锚点做最小二乘刚体对齐；SVD 的行列式修正排除镜像反射。最终将同一个旋转 R 与平移 b 施加到全部参考点。

The network predicts a 3D acceleration for each anchor and uses Verlet integration to obtain candidate next positions. These candidates may stretch relative to each other, so differentiable Kabsch alignment finds the least-squares rigid fit from reference anchors to candidates; an SVD determinant correction excludes reflections. The same rotation R and translation b are then applied to every reference point.

它严格保证物体内部距离不变，但不保证不同物体之间不穿透，也不自动保证动量、能量或摩擦规律正确。论文用 RoMa 实现可微刚体配准，以改善 SVD 退化附近的梯度稳定性。

This exactly preserves distances within each object, but does not ensure that different objects avoid penetration or that momentum, energy, and friction laws are satisfied. The paper uses RoMa for differentiable registration to improve gradient stability near SVD degeneracies.

4. ARoPE：排列对称性明确，形状范围声明需澄清

4. ARoPE: clear ordering symmetry, ambiguous shape-extent claim

作者为每个锚点构造 96 维相位描述，并在物体内平均，再用其旋转 attention 的 query/key 通道。平均聚合确实不受锚点编号影响；附录 B 也明确指出整体模型不具有完整 SE(3) 等变性，因为重力和地面都在世界坐标系中表达。

The authors construct a 96-dimensional phase descriptor for each anchor, average it within the object, and use it to rotate query/key attention channels. Mean pooling is indeed invariant to anchor indexing. Appendix B explicitly states that the complete simulator is not fully SE(3)-equivariant, since gravity and ground contact are expressed in world coordinates.

5. 训练：在投影前后都监督锚点

5. Training: supervise anchors before and after projection

四项都是锚点处的 Smooth L1 loss；全分辨率顶点由刚体变换间接受监督。主 MOVi 模型训练 300 epochs，序列长度 8，近似均匀抽取步长 {1, 5, 10}；没有 curriculum learning 或 scheduled sampling。AdamW 的初始基础学习率为 10⁻⁴，warmup 10 epochs 后余弦衰减至 10⁻⁶。所有这些步长由同一个条件模型支持，但没有证明任意未训练步长都同样可靠。

All four terms are Smooth L1 losses at anchor locations; full-resolution vertices receive indirect supervision through the rigid transform. The main MOVi model trains for 300 epochs with sequence length 8 and approximately uniform sampling of steps {1, 5, 10}, without curriculum learning or scheduled sampling. AdamW uses a base learning rate of 10⁻⁴, a 10-epoch warmup, and cosine decay to 10⁻⁶. A single conditioned model supports these step sizes, but reliability at arbitrary unseen steps is not established.

最关键的一句话：表面点负责提供接触证据，物体 token 负责交换交互信息，四个锚点负责提出运动，刚体配准负责保证不变形。 方法依据：§3 与 Appendix G。

The key mechanism: surface points supply contact evidence, object tokens exchange interaction context, four anchors propose motion, and rigid alignment prevents deformation. Method sources: §3 and Appendix G.

主要实验结果：先看相同步长，再看效率

Results: matched steps first, then efficiency

MOVi-A、MOVi-B、MOVi-Sphere 各有 1,200 个合成场景，每个场景 480 帧，训练／验证／测试划分为 960／120／120。指标是物体质心位置的 RMSE（米）与四元数测地距离对应的姿态 RMSE（度），越低越好。测试先给两帧，再自回归展开；主表按步长 1 比较。以下数值全部来自 v1 论文，百分比为本研究依据表值计算。

MOVi-A, MOVi-B, and MOVi-Sphere each contain 1,200 synthetic scenes of 480 frames, split 960/120/120 for training/validation/testing. Metrics are center-of-mass position RMSE in meters and orientation RMSE based on quaternion geodesic distance in degrees; lower is better. Evaluation starts from two frames and then rolls out autoregressively; the main table compares step size 1. All values below come from the v1 paper; percentages are calculated here from its tables.

相同步长下的精度：优势存在，但不是每项都领先

Accuracy at matched steps: gains, but not universal wins

以上摘自 Table 2，粗体是此处各列的最优值。MOVi-B 的 100 帧误差相对 HopNet 降低约 8.5%（位置）和 14.4%（姿态）。但在 MOVi-B 的 50 帧位置误差上，HopNet 的 0.047 m 优于 RigidFormer 的 0.050 m，因此不能概括为全面胜出。主比较匹配点数与基线使用的网格顶点数。

These values are from Table 2; bold marks the best values in these columns. At 100 frames on MOVi-B, RigidFormer reduces error relative to HopNet by about 8.5% in position and 14.4% in orientation. However, at 50 frames on MOVi-B, HopNet’s 0.047 m position error is better than RigidFormer’s 0.050 m, so the improvement is not universal. The main comparison matches point counts to the mesh vertex counts used by baselines.

步长：更少的网络调用能降低长时误差

Step size: fewer network calls can reduce long-horizon error

在 MOVi-B 的 100 帧时域，步长 10 相对步长 1 的位置误差降低约 28.6%。这反映了粗粒度未来状态预测的优势，不证明接触瞬间也更精确：大步长会跳过中间状态。论文称已将预测映射到相同物理帧，但没有充分说明步长 10 如何对应 75 帧等非整除时域；复现时应核对具体采样、对齐或插值规则。

At the 100-frame MOVi-B horizon, step 10 reduces position error by about 28.6% relative to step 1. This supports coarse future-state prediction, not necessarily more accurate contact timing: larger steps skip intermediate states. The paper states that predictions are mapped to matching physical frames, but does not fully explain how step 10 aligns with non-divisible horizons such as 75 frames; reproduction should verify the sampling, alignment, or interpolation rule.

消融：门控的贡献比可微配准增益更大

Ablations: gating contributes more than differentiable alignment

Tables 6–8 显示，开启 gate 后此处位置误差从 0.259 降至 0.161 m，约降低 37.8%；可微配准对应 0.169 → 0.161 m，约降低 4.7%。论文将后者解释为投影梯度流的收益，因此不应把这一行自动理解为完全删除所有刚体约束。4 个锚点是折中选择，8 个锚点有时姿态更好，但位置未必更好。ARoPE 也不是所有单元格最优：此处 sinusoidal 的 15.30° 略优于 15.33°。这些表未提供误差条，微小差距应谨慎解释。

Tables 6–8 show that gating reduces position error here from 0.259 to 0.161 m, about 37.8%; differentiable alignment changes 0.169 to 0.161 m, about 4.7%. The paper interprets the latter as a benefit of gradient flow through projection, so this row should not automatically be read as removing every rigidity constraint. 4 anchors are a trade-off: 8 sometimes improve orientation without improving translation. ARoPE is not best in every cell either: sinusoidal PE’s 15.30° is slightly better than 15.33° here. These tables provide no error bars, so small gaps deserve caution.

运行时间：端到端瓶颈是邻近点搜索

Runtime: nearest-neighbor search is the end-to-end bottleneck

Table 10 的测试在 NVIDIA GeForce RTX 5080 上进行，对 50 步展开重复 10 次取平均。按每步毫秒数计算，RigidFormer 比 FIGNet 快约 8.0×，比 HopNet 快约 100.9×。这不是相对经典物理引擎的速度比较，也不能把四舍五入后的 FPS 直接相除来重算 101×。

Table 10 measures a 50-step rollout averaged over 10 iterations on an NVIDIA GeForce RTX 5080. The milliseconds-per-step values imply about 8.0× speedup over FIGNet and 100.9× over HopNet. This is not a speed comparison against classical physics engines, and dividing rounded FPS values does not recover the reported 101× ratio.

细分时间为编码器 2.67 ms、物体交互 6.01 ms、锚点交互 9.94 ms、KNN 22.80 ms、Kabsch 0.45 ms；分项因四舍五入合计为 41.87 ms，论文总计为 41.86 ms。模型有 174.8M 参数，报告峰值显存为 1.80 GB。物体 attention 的二次项确实更小，但编码、KNN 和坐标广播仍随点数变化，整个系统并非与点数无关。

The breakdown is 2.67 ms for the encoder, 6.01 ms for object interaction, 9.94 ms for anchor interaction, 22.80 ms for KNN, and 0.45 ms for Kabsch. Rounded components sum to 41.87 ms; the paper reports 41.86 ms overall. The model has 174.8M parameters and a reported 1.80 GB peak memory footprint. Object attention reduces the quadratic term, but encoding, KNN, and coordinate broadcasting still depend on point count; the full system is not independent of resolution.

泛化：点数适应较明确，跨分布仍有反例

Generalization: resolution flexibility, but mixed transfer

点数实验使用 {128, 256, 512, 1024} 个点训练，在未见过的 768 点输入上测试。MOVi-B 的 100 帧误差在步长 1 为 0.189 m / 16.22°，步长 10 为 0.137 m / 11.13°（Table 5）。这验证了同一基准内的点分辨率适应，不能等同于任意传感器或任意新形状泛化。

The resolution experiment trains with {128, 256, 512, 1024} points and tests at an unseen count of 768. At 100 frames on MOVi-B, errors are 0.189 m / 16.22° with step 1 and 0.137 m / 11.13° with step 10 (Table 5). This supports point-resolution flexibility within the benchmark, not arbitrary sensor or shape generalization.

跨数据集结果并非始终优于 HopNet。例如 B → A、步长 1、100 帧时，RigidFormer 为 0.234 m / 20.28°，HopNet 为 0.202 m / 19.15°；B → Sphere 也落后。相反，Sphere → B 时 RigidFormer 的 0.161 m / 16.81° 优于 HopNet 的 0.188 m / 17.13°（Table 3）。训练几何更丰富并不自动带来更强迁移。

Cross-dataset transfer does not always beat HopNet. For B → A at step 1 and 100 frames, RigidFormer gives 0.234 m / 20.28° versus HopNet’s 0.202 m / 19.15°; B → Sphere also underperforms. Conversely, for Sphere → B, RigidFormer’s 0.161 m / 16.81° beats HopNet’s 0.188 m / 17.13° (Table 3). More diverse training geometry does not automatically improve transfer.

大场景与关节体：有用的扩展，但证据范围有限

Large scenes and articulated bodies: useful, limited extensions

WreckingBall 在 28、65、126、217 个物体的四种配置上联合训练，共 72 个训练样本，步长固定为 8。最大场景只有 1,771 个顶点，因为每个方块仅用 8 个角点。作者报告约 20 FPS；64、125、216 个方块场景在 50 个物理步的误差依次为 1.210 m / 20.50°、0.690 m / 16.50°、0.130 m / 4.60°。不同场景难度与大量静止物体可能影响平均值，因此这些数值不证明物体越多就越准确。

WreckingBall is trained jointly on four configurations of 28, 65, 126, and 217 objects, using 72 training samples and fixed step size 8. The largest scene has only 1,771 vertices because each cube uses 8 corners. The authors report about 20 FPS; at 50 physical steps, the 64-, 125-, and 216-cube cases have errors of 1.210 m / 20.50°, 0.690 m / 16.50°, and 0.130 m / 4.60°, respectively. Different scene difficulty and the many stationary objects can affect the averages, so these values do not establish that more objects improve accuracy.

关节体把每个身体部件当作物体，通过额外 FiLM 路径加入目标速度与方向。ASE 实际为 15 个身体链接加剑盾，共 17 个部件；G1 为 31 个。100 个物理步的误差为 0.062 m / 14.47° 与 0.072 m / 16.26°。它们以 30 Hz 数据按步长 10 训练，即 3 Hz 的有效预测率；数据来自既有控制策略，因此这里主要验证命令条件下的运动预测，尚不是新的底层控制器或显式关节约束求解器。

For articulated bodies, each part is treated as an object, with an additional FiLM path for target speed and direction. ASE uses 15 body links plus sword and shield, totaling 17 parts; G1 has 31. At 100 physical steps, errors are 0.062 m / 14.47° and 0.072 m / 16.26°, respectively. They train on 30 Hz data with step 10, giving an effective 3 Hz prediction rate. Data come from existing control policies, so this primarily tests command-conditioned motion prediction, not a new low-level controller or an explicit joint-constraint solver.

数值来源：§4, Tables 2–10、Appendix F、Appendix H、Appendix I。

Numerical sources: §4, Tables 2–10, Appendix F, Appendix H, and Appendix I.

局限与批判性定位

Limitations and critical assessment

怀疑者会怎样概括？这是 PointNet 编码、物体级 Transformer、稀疏点运动预测与 Kabsch 投影的组合，且大步长会减少误差积累次数。把这些部件视为全新物理原理会夸大贡献。最具体的新东西是将局部表面特征通过 AVP 接入稀疏锚点预测，并与物体交互、多步长条件及可微刚体投影结合，在不依赖网格连接的情况下取得有竞争力的接触轨迹与实测速率。

What would a skeptic say? This combines PointNet encoding, an object-level Transformer, sparse point-motion prediction, and Kabsch projection, while larger steps reduce the number of error-accumulating updates. Treating these components as a new physical principle would overstate the contribution. The most concrete novelty is coupling local surface features through AVP to sparse anchor prediction, object interactions, step conditioning, and differentiable rigid projection, obtaining competitive contact trajectories and measured speed without mesh connectivity.

主要限制包括：需要物体标签和稳定的几何对应；部分可见实验只随机删除 25% 的点；接触是从数据学习而非显式求解；结果以合成数据为主；缺少实测不确定性、穿透量和守恒误差分析。论文未给出主模型训练总耗时，174.8M 参数也使“计算轻量”只能针对报告的推理比较来理解。

Key limitations include required object labels and stable geometric correspondence; partial observations tested only by random removal of 25% of points; learned rather than explicitly solved contacts; predominantly synthetic data; and no measured uncertainty, penetration, or conservation-error analysis. Total main-model training time is not reported, and the 174.8M parameters mean that “lightweight” should be understood only relative to the reported inference comparisons.

值得继续验证的方向是：真实遮挡与分割噪声、未知材质、未训练物体数、碰撞时刻误差、以及带轻量接触修正的混合模型。对规划而言，大步长可能适合快速比较远期结果，再用小步长细化；这是论文提出的应用方向，不是已经完成的规划或自适应步长实验。

Useful next tests include real occlusion and segmentation noise, unknown materials, unseen object counts, collision-timing error, and hybrid models with lightweight contact correction. For planning, large steps could cheaply compare distant outcomes before smaller steps refine them. This is a proposed use direction, not a completed planning or adaptive-step experiment.

复现与阅读要点

Reproduction and reading notes

代码状态：v1 摘要仅承诺“发表后发布代码”，本文所读来源没有可核实的代码链接。本研究没有将这一承诺当作当前已开源的证据。

Code status: the v1 abstract only promises release upon publication, and the sources read for this study provide no verifiable code link. This study does not treat that promise as evidence that code is currently public.

来源与版本范围

Sources and version scope

主要来源是 2026-05-09 提交的 arXiv v1，包括正文和附录，共 26 页。本页面研究日期为 2026-10-14；这不是论文发表日期。图 1、4、5 直接取自论文的 HTML 版本；图 2、3 根据论文表格重绘。

The primary source is arXiv v1, submitted 2026-05-09, including the main text and appendices across 26 pages. This study is dated 2026-10-14; that is not the paper’s publication date. Figures 1, 4, and 5 are extracted from the paper’s HTML version; Figures 2 and 3 replot its tables.

• RigidFormer — 摘要与版本记录 abstract and version history
• RigidFormer — 完整 HTML v1 full HTML v1
• RigidFormer — 原始 PDF v1 source PDF v1