Interactive Multilayer Gaussian Garments for Low-Cost Try-On

作者 Authors Ryan S. Zesch, I-Chao Shen, Haoran Xie, Bo Zhu, Shinjiro Sueda, Takeo Igarashi

机构 Affiliations Texas A&M University; The University of Tokyo; Japan Advanced Institute of Science and Technology; Georgia Institute of Technology

发表 Publication Graphics Interface (GI) 2025

链接 Links Paper PDF · Author page · Video

把衣服看成由粗网格带着运动的彩色粒子：先快速模拟，再从多个虚拟相机找出穿到外衣前面的内衣面片，渲染时隐藏它们。它修正的是画面中“谁挡住谁”，不会把相交的三维衣服真正解开。

Think of clothing as colored particles carried by a coarse mesh: simulate it quickly, then use several virtual cameras to find inner-clothing triangles poking through outer clothing and hide them in the rendering. This corrects what occludes what in the image; it does not untangle the intersecting 3D garments.

Original Figure 1: layered garments and webcam try-on — 原论文图 1：分别采集的服装可以组合成多层穿搭，并用于摄像头驱动的交互试穿。服装外观来自静态采集，动态来自网格仿真。

17虚拟视点 virtual viewpoints

18–21 FPS启用 SH 的动画回放 animation playback with SH

12–16 FPS摄像头试穿 webcam try-on

数字均为论文报告；性能测试使用单张 RTX 4090。这里的低成本主要指手机采集和免用相机阵列，并不表示普通手机或入门级 GPU 已达到上述速度。

All numbers are reported by the paper; performance was measured on one RTX 4090. Low cost primarily refers to phone-based capture without a camera array, rather than demonstrated operation at these speeds on a phone or entry-level GPU.

研究动机

Motivation

多层试穿有两种不同难题：衣服要随身体运动，内衣又不能从外衣中露出来。精细的布料碰撞求解开销较高；简单仿真虽快，却会留下相交。即使网格没有相交，偏离表面的 Gaussian 粒子仍可能在错误的层上显露。

Layered try-on has two distinct challenges: garments must move with the body, and inner garments must not show through outer ones. Detailed cloth collision handling is expensive; a simple simulation is fast but leaves intersections. Even without mesh intersections, Gaussian particles extending away from the surface can appear on the wrong layer.

论文把已有方案分为：整体建模的 clothed avatars，难以独立换衣；依赖多相机动态采集的分层表示，采集成本高；以及只靠身体蒙皮或几何约束驱动的服装，难以表现外套滑落等运动。这里的目标是在同一系统中结合独立服装采集、多层混搭、物理运动和交互显示。此定位来自原论文的相关工作讨论，不是对截至今天所有方法的最新排名。

The paper distinguishes whole-body clothed avatars that do not easily support independent garment replacement, layered representations requiring costly multi-camera motion capture, and garments driven mainly by body skinning or geometric constraints that struggle with motions such as a vest sliding off. Its goal is to combine independent capture, layered mix-and-match outfits, physical motion, and interactive display. This positioning follows the paper’s related-work discussion, rather than a current ranking of all methods.

核心假设：用户事先指定内外层顺序；仿真只需大体合理；剩余、局部的视觉穿透可以在显示时修正。适用目标是令人信服的交互预览，论文没有验证精确尺码、接触力或无穿透几何。

Core assumptions: the user supplies the inner-to-outer order, the simulation remains broadly plausible, and residual local visual penetrations can be corrected at display time. The demonstrated target is convincing interactive preview; the paper does not validate accurate sizing, contact forces, or intersection-free geometry.

核心方法

Method

Original Figure 2: capture, simulation, visibility processing and Gaussian rendering — 原论文图 2：指定服装及层序 → 快速网格仿真 → 多视点可见性修正 → Gaussian 渲染。中间的穿透可以保留在仿真状态中，显示阶段选择哪些面片绑定的粒子可见。

术语速查

Terms grounded in this task

术语 Term	在本文中具体是什么 Concrete meaning here
Gaussian / splat	有位置、大小、方向、不透明度及颜色参数的三维粒子；大量粒子重建衣料外观。 A 3D particle with position, size, orientation, opacity, and color parameters; many particles reconstruct garment appearance.
Simulation mesh	每件衣服约 1,000–2,500 个顶点的粗三角网格，用于运动与碰撞近似。 A coarse triangle mesh of about 1,000–2,500 vertices per garment, used for motion and approximate collisions.
Skinning triangle	某个 Gaussian 所绑定的仿真面片；其变形驱动该粒子，处理后的可见性决定是否显示。 The simulation triangle associated with a Gaussian; its deformation drives the particle, and its processed visibility controls display.
Depth peel	每个像素沿视线遇到的第 1、第 2 等层三角形记录，不是不同相机拍到的照片。 A record of the first, second, and subsequent triangles encountered along each pixel’s ray, rather than photographs from different cameras.
Intra-view / inter-view	先在一个视点发现层序错误，再把可疑三角形信息传播到其他视点。 First detect ordering violations in one view, then propagate the implicated triangles to other views.
SH	Spherical harmonics，用于视角相关颜色；布料转动时，利用局部旋转调整求值方向。 Spherical harmonics for view-dependent color; the evaluation direction is adjusted using local rotation as the cloth turns.

1. 离线采集：外观和仿真网格要一起准备

1. Offline capture: prepare appearance and simulation geometry together

把单件衣服穿在静止人台上，用一部手机绕拍，抽取 200–300 帧，再用 Segment Anything 分割为黑色背景。这里的“单目”指一部移动相机获取多个角度，而非从一张照片直接恢复整件衣服。

Place one garment on a stationary mannequin, film it from multiple angles with a phone, sample 200–300 frames, and segment the garment onto a black background with Segment Anything. “Monocular” means one moving camera captures multiple angles, not that a complete garment is recovered from one photograph.

先拟合 Gaussian：在 L1、SSIM 外加入尺度各向异性与偏离中位尺寸的惩罚，得到较均匀的初始外观点集。
Fit initial Gaussians: supplement L1 and SSIM with penalties on scale anisotropy and deviation from median size to obtain a more uniform initial appearance representation.
再拟合网格：把裁剪后的 SMPL-X 模板拟合到 Gaussian；Chamfer 距离负责贴近衣服，面积、角度、法线和边长正则负责网格质量。
Fit a mesh: fit a cropped SMPL-X template to the Gaussians; Chamfer distance attracts it to the garment, while area, angle, normal, and edge-length regularizers maintain mesh quality.
最后重新训练外观：在网格表面用 Poisson-disc sampling 初始化新的 Gaussian，并约束其离表面距离、不透明度和尺寸。训练时只显示最近网格面片可见的 Gaussian，使每片衣料学会独立覆盖自身区域；否则正面可能依赖背面粒子补色，运行时剔除背面后就会露出斑点。
Reoptimize appearance: initialize new Gaussians on the mesh with Poisson-disc sampling and regularize surface distance, opacity, and size. During training, render a Gaussian only when its nearest mesh triangle is visible, so each piece learns to cover its own region. Otherwise the front may borrow color from back-side particles, leaving blotches when those particles are culled at runtime.

重建仍包含半自动清理：过滤背景色粒子，并在 Blender 中手工去除残留。作者不强制所有 Gaussian 都贴平表面，以保留毛绒背心的体积细节；这也保留了粒子超出网格边界的风险。

Reconstruction still includes semi-automated cleanup: filtering background-colored particles and manually removing leftovers in Blender. The authors do not force every Gaussian to lie flat on the surface, preserving volumetric detail such as fur; this also retains the risk of particles extending beyond mesh boundaries.

2. 在线仿真：允许不完美的碰撞处理

2. Online simulation: permit imperfect collision handling

系统用 NVIDIA Warp 中的 XPBD 模拟布料，以身体骨架间的胶囊体近似人体碰撞；衣服间用外层粒子到最近内层三角形的单侧弹簧约束，并加入简单摩擦。步长为 1/40 秒，每步 30 个子步。它处理碰撞，但不保证彻底消除相交；1/40 秒是仿真步长，不是系统达到 40 FPS 的证据。

The system simulates cloth with XPBD in NVIDIA Warp, approximating body collisions with capsules between skeleton joints. Cloth–cloth contacts use one-sided spring constraints from outer-layer particles to nearest inner-layer triangles, with simple friction. The time step is 1/40 second with 30 substeps. Collision handling is present but does not guarantee removal of all intersections; the 1/40-second simulation step is not evidence of 40 FPS system performance.

3. 关键步骤：多个视点共同决定哪些面片该隐藏

3. The key step: let multiple views decide which triangles to hide

准备深度层：用 17 个正交虚拟视点，以 512 × 512 分辨率同时光栅化身体和衣服；每个视点保存 5 层三角形 ID、深度和法线。
Prepare depth layers: rasterize the body and garments from 17 orthographic virtual viewpoints at 512 × 512 resolution, retaining five layers of triangle IDs, depths, and normals for each view.
视点内检测：逐层把当前最前面片与更深面片比较。只有层序错误、深度足够接近、且满足朝向条件时，才标记穿透；实现使用 5 cm 深度阈值和相对相机方向 70° 内的法线限制。不能仅凭“先看见内衣”就判断错误，例如领口本就应该露出内层。
Detect within a view: compare the current front triangle with deeper ones. Mark penetration only when ordering is wrong, depths are sufficiently close, and facing conditions hold; the implementation uses a 5 cm depth threshold and a normal-facing limit of 70°. Seeing an inner garment first is not sufficient evidence of an error: openings can legitimately expose it.
跨视点传播：把标记的三角形重新光栅化为候选掩码；在其他视点的顶层中找到它们后，以当前处理的更深层替换对应记录。侧面的穿透可能不在正面视线中同时遇到内外层，但另一台虚拟相机可以发现它。
Propagate across views: rerasterize marked triangles into candidate masks; when they appear in another view’s top layer, replace the corresponding records with the deeper layer currently being processed. A front-facing ray may not encounter both surfaces of a sideways penetration, but another virtual camera can detect it.
输出可见集合：遍历各个深度层，最后仅渲染绑定到处理后可见三角形的 Gaussian。简单历史滤波保留最近 m 帧中可见的面片，以缓解低分辨率判断的不稳定；正文未给出 m 的具体数值。
Produce the visible set: iterate over the depth layers, then render only Gaussians bound to triangles visible after processing. A simple history filter retains triangles visible in any of the last m frames to reduce low-resolution instability; the text does not specify a numerical value for m.

这不是从 17 个视点重新生成 17 份服装，也不是对最终 RGB 图做补画。各视点贡献三角形可见性判断；被隐藏的面片不会因此从仿真中删除，也不会自动移回正确位置。

This does not generate 17 separate garments or repaint the final RGB image. The views contribute triangle-visibility decisions; hiding a triangle neither deletes it from the simulation nor automatically moves it back to a correct position.

4. 粒子如何随衣服运动

4. How the particles follow the garment

每个三角形通过隐式四面体计算局部变形梯度 F，Gaussian 中心更新为 x = X₀ + F r，其中 r 是静止状态下相对于顶点的偏移。系统还对 F 做 SVD，提取旋转以调整 SH 的观察方向。这解释了为什么启用 SH 会增加开销，也解释了为什么它能随布料转动改变外观，却不能由此恢复未知的新褶皱或新光照。

Each triangle uses an implicit tetrahedron to compute a local deformation gradient F. A Gaussian center updates as x = X₀ + F r, where r is its rest-state offset from a vertex. The system also applies SVD to F to extract rotation and adjust the SH viewing direction. This explains both the added cost of SH and why appearance can rotate with the cloth without recovering unknown new wrinkles or new illumination.

最关键的一句话：让训练时的外观覆盖方式匹配运行时的面片剔除，再以多视点层序检测掩盖快速仿真的可见错误。

The central mechanism: train appearance to remain complete under triangle culling, then use multi-view ordering checks to conceal visible errors from fast simulation.

主要实验结果

Results

实验范围与读数方式

Experimental scope and how to read the numbers

作者采集 7 件衣服，包括 T-shirt、running singlet、hoodie 和 fur vest，使用 Samsung S10 / S24 与男装 L 码人台，展示 1–3 层组合，并用 AMASS 动作驱动。测试机为 Windows 10、Intel Core i9-14900KF、64 GB RAM 和 RTX 4090 24 GB；摄像头演示使用 1080P Logitech Brio 100。单件服装有约 200k–500k Gaussian；身体另有约 1.2M。

The authors capture seven garments, including T-shirts, a running singlet, a hoodie, and a fur vest, using Samsung S10 / S24 phones and a men’s size-L mannequin. They demonstrate one to three garment layers driven by AMASS motions. Tests use Windows 10, an Intel Core i9-14900KF, 64 GB RAM, and an RTX 4090 with 24 GB memory; the webcam demonstration uses a 1080P Logitech Brio 100. Each garment has about 200k–500k Gaussians, and the body adds about 1.2M.

原论文表 1 的 FPS。B = “Bad Books”，P = “Pigeons”，S = “Singlet”，按表中顺序组合；SH 表示 spherical harmonics。Naïve 渲染全部粒子，速度更高但保留可见穿透。

FPS from original Table 1. B = “Bad Books”, P = “Pigeons”, S = “Singlet”, combined in the listed order; SH denotes spherical harmonics. Naïve renders all particles and is faster, but retains visible penetrations.

服装 Garments	本文 / SH 开 Ours / SH on	Naïve / SH 开 Naïve / SH on	本文 / SH 关 Ours / SH off	Naïve / SH 关 Naïve / SH off
B	21	31	24	45
B, P	19	26	21	39
B, P, S	18	24	20	34

三层、启用 SH 时，本文为 18 FPS，Naïve 为 24 FPS，即吞吐率降低 25%（由表 1 计算）；这笔成本换取层间可见性修正。该表不是与另一完整试穿系统的公平速度竞赛，也没有逐模块耗时。剔除不可见面片后，SH 的 SVD 计算也可减少，因此启用 SH 时相对 Naïve 的额外开销较小。

With three garments and SH enabled, the method reaches 18 FPS versus 24 FPS for Naïve, a 25% throughput reduction calculated from Table 1; that cost buys visibility correction between layers. This is not a speed comparison against another complete try-on system, and the table provides no per-module timings. Culling invisible triangles also avoids their SH-related SVD work, reducing the relative overhead versus Naïve when SH is enabled.

摄像头试穿降到 12–16 FPS：ExPose 在另一线程估计姿态并更新 SMPL-X，系统把可见服装合成到实时视频中；论文指出线程同步是瓶颈，跟踪误差还会造成遮挡错误。离线第二阶段的可见性增强 Gaussian 优化每件需 20–30 分钟，对照普通 3DGS 优化的 5–8 分钟；这不包含一个已量化的完整采集到可用资产总耗时。

Webcam try-on reaches 12–16 FPS: ExPose estimates pose in another thread and updates SMPL-X, while visible garments are composited onto the live video. The paper identifies thread synchronization as a bottleneck, and tracking errors also cause occlusion artifacts. Offline visibility-augmented Gaussian optimization takes 20–30 minutes per garment, versus 5–8 minutes for standard 3DGS optimization; this is not a measured total time from capture to a finished usable asset.

消融：究竟哪些环节有证据支持

Ablations: which components have supporting evidence

Original Figure 10: Naive, single-view processing, single-view visibility, full method — 原论文图 10，从左到右：渲染全部粒子、仅单视点处理、单视点可见性但不处理层序、完整方法。完整方法减少了腹部及肩部穿透；这是定性图像证据。

改动 Change	观察与含义 Observation and implication
只用单视点 Use one view	仍有侧面和朝向阈值相关错误；多视点是解决视线盲区的关键。 Side and facing-threshold errors remain; multiple views address visibility blind spots.
移除训练中的网格可见性条件 Remove mesh-visibility conditioning in training	运行时只保留可见面片后出现斑块；训练必须适应后续剔除（图 11）。 Blotches appear when runtime keeps only visible triangles; training must accommodate later culling (Figure 11).
省略第二次 Gaussian 优化 Skip the second Gaussian optimization	表面覆盖不足、黑背景透出；第二阶段不只是细化颜色（图 12）。 Incomplete surface coverage exposes the black background; the second pass does more than refine color (Figure 12).
改变虚拟相机数量 Change the number of virtual cameras	5 / 9 个视点比 17 个伪影多；33 个没有明显额外收益。这是定性比较，不是全局最优证明（图 13）。 Five or nine views show more artifacts than 17; 33 provide no obvious further benefit. This is qualitative evidence, not proof of a global optimum (Figure 13).
关闭衣服间碰撞 Disable cloth–cloth collision handling	在深度容差内仍能改善画面，但变形差异过大时失败；渲染修正不能完全替代合理仿真（图 9）。 Images still improve within the depth tolerance, but widely diverging deformations can fail; rendering correction cannot fully replace plausible simulation (Figure 9).

证据边界：论文主要提供定性结果、模块消融和 FPS，没有报告 PSNR / SSIM 质量对比表、用户偏好研究、碰撞率或系统性尺码泛化评估。不能据此断言其视觉质量量化优于所有先前方法。正文对图 10 的 A–D 引用和图注不一致；本研究遵循图中标签及图注，避免把方法对应错位。

Evidence boundary: the paper primarily provides qualitative results, component ablations, and FPS. It does not report a PSNR / SSIM quality comparison table, user preference study, collision rate, or systematic sizing-generalization evaluation. These results do not establish quantitative visual superiority over all earlier methods. The prose references to A–D in Figure 10 disagree with the figure caption; this study follows the labels in the figure and its caption to avoid misidentifying methods.

局限与展望

Limitations and outlook

Original Figure 14: large penetrations, restricted visibility, and boundary-triangle artifacts — 原论文图 14：黄色为较大穿透，蓝色为可见性受限，红色为较大的边界三角形。方法明显改善画面，但这些位置仍可能出现伪影。

三角形粒度有两难：隐藏整个面片会误伤其本应可见的部分；面片过小又可能在低分辨率光栅化中消失。粒子本身具有体积和透明度，面片可见不等于每个像素都应显示它。
Triangle granularity is a tradeoff: hiding a whole triangle can erase a portion that should remain visible, while very small triangles may disappear in low-resolution rasterization. Particles have extent and opacity, so triangle visibility does not mean every pixel of a particle should be displayed.
层序与穿透幅度有前提：衣服顺序由用户预先固定，仿真中不能自由改变；大幅穿透或不同层变形严重分离可能超出深度阈值，仍需合理的基础仿真。
Ordering and penetration size matter: garment order is fixed by the user rather than changing freely during simulation. Severe penetration or strongly diverging layer deformations can exceed the depth tolerance, so a plausible underlying simulation remains necessary.
静态采集有缺失：没有衣服内部外观，翻面时会露馅；细褶皱部分烘焙在粒子外观中。旋转 SH 不等于动态重建细褶皱或完整重打光。
Static capture leaves missing information: garment interiors are not captured, producing artifacts when cloth flips over, and fine wrinkles are partly baked into particle appearance. Rotating SH does not reconstruct evolving fine wrinkles or provide full relighting.
系统成本与适用范围：手机采集降低了设备门槛，但仍需人台、多角度视频、分割、网格拟合、手工清理和较强 GPU。少量服装与 1–3 层演示尚不足以验证任意体型、任意衣料或更多层的稳定性能。
System cost and scope: phone capture lowers the capture-hardware barrier, but the process still requires a mannequin, multi-angle video, segmentation, mesh fitting, manual cleanup, and a powerful GPU. A small garment set and demonstrations with one to three layers do not establish stable performance for arbitrary body shapes, materials, or additional layers.

批判性定位

Critical positioning

怀疑者会说：这是一套用可见性遮盖仿真错误的工程系统，并没有解决多层布料的几何解缠。这一判断准确，但仍需看到其具体贡献：在已知层序下，用多视点深度剥离传播面片剔除信息，并让 Gaussian 训练适应这种剔除，从而把“廉价近似运动”和“完整外观覆盖”接起来。单纯使用 3DGS 或 XPBD 本身不是这里的新意。

A skeptic’s reading: this is an engineering system that conceals simulation errors through visibility processing, rather than solving geometric untangling for layered cloth. That reading is accurate, but the concrete contribution is still meaningful: under known layer ordering, multi-view depth peeling propagates triangle-culling decisions, while Gaussian training accommodates that culling, connecting cheap approximate motion with complete appearance coverage. Using 3DGS or XPBD alone is not the novelty.

作者提出的方向包括更细粒度的重心坐标可见性处理、优化相机位置，以及考虑光照或环境遮蔽。进一步把检测结果反馈给物理状态也是设想，尚未验证。作为后续评估建议，可测量时序闪烁、视点变化一致性、不同 GPU 的延迟，以及不同体型和层数下的失败率；这些是本研究提出的检验项，不是论文已有结果。

The authors suggest finer visibility handling with barycentric coordinates, optimized camera placement, and lighting- or ambient-occlusion-aware appearance. Feeding detection results back into the physical state is also proposed but unvalidated. Useful further evaluations would measure temporal flicker, consistency as the viewpoint changes, latency on different GPUs, and failure rates across body shapes and layer counts; these are assessment suggestions from this study, not results already reported by the paper.

阅读与复现时优先核对什么 What to check first when reading or reproducing the method

先核对网格与 Gaussian 的绑定及训练时可见性条件，再检查层序 ID、深度符号、法线朝向和单位。随后分别关闭跨视点传播与历史滤波，观察侧面错误和时序稳定性。要复现实验，还需要补齐正文未量化的历史长度、完整损失权重、捕获标定及完整预处理耗时；本文不把这些缺项替作者猜成确定配置。

First check mesh–Gaussian correspondence and visibility conditioning during training, then layer-order IDs, depth sign, normal direction, and units. Next disable cross-view propagation and temporal filtering separately to inspect side artifacts and temporal stability. Reproduction also requires details not quantified in the text, including the history length, complete loss weights, capture calibration, and total preprocessing time; this study does not invent definite settings for those omissions.

来源与版本

Sources and version

技术机制、数字及图像均来自用户提供的 11 页作者托管 PDF，尤其是 §3、§4、§5、表 1–2 及图 1–14。I-Chao Shen 的主页将其列为 Graphics Interface (GI) 2025，并标注 Best Graphics Paper。

Technical mechanisms, numbers, and figures come from the user-provided 11-page author-hosted PDF, especially Sections 3–5, Tables 1–2, and Figures 1–14. I-Chao Shen’s homepage lists the work at Graphics Interface (GI) 2025 and marks it Best Graphics Paper.

版本注意：PDF 元数据显示创建于 2025-04-23，但创建日期不等于正式发表日；文内仍有 Conference’17 和占位 DOI，不能把它们当成真实会议信息。研究日期记录的是本页面制作时间。未运行作者系统，所有速度均为作者报告；上述定量推导仅为 18 / 24 对应的 25% 降幅。

Version note: PDF metadata gives a creation date of 2025-04-23, which is not necessarily the formal publication date. The file still contains a Conference’17 placeholder and a placeholder DOI; these are not treated as actual venue metadata. The study date records creation of this page. The authors’ system was not run, so all speeds are author-reported; the quantitative calculation above is only the 25% reduction implied by 18 / 24.