A Method for Animating Children’s Drawings of the Human Figure
机构按论文署名列出；Qingyuan Zheng、Yifei Li 与 Jessica K. Hodgins 在参与本工作时隶属 Meta AI Research。
Affiliations follow the paper’s byline; Qingyuan Zheng, Yifei Li, and Jessica K. Hodgins were affiliated with Meta AI Research while contributing to this work.
把画中的人物剪成纸偶，装上骨架，再用真人动作记录来牵动它。为了看清动作，手臂和双腿可以分别从正面或侧面展示，而不必服从同一台相机。
Cut the drawn character out like a paper puppet, attach a skeleton, and drive it with recorded human motion. To keep the action readable, show the arms and legs from separate front or side views instead of forcing one camera view.
研究动机：让一张画成为可操纵的角色
Motivation: turn one drawing into a controllable character
输入是一张纸上绘画的普通照片，输出是保留原画纹理的二维动画。儿童无需重复画多个姿势，也无需自己建模或制作逐帧动画。问题在于，手臂可能只是一条线，头和躯干可能合为一体，身体内部可能和纸张一样空白；直接套用真实人体的视觉模型或三维比例假设会失败。
The input is an ordinary photograph of a drawing on paper; the output is a 2D animation retaining the drawing’s texture. Children need not redraw the character in several poses, build a model, or animate frame by frame. An arm may be one line, the head and torso may merge, and the body interior may look exactly like the paper. Models and 3D proportion assumptions designed for real people therefore transfer poorly.
相比需要用户标注、多个角色视图或明确三维模型的既有方法，本系统组合自动预测和简单修改界面。关键洞见是让动画服从绘画的表现习惯：儿童画常常避免透视缩短，并把不同身体部位画成最容易辨认的角度，因此动作也可以混合视角。§§2–3
Compared with previous approaches requiring user annotations, multiple character views, or an explicit 3D model, this system combines automatic predictions with simple correction tools. The key insight is to make the animation follow the drawing’s visual conventions: children often avoid foreshortening and depict different body parts from their most recognizable angles, so the motion can also mix viewpoints. §§2–3
核心方法：检测、剪出、绑定、驱动
Method: detect, cut out, rig, and animate
术语速查：每个抽象词具体指什么
Terminology: what each term means here
| 术语 Term | 在本文中的具体对象 Concrete object in this paper |
|---|---|
| 分割掩码 Segmentation mask | 区分人物与背景的二值图，最终保留一个实心连通轮廓。 A binary foreground map reduced to one solid connected silhouette. |
| 姿态 Pose | 图像中 17 个预测关键点的位置；动画使用其中的鼻子和主要四肢关节。 17 predicted image keypoints; animation uses the nose and main limb joints. |
| 角色绑定 Character rig | 带原画纹理的三角网格，加上能控制网格的关节和骨骼。 A triangulated mesh textured with the drawing, plus joints and bones controlling it. |
| 动作重定向 Motion retargeting | 把真人动作的二维骨骼方向转移到比例不同的纸偶上。 Transferring projected human bone directions to a differently proportioned puppet. |
| Twisted perspective | 上肢和下肢分别选择正面或侧面投影，而非恢复一个统一的三维人物。 Choosing frontal or sagittal projections separately for upper and lower limbs rather than recovering one coherent 3D person. |
| 适合动画的预测 Animation-ready prediction | 人工检查认为无需修改即可用于动画的检测框、掩码或关节预测。 A box, mask, or joint prediction visually judged usable for animation without correction. |
1. 先找到人物，再得到可变形的完整轮廓
1. Find the character and extract a deformable silhouette
人物检测采用 MS-COCO 预训练的 Mask R-CNN，骨干为 ResNet-50 + FPN；固定骨干，微调只预测“人形角色”的检测头。检测框裁剪减少纸张外部和场景背景的干扰。与检测框相比，动画掩码对细节更苛刻：少掉一只脚，或把手臂粘到躯干上，都可能破坏后续运动。
Figure detection uses MS-COCO-pretrained Mask R-CNN with ResNet-50 + FPN, freezing the backbone and fine-tuning a head for one human-figure class. Cropping limits interference from the surrounding scene and surfaces outside the paper. Animation masks impose stricter requirements than boxes: losing a foot or joining an arm to the torso can break subsequent movement.
最终管线的分割采用图像处理：裁剪图缩放到宽 400 像素 → 灰度化与自适应阈值 → 3×3 核闭运算和膨胀 → 从边界泛洪填充以填实闭合轮廓 → 只保留最大前景多边形。这样可保留细线肢体并填满空心人物，但未闭合轮廓、分离肢体、阴影或横格纸仍会导致错误。§§3.1–3.2
The deployed segmentation pipeline uses image processing: resize the crop to width 400 pixels → grayscale and adaptive thresholding → closing and dilation with 3×3 kernels → flood fill from the boundary to fill enclosed regions → retain the largest foreground polygon. This can preserve thin limbs and fill hollow figures, but open outlines, detached limbs, shadows, and ruled paper still cause failures. §§3.1–3.2
2. 预测关节，把像素变成二维纸偶
2. Predict joints and turn pixels into a 2D puppet
姿态估计使用 ImageNet 预训练 ResNet-50 与 top-down heatmap head；输入裁剪图为 192×256，每个热图的最大值给出一个关键点。虽然预测 17 个 MS-COCO 关键点，绑定只使用鼻子以及肩、肘、腕、髋、膝、踝；双髋中点是根关节，双肩中点是胸部关节。
Pose estimation uses an ImageNet-pretrained ResNet-50 and a top-down heatmap head. A 192×256 crop produces heatmaps whose maxima locate the keypoints. Although 17 MS-COCO keypoints are predicted, the rig uses the nose and shoulders, elbows, wrists, hips, knees, and ankles; hip and shoulder midpoints become the root and chest joints.
对掩码做 Delaunay 三角剖分并贴上原图纹理。按三角形中心离哪根骨骼最近，将网格分到 9 个身体区域；绘制顺序可制造四肢在身体前后穿过的效果。移动关节后，用 ARAP（as-rigid-as-possible）形变更新网格，尽量避免局部形状被任意拉扯。这仍是二维纹理网格，不是三维身体重建。§§3.3–3.4
Delaunay triangulation converts the mask into a mesh textured with the original image. Each triangle is assigned to one of nine body regions using the nearest bone to its centroid; rendering order can suggest limbs passing in front of or behind the body. Moving joints drives ARAP (as-rigid-as-possible) deformation, which discourages arbitrary local distortion. This remains a textured 2D mesh, not a reconstruction of a 3D body. §§3.3–3.4
3. 从动作中选择视角，而不是强迫全身使用一个视角
3. Choose viewpoints from the motion instead of imposing one view
- 归一化动作。对预选真人动作捕捉片段逐帧去掉根节点在地面上的平移，并绕竖直轴统一朝向；之后单独把根位移按腿长比例缩放后加回角色。
- Normalize the motion. For each frame of a selected human motion-capture clip, remove ground-plane root translation and normalize heading about the vertical axis; root displacement is later scaled using leg lengths and reapplied separately.
- 按部位选择投影。分别收集整个片段的上肢、下肢三维关节位置，对各自点云做 PCA。前两个主成分描述变化最大的平面，第三个给出法向；再依据法向相似度，从人体正面平面与矢状平面中选择一个。这是每个片段、每组肢体的选择，不是每帧自由切换摄像机。
- Select projections by body group. Collect upper- and lower-limb 3D joint positions across the clip and apply PCA separately. The first two components describe the plane of greatest positional variation and the third its normal; normal similarity then selects the frontal or sagittal body plane. Selection is per clip and limb group, not an unrestricted camera change every frame.
- 转移方向，保留长度。投影后计算上臂、前臂、大腿、小腿、颈部和脊柱骨骼的全局方向，再旋转纸偶对应骨骼。纸偶骨长保持不变，因此避免引入原画少见的透视缩短；新的关节位置成为 ARAP 的控制点。
- Transfer directions and retain lengths. Compute projected global bone orientations for upper arms, forearms, thighs, lower legs, neck, and spine, then rotate the corresponding puppet bones. Puppet bone lengths stay fixed, avoiding foreshortening uncommon in the drawings; the new joint positions become ARAP handles.
展开：方向转移的数学直觉
Expand: the geometry of direction transfer
以下是对方向转移的解释性写法，不是论文新增的训练损失。令三维关节为 q，选定二维投影为 P，纸偶对应骨骼长度为 L。投影只提供单位方向，长度来自纸偶自身：
The following is an explanatory formulation of direction transfer, not a new training loss from the paper. Let q denote 3D joints, P the selected 2D projection, and L the puppet’s bone length. Projection supplies a unit direction; length comes from the puppet:
\[d_{ij}=P(q_j-q_i),\qquad x_j=x_i+L_{ij}\frac{d_{ij}}{\lVert d_{ij}\rVert}.\]
只在投影方向非零时，上式才有定义。直觉上，正面投影可能看不清前后摆动的腿，而侧面投影可能压扁向两侧张开的手臂；分别选投影能保留这两种动作。PCA 最大化位置变化并不等价于最大化可辨认性，这是方法的启发式假设。
The expression is defined only when the projected direction is nonzero. Intuitively, a front view can hide legs moving forward and backward, while a side view can collapse arms opening sideways; separate projections can preserve both motions. Maximizing positional variation with PCA is not equivalent to maximizing recognizability: that connection is a heuristic assumption.
4. 自动预测之外，允许用户逐步修正
4. Let users correct each stage
界面依次允许用户拖动检测框、画入或擦除掩码、移动关节，最后从固定动作库中选择动作。用户不能增加或删除关节；论文让用户把不需要的关节拖离人物。无需修改时，作者报告在其 AWS 部署上整个用户流程少于 10 秒；这不是包含人工修改时间的通用速度保证。§3.5
The interface lets users resize the box, paint or erase the mask, and move joints before choosing from a fixed motion gallery. Users cannot add or delete joints; the paper instructs them to move unwanted joints away from the figure. With no corrections, the authors report a complete user flow under 10 seconds on their AWS deployment; this is not a general speed guarantee including manual correction time. §3.5
最关键的一句话：把三维动作转换成符合二维画风的骨骼方向，再让原画网格跟着这些方向变形。
The essential mechanism: convert 3D motion into bone directions compatible with the drawing’s 2D style, then deform the original textured mesh to follow them.
主要实验结果：精度高，不等于无需修改
Results: high accuracy does not imply correction-free animation
评估分成三个不同问题
Three distinct evaluation questions
mAP 实验从人工筛选的 2,500 个干净样本中留出 500 个验证，其余 2,000 个构成不同大小的训练集。另一个实验对额外 571 张适合动画的上传图做人工检查，判断每阶段预测是否无需修改即可用于动画。该实验没有使用这些图的用户标注作为真值；它还评估了用全部 2,500 个干净样本训练的模型。因此不能把 mAP 表中的 2,000 样本结果误标为 2,500 样本结果。
The mAP experiment reserves 500 of 2,500 manually screened clean examples for validation, drawing training sets of different sizes from the remaining 2,000. A separate experiment visually inspects predictions on 571 additional uploaded images considered suitable for animation, judging whether each stage needs correction. It does not use those images’ user annotations as ground truth and additionally evaluates models trained on all 2,500 clean examples. The 2,000-sample mAP results must therefore not be relabeled as 2,500-sample results.
第三个问题是“混合视角是否更讨喜”，通过对已经成功绑定的角色做偏好实验回答。它与前处理成功率、用户访问量都不是同一指标。下面的数值均来自 v2 的表 1–3。§4
The third question is whether mixed perspectives look more appealing, tested using characters that were already successfully rigged. This is separate from preprocessing success and public usage counts. All numerical results below come from Tables 1–3 in v2. §4
训练数据：少量领域样本很有用，但分割仍然最难
Training data: domain examples help, but segmentation remains hardest
| 训练集 Training set | 框 mAP Box mAP | 掩码 mAP Mask mAP | 姿态 mAP Pose mAP |
|---|---|---|---|
| 0，未微调 0, no fine-tuning | 0.06 | 0.04 | 0.09 |
| 10 | 0.27 | 0.30 | 0.34 |
| 100 | 0.51 | 0.51 | 0.76 |
| 250 | 0.58 | 0.57 | 0.80 |
| 500 | 0.69 | 0.63 | 0.82 |
| 1,000 | 0.77 | 0.68 | 0.84 |
| 1,500 | 0.80 | 0.70 | 0.85 |
| 2,000 | 0.81 | 0.71 | 0.85 |
| 177,666，有噪声 177,666, noisy | 0.82 | 0.49 | 0.90 |
表 1 的 mAP 使用 [.5:.95] 阈值范围；掩码列评估的是 Mask R-CNN，不是图像处理方法。大量有噪声样本使姿态 mAP 达到 0.90，但分割 mAP 从 2,000 个干净样本的 0.71 降到 0.49。作者认为掩码修改更费事，导致接受的标注中残留更多错误；这是解释，不是单独受控验证的因果结论。
Table 1 reports mAP over [.5:.95]; the mask column evaluates Mask R-CNN, not image processing. Many noisy examples raise pose mAP to 0.90, but mask mAP falls from 0.71 with 2,000 clean examples to 0.49. The authors suggest that tedious mask correction leaves more errors in accepted annotations; this is an explanation, not a separately controlled causal finding.
| 训练集 Training set | 框成功 % Box success % | 姿态成功 % Pose success % | 全阶段：模型掩码 % All stages: model mask % | 全阶段：图像处理 % All stages: image processing % |
|---|---|---|---|---|
| 100 | 60.9 | 54.1 | 4.9 | 19.4 |
| 2,000 | 91.8 | 89.5 | 21.2 | 38.9 |
| 2,500 | 92.5 | 90.2 | 23.3 | 39.4 |
| 177,666，有噪声 177,666, noisy | 92.5 | 94.6 | 16.1 | 40.6 |
真正限制自动化的是掩码。在同一 571 张测试图上，图像处理分割成功率为 42.4%，高于用 2,500 个干净样本训练的 Mask R-CNN 的 24.7%，也高于有噪声大数据训练的 16.1%。采用图像处理时，全阶段成功率分别为 39.4% 和 40.6%。这些是三个阶段同时成功的实际统计，不是三个边际成功率的乘积，也不是任何绘画都能成功的概率。
The mask is the practical automation bottleneck. On the same 571 test images, image-processing segmentation succeeds on 42.4%, versus 24.7% for Mask R-CNN trained on 2,500 clean examples and 16.1% with the large noisy set. With image processing, all-stage success reaches 39.4% and 40.6%, respectively. These are observed joint successes across three stages, not products of marginal rates or probabilities of success on arbitrary drawings.
感知实验：混合视角通常更受欢迎，但有边界
Perceptual study: mixed views usually help, within a limited scope
66 位 Mechanical Turk 参与者比较 4 个角色 × 5 个动作，共 20 组并排视频；混合视角左右位置随机，问题是强制二选一的“动作更有吸引力”，并加入注意力检查。16/20 组对混合视角有显著偏好，另外 4 组没有显著差异；没有显著偏好单一视角的组合。
66 Mechanical Turk participants compared four characters performing five motions, for 20 side-by-side video pairs. The mixed-perspective side was randomized; participants made a forced choice about motion appeal, with attention checks included. Mixed perspective was significantly preferred in 16/20 pairs; four showed no significant difference, and none significantly favored the single-view condition.
例如，Walk & Clap 的四个角色分别有 76%、78%、85%、78% 的人偏好混合视角，均 p<0.01。Wave Hello 对应为 49%、73%、49%、46%，只有第二个角色显著（p<0.01）。作者解释：挥手时腿部运动很少，两个版本差别更小。此结果支持特定角色和动作的吸引力，不能推出所有三维动作都能良好展平。
For Walk & Clap, mixed-view preferences were 76%, 78%, 85%, and 78% across the four characters, all p<0.01. Wave Hello yielded 49%, 73%, 49%, and 46%, with only the second character significant (p<0.01). The authors attribute the weaker effect to limited leg movement, making the two versions similar. These results support appeal for the tested characters and motions, not successful flattening of every 3D action.
展开：实验能支持什么、没有支持什么
Expand: what the experiments establish and leave open
数据量实验固定模型结构和超参数，分割替换与视角对比也提供了具体消融证据；但论文没有对 ARAP、骨长保持、PCA 选平面分别给出完整消融。感知实验只覆盖 4 个成功案例角色与 5 个动作，没有证明在前处理失败的角色上仍有相同偏好，也没有测量动作识别准确率。
The data-size experiment fixes architecture and hyperparameters, while segmentation replacement and viewpoint comparisons provide concrete component evidence. However, the paper does not separately ablate ARAP, fixed bone lengths, and PCA plane selection comprehensively. The perceptual study covers only four successfully prepared characters and five motions; it neither establishes the same preference after preprocessing failures nor measures motion-recognition accuracy.
本研究整理未运行模型训练或动画代码；这里重绘的是论文报告的数据，而不是新的复现实验。
This study did not run model training or animation code. Its chart reproduces reported paper data rather than new replication results.
数据集与反馈闭环
Dataset and feedback loop
Amateur Drawings Dataset 包含 178,166 张图，附用户接受的框、掩码和关节。它不是经过专业人员逐项验证的真值集合，也不是已确认全部出自儿童的集合：系统未记录作者年龄。公开演示让用户获得动画，同时在可选研究授权下贡献输入与修正，形成真实领域数据收集闭环。拒绝授权不影响使用。§5
The Amateur Drawings Dataset contains 178,166 images with user-accepted boxes, masks, and joints. These are neither exhaustively expert-verified labels nor confirmed drawings exclusively by children: artist ages were not recorded. The public demo gives users animations while collecting inputs and corrections under optional research consent, creating a source of real domain data. Declining consent did not restrict use. §5
- 截至 2022-04-20，超过 350 万张上传图中约 170 万张取得研究使用授权。
- By 2022-04-20, approximately 1.7 million of over 3.5 million uploaded images had research-use consent.
- 自监督 ResNet-50 特征与 k-means 将图像分为 100 簇，保留 32 个主要包含手绘角色的簇；在嵌入空间去除近重复后剩 471,405 张。
- Self-supervised ResNet-50 features and k-means formed 100 clusters; 32 mainly containing hand-drawn characters were retained. Near-duplicate filtering in embedding space left 471,405 images.
- 人工审查其中 283,146 张，按手绘完整人形、隐私与内容标准筛选，最后保留 178,166 张。
- Reviewers inspected 283,146 of those images for freehand complete humanoid figures, privacy, and content criteria, retaining 178,166.
从随机 5,000 个样本看，用户修改过 35% 的框、20% 的掩码、29% 的骨架。未修改不等于正确，这也解释了为何数据量增加可能无法弥补分割标注噪声。人工审核主要由英语使用者完成，带非英文文字的图被排除，带来明确的语言选择偏差。
In a random sample of 5,000 examples, users had modified 35% of boxes, 20% of masks, and 29% of skeletons. Unmodified does not mean correct, helping explain why more data may not compensate for mask-label noise. Reviewers were primarily English speakers and excluded images containing non-English words, introducing an explicit language selection bias.
影响力应按论文时期理解：演示于 2021-12-16 发布，前 9 个月超过 320 万独立访客上传 670 万张图，生成超过 2,400 万段动画。这证明需求和可用性，不等同于对全部上传图的自动成功率评估。§§4.1, 6
Usage figures refer to the paper’s reporting period: the demo launched on 2021-12-16 and, in its first nine months, received over 3.2 million unique visitors and 6.7 million images, producing over 24 million animations. This demonstrates demand and practical use, not measured automatic success across all uploads. §§4.1, 6
局限与研究判断
Limitations and assessment
- 输入拓扑限制。最大连通轮廓假设会丢失分离身体部件；手放在腰上等肢体接触会产生不该连在一起的网格。强阴影、背景线条和轮廓缺口使分割脆弱。
- Input topology. Keeping one connected silhouette can discard detached body parts. Contacts such as hands on hips can fuse mesh regions that should move separately. Hard shadows, background lines, and open contours make segmentation fragile.
- 运动与画风限制。在三维空间大幅绕转的动作难以压到二维。系统采用固定的人形骨架和预选动作，不能直接推广到任意四足动物或面部表情；真实透视、明显透视缩短的画作也可能与这种动画风格冲突。
- Motion and style. Motions sweeping through all three spatial dimensions are difficult to flatten. A fixed humanoid skeleton and selected motion library do not directly generalize to arbitrary quadrupeds or facial expressions. Drawings with consistent perspective and strong foreshortening may also clash with the animation style.
- 证据边界。571 张测试图已被筛选为适合动画，20 组偏好刺激覆盖范围有限，用户接受的标签也有噪声。论文没有给出面向任意上传图的普适成功率。
- Evidence limits. The 571 test images were screened as suitable for animation, the 20 preference stimuli cover limited cases, and user-accepted labels are noisy. The paper does not establish a universal success rate for arbitrary uploads.
怀疑者会如何概括？这是已有检测、关键点估计、形态学、三角剖分和 ARAP 的系统集成；不是全新的视觉骨干，也不是自动推断角色三维几何的系统。真正有辨识度的动画贡献是按上下肢运动分别选取投影，以混合视角和固定骨长匹配儿童画风，并用感知实验验证其吸引力；另一个重要贡献是通过公开工具收集的大规模真实绘画与标注。
How would a skeptic summarize it? It integrates existing detection, keypoint estimation, morphology, triangulation, and ARAP; it introduces neither a new visual backbone nor automatic recovery of character 3D geometry. The distinctive animation contribution is selecting projections separately for upper- and lower-limb motion, combining viewpoints and fixed bone lengths to match children’s drawing style, with perceptual evidence for its appeal. A second important contribution is large-scale real drawing and annotation collection through a public tool.
下一步的合理方向（研究判断）。最直接的收益可能来自更可靠的身体分区与可动画拓扑，而不仅是提高掩码像素重叠率。将框、关节和轮廓约束结合，用于修正粘连与断肢，是值得检验的方向；论文也建议更好地选择分割方法和理解角色类型。但这些扩展尚未由本文实验实现或验证。§6
A plausible next step (analysis). The most immediate gains may come from reliable body-part separation and animatable topology rather than merely improving pixel overlap. Combining box, joint, and silhouette constraints to repair fused or detached limbs is worth testing; the paper also proposes better segmentation selection and character-type understanding. These extensions were not implemented or validated by its experiments. §6
来源与版本
Sources and version
本页以 arXiv v2（2023-04-04）的全文、表 1–3 与图 1、3、6、8 为技术依据；初稿提交于 2023-03-07。图片来自原论文，已缩放用于网页阅读，点击可查看资源。研究整理日期为 2026-11-12，不是论文发表日期。
This study is based on the full arXiv v2 text (2023-04-04), Tables 1–3, and Figures 1, 3, 6, and 8; v1 was submitted on 2023-03-07. Images are extracted from the paper and resized for web reading; click them to view the assets. The study date, 2026-11-12, is not the paper’s publication date.
代码状态与论文结果分开记录：官方仓库于 2025-09-03 归档，当前为只读；它仍提供论文实现的参考资料，但不能视作仍在维护的服务承诺。此状态来自本次查阅的官方仓库，不改变上面的 2023 年实验结果。
Code status is separate from the paper’s results: the official repository was archived on 2025-09-03 and is read-only. It remains a reference implementation, but is not a promise of an actively maintained service. This status comes from the official repository checked for this study and does not alter the 2023 experimental results.