Contents

Codex effort 四档对比


同一个任务、同一个容器、同一个模型 gpt-6-astra,只改 model_reasoning_effort。 四档全部产出合法 schematic 并自测通过;差别在花多少时间、烧多少 token、以及建筑细节密度。

4/4passed
2.5×xhigh 比 low 的 token
2.5×xhigh 比 low 的耗时
1.70×palette state 增幅

关键观察

按从表面到深刻排序。

  1. 模型价格如下: $0.41, $0.46, $0.66, $1.01
  2. low和medium表现差不多, high和xhigh差不多, 后者耗时x2, output和reasoning正比于代码token长度
  3. 代码结构高度一致, 重名非常多, 震惊
  4. xhigh 会干多余的事, 比如yapf和对齐渲染图背景与参考图背景
  5. 4种测试失败率相当, 但low和medium会有语法错误, high/xhigh则没有, 难不成low/medium掺水了?
  6. effort 买的是细节密度和没有gap
  7. high喜欢看图, 而xhigh喜欢写测试, 两者消耗token比例相当, 说明看图非必须, 可用测试平替达到同样效果。 但奇怪的是所有结构缺陷全是测试检查出来的, 不懂效果是否能归因到图片检测上
  8. 除读API外, 检测和修bug是最耗token的, 考虑static check -> 能compile就没bug
  9. 已知bug:
    • 1/3 token花在了探索环境 -> 可加API速查表和/或调用渲染代码
    • 并且有坐标系不明确的bug
    • 不应要求写REPORT, 而是我们跑完后翻trace看, 以更好统计token消耗
    • 四档都花命令去确认旗帜图案, 展示框, 讲台书在无头渲染器里显示不全, 需要解释
    • 需明确规定seed random

实验设置

容器镜像 mcgen:latest,库栈只读挂载(MCIO + MCRender + 1.21.1 资源), workspace 可写,无 web 浏览,打分容器单独以 harness 用户运行。任务 prompt 就是 task/BRIEF.md 全文, 两张参考图用 codex exec --image= 附到第一轮。

./run_agent.sh --model gpt-6-astra --effort low --timeout 5400 ./run_agent.sh --model gpt-6-astra --effort medium --timeout 5400 --memory 4g --cpus 3 ./run_agent.sh --model gpt-6-astra --effort high --timeout 5400 --memory 6g ./run_agent.sh --model gpt-6-astra --effort xhigh --timeout 5400 --memory 6g
reference exterior
参考图 1:外观
reference interiors
参考图 2:三张内饰

契约

产物固定路径 output.litematic;报告 report.json; agent 可改 task/ 拷贝进来的任何文件;打分只验 schematic 是否可加载 + 环境自检。


对比表

点表头排序。token 为 codex rollout 里 token_usage_record 的实测合计,非估算。

effort耗时 s请求total tokenoutputreasoning 命令失败命令方块尺寸 XYZpalette种类 生成器 KB测试 KB渲染张数
low66727 1,203,71718,9064,281 26410,03145x41x35 745517.07.3 12
medium73528 1,407,61420,2815,216 24510,66145x40x35 795819.06.8 12
high156831 2,061,93631,0207,793 33112,34147x41x37 1017028.813.8 9
xhigh166929 2,443,14049,05117,264 32411,99247x43x37 1267841.77.8 8

四档 pytest 全部退出码 0,stacklinks 自检全部通过, 即没有任何一次尝试改动只读库或替换 symlink。


图表

每张图只画一个量纲。跨量纲的比较放在第一张归一化图里,单位是"相对 low 的倍数"。

Fig 1. 归一化总览,low = 1.0。唯一一张允许放在一起的图,因为单位统一成倍数。 reasoning token 是涨得最猛的一项(4.0×),方块数几乎没动(1.2×)。
Fig 2. 墙钟耗时(秒)。low 与 medium 基本持平,medium 到 high 翻倍,high 到 xhigh 只多 6%。
Fig 3. 次数类(次)。模型请求数四档几乎不变(27–31),shell 命令数也只在 24–33 之间, 说明差别不在"跑了多少步"。
Fig 4. 输入 token(百万级,堆叠)。92–96% 是缓存命中,真正新增的输入只有 5–10 万。
Fig 5. 输出 token(千级,堆叠)。这里才是 effort 真正花钱的地方: xhigh 的 reasoning 部分是 low 的 4 倍,可见输出也翻了 2.6 倍。
Fig 6. 非空气方块数(个)。规模基本持平,xhigh 甚至比 high 少 349 块。体量不是 effort 的产物。
Fig 7. 材质丰富度(个)。palette state 与方块种类单调上升,这才是 effort 的实际体现: 同样大的房子,用了更多不同的方块和朝向。
Fig 8. 代码体积(KB)。生成器从 17 KB 长到 42.7 KB;测试在 high 达峰, xhigh 把结构检查拆进了单独的 13 KB 模块(未计入此处的测试列)。

四个 schematic

浏览器内直接渲染四份产物,可拖动旋转。

effort = low  |  10,031 blocks  |  45x41x35  |  74 states

effort = medium  |  10,661 blocks  |  45x40x35  |  79 states

effort = high  |  12,341 blocks  |  47x41x37  |  101 states

effort = xhigh  |  11,992 blocks  |  47x43x37  |  126 states


全部渲染图

agent 自己渲的预览,含它保留的迭代前版本。渲染在容器内跑,512 px 约 1.4 秒。

effort = low  (12 renders)

Final renders

low bedroom.png
bedroom.png
low exterior_front.png
exterior_front.png
low exterior_rear.png
exterior_rear.png
low ground.png
ground.png
low living.png
living.png
low study.png
study.png

Earlier pass, kept by the agent for comparison

low initial_bedroom.png
initial_bedroom.png
low initial_exterior_front.png
initial_exterior_front.png
low initial_exterior_rear.png
initial_exterior_rear.png
low initial_ground.png
initial_ground.png
low initial_living.png
initial_living.png
low initial_study.png
initial_study.png

effort = medium  (12 renders)

Final renders

medium bedroom.png
bedroom.png
medium exterior_back.png
exterior_back.png
medium exterior_front.png
exterior_front.png
medium ground.png
ground.png
medium living.png
living.png
medium study.png
study.png

Earlier pass, kept by the agent for comparison

medium first_pass_bedroom.png
first_pass_bedroom.png
medium first_pass_exterior_back.png
first_pass_exterior_back.png
medium first_pass_exterior_front.png
first_pass_exterior_front.png
medium first_pass_ground.png
first_pass_ground.png
medium first_pass_living.png
first_pass_living.png
medium first_pass_study.png
first_pass_study.png

effort = high  (9 renders)

Final renders

high attic.png
attic.png
high bedroom.png
bedroom.png
high bedroom_window.png
bedroom_window.png
high exterior_front.png
exterior_front.png
high exterior_rear.png
exterior_rear.png
high ground.png
ground.png
high living.png
living.png

Earlier pass, kept by the agent for comparison

high initial_bedroom.png
initial_bedroom.png
high initial_exterior_front.png
initial_exterior_front.png

effort = xhigh  (8 renders)

Final renders

xhigh attic.png
attic.png
xhigh bedroom.png
bedroom.png
xhigh bedroom_detail.png
bedroom_detail.png
xhigh exterior_front_left.png
exterior_front_left.png
xhigh exterior_rear_right.png
exterior_rear_right.png
xhigh forge_detail.png
forge_detail.png
xhigh ground.png
ground.png
xhigh living.png
living.png

命令轨迹

每条 shell 命令的首行、退出码、耗时,从 codex rollout 提取。

low — 26 commands, 4 failed
[01] exit=0 0s cat /work/AGENTS.md [02] exit=0 0s ls -la; rg -n 'class LitematicCanvas|def |class Material' /work/generator/MCIO/mcio/sketch.py [03] exit=0 0s cat /work/generator/MCRender/src/headless/index.ts [04] exit=0 0s sed -n '1,175p' /work/generator/MCIO/mcio/sketch.py; rg -n 'def load_schematic|class .*|def ' /work/generator/MCIO/mcio/schematic.py | head -65; rg -n 'class... [05] exit=0 0s sed -n '636,690p' /work/generator/MCIO/mcio/litematica.py; sed -n '819,840p' /work/generator/MCIO/mcio/litematica.py; sed -n '44,65p' /work/generator/MCIO/mc... [06] exit=0 0s rg -n 'def save|class File|def write' /work/generator/MCIO/mcio/litematica.py /work/generator/MCIO/mcio/nbt/__init__.py | tail -15; sed -n '800,830p' /work/g... [07] exit=0 0s sed -n '1315,1335p' /work/generator/MCIO/mcio/litematica.py; head -40 /work/generator/MCIO/mcio/nbt/__init__.py [08] exit=1 0s cat > generate_lodge.py <<'PY' <-- FAILED [09] exit=1 0s python - <<'PY' <-- FAILED [10] exit=0 3s sed -i "221s/$/})/" generate_lodge.py [11] exit=0 2s cat > render_previews.mjs <<'JS' [12] exit=0 0s tail -8 previews/render.log; rg --files previews | head -20 [13] exit=1 0s python - <<'PY' <-- FAILED [14] exit=0 1s python - <<'PY' [15] exit=0 2s sed -i "s/'study': (30, 22, 18)/'study': (31, 22, 18)/" generate_lodge.py [16] exit=0 0s rg -n 'lightDirection|ambientIntensity' /work/generator/MCRender/src -g '*glsl*' -g '*.ts' | head -35; rg -n 'shader|diffuse|dot\(' /work/generator/MCRender/... [17] exit=1 0s rg -n 'uLight|ambient|diffuse|lightDirection' /work/generator/MCRender/src/render* /work/generator/MCRender/src/Renderer.ts /work/generator/MCRender/src/shad... <-- FAILED [18] exit=0 0s rg -n 'dot\(.*[Ll]ight|uAmbient|uLight|aNormal' /work/generator/MCRender/src | head -35 [19] exit=0 3s sed -i "s/ambientIntensities: \[.85\]/ambientIntensities: [.75]/; s/vec3.fromValues(-.5,1,-.7)/vec3.fromValues(.12,.25,.10)/" render_previews.mjs [20] exit=0 3s python - <<'PY' [21] exit=0 0s cat previews/tests.txt; tail -3 previews/render.log [22] exit=0 2s python - <<'PY' [23] exit=0 3s python - <<'PY' [24] exit=0 0s cat previews/tests.txt [25] exit=0 0s sed -i "s/lecterns\[0\]\['Book'\]\['count'\] == 1/int(lecterns[0]['Book']['count']) == 1/" test/test_structure.py [26] exit=0 0s cat > REPORT.md <<'MD'
medium — 24 commands, 5 failed
[01] exit=0 0s cat /work/AGENTS.md; pwd; ls -la; ls /work/generator/MCIO/mcio; cat /work/generator/MCRender/src/headless/index.ts [02] exit=0 0s rg -n 'class LitematicCanvas|def |entities' /work/generator/MCIO/mcio/sketch.py | tail -65; rg -n 'def load_schematic|class |def ' /work/generator/MCIO/mcio/... [03] exit=0 0s sed -n '1,175p' /work/generator/MCIO/mcio/sketch.py; sed -n '44,85p' /work/generator/MCIO/mcio/schematic.py; rg -n 'entit|class|def save' /work/generator/MCI... [04] exit=0 0s sed -n '636,706p' /work/generator/MCIO/mcio/litematica.py; sed -n '1250,1338p' /work/generator/MCIO/mcio/litematica.py; rg -n 'Entities|TileEntities' /work/g... [05] exit=1 0s cat > generate.py <<'PY' <-- FAILED [06] exit=0 0s rg -n 'import.*NBTFile' /work/generator/MCIO/mcio/litematica.py; rg --files /work/generator/MCIO/mcio/nbt | head -15 [07] exit=1 0s sed -i 's/from mcio.nbt.file import NBTFile/from mcio.nbt.nbt import File as NBTFile/' generate.py <-- FAILED [08] exit=0 3s sed -i "s/(16,31,26)/(16,32,26)/; s/radius=max(1,(height-y)\/\/5)/radius=min(2,max(1,(height-y)\/\/5))/" generate.py [09] exit=0 0s tail -8 previews/render.log; rg --files previews | head -20 [10] exit=1 0s mkdir -p test previews/first_pass <-- FAILED [11] exit=1 0s python - <<'PY' <-- FAILED [12] exit=0 0s rg -n 'sin|cos|sphericalCameraPose' /work/generator/MCRender/src/renderHelpers.ts | head -20 [13] exit=0 0s sed -n '59,83p' /work/generator/MCRender/src/renderHelpers.ts; rg -n 'ambientIntensity|lightDirection' /work/generator/MCRender/src/StructureRenderer.ts | he... [14] exit=0 0s rg -n 'ambientIntensity|lightDirection|uLight' /work/generator/MCRender/src -g '*.glsl' -g '*.ts' | head -35 [15] exit=0 3s python - <<'PY' [16] exit=0 0s sed -n '175,235p' test/test_structure.py; sed -n '10,52p' generate.py; tail -35 generate.py [17] exit=0 0s sed -n '140,172p' test/test_structure.py; rg -n 'x1, x2, z1|def door' generate.py [18] exit=0 0s sed -n '183,207p' generate.py [19] exit=1 0s python - <<'PY' <-- FAILED [20] exit=0 0s sed -i "s/b.put(13, 11, 26, 'smoker'/b.put(15, 11, 26, 'smoker'/" generate.py [21] exit=0 0s sed -n '87,116p' generate.py [22] exit=0 3s python - <<'PY' [23] exit=0 0s PYTHONDONTWRITEBYTECODE=1 python - <<'PY' [24] exit=0 0s cat > REPORT.md <<'MD'
high — 33 commands, 1 failed
[01] exit=0 0s cat /work/AGENTS.md [02] exit=0 0s ls -la; rg --files /work/generator/MCIO/mcio /work/generator/MCRender/src/headless [03] exit=0 0s sed -n '1,260p' /work/generator/MCIO/mcio/sketch.py [04] exit=0 0s cat /work/generator/MCRender/src/headless/index.ts [05] exit=0 0s sed -n '1,240p' /work/generator/MCIO/mcio/schematic.py; sed -n '1,180p' /work/generator/MCIO/mcio/litematica.py [06] exit=0 0s rg -n 'tile_entities|entities|TileEntities|Entities|def save|class LitematicaSchematic' /work/generator/MCIO/mcio/litematica.py; rg -n 'export function (crea... [07] exit=0 0s sed -n '660,820p' /work/generator/MCIO/mcio/litematica.py; sed -n '940,1020p' /work/generator/MCRender/src/headless/index.ts; cat /work/generator/MCIO/mcio/n... [08] exit=0 0s sed -n '1310,1345p' /work/generator/MCIO/mcio/litematica.py; sed -n '1128,1157p' /work/generator/MCRender/src/headless/index.ts; sed -n '1220,1245p' /work/ge... [09] exit=0 0s rg --files -g AGENTS.md /work/generator /work/.agent 2>/dev/null | head -30; rg -n 'sphericalCameraPose|Math.cos|Math.sin' /work/generator/MCRender/src/rende... [10] exit=0 0s cat > generate_lodge.py <<'PY' [11] exit=0 0s sed -n '59,85p' /work/generator/MCRender/src/renderHelpers.ts; sed -n '1150,1185p' /work/generator/MCRender/src/headless/index.ts [12] exit=0 3s cat > render_previews.mjs <<'JS' [13] exit=1 0s mkdir -p test <-- FAILED [14] exit=0 0s python - <<'PY' [15] exit=0 0s rg -n 'ambientIntensity|lightDirection' /work/generator/MCRender/src -g '*glsl*' -g '*.ts' | head -50 [16] exit=0 1s python - <<'PY' [17] exit=0 2s python - <<'PY' [18] exit=0 0s rg --files /work/generator/MCRender/resources | head -25; tail -20 previews/render.log [19] exit=0 1s python - <<'PY' [20] exit=0 2s xvfb-run -a /work/generator/MCRender/node_modules/.bin/vite-node --script render_previews.mjs > previews/render.log 2>&1 [21] exit=0 1s python - <<'PY' [22] exit=0 0s rg -n 'banner|item_frame|lectern' /work/generator/MCRender/src -g '*.ts' | head -35; sed -n '35,105p' generate_lodge.py [23] exit=0 0s sed -n '692,716p' /work/generator/MCRender/src/render/SpecialRenderer.ts; rg -n 'def cutaways' generate_lodge.py [24] exit=0 3s python - <<'PY' [25] exit=0 1s python -B - <<'PY' [26] exit=0 0s rg -n 'TileEntities|block_entities|tileEntities|patterns' /work/generator/MCRender/src/Schematics.ts /work/generator/MCRender/src/Structure.ts /work/generato... [27] exit=0 0s python -B - <<'PY' [28] exit=0 0s rg -n 'nbt|getBlock' /work/generator/MCRender/src/MultiStructure.ts /work/generator/MCRender/src/Structure.ts /work/generator/MCRender/src/StructureRenderer.... [29] exit=0 0s sed -n '20,45p' /work/generator/MCRender/src/MultiStructure.ts; rg -n 'addBlock|initStructure|nbt' /work/generator/MCRender/src/headless/index.ts | tail -25;... [30] exit=0 0s sed -n '115,173p' /work/generator/MCRender/src/render/ChunkBuilder.ts; sed -n '750,790p' /work/generator/MCRender/src/render/SpecialRenderer.ts; sed -n '1087... [31] exit=0 0s python -B - <<'PY' [32] exit=0 0s cat > REPORT.md <<'MD' [33] exit=0 0s python -B - <<'PY'
xhigh — 32 commands, 4 failed
[01] exit=0 0s pwd && rg --files -g 'AGENTS.md' -g '*.py' -g '*.mjs' -g 'README*' . /work/generator/MCIO /work/generator/MCRender/src/headless /work 2>/dev/null | head -90 [02] exit=0 0s cat /work/AGENTS.md [03] exit=0 0s sed -n '1,260p' /work/generator/MCIO/mcio/sketch.py [04] exit=0 0s for p in /work/generator/AGENTS.md /work/generator/workspace/AGENTS.md; do if test -f "$p"; then cat "$p"; fi; done [05] exit=0 0s sed -n '1,260p' /work/generator/MCIO/mcio/schematic.py && sed -n '1,220p' /work/generator/MCRender/src/headless/index.ts [06] exit=0 0s rg -n 'entities|Entities|tile_entities|TileEntities|block_entities|def save|DATA_VERSION' /work/generator/MCIO/mcio/litematica.py [07] exit=0 0s python - <<'PY' [08] exit=0 0s rg -n 'function createRenderScene|function renderLitematicWithMetadata|cameraDistance|objectCenter|ambientIntensity' /work/generator/MCRender/src/headless/in... [09] exit=0 0s sed -n '765,850p' /work/generator/MCRender/src/headless/index.ts [10] exit=0 0s sed -n '1128,1258p' /work/generator/MCRender/src/headless/index.ts [11] exit=1 0s cat > generate_lodge.py <<'PY' <-- FAILED [12] exit=0 4s python - <<'PY' [13] exit=1 0s mkdir -p test <-- FAILED [14] exit=0 0s python - <<'PY' [15] exit=0 0s rg -n 'lightDirection|uLight|ambient' /work/generator/MCRender/src/StructureRenderer.ts /work/generator/MCRender/src/render* /work/generator/MCRender/src/*Sh... [16] exit=1 0s python - <<'PY' <-- FAILED [17] exit=0 3s python - <<'PY' [18] exit=0 1s rg -n 'lightDirection|uLight|ambient' /work/generator/MCRender/src/StructureRenderer.ts /work/generator/MCRender/src/Renderer.ts | head -40 [19] exit=0 0s rg -n 'banner|patterns|TileEntities|ItemFrame' /work/generator/MCRender/src/Schematics.ts /work/generator/MCRender/src/StructureRenderer.ts /work/generator/M... [20] exit=0 3s python - <<'PY' [21] exit=0 0s rg -n 'def save|def write' /work/generator/MCIO/mcio/nbt/nbt.py [22] exit=0 0s sed -n '223,242p' /work/generator/MCIO/mcio/nbt/nbt.py [23] exit=1 1s python - <<'PY' <-- FAILED [24] exit=0 0s python - <<'PY' [25] exit=0 5s python - <<'PY' [26] exit=0 0s cat previews/test_results.txt [27] exit=0 0s python - <<'PY' [28] exit=0 1s python - <<'PY' [29] exit=0 5s python -m yapf --style /work/.style.yapf -i generate_lodge.py structural_checks.py test/test_lodge.py [30] exit=0 0s cat previews/generation.log [31] exit=0 0s cat > REPORT.md <<'MD' [32] exit=0 0s PYTHONDONTWRITEBYTECODE=1 python - <<'PY'

token 花在哪

把每个模型请求归到它当时在做的事,再把该请求的 token 记到那件事头上。 分类依据是这个请求触发的 shell 命令:读源码算探索,cat > *.py 算写生成器, pytest 算测试,xvfb-run 算渲染,没有命令的算规划与说明。

Fig 9. 按目的聚合。每档两行:新增输入、输出。段的顺序按用途分组,不按时间。 hover 看该段是什么事、几个请求、几条命令、折合多少钱。缓存命中的输入不画在这里(它是重复上下文, 量级是这两行的十倍),但计入下面的金额。

逐请求细切(按时间)

low 逐请求细切 — 27 个请求,按时间排序
每段是一个模型请求,左起为时间顺序。hover 看该请求触发了什么命令、花了多少 token、折合多少钱。
medium 逐请求细切 — 28 个请求,按时间排序
每段是一个模型请求,左起为时间顺序。hover 看该请求触发了什么命令、花了多少 token、折合多少钱。
high 逐请求细切 — 31 个请求,按时间排序
每段是一个模型请求,左起为时间顺序。hover 看该请求触发了什么命令、花了多少 token、折合多少钱。
xhigh 逐请求细切 — 29 个请求,按时间排序
每段是一个模型请求,左起为时间顺序。hover 看该请求触发了什么命令、花了多少 token、折合多少钱。

按 GPT-5 档位估算的花费

费率:新增输入 $1.25/M,缓存输入 $0.125/M, 输出 $10.0/M(输出含 reasoning)。这是估算gpt-6-astra 没有公开定价,且这四次跑用的是订阅额度,真实支出为 0。

effort缓存输入 token缓存费新增输入 token新增输入费 输出 token输出费合计
low 1,119,488$0.140 65,323$0.082 18,906$0.189 $0.411
medium 1,313,024$0.164 74,309$0.093 20,281$0.203 $0.460
high 1,943,040$0.243 87,876$0.110 31,020$0.310 $0.663
xhigh 2,198,400$0.275 195,689$0.245 49,051$0.490 $1.010
$2.54四档合计估算
2.5×xhigh / low 费用比
49%xhigh 花在输出上
34%low 花在缓存输入上

结构性结论:低档的钱花在读,高档的钱花在写。low 有 34% 的估算费用来自缓存输入(一遍遍重喂上下文), 只有 46% 来自输出;xhigh 反过来,48% 在输出,缓存只占 27%。 要压成本,低档该优化的是上下文,高档该优化的是让它少写废代码。


问题与分类

14 次失败命令,四类:

类别含义lowmediumhighxhigh
A 一次性大文件语法/导入错heredoc 写 20-40 KB python 后首次运行即炸2100
B 坐标越界写块时 assert / ValueError out of bounds0101
C 自写结构测试失败自检抓到真实结构缺陷,随后修复1313
D shell 用法错zsh glob 无匹配1000

low

medium

high

xhigh


测试对比

四档都自己写了 pytest 结构测试并跑到全绿,但"全绿"的含金量差别很大。 下面按类别拆开看:每类测了什么、用了多少行、谁漏了哪类。

8 / 9 / 12 / 12测试数量 low→xhigh
0 / 1 / 2 / 3负控制测试数量
146 / 152 / 256 / 465测试相关总行数
3 / 4产物字节可复现(实测)

类别覆盖矩阵

单元格是"该类别的测试体行数 / 测试个数"。行数只算测试函数本身,不含辅助函数。

类别lowmediumhighxhigh
导出回读14 行 / 1 个13 行 / 1 个13 行 / 1 个11 行 / 1 个
尺寸与方块状态合法性9 行 / 1 个7 行 / 1 个11 行 / 1 个
确定性10 行 / 1 个16 行 / 2 个
门与床配对17 行 / 1 个3 行 / 1 个3 行 / 1 个18 行 / 2 个
可达性与净高24 行 / 1 个6 行 / 1 个19 行 / 1 个22 行 / 1 个
封闭性25 行 / 1 个2 行 / 1 个12 行 / 1 个10 行 / 1 个
支撑与悬空22 行 / 1 个23 行 / 1 个51 行 / 2 个8 行 / 1 个
窗洞与实体载荷18 行 / 1 个11 行 / 1 个10 行 / 1 个
照明4 行 / 1 个4 行 / 1 个14 行 / 1 个4 行 / 1 个
功能性家具与实体11 行 / 1 个17 行 / 1 个
交付物一致性4 行 / 1 个
负控制17 行 / 1 个17 行 / 2 个
Fig 10. 各类测试的代码行数(行)。同一单位,可直接比高度。 high 把一半篇幅压在"支撑与悬空"(51 行);xhigh 把篇幅摊得最均匀,且额外有 325 行独立校验模块没算进来。

逐个测试

low — test_structure.py,8 个测试, 133 行测试体 + 13 行辅助函数
测试行数类别含负控制目的(docstring)
test_registry_and_extent9尺寸与方块状态合法性(无 docstring)
test_door_and_bed_pairs17门与床配对(无 docstring)
test_two_block_headroom_reachability24可达性与净高(无 docstring)
test_closed_envelope25封闭性(无 docstring)
test_supports_and_no_floating_components22支撑与悬空(无 docstring)
test_each_room_is_lit4照明(无 docstring)
test_export_roundtrip_and_entities14导出回读(无 docstring)
test_window_clearance_and_textile_data18窗洞与实体载荷(无 docstring)
medium — test_structure.py,9 个测试, 86 行测试体 + 66 行辅助函数
测试行数类别含负控制目的(docstring)
test_registry_and_determinism7尺寸与方块状态合法性(无 docstring)
test_doors_and_beds3门与床配对(无 docstring)
test_reachability_and_headroom6可达性与净高(无 docstring)
test_enclosure2封闭性(无 docstring)
test_no_floating_decoration23支撑与悬空(无 docstring)
test_room_lighting4照明(无 docstring)
test_full_export13导出回读(无 docstring)
test_validators_reject_damage17负控制(无 docstring)
test_windows_not_obstructed11窗洞与实体载荷(无 docstring)
high — test_lodge.py,12 个测试, 154 行测试体 + 102 行辅助函数
测试行数类别含负控制目的(docstring)
test_registry_and_full_state_round_trip13导出回读Require canonical legal states, one region, version and equality of all cells.
test_delivered_file_matches_generator4交付物一致性Check the contract artifact, rather than only a newly generated copy.
test_fixed_seed_is_reproducible10确定性Require repeatable blocks and entity data from the documented seed.
test_doors_and_beds3门与床配对Check all paired blocks in the reloaded file.
test_pair_check_detects_damage11负控制Ensure pairing validation fails for missing halves and mismatched facings.
test_every_room_reachable_with_headroom19可达性与净高Require routes through the front doors to every furnished room group.
test_enclosure_and_roof_join12封闭性Require every interior air cell to be isolated from a six-neighbor sky flood.
test_enclosure_check_detects_hole6负控制Prove that a single missing wall block leaks to the exterior flood.
test_decoration_support_and_window_clearance37支撑与悬空Check directional attachment for every fragile decoration and entity.
test_no_disconnected_blocks14支撑与悬空Require every block to connect through face neighbors to the terrain.
test_lights_furnishings_and_pines14照明Require nearby light in every room and the specified functional furnishings.
test_forge_flue_and_fuel11功能性家具与实体Check the permanent fire, open smoke route and active furnace inventories.
xhigh — test_lodge.py,12 个测试, 127 行测试体 + 13 行辅助函数 + 325 行独立校验模块
测试行数类别含负控制目的(docstring)
test_full_state_round_trip_and_single_region11导出回读Check every air and non-air cell plus metadata and complete state counts.
test_every_state_is_vanilla_and_explicit11尺寸与方块状态合法性Check all names, keys, values and defaults against the 1.21.1 registry.
test_door_pairing_and_negative_control10门与床配对Check every door and demonstrate detection of a missing or reversed half.
test_bed_pairing_and_negative_control8门与床配对Check both green beds, support, facing, and missing-half detection.
test_front_door_reaches_every_room_and_stair_route22可达性与净高Require walkable, two-cell-high routes through all floors and room groups.
test_enclosure_and_missing_roof_negative_control10封闭性Check sealed room air components and detect a removed roof patch.
test_no_floating_or_clipping_decorations8支撑与悬空Check grounded assemblies, props, vines, ladder mounts and entity backing.
test_window_openings_are_preserved10窗洞与实体载荷Ensure every designed window survived roofs, trees, flues, and furniture.
test_each_room_is_lit4照明Require a working light source close to every named room target.
test_furnishings_and_entity_payloads17功能性家具与实体Inspect banner patterns, readable books, item frames, and forge fuel.
test_seed_reproduces_geometry7确定性Check deterministic regeneration including tile payloads and entity UUIDs.
test_export_bytes_are_reproducible9确定性Require identical gzip bytes regardless of output filename or save time.

自测失败统计:哪类测试被抓到过

这里统计的是 agent 自己写的测试在迭代过程中真的失败过多少次, 按测试类别归并。一次 pytest 跑出多条 FAILED 算多次。四档最终都是全绿收工, 下面是收工前被自己的检查拦下来的记录。

测试类别lowmediumhighxhigh合计
可达性与净高2 次1 次01 次4
支撑与悬空1 次2 次2 次1 次6
窗洞与实体载荷2 次1 次01 次4
封闭性001 次1 次2
导出回读1 次0001
负控制01 次001
合计6 次 / 5 轮5 次 / 3 轮3 次 / 1 轮4 次 / 3 轮18
Fig 11. 各类测试的失败次数(次)。支撑与悬空是最容易写错的一类,四档里三档栽在这上面,共 6 次。
逐轮明细:每次 pytest 失败具体挂了哪些测试

low

  • 第 13 条命令:test_export_roundtrip_and_entitiestest_two_block_headroom_reachability
  • 第 14 条命令:test_two_block_headroom_reachability
  • 第 21 条命令:test_supports_and_no_floating_components
  • 第 23 条命令:test_window_clearance_and_textile_data
  • 第 24 条命令:test_window_clearance_and_textile_data

medium

  • 第 10 条命令:test_no_floating_decorationtest_reachability_and_headroomtest_validators_reject_damage
  • 第 11 条命令:test_no_floating_decoration
  • 第 19 条命令:test_windows_not_obstructed

high

  • 第 13 条命令:test_decoration_support_and_window_clearancetest_enclosure_and_roof_jointest_no_disconnected_blocks

xhigh

  • 第 13 条命令:test_front_door_reaches_every_room_and_stair_route
  • 第 16 条命令:test_enclosure_and_missing_roof_negative_controltest_no_floating_or_clipping_decorations
  • 第 23 条命令:test_window_openings_are_preserved

读出来的结论:

读出来的差别


四份代码

四次跑互相看不见对方,起点只有同一份 BRIEF.md 和两张参考图。 结果不只是"都写了个生成器",而是连类名、方法名、调用层次都撞在一起

四档都写了一个叫 Lodge 的类,都有 put() 放单块、box() 填区域、 door() 放门、build() 串流程、main() 做入口。BRIEF 里没有任何一句要求这么组织。

函数名重合度

单元格是两档之间同名函数的个数(含类方法,不含类名本身)。

lowmediumhighxhigh
low789
medium789
high8816
xhigh9916

四档共有的名字:__init__boxbuilddoormainput。 high 与 xhigh 之间重合 16 个——连 shell()(外壳)、forge()(铁匠炉)、 carpet()landscape()frame()state() 都叫一样。

函数对照表

只列至少两档都出现的名字,单元格是该函数的行数。

函数lowmediumhighxhigh
__init__5 行6 行8 行11 行
box6 行6 行7 行14 行
build219 行249 行9 行18 行
carpet5 行11 行
door4 行4 行4 行11 行
export23 行19 行43 行
forge44 行46 行
frame3 行20 行34 行
get3 行9 行
landscape26 行33 行
main10 行9 行16 行16 行
put11 行8 行9 行17 行
shell47 行80 行
state7 行9 行17 行
tile3 行3 行9 行
verify_roundtrip14 行10 行23 行
window15 行29 行

差别在拆分粒度,不在结构:low 与 medium 把整栋楼塞进一个 219 / 249 行的 build(); high 拆成 terrain / shell / roof / circulation / forge / interiors / landscape; xhigh 拆得最细,光内饰就分成 furnished_ground / furnished_living / furnished_bedroom / furnished_atticbuild() 只剩 18 行胶水。行数也一路涨:354 → 397 → 614 → 915。

完整源码

点标签切换档位。每档第一个文件默认展开,其余点开。

low medium high xhigh
generate_lodge.py — 354 行
# Generate Amberstone Lodge and reproducible cutaways with MCIO. from pathlib import Path import sys import json import random from collections import Counter import types sys.path.insert(0, '/work/generator/MCIO') from mcio.sketch import LitematicCanvas from mcio.schematic import load_schematic from mcio.nbt.tag import Compound, String, Int, Byte, Double, List, IntArray CONFIG = { 'seed': 73129, 'size': (41, 35, 45), 'data_version': 3955, 'tower': (25, 37, 10, 26), 'lower': (10, 25, 10, 26), 'floors': (3, 9, 15, 21), 'ladder': (35, 24), 'stone': ('stone_bricks', 'cobblestone', 'andesite'), 'plaster': ('sandstone', 'smooth_sandstone', 'end_stone_bricks'), 'roof': ('deepslate_tiles', 'deepslate_bricks', 'cobbled_deepslate'), 'wood': ('spruce_planks', 'dark_oak_planks', 'spruce_planks'), 'rooms': { 'entrance': (30, 4, 13), 'storage': (16, 4, 18), 'living': (17, 10, 16), 'hall': (30, 10, 16), 'bedroom': (30, 16, 18), 'study': (31, 22, 18) }, } REGISTRY = json.loads(Path('/work/generator/MCRender/resources/1.21.1-blocks.json').read_text()) class Lodge: """Hold deterministic blocks, furniture supports and export metadata.""" def __init__(self): self.blocks = {} self.rng = random.Random(CONFIG['seed']) self.tiles = [] self.entities = [] def put(self, x, y, z, name, **props): """Place a registry-normalized block at XYZ.""" if name == 'air': self.blocks.pop((x, y, z), None) return allowed, defaults = REGISTRY[name] state = dict(defaults) state.update({k: str(v) for k, v in props.items()}) assert all(k in allowed and v in allowed[k] for k, v in state.items()), (name, state) suffix = '[' + ','.join(f'{k}={v}' for k, v in sorted(state.items())) + ']' if state else '' self.blocks[x, y, z] = 'minecraft:' + name + suffix def box(self, x1, x2, y1, y2, z1, z2, material, **props): """Fill an inclusive box with a seeded palette.""" for x in range(x1, x2 + 1): for y in range(y1, y2 + 1): for z in range(z1, z2 + 1): self.put(x, y, z, self.rng.choice(material) if isinstance(material, tuple) else material, **props) def door(self, x, y, z): """Place a matching glazed oak door pair.""" for half, dy in [('lower', 0), ('upper', 1)]: self.put(x, y + dy, z, 'oak_door', half=half, facing='south', hinge='left') def pot(self, x, y, z): """Place a potted fern on a cabinet.""" self.put(x, y, z, 'potted_fern') def frame(self, x, y, z): """Attach a real item-frame entity to a north wall.""" self.entities.append(Compound({'id': String('minecraft:item_frame'), 'Pos': List[Double]([Double(x + .5), Double(y + .5), Double(z + 1.03125)]), 'Facing': Byte(3), 'TileX': Int(x), 'TileY': Int(y), 'TileZ': Int(z + 1), 'ItemRotation': Byte(0), 'Item': Compound({'id': String('minecraft:iron_axe'), 'count': Int(1)}), 'UUID': IntArray([73129, x, y, z])})) def export(self, path: Path, bounds=None): """Export one MCIO region, optionally cropped for an interior inspection.""" if bounds is None: bounds = (0, 44, 0, 40, 0, 34) x1, x2, y1, y2, z1, z2 = bounds canvas = LitematicCanvas((y2 - y1 + 1, z2 - z1 + 1, x2 - x1 + 1)) for (x, y, z), state in self.blocks.items(): if x1 <= x <= x2 and y1 <= y <= y2 and z1 <= z <= z2: canvas.block((y - y1, z - z1, x - x1), state) schem = canvas.to_litematica(name='Amberstone Lodge', author='Codex', minecraft_data_version=CONFIG['data_version']) schem.metadata.time_created = 0 schem.metadata.time_modified = 0 if bounds == (0, 44, 0, 40, 0, 34): original = schem.write_sub_region_to_nbt def region(this, name): tag = original(name) tag['TileEntities'] = List[Compound](self.tiles) tag['Entities'] = List[Compound](self.entities) return tag schem.write_sub_region_to_nbt = types.MethodType(region, schem) schem.save(path) def build(): """Construct terrain, connected timber shells, roofs, furnishings and trees.""" b = Lodge() # Irregular sod island and raised terrace. for x in range(2, 43): for z in range(2, 33): if ((x - 22) / 23)**2 + ((z - 18) / 18)**2 < 1: b.put(x, 0, z, 'dirt') b.put(x, 1, z, 'grass_block') b.box(7, 39, 1, 2, 8, 28, CONFIG['stone']) b.box(8, 38, 3, 3, 8, 28, ('stone_bricks', 'mossy_stone_bricks', 'andesite')) for z in range(0, 9): center = 30 + round(2 * __import__('math').sin(z * .7)) for x in range(center - 2, center + 3): b.put(x, 1, z, b.rng.choice(('dirt_path', 'gravel', 'coarse_dirt', 'cobblestone'))) for z, y in [(6, 1), (7, 2)]: b.box(28, 32, y, y, z, z, 'stone_brick_stairs', facing='south') # Two joined masses with common floor heights. for (x1, x2, z1, z2), top in [(CONFIG['lower'], 15), (CONFIG['tower'], 23)]: for y in range(4, top + 1): for x in range(x1, x2 + 1): for z in range(z1, z2 + 1): if x in (x1, x2) or z in (z1, z2): mat = CONFIG['stone'] if y < 9 else CONFIG['plaster'] b.put(x, y, z, b.rng.choice(mat)) if y in (9, 14, 15, 20, 21, 23): b.put(x, y, z, 'dark_oak_log', axis='x' if z in (z1, z2) else 'z') if (x in (x1, x2) and (z - z1) % 5 == 0) or (z in (z1, z2) and (x - x1) % 5 == 0): b.put(x, y, z, 'stripped_spruce_log') for fy in CONFIG['floors']: if fy <= top: b.box(x1 + 1, x2 - 1, fy, fy, z1 + 1, z2 - 1, CONFIG['wood']) # Open the shared wall on both occupied lower-wing levels. for fy in (3, 9): b.box(25, 25, fy + 1, fy + 4, 17, 20, 'air') b.box(25, 25, fy, fy, 17, 20, 'spruce_planks') # Tower gable: steep two-rise roof, closed front and rear triangles. for x in range(25, 38): top = 37 - 2 * abs(x - 31) for z in (10, 26): b.box(x, x, 24, top, z, z, CONFIG['plaster']) if x == 31: b.box(x, x, 24, top, z, z, 'stripped_spruce_log') for x in range(23, 40): h = 38 - 2 * abs(x - 31) b.box(x, x, h - 2, h, 8, 28, CONFIG['roof']) b.box(x, x, h, h, 8, 28, 'deepslate_tile_stairs', facing='east' if x < 31 else 'west') for z in (8, 28): b.put(x, h, z, 'polished_deepslate') b.box(31, 31, 39, 39, 9, 27, 'deepslate_tile_slab') # Lower transverse gable, overlaps tower shell to make a sealed valley. for z in range(8, 29): h = 23 - abs(z - 18) // 2 for x in range(8, 26): b.put(x, h, z, b.rng.choice(CONFIG['roof'])) b.put(x, h + 1, z, 'deepslate_tile_slab' if abs(z - 18) % 2 == 0 else 'deepslate_tile_stairs', **({} if abs(z - 18) % 2 == 0 else {'facing': 'south' if z < 18 else 'north'})) if 10 <= z <= 26: for x in (10,): b.box(x, x, 16, h - 1, z, z, CONFIG['plaster']) # Sealed ceiling above lower living room. b.box(11, 24, 16, 16, 11, 25, 'spruce_planks') # Windows: full glass keeps a watertight envelope, exterior shutters beside openings. for y in (5, 11, 17, 23, 29): xs = (13, 19, 28, 33) if y < 15 else ((28, 33) if y < 25 else (30,)) for x in xs: for z in (10, 26): b.box(x, x + 1, y, y + 2, z, z, 'glass') for sx in (x - 1, x + 2): b.box(sx, sx, y, y + 2, z + (-1 if z == 10 else 1), z + (-1 if z == 10 else 1), 'spruce_trapdoor', facing='south' if z == 10 else 'north', open='true') b.box(x, x + 1, y - 1, y - 1, z + (-1 if z == 10 else 1), z + (-1 if z == 10 else 1), 'spruce_stairs', facing='north' if z == 10 else 'south') # Side elevations use the same plaster panels, framed windows and sills. for wall_x, levels in ((10, (5, 11)), (37, (5, 11, 17))): for y in levels: for z in (14, 21): b.box(wall_x, wall_x, y, y + 2, z, z + 1, 'glass') outside = wall_x + (-1 if wall_x == 10 else 1) b.box(outside, outside, y - 1, y - 1, z, z + 1, 'spruce_stairs', facing='east' if wall_x == 10 else 'west') for sz in (z - 1, z + 2): b.box(outside, outside, y, y + 2, sz, sz, 'spruce_trapdoor', facing='east' if wall_x == 10 else 'west', open='true') b.door(30, 4, 10) b.put(30, 5, 9, 'air') # Keep the neighboring shutter out of the door approach. for x in (28, 32): b.put(x, 6, 9, 'spruce_fence') b.put(x, 5, 9, 'lantern', hanging='true') # Ladder hall: continuous back support and a two-cell clear landing at every floor. for y in range(4, 24): b.put(35, y, 25, 'spruce_planks') b.put(35, y, 24, 'ladder', facing='north') b.put(34, y, 24, 'air') for fy in CONFIG['floors']: b.put(34, fy, 24, 'spruce_planks') # Ground entrance, workshop and storage. for x in (12, 15, 18, 21): b.put(x, 4, 24, 'barrel', facing='north') b.put(x, 5, 24, 'chest', facing='north') b.box(12, 17, 4, 4, 12, 12, 'spruce_planks') for x in (12, 14, 16): b.put(x, 5, 12, 'crafting_table' if x == 14 else 'barrel') b.put(20, 4, 14, 'anvil') b.box(28, 32, 4, 4, 13, 17, 'red_carpet') b.box(36, 36, 4, 6, 13, 18, 'bookshelf') b.put(27, 4, 12, 'barrel') b.pot(27, 5, 12) b.frame(23, 6, 10) # Living and dining: upholstered seating, large oak table and kitchen cabinets. for x in range(13, 19): b.put(x, 10, 23, 'spruce_stairs', facing='south') b.box(14, 20, 10, 10, 15, 20, 'green_carpet') b.box(15, 18, 10, 10, 17, 18, 'dark_oak_planks') for x in (15, 18): b.put(x, 11, 17, 'candle', candles='3', lit='true') b.put(x, 10, 15, 'spruce_stairs', facing='north') for x in range(12, 23): b.put(x, 10, 12, 'barrel', facing='south') if x % 3 == 0: b.put(x, 13, 12, 'barrel', facing='south') b.put(x, 13, 11, 'spruce_planks') b.put(13, 11, 12, 'smoker', facing='south', lit='true') b.pot(20, 11, 12) b.box(27, 27, 10, 13, 17, 21, 'bookshelf') b.box(29, 32, 10, 10, 13, 20, 'red_carpet') b.put(36, 10, 12, 'barrel') b.pot(36, 11, 12) # Bedroom platform and matching green beds. b.box(27, 31, 16, 16, 20, 23, 'dark_oak_planks') b.box(28, 30, 16, 16, 19, 19, 'dark_oak_stairs', facing='south') for x in (28, 29): b.put(x, 17, 22, 'green_bed', facing='south', part='foot') b.put(x, 17, 23, 'green_bed', facing='south', part='head') b.box(27, 31, 19, 19, 25, 25, 'bookshelf') for x in (27, 31): b.put(x, 17, 23, 'barrel') b.put(x, 18, 23, 'lantern') b.put(x, 20, 25, 'barrel') b.box(28, 32, 16, 16, 14, 17, 'green_carpet') for x in (27, 30, 32, 35): b.put(x, 19, 11, 'white_wall_banner', facing='south') b.tiles.append(Compound({'id': String('minecraft:banner'), 'x': Int(x), 'y': Int(19), 'z': Int(11), 'patterns': List[Compound]([Compound({'pattern': String('minecraft:stripe_center'), 'color': String('green')})])})) b.box(28, 34, 16, 16, 12, 12, 'dark_oak_stairs', facing='north') b.put(36, 16, 19, 'barrel') b.pot(36, 17, 19) # Attic study. b.box(27, 27, 22, 25, 16, 22, 'bookshelf') b.box(29, 32, 22, 22, 20, 21, 'dark_oak_planks') b.put(30, 23, 20, 'lectern', facing='north', has_book='true') book = Compound({'id': String('minecraft:writable_book'), 'count': Int(1), 'components': Compound({'minecraft:writable_book_content': Compound({'pages': List[Compound]([Compound({'raw': String('Forge orders and winter provisions.')})])})})}) b.tiles.append(Compound({'id': String('minecraft:lectern'), 'x': Int(30), 'y': Int(23), 'z': Int(20), 'Book': book, 'Page': Int(0)})) b.put(32, 23, 20, 'candle', candles='3', lit='true') b.put(30, 22, 18, 'spruce_stairs', facing='north') for x in (28, 32, 36): b.put(x, 22, 12, 'barrel') b.put(x, 23, 12, 'chest', facing='south') b.box(29, 33, 22, 22, 14, 16, 'green_carpet') b.frame(31, 25, 10) # Supported ceiling lanterns on every occupied level. for fy in CONFIG['floors']: for x, z in ((29, 18), (33, 14)) + (((17, 21), (22, 15)) if fy < 15 else ()): ceiling = fy + 5 b.put(x, ceiling, z, 'spruce_planks') b.put(x, ceiling - 1, z, 'lantern', hanging='true') # Tie attic beams into exterior walls. if fy == 21: b.box(26, 36, ceiling, ceiling, z, z, 'dark_oak_log', axis='x') # Covered blacksmith lean-to on left side, independent from room windows. b.box(3, 9, 2, 2, 12, 23, CONFIG['stone']) for x in (3, 8): for z in (12, 23): b.box(x, x, 3, 8, z, z, 'spruce_log') for x in range(2, 11): h = 8 + (x - 2) // 3 b.box(x, x, h, h, 11, 24, 'deepslate_tile_slab') b.box(5, 7, 3, 4, 19, 21, 'stone_bricks') b.put(6, 5, 20, 'campfire', lit='true') b.put(6, 4, 19, 'blast_furnace', facing='north', lit='true') b.box(5, 7, 5, 7, 21, 22, CONFIG['stone']) b.box(5, 7, 8, 24, 21, 22, CONFIG['stone']) b.box(4, 8, 25, 25, 20, 23, 'stone_brick_slab') b.put(5, 3, 15, 'anvil') b.put(8, 3, 17, 'barrel') b.put(8, 4, 17, 'chest') b.put(4, 3, 13, 'lantern') # Climbing foliage on south/front masonry, attached to wall and planter. for x in (25, 26, 36, 37): for y in range(4, 14 if x < 30 else 9): if b.rng.random() < .78: b.put(x, y, 9, 'vine', south='true') # Two hand-built tiered pines, rear left away from the roof. for tx, tz, height in ((5, 29, 22), (17, 30, 29)): b.box(tx, tx, 2, height, tz, tz, 'spruce_log') for y in range(5, height, 4): radius = max(1, (height - y) // 6) for dx, dz in ((1, 0), (-1, 0), (0, 1), (0, -1)): for r in range(1, radius + 1): b.put(tx + dx * r, y, tz + dz * r, 'spruce_log', axis='x' if dx else 'z') for dy in (0, 1): for dx in range(-radius, radius + 1): for dz in range(-radius, radius + 1): if abs(dx) + abs(dz) <= radius + 1 - dy and (dx or dz) and (tx + dx, y + dy, tz + dz) not in b.blocks: b.put(tx + dx, y + dy, tz + dz, 'spruce_leaves', persistent='true', distance='1') b.box(tx, tx, height - 2, height + 1, tz, tz, 'spruce_leaves', persistent='true', distance='1') # Stone chimney breast and a guarded living-room hearth, away from side windows. b.box(36, 36, 4, 32, 17, 18, CONFIG['stone']) b.box(35, 37, 33, 33, 16, 19, 'stone_brick_slab') b.put(35, 9, 18, 'stone_bricks') b.put(35, 10, 18, 'campfire', lit='true') b.put(35, 10, 17, 'iron_bars', east='true', west='true') b.put(36, 17, 19, 'air') # Terrace edging and planted ground soften the rectangular foundation. for x in range(8, 39): if x not in range(27, 34): b.put(x, 4, 8, 'stone_brick_slab') b.put(x, 4, 28, 'stone_brick_slab') for z in range(9, 28): b.put(39, 3, z, 'stone_brick_slab') for x in range(3, 43): for z in (5, 6, 29, 31): if b.blocks.get((x, 1, z), '').startswith('minecraft:grass_block') and (x, 2, z) not in b.blocks and b.rng.random() < .35: b.put(x, 2, z, b.rng.choice(('short_grass', 'fern', 'poppy', 'dandelion'))) return b def verify_roundtrip(b, path: Path): """Compare every cell including air and all normalized state properties.""" loaded = load_schematic(path) assert loaded.size_yzx == CONFIG['size'] ids = loaded.read_flat(0, loaded.volume).reshape(loaded.size_yzx) actual = {} for y in range(ids.shape[0]): for z in range(ids.shape[1]): for x in range(ids.shape[2]): state = loaded.palette[int(ids[y, z, x])] if state != 'minecraft:air': actual[x, y, z] = state assert actual == b.blocks return {'dimensions_yzx': loaded.size_yzx, 'non_air_blocks': len(actual), 'unique_states': len(set(actual.values())), 'full_state_roundtrip': True, 'sha256': loaded.sha256, 'block_counts': dict(Counter(s.split('[')[0] for s in actual.values()))} def main(): """Run from any clean writable directory with the supplied MCIO installation.""" b = build() b.export(Path('output.litematic')) previews = Path('previews') previews.mkdir(exist_ok=True) for name, bounds in {'ground': (10, 37, 3, 7, 10, 26), 'living': (10, 37, 9, 13, 10, 26), 'bedroom': (25, 37, 15, 20, 10, 26), 'study': (25, 37, 21, 25, 10, 26)}.items(): b.export(previews / (name + '.litematic'), bounds) (previews / 'validation.json').write_text(json.dumps(verify_roundtrip(b, Path('output.litematic')), indent=2)) print((previews / 'validation.json').read_text()[:220]) if __name__ == '__main__': main()
test/test_structure.py — 182 行
# Verify the complete lodge's state, topology, supports and serialized content. from pathlib import Path import sys from collections import deque, Counter import tempfile import pytest sys.path.insert(0, str(Path(__file__).resolve().parents[1])) from generate_lodge import build, CONFIG, REGISTRY, verify_roundtrip from mcio.nbt import tag from mcio.litematica import LitematicaSchematic @pytest.fixture(scope='module') def lodge(): return build() def state(lodge, p): s = lodge.blocks.get(p, 'minecraft:air').removeprefix('minecraft:') name = s.split('[')[0] props = dict(v.split('=') for v in s.split('[')[1][:-1].split(',')) if '[' in s else {} return name, props def neighbors(p): x, y, z = p return [(x + 1, y, z), (x - 1, y, z), (x, y + 1, z), (x, y - 1, z), (x, y, z + 1), (x, y, z - 1)] def passable(lodge, p, doors=True): n, _ = state(lodge, p) return n == 'air' or n == 'ladder' or n.endswith('carpet') or (doors and n.endswith('_door')) or n in ('vine', 'white_wall_banner') def test_registry_and_extent(lodge): for p in lodge.blocks: x, y, z = p assert 0 <= x < 45 and 0 <= y < 41 and 0 <= z < 35 n, props = state(lodge, p) assert n in REGISTRY assert props.keys() == REGISTRY[n][1].keys() assert all(v in REGISTRY[n][0][k] for k, v in props.items()) assert max(y for x, y, z in lodge.blocks) == 39 def test_door_and_bed_pairs(lodge): counts = Counter() for (x, y, z) in lodge.blocks: n, p = state(lodge, (x, y, z)) if n.endswith('_door'): counts['doors'] += 1 other, op = state(lodge, (x, y + (1 if p['half'] == 'lower' else -1), z)) assert other == n and op['half'] != p['half'] assert all(op[k] == p[k] for k in ('facing', 'hinge', 'open', 'powered')) if n.endswith('_bed'): counts['beds'] += 1 dx, dz = {'south': (0, 1), 'north': (0, -1), 'east': (1, 0), 'west': (-1, 0)}[p['facing']] sign = 1 if p['part'] == 'foot' else -1 other, op = state(lodge, (x + sign * dx, y, z + sign * dz)) assert other == n and op['facing'] == p['facing'] and op['part'] != p['part'] assert not passable(lodge, (x, y - 1, z)) assert counts['doors'] >= 2 and counts['beds'] == 4 def test_two_block_headroom_reachability(lodge): def stand(p): x, y, z = p return passable(lodge, p) and passable(lodge, (x, y + 1, z)) and (not passable(lodge, (x, y - 1, z)) or state(lodge, p)[0] == 'ladder' or state(lodge, (x, y - 1, z))[0] == 'ladder') start = (30, 2, 5) assert stand(start) seen = {start} q = deque([start]) while q: x, y, z = q.popleft() candidates = [(x + dx, y + dy, z + dz) for dx, dz in ((1, 0), (-1, 0), (0, 1), (0, -1)) for dy in (-1, 0, 1)] if state(lodge, (x, y, z))[0] == 'ladder': candidates += [(x, y - 1, z), (x, y + 1, z)] for p in candidates: if p not in seen and 0 <= p[0] < 45 and 0 <= p[1] < 40 and 0 <= p[2] < 35 and stand(p): seen.add(p) q.append(p) assert (30, 4, 10) in seen for name, p in CONFIG['rooms'].items(): assert p in seen, (name, p) for fy in CONFIG['floors']: assert (35, fy + 1, 24) in seen def test_closed_envelope(lodge): # Flood from every exterior boundary with doors closed. Glass and roof count as sealed. def empty(p): return passable(lodge, p, doors=False) seen = set() q = deque() for x in range(45): for y in range(41): for z in range(35): if (x in (0, 44) or y in (0, 40) or z in (0, 34)) and empty((x, y, z)): seen.add((x, y, z)) q.append((x, y, z)) while q: for p in neighbors(q.popleft()): x, y, z = p if 0 <= x < 45 and 0 <= y < 41 and 0 <= z < 35 and p not in seen and empty(p): seen.add(p) q.append(p) for name, p in CONFIG['rooms'].items(): assert p not in seen, ('Exterior leak', name, p) # Every interior air column has a solid ceiling before the sky. for x in range(26, 37): for z in range(11, 26): assert any((x, y, z) in lodge.blocks for y in range(27, 40)) def test_supports_and_no_floating_components(lodge): # Every non-air block belongs to a face-connected component anchored in the terrain. seen = {p for p in lodge.blocks if p[1] == 0} q = deque(seen) while q: for p in neighbors(q.popleft()): if p in lodge.blocks and p not in seen: seen.add(p) q.append(p) assert not set(lodge.blocks) - seen, sorted(set(lodge.blocks) - seen)[:20] for (x, y, z) in lodge.blocks: n, p = state(lodge, (x, y, z)) if n in ('candle', 'potted_fern', 'anvil', 'chest', 'lectern') or n.endswith('_carpet') or n in ('fern', 'short_grass', 'poppy', 'dandelion'): assert (x, y - 1, z) in lodge.blocks, (n, (x, y, z)) if n == 'lantern': assert (x, y + (1 if p['hanging'] == 'true' else -1), z) in lodge.blocks if n == 'ladder': assert (x, y, z + 1) in lodge.blocks if n == 'white_wall_banner': assert (x, y, z - 1) in lodge.blocks for e in lodge.entities: assert (int(e['TileX']), int(e['TileY']), int(e['TileZ']) - 1) in lodge.blocks def test_each_room_is_lit(lodge): lights = [p for p in lodge.blocks if state(lodge, p)[0] == 'lantern' or state(lodge, p)[1].get('lit') == 'true'] for name, (x, y, z) in CONFIG['rooms'].items(): assert any(abs(x - lx) + abs(y - ly) + abs(z - lz) <= 10 and y <= ly < y + 5 for lx, ly, lz in lights), name def test_export_roundtrip_and_entities(lodge): # Temp directory is explicitly inside test/ and removed even on failure. with tempfile.TemporaryDirectory(dir=Path(__file__).parent) as tmp: path = Path(tmp) / 'lodge.litematic' lodge.export(path) result = verify_roundtrip(lodge, path) assert result['non_air_blocks'] == len(lodge.blocks) from mcio.nbt.nbt import File as NBTFile nbt = NBTFile.load_regardless_of_gzipped(path) assert len(nbt['Regions']) == 1 assert int(nbt['MinecraftDataVersion']) == 3955 region = next(iter(nbt['Regions'].values())) assert len(region['Entities']) == 2 assert len(region['TileEntities']) == 5 def test_window_clearance_and_textile_data(lodge): # Windows remain glass throughout each opening, never roof or chimney blocks. for y in (5, 11, 17, 23, 29): xs = (13, 19, 28, 33) if y < 15 else ((28, 33) if y < 25 else (30,)) for x in xs: for z in (10, 26): for xx in (x, x + 1): for yy in range(y, y + 3): assert state(lodge, (xx, yy, z))[0] == 'glass' banners = [t for t in lodge.tiles if str(t['id']) == 'minecraft:banner'] assert len(banners) == 4 for tile in banners: p = tuple(int(tile[k]) for k in ('x', 'y', 'z')) assert state(lodge, p)[0] == 'white_wall_banner' assert str(tile['patterns'][0]['pattern']) == 'minecraft:stripe_center' assert str(tile['patterns'][0]['color']) == 'green' lecterns = [t for t in lodge.tiles if str(t['id']) == 'minecraft:lectern'] assert len(lecterns) == 1 and int(lecterns[0]['Book']['count']) == 1

REPORT.md 对比

effort行数字节markdown 小节楼层表写了已知缺陷特点
low556,2555结构规整,承认"看不见的房间比参考图更宽敞"
medium636,2326单列"Visual inspection and fixes",记录了看图后改了什么
high637,7590没有 markdown 标题,全用粗体行分段,机器不好解析
xhigh12711,6957有(单独小节)最长,含 Validation results 与 Render review and fixes 两个独立小节

四份都做到了 BRIEF 要求的三件事:写清设计取舍、列出验证方式、诚实交代未完成项。都提到同一批已知限制: 无头渲染器不还原 Minecraft 着色器光照与粒子、cutaway 导出丢实体导致窗帘条纹/展示框在预览里看不到、没有真机 paste 验证。

各档 REPORT.md 的"已知限制"原文节选
low: This is a block-scale reconstruction, not a pixel-exact copy ... The headless renderer does not reproduce Minecraft shader lighting, smoke particles or all entity/block-entity visuals ... I did not launch a Minecraft client; reachability and light coverage are geometric checks. medium: No known structural failures remain in the implemented checks ... Cutaways remove ceiling supports, so some hanging lights appear unsupported in these inspection views but are attached in the complete schematic ... Furnace `lit` states can reset on game updates without fuel. high: (无 markdown 标题, 以粗体段落给出) 同样声明渲染器不显示旗帜图案与展示框物品, 并说明主楼梯与塔梯各自独立可达。 xhigh: 单独 "## Known limitations" 小节, 逐条列出。

解题方式对比

四档完全一致的部分

差异

维度lowmediumhighxhigh
读源码命令数74910
写生成器的方式一次性 heredoc,炸两次一次性 heredoc,炸一次一次性 heredoc,一次过一次性 heredoc + 自带边界检查
渲染迭代1 轮1 轮2 轮多轮 + 局部特写(forge、bedroom detail)
代码组织单文件 17 KB单文件 19 KB单文件 29 KB42 KB + 独立 structural_checks.py
测试规模8 个 / 7.3 KB9 个 / 6.8 KB11 个 / 13.8 KB12 个 / 7.9 KB + 负控制
对尺寸约束的态度守 45×35守 45×35自行放宽到 47×37自行放宽到 47×37×43
渲染呈现白底白底白底改成深色背景,更像参考图的展示风格
格式化未跑 yapf未跑 yapf未跑 yapf跑了 yapf --style /work/.style.yapf

高档位不是"同样的做法做得更久",而是多做了三件事:把结构检查独立成模块、对同一处做局部特写复查、主动遵守仓库约定(yapf、负控制测试)。


harness 可优化点

下面每条都有轨迹证据,按预期收益排序。

  1. 在 BRIEF 里内嵌 API 速查表。四档开局各花 4–10 条命令读 sketch.py / schematic.py / litematica.py 才知道怎么写块、怎么存。medium 甚至猜错导入路径(mcio.nbt.file)直接失败。给出 LitematicCanvas((y,z,x))canvas.block(pos, material)canvas.save(path, name=..., minecraft_data_version=...)from mcio.nbt.nbt import File as NBTFile 这几行,能省掉几千 input token 和一次失败。
  2. 提供现成渲染脚本模板。四档都自写了功能几乎相同的 render_previews.mjs,并且都去翻渲染器源码调 ambientIntensitieslightDirection。 harness 直接给一个带合理相机/光照默认值的模板(含 cutaway 导出函数),预计每档省 3–6 条命令和一轮渲染。
  3. 提供结构校验库。门/床配对、可达性、封闭性、悬空装饰——四档各写了一份,xhigh 单独写了 13 KB 的 structural_checks.py。 harness 侧提供同一套检查(agent 可调用),既省重复代码,也让不同 run 的"通过"口径一致。
  4. 引导增量写文件。三次失败源于一次性 heredoc 写 20–40 KB python,语法错后整块重来。 BRIEF 里提示"先写骨架再分段追加",或提供一个 patch 工具,可消除 A 类失败。
  5. 坐标系与边界写进 BRIEF。两次越界失败都是 YZX 与 XYZ 混淆、尺寸边界取错。 明确写出 canvas 是 YZX、索引范围 [0, size),并建议给 put() 加断言(xhigh 自己加了,于是第一次就被自己拦住,比 medium 的裸 assert 更早暴露)。
  6. 把结构指标纳入 report.json。现在打分只验"能加载"。四档都自测通过,但那是各自写的标准。 harness 侧独立算门/床配对、可达格数、封闭空腔,才能横向比较,也避免自测自评。
  7. 给 REPORT.md 模板。high 那份没有任何 markdown 标题,跨 run 自动比对会卡住。 BRIEF 给定小节顺序(设计取舍 / 验证 / 渲染复查 / 已知限制)即可。
  8. 缓存友好的只读文档。每档都重复读同样的大文件(headless/index.ts 上千行)。 预生成一份 docs/api_summary.md 放进镜像,可把 input token 压下来——注意四档 input 里 92–95% 已经是 cache 命中,真正省的是首轮。
  9. 渲染器能力清单。四档都花命令去确认旗帜图案、展示框、讲台书在无头渲染器里显示不全,然后在 REPORT 里解释。 BRIEF 直接写明"这些实体在预览里不显示,请用 NBT 断言验证",省一轮排查。
  10. run 之间的 workspace 预热。目前每次从空目录开始。若要做多轮实验,可加 --seed-workspace 选项,从上一次的产物继续,考察改进能力而非从零重建。

数据来源:四个 run 的 out/report.jsonout/session/*,由容器内 harness 从 codex rollout 提取。