MCDiffuse - Architecture & Design

Design Philosophy

System Context

At the highest level, MCDiffuse sits between the Minecraft player and a remote Python diffusion model server. The plugin receives a text prompt, captures the world snapshot, sends a request, and renders the streamed response as holographic previews and final world edits.

Counts as of 2026-07-05 (src/main 191 + vanilla 9 + app 9 + fabric 2 + neoforge 1; @Test methods). These numbers rot — recount before quoting, or treat as order-of-magnitude.

flowchart LR
  Player["Player\n/mcdiffuse ..."] -->|"typed command"| Platform["Platform Bootstrap\nBukkit / Fabric / NeoForge"]
  Platform -->|"CommandContext"| Router["Command Router"]
  Router -->|"routes to"| Cmd["Command Impl"]
  Cmd -->|"Actions"| Adapter["Action Executor\nplatform adapter"]
  Adapter -->|"block edits"| World["Minecraft World"]
  Adapter -->|"HTTP POST"| ModelSrv["Model Server\nPython"]
  ModelSrv -->|"NDJSON stream"| Listeners["Model Response\nListeners"]
  Listeners -->|"Actions"| Adapter
        

Module Map

180 files collapse into 14 named modules. Each module has a designated public surface — the handful of files that other modules are allowed to import. Everything else is module-internal.

flowchart TD
  PB["Platform Bootstrap"] -->|"implements"| CR["Command Router"]
  CR -->|"routes"| C["Commands"]
  C -->|"reads"| GS["Generation State"]
  C -->|"calls"| GL["Generation Logic"]
  GL -->|"produces"| AC["Action Catalog"]
  C -->|"produces"| AC
  AC -->|"executed by"| AE["Action Executor"]
  AE -->|"world reads via"| WS["World Snapshot"]
  AE -->|"sends to"| MC["Model Client"]
  MC -->|"response via"| GL
  AE -->|"renders"| H3D["Holo 3D"]
  AE -->|"renders"| H2D["Holo 2D"]
  AE -->|"opens"| UI["Inventory UI"]
  H3D -->|"packets"| PI["Platform Infra"]
  H2D -->|"packets"| PI
  UI -->|"packets"| PI
        

Command Dispatch Flow

Every player command follows the same path regardless of platform. The queue is the only branching point: if the model is already running for this player, the command waits; otherwise it runs immediately.

flowchart TD
  PE["Platform Event\nonCommand / LiteralArg"] -->|"raw args"| CR["CommandRouter\nparse CommandAction"]
  CR -->|"CommandContext\nplayer, world, args, provider"| CD["CommandDispatcher\nlook up registered command"]
  CD -->|"QueuePolicy START_MODEL"| GCQ["GenerationCommandQueue\nidle?"]
  CD -->|"QueuePolicy RUN_NOW"| CMD["Command.execute()"]
  GCQ -->|"yes"| CMD
  GCQ -->|"no, queue"| Q["Pending Queue\nper player"]
  Q -->|"model done"| CMD
  CMD -->|"Actions"| AE["Action Executor\nBukkitActionAdapter / FabricActionAdapter"]
  AE -->|"switch on type"| Effects["World edits\nDisplay updates\nMessages\nUI opens"]
        

Key design rule: CommandDispatcher is the only place that maps a CommandAction enum to a Command instance. Adding a new command means adding one entry in the dispatch table and one new file in commands/. Nothing else changes.

Queue policies - each command declares one:

Generation Request Flow

When a model command runs, it assembles a GenerationRequestContext from the current generation state and world snapshot, then sends it to the model server. Responses arrive as a streaming NDJSON sequence; each chunk becomes another list of actions.

flowchart LR
  MC["ModelCommand\nShapeCommand\nTerrainGenerateCmd"] -->|"read"| GCS["ContainerStore\ncurrent settings"]
  MC -->|"read"| WS["WorldSnapshotProvider\nblock data + height"]
  GCS -->|"assemble"| RC["GenerationRequestContext\nRequestObject"]
  WS -->|"assemble"| RC
  RC -->|"HTTP POST"| MSC["ModelServerClient\nstreaming"]
  MSC -->|"loading"| MRL_S["ModelResponseListeners\nonModelResponse()"]
  MSC -->|"chunk N"| MRL_C["ModelResponseListeners\nonModelResponse()"]
  MSC -->|"done"| MRL_D["ModelResponseListeners\nonModelResponse()"]
  MRL_S -->|"status action"| AE["Action Executor"]
  MRL_C -->|"RenderPreviewAction\nstore preview"| AE
  MRL_D -->|"CommitBlocksAction\nstore history"| AE
  AE -->|"Holo3D render"| Preview["Holographic Preview"]
  AE -->|"world writes"| World["Minecraft World\non commit"]
  AE -->|"update"| HS["HistoryStore\nPreviewOverlayStore"]
        

Streaming design: Each NDJSON chunk calls onModelResponse() independently. Progress steps produce preview actions; the final step (or any error) produces a terminal action. The caller never polls - it just executes each returned list immediately.

Generation State after a successful generation:

History & Undo/Redo Flow

History is a per-player stack of GenerationContainer snapshots. Undo/redo/commit are all stateless reads of that stack; HistoryActionPreparer translates a stack position into an action list.

flowchart LR
  UC["UndoCommand"] -->|"prepareUndo"| HAP["HistoryActionPreparer"]
  RC["RedoCommand"] -->|"prepareRedo"| HAP
  CC["CommitCommand"] -->|"prepareCommitSelected"| HAP
  HC["HideCommand"] -->|"prepareHideSelected"| HAP
  SC["ShowCommand"] -->|"prepareShowIndexed"| HAP
  HAP -->|"read"| HS["HistoryStore\nper-player stack"]
  HAP -->|"read"| WS["WorldSnapshotProvider\nbaseline snapshot"]
  HAP -->|"Actions"| AE["Action Executor"]
  AE -->|"RestoreWorldBlocksAction"| World["Minecraft World"]
  AE -->|"RenderGeneratedPreviewAction"| H3D["Holographic 3D Display"]
  AE -->|"ClearHistoryRegionDisplayAction"| H3D
        

History entry lifecycle:

Display Pipeline

MCDiffuse renders previews as floating holographic displays using Minecraft armor-stand entities (Holo3D) and block-face projections (Holo2D). The neutral half defines what to show; the Bukkit half decides how to show it via packets.

Holo3D Pipeline

Holo3D is split: 11 neutral files compute cell layout and manage object state; 5 Bukkit files send packets to the player.

flowchart TD
  A["PrepareHolo3DOperationAction\nneutral - describes op"] -->|"executed by"| AE["Action Executor"]
  AE --> H3DP["Holo3DOperationPreparer\nneutral - computes cell layout"]
  H3DP -->|"Holo3DOperationRequest"| H3DL["Holo3DListeners\nneutral - manages object state"]
  H3DL -->|"Holo3DObjectState"| H3DUI["Holographic3DUI\nBukkit - sends packets"]
  H3DUI -->|"spawn/move/destroy\narmor stands"| PL["PacketLayer\nProtocolLib"]
  PL --> PC["Player Client"]
        

Neutral half: Holo3DOperationPreparer, Holo3DListeners, Holo3DObjectState, Holo3DCellKey, Holo3DPivot, Holographic3DDisplayTransform, Holo3DCommandSpec, Holo3DAction, Holo3DOperationRequest, Holo3DOperationSnapshot, Holo3DBrightnessOverride

Bukkit half: Holographic3DUI (entity management), Holographic3DHitTest (raycasting), Holographic3DRenderedCell (entity handle), Holo3DRenderExecutor (scheduler), Holographic3DRenderHandle (lifecycle)

Holo2D Pipeline

Holo2D renders 2D images as Minecraft block faces. All 12 files are Bukkit-only; no neutral split is needed since no other platform requires this subsystem.

flowchart TD
  A["Holo2D Action\nfrom Action Catalog"] -->|"executed by"| AE["Action Executor"]
  AE --> H2D["Holographic2DUI\nBukkit - widget host"]
  H2D -->|"layout"| W["Widget Tree\nholo2d/widget/"]
  W -->|"block-face frames"| RL["Render Layer\nbukkit/render/"]
  RL -->|"map art packets"| PL["PacketLayer\nProtocolLib"]
  PL --> PC["Player Client"]
        

Inventory UI

A Minecraft chest inventory repurposed as a GUI. Slot clicks re-enter the action loop, allowing menus to compose commands.

flowchart TD
  A["OpenMainMenuAction\nOpenModelManagerAction"] -->|"executed by"| AE["Action Executor"]
  AE --> VUI["VirtualUI\nBukkit - chest inventory"]
  VUI -->|"slot layout"| VCB["VirtualUICallback\nslot click handler"]
  VCB -->|"returns Actions"| AE
  VUI -->|"listens"| VUL["VirtualUIListeners\ninventory close / item drag"]
        

Platform Boundary

The neutral/platform split is enforced structurally. The :core Gradle module holds all neutral sources (via srcDir("../src/main/java") + exclude("**/bukkit/**")); :bukkit, :fabric, and :neoforge depend on it, so a Bukkit import in neutral code is a compile error for every platform build. The remaining gap is physical: neutral and Bukkit sources share one directory tree, so the boundary is invisible in the IDE until Gradle compiles :core. Target state is moving neutral sources into core/src/main/java outright.

flowchart LR
  subgraph Neutral ["Neutral - shared by all platforms"]
    direction TB
    Types["Type Bridge\nabstraction/"]
    Actions["Action Catalog\naction/"]
    Router["Command Router\ncommands/core/"]
    Cmds["Commands\ncommands/"]
    GenState["Generation State\ngeneration/store/"]
    GenLogic["Generation Logic\ngeneration/logic/"]
    Holo3DNeut["Holo3D neutral\nholo3d/ 11 files"]
  end
  subgraph BukkitPlat ["Bukkit"]
    direction TB
    BPlugin["Plugin Bootstrap\nMCDiffuse + Config"]
    BExec["Action Executor\nBukkitActionAdapter"]
    BWS["World Snapshot\nBukkitGenerationWorldSnapshotProvider"]
    BH3D["Holo3D Bukkit\n5 files"]
    BH2D["Holo 2D\n12 files"]
    BUI["Inventory UI\n6 files"]
    BInfra["Platform Infra\npacket + render + selection\nhandler + modelmanager"]
  end
  subgraph VanillaShared ["Vanilla shared - Fabric + NeoForge"]
    direction TB
    FExec["Action Executor\nVanillaActionAdapter"]
    FWS["World Snapshot\nVanillaGenerationWorldSnapshotProvider"]
    FSM["Model Client sink\nVanillaStreamingModelServerClient"]
    FBrig["Brigadier tree\nVanillaBrigadierCommands"]
  end
  subgraph FabricPlat ["Fabric bootstrap"]
    FPlugin["MCDiffuseFabric"]
  end
  subgraph NeoForgePlat ["NeoForge bootstrap"]
    NPlugin["MCDiffuseNeoForge"]
  end
  FabricPlat -->|"events + metadata"| VanillaShared
  NeoForgePlat -->|"events + metadata"| VanillaShared
  Neutral -->|"compile dep"| BukkitPlat
  Neutral -->|"compile dep"| VanillaShared
        

Execution path comparison: same command on two platforms

Steps 2-4 are identical across all three platforms. Platform code is only ever at the edges: receiving player input (step 1) and executing actions against the world (steps 5-6). NeoForge shares the Fabric implementation via the loader-agnostic vanilla/ source tree (see Blind Spot #2).

Blind Spots & Design Debt

The philosophy above describes the intended shape. This section records where the implementation diverges from it, why that matters for maintainability, and what the fix looks like. Verified against source on 2026-07-05.

1. Action parity is asymmetrically enforced

MCDiffuseAction is a sealed interface. BukkitActionAdapter dispatches with an exhaustive switch, so adding an action without a Bukkit handler is a compile error. FabricActionAdapter dispatches with an if instanceof chain and a fallthrough that string-matches class names (className.contains("Holo"), "History", "Display", ...) to decide whether to log at FINE or WARNING — then skips either way. Consequences:

• Fabric implements 21 of 34 action types. A Fabric player running a Holo3D or history-display command gets silent no-ops, not an error message.
• A renamed action class silently changes which log branch it hits; a new action whose name happens to contain "Display" is silently swallowed instead of warned.
• The philosophy "commands return data, adapters execute" implicitly promises platform-equivalent behaviour that nothing enforces or even measures.

Fix: exhaustive switch in every adapter (the sealed interface already pays for it). Unsupported actions get explicit case arms that produce a player-visible "not supported on Fabric" message, and the support matrix becomes greppable code instead of string heuristics.

Status: fixed 2026-07-05. FabricActionAdapter now dispatches via exhaustive sealed switch — a new action type without a Fabric arm is a compile error. Player-initiated unsupported actions (7) send a yellow "not supported on Fabric yet" chat notice; downstream Holo3D plumbing actions (6) log at FINE only, since their entry point already messaged the player. Feature coverage is still 21/34 implemented — the parity enforcement is fixed, the missing Fabric features are separate work.

2. Two command architectures coexist

Bukkit and Fabric enter through CommandRouter → Command.execute() → actions. NeoForge bypasses all of it: MCDiffuseNeoForge builds an MCDiffuseApp facade (:app) exposing executeCommand() / executeSetModelDevice() returning CompletableFuture<MCDiffuseCommandResult> — a second, parallel dispatch system with its own result type. "3 platforms" in the KPI overstates parity: only two ride the action pipeline. Every new command now poses the question "which system?", and the facade will accrete duplicates of router features (queueing, arg parsing) one at a time.

Fix: pick a target. Either NeoForge grows a NeoForgeActionAdapter and joins the pipeline, or the facade is documented as a deliberately minimal tier with a frozen command surface. The worst outcome is both systems growing indefinitely.

Status: fixed 2026-07-05. One architecture remains. Key insight: Fabric and NeoForge both use official Mojang mappings, and Fabric's implementation was already ~90% loader-agnostic (zero net.fabricmc imports in the adapter, renderer, snapshot provider, selection state, and streaming sink). Those files moved to a shared vanilla/ source tree (me.kokecacao.mcdiffuse.vanilla, classes Vanilla*) compiled into both loader modules via srcDir — the same sharing mechanism :core uses. Each loader keeps only a thin bootstrap (~1 file): mod init, metadata lookup, event-bus registration (Fabric callbacks vs NeoForge @SubscribeEvent), selection-tool interaction hook. NeoForge now has full Fabric-parity gameplay through CommandRouter → VanillaBrigadierCommands → VanillaActionAdapter. The MCDiffuseApp facade and its family (MCDiffuseCommandResult, MCDiffuseAppConfig, MCDiffusePlatform, both platform schedulers, ModelServerDisplayFormatter) were deleted as dead code. ModuleSurfaceTest.vanillaTreeStaysLoaderAgnostic locks the new boundary: any net.fabricmc/net.neoforged import in the shared tree fails the build. 2026-07-06 follow-up (P4): the last duplicated bootstrap logic — the ~30-line selection-tool right-click handler and the 17-line resolveSelectionToolItem, byte-identical in both bootstraps — moved to VanillaSelectionToolHandler in the shared tree; each loader now keeps only its event plumbing (Fabric returns InteractionResult, NeoForge cancels the event).

3. Model client is implemented three times

HTTP + NDJSON streaming exists as ModelServerClient (Bukkit, 409 lines), FabricStreamingModelServerClient (230 lines), and SharedModelServerClient (:app, 164 lines). java.net.http is platform-neutral — nothing about streaming NDJSON requires Bukkit or Fabric. The philosophy constrains what neutral code may touch, but says nothing about what platform code may host, so neutral logic leaked into platform modules where it now drifts (retry policy, error parsing) in triplicate.

Fix: add the missing dual of the boundary rule — platform modules contain only code that needs platform APIs — and collapse the three clients into one neutral streaming client injected with a platform scheduler for thread-hopping.

Status: fixed 2026-07-05. Neutral StreamingModelServerClient (in modelserver/, compiled into :core) now owns the HTTP + NDJSON loop, transport-failure classification, and the error-category constants; platform events alias those constants. Platform classes became thin sinks with unchanged public APIs: Bukkit's ModelServerClient wraps chunks into ModelResponseEvent and delegates JSON endpoints to SharedModelServerClient; Fabric's client converts blocks against its registry and hops via server.execute. Design rule adopted: the neutral client never hops threads — each sink performs its own platform hop. Registry-validating block conversion stays platform-side by necessity (Bukkit.createBlockData / Fabric block registry). Covered by StreamingModelServerClientTest against a local HTTP server. 2026-07-05 follow-up: the vanilla sink originally called ModelResponseListeners.onModelResponse on HTTP client threads, racing main-thread store reads (the neutral stores are unsynchronized by design); the sink now hops to the server thread before the listener call, mirroring Bukkit's scheduler-first order — thread confinement is documented as store precondition.

4. Store lifecycle is unmanaged

"State in Stores" is implemented as static HashMaps with static methods (GenerationContainerStore, GenerationContainerHistoryStore). Three costs the philosophy never mentions:

• Unbounded growth: history entries hold full world-block baselines, and there is no per-player eviction on quit — only an all-or-nothing clear(). A long-running server accumulates snapshots for every player who ever generated.
• Test isolation: every test that touches a store must remember to clear(); forgetting produces order-dependent tests.
• No instancing: static state means one store per JVM, which blocks Folia-style multi-region servers and side-by-side test fixtures.

Fix: per-player eviction hooks on quit (bounded ring per player at minimum), and long-term, instance stores owned by the bootstrap and passed through CommandContext — the injection plumbing already exists.

Status: fixed 2026-07-05 (revised diagnosis). Quit-eviction of history was rejected on purpose: history is designed to survive sessions — undo works after rejoin, and RuntimeStatePersistenceCodec persists containers/history/displays across restarts (Holo3D state is likewise parked, not evicted, on quit). Evicting on quit would silently break that contract. Instead memory is bounded at the source: GenerationContainerHistory.push() now returns dropped container ids (overwritten redo branches were the real leak — dropped ids previously left orphaned block volumes in GenerationContainerStore forever, which had no remove API), the history store evicts unreferenced dropped containers, and a depth cap (HISTORY_MAX_UNDO_DEPTH = 16, baseline never dropped) bounds each region stack. Session-only state is evicted on quit: PreviewBlockOverlayStore.clearPlayer() existed but had zero callers; the quit handler now calls it. Test-isolation and instancing costs remain open. 2026-07-05 P2 follow-up: shadow stores repaired — ModelResponseListeners’ three hidden static maps extracted into ModelResponseStateStore (the listener is now the stateless translator the doc always claimed), Holo3DOperationPreparer’s object registry extracted into Holo3DObjectStateStore (fixing the false claim that Holo3DListeners manages state), VirtualUI’s public static map encapsulated behind register/get/remove, debug test-render handles now evicted on quit, and the remaining module-internal registries were blessed with explicit store-role comments and eviction paths.

P3 stage 1 (2026-07-05): committed long-term direction — one bootstrap-owned runtime aggregate with instance stores behind interfaces, reached in shippable stages. Stage 1 done: commands now read generation state only through the injected GenerationStateView (CommandContext.stores(), store-backed default impl StoreBackedGenerationStateView); the history store’s nested result records moved to generation/data as top-level records, misplaced pure helpers moved out (CommandArgs.parseHistoryIndex, DisplayText); zero generation/store imports remain under commands/ (arch-test enforced); command tests can inject mock views with no store setup or teardown (GenerationStateViewSeamTest). Stage 2: instance stores behind unchanged interfaces. Stage 3: aggregate into listeners/preparers.

P3 stage 2 (2026-07-06): store state is now instance state. New instance classes ContainerStore, HistoryStore (constructor-wired to its container store so eviction frees volumes in the same runtime), ModelResponseState, PreviewOverlayStore, Holo3DObjectStates are owned by a new GenerationState aggregate. The old static store classes remain as thin facades delegating to GenerationState.DEFAULT so the ~60 listener/preparer/adapter/persistence call sites stay put until stage 3, which moves aggregate ownership into the platform bootstraps and deletes the facades. Payoff now: isolated fixtures — new GenerationState() per test, zero clear() discipline (GenerationStateIsolationTest proves container, history-eviction wiring, model-response, and Holo3D isolation).

P3 stage 3 (2026-07-06) — done; item closed. Static generation state is gone. Each platform bootstrap owns one aggregate: MCDiffuse (Bukkit) holds a generationState field (lazy accessor for CALLS_REAL_METHODS partial mocks), MCDiffuseFabric and MCDiffuseNeoForge each construct one and wire it into CommandRouter, ModelResponseListeners, Holo3DListeners, the streaming sink, and VanillaActionAdapter.init. GenerationCommandQueue became an instance owned by the aggregate (state.queue()); preparers take a leading GenerationState parameter; Bukkit handlers reach state via plugin.generationState(); SelectionListeners takes an injected HistoryStore; RuntimeStatePersistenceCodec and Holo3DRenderExecutor.flushPlayerState take explicit state parameters. The five facades, GenerationState.DEFAULT, and StoreBackedGenerationStateView.INSTANCE were deleted; BlockKey moved to its own top-level record in generation/store. Tests seed per-fixture aggregates (mock plugins stub generationState()). Remaining honest caveat: Holo3DRenderExecutor's live render-handle registry and SelectionListeners' cursor registry are still static module-internal stores at the Bukkit edge — session-only packet state, quit-evicted, out of this item's scope.

5. Conventions are prose, not checks

"Narrow module interfaces", "internal files talk only to siblings", and "file names = control flow" are enforced by nothing. The module table's public surface column is aspiration; any file can import any other file inside the source tree and no build step objects. Conventions that survive only by review attention decay at exactly the rate reviewers get busy.

Fix: encode the module table as an architecture test (ArchUnit or a simple import-grep test in CI) that fails when a module is imported outside its declared surface. The table in this document then becomes generated-from or verified-against code instead of hand-maintained.

Status: fixed 2026-07-05. Drift was fixed in code first, then locked: ShowHistoryRegionDisplayAction/ClearHistoryRegionDisplayAction constructors no longer call GenerationContainerHistoryStore.validateHistoryOwner — ownership checks live in the preparers that construct them (HistoryDisplayPreparer validates explicit history args; HistoryActionPreparer only resolves from the player's own stack) — so the "plain data records" claim is true again. New ModuleSurfaceTest (same source-walking pattern as the existing raw-volume-array boundary test) enforces four rules: command implementations importable only by the dispatch table; action catalog may not import behavior modules; cross-module imports must hit each restricted module's declared surface (the table above now mirrors the enforced allowlist); neutral sources may not import platform APIs. Verified by mutation: each forbidden-import injection fails the corresponding check. Still unchecked: bukkit/* internal layering.

6. This document itself

Hand-written counts (file totals, action counts, test counts) and hand-drawn module tables drift from source — several KPIs here were stale until the 2026-07-05 audit, and the platform-boundary section described an already-completed migration as future work. Prefer describing invariants (which are stable) over inventories (which rot), and date-stamp anything counted.