MirrorCraft: Paired Evaluation under Hidden Rule Changes in Minecraft

Jianxin Gao 1 \equalcontrib , Beini Hu 2 \equalcontrib , Runze Li 3 \equalcontrib , Wanli Peng 1 \corresponding Ruohan Lei 1 , Jinyuan Zhang 1 , Linna Deng 1 , Tianyi Yu 4 , Zining Wang 5

Abstract With the prosperity of the large language models (LLMs), it has become an interesting topic: how do LLM-based agents work in Minecraft? Unfortunately, most existing benchmarks evaluate them under fixed game mechanics. High performance in these settings does not show whether an agent can continue making progress when familiar recipes, drops, and other rules change. In this paper, we introduce MirrorCraft, a paired benchmark for evaluating agents under hidden rule changes in Minecraft. Each Mirror world is a copy of its paired Vanilla world, with selected server-side rules modified by the corresponding datapack. Terrain, spawn, resource placement, objective, interface, and action budget remain matched within every Vanilla-Mirror pair. MirrorCraft includes five controlled biomes, six rule suites, three progression objectives, two model families, and six agent configurations under a shared Mineflayer interface. We evaluate task progress with deterministic advancement milestones and success rate and use the Rule Intervention Effect (RIE) to measure the performance change between matched Vanilla and Mirror worlds. The experiments show that hidden rule changes have strongly different effects across suites. Among the configurations evaluated without rule descriptions, ReAct achieves the highest pooled Mirror score. Providing the exact rules yields modest gains in average progress and completion across all three objectives. MirrorCraft extends Minecraft evaluation beyond fixed mechanics and provides a controlled setting for studying how agents use gameplay outcomes when the rules of the current world differ from familiar ones.

Introduction

Minecraft is widely played and documented online (Fan et al. 2022 ) , so agents may draw on familiar mechanics encoded during language model pretraining rather than infer the rules of the current world (Yu and Lu 2025 ) . High performance under fixed mechanics may therefore reflect such priors rather than inference from outcomes in the current world. When recipes, drops, or tool requirements change, one wrong assumption can distort later plans; evaluation must distinguish performance under changed rules from the difference relative to standard rules.

MineRL, MineDojo, MCU, MineExplorer, and SciCrafter cover diverse tasks, generalization, exploration, and parameterized discovery while keeping game mechanics fixed (Guss et al. 2019 ; Fan et al. 2022 ; Zheng et al. 2025 ; Ju et al. 2026 ; Zhou et al. 2026 ) . High performance on these benchmarks therefore does not show whether an agent can continue making progress when familiar recipes, drops, and other rules change. Mars, NovelGym, and NovelCraft evaluate changed mechanics or novelty against a default or unperturbed condition, while XENON studies an agent designed to revise task knowledge after perturbations (Tang et al. 2024 ; Goel et al. 2024 ; Feeney et al. 2023 ; Lee et al. 2026 ) . Table 1 distinguishes online evaluation, changed mechanics or introduced novelty, and whether standard and changed runs begin from matched copies of the same world. None of the prior benchmarks with changed mechanics in the table, however, uses this form of pairing.

Benchmark(s) World Online Change

Matched copies

MineRL / MineDojo MC ✓ \checkmark – –

MCU / MineExplorer MC ✓ \checkmark – –

SciCrafter MC ✓ \checkmark – –

Mars Crafter ✓ \checkmark ✓ \checkmark –

NovelCraft PolyCraft – ✓ \checkmark –

NovelGym Grid ✓ \checkmark ✓ \checkmark –

MirrorCraft MC ✓ \checkmark ✓ \checkmark ✓ \checkmark

Table 1: Representative benchmark protocols. “Matched copies” denotes separate standard and changed runs initialized from copies of the same world; MC denotes Minecraft.

We introduce MirrorCraft, a paired benchmark for hidden rule changes in Minecraft Java 1.19. For every Vanilla world under standard mechanics, six paired Mirror worlds are copied from its save, each loading the datapack for a different rule suite. The Vanilla world and its Mirror versions use the same terrain, spawn, placed resources, task, observation interface, action interface, and action budget. Ordinary actions reveal their outcomes, but the agent is not told which rules have changed. This design compares standard and modified rules without changing the starting map or the way the agent interacts with the game.

MirrorCraft contains 10 Vanilla worlds across five controlled biomes, six rule suites, and three progression tasks: Iron Armor , Diamond , and Enchantment . The main study evaluates two models and six agent configurations in both Vanilla and Mirror worlds through the same Mineflayer body and semantic skills. A separate rule-disclosure condition gives ReAct the exact Mirror rules. Milestones verified by the server provide a Score for partial progress, while success rate (SR) records completion of the final objective. For the same Vanilla world, model, task, agent configuration, and action budget, RIE SC \mathrm{RIE}{\mathrm{SC}} and RIE SR \mathrm{RIE}{\mathrm{SR}} report the average Vanilla result minus the corresponding paired Mirror result for Score and SR. Reporting Mirror performance together with the two RIEs separates performance under modified rules from the change relative to standard rules.

The main study contains 8,640 episodes over the 10 Vanilla worlds and six designed rule suites. The RIE values vary in both sign and magnitude across suites, so Mirror worlds are not uniformly harder than their Vanilla versions. Among the six configurations that do not receive rule descriptions, ReAct achieves the highest pooled Mirror Score. The configurations with the smallest RIE values do not always achieve the strongest Mirror performance, showing why both quantities are needed. For ReAct, providing the exact rules improves average Score and SR for all three tasks, but still does not ensure completion.

Our contributions are threefold. First, we introduce a Minecraft benchmark that changes selected rules while preserving the starting world and interaction protocol used for comparison. Second, we provide a common agent interface, measures of progress and completion verified by the server, and two RIE measures that compare each Vanilla world with its paired Mirror worlds. Third, our 8,640-episode main study reveals substantial variation across rule suites and persistent ReAct failures even when the rules are disclosed.

Related Work

Minecraft agents.

Minecraft agents combine language model planning with reusable skills, feedback, and structured knowledge (Wang et al. 2023b , a ; Zhu et al. 2023 ; Wang et al. 2025 ; Liu et al. 2025 ) . Goal-Oriented Graphs retrieve prerequisite chains from Minecraft knowledge sources (Leung et al. 2026 ) . ReAct tracks recent reasoning, actions, and observations, Reflexion converts feedback into verbal reflections, Voyager retrieves reusable procedures, and XENON and ADAM revise task relations from outcomes (Yao et al. 2023 ; Shinn et al. 2023 ; Wang et al. 2023a ; Lee et al. 2026 ; Yu and Lu 2025 ) . MirrorCraft evaluates Direct LLM and these five configurations through the same Mineflayer interface (PrismarineJS 2026 ) ; all share observations, semantic skills, action budgets, and state resets between episodes. Agents that act directly on pixels instead emphasize visual perception and control through primitive actions (Li et al. 2024 , 2025 ; Park et al. 2025 ) .

Minecraft capability benchmarks.

MineRL and MineDojo provide diverse tasks and demonstrations, MCU studies compositional generalization, SmartPlay probes abilities across six games, and OpenHA compares action abstractions across more than 800 tasks (Guss et al. 2019 ; Fan et al. 2022 ; Zheng et al. 2025 ; Wu et al. 2024 ; Wang et al. 2026 ) . Crafter uses achievements to cover several abilities, MineExplorer emphasizes exploration by filtering tasks that depend heavily on Minecraft knowledge, and SciCrafter connects discovery with application through parameterized redstone construction (Hafner 2022 ; Ju et al. 2026 ; Zhou et al. 2026 ) . More focused environments test whether a crafting plan is feasible or whether an agent can complete spatial construction (Dagan et al. 2025 ; Wei et al. 2025 ) . These benchmarks broaden task coverage or isolate capabilities but generally retain fixed mechanics. MirrorCraft keeps familiar progression tasks and Vanilla worlds but changes selected mechanics required by those tasks.

Common interfaces and progress measures.

BrowserGym provides a common observation and action interface for web agents, whereas AndroidWorld combines dynamic tasks with programmatic initialization and success checks (de Chezelles et al. 2025 ; Rawles et al. 2025 ) . AgentBoard complements binary success with graded progress, and GameWorld compares raw and semantic game interfaces using progress and completion verified from environment state (Ma et al. 2024 ; Ouyang et al. 2026 ) . ProEvolve extends this line of evaluation to changes between versions of programmable tool environments (Li et al. 2026 ) . MirrorCraft likewise uses one runtime and outcomes verified by the server so that the agent body and success judge remain constant across configurations. It reports Score and SR separately for Vanilla and Mirror worlds. The two RIE measures then subtract the paired Mirror result from the Vanilla result for comparisons that use the same Vanilla world, model, task, and agent configuration.

Changed mechanics and novelty.

Benchmarks that alter environment mechanics ask a different question from benchmarks that only expand task coverage. Mars is the closest benchmark because it changes terrain, survival settings, and task dependencies in Crafter to study reasoning when mechanics contradict familiar expectations (Tang et al. 2024 ) . NovelGym composes changes to entities, recipes, actions, transition dynamics, and costs in a gridworld inspired by Minecraft (Goel et al. 2024 ) . NovelCraft provides multimodal episodes from PolyCraft, a modified Minecraft environment for novelty detection and category discovery, while NovelGridWorlds evaluates planning and learning after novelty is introduced (Feeney et al. 2023 ; Goel et al. 2021 ) . XENON revises dependency and action knowledge from experience, while WALL-E learns symbolic rules from trajectories for world model alignment (Lee et al. 2026 ; Zhou et al. 2025 ) . Several include a default, standard, or pre-novelty condition, but none evaluates the standard and changed conditions in separate runs initialized from copies of the same world. For every comparison, MirrorCraft preserves terrain, spawn, placed resources, task, interface, and action budget while changing selected rules. This design supports both specialized methods and general agents, and reports performance in the Mirror world separately from the difference relative to Vanilla.

The MirrorCraft Benchmark

MirrorCraft evaluates how rule interventions affect agent progress. For every Vanilla world under standard gameplay rules, we create six paired Mirror worlds by copying its save. Each copy loads the datapack for its rule intervention suite (M01–M06); the datapack modifies selected gameplay rules. Terrain, spawn, placed resources, task, interfaces, and action budget remain fixed within each pairing.

Vanilla and Mirror Worlds

We construct two Vanilla worlds in each of five controlled biomes: Plains, Taiga, Snowy Plains, Jungle, and Savanna. A biome datapack defines surface blocks, vegetation, water margins, and all required task resources and remains unchanged in each Vanilla world and its copies. Two worlds per biome yield 10 Vanilla worlds with validated solution routes; six copies of each yield 60 paired Mirror worlds. A fresh world copy resets inventory, advancements, entities, dropped items, and block state; agent state is cleared separately before every episode. Figure 1 summarizes the three stages of evaluation and separates observations available through Mineflayer from the modified rules hidden from the agent.

Figure 1: Overview of benchmark construction and paired evaluation in Vanilla and Mirror worlds. The paired Mirror world is copied from the Vanilla save, evaluated under matched controls, and loads the datapack for its rule intervention suite.

Rule Suites

Each rule suite changes material availability or the route needed to complete a progression task. In the hidden setting, a valid intervention remains undisclosed, produces an observable outcome when triggered, affects a dependency used by at least one task, and preserves a complete route to every objective.

A static program traces milestone dependencies and identifies editable block drops, entity loot, and crafting or processing recipes. An LLM proposes candidate manifests, and an LLM judge screens them for relevance, observability, coherence, and diversity. The static program compiles accepted manifests into Java 1.19 datapacks, after which deterministic tests verify loading, declared outcomes, hidden-rule integrity, and complete solution routes through the available skills. Together, this pipeline supports batch datapack construction without using an LLM to evaluate agent episodes or determine success. Complete manifests for M01–M06 are provided in the supplementary material.

The six suites implement five forms of route change. M01 reduces several material yields and creates quantity scarcity. M02 and M06 redistribute useful task items as byproducts of ordinary actions. M03 expands yields and replaces selected outputs with more useful items. M04 replaces the standard mineral drops of coal, iron, diamond, and lapis ores with the corresponding ore blocks and adds a bookshelf drop. M05 replaces the familiar pickaxe route with a route involving a blast furnace and several additional steps. Because each suite modifies connected rules, an observed outcome can alter later progression.

Information and Action Interface

Every configuration uses the same Mineflayer interface and receives the same observation fields: inventory, position, task milestones, nearby blocks and entities, action outcomes, available skills, equipped items, survival state, and biome (Figure 1 ). It may retain configuration-specific state within an episode.

In the hidden setting, the agent receives no description of the rules modified by the active datapack and can infer a modification only from the observed outcome of an affected action. Crafting reports the produced item, mining reports the broken block and collected drops, attacks report collected loot, and processing reports the inventory change. The agent cannot query recipes, loot tables, processing rules, datapacks, or server registries. Clearly marked reference grids for standard Java 1.19 recipes may be supplied, but they are identical in Vanilla worlds and paired Mirror worlds and never query the connected server.

The Mineflayer body provides skills for exploration, mining, collection, crafting, smelting, combat, enchanting, and limited obstacle recovery. The craft_grid skill accepts an explicit grid and workstation rather than a desired output, exposing the item produced under current rules. During a call, the body may equip an available tool, collect nearby drops, or clear at most one local obstruction; it cannot create missing items, select semantic subgoals, inspect rules, override the chosen action, or retry indefinitely.

Progression Tasks and Scoring

We evaluate three increasingly deep objectives along one Minecraft progression route: Iron Armor , Diamond , and Enchantment . Each task has an ordered sequence of server-verified advancement milestones. Score records the deepest completed milestone, whereas success rate (SR) records whether the final objective is completed; the final milestone has value 100. Together, the metrics distinguish partial progress from task completion.

Let 𝒞 t \mathcal{C}{t} be the milestone set for task t t , let w t ​ ( c ) w{t}(c) be the value of milestone c c , and let 𝒜 ​ ( e ) \mathcal{A}(e) be the advancements completed in episode e e . The episode Score and aggregate SR are

Score ⁡ ( e , t ) \displaystyle\operatorname{Score}(e,t) = max ⁡ ( { 0 } ∪ { w t ​ ( c ) : c ∈ 𝒜 ​ ( e ) ∩ 𝒞 t } ) , \displaystyle=\max\left({0}\cup{w{t}(c):c\in\mathcal{A}(e)\cap\mathcal{C}{t}}\right), (1)

SR ⁡ ( t ) \displaystyle\operatorname{SR}(t) = 1 N t ​ ∑ e = 1 N t 𝟏 ​ [ Score ⁡ ( e , t ) = 100 ] . \displaystyle=\frac{1}{N{t}}\sum{e=1}^{N_{t}}\mathbf{1}[\operatorname{Score}(e,t)=100]. (2)

Using only the deepest milestone avoids treating milestones on the same route as independent rewards. Score ranges from 0 to 100, but equal differences need not represent equal task difficulty, so we interpret it with SR and results for individual tasks. N t N_{t} includes every valid episode, including deaths, budget exhaustion, and failures to produce an executable action. The three tasks receive equal weight in pooled results. The displayed milestones reveal evaluation progress but not the modified dependencies needed to reach them.

Rule Intervention Effects

Score and SR measure performance within Vanilla and paired Mirror worlds. The Rule Intervention Effects quantify the performance difference between a Vanilla world and a paired Mirror world copied from it. In the paired Mirror world, a datapack modifies selected gameplay rules; the model, task, agent configuration, interfaces, and action budget are held fixed. For each Vanilla world, model, task, and agent configuration, we first average the three Vanilla repetitions. We separately average the three repetitions in each paired Mirror world and compare that mean with its Vanilla counterpart. Let 𝒫 \mathcal{P} contain the resulting comparisons. For comparison p ∈ 𝒫 p\in\mathcal{P} , let S ¯ p V \bar{S}{p}^{\mathrm{V}} and S ¯ p M \bar{S}{p}^{\mathrm{M}} be the mean Scores in the Vanilla world and its paired Mirror world, and let Y ¯ p V \bar{Y}{p}^{\mathrm{V}} and Y ¯ p M \bar{Y}{p}^{\mathrm{M}} be the corresponding mean success percentages. We define

RIE SC \displaystyle\mathrm{RIE}{\mathrm{SC}} = 1 | 𝒫 | ​ ∑ p ∈ 𝒫 ( S ¯ p V − S ¯ p M ) , \displaystyle=\frac{1}{|\mathcal{P}|}\sum{p\in\mathcal{P}}\left(\bar{S}{p}^{\mathrm{V}}-\bar{S}{p}^{\mathrm{M}}\right), (3)

RIE SR \displaystyle\mathrm{RIE}{\mathrm{SR}} = 1 | 𝒫 | ​ ∑ p ∈ 𝒫 ( Y ¯ p V − Y ¯ p M ) . \displaystyle=\frac{1}{|\mathcal{P}|}\sum{p\in\mathcal{P}}\left(\bar{Y}{p}^{\mathrm{V}}-\bar{Y}{p}^{\mathrm{M}}\right). (4)

Positive values indicate lower performance in paired Mirror worlds, while negative values indicate higher performance in paired Mirror worlds. We report both RIE measures because an intervention can affect partial progress and final completion differently.

Each RIE term compares a Vanilla world with a paired Mirror world copied from it under the same model, task, configuration, interfaces, and action budget. It includes every valid episode, even if the agent never attempts an affected action. RIE therefore combines changes in route requirements, exposure to modified outcomes, and subsequent use of those outcomes; it does not isolate adaptation after an encounter.

To measure the value of direct rule access, we follow Mars (Tang et al. 2024 ) and compare ReAct under two disclosure conditions in paired Mirror worlds. ReAct receives no rule descriptions, while ReAct w/rules receives an exact description of the modified rules. They otherwise use the same task, interface, action budget, and ReAct implementation. We define Δ ​ Score \Delta\mathrm{Score} and Δ ​ SR \Delta\mathrm{SR} as the disclosed result minus the hidden result. These differences measure the value of rule descriptions within paired Mirror worlds and are distinct from the two RIE measures.

Behavior after Rule Encounters

Score, SR, and RIE characterize task outcomes but not the actions that follow a modified gameplay outcome. A server evaluator invisible to the agent records every action that triggers such an outcome. For each changed rule, the first action that triggers its modified outcome in an episode defines a rule encounter. An encounter is eligible for Immediate Repeat Rate (IRR) when another action that consumes budget follows it.

We map each action a a to a semantic signature σ ​ ( a ) \sigma(a) . For craft_grid , the signature contains the workstation and grid; for smelt , the input and device; for mine , the block type; for attack , the mob type; for place , the item and use; and for enchant , the item and option. All other skills use the skill name. The signatures ignore crafting quantity, smelting fuel and quantity, mining quantity and coordinates, attack entity identity, and placement coordinates. Mining the same block type at different locations therefore counts as the same semantic action.

Let ℰ + \mathcal{E}^{+} contain encounters with a subsequent action, and let i e i_{e} be the index of encounter e e . IRR is

IRR = 1 | ℰ + | ​ ∑ e ∈ ℰ + 𝟏 ​ [ σ ​ ( a i e + 1 ) = σ ​ ( a i e ) ] . \operatorname{IRR}=\frac{1}{|\mathcal{E}^{+}|}\sum{e\in\mathcal{E}^{+}}\mathbf{1}!\left[\sigma(a{i{e}+1})=\sigma(a{i_{e}})\right]. (5)

IRR measures immediate reuse of the same semantic action signature but does not determine whether the repetition was useful.

Recovery Latency (RL) measures the number of semantic action steps from a rule encounter to the first increase above the Score before that encounter. For encounter e e , let S k S{k} be the milestone Score after action k k in the same episode, with S 0 S{0} denoting the initial Score. Then s e − = S i e − 1 s{e}^{-}=S{i{e}-1} is the Score immediately before encounter action a i e a{i_{e}} . Define

j e = min ⁡ { j ≥ i e : S j > s e − } . j{e}=\min{j\geq i{e}:S{j}>s{e}^{-}}. (6)

If the encounter action increases the Score, j e = i e j{e}=i{e} and the latency is zero. Let ℛ \mathcal{R} contain encounters for which j e j_{e} exists. We report

RL = 1 | ℛ | ​ ∑ e ∈ ℛ ( j e − i e ) . \operatorname{RL}=\frac{1}{|\mathcal{R}|}\sum{e\in\mathcal{R}}(j{e}-i_{e}). (7)

An encounter is excluded if Score never exceeds its value before the encounter at any point from that action onward. RL therefore describes timing only when progress resumes; it measures neither recovery frequency nor causal adaptation. IRR and RL are secondary diagnostics reported with Score, SR, and the two RIE measures.

Integrity and Reachability Controls

Before evaluation, we verify the game version, datapack loading, advancements, spawn conditions, and placed resources in every Vanilla world and paired Mirror world, together with an executable route for every task under every rule suite. These tests establish reachability through the available skills but do not require equal route lengths, which may change under an intervention while the action budget remains fixed.

For each Vanilla world, a scripted test executes the same action sequence there and in its paired Mirror worlds without triggering a changed rule; the serialized observations must match. It also verifies that the interface cannot query server rules, rule text, or an identifier for a paired Mirror world. Missing datapacks, unavailable advancements, and disconnections are treated as infrastructure faults and rerun with the same episode identifier. Deaths, exhausted budgets, ineffective strategies, and policy errors remain evaluation outcomes.

Experiments

Experimental Setup

Table 2 pools outcome measures and evaluation costs by configuration. The six agent configurations receive no rule descriptions; ReAct w/rules is a separate condition receiving exact descriptions only in paired Mirror worlds. Actions and API cost are averaged over paired Mirror episodes.

Gemini DeepSeek

Configuration Vanilla Mirror RIE SC \mathrm{RIE}{\mathrm{SC}} RIE SR \mathrm{RIE}{\mathrm{SR}} Actions Cost Vanilla Mirror RIE SC \mathrm{RIE}{\mathrm{SC}} RIE SR \mathrm{RIE}{\mathrm{SR}} Actions Cost

Direct LLM 70.6 / 13.3 65.1 / 12.4

• 5.5 +5.5
• 0.9 +0.9 56.1 0.05 51.3 / 25.6 47.7 / 8.0
• 3.7 +3.7

• 17.6 +17.6 54.6 0.01

  ReAct 94.4 / 66.7 89.7 / 54.8

• 4.7 +4.7
• 11.9 +11.9 50.8 0.10 90.4 / 61.1 83.8 / 51.1
• 6.5 +6.5

• 10.0 +10.0 51.5 0.04

  Reflexion 92.8 / 63.3 90.0 / 57.4

• 2.8 +2.8
• 5.9 +5.9 50.6 0.10 83.0 / 55.6 78.7 / 46.3
• 4.3 +4.3

• 9.3 +9.3 52.0 0.04

  Voyager 92.1 / 62.2 88.3 / 56.1

• 3.8 +3.8
• 6.1 +6.1 50.0 0.07 78.3 / 52.2 76.3 / 42.6
• 2.0 +2.0

• 9.6 +9.6 52.9 0.03

  XENON 73.6 / 12.2 69.7 / 12.4

• 4.0 +4.0 − 0.2 -0.2 56.0 0.12 83.9 / 52.2 83.2 / 43.1
• 0.7 +0.7

• 9.1 +9.1 52.1 0.05

  ADAM 75.2 / 24.4 71.2 / 18.3

• 4.0 +4.0
• 6.1 +6.1 56.3 0.17 82.1 / 51.1 81.5 / 44.8
• 0.6 +0.6

• 6.3 +6.3 52.9 0.05

  ReAct w/rules – 92.0 / 61.7 – – 50.1 0.10 – 87.2 / 55.4 – – 50.6 0.04

  Table 2: Results by configuration and disclosure condition, pooled over three tasks and 10 Vanilla worlds; Mirror results also pool M01–M06. Vanilla and Mirror report Score / SR, with SR in percent and RIE SR \mathrm{RIE}_{\mathrm{SR}} in percentage points. Cost is reported in US dollars.

  Our main study evaluates Gemini 3.1 Flash Lite (Google 2026 ) and DeepSeek V4 Flash (DeepSeek-AI 2026 ) . GPT-5.6 Luna (OpenAI 2026 ) is evaluated outside the main study; its results appear in the supplementary material. For each combination of model, Vanilla world, task, and one of the six configurations evaluated in both settings, we run three repetitions in the Vanilla world and three in each of its six paired Mirror worlds. ReAct with rules also uses three repetitions per paired Mirror world. Model responses use a temperature of 0.7 and a limit of 2,048 tokens. The six configurations differ in the reasoning, memory, review, retrieval, and probing mechanisms used during an episode.

  All configurations receive the same task, ordered milestones, observation fields, latest action result, skill interface, and Mineflayer execution body in Vanilla and paired Mirror episodes. Each attempted Mineflayer skill consumes one semantic action; Iron Armor and Diamond allow 50 actions, whereas Enchantment allows 75. The common body holds skill execution constant across models and configurations, so the benchmark evaluates decisions through a semantic interface rather than pixel perception or low-level movement control.

  Direct LLM selects each action from the current observation and recent verified outcomes without an additional memory module. ReAct retains a recent trace of thoughts, actions, and observations (Yao et al. 2023 ) , while Reflexion records temporary lessons when progress stalls or actions repeatedly fail (Shinn et al. 2023 ) . Voyager builds and revises a small library of procedures that it can retrieve during an episode (Wang et al. 2023a ) . Our XENON adaptation maintains a graph of task dependencies and updates edges that conflict with observed outcomes (Lee et al. 2026 ) . Our ADAM adaptation represents causal dependencies and can invoke an intervention probe when uncertainty blocks further progress (Yu and Lu 2025 ) .

  Memory, retrieval, review, and probe limits remain fixed across models and worlds, and all internal state is cleared after each episode. Neither XENON nor ADAM receives evaluator labels identifying incorrect dependencies, and each intervention probe consumes a semantic action. ReAct with rules differs from ReAct only by receiving the exact rule descriptions on every model call. Reflection, critique, retrieval, and analysis calls consume no action budget because they do not act in Minecraft, although their API cost is included; every attempted semantic action consumes one unit.

  We fix the time of day, use Peaceful difficulty, and apply identical movement, mining, defense, combat, and experience settings; these controls do not supply items, select subgoals, disclose rules, or skip task dependencies. Each episode begins from a fresh world copy with an empty inventory and cleared agent state. The run record identifies the selected Vanilla world or paired Mirror world, spawn, task, action budget, model settings, and rule disclosure condition. The episode ends when the final milestone is reached, the budget is exhausted, the agent dies, or no executable semantic action can be produced. Deaths, ineffective actions, failed plans, and exhausted budgets remain in the aggregates; only infrastructure failures are rerun.

  Across the 10 Vanilla worlds and their paired Mirror worlds, three tasks, two models, and three repetitions, the six configurations evaluated in both settings produce 1,080 Vanilla episodes and 6,480 paired Mirror episodes with hidden rules. ReAct with rules contributes another 1,080 paired Mirror episodes, yielding 7,560 paired Mirror episodes and bringing the main study total to 8,640 episodes.

  For each Vanilla world, model, task, and configuration evaluated in both settings, we average Score and binary success over its three Vanilla repetitions and separately over three repetitions in each paired Mirror world. RIE subtracts each Mirror mean from its paired Vanilla mean before averaging over the reported factors. The comparison unit is thus a mean over three runs rather than an individual trajectory. The paired worlds match in spawn, placed resources, task, and interfaces at initialization, although trajectories may diverge after a changed outcome. An episode is successful only when the server verifies the final milestone, and values are rounded only after aggregation.

  IRR and RL are computed by rule suite and configuration, then averaged equally across M01–M06. Actions and API cost are reported descriptively and do not enter Score, SR, or RIE.

  Results

  Absolute Performance and Change from Vanilla

  Among the six configurations evaluated without rule descriptions, ReAct achieves the highest pooled Mirror Score (86.8) and SR (53.0%) (Table 2 ). ADAM has the smallest pooled RIE SC \mathrm{RIE}{\mathrm{SC}} ( + 2.3 +2.3 ), whereas XENON has the smallest pooled RIE SR \mathrm{RIE}{\mathrm{SR}} ( + 4.4 +4.4 percentage points). Their Mirror Scores are 76.3 and 76.5, respectively, both below ReAct’s 86.8. A small RIE therefore does not, by itself, indicate strong performance in paired Mirror worlds.

  The observed configuration ordering differs between the two models. With Gemini, Reflexion has a slightly higher mean Mirror Score than ReAct (90.0 versus 89.7), whereas ReAct has a higher mean with DeepSeek (83.8 versus 78.7). The difference between the two models is larger for Direct LLM (65.1 versus 47.7) than for ReAct (89.7 versus 83.8). These comparisons apply to the evaluated model and configuration combinations and do not establish an ordering of agent designs independent of model choice.

  XENON and ADAM have lower pooled Mirror SRs than ReAct, Reflexion, and Voyager (27.8% and 31.6% versus 49.4–53.0%), especially with Gemini. XENON revises dependencies after observed failures, whereas ADAM probes uncertain links (Lee et al. 2026 ; Yu and Lu 2025 ) ; the required attempts and probes consume semantic actions in our adaptations, but graph analysis does not.

  Enchantment extends the Vanilla Diamond route with leather and sugar cane for a book, additional diamonds and obsidian for an enchanting table, lapis lazuli, and the final enchantment; Mirror rules may alter this route. Despite a 75-action budget, no configuration under hidden rules exceeds 20.0% SR with either model, while the pooled Mirror Score/SR is 69.1 / 3.1%, indicating late progress without completion.

  To test whether the interaction horizon constrains completion, we run ReAct, Reflexion, XENON, and ADAM with Gemini under a 200-action cap and record cumulative SR at actions 100, 150, and 200 (Table 3 ). These checkpoints share trajectories and measure completion timing, not separate budgets. This horizon study uses four runs per world and task: 25 Vanilla worlds (five per biome) and 60 Mirror worlds from 10 source worlds (two per biome, six suites each). SR is reported separately by condition; neither result enters RIE.

  100 actions 150 actions 200 actions

  Configuration Vanilla Mirror Vanilla Mirror Vanilla Mirror

  ReAct 77.0 64.4 85.3 72.1 94.7 73.9

  Reflexion 72.7 68.6 83.7 74.4 93.7 77.1

  XENON 39.3 37.4 63.3 76.9 85.3 86.8

  ADAM 49.0 28.5 77.7 48.2 82.3 71.5

  Table 3: Cumulative Gemini SR (%), averaged equally over Iron Armor, Diamond, and Enchantment.

  At action 100, XENON and ADAM trail ReAct and Reflexion in both settings, but from action 100 to 200 their Mirror SRs rise by 49.4 and 43.0 percentage points, versus 9.5 and 8.5 for ReAct and Reflexion. At action 200, XENON leads Mirror SR (86.8%), while ReAct leads Vanilla (94.7%). This delayed improvement is consistent with the actions required for evidence collection and knowledge revision, which can preclude completion under the main horizons.

  Variation Across Rule Suites and Biomes

  Suite Rule change Score SR RIE SC \mathrm{RIE}{\mathrm{SC}} RIE SR \mathrm{RIE}{\mathrm{SR}}

  M01 Quantity scarcity 67.2 21.5

• 13.5 +13.5

• 23.5 +23.5

  M02 Byproduct redistribution 82.1 45.2 − 1.5 -1.5 − 0.2 -0.2

  M03 Yield expansion 84.5 48.0 − 3.8 -3.8 − 3.0 -3.0

  M04 Ore block replacement 66.6 25.4

• 14.0 +14.0

• 19.6 +19.6

  M05 Route substitution 78.6 38.1

• 2.0 +2.0

• 6.9 +6.9

  M06 Byproduct redistribution 83.5 45.6 − 2.9 -2.9 − 0.6 -0.6

  Pooled All suites 77.1 37.3

• 3.6 +3.6

• 7.7 +7.7

  Table 4: Performance in paired Mirror worlds and RIE (Vanilla minus Mirror) by rule suite, pooled over models, tasks, six configurations evaluated without rule descriptions, and 10 Vanilla worlds. SR is a percentage; RIE SR \mathrm{RIE}_{\mathrm{SR}} is in percentage points.

  Quantity scarcity and ore block replacement produce the largest declines from Vanilla, whereas yield expansion and both byproduct redistribution suites improve average outcomes; route substitution produces a smaller decline (Table 4 ). The interventions therefore do not form a uniformly harder test set, and the pooled RIE values mask substantial differences among suite types. These aggregates do not establish whether an agent encountered, inferred, or later used any particular changed rule.

  Both RIE point estimates are positive in all five biome groups. Across the two Vanilla worlds and their paired Mirror worlds in each group, RIE SC \mathrm{RIE}{\mathrm{SC}} ranges from + 1.4 +1.4 to + 7.3 +7.3 , RIE SR \mathrm{RIE}{\mathrm{SR}} from + 4.4 +4.4 to + 13.0 +13.0 percentage points, and the average Mirror Score from 69.8 in Savanna to 82.0 in Taiga. Thus, the positive pooled value is not an artifact of averaging positive and negative biome groups, although its magnitude varies. Because each biome contains only two Vanilla worlds, the breakdown characterizes the evaluated worlds rather than arbitrary worlds from the same biome. The biome breakdown pools rule suites, whereas the suite results pool the evaluated world pairs. Complete results for each biome and Mirror rule suite are reported in the supplementary material.

  Rule Disclosure and Behavioral Diagnostics

  Task Hidden Disclosed Δ ​ Score \Delta\mathrm{Score} Δ ​ SR \Delta\mathrm{SR}

  Iron Armor 89.3 / 76.7 92.8 / 83.1

• 3.5 +3.5

• 6.4 +6.4

  Diamond 89.0 / 72.8 91.9 / 78.9

• 2.9 +2.9

• 6.1 +6.1

  Enchantment 82.1 / 9.4 84.1 / 13.6

• 2.0 +2.0

• 4.2 +4.2

  Table 5: ReAct performance by task in paired Mirror worlds with hidden or disclosed rules, pooled over models and M01–M06. Hidden and Disclosed report Score / SR; SR is a percentage, Δ ​ SR \Delta\mathrm{SR} is in percentage points, and deltas are Disclosed minus Hidden.

  Disclosure improves mean Score and SR in all three tasks, but the gains decrease with task depth and Enchantment reaches only 13.6% SR (Table 5 ). Exact rules therefore help on average without ensuring completion of a longer route. The task averages do not imply positive gains for every suite and model combination, and the remaining failures cannot be attributed separately to route requirements, planning, or execution.

  Figure 2: IRR and RL by configuration, averaged over models, tasks, and paired Mirror worlds with equal weight for M01–M06. Circles on the left axis show IRR (%); squares on the right show RL (semantic actions). The six base configurations receive no rule descriptions; ReAct w/rules is the disclosure condition.

  Direct LLM has the lowest IRR (1.1%) and Voyager the highest (11.9%); their mean RL values are 6.2 and 6.8 actions, respectively (Figure 2 ). ReAct with disclosed rules has a higher IRR than ReAct with hidden rules (9.9% versus 7.1%) and a lower RL (5.4 versus 7.2 actions). A higher IRR can therefore coincide with a lower RL; immediate repetition alone is not an error signal. Because the encounter action can itself raise Score, RL may be zero. IRR considers only the next semantic signature, whereas RL excludes encounters without renewed progress; both are descriptive trajectory summaries, not measures of adaptation or explanations for outcome differences.

  Conclusion

  MirrorCraft compares each Vanilla world with paired Mirror worlds copied from its save, preserving terrain, spawn, placed resources, task, interfaces, and action budget while changing only the rule suite. Across the evaluated settings, hidden rule changes do not produce a uniform penalty. The differences span both directions: RIE SC \mathrm{RIE}{\mathrm{SC}} ranges from − 3.8 -3.8 to + 14.0 +14.0 , while RIE SR \mathrm{RIE}{\mathrm{SR}} ranges from − 3.0 -3.0 to + 23.5 +23.5 percentage points. Among the six configurations evaluated without rule descriptions, ReAct achieves the highest pooled Mirror Score. For ReAct, providing the exact rules improves mean Score and SR in all three tasks but does not eliminate failures. Because Mirror performance and RIE rankings can disagree, both should be reported; a small RIE alone does not imply strong Mirror performance. The RIE measures summarize the overall outcome difference associated with rule replacement under the benchmark protocol; they do not isolate adaptation after a rule encounter.

  Appendix A Limitations

  MirrorCraft evaluates decision making through a common semantic Mineflayer interface; it does not measure visual perception or motor control. The main study uses a fixed set of controlled worlds, so the biome analysis describes the tested instances rather than arbitrary Minecraft seeds; repeated trials do not increase environmental diversity. Each rule suite combines several designed changes rather than sampling one change from a defined population. Results for a suite therefore cannot be attributed to individual edits or generalized to all Minecraft mechanics. The three tasks are nested along one progression chain and share early milestones, so pooled results depend on the chosen equal weighting of tasks.

  Score records the deepest completed milestone but is not a calibrated interval scale on which equal differences represent equal increments of progress. It should therefore be interpreted together with SR and the results for individual tasks. Every RIE comparison uses the same Vanilla world, model, task, agent configuration, interfaces, and action budget in Vanilla and Mirror. However, RIE includes episodes that never encounter a changed rule and does not separately identify changes in route length, the absence of rule descriptions, planning or execution failures, or adaptation after an encounter. Only ReAct is evaluated with the exact rules, so its gains from disclosure should not be generalized to other configurations. Finally, the current tasks do not cover building, collaboration, or other broader forms of Minecraft play.

  Appendix B Supplementary Overview

  This appendix provides the experimental and implementation details omitted from the main paper for space. It reports disaggregated outcomes; specifies agent configurations, execution controls, and automated datapack construction; and documents task milestones, behavioral diagnostics, biome-specific setups, and the six rule-suite manifests.

  Unless stated otherwise, Score and success rate (SR) use the definitions in the main paper. Vanilla–Mirror comparisons are paired by Vanilla world, model, task, agent configuration, and action budget. Reported values follow the aggregation level specified for each analysis and are rounded to the displayed precision.

  Appendix C Additional Model Evaluation

  We additionally evaluate GPT-5.6 Luna on the Diamond task. For each of the six configurations evaluated in both settings, Vanilla covers all five controlled biomes with two maps per biome and two repetitions per map. Mirror covers Plains and Savanna with two source maps per biome, all six rule suites, and two repetitions; ReAct w/rules is evaluated only in Mirror. Table 6 reports averages over the corresponding evaluation sets. Because the Vanilla and Mirror aggregates cover different biome sets, their difference is descriptive and is not reported as RIE.

  Configuration Vanilla Score / SR Mirror Score / SR

  Direct LLM 64.8 / 45.0 60.4 / 39.6

  ReAct 80.0 / 65.0 82.8 / 68.8

  Reflexion 88.8 / 80.0 83.3 / 66.7

  Voyager 90.5 / 85.0 72.9 / 54.2

  XENON 90.0 / 80.0 73.0 / 47.9

  ADAM 90.0 / 80.0 77.6 / 62.5

  ReAct w/rules – 89.1 / 81.2

  Configuration Actions (V/M) Time (V/M) Cost (V/M)

  Direct LLM 43.0 / 42.2 9.6 / 8.5 0.16 / 0.17

  ReAct 39.5 / 38.4 10.9 / 9.0 0.27 / 0.26

  Reflexion 34.1 / 33.6 7.4 / 8.0 0.22 / 0.23

  Voyager 35.9 / 37.6 9.7 / 10.0 0.21 / 0.23

  XENON 39.5 / 38.9 10.2 / 9.5 0.33 / 0.37

  ADAM 33.2 / 41.3 11.5 / 15.3 0.35 / 0.50

  ReAct w/rules – / 34.0 – / 7.3 – / 0.23

  Table 6: GPT-5.6 Luna on Diamond. The upper panel reports Vanilla and Mirror Score / SR; the lower panel reports Vanilla / Mirror Actions, Time, and Cost. SR is in percent, Time in minutes, and Cost in US dollars.

  Within the evaluated Mirror subset, Reflexion obtains the highest Score among configurations without rule descriptions (83.3), while ReAct obtains the highest SR (68.8%). Providing the exact rules increases ReAct’s Score from 82.8 to 89.1 and SR from 68.8% to 81.2%, while reducing its mean semantic-action count from 38.4 to 34.0. These results extend the model coverage of the benchmark but should not be pooled with the two-model main study because the GPT-5.6 Luna evaluation uses only the Diamond task and a different world subset.

  Appendix D Additional Aggregate Results

  Table 7 disaggregates the main results by task and model. Vanilla and Mirror entries aggregate the corresponding evaluations, with Mirror results pooled equally across the six rule suites. The six configurations shared across settings receive no rule descriptions in Mirror; ReAct w/rules receives the exact descriptions. For RIE, each per-world Mirror mean is subtracted from its matched Vanilla-world mean for the same model, task, and agent configuration before pooling. Actions, time, and cost are averaged over the corresponding Mirror evaluations and do not enter RIE.

  Gemini

  Task Configuration Vanilla Mirror RIE SC \mathrm{RIE}{\mathrm{SC}} RIE SR \mathrm{RIE}{\mathrm{SR}} Actions Time Cost

  Iron Armor Direct LLM 73.5 / 30.0 69.3 / 23.3

• 4.2 +4.2

• 6.7 % +6.7% 45.7 9.9 0.04

  ReAct 95.3 / 86.7 90.8 / 75.6

• 4.6 +4.6

• 11.1 % +11.1% 38.9 9.2 0.07

  ReAct w/rules – 93.6 / 81.7 – – 37.4 9.1 0.07

  Reflexion 95.3 / 86.7 95.4 / 87.8 − 0.1 -0.1 − 1.1 % -1.1% 36.3 9.5 0.07

  Voyager 97.7 / 93.3 95.6 / 88.9

• 2.1 +2.1

• 4.4 % +4.4% 35.5 9.4 0.05

  XENON 71.2 / 23.3 65.8 / 17.8

• 5.4 +5.4

• 5.5 % +5.5% 46.1 9.7 0.08

  ADAM 74.7 / 33.3 69.5 / 27.2

• 5.1 +5.1

• 6.1 % +6.1% 46.6 13.3 0.12

  Diamond Direct LLM 65.8 / 10.0 66.2 / 13.9 − 0.3 -0.3 − 3.9 % -3.9% 47.7 12.9 0.04

  ReAct 98.0 / 93.3 91.4 / 73.3

• 6.6 +6.6

• 20.0 % +20.0% 40.3 12.1 0.07

  ReAct w/rules – 94.8 / 82.8 – – 40.2 11.7 0.07

  Reflexion 96.0 / 86.7 92.8 / 77.8

• 3.2 +3.2

• 8.9 % +8.9% 41.1 12.1 0.07

  Voyager 96.0 / 86.7 92.2 / 75.6

• 3.8 +3.8

• 11.1 % +11.1% 40.6 12.4 0.05

  XENON 67.7 / 13.3 68.0 / 16.7 − 0.3 -0.3 − 3.4 % -3.4% 47.2 12.0 0.08

  ADAM 77.5 / 36.7 73.8 / 26.7

• 3.7 +3.7

• 10.0 % +10.0% 47.5 17.1 0.13

  Enchantment Direct LLM 72.5 / 0.0 59.8 / 0.0

• 12.7 +12.7 0.0 % 0.0% 75.0 24.4 0.08

  ReAct 90.0 / 20.0 87.0 / 15.6

• 3.0 +3.0

• 4.4 % +4.4% 73.3 22.9 0.15

  ReAct w/rules – 87.5 / 20.6 – – 72.6 23.3 0.15

  Reflexion 87.2 / 16.7 81.8 / 6.7

• 5.4 +5.4

• 10.0 % +10.0% 74.3 29.4 0.16

  Voyager 82.7 / 6.7 77.2 / 3.9

• 5.5 +5.5

• 2.8 % +2.8% 73.9 22.6 0.12

  XENON 82.0 / 0.0 75.2 / 2.8

• 6.8 +6.8 − 2.8 % -2.8% 74.7 26.1 0.20

  ADAM 73.3 / 3.3 70.3 / 1.1

• 3.1 +3.1

• 2.2 % +2.2% 74.9 32.7 0.26

  DeepSeek

  Task Configuration Vanilla Mirror RIE SC \mathrm{RIE}{\mathrm{SC}} RIE SR \mathrm{RIE}{\mathrm{SR}} Actions Time Cost

  Iron Armor Direct LLM 71.7 / 50.0 62.0 / 18.9

• 9.6 +9.6

• 31.1 % +31.1% 42.8 8.3 0.01

  ReAct 96.7 / 93.3 87.8 / 77.8

• 8.9 +8.9

• 15.5 % +15.5% 38.7 8.1 0.03

  ReAct w/rules – 92.0 / 84.4 – – 38.0 8.1 0.03

  Reflexion 94.5 / 90.0 88.7 / 76.7

• 5.8 +5.8

• 13.3 % +13.3% 39.4 8.1 0.03

  Voyager 95.5 / 93.3 91.6 / 81.7

• 3.9 +3.9

• 11.6 % +11.6% 39.5 8.8 0.02

  XENON 86.7 / 83.3 88.2 / 73.9 − 1.6 -1.6

• 9.4 % +9.4% 38.8 9.5 0.04

  ADAM 89.2 / 83.3 90.2 / 76.7 − 1.1 -1.1

• 6.6 % +6.6% 39.8 9.7 0.04

  Diamond Direct LLM 41.8 / 26.7 48.7 / 5.0 − 6.9 -6.9

• 21.7 % +21.7% 46.8 8.9 0.01

  ReAct 90.8 / 83.3 86.5 / 72.2

• 4.3 +4.3

• 11.1 % +11.1% 41.9 8.5 0.03

  ReAct w/rules – 88.9 / 75.0 – – 40.8 8.4 0.03

  Reflexion 86.3 / 76.7 84.4 / 62.2

• 1.9 +1.9

• 14.5 % +14.5% 42.3 9.0 0.03

  Voyager 79.7 / 63.3 79.8 / 46.1 − 0.1 -0.1

• 17.2 % +17.2% 45.0 10.3 0.02

  XENON 80.3 / 66.7 83.5 / 52.8 − 3.1 -3.1

• 13.9 % +13.9% 43.6 9.7 0.04

  ADAM 83.2 / 70.0 84.0 / 57.2 − 0.8 -0.8

• 12.8 % +12.8% 44.9 10.3 0.04

  Enchantment Direct LLM 40.5 / 0.0 32.3 / 0.0

• 8.3 +8.3 0.0 % 0.0% 74.3 8.2 0.02

  ReAct 83.7 / 6.7 77.3 / 3.3

• 6.4 +6.4

• 3.4 % +3.4% 74.0 8.1 0.05

  ReAct w/rules – 80.8 / 6.7 – – 73.1 8.0 0.05

  Reflexion 68.2 / 0.0 62.9 / 0.0

• 5.2 +5.2 0.0 % 0.0% 74.4 10.7 0.06

  Voyager 59.8 / 0.0 57.5 / 0.0

• 2.3 +2.3 0.0 % 0.0% 74.3 9.1 0.04

  XENON 84.7 / 6.7 77.8 / 2.8

• 6.9 +6.9

• 3.9 % +3.9% 74.0 10.2 0.08

  ADAM 74.0 / 0.0 70.2 / 0.6

• 3.8 +3.8 − 0.6 % -0.6% 74.1 9.6 0.08

  Table 7: Vanilla and paired Mirror performance, RIE, and evaluation cost by task, model, and agent configuration. Vanilla and Mirror report Score / SR; Time is in minutes and Cost in US dollars.

  Task and model breakdown.

  For Enchantment, several configurations reach late milestones while rarely completing the final milestone, whereas Iron Armor and Diamond show closer agreement between progress and completion (Table 7 ). For some combinations of task and model, RIE SC \mathrm{RIE}_{\mathrm{SC}} is negative, meaning that the mean Mirror Score exceeds the Vanilla mean for the same Vanilla worlds. This outcome alone does not establish that an agent recognized or exploited a shortcut.

  Table 8 pools models, tasks, and the six configurations evaluated without rule descriptions in both settings.

  Biome Vanilla Mirror RIE SC \mathrm{RIE}{\mathrm{SC}} RIE SR \mathrm{RIE}{\mathrm{SR}}

  Score SR Score SR

  Plains 81.5 46.3 78.9 40.4

• 2.6 +2.6

• 5.9 % +5.9%

  Taiga 84.9 50.0 82.0 42.1

• 2.9 +2.9

• 7.9 % +7.9%

  Snowy Plains 84.5 50.0 81.0 42.6

• 3.5 +3.5

• 7.4 % +7.4%

  Jungle 75.3 36.1 73.8 31.7

• 1.4 +1.4

• 4.4 % +4.4%

  Savanna 77.1 42.6 69.8 29.6

• 7.3 +7.3

• 13.0 % +13.0%

  Table 8: Vanilla and paired Mirror Score, SR, and RIE by biome.

  Biome breakdown.

  Across the tested worlds in each biome, both RIE point estimates are positive but vary in magnitude (Table 8 ). Savanna has the lowest Mirror Score and SR and the largest decreases from Vanilla to Mirror, whereas Taiga has the highest Mirror Score. These descriptive results apply to the tested instances and do not establish generalization to other worlds from the same biomes or calibrated differences in task difficulty.

  Appendix E Agent Configuration Details

  All configurations use the same serialized observations, semantic skills, and action budgets. They also follow the same loop for observing, updating, deciding, and executing. Beyond this common interface, they implement distinct reasoning, memory, review, retrieval, and probing mechanisms within each episode. Each episode starts with empty internal state, and all traces, lessons, procedures, and graphs are discarded when it ends. The main study uses gemini-3.1-flash-lite and deepseek-v4-flash . All calls use temperature 0.7 and an output limit of 2,048 tokens. All evaluated models, including GPT-5.6 Luna, were run with any provider-side extended-thinking or reasoning mode disabled. This setting is distinct from the explicitly implemented reasoning traces and auxiliary agent procedures described below.

  World and execution controls.

  A common construction policy is instantiated as one world-generation datapack for each controlled biome, with a shared placement schedule for task resources. Daytime is fixed, and Peaceful difficulty removes variation from hostile mobs and night. Every agent receives the same Haste, Speed, Resistance, and Strength effects and begins at experience level 100. These controls shorten mining, traversal, combat, and enchanting without choosing an agent subgoal or revealing a changed rule. Iron Armor and Diamond use a budget of 50 semantic actions, while Enchantment uses 75.

  Direct LLM.

  Direct LLM uses the current observation and a bounded record of verified action effects, without an auxiliary memory module. Each model call selects and executes one semantic skill.

  ReAct.

  ReAct maintains a trace of thoughts, actions, and observations for the 16 most recent transitions and uses this trace to select the next semantic skill.

  ReAct w/rules.

  This configuration uses the same trace of 16 transitions and the same procedure for selecting actions as ReAct, but includes the exact active rule descriptions in every model call.

  Reflexion.

  Reflexion augments the ReAct trace with up to three temporary lessons. It creates or revises lessons after repeated failure, conflicting evidence, stalled progress, or a scheduled review, with at most six reviews per episode and a retention window of 16 actions per lesson.

  Voyager.

  Voyager maintains a procedure library of at most 16 entries within each episode. It generates programs of four skills, critiques their execution, retrieves up to five procedures, and permits at most four repairs.

  XENON.

  XENON maintains a graph of task dependencies and a memory of observed effects, each with capacity 32. It retrieves up to three entries, records action outcomes, corrects contradicted edges, and replans from the updated frontier after two failures.

  ADAM.

  ADAM maintains a causal graph of at most 48 nodes and a transition memory of at most 32 entries. It learns from two comparable samples and may execute at most two probes when an uncertain dependency blocks progress; probing is disabled during the final 12 actions.

  Neither XENON nor ADAM receives evaluator feedback that identifies an incorrect dependency. An ADAM probe is an ordinary semantic action and consumes the same action budget as any other skill call. Additional model calls used for reflection, critique, retrieval, or graph analysis do not act in Minecraft.

  Deaths, policy failures, and exhausted budgets remain in the denominator. Disconnected clients, missing datapacks, and unavailable advancements are rerun under the same episode identifier. Before aggregation, we check the recorded biome, identifier of the Vanilla world, whether the episode uses Vanilla or Mirror rules, task, configuration, rule disclosure status, and action budget. Score, SR, both RIEs, Δ ​ Score \Delta\mathrm{Score} , and Δ ​ SR \Delta\mathrm{SR} are computed only after these checks.

  Appendix F Direct LLM Input

  Direct LLM maps the current serialized observation to one semantic skill call from the common interface. Its input contains inventory, position, task and milestones, nearby blocks and entities, outcomes of previous actions, and available skills. The serialized observation also reports equipment, current biome, survival state, nearby dropped items, the next unfinished milestone, and known failure constraints. For Direct LLM, previous outcomes include the latest action result and a bounded record of verified action effects.

  Appendix G Automated Datapack Construction

  Minecraft Java datapacks express recipes, loot tables, advancements, and resources for world generation as JSON files loaded by the server (Minecraft Wiki contributors 2026d , b , a , c ) . MirrorCraft instantiates two datapack families through one parameterized offline construction system: world-generation specifications define the controlled Vanilla environments, while rule-intervention manifests define the gameplay changes applied to their paired Mirror copies. Both families share a versioned JSON protocol, compiler interface, feedback loop, and termination policy. The domain flag kind ∈ { biome , mirror } \in{\texttt{biome},\texttt{mirror}} selects the appropriate specification schema, allowlist, compiler, design criteria, and audit criteria.

  Shared Construction Workflow

  The workflow begins from a complete specification supplied by the user or generated by a deterministic seed-based sampler. When LLM-assisted design is enabled, the Designer returns a complete replacement specification conditioned on the constraints, allowlist, current static report, and preceding Judge report. The domain compiler validates and provisionally materializes each candidate; static failures and Judge revisions are returned for another bounded round. A candidate is finalized when compilation succeeds and the Judge reports no hard failure and deems every selected task solvable, while static-only runs end after the compiler audit. Construction-level acceptance is followed by the deterministic loading, outcome, information-integrity, and route controls described in the main paper.

  World-Generation Datapacks

  For kind=biome , MirrorCraft represents each controlled biome as a versioned world-generation specification. The specification factorizes a world into terrain, resources, ecology, and runtime controls. Terrain fields determine the vertical range, sea level, relief noise, surface materials, and water generation. Resource entries define one or more height bands per material, with separate controls for vein size, placement attempts, vertical range, and air-exposure discard. Ecology fields select a benchmark or Vanilla feature backbone, or a minimal feature policy, and parameterize trees, vegetation, decorations, water features, and passive creatures. Runtime fields configure difficulty, initial time, and day and weather cycles.

  Pre-compilation validation checks the schema, numeric bounds, whitelisted identifiers, ore-band parameters, ecology fields, creature spawning, and the presence of resource categories required by the selected tasks. A deterministic whitelist compiler then instantiates the world preset, biome, noise settings, density functions, configured features, placed features, and setup function for Minecraft Java 1.19.0 (pack format 10), and packages the resulting files deterministically for installation in the corresponding evaluation save. A post-packaging audit parses every JSON document and checks the pack format, required world-generation resource types, custom resource identifiers, internal references, feature-stage structure, ore placements, noise settings, and artifact metadata. The Judge then assesses the static plausibility of spawn safety, traversal, biome recognizability, resource density, and task completion from the specification and audit report. The biome-specific setup pages below report the resulting terrain and placement controls.

  Rule-Intervention Datapacks

  For kind=mirror , the current specification is a task-aware rule manifest, and the injected catalog contains the allowed operators, editable targets, processing inputs and stations, output items, and dependency routes for the selected tasks. The compiler constructs an abstract dependency graph from predefined task routes and task-relevant transformations, then identifies recipe, loot, drop, and processing transitions that can be modified without changing the map or action schema. When enabled, the Designer proposes a compact set of interpretable interventions; the compiler materializes the manifest through standard datapack mechanisms and reports statically estimated relevance, abstract route reachability, and limited shadowing-based encounterability; and the Judge audits observability, coherence, interactions among changes, and task feasibility. In Mirror episodes with hidden rules, agents receive neither rule descriptions nor access to a server rule query and can detect a changed rule only from the outcome of an affected action. For rule-intervention construction, kind=mirror ; the selected task set determines the dependency routes that must remain reachable. After construction-level acceptance, the package, specification, and reports are finalized as a reproducible artifact; benchmark admission then follows the Integrity and Reachability Controls in the main paper.

  World-Generation Prompts

  The world-generation prompts use the structured response contracts described below. The Designer receives the user constraints, allowed catalog, current specification, static compiler/reachability report, and preceding Judge report; when no preceding report exists, its verdict is not_run . The Judge receives the candidate specification and static compiler report. The world-generation catalog contains the permitted base biomes, surface blocks, ores and ore schema, creatures, tree and feature policies, surface patches, decorations, and numeric bounds.

  Designer prompt.

  You are the MirrorCraft LLM Designer. Design a complete, machine-readable candidate specification for Minecraft Java 1.19.0. The static compiler writes the datapack. Output no commands, source code, file paths, Markdown fences, comments, or prose outside one JSON object. The object contains schemaversion , kind , a brief designsummary , one resolvedfeedback list item per addressed problem, and a complete replacement spec ; the outer object and nested specification both use schema version 1.0 and kind biome . Preserve the requested selectedtasks , keep every numeric field inside the supplied bounds, treat the static report as authoritative, and correct every hard failure.

  Construct an Overworld single-biome evaluation world rather than a decorative showcase, with a traversable spawn surface and practically reachable resources for every selected task. The benchmark routes may require wood, stone, fuel, iron, diamond, lapis, obsidian, sugar cane, cows or leather, and experience; a listed resource is insufficient when its height range lies outside the generated terrain or its density is impractically low. Preserve the recognizable ecology of the base biome while varying terrain, surface blocks, resource distributions, vegetation, water, and passive mobs within the whitelist. Exclude Nether- or End-only assumptions, structures, chests, commands, and arbitrary identifiers. Avoid underwater spawn traps, impassable clutter, and settings that trivialize every task. Return one corrected full specification from the supplied constraints, catalog, current specification, static report, and preceding Judge report.

  Judge prompt.

  You are the independent MirrorCraft LLM Judge. Audit the compiled Minecraft Java 1.19.0 candidate without redesigning the datapack or inspecting agent outcomes. Be conservative, use the static report as hard evidence, and output no Markdown or prose outside one JSON object. The object uses schema version 1.0 and kind biome , and contains verdict , confidence , summary , hardfailures , warnings , taskassessments , changeassessments , and specific, machine-actionable revisioninstructions . Each task assessment reports the task identifier, solvability, ordered dependency route, and principal risk. Use accept only when the hard-failure list is empty, revise when a bounded specification change can repair the candidate, and reject when its central design must be replaced.

  Audit spawn safety, traversal, the surface–sea relationship, biome recognizability, ore height ranges, every ore band, resource density, weighted tree placement, controlled decorations, wood and fuel availability, cows and leather, sugar cane, diamond, lapis, obsidian, and selected-task feasibility. A resource generated outside usable terrain does not count. Distinguish valid biome character from needless motor difficulty. Treat the ZIP as a deterministic serialization of the specification, and judge only the transitions and task routes represented by the supplied specification and static compiler report without assuming any unlisted resource or hidden interface.

  Rule-Intervention Prompts

  The rule-intervention prompts use the same structured response contracts. The Designer receives the user constraints, allowed catalog, current specification, static compiler/reachability report, and preceding Judge report; when no preceding report exists, its verdict is not_run . The Judge receives the candidate specification and static compiler report. The rule-intervention catalog contains the permitted operators, editable targets, processing inputs and stations, output items, and dependency routes for the selected tasks.

  Designer prompt.

  You are the MirrorCraft LLM Designer. Design a complete, machine-readable candidate specification for Minecraft Java 1.19.0. The static compiler writes the datapack. Output no commands, source code, file paths, Markdown fences, comments, or prose outside one JSON object. The object contains schemaversion , kind , a brief designsummary , one resolvedfeedback list item per addressed problem, and a complete replacement spec ; the outer object and nested specification both use schema version 1.0 and kind mirror . Preserve the requested selectedtasks , keep every numeric field inside the supplied bounds, treat the static report as authoritative, and correct every hard failure.

  Make each changed transition normally encounterable while an agent pursues at least one selected task, and keep every selected task solvable through Mineflayer-observable interaction. Prefer two to six interpretable changes, combining quantity, coproduct, or route modifications only when their interaction remains auditable. Restrict edits to the supplied operators, targets, stations, inputs, and outputs, without modifying unrelated food, decoration, redstone, village, Nether, or End mechanics. The changed rule must be learnable from actual action outcomes rather than recipe, loot-table, datapack, recipe-book, or hidden-state inspection. Avoid changes that make completion immediate or require an item absent from the allowed dependency graph. Return one corrected full specification from the supplied constraints, catalog, current specification, static report, and preceding Judge report.

  Judge prompt.

  You are the independent MirrorCraft LLM Judge. Audit the compiled Minecraft Java 1.19.0 candidate without redesigning the datapack or inspecting agent outcomes. Be conservative, use the static report as hard evidence, and output no Markdown or prose outside one JSON object. The object uses schema version 1.0 and kind mirror , and contains verdict , confidence , summary , hardfailures , warnings , taskassessments , changeassessments , and specific, machine-actionable revisioninstructions . Each task assessment reports the task identifier, solvability, ordered dependency route, and principal risk. Use accept only when the hard-failure list is empty, revise when a bounded specification change can repair the candidate, and reject when its central design must be replaced.

  For every rule change, audit task relevance, encounterability before task completion, observable outcome, the complete modified dependency route, interactions with other changes, and difficulty. Reject changes unrelated to all selected routes and reject a pack when any selected task is unreachable. Treat extreme scarcity or shortcuts as revision requirements unless explicitly requested. The agent may observe action inputs and actual outputs but cannot inspect recipes, loot tables, datapack manifests, or the compiler’s rule sheet. Treat the ZIP as a deterministic serialization of the specification, and judge only the transitions and task routes represented by the supplied specification and static compiler report without assuming any unlisted resource or hidden interface.

  Figure 3: Server-verified task milestones and their assigned Score values.

  M01 M02 M03 M04 M05 M06 Average

  Configuration IRR RL IRR RL IRR RL IRR RL IRR RL IRR RL IRR RL

  Direct LLM 0.7% 14.0 5.3% 3.4 0.3% 4.3 0.0% 7.2 0.2% 4.5 0.1% 3.8 1.1% 6.2

  ReAct 10.7% 15.3 24.0% 4.1 1.8% 3.9 4.2% 9.1 1.1% 6.0 0.7% 5.0 7.1% 7.2

  Reflexion 7.7% 15.2 20.8% 4.0 3.3% 3.7 3.1% 8.9 0.8% 5.6 0.5% 5.1 6.0% 7.1

  Voyager 11.3% 15.0 35.9% 3.8 7.8% 3.7 11.5% 8.1 3.0% 5.4 1.8% 4.8 11.9% 6.8

  XENON 11.7% 15.0 33.3% 3.3 5.3% 3.9 9.1% 7.5 4.6% 5.8 1.4% 4.3 10.9% 6.6

  ADAM 7.0% 13.7 33.7% 4.0 4.7% 3.7 7.6% 7.6 2.0% 5.1 1.2% 4.5 9.4% 6.4

  ReAct w/rules 18.9% 10.5 26.6% 4.2 3.4% 3.1 7.5% 6.0 1.7% 4.3 1.2% 4.0 9.9% 5.4

  Table 9: IRR (%) and RL (semantic actions) by rule suite and configuration, averaged over models, tasks, and Mirror worlds. Average weights M01–M06 equally; only ReAct w/rules receives rule descriptions.

  Appendix H Advancement Milestones

  MirrorCraft measures ordered task progress with the server-verified milestones in Figure 3 . Each task has a fixed milestone sequence that follows its main resource and tool dependencies. An episode receives the value of its deepest completed milestone rather than the sum of all completed milestones. The final milestone has value 100 and indicates task completion. This design preserves partial progress when an episode reaches a late prerequisite but does not finish the full task. All milestone identifiers below use the minecraft:story/ namespace.

  Iron Armor

  The Iron Armor task is completed by obtaining any one iron armor piece rather than a complete four-piece set. The ordered codes used by the game are minestone , upgradetools , smeltiron , and obtainarmor .

  Diamond

  The ordered codes are minestone , upgradetools , smeltiron , irontools , and mine_diamond .

  Enchantment

  The ordered codes are minestone , upgradetools , smeltiron , irontools , minediamond , formobsidian , and enchant_item .

  The milestone sequence is shown to the agent, while the server independently verifies completion.

  Appendix I Detailed Behavioral Diagnostics

  The results for each rule suite below retain the same encounter definitions used in the benchmark. IRR compares the semantic signature of the action that triggers a rule with that of the next action counted against the budget. RL counts actions until the first higher milestone and includes only encounters followed by such an increase. The final columns weight M01–M06 equally.

  M02 produces the highest IRR for every configuration while retaining short RL. This pattern is consistent with useful byproducts, for which repeating the same semantic action can remain productive. M01 instead yields the longest RL for every configuration evaluated with hidden rules. Across suites, rule disclosure raises ReAct’s IRR from 7.1% to 9.9% while reducing RL from 7.2 to 5.4 actions. The two diagnostics therefore capture different behavior, and IRR should not be read as an error rate.

  Appendix J Biome-Specific Datapack Setups

  Plains

  Game view Cross-section schematic

  The Plains artifact targets Minecraft Java 1.19.0 (pack format 10) and defines abplains:benchmarkplains as a single-biome Overworld preset spanning Y = − 64 ​ – ​ 319 Y=-64\text{--}319 , with sea level 63, aquifers, and the standard minecraft:cave and minecraft:caveextra_underground carvers. Its setup function selects Peaceful difficulty, fixes time at noon and weather to clear, retains natural mob spawning, and disables patrol, wandering-trader, and insomnia spawning. The Nether and End keep their Vanilla generators, while every Overworld climate point maps to the controlled Plains biome.

  Terrain density combines a vertical gradient from 1.0 1.0 at Y = 50 Y=50 to − 1.0 -1.0 at Y = 90 Y=90 with cached broad-contour and local-undulation noise weighted by 0.56 and 0.18. Their first octaves are − 8 -8 and − 5 -5 , with amplitudes [ 1 , .5 , .25 ] [1,.5,.25] and [ 1 , .5 ] [1,.5] ; the blended density is scaled by 0.64, yielding low, open hills. Grass over dirt is the default surface. Surface-noise intervals introduce podzol on [ − .95 , − .80 ] [-.95,-.80] , coarse dirt on [ .51 , .60 ] [.51,.60] and [ .66 , .79 ] [.66,.79] , and gravel on [ .88 , .95 ] [.88,.95] . The feature list retains Vanilla amethyst geodes, underground lava lakes, springs, glow lichen, stone variants, sediments, and the ordinary Plains flower, grass, mushroom, and pumpkin patches.

  Surface ecology remains stochastic rather than placing task resources at fixed coordinates. The minecraft:trees_plains wrapper selects zero or one placement attempt with equal probability (0.5 per chunk in expectation) and rejects locations with nonzero surface-water depth. Sugar cane and water lilies receive four and two attempts per chunk. Rarity filters request a surface lake, large ferns, a mossy boulder, and an additional pumpkin patch once per 15, 12, 18, and 180 chunks in expectation. Cows spawn naturally with weight 24 in groups of four, and their base loot provides 1–2 leather before looting modifiers. The boulder is a loot-free world-generation feature rather than a structure. The archive must be enabled alone when creating a new world with this preset; existing chunks do not acquire its terrain, resources, vegetation, or landmarks retroactively.

  Band

  Attempts per chunk

  Size

  Origin Y Y (inclusive)

  Coal (accessible)

  96

  20

  44–112

  Iron (accessible)

  84

  16

  40–92

  Copper (accessible)

  72

  20

  40–96

  Gold (accessible)

  48

  12

  32–72

  Redstone (accessible)

  48

  14

  28–68

  Lapis (accessible)

  36

  12

  30–68

  Emerald (accessible)

  24

  8

  40–80

  Diamond (main)

  40

  10

  36–68

  Diamond (shallow)

  8

  6

  52–72

  Obsidian (main)

  8

  12

  36–64

  Obsidian (shallow)

  10

  10

  50–72

  Table 10: Custom resource bands for Plains; retained Vanilla ore placements are omitted.

  Note. Attempts count sampled origins per chunk; size is the configured ore-feature parameter. The final column bounds the sampled origin, not every generated block.

  The biome definition retains ordinary Vanilla placements for coal, iron, copper, gold, redstone, lapis, and diamond, then appends the 11 custom entries in Table 10 ; the table is therefore not a complete ore distribution. Emerald and obsidian occur only through custom entries in this biome list. Every custom band applies count , in_square , an inclusive uniform height range, and a biome filter. Its configured feature replaces stone- or deepslate-replaceable blocks and uses air-exposure discard 0.0. Attempts can fail or overlap, and configured size is not a guaranteed realized block count.

  The main and shallow diamond and obsidian bands reduce search and descent costs without placing task items directly in inventory or containers. Setting oreveinsenabled to false disables Minecraft’s separate large noise-vein system, not the ordinary Vanilla ore placements retained in the biome definition. The cross-section above is illustrative rather than a block-exact rendering of these attempts.

  Taiga

  Game view Cross-section schematic

  The Taiga artifact targets Minecraft Java 1.19.0 (pack format 10) and defines abtaiga:benchmarktaiga as a single-biome Overworld preset spanning Y = − 64 ​ – ​ 319 Y=-64\text{--}319 , with sea level 63, aquifers, and the standard minecraft:cave and minecraft:caveextra_underground carvers. Its setup function selects Peaceful difficulty, fixes time at noon and weather to clear, retains ordinary mob spawning, and disables patrol, wandering-trader, and insomnia spawning. These controls remove hostile, day–night, and weather variation without suppressing passive animals or the normal collection interface.

  Terrain density combines a vertical gradient from 1.0 1.0 at Y = 48 Y=48 to − 1.0 -1.0 at Y = 94 Y=94 with two cached two-dimensional noise fields. The broad contour field has coefficient 0.6, first octave − 8 -8 , and amplitudes [ 1 , .5 , .25 ] [1,.5,.25] ; the local-undulation field has coefficient 0.2, first octave − 5 -5 , and amplitudes [ 1 , .5 ] [1,.5] . The combined density is scaled by 0.64, producing rolling relief and shallow valleys rather than sharp mountain barriers. Grass over dirt is the default surface, while surface-noise intervals introduce podzol on [ − .95 , − .80 ] [-.95,-.80] , coarse dirt on [ .51 , .60 ] [.51,.60] and [ .66 , .79 ] [.66,.79] , and moss on [ .88 , .95 ] [.88,.95] . The biome feature list also retains Vanilla amethyst geodes, water and lava springs, glow lichen, and the ordinary stone-variant and sediment features, so the controlled resource bands do not reduce the underground to a bare test grid.

  Surface ecology is generated rather than placed at fixed task coordinates. Spruce placement draws 14 attempts with weight 9 or 15 with weight 1 (14.1 in expectation) and rejects locations with nonzero surface-water depth. Sugar cane, large ferns, and water lilies receive 4, 2, and 1 attempts per chunk, respectively. Rarity filters request berries, a surface lake, a mossy boulder, a huge brown mushroom, and pumpkins once per 12, 15, 8, 48, and 160 chunks in expectation. Cows spawn naturally with weight 24 in groups of four, and their base loot provides 1–2 leather before looting modifiers. The boulder and mushroom are visual landmarks only: neither contains a chest or task reward. The archive must be enabled alone when creating a new world with the specified preset; existing chunks do not acquire its ores, vegetation, or landmarks retroactively.

  Band

  Attempts per chunk

  Size

  Origin Y Y (inclusive)

  Coal (accessible)

  96

  20

  44–112

  Iron (accessible)

  84

  16

  40–92

  Copper (accessible)

  72

  20

  40–96

  Gold (accessible)

  48

  12

  32–72

  Redstone (accessible)

  48

  14

  28–68

  Lapis (accessible)

  36

  12

  30–68

  Emerald (accessible)

  24

  8

  40–80

  Diamond (main)

  40

  10

  36–68

  Diamond (shallow)

  8

  6

  52–72

  Obsidian (main)

  8

  12

  36–64

  Obsidian (shallow)

  10

  10

  50–72

  Table 11: Custom resource bands for Taiga; retained Vanilla ore placements are omitted.

  Note. Attempts count sampled origins per chunk; size is the configured ore-feature parameter. The final column bounds the sampled origin, not every generated block.

  The biome first retains ordinary Vanilla placements for coal, iron, copper, gold, redstone, lapis, and diamond, then appends the 11 custom entries in Table 11 ; the table is therefore not a complete ore distribution. Emerald and obsidian occur only through custom entries in this biome list. For each attempt, in_square selects a horizontal origin in the chunk and uniform samples an integer height from the listed band. A biome modifier checks the sampled position, after which the configured feature replaces stone- or deepslate-replaceable blocks. Air-exposure discard is 0.0, so a candidate is not rejected merely for touching air. Attempts may fail or overlap, and configured size is not a guaranteed realized block count; consequently, attempts multiplied by size is not an ore count.

  Diamond and obsidian each use main and shallow bands to reduce search and descent costs without placing task items directly in inventory or containers. The boulder and mushroom are world-generation features rather than structures, so disabling generated structures does not remove them. The accompanying archive contains the complete namespace and JSON definitions; the cross-section above is illustrative rather than a block-exact rendering of these placement counts.

  Snowy Plains

  Game view Cross-section schematic

  The Snowy Plains artifact targets Minecraft Java 1.19.0 (pack format 10) and defines absnow:benchmark_snow as a single-biome Overworld preset spanning Y = − 64 ​ – ​ 319 Y=-64\text{--}319 , with sea level 63, aquifers, and the standard cave carvers. Every Overworld climate point maps to this biome; the Nether and End retain their Vanilla generators. The setup function selects Peaceful difficulty, fixes noon and clear weather, keeps natural mob spawning enabled, and disables patrol, wandering-trader, and insomnia spawning.

  Terrain density combines a vertical gradient from 1.0 1.0 at Y = 50 Y=50 to − 1.0 -1.0 at Y = 88 Y=88 with broad-contour and local-undulation noise weighted by 0.45 and 0.14. The two noises use first octaves − 8 -8 and − 5 -5 and amplitudes [ 1 , .5 , .25 ] [1,.5,.25] and [ 1 , .5 ] [1,.5] ; the blended field is scaled by 0.64, producing the lowest-relief terrain among the five setups. The base surface is grass over dirt, with podzol on [ − .95 , − .80 ] [-.95,-.80] , coarse dirt on [ .51 , .60 ] [.51,.60] and [ .66 , .79 ] [.66,.79] , and gravel on [ .88 , .95 ] [.88,.95] . Snow precipitation, temperature 0.0, and the retained minecraft:freezetoplayer feature apply the visible snow and ice rather than hard-coding snow into the surface rule. Caves, underground lava lakes, amethyst geodes, springs, glow lichen, stone variants, and sediments remain enabled.

  Spruce placement selects zero or one attempt with equal probability (0.5 per chunk in expectation) and rejects positions with nonzero surface-water depth. The executable placed-feature JSON assigns six sugar-cane attempts per chunk. Rarity filters request large ferns, berry bushes, an extra pumpkin patch, a mossy boulder, an ice patch, an ice spike, and a surface lake once per 14, 24, 200, 24, 18, 56, and 18 chunks in expectation. These ice and rock landmarks are loot-free placed features rather than structures. Cows retain weight 24 in groups of four and provide 1–2 base leather; sheep, pigs, chickens, rabbits, bats, and glow squid also remain in their listed spawn categories. The archive must be enabled alone for a new world because existing chunks are not regenerated retroactively.

  Band

  Attempts per chunk

  Size

  Origin Y Y (inclusive)

  Coal (accessible)

  96

  20

  44–112

  Iron (accessible)

  84

  16

  40–92

  Copper (accessible)

  72

  20

  40–96

  Gold (accessible)

  48

  12

  32–72

  Redstone (accessible)

  48

  14

  28–68

  Lapis (accessible)

  36

  12

  30–68

  Emerald (accessible)

  24

  8

  40–80

  Diamond (main)

  40

  10

  36–68

  Diamond (shallow)

  8

  6

  52–72

  Obsidian (main)

  8

  12

  36–64

  Obsidian (shallow)

  10

  10

  50–72

  Table 12: Custom resource bands for Snowy Plains; retained Vanilla ore placements are omitted.

  Note. Attempts count sampled origins per chunk; size is the configured ore-feature parameter. The final column bounds the sampled origin, not every generated block.

  The biome definition retains ordinary Vanilla placements for coal, iron, copper, gold, redstone, lapis, and diamond, then appends the 11 custom entries in Table 12 . Emerald and obsidian occur only through custom entries in this biome list. Each added band applies count , in_square , inclusive uniform height sampling, and a biome filter, then replaces stone- or deepslate-replaceable blocks with air-exposure discard 0.0. The reported attempts may fail or overlap, and configured size is not a realized block count.

  The two diamond bands and two obsidian bands reduce search and descent costs without bypassing mining. The separate large noise-vein system is disabled, but the ordinary Vanilla placed ores explicitly retained in the biome remain active. Thus the Snowy Plains setup changes surface traversal and resource visibility while using the same added underground bands as the other controlled biomes. The cross-section is an interpretive schematic rather than a block-exact reconstruction.

  Jungle

  Game view Cross-section schematic

  The Jungle artifact targets Minecraft Java 1.19.0 (pack format 10) and defines abjungle:benchmark_jungle as a single-biome Overworld preset spanning Y = − 64 ​ – ​ 319 Y=-64\text{--}319 , with sea level 63, aquifers, and the standard cave carvers. Its full multi-noise domain maps to the controlled Jungle biome, while the Nether and End retain Vanilla generation. The setup function fixes Peaceful difficulty, noon, and clear weather, leaves natural mob spawning active, and disables patrol, wandering-trader, and insomnia spawning.

  Terrain density uses a vertical gradient from 1.0 1.0 at Y = 46 Y=46 to − 1.0 -1.0 at Y = 94 Y=94 plus broad-contour and local-undulation noise weighted by 0.65 and 0.22. The fields use first octaves − 8 -8 and − 5 -5 , amplitudes [ 1 , .5 , .25 ] [1,.5,.25] and [ 1 , .5 ] [1,.5] , and a final density scale of 0.64, creating substantial but continuous relief. Grass over dirt is interrupted by moss on [ − .95 , − .80 ] [-.95,-.80] , coarse dirt on [ .51 , .60 ] [.51,.60] and [ .66 , .79 ] [.66,.79] , and gravel on [ .88 , .95 ] [.88,.95] . The biome retains caves, underground lava lakes, amethyst geodes, springs, glow lichen, stone variants, sediments, bamboo, vines, warm flowers, Jungle grass, mushrooms, pumpkins, and a Vanilla melon patch.

  The minecraft:trees_jungle wrapper selects 68 attempts with weight 9 or 69 with weight 1 (68.1 per chunk in expectation), applying horizontal, surface-water, heightmap, and biome filters. This high count produces the dense canopy and short sight lines visible above rather than a manually placed corridor layout. Additional wrappers assign five sugar-cane attempts, two large-fern attempts, and two water-lily attempts per chunk. Rarity filters request an extra melon patch, mossy boulder, huge brown mushroom, and surface lake once per 4, 10, 40, and 14 chunks in expectation. The boulder and mushroom are loot-free visual features. Cows spawn with weight 24 in groups of four and retain 1–2 base leather; parrots and pandas preserve biome-specific passive variation. The archive must be selected for a new world and enabled without another scene pack.

  Band

  Attempts per chunk

  Size

  Origin Y Y (inclusive)

  Coal (accessible)

  96

  20

  44–112

  Iron (accessible)

  84

  16

  40–92

  Copper (accessible)

  72

  20

  40–96

  Gold (accessible)

  48

  12

  32–72

  Redstone (accessible)

  48

  14

  28–68

  Lapis (accessible)

  36

  12

  30–68

  Emerald (accessible)

  24

  8

  40–80

  Diamond (main)

  40

  10

  36–68

  Diamond (shallow)

  8

  6

  52–72

  Obsidian (main)

  8

  12

  36–64

  Obsidian (shallow)

  10

  10

  50–72

  Table 13: Custom resource bands for Jungle; retained Vanilla ore placements are omitted.

  Note. Attempts count sampled origins per chunk; size is the configured ore-feature parameter. The final column bounds the sampled origin, not every generated block.

  The biome definition retains ordinary Vanilla coal, iron, copper, gold, redstone, lapis, and diamond placements, then appends the 11 custom entries in Table 13 ; these rows are not total ore counts. Emerald and obsidian occur only through custom entries in this biome list. Each entry samples horizontal and vertical origins with in_square and an inclusive uniform height range, checks the biome, and targets stone- or deepslate-replaceable blocks. Air-exposure discard is 0.0, while failed targets, overlap, and ore geometry can still reduce realized output.

  Main and shallow diamond and obsidian bands keep underground accessibility consistent with the other controlled biomes. The disabled large noise-vein system is separate from the ordinary Vanilla placed ores retained above. Consequently, the Jungle page isolates dense surface navigation and target occlusion without introducing a different custom resource-band schedule. The cross-section summarizes this schedule but is not a block-exact rendering of any generated chunk.

  Savanna

  Game view Cross-section schematic

  The Savanna artifact targets Minecraft Java 1.19.0 (pack format 10) and defines absavanna:benchmark_savanna as a single-biome Overworld preset spanning Y = − 64 ​ – ​ 319 Y=-64\text{--}319 , with sea level 63, aquifers, and the standard cave carvers. Its biome has no precipitation, temperature 2.0, and downfall 0.0; the Nether and End retain their Vanilla generators. The setup function fixes Peaceful difficulty, noon, and clear weather, keeps natural mob spawning active, and disables patrol, wandering-trader, and insomnia spawning.

  Terrain density combines a vertical gradient from 1.0 1.0 at Y = 46 Y=46 to − 1.0 -1.0 at Y = 96 Y=96 with broad-contour and local-undulation noise weighted by 0.68 and 0.22. The two cached fields use first octaves − 8 -8 and − 5 -5 , amplitudes [ 1 , .5 , .25 ] [1,.5,.25] and [ 1 , .5 ] [1,.5] , and a final scale of 0.64, producing the widest vertical relief range among the five setups. Grass over dirt is the base surface. Surface-noise intervals add coarse dirt on [ − .95 , − .80 ] [-.95,-.80] and [ .51 , .60 ] [.51,.60] , gravel on [ .66 , .79 ] [.66,.79] , and terracotta on [ .88 , .95 ] [.88,.95] , yielding dry patches and shallow terracing. Caves, underground lava lakes, amethyst geodes, springs, glow lichen, stone variants, sediments, tall grass, warm flowers, Savanna grass, mushrooms, and pumpkins remain in the feature list.

  Acacia placement selects two or three attempts with equal probability (2.5 per chunk in expectation) and rejects positions with nonzero surface-water depth. Sugar cane receives six attempts and water lilies one attempt per chunk. Rarity filters request an extra melon patch, an additional pumpkin patch, a mossy boulder, a small well, and a surface lake once per 16, 180, 14, 128, and 16 chunks in expectation. The boulder and well are loot-free placed features rather than structures, so the optional Generate Structures setting does not remove them. Cows spawn with weight 24 in groups of four and provide 1–2 base leather; horses and donkeys retain characteristic open-biome variation. The archive must be enabled alone when creating a new world because old chunks are not regenerated.

  Band

  Attempts per chunk

  Size

  Origin Y Y (inclusive)

  Coal (accessible)

  96

  20

  44–112

  Iron (accessible)

  84

  16

  40–92

  Copper (accessible)

  72

  20

  40–96

  Gold (accessible)

  48

  12

  32–72

  Redstone (accessible)

  48

  14

  28–68

  Lapis (accessible)

  36

  12

  30–68

  Emerald (accessible)

  24

  8

  40–80

  Diamond (main)

  40

  10

  36–68

  Diamond (shallow)

  8

  6

  52–72

  Obsidian (main)

  8

  12

  36–64

  Obsidian (shallow)

  10

  10

  50–72

  Table 14: Custom resource bands for Savanna; retained Vanilla ore placements are omitted.

  Note. Attempts count sampled origins per chunk; size is the configured ore-feature parameter. The final column bounds the sampled origin, not every generated block.

  The biome definition retains ordinary Vanilla coal, iron, copper, gold, redstone, lapis, and diamond placements, then appends the 11 custom entries in Table 14 . Emerald and obsidian occur only through custom entries in this biome list. Each band uses a per-chunk count, in_square , inclusive uniform origin heights, and a biome filter, then replaces stone- or deepslate-replaceable blocks with air-exposure discard 0.0. Attempts can fail or overlap, and configured size does not guarantee the realized number of blocks.

  The paired main and shallow diamond and obsidian bands match the custom schedule used in the other controlled biomes, while the Savanna-specific terrain and surface ecology determine the visible navigation differences. Disabling the separate large noise-vein system does not remove the ordinary Vanilla placed ores listed in the biome. The cross-section above is illustrative rather than block-exact.

  Appendix K Rule Suite Manifests and Detailed Results

  Table 7 gives the matched Vanilla references; the next three pages report detailed Mirror results. The six cards summarize rule changes and route effects provided only to ReAct w/rules. Archive suites Mirror01–Mirror04, Mirror06, and Mirror08 are relabeled M01–M06 in that order.

  M01: Quantity scarcity Rules. One trunk yields two matching planks; two planks yield one stick; raw iron smelts into three nuggets; three sugar canes yield one paper. Effect. Lower yields add gathering and repeated processing to otherwise familiar crafting and smelting routes.

  M02: Byproduct redistribution Rules. A trunk drops four matching planks; iron ore adds an ingot; diamond ore adds obsidian; sugar cane adds leather; the book recipe yields an enchanting table. Effect. Task resources become byproducts or substituted outputs, exposing useful noncanonical routes.

  M03: Yield expansion Rules. One trunk yields six matching planks; smelting or blasting raw iron yields two ingots; the diamond-pickaxe recipe yields a netherite pickaxe; three sugar canes yield one book; each diamond-ore variant yields two diamonds and one lapis lazuli. Effect. Expanded and substituted outputs shorten collection, crafting, and smelting routes.

  M04: Compressed resource drops Rules. Coal, iron, diamond, and lapis ore variants respectively drop a coal block, raw-iron block, diamond block, and lapis block; cows drop one to two leather, one to three beef, and one bookshelf. Effect. Mining yields compressed resources that can be unpacked through crafting into ordinary task inputs, while cows create a direct bookshelf route.

  M05: Route substitution Rules. The furnace recipe yields a blast furnace; iron- and diamond-pickaxe recipes yield their corresponding hoes; smelting or blasting an iron hoe yields an iron pickaxe, while blasting a diamond hoe yields a diamond pickaxe. Effect. Pickaxe production becomes an ordered detour through hoes and furnace processing.

  M06: Byproduct redistribution Rules. Each trunk drops four matching planks and two sticks; iron ore adds one paper; diamond ore adds one lapis lazuli; lapis ore adds two obsidian; cows retain their standard drops and add two paper. Effect. Paper, lapis, obsidian, and sticks become byproducts of ordinary gathering.

  Detailed Mirror Results M01 and M02

  Gemini DeepSeek

  Task Configuration Score / SR Actions Time Cost Score / SR Actions Time Cost

  Iron Armor Direct LLM 54.0 / 0.0 50.0 13.1 0.04 40.3 / 0.0 48.0 12.0 0.01

  ReAct 77.2 / 43.3 47.6 12.8 0.08 78.3 / 60.0 45.2 11.6 0.03

  ReAct w/rules 84.8 / 56.7 46.5 12.9 0.07 84.5 / 63.3 44.9 12.0 0.03

  Reflexion 85.2 / 63.3 45.5 13.2 0.07 76.3 / 46.7 46.9 11.0 0.03

  Voyager 84.2 / 63.3 42.7 13.5 0.06 75.3 / 46.7 45.3 11.5 0.02

  XENON 45.2 / 0.0 48.7 13.2 0.09 68.2 / 36.7 47.1 12.4 0.04

  ADAM 57.5 / 10.0 49.1 18.6 0.14 71.2 / 36.7 46.7 13.6 0.04

  Diamond Direct LLM 58.2 / 0.0 50.0 17.7 0.04 40.7 / 0.0 50.0 12.0 0.01

  ReAct 84.7 / 53.3 47.0 16.9 0.08 88.3 / 63.3 47.4 12.3 0.03

  ReAct w/rules 92.0 / 73.3 46.2 16.4 0.07 87.0 / 56.7 47.3 11.2 0.03

  Reflexion 88.7 / 66.7 46.8 16.8 0.07 77.3 / 26.7 48.3 12.2 0.03

  Voyager 87.3 / 60.0 47.0 16.8 0.06 72.2 / 23.3 49.6 14.1 0.02

  XENON 59.0 / 10.0 49.9 17.0 0.09 69.7 / 20.0 49.7 14.0 0.05

  ADAM 64.8 / 3.3 50.0 24.5 0.14 81.0 / 36.7 49.2 14.4 0.05

  Enchantment Direct LLM 48.0 / 0.0 75.0 32.3 0.10 23.7 / 0.0 75.0 10.5 0.02

  ReAct 82.3 / 3.3 75.0 30.1 0.17 73.0 / 0.0 75.0 11.1 0.06

  ReAct w/rules 80.7 / 6.7 75.0 31.4 0.16 73.5 / 0.0 75.0 11.2 0.06

  Reflexion 77.5 / 0.0 75.0 43.2 0.18 53.3 / 0.0 75.0 14.8 0.06

  Voyager 64.8 / 0.0 75.0 30.8 0.13 44.2 / 0.0 75.0 12.5 0.05

  XENON 69.3 / 0.0 75.0 35.7 0.22 73.5 / 0.0 75.0 13.8 0.09

  ADAM 64.5 / 0.0 75.0 44.8 0.28 59.0 / 0.0 75.0 14.1 0.09

  Table 15: Results for M01 (Quantity scarcity) by task, agent configuration, and model.

  Gemini DeepSeek

  Task Configuration Score / SR Actions Time Cost Score / SR Actions Time Cost

  Iron Armor Direct LLM 76.7 / 33.3 40.9 3.9 0.02 70.0 / 20.0 43.7 3.5 0.01

  ReAct 97.7 / 93.3 25.9 3.6 0.04 100.0 / 100.0 26.1 3.2 0.02

  ReAct w/rules 98.8 / 96.7 25.1 3.9 0.04 100.0 / 100.0 26.2 3.2 0.02

  Reflexion 100.0 / 100.0 23.8 4.1 0.04 96.5 / 90.0 28.0 3.1 0.02

  Voyager 100.0 / 100.0 22.9 3.9 0.03 97.7 / 93.3 27.5 3.6 0.01

  XENON 72.3 / 26.7 42.5 3.6 0.05 97.7 / 93.3 28.9 3.6 0.02

  ADAM 76.2 / 43.3 40.2 5.4 0.07 94.2 / 83.3 30.5 3.6 0.02

  Diamond Direct LLM 76.0 / 26.7 43.5 5.3 0.02 51.8 / 0.0 50.0 3.5 0.01

  ReAct 99.0 / 96.7 26.2 5.0 0.04 93.0 / 76.7 33.1 3.2 0.02

  ReAct w/rules 98.0 / 93.3 26.3 5.1 0.04 99.0 / 96.7 26.4 3.5 0.02

  Reflexion 97.0 / 90.0 29.0 5.4 0.04 93.0 / 76.7 34.3 3.6 0.02

  Voyager 94.0 / 80.0 29.8 5.2 0.03 87.0 / 56.7 37.5 4.2 0.01

  XENON 79.3 / 33.3 41.4 5.0 0.05 84.0 / 46.7 39.3 3.9 0.03

  ADAM 83.7 / 50.0 41.9 7.3 0.08 86.0 / 53.3 38.9 4.2 0.03

  Enchantment Direct LLM 63.5 / 0.0 75.0 10.4 0.05 32.8 / 0.0 70.9 3.2 0.01

  ReAct 91.2 / 36.7 67.0 9.5 0.10 77.0 / 0.0 71.0 3.6 0.03

  ReAct w/rules 91.0 / 30.0 67.1 10.1 0.10 80.2 / 3.3 69.5 3.4 0.03

  Reflexion 83.5 / 10.0 72.0 11.7 0.10 59.5 / 0.0 71.1 4.5 0.03

  Voyager 82.2 / 6.7 72.6 9.1 0.06 59.8 / 0.0 70.8 3.6 0.03

  XENON 77.3 / 3.3 74.4 11.5 0.13 77.3 / 3.3 70.1 4.1 0.05

  ADAM 76.3 / 3.3 75.0 13.0 0.16 74.0 / 0.0 70.7 3.8 0.05

  Table 16: Results for M02 (Byproduct redistribution) by task, agent configuration, and model.

  Detailed Mirror Results M03 and M04

  Gemini DeepSeek

  Task Configuration Score / SR Actions Time Cost Score / SR Actions Time Cost

  Iron Armor Direct LLM 78.2 / 43.3 42.4 9.5 0.03 74.5 / 30.0 43.2 7.6 0.01

  ReAct 96.5 / 90.0 34.6 9.0 0.06 98.8 / 96.7 36.6 8.1 0.02

  ReAct w/rules 98.8 / 96.7 32.7 8.3 0.06 100.0 / 100.0 34.2 7.6 0.02

  Reflexion 98.8 / 96.7 34.2 9.2 0.06 98.8 / 96.7 34.3 7.6 0.03

  Voyager 98.8 / 96.7 32.8 9.1 0.04 100.0 / 100.0 37.9 8.5 0.02

  XENON 78.0 / 40.0 41.9 9.2 0.07 98.8 / 96.7 33.8 9.8 0.03

  ADAM 80.3 / 46.7 45.2 12.7 0.11 97.7 / 93.3 36.7 9.4 0.03

  Diamond Direct LLM 76.5 / 33.3 44.6 11.9 0.03 52.0 / 3.3 49.7 9.3 0.01

  ReAct 99.0 / 96.7 37.2 11.0 0.06 100.0 / 100.0 39.5 8.3 0.02

  ReAct w/rules 97.0 / 90.0 37.8 10.2 0.06 99.0 / 96.7 37.0 8.8 0.02

  Reflexion 95.0 / 83.3 38.8 11.7 0.07 97.0 / 90.0 38.7 9.2 0.03

  Voyager 96.0 / 86.7 38.8 11.1 0.04 85.7 / 56.7 42.7 10.1 0.02

  XENON 68.7 / 10.0 48.3 11.1 0.08 86.5 / 60.0 41.8 9.8 0.04

  ADAM 77.5 / 36.7 46.0 14.7 0.12 95.0 / 83.3 42.2 10.7 0.04

  Enchantment Direct LLM 66.2 / 0.0 75.0 25.1 0.08 39.3 / 0.0 75.0 8.1 0.02

  ReAct 91.5 / 20.0 73.8 23.5 0.13 83.0 / 0.0 75.0 7.6 0.04

  ReAct w/rules 91.8 / 33.3 71.8 23.0 0.14 90.8 / 23.3 70.1 7.2 0.04

  Reflexion 85.3 / 3.3 74.3 27.2 0.14 77.0 / 0.0 75.0 10.6 0.05

  Voyager 85.7 / 16.7 71.0 21.1 0.10 66.5 / 0.0 75.0 8.2 0.04

  XENON 80.8 / 13.3 74.0 25.6 0.18 82.3 / 3.3 74.7 9.9 0.07

  ADAM 75.2 / 3.3 74.4 31.2 0.23 79.5 / 0.0 75.0 8.8 0.06

  Table 17: Results for M03 (Yield expansion) by task, agent configuration, and model.

  Gemini DeepSeek

  Task Configuration Score / SR Actions Time Cost Score / SR Actions Time Cost

  Iron Armor Direct LLM 62.7 / 13.3 47.8 10.1 0.05 45.7 / 16.7 32.1 8.0 0.01

  ReAct 82.7 / 53.3 44.3 9.9 0.08 50.5 / 13.3 46.6 9.0 0.04

  ReAct w/rules 84.8 / 56.7 44.1 9.5 0.08 67.5 / 43.3 43.5 8.9 0.04

  Reflexion 94.2 / 83.3 41.0 9.5 0.07 64.0 / 36.7 45.3 9.6 0.04

  Voyager 93.0 / 80.0 40.2 9.2 0.05 77.7 / 53.3 44.9 10.0 0.03

  XENON 60.3 / 6.7 48.7 10.3 0.09 75.2 / 46.7 43.3 10.9 0.05

  ADAM 58.7 / 13.3 49.3 15.4 0.15 84.2 / 63.3 42.3 10.0 0.05

  Diamond Direct LLM 55.7 / 0.0 50.0 14.5 0.05 36.2 / 13.3 34.0 9.5 0.01

  ReAct 82.0 / 40.0 46.3 13.1 0.08 48.5 / 26.7 47.1 9.7 0.04

  ReAct w/rules 87.0 / 56.7 47.3 12.8 0.09 58.0 / 30.0 46.4 9.0 0.04

  Reflexion 89.0 / 63.3 46.3 12.8 0.08 52.8 / 23.3 48.6 10.3 0.04

  Voyager 92.0 / 73.3 43.3 13.2 0.05 67.0 / 40.0 47.1 10.5 0.03

  XENON 63.3 / 3.3 49.9 12.9 0.09 88.0 / 76.7 42.6 10.2 0.05

  ADAM 64.5 / 10.0 49.5 18.3 0.15 70.0 / 56.7 45.8 10.8 0.05

  Enchantment Direct LLM 51.3 / 0.0 75.0 24.7 0.10 22.5 / 0.0 75.0 9.1 0.02

  ReAct 81.0 / 0.0 75.0 25.1 0.18 66.5 / 0.0 75.0 8.3 0.06

  ReAct w/rules 81.3 / 3.3 75.0 23.1 0.18 69.3 / 0.0 75.0 8.7 0.07

  Reflexion 75.7 / 6.7 75.0 32.0 0.19 53.7 / 0.0 75.0 11.6 0.07

  Voyager 72.5 / 0.0 75.0 24.2 0.14 48.8 / 0.0 75.0 9.3 0.06

  XENON 72.0 / 0.0 75.0 28.7 0.22 69.8 / 0.0 75.0 10.5 0.10

  ADAM 65.0 / 0.0 75.0 35.9 0.31 62.2 / 0.0 75.0 10.1 0.09

  Table 18: Results for M04 (Compressed resource drops) by task, agent configuration, and model.

  Detailed Mirror Results M05 and M06

  Gemini DeepSeek

  Task Configuration Score / SR Actions Time Cost Score / SR Actions Time Cost

  Iron Armor Direct LLM 66.7 / 13.3 49.1 11.5 0.04 69.3 / 20.0 46.2 9.0 0.01

  ReAct 95.3 / 86.7 42.1 10.2 0.07 98.8 / 96.7 40.9 8.3 0.03

  ReAct w/rules 96.5 / 90.0 40.1 9.6 0.07 100.0 / 100.0 41.8 8.9 0.03

  Reflexion 95.3 / 86.7 39.2 10.4 0.08 98.8 / 96.7 43.3 8.4 0.03

  Voyager 97.7 / 93.3 39.3 10.0 0.05 100.0 / 100.0 42.0 9.5 0.02

  XENON 67.8 / 16.7 47.6 10.5 0.09 95.3 / 86.7 41.3 10.3 0.04

  ADAM 72.2 / 23.3 49.1 13.8 0.13 97.7 / 93.3 43.0 10.3 0.04

  Diamond Direct LLM 58.7 / 3.3 50.0 13.8 0.04 50.7 / 0.0 50.0 9.0 0.01

  ReAct 89.0 / 70.0 45.4 13.3 0.08 94.0 / 80.0 45.3 9.1 0.03

  ReAct w/rules 97.0 / 90.0 45.1 13.0 0.08 95.0 / 83.3 46.4 9.0 0.03

  Reflexion 90.0 / 73.3 45.7 13.6 0.08 88.3 / 63.3 44.7 9.3 0.03

  Voyager 87.0 / 63.3 45.4 14.5 0.06 78.3 / 36.7 48.7 11.7 0.03

  XENON 62.5 / 13.3 48.3 13.1 0.10 85.3 / 53.3 46.1 9.6 0.05

  ADAM 72.0 / 20.0 49.7 20.6 0.15 86.0 / 53.3 47.7 11.0 0.05

  Enchantment Direct LLM 59.0 / 0.0 75.0 25.9 0.09 34.7 / 0.0 75.0 9.1 0.02

  ReAct 85.3 / 13.3 74.9 23.9 0.17 81.7 / 6.7 74.8 8.6 0.06

  ReAct w/rules 87.7 / 16.7 74.8 25.8 0.17 85.2 / 6.7 75.0 8.5 0.06

  Reflexion 81.8 / 3.3 75.0 29.7 0.18 65.8 / 0.0 75.0 10.8 0.07

  Voyager 75.0 / 0.0 75.0 25.7 0.13 61.0 / 0.0 75.0 10.7 0.05

  XENON 69.5 / 0.0 75.0 26.4 0.24 81.8 / 3.3 75.0 10.8 0.09

  ADAM 65.5 / 0.0 75.0 34.1 0.29 72.5 / 0.0 75.0 10.5 0.08

  Table 19: Results for M05 (Route substitution) by task, agent configuration, and model.

  Gemini DeepSeek

  Task Configuration Score / SR Actions Time Cost Score / SR Actions Time Cost

  Iron Armor Direct LLM 77.8 / 36.7 43.8 11.1 0.04 72.3 / 26.7 43.3 9.4 0.01

  ReAct 95.3 / 86.7 38.7 10.0 0.07 100.0 / 100.0 36.5 8.7 0.03

  ReAct w/rules 97.7 / 93.3 35.9 10.7 0.08 100.0 / 100.0 37.5 8.1 0.03

  Reflexion 98.8 / 96.7 33.9 10.9 0.07 97.7 / 93.3 38.5 8.9 0.03

  Voyager 100.0 / 100.0 35.0 10.6 0.05 98.8 / 96.7 39.1 9.6 0.02

  XENON 70.8 / 16.7 47.0 11.1 0.09 94.2 / 83.3 38.2 10.2 0.04

  ADAM 72.3 / 26.7 46.5 14.2 0.13 96.5 / 90.0 39.5 11.5 0.04

  Diamond Direct LLM 72.0 / 20.0 48.1 14.1 0.05 60.8 / 13.3 46.7 9.9 0.01

  ReAct 95.0 / 83.3 39.6 13.2 0.08 95.3 / 86.7 38.9 8.5 0.03

  ReAct w/rules 98.0 / 93.3 38.3 12.8 0.07 95.3 / 86.7 41.1 8.8 0.03

  Reflexion 97.0 / 90.0 40.0 12.3 0.08 98.0 / 93.3 39.4 9.2 0.03

  Voyager 97.0 / 90.0 39.5 13.6 0.05 88.3 / 63.3 44.3 11.0 0.03

  XENON 75.0 / 30.0 45.4 12.9 0.09 87.3 / 60.0 42.4 10.7 0.04

  ADAM 80.0 / 40.0 47.7 17.1 0.14 86.0 / 60.0 45.6 10.6 0.05

  Enchantment Direct LLM 71.0 / 0.0 75.0 28.0 0.09 40.5 / 0.0 75.0 9.2 0.02

  ReAct 90.5 / 20.0 73.9 25.2 0.17 82.3 / 13.3 73.0 9.3 0.06

  ReAct w/rules 92.3 / 33.3 72.0 26.6 0.16 85.7 / 6.7 73.9 9.3 0.06

  Reflexion 86.7 / 16.7 74.3 32.5 0.18 68.3 / 0.0 75.0 11.9 0.07

  Voyager 83.0 / 0.0 75.0 24.8 0.13 64.8 / 0.0 75.0 10.4 0.05

  XENON 82.0 / 0.0 75.0 28.7 0.23 81.7 / 6.7 74.3 12.2 0.09

  ADAM 75.0 / 0.0 75.0 37.3 0.30 73.8 / 3.3 74.1 10.6 0.09

  Table 20: Results for M06 (Byproduct redistribution) by task, agent configuration, and model.