The capstone of the positive-chase (Part 8). This week we ran the control experiment for every claim we wanted to make — an off-policy baseline, a shaped-flat baseline, a config-bug fix, a 4× budget extension, and a per-loss ablation — on byte-identical environments. Every control changed its claim. What’s left is smaller than what we started with, and solid. All numbers trace to JSONs/logs in the campaign ledger; two corroborative cells are still running (marked).

The final scorecard

1. The wall: real, on-policy, gait-specific

On the identical MJX HopperHop (verified byte-for-byte — same registry env, episode length, action repeat):

• PPO (tuned config): peak 53.8, 0/5 seeds ≥200, through 472M steps/seed. Categorically walled.
• SAC (tuned config): 3/3 seeds cross 200 within 5M. The wall is an on-policy pathology — replay + entropy exploration escapes it.
• TD-MPC2: ~282 at 1M (n=4, best 367); 5M bests 295–610, mean 420 ± 113 (final, n=12, 07-05) — the flagship efficiency anchor at publication n: eight fresh seeds joined the original four (best 610, worst 295; a wide but honest spread).

And the wall does not generalize: FingerTurnHard PPO ≈973 (3/3, no wall); Pendulum solves once an upstream config-case bug (PendulumSwingUp vs PendulumSwingup silently skipping the tuned override) is fixed — peaks 842/831 (n=2); BallInCup is discovery-luck for both algorithms (PPO 1/3, TD-MPC2 1/2). (Addendum, later that day — a fourth control: CheetahRun, which our own older Pareto study had scored “PPO never reaches 500 at 30M.” With the tuned config and adequate budget, PPO reaches 892–922 (3/3 seeds at 285M), matching SAC (918/912 at 10M) — the old number was a budget/config artifact, corrected in the synthesis doc. CheetahRun is slow-but-converging, not walled; TD-MPC2’s edge there is purely efficiency, ~600+ within 0.55M steps.)

🚩 Framing revisions (2026-07-03, the wall-boundary probe — twice, because the controls kept teaching): we asked whether the wall is the hop gait or the hopper robot, running the same three arms on HopperStand (same morphology, easier objective). At n=2 PPO looked walled there too (149/142 @285M) and we briefly framed the wall as morphology-categorical. Extending to n=4 corrected that: PPO Stand = 149 / 142 / 144 / 681 — one seed in four escapes; and SAC Stand = 492 / 754 / 464 / 33 — one seed in four fails. The honest final picture, with five no-wall control tasks:

• HopperHop: a categorical on-policy wall, budget-indexed for the rest — PPO 0/5 ≥200 at 472M; SAC crosses 200 in 6/12 seeds by 5M (final, 07-04) and 6/9 by 8M (final 07-05: the 20M cohort crossed 5/5 at 4.1–7.7M, but four fresh direct-8M seeds went 1/4 — 374 vs 117–140 — SAC’s crossing by 8M is a ~2/3 rate, not a certainty); TD-MPC2 6/6 by ~1M. (07-05: the wall also survives the standard exploration knob — entropy_cost ×3 and ×10 leave PPO at peaks 4–74, n=2 each at 150M, no better than baseline; under-exploration hyperparameters are not the explanation. Instructive contrast on the graded Stand barrier below (n=4/arm final): entropy ×3 put 1/4 seeds over — 627 vs 118–150 — a rate consistent with the baseline’s 2/16 escape lottery, not a repair; ×10 walled all four (116–185). The knob neither repairs the graded barrier nor dents the categorical wall.)
• HopperStand: a graded but near-categorical barrier — PPO escapes 2/16 (final, 07-04: eight more seeds at the full 285M budget ALL walled at 105–195; both escapes — 681/749 — came from earlier 120M runs); SAC 0/3 at 1M and 7/10 by 5M (final, 07-04: solves 464–932; fails 33/100/300); TD-MPC2 3/3 by ≤0.9M (962 / 948 / 943 — the third seed by just 0.3M). The final three-method gradient on Stand — TD-MPC2 3/3 ≫ SAC 7/10 ≫ PPO 2/16 — and the matched-1M column (TD-MPC2 solves both hopper tasks at a budget where neither model-free method solves either) are the paper-grade statements.

So the durable, defensible claim is a reliability × efficiency ordering on hopper dynamics — TD-MPC2 ≫ SAC ≫ PPO, with orders-of-magnitude budget gaps — hardest exactly where the dynamics are unstable and contact-timing-critical, categorical only for the hop itself. Both of today’s framings (“gait-specific”, then “morphology-categorical”) died under one more seed each; this one is stated at the n we actually have. (TD-MPC2 Stand later firmed to 5/5: 962/948/943/959/959, all within 1M.)

Discriminator (07-04 evening, AcrobotSwingup — unstable but contact-free; all cells n=4 by 07-05): PPO learns fine (267–344 at 285M) — no wall, so the on-policy wall requires contact-criticality, not instability alone. And SAC is slow and inconsistent here even at 20M (42/66/123/207; 54–89 at 5M) — the off-policy advantage is also task-class-specific, not a generic superiority. TD-MPC2 leads on both task classes (422–454 within 1M, n=4) — of the three methods, the value-pathway advantage is the most portable. (Completing the humanoid matrix: SAC also solves HumanoidStand, 918/922 — off-policy SAC solves both humanoid tasks while PPO can’t run and TD-MPC2 diverges on both.)

2. Level vs efficiency: the 4× budget test

If the world model bought a higher level, SAC shouldn’t reach TD-MPC2’s plateau at any reasonable budget. At 20M steps (4× TD-MPC2’s), SAC’s finals are 246 / 277 / 285 / 414 / 572 (n=5; peaks to 574). The mean (~359) sits below TD-MPC2’s ~477, but the best seeds reach or exceed it — the distributions overlap. So:

The world model’s robust advantages are consistency (TD-MPC2: 4/4 seeds ≥282 at 1M; SAC: wild seed spread even at 20M) and ~4–5× sample-efficiency. Not a level ceiling.

This is the campaign’s own law — “abstraction buys speed, not ceiling” — arriving from a fourth independent direction, and it agrees with where the field has landed (BRO, SimBa, TD7, CrossQ: scaled/regularized model-free methods keep erasing model-based “level” gaps; the efficiency gap is the durable one).

3. Which part of the “world model” does the work? The value signal.

Per-loss ablation (zero ONE loss term, from scratch, mask verified live in logs; MPPI + π-only eval returns):

CheetahRun (decisive arms at n=4 as of 07-04, 1M): full = 738/782/721/795 (MPPI bests). Ablate value → 16/37/58 — dead 4/4 (π ≤12). Ablate reward → MPPI dead at 5/31/26 but π at FULL strength 761/796/805 — n=4 (planning-only, partly by construction — MPPI scores rollouts with it). Ablate policy prior → MPPI limps at 123/192/141 while π is dead at 2.5/9/11 — n=4. Ablate consistency — the self-predictive latent-dynamics loss itself → 367/516/558 MPPI (π 541/575/623), the smallest drop of the four — n=4. (07-04: with these, every arm of the five-loss ablation is at n≥4 on all three tasks.)

HopperHop — the exploration task (all five arms at n=4; the headline value cell firmed to n=6 on 07-03): full finds the gait on 6/6 seeds (MPPI best 287–570). Value-ablated: 0.0 / 0.0 / 3.2 / 0.0 / 0.0 / 0.1 (n=6) — the gait is never found. Ablating the TD value loss reproduces the exploration wall, six seeds out of six. Consistency-ablated still finds it at roughly half strength on 4/4 seeds (MPPI 185–245). Reward-ablated matches CheetahRun’s pattern: the planner is dead by construction (MPPI ≈0 — it scores rollouts with that head) but the policy still learns the gait (π 519 / 241 / 226 / 189, n=4). Policy-ablated is ≈0 on both readouts (n=4) — and here HopperHop is harsher than CheetahRun (where the planner alone still limped to 123): on the exploration task the planner cannot compensate for a missing prior at all. So the precise statement: the value-learning pathway — the TD value loss and the policy trained from it — is individually necessary (each ablation reproduces total failure); the reward head matters only for planning; and the self-predictive consistency loss is the only component whose removal merely degrades.

Replicated on a third task (WalkerRun, decisive arms at n=4, 1M): full 731/680/699/723 (MPPI best); value-ablated 56/28/39/38 — dead 4/4; policy-ablated 76/64/83/53 — dead 4/4 (seeds 3/4 dead through ≥900k); reward-ablated MPPI dead (44/44) but π at full strength 711/728/681/684 (n=4); consistency-ablated 547/522 & 533/483 (π to 667) — the mildest cut yet again, now n=4. The mechanism table now spans three tasks — CheetahRun n=4, HopperHop value-cell n=6, WalkerRun n=4 on every arm — with the same shape everywhere.

Sufficiency grid (revised 07-07): removability is the exception, not a task-class rule. Training consistency-OFF from scratch at the full 5M budget, four tasks in:

• HopperHop (exploration-bound): stripped 165/475/481/511 vs full 420 ± 113 (n=12) — three seeds at the top of the full band, reaching 440+ by ~2M. Removable for typical seeds; residual role = worst-seed insurance.
• WalkerRun (dense): stripped 537/574/554/594 (mean 565) vs full 709/705/753/782 (mean 737) — −23%, non-overlapping.
• CheetahRun (dense): stripped 527/528/516/524 (mean 524) vs full 903/904/782/806 (mean 849) — −38%, non-overlapping.
• AcrobotSwingup (exploration-flavored, 07-07 morning): stripped 297/233/352/256 (mean 284) vs full 533/511/513/488 (mean 511) — −44%, non-overlapping. Honest revision: our 07-06 “exploration vs dense” split — written when only Hop and Walker were in — breaks on Acrobot, which is exploration-flavored yet shows the largest gap. The reading that survives all four tasks: the consistency loss supports MPPI rollout quality wherever the planner carries learning (Acrobot, Walker, Cheetah are all planner-led here); HopperHop — where the policy head learns the behavior directly and the planner mostly amplifies it — is the removable case. CartpoleSwingupSparse was queued as the fifth test, but the full-model baseline itself stalls on this sparse task (bests 0.0/0.0/1.3/0.0 at 5M, n=4 — a known TD-MPC2 failure mode with default exploration), so that cell is a degenerate both-fail rather than an informative sufficiency probe; the grid’s evidentiary core stays the four tasks the full model solves. The HopperHop removable cell now holds at n=8: four fresh stripped seeds land at 306/449/443/477 (mean 419), so 7 of 8 stripped seeds sit inside the full-model band 420±113 — removability confirmed, with the lone 165 floor-dip the only exception.

Replicated on a FOURTH task (HopperStand, n=4/arm, 07-05) — a task the full model solves in 0.3M steps: none 911–946 (all solve); value-ablated 6–13 — dead; policy-ablated 9–34 (π ≤5) — dead; reward-ablated MPPI degraded to 265–542 but π at 944/515/943/944; consistency-ablated 816–898 (π to 930; one seed at eval 13/20 when written) — here the mildest cut is nearly no cut at all. Same shape, fourth task: without the TD value loss, TD-MPC2 cannot even learn to stand.

“The world model explores” decomposes into: the TD value signal trained through the latent is what discovers and ranks behavior; the self-predictive dynamics loss is a helpful regularizer, not the key. This rhymes with the entire campaign — from the redundancy nulls to the anti-collapse reversal, the value anchor keeps being the thing that matters.

4. Hierarchy: the win was the signal, not the structure

The fourroom result (feudal 4/6, sparse-flat 0/6) survived n=6 — and then the right control killed its interpretation: flat TD3 + dense potential shaping toward the goal solves 3/6 — statistically the same as feudal. What carried the win was density of learning signal, not hierarchy. The surviving claim is narrower and still interesting: the feudal agent manufactured that dense signal itself (its low level trains on distance to self-proposed subgoals; no privileged goal information), whereas the control had to be handed the true goal. This independently replicates Nachum et al. (2019): most of HRL’s measurable benefit reproduces on flat agents given better exploration/shaping.

4b. Addendum (07-04): does the ordering generalize? The humanoid says no — it inverts

The hopper result predicts “TD-MPC2 ≫ SAC ≫ PPO wherever dynamics are unstable and contact-critical.” We tested that prediction on a second unstable morphology, HumanoidWalk (21-DoF), same three-arm design. The prediction failed — informatively. Every method broke, each along a different axis:

• PPO: numerically unusable — reward=nan under all 3 configs tried (brax defaults; a tuned-WalkerRun transplant with 512³ networks + reward_scaling 0.1; same + reward_scaling 1.0 + lr 1e-4). Notably there is no official tuned humanoid config in mujoco_playground’s DMC params — the fragility is upstream reality, not our bug.
• TD-MPC2 (final, 9 runs): under the default config, training diverges to loss=nan on 6/6 walk seeds (onsets 0.53–2.51M) and on the easier HumanoidStand (0.28M) — 7/7, effectively deterministic with variable onset; best return before any nan was 30.4. Lowering the update-to-data ratio shifts the onset stochastically rather than fixing it (k_update=32 diverged at 3.13M; k_update=64 was the only run of nine to finish 4M) — and the nan-free survivor never learned anyway (best 21.8, flat ~20 for 4M steps). So the humanoid failure is not merely numerical: under every setting tried, TD-MPC2 does not crack the 21-DoF task at these budgets. The same architecture solved hopper Stand at 0.3M.
• SAC: the only robust method — 4/5 seeds solve (625–909 at 5M), 1/5 hit a nan of its own.

So the reliability ordering is task-class-dependent, and on the high-DoF humanoid it inverts: plain off-policy SAC beats both the on-policy and the model-based method — not on sample-efficiency but on working at all. Config/numerical fragility is a distinct failure axis from exploration, and any “X beats PPO” claim needs to say which axis it lives on. This sharpens rather than weakens the paper: the hopper ordering is real, matched, and quantified — and it is a statement about hopper-class dynamics, not a universal law.

5. What this week actually taught

1. Run the control before the claim. Four headline claims went into this week; four came out rewritten by their own controls — every time into something smaller and truer. (The full adversarial review that mandated these controls, and the retraction bookkeeping, is in the campaign ledger and the R²/Pareto explainer.)
2. The durable positives of the whole program: (a) explicit abstraction is redundant on a strong value-anchored latent (Paper A, submission-ready); (b) structured priors and world models buy sample-efficiency and consistency — never a ceiling — across priors, hierarchy, planning, and model-based RL (the speed-of-learning paper); (c) on-policy PPO has a real, reproducible, task-specific exploration failure mode that off-policy and model-based methods don’t share — and its mechanism, in TD-MPC2’s case, is value learning, not dynamics modeling.
3. Complete. All 20 HopperHop ablation runs finished (the mid-campaign host outage turned out to be network-only; the jobs never stopped). Nothing in this post is pending.

What’s next (07-05): the 5M efficiency anchor is being extended to n=12 overnight (last wide-CI number); then this post’s content gets assembled into a conference paper (matched-env wall + budget-indexed rates + 4-task value-pathway mechanism + Acrobot/humanoid boundary results), with the redundancy-criterion paper (Paper A) already at submission-readiness in parallel.

Every number: campaign ledger bet2_null_results.md (disk paths per entry). Same-env verification, config audits, and the two bugs found along the way (Pendulum case mismatch; an A1 noise confound) are documented in the 2026-07-02 full-arc review.

Part 21 was an audit. This one is a glossary and a reckoning. Every term the last two days of results have leaned on, explained from scratch with the actual numbers — and then the question that matters most: have we been ablating the thing we said we were ablating?

Part 1 — The setup, in plain language

We train the same agent twice on the same task. One copy is normal. The other has one loss term switched off. Then we ask whether the second copy is worse.

That’s it. Everything else is bookkeeping about how much worse, how reliably, and what kind of worse.

Two details make the bookkeeping harder than it sounds:

Runs are random. Each training run starts from a different random seed — different network initialisation, different exploration noise, different data. Two identical configurations can land far apart. So we don’t run each configuration once; we run it many times and look at the spread.

The number we compare is not the last number. An agent’s score bounces between evaluations. On pendulum a single run read 360 at one checkpoint and 0 at the next. So we take the last-6-evaluation median: the six most recent evaluations of a finished run, sorted, middle value. One number per run, robust to a single bad checkpoint.

Part 2 — The vocabulary

“A cell”

Our main table is a grid: rows are tasks (cheetah, walker, ball-in-cup, …), columns are the four combinations of full vs stripped agent × policy π vs planner Π. Each box in that grid is a cell.

“Which cell moved” is shorthand for: when we switched off the loss term, which of those four boxes changed? If the full-vs-stripped difference shows up under the planner but not under the policy, that tells you the effect lives in planning, not in the representation. Acrobot does exactly that, and that’s how we know.

“Per-seed values”

Instead of reporting one number per configuration, we report the whole list. Cheetah’s stripped arm at n=12:

498.3  543.5  566.8  572.6  586.4  587.6  603.4  612.1  615.7  617.5  627.8  648.1

Twelve runs, twelve numbers. Reporting the list rather than the average is the single most useful thing this campaign started doing, because averages hide the two shapes below.

“Unimodal” vs “bimodal”

Unimodal means the numbers cluster around one value. Cheetah’s stripped arm above is unimodal: it runs smoothly from 498 to 648 with no gaps. Every run learned the task; they just learned it to slightly different degrees.

Bimodal means the numbers fall into two separate clumps with a gap between. Finger-spin’s stripped arm at n=9:

641.1  701.3  702.6  │   926.4  930.5  948.9  953.9  957.5  966.7
└──── three runs ────┘   └──────────── six runs ─────────────────┘
                    224-point gap

That is not “the average is 838.” No run scored 838. Three runs failed to learn properly and six learned fine. The mean describes nothing that happened.

Why it matters so much: the right summary statistic depends on the shape. For a unimodal arm, an average or a gap is meaningful. For a bimodal arm, the only honest summary is a success rate — how many runs landed in the good clump. Using an average on a bimodal arm is how we ended up retracting two claims.

“Graded” vs “all-or-nothing”

This follows directly.

• Graded: every run learns, and the ablation changes how well. Cheetah’s full arm averages 753, stripped averages 574. Everyone succeeds; the stripped ones succeed less. The effect is a continuous quantity.
• All-or-nothing: runs either succeed completely or fail completely, and the ablation changes how often. Ball-in-cup’s full arm solves 8 times out of 14 — and when it solves, it scores 972–976, essentially a perfect score. When it doesn’t, it scores 0. There is no middle.

These are different phenomena and they need different statistics. That distinction is the backbone of the current hypothesis.

“Inversions”, and “0/144”

Take every full-arm run and pair it with every stripped-arm run. With 12 runs each, that’s 12 × 12 = 144 pairs. An inversion is a pair where the stripped run beat the full run — the “wrong” way round.

• Cheetah at n=12: 0/144. Every single full run scored higher than every single stripped run. The two lists don’t overlap at all.
• Walker at n=12: 16/144. Mostly ordered, but 16 pairs go the wrong way — the lists overlap.

“0/144 inversions” is the cleanest possible result for a comparison of this kind. It’s what “complete separation” means, stated as a count.

“p = 3.7 × 10⁻⁷” — where that number comes from

This is an exact permutation test, and it’s simpler than it looks.

Ask: suppose the labels are meaningless. Suppose those 24 cheetah numbers are just 24 numbers, and calling twelve of them “full” and twelve “stripped” is arbitrary. How many ways are there to split 24 numbers into two groups of 12?

Of those 2.7 million possible splits, how many put all twelve highest numbers in one group? Exactly one. So if the labels meant nothing, the chance of seeing a split this extreme is 1 in 2,704,156 —

No distributional assumptions, no normality, no t-test. Just counting arrangements. That’s why it’s called exact.

“The design floor”

Notice what that implies: with 12 runs per arm, the smallest p you can ever get is 1/2,704,156, because perfect separation is the most extreme outcome available. That’s the design floor — the best possible p-value for that sample size.

This table cost us a day. The DreamerV3 hopper comparison was run at 2 seeds per arm, where the best possible p is 0.167 — it could never have reached significance no matter how large the effect was. We’d have run it, seen a gap, and reported something unsupportable. Now the first question about any design is: what’s the best p this could produce?

“The margin”

The margin is the gap between the worst full run and the best stripped run. Cheetah at n=12: worst full is 681.25, best stripped is 648.10, so the margin is 33.15 points.

An inversion count and a margin measure different things. Zero inversions says the lists don’t overlap. The margin says by how much. You can have zero inversions with a huge gap (robust) or with a hair’s-breadth gap (fragile).

And here’s the finding that surprised us: margins shrink as you add runs. Cheetah’s margin was 92.85 at n=3, n=5, and n=8 — rock stable across ten added runs — and then fell to 33.15 at n=12, when an unusually weak full run (681) and an unusually strong stripped run (648) both showed up.

That’s not bad luck. It’s arithmetic. The margin depends on the extremes of both lists, and the more runs you do, the more extreme your extremes get. A margin is an extreme-order statistic, and it drifts downward with n by construction. So a stable margin is evidence only relative to the n at which you measured it. We had asserted cheetah’s margin was stable; at n=12 that turned out to be wrong, and correcting it produced a better methodological point than the original claim.

Part 3 — The hard question: did we ever remove the world model?

Short answer: no, and this is the most important thing in the paper.

What TD-MPC2’s latent dynamics is trained by

TD-MPC2 has a learned dynamics network d(z, a) — given a latent state and an action, predict the next latent state. Three separate signals train it:

1. Self-prediction (the “consistency” loss). Roll d forward one step, and pull the result toward a target encoding of the actual next observation. This is what grounds the latent in the world.
2. The reward head. Predict reward from latents that were produced by rolling d forward. Errors flow backward through d.
3. The value head. Same structure: Q-values are computed on rolled-out latents, so its gradients also flow back through d.

Our ablation sets consistency_coef = 0. That deletes only signal 1.

Signals 2 and 3 remain untouched, and the planner still rolls trajectories through d when choosing actions. So the “stripped” agent:

• still has a dynamics model
• still trains it, using reward and value prediction
• still plans through it

It has lost exactly one thing: the requirement that its imagined next state resemble the actual next state. It no longer has to model the world — only the parts of the world that affect reward and value.

Why this isn’t a flaw, and what it makes the paper about

That object has a name. It’s a value-equivalent model — Grimm et al. (2020), and the principle behind MuZero. The idea: a model doesn’t need to predict observations, only quantities that affect decisions.

So our experiment is not “world model vs no world model.” It’s:

Does observation-grounded self-prediction earn its keep on top of value equivalence?

This is a narrower question and a better one. It’s the question the value-equivalence literature actually poses, and it’s answerable — where “does a world model help?” is under-specified, because it depends entirely on what you mean by removing it.

It also explains our results in a way the loose framing couldn’t. The stripped agent stays competitive on dense-reward tasks like cheetah and walker, because reward and value gradients alone are enough to shape a usable latent there. It falls apart on acrobot’s planner, because planning three steps through a latent that was never grounded in observations produces garbage. The old framing had to call those unrelated. The value-equivalence framing predicts both.

What we genuinely cannot test: a true model-free TD-MPC2 — no dynamics network at all. That’s not an ablation, it’s a different algorithm. The closest reference point is the tuned SAC baseline from the official TD-MPC2 release, and we use it as exactly that.

Part 4 — So is that why Dreamer behaves differently?

Partly — but let me correct a framing first, because the difference is real and the word “crash” is doing damage.

What actually happened. In DreamerV3 on hopper, the ablated agent scored 188 against the full agent’s 322. That’s a substantial degradation. It is not a crash to zero. The thing that genuinely collapsed in this campaign was the JEPA arm — and that was a configuration pathology (the anti-collapse regulariser saturating against a one-hot softmax), not a consequence of the ablation.

Why the same nominal ablation has different leverage. DreamerV3’s ablation is loss_scales.dyn = 0, which switches off the KL term pulling the RSSM’s prior toward its posterior. And here’s the structural difference:

• DreamerV3 trains its actor and critic entirely inside imagination. They never see real states. Every gradient they receive comes from trajectories the model dreamed. Break the prior and you have poisoned the only source of training signal the controller ever gets.
• TD-MPC2 trains its policy on real latents, and its planner only rolls 3 steps before falling back on a learned value estimate. Break the self-prediction and you’ve degraded a 3-step lookahead that was already backstopped.

So: same switch, very different blast radius. In Dreamer the model is the training environment. In TD-MPC2 the model is a short-horizon lookahead attached to a controller that has other sources of signal.

This is a genuine confound for cross-family comparison, and the paper says so. If we reported “Dreamer degrades more than TD-MPC2, therefore generative world models matter more,” we’d be measuring the ablations’ leverage, not the architectures’ reliance.

Part 5 — Do we need to fix it?

Three options were on the table.

Option 1 — actually delete the model from TD-MPC2. No dynamics network at all. Rejected: that’s not an ablation of the model, it’s a swap to a different algorithm, and the comparison it produces (model-based vs model-free) is the one the SAC baseline already provides.

Option 2 — match the ablations across families. Find a Dreamer intervention with the same leverage as TD-MPC2’s. Rejected as intractable inside a week: the architectures differ in exactly the way that makes “the same leverage” hard to define. Dreamer has no equivalent of “the reward head still trains the dynamics,” because in Dreamer the dynamics is the substrate everything else runs on.

Option 3 — state precisely what is removed and make the claim about that. This is what we did, and I want to be clear it isn’t a consolation prize. Naming the ablation correctly turned a vague question into a sharp one, made the results explicable rather than merely tabulated, and connected the work to an existing theoretical framework instead of leaving it as a pile of ablation numbers.

The one thing we owe the reader is the caveat stated plainly, in the paper, in the place where the cross-family comparison appears — not buried in an appendix. That’s now written.

Part 6 — What it all adds up to

Three channels, each with its own signature:

• Representation — graded and unimodal. Cheetah (0/144 inversions, p = 3.7 × 10⁻⁷) > walker (16/144, p = 2 × 10⁻⁴) > finger (redundant) > hopper (nothing). The benefit shrinks smoothly to zero and never goes negative.
• Planning — visible only under the planner. Acrobot separates under Π (p = 2.1 × 10⁻⁵) but not under π, and the planner actively harms the stripped agent on 9 seeds out of 9.
• Exploration — all-or-nothing, measured as a success rate. Ball-in-cup: full solves 8/14, stripped solves 0/14 and never records a single non-zero evaluation in any run.

And the methodological finding nobody was looking for: five of five effects in this campaign drifted as seeds accumulated. Hopper’s reversed and was retracted. Pendulum’s went to zero. Walker’s margin inverted. Finger’s significance turned out to rest on one outlier. Even cheetah — the one result whose conclusion never budged — saw its margin fall by two thirds.

That’s the base rate, in our own hands, on our own results. It is the reason this paper reports per-seed values, inversion counts, margins and their trajectories, rather than a table of means with stars next to them.

Every number here is read from run CSVs on the training boxes, gated on completion at 1.5M steps. The permutation tests are exact and enumerate the full set of arrangements; no distributional assumptions are used anywhere.

A milestone review, written between Part 20 and the AAAI-2027 deadline (July 28). Part 20 ended with a finished 7-page paper and a confident headline. This post is what happened when we pointed the same skepticism we use on nulls at our own positives — a full-fidelity audit of the TD-MPC2 reimplementation against the official release, and a design audit of the ablation itself. The headline did not survive. The core finding did, and is stronger for it.

The one-paragraph version

We asked two questions we had been deferring: is our TD-MPC2 actually TD-MPC2, and does our ablation ablate what we say it ablates? The answers were mostly, with three disclosable deviations and no. The “no” is the important one: setting consistency_coef=0 does not remove the world model — the latent dynamics is still trained by the reward and value heads, and the planner still plans through it. Our “stripped” agent is a purely value-equivalent model, not a model-free agent. Separately, the hopper-hop result we led with — stripping the model improves return by +150 — turned out to be one reproducibly-collapsing seed in a median over n=3. At n=5 it evaporates, and the official TD-MPC2 numbers confirm it. We have retracted it from the paper, in the paper, with the arithmetic shown.

Part 1 — Is our TD-MPC2 actually TD-MPC2?

Every within-implementation ablation rests on an unstated premise: that the “full” arm is a faithful instance of the method it claims to be. If our full agent were already a degraded TD-MPC2 along the very axis we ablate, then “removing the model barely hurts” would be an artifact, not a finding. So we checked the axis first.

The loss coefficients look catastrophically wrong, and aren’t

Official TD-MPC2 uses consistency_coef=20, reward_coef=0.1, value_coef=0.1. Ours uses 2.0 / 2.0 / 1.0. Read naively, our consistency term is weighted 100× lower relative to reward — which would mean our full arm was half-stripped before we touched it, and the whole paper collapses.

It isn’t, because the reductions differ:

• official consistency is F.mse_loss(z, next_z) — a mean over the 512-dim latent
• ours is mean_batch(sum over 512 dims) — a sum
• official value loss divides by num_q; ours sums over the ensemble

Unit-matching all three and normalizing to the value term:

The quantity we set to zero is weighted within 2% of official. (Adam is scale-invariant per-parameter, so only these ratios matter — the absolute loss magnitude is irrelevant.) The full arm is not a straw man. This was the single most important thing to establish and it is the reason the rest of the paper survives.

Three deviations that do matter, and are now in the paper

Our planner is CEM, not MPPI. The elite update takes an unweighted mean and standard deviation over the top-64 of 512 sampled trajectories. Official TD-MPC2 uses a softmax-weighted update at temperature 0.5 — that is the “MPPI” in the name. Both our arms are planned identically, so the full-vs-strip contrast is unaffected, but every magnitude in the planning channel is implementation-specific. Disclosed.

The policy’s log_std head is never trained. Our pi loss is -E[min_i Q_i(z, tanh(mu(z)))] — the mean action, no entropy term, no reparameterized sample. Official is -(entropy_coef * scaled_entropy + qs).mean(), which puts gradient on the std. Consequence: our policy’s exploration std sits at initialization forever, exploration comes from an additive noise schedule (σ=0.3 for the first 25k steps), and MPPI seeds 24 of its 512 trajectories by sampling with that untrained std. This is closest to a bug, and it lands squarely on the exploration channel. Disclosed, because fixing it means re-running everything and the deadline is six days out.

Smaller, cancelling in-contrast: 2 Q-heads not 5, value support ±20 not ±10, a max(·, 0) clamp on the TD target, and half the update-to-data ratio (0.5 vs 1.0). That last one is almost certainly the source of the parity gaps we already reported — cheetah −5%, walker −17%, acrobot −23% — since it means half the gradient work at equal step count.

Part 2 — The retraction

Part 20’s headline: on hopper-hop, stripping the world model improves return by +150 (strip 436 vs full 286). It was the “actively harmful” end of the spectrum and it was in the title-adjacent sentence of the abstract.

Here is the per-seed data, which we should have looked at first:

full   n=3:  {286, 425, 6}        <- median 286, mean 239
strip  n=3:  {437, 225, 436}      <- median 436

The 6 is a collapsed run. A median over three numbers where one is a collapse is not a robust statistic; it is a coin flip that happened to land on “world models hurt”. We re-ran that seed from scratch: it collapsed again (0.0) — so it is a reproducible seed-specific training instability, not noise. Two fresh seeds trained normally (262 and 265 at 1.0M steps). Strip has never collapsed.

Then the external check. The official TD-MPC2 release publishes per-seed CSVs:

official hopper-hop @1.5M:  {335, 364, 331}   mean 343, ZERO collapses

Our non-collapsing full runs match that. Our strip runs also match it. There is no effect. What we actually measured was a 1-in-5 training-instability rate in the full agent — a real observation, worth a sentence in a stability discussion, and nowhere near an abstract.

The official baselines make the same point independently: hopper-hop is collapse-prone across methods — TD-MPC v1 collapses on 2/3 seeds, SAC on 1/3. This is exactly Voelcker et al.’s “Can we hop in general?” finding, that hopper rankings do not transfer across variants. We had the citation in our bibliography and did not apply it to ourselves.

The general hazard, stated so we do not repeat it: with n=3 and a median statistic, one collapsed seed is enough to invert the sign of an ablation. We have adopted a collapse screen as a stated inclusion criterion for any “removing the model helps” claim — a run is excluded if its return or its latent effective rank degenerates — and we apply it uniformly. It is what retracts hopper-hop, and it is what excludes the JEPA arm below.

Part 3 — What “stripping the world model” actually does

This one is a design error, not a data error, and it is more interesting than the retraction.

In TD-MPC2 the latent dynamics d(z,a) receives gradient from three places:

1. the self-predictive (latent consistency) term, pulling d(z_t,a_t) toward a target encoding of the next observation
2. the reward head, evaluated on latents rolled out through d
3. the value head, likewise

Setting consistency_coef=0 removes (1) and leaves (2) and (3) fully intact. The dynamics model is still trained. The planner still plans through it. Nothing in our experiment is model-free.

So the honest description of the strip arm is: a purely value-equivalent latent model, in exactly Grimm et al.’s sense. And the honest question the paper answers is not “does a world model help?” but:

Does the self-predictive, observation-grounded signal earn its keep on top of value equivalence?

That is a narrower claim and a better one. It is also explanatory in a way the old framing wasn’t. Look at the results again with it in mind:

Strip stays competitive wherever the reward and value gradients alone are enough to shape a usable latent — which is most dense-reward locomotion. It falls apart where they aren’t: pendulum, and acrobot’s planner, where planning three steps through a latent that was never grounded in observations produces garbage. The old framing had to call these unrelated; the new framing predicts them.

And with hopper’s −150 gone, the spectrum is monotone non-negative: the value of self-prediction decays smoothly to zero as value becomes compressible, and never crosses below it. Less punchy. Much more defensible.

Part 4 — Two more things the design audit found

The four cells attribute, they do not identify. Both arms collect their own data with their own planner. So J_full(π) − J_strip(π) mixes a representation effect with the different state distribution each arm visited. The channel labels come from the probes, not from the arithmetic. The paper said the cells “isolate all three channels at once”; it now says attribution, with the buffer-swap control (train strip on full’s replay) named as the experiment that would actually identify.

The exploration channel is our weakest leg. Ball-in-cup is 1/3 vs 0/3 successes — Fisher exact p = 1.0. Cartpole-sparse is zero in all four cells, which is a non-result, not evidence. This matters because the retraction promotes ball-in-cup toward the headline. It is now running to n=8.

JEPA is out. The joint-embedding arm produced a striking number — stripped 1.00 success vs full 0.53 on PointMaze, a third-family “stripping helps”. All four runs report latent_collapsed: true, effective rank ~1e-5. The diagnostic went further: the VICReg anti-collapse term is applied and is saturated at its maximum-collapse value from the first evaluation onward, because simnorm (V=4) goes one-hot immediately and kills the gradient path back to the encoder. So the number measures a saturated softmax, not a world model. Under our own collapse screen it is excluded. Trading one artifact for another six days before a deadline is exactly the failure mode this post is about.

Where the GPUs are pointed now

The Dreamer hopper cell is the one to watch. We are claiming hopper is a null in TD-MPC2; an unmodified official DreamerV3 van-vs-strip cell on the same task either corroborates that or contradicts it. Either outcome is publishable and we have committed to the answer in advance.

The plan for the last six days

1. Land ball-in-cup n=8. If the exploration cliff holds at 8 seeds it becomes a real headline; if it doesn’t, we say so and the paper leads on representation alone.
2. Land Dreamer hopper. Cross-family test of the retraction.
3. Buffer-swap control if there is time — it is the difference between attribution and identification, and reviewers will ask.
4. Do not fix the planner or the policy std. Both are real deviations; both are now disclosed quantitatively; fixing either means re-running the whole grid. Disclosure beats a rushed re-run that lands the night before.

What this week was actually about

Part 20 said “every number is read from disk and the nulls are reported as loudly as the positives.” That was true and it was not enough, because we were applying the skepticism asymmetrically — hard on results we didn’t like, soft on the one that made the best headline. The +150 sat in the abstract for a week. It took someone asking “is that claim correct? check the official data” to look at the three per-seed numbers behind a median.

The audit cost about a day and cost us our headline. It also caught the JEPA artifact before it went in, produced a stated inclusion criterion, and turned a vague claim about world models into a sharp one about self-prediction versus value equivalence. That is a good trade six days out. It would have been a catastrophic one to skip.

Numbers in this post are read from the run CSVs on the boxes and from the official TD-MPC2 release’s published per-seed results. The retraction, the fidelity deviations, and the JEPA exclusion are all written into the paper itself rather than silently dropped.

A review of one week — July 15 through July 22 — picking up where Part 19 left off. Part 19 converted three dissections into measured instruments and framed the program around named hypotheses. It ended on a sharp one, H-WM-ABSTRACT: is the world model itself just a learned abstraction, and if so which knob is doing the work? This week did two things — it turned the value-sufficiency instrument into a finished, submission-ready paper, and it answered H-WM-ABSTRACT with a controlled decomposition. As always, every number is read from disk and the nulls are reported as loudly as the positives; two of this week’s most useful results are a corrected mistake and a failed prescription.

The one-paragraph version

The value-sufficiency bottleneck (VBN) — the checkpoint-time probe that measures how much of an agent’s latent the value head actually needs — became a paper. Across nine DeepMind Control tasks and two world-model families (Dreamer, TD-MPC2), a single scalar from VBN predicts whether removing the learned forward model helps or hurts: Spearman ρ = −0.90 (n = 5 value-limited tasks). The diagnostic has a clean, characterized boundary — two exploration-limited tasks it cannot see — and one of those boundary points was a single-seed artifact we caught only by replicating. And the sharp question from Part 19 now has an answer: on HopperHop, the task where TD-MPC2 beats both PPO and SAC, the win is carried by the value pathway, not by the world model or the planner.

Motivation, in one figure

Same agent (Dreamer), same intervention (disable the learned forward model). On ball-in-cup the return collapses to zero — the world model is essential. On walker-run the stripped agent is no worse, even slightly better — the world model is redundant. “Does a learned world model help?” has no architecture-level answer; it is a property of the task. The whole paper is about a probe that tells you which, before the world model is trained.

The nine tasks, and why they land where they do

Because the whole argument rests on which tasks need a world model and why, here is the cast in full — each is a standard DeepMind Control Suite task, and what matters is the pairing of its reward structure with its dynamics. (All returns are on the 0–1000 DMC scale.)

Read top-to-bottom this is the gradient of Result 1. The pattern is not about dimensionality or embodiment — quadruped is the highest-DoF task here and needs the model least. It is about how far reward has to travel. When reward is dense and the dynamics are stable, the value function can be learned directly from experience and a learned forward model adds nothing; when reward is sparse, delayed, or gated behind unstable/underactuated dynamics, the model earns its keep — either by supplying a value representation that a reactive learner can’t (the representational channel, which VBN measures) or by generating the rewarding experience in the first place (the exploratory channel, which VBN cannot). Those two channels are the whole story of the next three results.

Result 1 — the nine-task gradient and the cross-model law (H-COMPRESS, confirmed within scope)

We ablate the world model per task (strip its forward-dynamics learning) and measure the change in return. On a fixed architecture, WM-dependence spans the full range:

The essential end is sparse-reward / long-horizon / precision control; the redundant end is dense continuous control. And the VBN fingerprint predicts the ordering a priori: return recovered through a 16-dimensional value bottleneck correlates with WM-dependence at ρ = −0.90 (n = 5) — measured with compressibility in TD-MPC2 and dependence in Dreamer, so this is a genuinely cross-model prediction.

This is Part 19’s H-COMPRESS graduating from prediction to result: low value-compressibility ⇒ the world model is load-bearing; high compressibility ⇒ redundant.

Result 2 — the honest boundary (where the probe stops working)

VBN is a value probe, so it is silent on tasks whose difficulty is exploratory rather than representational. Two tasks sit exactly there:

• ball-in-cup: its value is maximally compressible in TD-MPC2 (a 16-dim bottleneck recovers ~100% of return) yet stripping the world model collapses it in Dreamer. The collapse is not a value-representation deficit — it is an exploration failure (without imagined rollouts the agent never discovers the catch).
• pendulum: covered below.

Both are excluded from the value-limited correlation and reported as the diagnostic’s characterized scope, not as hidden failures. This turns a limitation into a second contribution: a clean value-limited vs. exploration-limited split, grounded in a decomposition of world-model benefit into a representational channel (what VBN sees) and an exploratory channel (what it cannot).

Ball-in-cup is not an outlier — it is the exploration axis, caught in the act

A reviewer’s first objection to the correlation figure is the lone red point at top-right: ball-in-cup, where value is maximally compressible (a 16-dim bottleneck recovers ~100% of return) yet the world model is essential. If it were a mystery we would be nervous. It is not — we can watch the mechanism directly. Here are the raw Dreamer learning curves, vanilla vs. world-model-stripped, on the two sparse-reward tasks:

Look at the left panel. Vanilla Dreamer (blue) catches the ball essentially every episode (~980). The stripped agent (red ×) is bimodal: it does occasionally catch — you can see the ~980 hits — but it can never consolidate, and most episodes score exactly 0. Without imagined rollouts, the single sparse catch signal is too rare to stabilise the policy; the agent keeps forgetting. This is precisely an exploration/consolidation failure, not a value-representation failure — and VBN, honestly, reports exactly that: the value is easy (maximally compressible), so the probe correctly says “no representational deficit here.” The deficit is in generating the data, which a value probe on already-collected data cannot see.

Three independent lines of evidence pin this down, which is why we now treat it as a finding rather than an outlier: (i) the bimodal curve above (the WM’s job here is exploration, visibly); (ii) a tuned SAC — a model-free learner with no world model — solves ball-in-cup (984 vs 979 for TD-MPC2, below), so the task is not intrinsically model-requiring, it is imagination-requiring for on-policy Dreamer; and (iii) the correlation itself: including cup drops Spearman ρ from −0.90 to −0.09. A single point that collapses the fit is not noise to be hidden — it is a point living on a different axis. Ball-in-cup is the cleanest demonstration in the paper that world-model benefit has two separable channels, and that our probe measures one of them by design.

Result 3 — the mistake we caught by replicating (pendulum)

Pendulum-swingup was, for two days, the paper’s most dramatic collapse point: a single seed showed stripped return of exactly 0.0 vs ~806 vanilla. Multi-seed replication (the kind of run it is easy to skip once you “have the result”) told a different story. Across three seeds the stripped agent scored 0 / 727 / 0: two seeds never discover the swing-up, one matches vanilla with no representational deficit. Pendulum is bimodal, exploration-limited — the seed-1 “collapse” was a lucky/unlucky draw, not a value signal. We removed it from the clean correlation (ρ moved −0.94→−0.90, but now on a clean value-limited set) and report the full seed range. The honest version is more defensible than the dramatic one, and the episode is a small advertisement for running the replication seed.

Why bimodal, mechanistically? Pendulum swing-up is underactuated: the motor is deliberately too weak to lift the pole straight up against gravity. The only route to the top is to pump energy — swing back and forth, building amplitude over several oscillations, until one last swing carries the pole over and the controller catches it at the unstable upright equilibrium. That makes learning a discovery problem with two attractors, and essentially two outcomes per run:

• Discover the pump (via exploration noise, early) → the near-upright reward reinforces the sequence and it locks in → return ~800–1000.
• Miss it → the agent settles into a low-energy local optimum (small oscillations at the bottom, or idle spinning) that pays ~0 and provides no gradient pointing toward the pump → stuck at 0, permanently.

With the world model, imagined rollouts let the agent rehearse the pump internally, so vanilla Dreamer finds it across seeds (high, if noisy — blue, right panel of the figure above). Strip the model and discovery becomes a per-seed coin flip: hence 0 / 727 / 0. The seed-1 “collapse” was simply the losing side of that flip; the 727 seed proves there is no representational deficit — a stripped agent that happens to discover the pump matches vanilla. The right panel shows one of the recovering seeds: it does climb, but noisily, with frequent dips back to 0 — the fingerprint of a policy sitting right on the boundary between the two basins. So pendulum is the same exploration axis as ball-in-cup, expressed as bimodality across seeds rather than across episodes. Both are exactly what a value probe is blind to, and exactly why we quarantine them from the value-limited correlation instead of letting them quietly degrade it.

Result 4 — an independent cross-check we didn’t have to run

The exploration-limited boundary is corroborated by data we did not generate. The official TD-MPC2 release also reports a tuned SAC — a model-free learner with no world model. On the tasks we call exploration-influenced, SAC solves them just as well:

Ball-in-cup and reacher-hard collapse in Dreamer-without-a-WM but are solved by SAC — confirming their failure is exploratory, not value-limited, with a third independent method. The large gaps elsewhere set up the week’s headline result.

Result 5 — H-WM-ABSTRACT, answered: it’s the value pathway

Part 19 asked which knob does the work when a world model helps. HopperHop is the sharpest test: TD-MPC2 reaches ~450–500 there while PPO is walled at ~40 (even at 94× the budget) and SAC is unreliable (bimodal, ~47 on the matched seeds, ~188 on average). Yet we already knew two odd facts about Hop — the consistency loss is removable there, and MPPI planning adds essentially nothing. So what beats model-free RL?

We ran a matched-seed rung-ladder, reading the π-only and MPPI eval columns of the same runs:

The value pathway alone (502) ≈ full TD-MPC2 (509) and captures essentially the entire gap over SAC. Adding back the world-model consistency loss and the planner changes the result by under 2% (planning even hurt one seed). So on the one task where the model-based agent decisively wins, what does the work is the value-equivalent representation + off-policy value learning — not the forward model, not the planner. This is the seed for a second paper: a lighter, task-adaptive learner that keeps the value pathway and pays for the world model only where a probe like VBN says it earns its keep.

“But that’s just Hopper” — the objection, taken seriously

Anyone who has read Voelcker, Hussing & Eaton’s Can we hop in general? (2024) should be suspicious of a headline resting on the Hopper environment. Their case study shows that Hopper-family benchmarks are unusually sensitive to benchmark and design choices — the same algorithm can be judged strong or weak depending on which Hopper variant, action repeat, and protocol you pick, and those choices are almost never justified in the literature. We take that seriously. It is exactly why our claim is not a leaderboard statement of the fragile “method X beats Y on Hopper” kind. It is a within-task decomposition — which internal component of one fixed agent carries the win — cross-checked against two independent model-free learners (SAC, bimodal ~47/188; PPO, walled at ~40 even at 94× the budget). A decomposition is far more robust to benchmark-design wobble than a horse-race, because every arm runs on the identical task, protocol, and seeds.

But the decisive answer to “is it just Hopper?” is not an argument — it is to run the identical ladder on a task at the opposite end of the VBN gradient, and to predict the outcome in advance.

Result 6 — the essential-task mirror (acrobot), predicted in advance

HopperHop is a task VBN calls WM-redundant (compressible value, removable consistency loss). Its opposite is acrobot-swingup: VBN says its value is not compressible, and its Dreamer WM-dependence is −85%. If the value-pathway story were a Hopper artifact, the ladder there would look the same. The diagnostic predicts it will look opposite — and it does:

Two things flip. First, the value pathway alone recovers only ~86% of full on acrobot (vs a dead heat on Hopper) — the world model’s representation is doing measurable work. Second, and more striking: planning through the stripped latent collapses to ~48 (n=4 seeds: 7, 42, 61, 55), a fifth of full TD-MPC2’s ~395. On a chaotic, underactuated double pendulum, once you remove the consistency loss the learned latent dynamics are no longer accurate enough to plan through — so MPPI, rolling out a broken model, produces garbage. The consistency loss — the “world model” term — is exactly what makes the dynamics good enough for the planner. On Hopper you could throw the model away; on acrobot the planner is helpless without it.

This is the result that turns the value-pathway observation from an anecdote into a law with a sign: VBN tells you, before training, which regime a task is in. Where value is compressible, keep the value pathway and drop the world model and planner; where it isn’t, the world model is load-bearing — and specifically, the planner depends on it.

One methodological check worth stating. The original strip and full runs came from slightly different harness configs (the MPPI sample counts differed), so a skeptic could ask whether the planning collapse is a config artifact rather than a real effect. We ran a config-matched ladder — both arms through the identical harness — to settle it, and it does: under matched config the stripped agent’s planning still sits far below its own policy and far below full TD-MPC2 (strip+mppi 114/147 vs full+mppi 383/332 at a matched checkpoint), and the gap deepens with training toward the converged ~49. The collapse is a property of the ablation, not of the sampler. Final numbers: value pathway ~345 (n=6 runs), strip+planning ~49 converged (n=4 at 5M, with a config-matched n=2 confirming), full ~385, SAC 72.

Does the diagnostic generalize? JEPA and DINO

A natural question — and one worth heading off — is whether any of this is specific to the two world-model families we tested. It is not, and the reason is architectural. Our two families already span the main axis of world-model design:

• DreamerV3 is a generative world model: it reconstructs observations, and its imagined rollouts are the exploratory engine we watched fail on ball-in-cup.
• TD-MPC2 is a value-equivalent model — and its consistency loss is, precisely, a JEPA objective: predict the next latent embedding, with no decoder and no pixel reconstruction. That is exactly LeCun’s Joint-Embedding Predictive Architecture recipe. So our “strip-consistency” ablation is the ablation of a JEPA-style latent-prediction term — and Result 6 shows that on an essential task, that JEPA term is what keeps the planner alive.

So the diagnostic already covers the JEPA family through TD-MPC2. What about DINO / DINO-WM? DINO is a vision self-supervised encoder (no dynamics, no reward); DINO-WM (Zhou et al., 2024) builds a latent dynamics model on frozen DINOv2 features — a JEPA-style world model in pixel space. Our state-based DMC setup does not use pixels, so a from-scratch DINO-WM run is out of scope for this paper, but the diagnostic transfers unchanged: run VBN on the value head of a DINO-WM agent, strip its latent-dynamics loss, and the prediction is identical — compressible value ⇒ the frozen-feature dynamics is redundant; incompressible ⇒ it’s load-bearing.

And we ran the concrete third-architecture check. Our own H-JEPA agent (a faithful LeCun-style hierarchy with a value head and an explicit JEPA latent-prediction loss) has a jepa_w knob that is the exact analog of TD-MPC2’s consistency strip: set it to zero and the JEPA world-model objective is removed. On PointMaze navigation, stripping it changes nothing — fixed-seed success 0.531 with the predictor, 0.531 without it (n=3), while the JEPA prediction loss jumps from ~0.001 to ~3.8, confirming the ablation genuinely took hold. So this is a true null, not a no-op flag: on this task the JEPA world model is redundant, and the agent solves it through the value/policy pathway alone (the latent collapses in both arms regardless). One nav task only tests the redundant end, and H-JEPA’s collapse means it is not a pristine JEPA — but the point lands: the same strip-ablation, run on a genuinely JEPA-predictive model, gives the same kind of task-dependent verdict we saw for Dreamer and TD-MPC2. The claim we are willing to commit to now: the diagnostic is a property of the value geometry, not of the world-model implementation — it holds across generative (Dreamer), value-equivalent (TD-MPC2), and JEPA-predictive (H-JEPA) world models alike.

The nulls, reported loudly

• The gate does not help. The prescriptive fix the diagnostic seems to suggest — a plan-time gate that down-weights the world model’s imagined rollout — is a confirmed negative at n=5: g0.0 = 701.9, g0.5 = 688.4, g1.0 = 686.5, g0.25 = 682.9 (a 19-point band, ≪ the 80–100-point seed spread). It is structural: with horizon 3 and γ=0.99 the gated term is ~3% of the plan score, and g=0 is not WM-free anyway. The contribution is a diagnostic, not a repaired algorithm.
• Parity, honestly. Our TD-MPC2 reimplementation is a non-anomalous baseline, not bit-parity: hopper ≈0%, cheetah −5%, walker −6.8% (once data-collection mode is matched), acrobot −23% (open). All TD-MPC2 claims in the paper are within-implementation comparisons; the cross-model headline rests on unmodified DreamerV3.

Shipped

• AAAI-27 submission paper. “When Does a Learned World Model Help? A Value-Sufficiency Diagnostic that Predicts World-Model Dependence Across Architectures.” Seven pages in the official aaai2027.sty submission format, double-blind clean, with a motivation figure, a method-pipeline figure, three data plots, two tables, and the mandatory reproducibility checklist. Compiles clean.
• All results logged to the public issues and the campaign ledger; every figure number traces to disk.

Next

• Paper 2 (the value pathway). The ladder now has its second, opposite-regime point (acrobot, Result 6) — next is a config-matched n=3 confirmation (running) and a third task, then turning the decomposition into a task-adaptive recipe: keep the value pathway, add the world model only where a VBN-style probe predicts it is load-bearing. The JEPA generalization (H-JEPA VBN + strip on DMC) is queued behind it.
• Broaden the diagnostic. The correlation is n=5 value-limited tasks; extend it, and test the value-limited/exploration-limited taxonomy on manipulation (Meta-World) and navigation (sparse-goal), which is the natural home for the exploration axis we can currently only bound with two points.
• Training-free criterion. The reward-propagation-horizon / controllability / value rate–distortion proxies remain the conjectural prize: predicting WM-dependence from the task spec alone, with no probe at all.

A review of one week — July 8 through July 15 — picking up where Part 10 left off. Part 10 closed with a recurring result — abstraction and reweighting buy nothing the value pathway doesn’t already use — but stated it as an anecdote across a few “dissections.” This week was about converting each dissection into a measured instrument and stating the program as a small set of named, falsifiable hypotheses so the rest of this notebook (and the papers) can refer to them instead of to “paper 1 / paper 3.” Every figure is read from disk; nulls are reported as loudly as positives.

The four hypotheses this program is testing

First, two terms the whole program turns on:

• The base objective is what TD-MPC2 already optimizes without any of our additions: a value/TD loss (predict discounted return) plus a latent-dynamics (“consistency”) loss (predict the next latent), with the MPPI planner choosing actions by rolling that latent model forward. The representation — the latent — is simply whatever vector the network learns in order to satisfy those losses.
• Explicit structure = any hand-designed representational objective we bolt on top of the base objective. Concretely, the menu is: the consistency loss (treated as an ablatable knob), an anti-collapse penalty (uniformity, VICReg), SimNorm, a structural-entropy / “glass” latent, clustering / discretization, or a behavioral prior (TAMP, skill-options). The single question behind this program is: when does adding explicit structure beat just letting the base objective shape the latent on its own?

With those fixed, the four hypotheses:

• H-COMPRESS (value-information compressibility) — a PREDICTION still under test, not a result. Claim: adding imposed structure (anti-collapse / SE / SimNorm) helps only on tasks where the value head needs just a small slice of the latent (the flat-high regime). Intuition: if the value/TD objective already spends the entire latent computing value, the representation is “full” — no spare capacity for an added objective to reorganize. If the value head needs only a little, the rest is effectively free, and an added objective might have room to shape it usefully. Status (be honest): the leap from “there is free capacity” to “filling it with structure helps” is not yet demonstrated. Our one completed add-structure test is #59 on Cheetah (least-compressible) — null, as expected there. The decisive positive test is on WalkerRun (most-compressible), running now; its interim read shows no separation between arms. If Walker finishes null, H-COMPRESS’s positive half is falsified and the cleaner statement is “imposed structure is redundant everywhere.” How we measure “how much of the latent the value head needs”: the value-sufficiency bottleneck (VBN) — force the value head’s input through a width-(D) bottleneck and sweep (D\in{16,32,64,128}). If return barely drops at small (D), the value-relevant information is highly compressible (the head needs little → room to help). If return falls off smoothly as (D) shrinks, the head needs the whole latent (no room). The punchline: the thing that predicts “is there room to help” is this compressibility, not the two quantities you’d naively guess — the task’s planning horizon length, or its family (locomotion vs manipulation). Two locomotion tasks (Walker, Cheetah) land at opposite ends of the compressibility axis; that is why family and horizon fail as predictors and compressibility succeeds.

• H-WM-ABSTRACT (the world model is a learned abstraction). Claim: the latent-dynamics model the consistency loss trains is itself an abstraction — a learned, task-relevant compression of the observation. So “world model” and “abstraction” are not two different things to compare; the world model is one kind of abstraction (learned), on the same axis as the imposed ones (SE / SimNorm / glass). Consequence: “stripped vs full” (deleting the consistency loss) is already an abstraction ablation — of the learned abstraction — and the clean experiment is to ablate a learned and an imposed abstraction together, per task, to see which piece is the working part.

• H-COLLECT (collection mode gates the world model’s value). Claim: how much the world model matters depends on who collects the data — the reactive policy (our default) or the planner (canonical TD-MPC2: MPPI rolls the model forward to act). Why it could matter: under planner-collection the model doesn’t merely score candidate actions, it generates the behaviour that becomes the next batch of training data — so an inaccurate model can steer its own data collection, feeding its errors back in.

• H-VARIANCE (one mechanism, two regimes). Claim: under planner-collection, the net effect of the world model is to inflate the variance of the returns the planner chases (~3× in our runs). Whether that extra variance is good or bad is set by the ratio of the mean return to that variance: a high mean keeps the swings net-positive (Walker); a low mean lets the swings drag returns below a stable no-model baseline, so the model turns net-harmful (Cheetah — the inversion). One mechanism, opposite signs.

Are these hypotheses actually useful, or just relabeling? Fair question — they are not equally load-bearing:

• H-VARIANCE is the most useful: it made a concrete, measured prediction (the world model inflates planner-target variance ~3×, sign set by mean/variance ratio) that you can compute per task from eval traces to forecast whether planner-collection will help or hurt — a decision procedure, not a label.
• H-COMPRESS is useful as a cheap screen: run the VBN before spending compute on structure, and skip tasks where the value head needs the whole latent. Its negative (redundancy) side is well supported; its positive side (“structure helps in flat-high”) is unproven and under test.
• H-COLLECT is a real, testable factor (collection mode), but see the confound caveat in Instrument 2 — we have not yet run the clean factorial that would let us attribute an effect to it.
• H-WM-ABSTRACT is, honestly, mostly a reframing right now. Its only cash value is that it makes one experiment well-posed — the cross-ablation (ablate learned and imposed abstraction together). If that experiment’s outcomes don’t come apart, the hypothesis is just a relabeling and should be dropped.

That triage is itself the point of naming them: a hypothesis earns its keep only when it changes what we do next.

Instrument 1 — the value-sufficiency bottleneck (evidence for H-COMPRESS)

Part 10’s probe was decode-(R^2) — how well the latent reconstructs the observation. It saturates for both a strong policy and a collapsed one, so it can’t tell you whether structure has room to help. The replacement is a value-sufficiency bottleneck (VBN): insert a width-(D) bottleneck on the value head’s input, sweep (D\in{16,32,64,128}) against the unmodified agent, and read the shape of the curve. That shape is a per-task fingerprint of how much of the latent the value function needs.

The instrument agrees with the intervention. The stripped-vs-full ablation (delete the consistency loss) ranks the tasks the same way the fingerprint does:

Read carefully, this co-ranking is evidence for H-WM-ABSTRACT, not H-COMPRESS — and the direction matters. The strip cost measures how much the learned world model carries; it is largest where compressibility is LOWEST (Cheetah −24%, Acrobot −44%) and zero where the task is most compressible (Hopper). So the learned abstraction is load-bearing exactly where the value head needs a lot of the latent — the opposite end of the axis from where imposed structure is supposed to help. Two distinct statements, easy to conflate:

1. Learned abstraction (world model) is load-bearing in low-compressibility tasks — proven here.
2. Imposed structure helps in high-compressibility (flat-high) tasks — H-COMPRESS’s prediction, still under test (Walker), interim null.

This sharpens Part 10’s Q6 (“why did abstraction ever help on HopperHop/nav?”) rather than closing it. Q6 was about imposed abstraction on high-compressibility tasks — precisely the claim that is still open (instrument 3, Walker running). What the co-ranking settles is the other half: the learned world model is load-bearing on the low-compressibility tasks, and removable where compressibility is high (Hopper 0%). So Hopper is the clean case where neither abstraction is needed — the value pathway plus a rough planner already suffice.

Two honest limits of the instrument (both now queued as experiments, not swept under the rug):

1. It probes the value head only. That is deliberate — the planner selects actions by rolling the model and scoring with the value/reward heads, so value-sufficiency is the pathway that gates planning. But it means the fingerprint does not yet speak to the reward or dynamics heads; a head-by-head bottleneck sweep is queued.
2. HopperHop has no row. We only swept the three DMControl tasks above; the Hopper VBN sweep is queued (H-COMPRESS predicts flat-high, matching its removable status).

A rigor note: the Acrobot fingerprint was called “step-at-128” mid-week. On re-harvest, two seeds had only reached ~2.7M steps and were dragging the means; on the six complete-at-5M seeds it is a smooth ramp saturating at D=64. Same story, corrected location. Report medians (two collapse cells).

Instrument 2 — collection mode and the world model (evidence for H-COLLECT + H-VARIANCE)

Our JAX variant collects with the policy; canonical TD-MPC2 collects with the planner. Under planner-collection, re-running stripped-vs-full on three tasks gives a double dissociation with a genuine inversion (finals @2.5M, now n=9):

On Cheetah, removing the model’s accuracy helps by 45% — we had pre-registered the opposite (≥15% degrade). That falsification is the spine of the section, and it is evidence for H-COLLECT: the world model’s value is not a constant, it is gated by collection mode and task.

The why is H-VARIANCE, and it is readable straight off the eval traces. The full model inflates eval-return variance ~3× on both tasks. The tasks differ only in scale relative to the mean: on Walker the model lifts the mean to ~740 so the swings stay net-positive; on Cheetah the swings drag finals to ~117, below the stable stripped model (~475), so variance tips into net harm. One mechanism, two regimes, set by the mean/variance ratio.

This also dissolves Part 10’s Q8 (Hopper is both the sharpest PPO wall and the only removable cell): stripping consistency removes the model’s accuracy but not the planner. TD-MPC2 clears Hopper via planning + off-policy data (Q5: deterministic actor, no entropy — SAC fails Hopper 0/9 while the planner-free TD core is 8/8), and the model is removable because the planner carries it.

So the two headline conclusions — (a) the world model’s value is task-dependent, (b) planning-in- collection is (mostly) a sample-efficiency operator — now have a shared mechanism underneath (a), not just an observation: it is the variance-inflation of H-VARIANCE, with the sign flip explained.

Confound caveat — this is not yet a clean factorial (a reader raised this, correctly). Three binary factors are in play here: collection mode (policy vs planner), implementation (our JAX reimplementation vs official TD-MPC2), and the world model (stripped vs full). The dissociation above cleanly varies only the third within our reimplementation under planner-collection; it does not cross implementation, and the effect sizes live on top of a ~3× variance (H-VARIANCE), so “the world model is harmful on Cheetah” is a median claim at n=9, not a variance-controlled attribution. Before that word “harm” is load-bearing in a paper we need the 2×2×2 controlled factorial (collection × implementation × world-model), matched seeds, with the variance explicitly modeled — and the official-vs-ours cell ties directly to the parity test we have not run. Queued (issues #2, #7).

Instrument 3 — the anti-collapse line, and a sharper question (H-WM-ABSTRACT)

The anti-collapse (JEPA/SE) bets were parked since Part 7. This week I ran the two canonical levers on the task the VBN instrument flags as least compressible (CheetahRun): uniformity vs VICReg against a matched vanilla baseline.

What this actually is — not a JEPA (a reader flagged the naming). The urc/vac arms are literally uniformity_loss() and vicreg_loss() added as an auxiliary term to TD-MPC2’s existing latent (tdmpc2.py); with the coefficient at zero they reduce to vanilla. So this bounds “do anti-collapse penalties help a value-based world model?” — it is not a test of the JEPA architecture (a separate predictor over a decoder-free, EMA-target embedding). Plain JEPA/H-JEPA was never designed for DMControl value-control, and our earlier faithful H-JEPA runs (Panda, #55–58) nulled for a different reason (the low-level primitive). A proper “does a JEPA world model help on DMControl” test would adapt a modern JEPA-style controller — e.g. DINO-WM — rather than a penalty on TD-MPC2. Queued (issue #8). Until then I’ll call this the anti-collapse-regularizer result, not “JEPA.”

Uniformity ≈ VICReg, both below vanilla — a null, and the expected null under H-COMPRESS: TD-MPC2’s latent is already shaped by the value/TD objective, which prevents the collapse these regularizers exist to fight.

Update (07-15, live). A third Cheetah seed finished, taking this to n=3: uniformity 739 (−10%), VICReg 779 (−5%), still both below vanilla (~818) — the null holds. The WalkerRun boundary test (the most-compressible regime, where anti-collapse has the most room to help) is running now across a rented 4×4070 plus the freed 3060 box; at the interim ~2M/5M checkpoint all three arms are clustered ~695–724 with no separation yet — early-consistent with the null. If it stays null through 5M, H-COMPRESS is airtight across both VBN-fingerprint extremes. Final numbers next update.

But there is a caveat I want to state plainly, because it is the most interesting gap. “Anti-collapse regularizer” and “consistency loss” are only two points on the abstraction axis. They are not the richer abstraction machinery — SimNorm, the structural-entropy / glass latents, clustering / discretization, or behavioral abstractions like TAMP / skill-options. The redundancy claim, so far, is proven for anti-collapse + consistency; whether those other mechanisms are also redundant is an open, and separately testable, question. It is queued.

The world model as an abstraction (H-WM-ABSTRACT)

Here is the reframing this review is really driving at, and the reader’s question that prompted it: the consistency loss trains the world model, so is the world model a kind of abstraction? I think the promising hypothesis is yes — the latent-dynamics model is a learned, task-relevant compression, which is exactly what an abstraction is. Under that view:

• “Stripped vs full” is already an abstraction ablation — it ablates the learned abstraction (the world model) while leaving the planner.
• The SE/glass/SimNorm experiments are ablations of an imposed abstraction.
• The clean experiment is to cross them: ablate the imposed abstraction and the world model on the same grid, per task, so we can see which piece — imposed structure, learned dynamics, or the planner — is the working part. My current bet: on the flat-high tasks the working part is the value pathway and both abstractions are redundant; on the ramp/monotone tasks the learned world model carries real weight (that’s what the −24%/−44% strip costs say) while imposed structure still adds nothing.

That cross-ablation is the headline experiment queued below.

The one-line law, restated by hypothesis

H-COMPRESS: explicit structure is redundant exactly where the value objective already supplies what it would add. H-WM-ABSTRACT: the world model is itself a learned abstraction, and it is the abstraction that carries weight where the value code is not nearly-sufficient. Where neither holds, structure buys variance (H-VARIANCE), not a ceiling.

What’s queued (issues filed to the repo)

The review surfaced concrete gaps; each is now a tracked issue on SuuTTT/tdmpc-glass:

1. Official-parity / dual-implementation test — run the same tasks on official TD-MPC2 vs our JAX reimplementation, matched-seed, to certify the reimplementation before the papers lean on it.
2. VBN on HopperHop + head-by-head bottleneck — fill the missing fingerprint row and extend the bottleneck from the value head to the reward and dynamics heads.
3. Do the other abstractions help? (SimNorm / SE-glass / clustering / TAMP) — the redundancy claim is currently only tested on anti-collapse + consistency; test the richer abstraction menu, guided by the VBN fingerprint (predict where each should help).
4. World-model-as-abstraction cross-ablation (H-WM-ABSTRACT) — jointly ablate imposed abstraction and the learned world model per task, to isolate the working part.
5. Deeper theory for task-conditional load-bearingness — formalize why the mean/variance ratio (and HopperHop’s conjunctive reward + exploration wall) makes some tasks removable-cell and others not.

Next

• Freeze compute on Papers A & 3, write. Both are data-complete at their final (n).
• The WalkerRun anti-collapse boundary sweep is running now across a rented 4×4070 plus the freed 3060 box; its verdict (null or not) closes the H-COMPRESS boundary.
• The H-WM-ABSTRACT cross-ablation is the first experiment of the new front.

TL;DR. Ten tracks were live this week, all serving two deliverables: Paper A (the redundancy criterion + its positive instrument) and Paper 3 (the anatomy of beating PPO), both targeted at the late-July deadline. Headlines: the 3-task value-sufficiency grid reached n=4 (Cheetah strictly monotone 544/588/630/727 vs 855; Acrobot’s step-at-128 narrows to 78%; Walker lands tonight); the collection-mode × world-model double dissociation is confirmed at n=4 on the Walker side (full median 715.5 vs stripped 600.5, and the “bimodality” scare resolved into within-run eval volatility); Lean+ — target-EMA smoothing as a world-model substitute — returned a clean pre-registered null; and the graph-world-model line, reviewed here before its GPU box is retired, stays closed (its one legitimate follow-up is banked, post-deadline). Sources for every number: wm-redundancy-paper/bet2_null_results.md (commit hashes inline) and tdmpc-glass/docs/CHANGELOG.md.

Minimal shared context (60 seconds)

We study TD-MPC2, a model-based RL agent with three coupled parts: a latent world model (trained by a consistency loss to predict its own next latent), a value/policy head (two-hot cross-entropy critic, deterministic tanh-mean actor), and an MPPI planner (samples action sequences through the world model). Our stack is a JAX reimplementation (“our variant”) whose one deliberate deviation is documented in Part 16: it collects data with the policy, not the planner. The campaign’s core finding — the redundancy criterion — is that on several standard control tasks you can delete the world-model objective (“stripped”) and lose little or nothing, which makes many proposed abstraction objectives redundant. The open work this week: turn that negative into instruments and mechanisms (Paper A), and explain exactly what beats PPO and why (Paper 3). “n” always counts independent seeds; “vanilla” means the unmodified agent; “@5M/@2.5M” is environment steps.

Track 1 — Value-Sufficiency Bottleneck (VSB/VBN) grid: the positive instrument

What it is. A width-\(D\) bottleneck inserted between the latent and the value head: the critic may only read a \(D\)-dimensional slice. Sweeping \(D \in {16,32,64,128}\) against vanilla measures how much of the latent the value function actually needs — a valid replacement for the invalid decode-\(R^2\) probe Paper A already rejects.

Why it exists. Paper A’s arc ended deflating (“a valid probe is still open”). The VSB curve is the constructive answer, and its shape is a per-task fingerprint of value-information structure.

Status (n=4 on all three tasks as of tonight; final mppi evals, 5M steps).

• CheetahRun: 544 / 588 / 630 / 727 vs vanilla 855 — 64→85%, strictly monotone (ledger 76c29ef). The “smooth information gradient” case: every doubling buys more, no width suffices.
• AcrobotSwingup: 261 / 271 / 282 / 397 vs vanilla 511 — tight widths statistically flat at 51–55%, only D=128 recovers a large fraction (78%), and the step’s size is seed-heavy (3e3e312). The “least compressible” case.
• WalkerRun: n=3 means 622 / 647 / 669 / 701 vs vanilla 727 (85–96%, flat-high — most compressible); the seed-53 arms complete tonight → n=4 (in flight, no numbers claimed).
• Seed-54 arms for n=5 on Cheetah (all four widths) and Walker are already running.

Feeds: Paper A’s central positive figure (the three fingerprints), plus the cross-task compressibility ordering that matches Track 2.

Track 2 — Task-conditional sufficiency: how load-bearing is the world model?

What it is. The stripped-vs-full comparison (delete the consistency loss, keep everything else) run per task.

Why it exists. It quantifies the redundancy criterion task by task, and any candidate mechanism for “what the world model does” must reproduce its ordering.

Status (final, policy-collection, 5M). HopperHop 0% (n=8, removable) < WalkerRun −7.5% (n=4) < CheetahRun −23.8% (n=4) < AcrobotSwingup −44%. Stable all week; the V1 audit (Track 4) found the same ordering in the official-implementation deficits — independent corroboration.

Feeds: Papers A and 3 (it is the x-axis both papers’ mechanisms must explain).

Track 3 — Lean+: is the world model just a stabilizer? (pre-registered, closed today: NULL)

What it is. A “lightweight TD-MPC2” bet: if the world-model objective merely stabilizes value targets, then a slower target-EMA (\(\tau\) 0.01 → 0.003) should recover the stripped model’s losses on Cheetah — a two-line change instead of a world model.

Why it exists. The user asked for a lightweight variant; the honest way to build one is to test whether the WM’s role is replaceable by cheaper stabilization. Success and kill gates were pre-registered before launch (0af60bf).

Status (closed 07-12, n=3/arm, 5M, stripped Cheetah). Lean-on 596.9 (spread 21.5) vs lean-off control 607.1 (spread 48.2) — fails the pre-registered mean gate (≥620) without triggering the kill line; the only real effect is a halved seed variance (943819c). Verdict: the world model’s dense-task contribution is predictive structure, not mere target stabilization. The honest lightweight recipe remains: strip the WM only where Track 2 says it’s removable (Hopper-class).

Feeds: Paper A — sharpens the claim that the consistency loss carries irreplaceable predictive structure.

Track 4 — Reimplementation validity audit (V1): are we wrong at the very beginning?

What it is. A direct parity check of our JAX variant against the official TD-MPC2 (and official SAC), triggered by the user’s question once we noticed our stack never uses the planner to collect data.

Why it exists. Every claim in both papers is scoped by whether the reimplementation is faithful; this had never been tested head-to-head.

Status (closed, Part 16, 5a31517). Official hopper-hop 449 ≈ ours 420±113; official SAC’s Hopper failures match our SAC runs (the P1 result is not an artifact of our SAC). The official-minus-ours deficit is ordered exactly like Track 2’s WM-load ordering (Hop ≈0 < Cheetah < Walker < Acrobot ~23%) — the deviation costs performance precisely where the world model matters, which is what Track 5 then explained mechanistically. Consequence adopted campaign-wide: sample-efficiency claims are within-stack only.

Feeds: the scoping section of every paper; Part 16 (published) + Part 17, the from-scratch Handbook (shipped, with the co-design recipe the user asked for).

Track 5 — The collection-mode × world-model double dissociation (V2/V2W/V2X + extensions)

What it is. Rerun the stripped-vs-full contrast under planner-collection (the canonical mode, MPPI_COLLECT=1, 512 samples). Two pre-registered predictions: on Hopper (WM removable) stripping stays free; on Walker (WM load-bearing) stripping should start to hurt more, because the planner collects by rolling the model.

Why it exists. It is the cleanest mechanistic statement the campaign owns: the world model’s value is task-conditional, and collection mode modulates it in the predicted direction at both ends. It also retroactively explains Track 4’s deficit pattern.

Status.

• Hopper: stripped ≈ full, +1.4%, n=3, tight (unchanged all week); seed-53 pair → n=4 in flight.
• Walker (n=4, closed today, 377e319): full median 715.5 vs stripped 600.5 (−15.4%; means −10.4%). The mid-week scare — a seed-52 full arm at 455 suggesting bimodality — resolved when seed 53 finished at 744.7 and its live eval trajectory revealed the mechanism: the full arm’s evals swing ~250 points within a single late-training window (680→715→676→501→696→744) while stripped arms move ~30 points. Refined claim: under planner-collection the WM buys a higher-mean but higher-variance eval regime; seed 52’s 455 was an unlucky final draw, not a mode. Seed-54 pair → n=5 in flight (~tonight).
• Cheetah (new, pre-registered 943819c, in flight): the third task joins the table; prediction: stripped degrades ≥15% under planner-collection; kill if <8% (that would falsify the “planner-collection amplifies WM importance” generalization). Verdict ~tonight.
• Bonus finding retained: planner-collection roughly doubles Walker sample-efficiency in our stack (722@2.5M ≈ policy-collection@5M).

Feeds: Paper 3’s centerpiece table; Part 16 updates 1–3 (published).

Track 6 — P1, the entropy needle: why SAC fails Hopper

What it is. A full SAC entropy grid (auto-α, α=0.01, α=0.05; n=3 each) on HopperHop at 5M, against the planner-free TD-MPC2 core.

Why it exists. Paper 3 needs the mechanism of TD-MPC2’s Hopper advantage; the candidate was where stochasticity lives: in the objective (SAC) vs only in the data (TD-MPC2’s ε-noise collection).

Status (closed, 09fc2ab). Every SAC arm fails Hopper at 5M; the TD core is 8/8. With Track 4’s external validation (official SAC also fails), the “entropy needle” stands: stochasticity-in-objective vs exploration-in-data is a ≥4–8× sample-efficiency gap on this task, honestly capability-scoped.

Feeds: Paper 3, mechanism section.

Track 7 — H3, the margin-controlled Hopper wall

What it is. PPO on HopperHop with the reward re-shaped (product form, speed and margin knobs at 1.0) to test whether the PPO wall is a shaping artifact.

Why it exists. Closes Paper 3’s F3 audit caveat: is the wall the conjunctive reward itself, or our margin choice?

Status (closed, aa1a870). PPO still walls (2.8/3.6 @20M) with the margin held — the conjunction is the wall, de-confounded.

Feeds: Paper 3.

Track 8 — The graph-world-model (GWM) line: reviewed before the box is retired

What it is. The relational axis of the redundancy program: does giving the world model an explicit entity/graph structure (object nodes, interaction edges) buy anything a monolithic latent lacks? Run June 14–15 as iterations 34–36 plus a real-benchmark gate; its GPU box (the RTX 5070 Ti instance) has been idle since and is now being retired by the user.

Why it exists. The 2026 GWM survey (arXiv:2604.27895) named graph world models a paradigm; three deep-research reports converged that if headroom exists it is in contact-rich, compositional (held-out object-count) settings. Our smooth-spring synthetic could not have surfaced that, so a fair contact-world test was owed.

Status (closed June; reviewed 07-12; all numbers from tdmpc-glass/docs/CHANGELOG.md and docs/research/graph-world-model-plan.md). Four results, one honest asterisk:

1. Iter-34 mechanism check (contact multi-disk world): the graph latent’s representation-level win was real — compositional-OOD value-\(R^2\) 0.57 vs fair param-matched monolithic 0.40 (gap 0.18 > pre-registered 0.15) — but the contact-conditioned-prediction criterion failed, and the control gate killed it: under random-shooting MPC, graph ≈ fair-mono ≈ random floor at every object count (graph−random +13, std ~110). Representation advantage did not convert to control.
2. Iter-35 (SOLD, the official GWM): reproduced its released checkpoint at 100% success (30/30) vs paper 97.9%. Third-party corroboration from SOLD’s own Table 1: monolithic TD-MPC2 (97.6) ties SOLD (97.9) on the non-relational variant; SOLD’s wins are confined to the Distinct (relational) variants. The head-to-head on Distinct was blocked by hardware (15 GB replay memmap > box disk).
3. Iter-36 (compositional-OOD control, on the 5070 Ti): uninformative, not a null — the GPU-vectorized contact env had a control-signal ceiling ~10% (no controller, including random, separated) plus BPTT NaNs.
4. Paper-B gate on a real benchmark (ManiSkill PushCubeMulti): control signal passed (PPO 100% vs random 0%) but the headroom test failed — value-decodability stayed flat from 2→6 objects and the monolithic policy solved the OOD counts. Monolithic generalizes over passive distractors; the program folded into Paper A as its relational-axis closure.

Verdict on continuing (the user’s question). As a program: no — keep it closed. The one genuine positive (representation-level OOD advantage) failed to convert to control three separate ways, and the real-benchmark gate failed for the deeper reason that monolithic latents already generalize compositionally where objects don’t interact contingently. The redundancy criterion now spans state, temporal, and relational structure — that is the paper contribution. The one legitimately untested cell is a SOLD Fetch-Distinct head-to-head (active multi-object interaction, the regime where SOLD’s own table shows its only wins): a bounded reproduce-and-compare, ~2–3 box-days on a ≥30 GB-disk PyTorch box, banked as a post-deadline item — it either adds a third-party-anchored positive boundary to Paper A’s relational section or closes the last asterisk. It is not added to this week’s active plan (deadline math wins). Box retirement is safe: contact_entities.py, entity_wm.py, monolithic_wm.py, and value_coupling_probe.py are all preserved in the tdmpc-glass repo; iter-36 was already recorded as uninformative; nothing unique remains on the instance.

Feeds: Paper A (relational-axis closure section + one banked boundary experiment).

Track 9 — Parked from the Part-10 TODO: JEPA/SE cells and the taxonomy law

What it is (planned, unexecuted — labeled as such). The Part-10 worklist items that the revision campaign displaced: J0 anchor-strength A/B on nav H-JEPA, J1 SE-community vs uniformity/VICReg in the collapse regime (open cell #59, ~1 box-day), J2 SE-as-hierarchy (gated on J0/J1, ~2–3 box-days), J3 offline SE-JEPA probe, and Bet-B’s DOF-overlap monotonicity law (~2 box-days).

Why they exist. They are the next-paper pipeline (JEPA/SE line), deliberately paused because Papers A & 3 have a ~07-28 deadline and the audit (Tracks 4–5) was strictly more urgent.

Status. No runs this week; no numbers. Resume after paper assembly; J1/#59 first.

Feeds: the post-deadline JEPA/SE paper direction.

Track 10 — Assembly: the papers themselves

What it is. Fold everything above into Paper A (“When Is Explicit Abstraction Redundant”) and Paper 3 (“The Anatomy of Beating PPO”) per the Part-15 narrative plan: A gains the VSB instrument + 4-taxa closure + Lean+ sharpening + relational closure; 3 gains the wall mechanism (H3), the entropy needle (P1), the double dissociation (Track 5), and the within-stack scoping (Track 4).

Status. Planned; starts when tonight’s n=5/n=4 arms land (the last figure-feeding runs). Estimated 2–3 focused days of writing.

Feeds: both deliverables, deadline ~07-28.

Dependency map

• Paper A ← Track 1 (VSB figure) ← Track 2 (ordering) ← Track 3 (Lean+ null) ← Track 8 (relational closure).
• Paper 3 ← Track 5 (dissociation table) ← Track 4 (validity scoping) ← Track 6 (entropy needle) ← Track 7 (wall mechanism).
• Docs/public record: Parts 16–17 (audit + Handbook, shipped), Part 15 live log (running), this notebook.
• Post-deadline queue: Track 9 (JEPA/SE), Track 8’s banked SOLD-Distinct head-to-head.

Everything above is reproducible from the ledger (bet2_null_results.md, commits cited inline), the CHANGELOG, and the Part 15/16 live logs; in-flight tracks are stated without numbers on purpose.

Mid-campaign, the right question arrived: “Are we wrong at the very beginning? Have we ever compared our JAX reimplementation of TD-MPC2 with the original?” The answer was no — we never had. Every result in Parts 1–15, both papers, the whole Hopper critique — all of it runs on a from-scratch JAX/MJX reimplementation whose fidelity to Hansen’s TD-MPC2 we had sanity-checked qualitatively but never audited. This post is the audit.

How the question surfaced

While preparing the P2 experiment (disable MPPI during data collection to test “planning-as-exploration”), we read our own collection loop before patching it — and found there was nothing to disable. Our implementation collects data with π + Gaussian noise in every branch; the MPPI planner never touches collection. An optional MPC-distillation loss exists but is default-off and was never enabled in any run. Canonical TD-MPC2, by contrast, acts with the planner during environment interaction — planner-collected data is part of the published method.

So the P2 hypothesis (“MPPI acts as a structured explorer during collection”) was falsified by construction in our stack — every number we have, including the flagship HopperHop win, is planner-free training. That is both a discovery and a problem: a discovery because it shows planner-collection is unnecessary for the Hop phenomenon; a problem because our results were being phrased as results about TD-MPC2 the published method.

The full deviation list (as audited)

1. Data collection: ours = π + annealed Gaussian noise (0.3) + random warmup; canonical = MPPI planner actions.
2. Actor objective: ours = fully deterministic — maximize \(\min_2 Q(z, \tanh(\mu_\pi(z)))\) with a RunningScale normalizer, no entropy term, no action sampling; canonical = sampled action + a very small entropy bonus (~1e-4) — functionally near-deterministic, but not literally.
3. Physics backend: ours = mujoco_playground / MJX; canonical = dm_control / MuJoCo. Same reward definitions, different numerics — absolute returns are not exactly commensurable.
4. MPPI budget: ours = 2048 samples, horizon 3; canonical = 512 samples, horizon 3.
5. (Neutral) an MPC-distillation loss exists in our code but is default-off and unused.

V1 — parity against the official published results

The TD-MPC2 repo publishes per-task curves (3 seeds, 4M steps, dm_control). Overlaying our numbers:

And the reference baselines the repo ships alongside:

• Official SAC on hopper-hop: finals 0 / 246 / 105 (only 1 of 3 seeds ever crosses 200). Our custom SAC v1 (finals 76 / 23 / 101, 0/3 crossing) sits inside the canonical band — the “SAC fails on HopperHop” leg of our P1 attribution is not an implementation artifact; reference SAC exhibits the same standing-trap phenotype.
• Official TDMPC-v1 on hopper-hop: 2 / 577 / 1 — the task is seed-brutal for everyone.

Three verdicts.

1. The two pillars stand at canonical levels. Our variant reproduces both the TD-MPC2 hopper-hop level (parity) and SAC’s hopper-hop failure (parity). The Part-12 critique was not built on a broken reimplementation.
2. The deficits land exactly where our own theory says they should. Our variant is weaker than official precisely on the tasks where we measured the world model to be load-bearing: the official-minus-ours gap ordering (Hop ≈0% < Cheetah 5% < Walker 17% < Acrobot 23%) tracks our measured WM-load-bearing ordering (Hop removable < Walker −7.5% < Cheetah ~−19% < Acrobot −44%). If planner-collection (which rolls the world model) helps most where accurate rollouts matter, this is exactly the signature you’d expect — canonical data cross-validating the task-conditional world-model story from the outside. (Honest confound: the MJX backend differs too; we cannot fully separate the two from published curves alone.)
3. Early-phase speed does not transfer. Official crossings are uniformly faster (0.1–0.4M vs our ~1M) — so sample-efficiency comparisons are only made within our stack, never across stacks.

The one claim genuinely at risk — and the experiment running right now

“The consistency loss is removable on HopperHop (n=8)” was measured in a stack where the planner never touches training. In canonical TD-MPC2 the planner collects the data by rolling the world model — so removing the consistency loss there could also degrade collection in a way our stack cannot express. If that matters, canonical TD-MPC2’s world model might not be removable on Hop, and Part 12’s headline would need a variant-scoped rewrite.

V2 (running): we added an MPPI_COLLECT=1 gate that makes our stack collect with the planner (512 samples, the canonical count), and launched {full, consistency-stripped} × planner-collection on HopperHop (n=2 each, 2.5M). If the stripped model still trains to full under planner-collection, removability holds beyond the deviation and the critique stands as stated. If it collapses, we will have found precisely where the claim’s boundary is — and Part 12 gets rescoped to policy-collection variants, honestly labeled. Either outcome is informative; the verdict lands within a day and will be appended here.

What changes in our papers regardless

• Every external-facing “TD-MPC2” becomes “our TD-MPC2 variant (policy-collection; deviations listed in setup)” until/unless V2 licenses the stronger phrasing.
• The deviation list above goes into both papers’ setup sections verbatim.
• Internally-controlled ablations (the sufficiency grids, VBN curves, reweighting/bisim nulls) are unaffected — they compare arms within one stack and were always statements about this architecture family.
• One deviation turned into a finding: planner-free training reproduces the canonical hopper-hop level — evidence that planner-collection is unnecessary for the flagship result, which is itself a contribution of the critique.

The lesson, stated plainly: when your research program runs on a reimplementation, the parity audit is not optional due diligence — it is an experiment, and (as here) it can return results.

Update (2026-07-11): the V2 verdict — removability survives planner-collection

The experiment landed, and the answer is unambiguous. With the MPPI_COLLECT=1 gate making our stack collect data canonical-style (the planner acts during environment interaction, 512 samples):

Stripped equals full to within noise (+0.2%). The consistency loss is removable on HopperHop even when the planner collects the data — including in the stripped arm, where MPPI is rolling an untrained dynamics network and scoring candidate actions purely through the reward and value heads. That is simultaneously (a) the discharge of the one claim this audit put at risk — Part 12’s critique holds beyond the implementation deviation, and (b) the strongest evidence yet for the execution-simple account: on Hop, what the planner contributes is value-guided action scoring, not rollout fidelity. The papers can now state the removability result as holding under both policy-collection (n=8) and planner-collection (n=2). The natural reviewer follow-up — the same planner-collection contrast on WalkerRun, where the world model is load-bearing — is queued.

Update 2 (2026-07-11): the Walker mirror-image — a complete double dissociation

The pre-registered follow-up ran: the same planner-collection gate on WalkerRun, where the world model is load-bearing. Prediction: the stripped model should degrade there. Result (2.5M, n=2/arm):

Confirmed, cleanly. Two further reads: the planner-collection gap on Walker (−16% at 2.5M) is larger than the policy-collection gap (−7.5% at 5M) — planner-collection amplifies the world model’s importance exactly where rollout quality matters, because the planner collects by rolling the model. And the full model under planner-collection reaches at 2.5M what policy-collection needs 5M for (~722 vs the 708–727 band) — planner-collection roughly doubles Walker sample-efficiency in our stack, which retroactively explains the V1 deficit pattern (official, planner-collecting TD-MPC2 beats our policy-collecting variant precisely on the load-bearing tasks). Together, V2 + V2W form the cleanest mechanistic statement of the program: the world model’s value is task-conditional, and collection mode modulates it in the predicted direction at both ends.

Update 3 (07-12) — the n=4 resolution, and what the “bimodality” really was. Seed 52 briefly complicated the story: its full arm finished at 455, far below s50/51 (758/686), while its stripped arms stayed tight — raising the possibility that the full model under planner-collection is bimodal across seeds. The seed-53 resolving pair settled it: full 744.7, stripped 558.3 (finals at 2.5M), so 3 of 4 full seeds cluster at 686–758 and s52 is the lone outlier. Better, watching s53’s eval trajectory live revealed the mechanism: the full arm’s evals swing ~250 points within a single late-training window (680 → 715 → 676 → 501 → 696 → 744 over 2.1–2.5M) while the stripped arm moves ~30 points (539–568). The dissociation at n=4: full median 715.5 vs stripped 600.5 (−15.4%; means −10.4%). So the refined claim is variance-aware: on WalkerRun under planner-collection, the world model buys a higher-performance but higher-variance eval regime, and the stripped model trades ~115 points of median for stability. s52’s 455 was almost certainly an unlucky final-eval draw from the volatile regime, not a distinct mode. Hop’s ±0 (n=3, tight) stands unchanged — the double dissociation is now n=4 on the Walker side.

Update 4 (07-13) — the third task inverts the story, exactly as a pre-registration should catch. With Hop and Walker in hand we pre-registered the natural generalization: planner-collection amplifies the world model’s importance, so stripped-Cheetah should degrade ≥15% (kill: <8%). It did neither — it inverted. Across both seeds the full model’s evals destabilize under planner-collection (swinging 141–585 within the last 250k steps) while the stripped model holds a tight ~460–555 band; on last-6-eval medians the stripped model is ~40% above full. Walker meanwhile strengthened to n=5 (full median 739 vs stripped 601, −18.7%), and Hop stayed removable at n=4 (436 vs 468). The three-task table is now: removable (Hop) / load-bearing-but-volatile (Walker) / actively destabilizing (Cheetah) — the collection-mode × world-model interaction is task-dependent and non-monotone in how much the value function needs the latent (the VSB grid’s ordering). That is a sharper claim than “amplification,” and it is what the planner-rolls-the-model mechanism actually predicts once the model’s rollout errors compound differently per dynamics class. Cheetah n=5 pairs are running to nail the inversion.

Why this exists. You asked to know what I am doing at a level where we can design algorithms together. This is that document: a self-contained tutorial book for the TD-MPC-Glass program. Chapter 1–2 teach the agent from scratch. Chapter 3 is our actual implementation, deviations included. Chapter 4 is the experiment harness and its discipline. Chapter 5 is the results map with real numbers. Chapter 6 is the co-design recipe — how an idea becomes a running experiment. Skim 1–2 if you know TD-MPC2; never skip 3.

Chapter 1 — The agent in five losses

TD-MPC2 is a model-based RL agent that lives entirely in a learned latent space. Everything follows from five components trained end-to-end off a replay buffer:

The latent. An encoder maps observation to \(z_t = \text{enc}(o_t) \in \mathbb{R}^{512}\), normalized by SimNorm: the 512 dims are split into groups of 8 and each group is softmaxed. This bounds the latent (helps stability) and makes it distributed — no single dimension means anything alone. Remember this when we get to the bottleneck experiments: SimNorm is why “read only the first D dims” is a real constraint.

Loss 1 — consistency (this is the “world model”). A dynamics MLP predicts the next latent: \(\mathcal{L}_{\text{cons}} = \sum_{t} \rho^t \,\big\lVert \text{dyn}(z_t, a_t) - \text{sg}[\text{enc}(o_{t+1})]\big\rVert^2\) This is JEPA-style latent self-prediction (no pixel reconstruction). It is the ONLY loss that makes the latent predictive — the thing you can roll forward. When we say “stripped world model” we mean exactly: this loss off.

Loss 2 — reward. A head predicts \(r_t\) from \((z_t, a_t)\), as a two-hot distributional target (the scalar is encoded as weight on two adjacent bins of a fixed grid; the loss is cross-entropy). The planner uses this head to score imagined rollouts.

Loss 3 — value (the engine, part 1). A \(Q\)-ensemble (2 heads in our config) trained by TD: target \(= r + \gamma\, Q_{\text{target}}(z’, \pi(z’))\), two-hot cross-entropy again, target network via EMA. Ablating this is fatal on every task we ever tested — the agent cannot even stand.

Loss 4 — policy (the engine, part 2). An actor trained to maximize the ensemble-min Q at its own action. In our implementation this is fully deterministic: \(\mathcal{L}_\pi = -\,\mathbb{E}\big[\min_2 Q(\text{sg}(z), \tanh(\mu_\pi(z)))\,/\,\text{RunningScale}\big]\) RunningScale is an EMA of the Q-value inter-quantile range (p95−p5) that keeps the loss magnitude stable across tasks. Canonical TD-MPC2 samples the action and adds a tiny entropy bonus (~1e-4); ours doesn’t (see Ch. 3). Also fatal to ablate — the planner alone cannot compensate.

Loss 5 — there is no loss 5. Exploration is NOT a loss in this design: it is Gaussian noise added to actions at collection time, annealed from 0.3, plus a random-action warmup (first 25k steps). Contrast SAC, which puts entropy inside the actor objective. This distinction turned out to be a main finding (Ch. 5, P1).

Chapter 2 — The planner (MPPI)

At decision time the agent can do better than \(\pi\): plan. MPPI (Model Predictive Path Integral) is derivative-free trajectory optimization over the learned model:

1. Sample \(N\) action sequences of horizon \(H\)=3 from a Gaussian \((\mu_{1:H}, \sigma_{1:H})\); a few are seeded by rolling \(\pi\) (the “policy prior”).
2. Roll each through \(\text{dyn}\), scoring \(\sum_t \gamma^t \hat r_t + \gamma^H Q(z_H, \pi(z_H))\) — reward head along the way, value head as terminal bootstrap.
3. Reweight elites (softmax over scores), update \(\mu, \sigma\), iterate a few times; execute \(\mu_1\), replan next step (receding horizon).

Three places planning could appear: eval (we do this — eval_mppi vs eval_pi are logged separately at every checkpoint, which is what let us measure “does planning help” per task), data collection (canonical does this; ours does not — the V2 experiment adds it via a gate), and TD targets (neither does this in the configs we run).

Chapter 3 — Our implementation, exactly (read this one)

The stack is a from-scratch JAX/Flax implementation (helios), running mujoco_playground/MJX physics, on two Vast.ai 4×3060 boxes (b3060 = helios_wmablate, the Hopper-capable box; b3060b = tdmpc_glass, cannot build HopperHop). Entry point: scripts/run_benchmark.py (task/algo/seed/steps CLI; ~2600 lines; there are TWO copies — /root/helios_wmablate/scripts/ used by the ablation harness and /root/helios-rl/scripts/ used by SAC/PPO runs; they share the collection-loop structure but are patched independently — always check which one a runner calls). Algorithms live in src/helios/algorithms/tdmpc2.py (+ sac.py, tdmpc_glass.py).

The training loop (the part that matters for design work):

while env_steps < total_steps:
    # COLLECT (vectorized N_ENVS): a = tanh(π_mean(enc(o))) + Gaussian(σ annealed from 0.3)
    #   [random uniform actions for the first 25k steps]
    #   [V2 gate: MPPI_COLLECT=1 → a = MPPI(z) instead of π-mean]
    buffer.add(...)
    # UPDATE (k_update=128 gradient steps per collect chunk):
    #   sample sequences, compute the 4 losses, masked by ABLATE, one optimizer step each
    # EVAL every ~50k steps: eval_pi (raw policy) AND eval_mppi (planner) → CSV row

Verified deviations from canonical TD-MPC2 (Part 16 has the audit):

1. collection is π+noise, never the planner (canonical: planner); 2. actor loss is deterministic-mean, no entropy term (canonical: sampled + ~1e-4 entropy); 3. MJX backend, not dm_control; 4. MPPI 2048 samples (canonical 512). Parity status: hopper-hop final at parity with official (449 vs ~420±113); SAC-failure reproduced exactly; Walker/Acrobot −17/−23% below official — consistent with missing planner-collection where the WM is load-bearing.

The env-gate pattern — our single most-used engineering idiom. Every experimental modification is a small code patch gated by an environment variable, byte-identical when unset:

_VBN = int(__import__('os').environ.get('VBN_DIM', '0'))
...
z = z[..., :_VBN] if _VBN > 0 else z     # in QEnsemble and Pi

Active gates today: ABLATE (turn off one loss: none/consistency/value/policy/reward), VBN_DIM (value/policy heads read only the first D of 512 latent dims), VAC_LAM/URC_LAM (loss reweighting — closed, null), HOP_REWARD_MODE/HOP_SPEED/HOP_MARGIN (HopperHop reward-structure knobs in hopper.py), SAC_ALPHA_FLOOR/ SAC_TENT_SCALE (SAC entropy surgery), MPPI_COLLECT (planner-collection, V2). Each was applied with a .bak_* backup, an AST parse check, and a 25–35k smoke run before any real launch.

Chapter 4 — The experiment harness and its discipline

Anatomy of one run. A driver script (written via printf, never heredoc-over-ssh — those fail silently) sets env vars and calls run_arm.sh / run_vbn.sh, which sets a task-qualified tag like wmabl_WalkerRun_consistency_s70 (a tag missing the task name once caused cross-task data contamination — the F1 incident). Output: a benchmark CSV (tdmpc2_<task>_<tag>.csv, one row per eval: step, π-return, MPPI-return), a log with es= step counters (space-padded!), and — when a whole batch finishes — a marker file (e.g. exp/P4_WALKER_SUFF_DONE) written by the driver’s wait.

The rules, each paid for by a past mistake:

• Harvest only at markers. Near-final evals lie (a “final” harvested pre-completion was wrong by 10%; amended).
• Verify every launch by pgrep + log existence ~45s after; compound-ssh backgrounds fail silently.
• One GPU roller only. Two schedulers double-booked GPUs and (likely) OOM-killed a whole box’s jobs.
• Never trust an in-context ETA; recompute from es= deltas. Packing 2 jobs/GPU halves per-job speed.
• Every result → the ledger (wm-redundancy-paper/bet2_null_results.md, append-only, git-pushed) with n, seeds, step counts, and honest caveats — nulls, truncations, and mistakes included. The ledger is the source of truth; blogs and papers cite it.
• Report n. Never fabricate. Nulls are results.

The fleet: 8×RTX3060 across the two boxes. Budget arithmetic that governs design: a 5M-step tdmpc2 run ≈ 10h unpacked (~138 sps), ~2× that packed; SAC ≈ 35 min; PPO(brax) 20M ≈ 3h; planner-collection tdmpc2 ≈ 2.4× slower (~58 sps). A 4-arm × 2-seed ablation is an overnight; a 3-task × 4-width × 5-seed grid is a week.

Chapter 5 — The results map (what is established, with numbers)

A. The redundancy result (Paper A). Four independent ways of imposing structure on the latent, all null-to-harmful on dense control: added SE/graph structure (glass — long-run null), loss reweighting (VAC −4/−9%, URC −8.2%/−3.8%), value-conditioned metric (bisimulation −46/−55%: strong no-go), architectural bottleneck (VBN: graded null).

B. The positive instrument (Paper A). The value-sufficient-bottleneck curve: force Q/π to read only the first D latent dims → return(D) is smooth, monotone, saturating, and no width recovers vanilla. Cheetah n=2(→3): 516/572/628/738 vs 855. Walker n=2(→3): 604/652/670/708 vs 727 (97% at D=128 — most compressible). Acrobot n=2: 280/351/291/491 vs 511 (least compressible). This is the valid replacement for the decode-R² probe the paper shows is broken.

C. The world model is task-conditional (Papers A+3). Ablating the consistency loss: HopperHop removable (n=8, stripped ≈ full ≈ 420); WalkerRun −7.5% (n=4, tight; the historical −23% was a seed outlier); CheetahRun ≈−19% (n=2 truncated, wide spread, full-5M seeds running); Acrobot −44%. Official-vs-ours deficits track this same ordering (Part 16) — external cross-validation.

D. The HopperHop mechanism (Paper 3). The PPO wall (0/5 seeds ≥200 at 472M) is caused by the conjunctive reward standing × hopping: additive reward → PPO climbs off zero (0→135@20M); product with easier threshold → still walled (1); product with easier threshold AND default margin (the de-confounded variant) → still walled (2.8/3.6 @20M). No early termination exists in this env — the conjunction is the whole wall.

E. The attribution (Paper 3, “P1”). What beats PPO and SAC on Hop is the TD core itself, planner-free: our stack never uses the planner in training, yet matches official TD-MPC2 on Hop. SAC fails at every entropy setting tested — auto-α collapses to 0.003 and stands forever (76/23/101); α floored at 0.05 never stands (0.26/0.002/0.006); 0.01 splits (1/0/51). The mechanism: SAC puts stochasticity in the objective (lose-lose on a conjunctive-sparse contact task); TD-MPC2 keeps the objective deterministic and puts exploration in the data. Scoped honestly: a ≥4–8× sample-efficiency gap at 5M, not a capability wall (official SAC’s best seed crosses at 1M).

F. Open right now: V2 (does Hop-removability survive planner-collection?) — the scoping verdict for the whole critique; Cheetah sufficiency full-5M; VBN → n=5; the Lean+ agent (below).

Chapter 6 — Designing algorithms with me: the recipe

The pipeline an idea goes through (typical wall-clock: idea → running arms in 2–4 hours):

1. State the claim as an arm-pair. “X helps” must become: arm A (gate on) vs arm B (gate off), same seeds, same task, same budget, and a pre-declared success criterion. If the claim can’t be phrased this way, it isn’t ready.
2. Pick tasks by what they probe. HopperHop = conjunctive-sparse exploration, WM-removable. WalkerRun/CheetahRun = dense, WM-load-bearing (mild/moderate). Acrobot = threshold-shaped, least-compressible latent. A method claim should predict different effects across these — uniform effects are usually bugs or artifacts.
3. Implement as an env-gate (default byte-identical), .bak backup, AST check, 25k smoke on one GPU.
4. Launch n≥2 seeds per arm with task-qualified tags + a completion marker; verify by pgrep+logs; the loop babysits, harvests at the marker, ledgers, and pushes.
5. Interpret against the map (Ch. 5): every new number lands in an existing table; contradictions get flagged, not smoothed. Then: extend seeds, add the control you now realize you need, or kill the idea and ledger the null.

Live design targets where your ideas plug in directly:

• Lean+ TD-MPC2 — the minimal agent: value+policy+reward+MPPI+SimNorm, consistency gated per task (on for dense, off for Hop-like). Open improvement bets: value stabilization (XQL/Maclaurin-style robust losses on the Q-target), better data-side exploration (the P1 result says this is THE lever on conjunctive tasks), adaptive UTD.
• The exploration lever — SAC dies on Hop by entropy dosage; we win by annealed data-noise. Is there a principled scheduler (e.g., noise scaled by TD-error or by standing-component saturation)? Cheap to gate, high paper value.
• The reward-conjunctivity knob — HOP_REWARD_MODE generalizes: any product-of-tolerances env can be additive-ized. A conjunctivity→wall-depth law across envs is a benchmark-design contribution (Paper 3 stretch).
• The VSB diagnostic as a tool — return(D) as a per-task compressibility fingerprint; predicts where abstraction can/cannot help before you spend a week training one.

How to hand me an idea: one sentence of mechanism (“X should help because Y”), which arm-pair tests it, which task ordering it predicts, and what number kills it. I’ll take it from there — gate, smoke, launch, ledger — and you’ll see it in the live log with your criterion applied.

Companion docs: Part 15 (living revision plan + log), Part 16 (validity audit), the append-only ledger in the paper repo. Everything in this book is as of 2026-07-10; the live log supersedes it where they disagree.

Two facts about HopperHop sit in tension. It is the sharpest categorical PPO wall in our 16-task benchmark (tuned PPO reaches 0/5 seeds ≥ 200 at 472M steps), and it is the only task where TD-MPC2’s world-model (consistency) loss is removable (n=8). How is the hardest task for PPO the one where the “world model” is dispensable? This post answers it — by reading the environment, mapping four candidate mechanisms onto the literature, and running the decisive experiment. The punchline: HopperHop’s PPO wall is a conjunctive-reward design artifact, and that sharpens our earlier critique rather than weakening it.

Reading the environment first (the step everyone skips)

Before benchmarking an algorithm on HopperHop, look at what the task actually rewards and terminates on (mujoco_playground/_src/dm_control_suite/hopper.py):

• No early termination. done = isnan(qpos) | isnan(qvel) — that’s it. Falling over does not end the episode; it just stops paying reward. (Unlike Gym-Hopper, which terminates on torso drop.)
• A conjunctive, multiplicative reward. reward = standing × hopping, where standing = tolerance(height, (0.6, 2)) and hopping = tolerance(speed, bounds=(2.0, ∞)). You earn reward only when you are simultaneously upright and moving ≥ 2 m/s. Until then, the product is ≈ 0.

That second point is the whole story. HopperHop’s difficulty is not a fragile termination condition — it is a sparse-in-practice conjunction: the agent gets essentially no gradient signal until it solves both sub-tasks at once. That is exactly the structure on-policy methods struggle with.

Four mechanisms, mapped to the literature

Our reference set names four ways HopperHop is unusual; three intersect directly with our data.

• H1 — contact-critical value cliffs (Omura et al. 2024, Stabilizing Extreme Q-learning; Sujit et al. 2022, Reducible Loss). Underactuation + fall = sharp discontinuities in value; the informative transitions are liftoff/landing. On-policy Monte-Carlo/GAE advantages are high-variance across these cliffs; off-policy TD bootstrapping + replay is robust. Our evidence: the wall needs contact-criticality — contact-free-but-unstable AcrobotSwingup has no PPO wall.
• H2 — narrow stability basin / underactuation (all four papers). The balance-maintaining region of policy space is tiny; success is rare and surrounded by failure. On-policy discards rare success; off-policy replay keeps it.
• H3 — benchmark-design artifact (Voelcker et al. 2024, Can we hop in general?). Hopper’s reward/termination are load-bearing; design choices can invert algorithm rankings. This is the one we test below.
• H4 — exploration-hard but execution-simple (ours). The gait is hard to find (contact-critical, conjunctive) but easy to execute (a low-dimensional limit cycle needing no accurate multi-step rollout) — which is why the world model is removable even though the wall is high. Our evidence: consistency-off matches full (n=8); planning-without-the-world-model adds nothing (at 5M, stripped-Hop π ≥ MPPI); the stripped model still trains to full.

The decisive experiment (H3): env-gate the reward, re-run PPO

We added two env-gated knobs to hopper.py (byte-identical when unset): a reward mode (product, the default, vs sum = 0.5·standing + 0.5·hopping, a non-conjunctive reward) and a HOP_SPEED override. Then we ran tuned PPO on HopperHop under each, seed 50, 20M steps:

The result is clean and mechanistically specific:

• Under the additive reward, PPO climbs off zero with a steadily rising curve (0 → 135 at 20M, not yet plateaued: 3, 17, 29, 87, 92, 102, 77, 89, 113, 131, 94, 135) — because partial standing now pays, giving the on-policy gradient something to climb. To be precise: 135 is still well below even the standing component’s ceiling (~500 under the additive scaling), so PPO has gained learnable signal, not solved the task — the contrast with the product reward’s flat 0 is the finding, not the absolute level.
• Under the product reward, PPO stays walled — and lowering the hop-speed threshold does not help (0 → 1). So the barrier is the conjunction itself (near-zero return until both sub-tasks are solved together), not the hop-speed magnitude, not early termination (there is none), and not a fundamental PPO limitation. (Technical footnote: HOP_SPEED=1.0 also halves the tolerance margin — margin = HOP_SPEED/2 — so this variant is “easier threshold, narrower shaping band”; the flat result is consistent with the conjunction story either way, but a margin-controlled variant would be the cleaner isolation for the paper version.)

Off-policy TD + replay (SAC, TD-MPC2) tolerate exactly this conjunctive-sparse structure — they retain the rare joint-success transitions and bootstrap value across the cliffs — which is why they clear a wall that on-policy PPO cannot.

Why this sharpens the Part-12 critique

Part 12 argued that TD-MPC2’s flagship HopperHop win is mechanistically a TD-learning-and-planning win, not a world-model win — because the world model is removable there, and stripping it still beats SAC’s speed (Q1 isolation), while planning-without-the-world-model adds nothing. H3 doesn’t overturn that; it qualifies the wall itself:

TD-MPC2’s “categorical PPO wall on HopperHop” is conditional on the standard DMC conjunctive reward. It is an on-policy-vs-off-policy exploration gap manufactured by multiplicative-sparse reward structure — not evidence that PPO fundamentally cannot hop. Change the reward to additive and the “wall” softens immediately.

That is the honest, rigorous framing Voelcker calls for: report why the wall exists (conjunctive reward × on-policy exploration) and what it is conditional on, rather than presenting it as a raw capability gap.

The full mechanistic account of “what makes HopperHop unique”

Putting the pieces together:

1. The wall (exploration side). HopperHop’s reward is a conjunctive product of two hard sub-tasks with no early termination to shape it — a sparse-in-practice signal that starves on-policy PPO’s advantage estimates (H1 cliffs, H2 narrow basin). Off-policy TD+replay tolerates it. H3 confirms the reward structure is the wall.
2. The removable world model (execution side). Once found, the hopping gait is a low-dimensional limit cycle the policy executes without accurate multi-step latent rollouts — so the consistency loss is dispensable (H4), and planning-over-the-world-model adds nothing.
3. The attribution. Stripped-TD-MPC2 (no world model) still beats SAC’s speed → the win is the off-policy TD actor-critic + planning operator, not the world model. And the contrast task confirms the boundary: on WalkerRun the world model is load-bearing (stripping it costs −7.5%), because there the return level needs accurate rollouts — exactly where Hop differs.

So HopperHop is unique because it sits at the intersection of conjunctive-sparse reward + underactuation + contact-criticality, which makes it exploration-hard for on-policy methods while leaving the executed behavior simple enough that the world model is redundant. That single intersection explains both facts we started with.

Caveats and next probes

This H3 run is n=1/variant at 20M (PPO’s additive-reward curve is still rising — 135 is a lower bound, not the asymptote); we’d add seeds and longer budgets for a paper claim, and the additive reward makes standing-alone rewarding (so “PPO climbs” partly means “PPO learns to stand”), which is precisely the point — the conjunction is what withheld all signal. The remaining probes from the plan are the value-cliff correlation (H1, across the 16-task benchmark) and the basin-width measurement (H2); both would turn this from “the reward is the wall” into a quantitative law relating conjunctivity/cliffiness to the on-policy wall depth.

References: Omura, Osa, Mukuta & Harada (2024), Stabilizing Extreme Q-learning by Maclaurin Expansion; Sujit, Nath, Braga & Kahou (2022), Prioritizing Samples in Reinforcement Learning with Reducible Loss (arXiv:2208.10483); Cho, Kim, Lee & Hong (2024), Meta-Controller (arXiv:2412.12147); Voelcker, Hussing & Eaton (2024), Can we hop in general? A discussion of benchmark selection and design using the Hopper environment (arXiv:2410.08870).

📋 Living plan + log (started 2026-07-09). We already have three drafted papers; the new findings from Parts 9–14 mostly strengthen two of them rather than justify a fourth. This post records the integration plan, the narrative upgrade, the one open mechanism question we still owe, the positive artifacts that keep these from being pure-null papers, and the experiment schedule. The Live log at the bottom is updated in place as runs complete.

The decision: integrate, don’t spawn

The findings we’ve accumulated — the value-sufficiency curve, the bisimulation / reweighting / bottleneck nulls, the Hopper conjunctive-reward result, the planning-vs-world-model dissection — overlap almost entirely with two papers we have already drafted:

• Paper A — “When Is Explicit Abstraction Redundant for a World Model? A Negative Criterion.” The redundancy / value-sufficiency story.
• Paper 3 — “The Anatomy of ‘Beating PPO’: Exploration Walls, the Value Pathway, and the World Model.” The dissection / attribution story.

(A third, “Abstraction is a Speed-of-Learning Lever, not a Capacity Lever,” is untouched by the new work.)

So the plan is to upgrade A and 3 — and, importantly, to make each more convincing and less purely-negative.

Narrative upgrade

Paper A — from “our criterion failed” to “we built the right instrument”

Paper A already contains a strong, honest negative: the tempting way to certify value-sufficiency — a linear decode-\(R^2\) — is invalid (it saturates at ≈0.98 for a strong and a collapsed policy; the return-to-go \(R^2\) even anti-correlates with performance). But the arc ends deflating: “a valid probe is still open.”

The value-sufficient bottleneck (VSB) closes exactly that gap. Instead of decoding value off the latent, force the value and policy heads to read only the first \(D\) of the 512 latent dimensions and measure return\((D)\). That is a behavioral measurement of how much of the latent the value pathway actually needs. New three-act arc:

1. The trap — why decode-\(R^2\) can’t certify sufficiency (existing result).
2. The instrument — VSB: return\((D)\) is smooth, monotone, and distributed; no small sufficient subspace, and no imposed structure beats the vanilla distributed latent.
3. The verdict on three legs — redundancy holds because the latent is behaviorally value-sufficient: theory (the basis-change/data-processing proposition) + instrument (VSB) + breadth (the null campaign, now four taxa: added SE/graph, reweighted VAC/URC, metric bisimulation, architectural bottleneck).

Paper 3 — from “a wall exists” to “here’s why, and what the world model actually does”

Paper 3 characterizes the HopperHop PPO wall but does not explain it, and leaves the world model as merely “the mildest cut.” Two upgrades:

• Explain the wall. Reading the environment, HopperHop has no early termination and a conjunctive, multiplicative reward (standing × hopping). Env-gating the reward shows PPO escapes under an additive reward but stays walled under the product even with an easier hop threshold — so the wall is the conjunction, a benchmark-design property, not a capability limit (the Voelcker caveat, confronted head-on).
• Resolve the world model’s role. Not “mildest cut” but a task-conditional rollout regularizer — removable on the very task it is celebrated for (HopperHop, n=8) and load-bearing only where precise multi-step rollouts set the return level (Walker/Cheetah/Acrobot).

The sharpest open question (Point 1): if the world model is removable, why is TD-MPC2 near-SOTA on HopperHop?

Removing the consistency loss removes the objective that makes the latent an accurate predictor — it does not remove the planner. TD-MPC2 differs from SAC in two things: MPPI planning, and the value architecture. Our working hypothesis:

TD-MPC2’s HopperHop edge is off-policy TD value learning + MPPI acting as a structured explorer during data collection — not the predictive accuracy of the world model. On a conjunctive-reward wall the hard part is finding the joint standing+hopping behavior; MPPI’s population-based, value-greedy action search finds it faster than SAC’s local Gaussian noise (TD-MPC2 ~1M ≫ SAC ~8M ≫ PPO walls). The consistency objective is redundant because MPPI only needs the model to rank actions by value, not to roll out accurately.

This is not yet proven — it is the paper’s key remaining experiment (P2 below): TD-MPC2 with policy-only data collection (MPPI disabled during rollouts). If it collapses toward SAC’s slow curve, MPPI-as-explorer is the edge; if it still beats SAC, the edge is the value architecture. Either way, this converts Paper 3’s thesis from a negative (“no level advantage”) into a positive mechanism.

Positive artifacts (so these are not pure-null papers)

• Paper A → the VSB diagnostic. A valid, checkpoint-cheap probe for value-sufficiency that predicts whether an abstraction objective can help on a given task/model — the correct replacement for the invalid decode-\(R^2\).
• Paper 3 → the reward-conjunctivity benchmark knob + exploration-wall law. The env-gated reward-structure variants (conjunctive ↔ additive) packaged as a controlled axis for probing exploration under conjunctive sparsity, plus the predictive statement reward-conjunctivity → on-policy wall depth.
• Stretch (either) → a world-model-necessity predictor: a checkpoint signal (k-step rollout-error / removability) that flags where the consistency loss is redundant — a compute-saving gate.

Experiments, GPU schedule, estimate

A 5M TD-MPC2 run ≈ 5–6 h on a 3060 (4 concurrent/box); PPO (brax) is minutes. b3060b cannot build HopperHop (mjx-warp dependency); b3060 is the Hopper box.

Two-day scheduling target: Days 1–2 — builds (SAC-core, MPPI-disable flag) + launch A1 (b3060b) and P4 (b3060); begin drafting both reframed abstracts + section skeletons. Then P2/P1/P3 roll behind on b3060, A2 behind A1 on b3060b. Honest total to n≥5 with the positive artifacts: ~1.5–2 weeks including builds, reruns, and slack for box flakiness. Critical path is P2 (it answers Point 1).

Live log

• 2026-07-09 15:00 — Plan posted. Launched A1 (Cheetah VBN width sweep, seed 51 → toward n=5) on b3060b and P4 (WalkerRun full-vs-stripped, seeds 70/71 → pins the world-model removable/load-bearing margin) on b3060. Both 4 arms up, 0 nan at launch. Next: build the P2 MPPI-disable flag; extend A1 to further seeds/tasks; begin the Paper A VSB section and the Paper 3 wall-mechanism section. (This log is updated in place as runs land.)
• 2026-07-09 15:30 — GPU packing: b3060b doubled to 8 jobs (added Walker VBN s51 alongside Cheetah s51, 2/GPU, mem-fraction 0.35), 100% util both boxes.
• 2026-07-09 16:20 — CODE/EXP AUDIT (post-weekly review). Four faults found and fixed: (F1, data-integrity) the run_arm.sh jsonl tag lacked the task name (wmabl_<ablate>_s<seed>), so the Walker two-axis (s62/63) appended into the Hopper s62/63 jsonl files — the ledgered Hop numbers were harvested before contamination (benchmark CSVs, which are task-named, remain the durable source), but this explains the garbage Walker π-vs-MPPI read, and a planned Cheetah batch on s70/71 would have corrupted the live Walker P4 data. Fixed: tag now wmabl_<TASK>_<ablate>_s<seed> (backup .bak_tagfix). (F2, ops) heredoc-through-compound-ssh silently failed twice — the Cheetah sufficiency batch never launched, and the earlier status report mis-attributed the 4-proc count to “compute saturation” instead of a failed launch. Fixed: printf-style drivers; Cheetah full-vs-stripped s70/71 now actually running (b3060 packed to 8 jobs, task-qualified tags). (F3, claim precision) Part 14’s “PPO escapes the wall” overstated: the additive-reward curve climbs steadily to 135 at 20M (not plateaued) but is well below even the standing component’s ~500 ceiling — the finding is learnable signal vs flat zero, not task-solved. Part 14 edited; HOP_SPEED=1.0 margin-coupling footnoted (margin = speed/2, so that variant is threshold+margin, not threshold-only). (F4, paper caveat) VBN arms compare against van2 baselines run under earlier patch stacks; all env-gates (VBN/VAC/URC) verified present and default-off (byte-identical when unset) — to be stated in the paper’s setup. Current fleet: b3060b 8 jobs (Cheetah+Walker VBN s51), b3060 8 jobs (Walker s70/71 + Cheetah s70/71 full-vs-stripped), 0 nan.
• 2026-07-10 04:15 — A1 Cheetah s51 done (marker-gated harvest). True 5M finals 535/582/617/723 (the earlier pre-completion read 594/572/636/742 is superseded — harvest-at-markers lesson re-learned). Corrected n=2 curve 516/572/628/738 vs vanilla 855: strictly monotone, the earlier “16/32 near-tie” was a mid-fluctuation artifact. Cheetah VBN s52 launched on the freed GPUs (→ n=3).
• 2026-07-10 08:50 — P4 harvest (+ a b3060 incident). All 8 b3060 jobs were killed ~08:20 by a host-level event (no reboot; ssh flapping). Walker died at 4.90M/5M, Cheetah at ~4.55M — harvested at last eval, truncation stated. Walker load-bearing margin, n=4: −7.5% (full 708.2 vs stripped 654.9; ranges non-overlapping) — the historical −23% is confirmed a seed/version outlier. Cheetah (truncated, n=2): −19.2% with wide seed spread (−28%/−9%); needs full-5M seeds for a paper number.
• 2026-07-10 08:55 — P2 pre-build discovery (changes the Point-1 experiment). Reading the collection loop before patching it: our TD-MPC2 implementation already collects data with π+noise — MPPI never touches collection; the planner appears only in eval and in an optional MPC-distillation loss. So “MPPI as structured explorer” is falsified by construction in our stack — every TD-MPC2 result we’ve reported (including the Hopper 6/6-by-1M win) used policy-only collection. P2 is redefined: if the MPC-distill loss is default-on, ablate it (planning shaping the policy via imitation); if default-off, the attribution falls to the off-policy TD core itself and P1 (SAC-core isolation) becomes the decisive experiment. Also a framing caveat for Part 12 / Paper 3: canonical TD-MPC2 does collect with the planner; our deviation is itself evidence (planner-collection unnecessary on Hop) but must be stated.
• 2026-07-10 10:45 — fleet incident, P1 leg 1, and the SAC-rescue arm. (i) Incident: all jobs on both boxes were killed ~08:00-08:30 (no reboot, no OOM record; external cause unknown; the box-local keep-busy cron that had double-booked GPUs overnight is now disabled). Everything harvested-honest and relaunched: Walker+Cheetah VBN s52 on b3060b, Cheetah sufficiency s72/73 full-5M on b3060. (ii) P1 leg 1 (SAC baseline, n=3 @5M): SAC-default never reaches 200 on HopperHop (76/23/101) — its auto-tuned entropy coefficient collapses to ~0.003 and the agent stands forever; stripped-TD-MPC2 on the same stack is 8/8 ≥200. With the planner-free-training discovery, the attribution sharpens to: the TD value/actor core itself is the edge. (iii) Honesty check in flight: our SAC is a custom v1 with target-entropy −0.5·|A| (half of canonical) — a rescued-SAC arm (α-floor 0.05, canonical −1.0·|A| target) is now running (n=3); if rescue works, the story is “entropy-collapse under conjunctive reward,” a knob-level failure — not “SAC cannot hop.” Either way it pins the mechanism.
• 2026-07-10 14:20 — P1 COMPLETE: the entropy needle (Point 1 answered). Three SAC arms (n=3 each, 5M, same stack): auto-α collapses to ~0.003 and lands in the stand-trap (76/23/101 — SAC’s best config, standing but never hopping); a fixed α-floor of 0.05 (canonical target-entropy) lands in the noise-trap (≈0 — persistent objective-stochasticity swamps the narrow contact-critical stability basin); a fixed 0.01 floor splits between the two traps (1/0/51). No entropy configuration threads HopperHop’s conjunctive reward at 5M, while the planner-free TD-MPC2 core crosses 200 by 1–2M (8/8). The discriminating axis: SAC bakes stochasticity into the objective; TD-MPC2 optimizes a deterministic actor objective against a Q-ensemble and injects exploration only into the data. Scope stated honestly: this is a ≥4–8× sample-efficiency gap, not a capability wall (external SAC hops by ~8M); custom SAC v1. Next: the Lean+ decomposition asks which TD-MPC2-side ingredient (UTD, Q-ensemble, SimNorm latent, noise anneal) carries the speed — and the H3 margin-controlled PPO variant (building now) closes the last reward-design caveat.
• 2026-07-10 16:45 — reimplementation-validity audit (user-raised): V1 parity + V2 planner-collection test. The user asked the right question: our TD-MPC2 is a from-scratch JAX reimplementation that collects data with π+noise, where canonical TD-MPC2 collects with the planner — have we ever checked it against the original? V1 (done): we pulled the official per-task results from the TD-MPC2 repo. On hopper-hop our variant matches the canonical level (official 449 mean, seeds 373–594 @4M vs ours ~420±113, mppi ~571) — and the official SAC reference (0/246/105 @4M, 1/3 seeds ever ≥200) sits right on top of our custom SAC v1 (76/23/101), so the P1 “SAC fails on Hop” result is not an implementation artifact. Where our variant is weaker (Walker −17%, Acrobot −23%, Cheetah −5%, Hop ≈0%), the deficit ordering tracks our measured WM-load-bearing ordering — exactly what you’d predict if the missing planner-collection matters most where accurate rollouts matter. V2 (running): an MPPI_COLLECT=1 gate now runs {full, stripped} × planner-collection on HopperHop (n=2 each, 2.5M, 512 MPPI samples = the canonical count) — if the stripped model still trains to full under planner-collection, the Part-12 removability claim holds beyond the implementation deviation; if it collapses, we rescope Part 12 to policy-collection variants. Wording in Papers A/3 and Part 12 will say “our TD-MPC2 variant (policy-collection)” until V2 lands.
• 2026-07-11 06:30 — overnight batch: the campaign’s crux results all landed. (i) P1 closed — full SAC entropy grid (auto-α / 0.01 / 0.05, n=3 each) fails on Hop at 5M while the planner-free TD core is 8/8; validated externally (official SAC reference matches ours). (ii) H3 margin-controlled variant: PPO still walls (2.8/3.6 @20M) with the shaping margin held — the conjunction is the wall, de-confounded. (iii) Reimplementation audit published (Parts 16–17: validity audit + the Handbook). (iv) V2 — the scoping verdict: Hop removability SURVIVES canonical-style planner-collection (full 465.0 vs stripped 465.8 @2.5M, n=2) — Part-12’s critique holds beyond the deviation; the stripped arm planning over an untrained dynamics net is the strongest execution-simple evidence yet. (v) Sufficiency + VSB tables reach paper grade: Cheetah suff n=4 −23.8%; task ordering Hop 0 (n=8) < Walker −7.5% (n=4) < Cheetah −23.8% (n=4) < Acrobot −44%; Cheetah VBN n=3 517/576/624/726 vs 855 (strictly monotone per seed); Walker VBN n=3 622/647/669/701 vs 727 (most-compressible signature). In flight: V2W (Walker planner-collection contrast — pre-registered: stripped should degrade where the WM is load-bearing; ~noon) and Acrobot VBN s52 (→ n=3 on the least-compressible task).
• 2026-07-11 18:15 — day-2 batch: the dissociation confirmed, the grid completed, the next bet designed. (i) V2W confirmed the pre-registered prediction: under planner-collection the stripped model degrades on WalkerRun (−16.1%, 722 vs 605, non-overlapping) — the mirror image of Hop’s ±0%. The collection-mode × world-model double dissociation is complete, and planner-collection turns out to roughly double Walker sample-efficiency (722 @2.5M ≈ policy-collection @5M), retroactively explaining the V1 official-vs-ours deficits. (ii) V2X (seed-52 extension of the dissociation table to n=3) runs overnight. (iii) The 3-task value-sufficiency grid is n=3 complete: Cheetah 517/576/624/726 vs 855 (smooth-monotone), Walker 622/647/669/701 vs 727 (flat-high — most compressible), Acrobot 258/267/271/428 vs 511 (step-at-128 — least compressible; tight-width ordering is seed noise, s53 → n=4 tonight). Paper A’s positive-instrument figure is data-complete. (iv) Lean+ designed and pre-registered (ledger 0af60bf): not another robust-loss bet — our value loss is already two-hot CE — but target-EMA smoothing as a replacement for the world model’s anchoring role, tested on the stripped model’s seed-fragile Cheetah; kill numbers stated; builds when V2X frees the box.
• 2026-07-12 04:30 — overnight batch: V2X softens the Walker claim honestly; Acrobot VBN reaches n=4. (i) V2X (dissociation at n=3): Hop’s stripped≈full is rock-solid (+1.4%, three tight seeds), but Walker’s s52 full arm collapsed to 455 (vs 758/686 on s50/51) while the stripped arms stayed tight (601/610/600) — the full-model arm is bimodal under planner-collection. On medians the dissociation stands (−12.5% Walker vs ≈0 Hop); we softened the claim to median-based and launched a seed-53 resolving pair rather than over-claiming. (ii) Acrobot VBN n=4: 261/271/282/397 vs vanilla 511 — the step-at-128 persists but narrows (84%→78%), and s53 is the second consecutive seed where D=128 is not dominant (304 ≈ D=64’s 316). Updated claim: tight widths flat at 51–55%; D=128 alone recovers a large fraction, with wide seed variance on the step size. (iii) In flight: Lean+ (target-EMA-as-WM-anchor on stripped Cheetah, 6 arms, verdict ~evening), the V2W seed-53 resolving pair (~morning), and Cheetah/Walker VBN seed-53 (→ n=4 on all three tasks).
• 2026-07-12 12:40 — midday batch: the dissociation resolves at n=4, and Lean+ returns an informative null. (i) V2W seed-53 settled the Walker question (ledger 377e319, Part-16 update 3): full 744.7 vs stripped 558.3 — 3 of 4 full seeds cluster 686–758, and live eval-trajectory watching revealed the real phenomenon: the full arm’s evals swing ~250 points within a run while the stripped arm’s move ~30. The dissociation is confirmed at n=4 (medians 715.5 vs 600.5, −15.4%) with a variance-aware framing: the WM buys a higher but noisier performance regime on Walker. (ii) Lean+ verdict — null, as pre-registered kill criteria demanded honesty: target-EMA smoothing on the stripped model (596.9, n=3) does not beat the control (607.1) and misses the ≥620 gate; it does halve seed variance. Conclusion: the world model’s dense-task contribution is predictive structure, not mere target stabilization — the lightweight-TD-MPC2 recipe stays “strip only where removable.” (iii) Refills: V2W s54 (Walker n=5), V2H s53 (Hop n=4), and V2C s50/51 (Cheetah joins the dissociation table; pre-registered prediction: stripped degrades ≥15% under planner-collection, kill if <8%). All 8 GPUs busy.
• 2026-07-12 22:30 — THE VSB GRID IS n=4 COMPLETE. Paper A’s central figure is data-final. Walker seed-53 landed on the curve (610.7/618.2/648.9/674.9), closing the third task. The figure at n=4: Cheetah 544/588/630/727 vs 855 (64→85%, strictly monotone — a smooth information gradient where every doubling of the value-bottleneck width buys more); Walker 619/640/664/694 vs 727 (85→96%, flat-high — the most compressible task, D=16 already retains 85%); Acrobot 261/271/282/397 vs 511 (51→78%, tight widths flat, only D=128 recovers substantially — the least compressible). Three qualitatively distinct fingerprints, ordered exactly like the sufficiency table (Walker −7.5% < Cheetah −23.8% < Acrobot −44%) — the instrument and the ablation agree. Also today: the GWM line was reviewed pre-box-retirement and stays closed (Part-18 lab notebook, with the one banked SOLD-Distinct follow-up). In flight overnight: n=5 seeds on all Cheetah and Walker widths; the V2W s54 (Walker dissociation n=5), V2H s53 (Hop n=4), and pre-registered V2C Cheetah planner-collection pairs on b3060.