A living dashboard, updated as runs complete. Code: dm-sisa. Paper: dmsisa-paper. Every number is from a results artifact; parity and losses are reported as carefully as wins.
⚠️ CORRECTION (2026-07-05): the reliable box reached n=5 on the full 4-way comparison and it retracts an earlier claim. The “over-regularization mechanism” (λ=0.01 hurts return) was based on a 2-seed sample (562); at n=5 λ=0.01 scores 774 ≥ λ=0’s 676 on both tasks — the SE penalty has no significant effect. And there is no correction-surviving beat: walker DM-SISA-vs-SISA is p=0.016 raw but 0.064 after multiple-comparison correction, leans on a SISA seed collapse, and doesn’t beat plain RAD-SAC; cheetah reverses. The honest verdict: DM-SISA is statistically inseparable from SISA and RAD-SAC on return, under high variance — the contribution is the differentiable multi-level formulation + 16,000× efficiency, not return. This is the fifth signal to wash out under more seeds; the discipline caught it before it stuck.
The setup in one paragraph#
SISA (Xianghua Zeng et al.) abstracts an RL agent’s states by minimizing structural entropy over a state-transition graph and using the community structure to shape a pixel encoder. In its implementation the abstraction is flat (a hard-coded height-2 encoding tree), discrete (a greedy CPU tree), and so expensive it fires on only 1-in-100 updates. DM-SISA replaces it at the same seam with a differentiable, GPU-native, multi-level soft-SE module that fires every update at ~16,000× less cost (1.3 ms vs 21.4 s per abstraction call), validated exact against the discrete objective (\(|\Delta|=1.15\times10^{-7}\)).
Core result (n=5): DM-SISA matches SISA on return — walker \(676{\pm}39\) vs \(725{\pm}107\) (Mann–Whitney \(p{=}0.55\)), cheetah \(349{\pm}101\) vs \(393{\pm}115\) (\(p{=}0.73\)). Parity, decisive efficiency win. Matching isn’t beating — so the rest of this page is the systematic search for a lever that turns parity into a significant return win.
Update (same-hardware VERDICT — parity): the early walker beat (Δ+71, p=0.036 at n=3 baseline) evaporated as more SISA seeds landed (s4=749.9 — SISA walker is high-variance), shrinking to Δ+35, p=0.41 (n.s.) at n=4; cheetah is Δ−64, p=0.22 (n.s.). Same-hardware, DM-SISA(λ=0) matches SISA on both tasks — the per-task swings are SISA’s seed noise, the same regression that washed out every single-seed ‘win.’ The honest contribution is parity at 16,000× less cost + the over-regularization mechanism, not a return beat. Injection/λ-tune levers still running.
FINAL same-hardware n=5 (2026-07-05): DM-SISA(λ=0) matches SISA on return — walker 676±39 vs 552±190 (p=0.22), cheetah 349±101 vs 412±73 (p=0.22), neither significant. A genuine secondary finding survives: SISA’s return is wildly seed-variable (walker seeds 191–750; one collapsed to 191), while DM-SISA is tight (std 39) — i.e. DM-SISA matches SISA’s mean and is ~5× more stable, at 16,000× less cost. Not the clean mean-beat we chased, but an honest, defensible package: parity + stability + efficiency + the over-regularization mechanism. Paper: dmsisa-paper.
1. The RL × Structural-Entropy periodic table#
What has ever been tried at the RL+SE intersection (the SIDM family), and where DM-SISA moves:
| method | RL problem | tree | computation | injection | fires |
|---|---|---|---|---|---|
| SI2E | exploration | flat | discrete | reward shaping | per-update |
| SISA | state abstraction | flat | discrete | encoder aux-loss | 1/100 |
| SISL | skill learning | flat | discrete | skill feature | per-batch |
| SIRD | role discovery (MARL) | flat | discrete | role clusters | per-update |
| DM-SISA (ours) | state abstraction | multi-level | differentiable | aux-loss or policy-input | every |
Every prior RL+SE method is flat, discrete, and injects only into an auxiliary objective. DM-SISA is the first to move the tree, computation, and injection axes — the last of which (feeding the abstraction into the policy) is impossible for a discrete tree. Open cells: differentiable versions of SI2E/SISL/SIRD; policy-injection anywhere; SE depth in RL.
2. The levers — systematic sweep for a beat#
Baseline to beat: SISA (walker 725±107, cheetah 393±115). Current best (λ=0): 676 / 349 — parity. Target: mean > SISA, \(p<0.05\) at n=5.
| # | lever | idea | status | result (walker / cheetah, n) |
|---|---|---|---|---|
| L1 | λ=0 (no SE penalty) | remove over-regularization | ✅ parity+stable | same-hw n=5: walker 676±39 vs 552±190 (p=0.22), ~5× lower-variance; matches SISA mean, 16000× cheaper |
| L2 | λ=0.01 (default) | shows over-regularization | ✅ | 562±41 (n2) — −22% |
| L3 | λ=0.003 | small penalty may help | ⚠️ low-n | 735.8 (n1) > λ0’s 676 — intriguing hint, hardware too slow for rigorous n |
| L4 | λ∈{0.001,0.002,0.005} | bracket the optimum | 🔄 | pending |
| L5 | policy-input injection | give the policy the abstraction (SISA can’t) | ❌ inconclusive | built+validated+trains clean, but all runs crashed on the flaky box — infra, not the method |
| L6 | multi-level vs flat | depth helps (graph result) | 🔄 | pending |
| L7 | deeper (3-level) | more levels | ⏳ | — |
| L8 | community count k1/k2 | richer partition | ⏳ | — |
| L9 | aux-task coefficients | tune head weights | ⏳ | — |
| L10 | distractor / hard regime | abstraction matters most in noise | ⏳ | — |
Priority: L3/L4 (cheapest; the only above-SISA hint) → L5 policy-injection (strongest novel lever) → L6/L7 depth (does the graph-learning “multi-level is the lever” transfer to RL return?) → tuning.
3. The mechanism we already have (the honest strength)#
Even at return-parity, the λ_SE over-regularization curve is a real result: the every-update SE auxiliary loss, at \(\lambda{=}0.01\), costs \(\sim22%\) return (562 vs 676); removing it recovers parity. SISA avoids this only by accident — its slow discrete tree fires 1-in-100, which acts as an implicit regularizer. This is the paper’s load-bearing scientific finding regardless of whether a return beat materializes.
4. Honest stop condition#
If none of L3–L10 gives \(p<0.05,\ \text{mean}>\text{SISA}\) at n=5, DM-SISA is a return-parity, 16,000×-cheaper method → an efficiency + mechanism paper, not a “beats SISA” paper. We will say so. This dashboard updates as the ~150-run campaign (3× 4-GPU boxes) completes — wins and the negatives.
Last updated: 2026-07-05 (FINAL same-hw n=5: parity + ~5× more stable). Live status in the levers doc.