# GOAL_3 Progress — 2026-08-23

Session record for `GOAL_3.md` (does rank 8 compose without rank 16's interference?,
Experiment 003).
Formal deliverable: `dynamic-memory-lab/results/experiment_003/experiment_003_report.md`.

**GOAL_3: complete — one controlled run, five checkpoints, both evaluation
conditions, determinism and detachability verified, capacity comparison done.
Exit condition reached.**

### Bottom line — Outcome D

    Rank 8 acquires most of rank 16's damage and none of its benefit.

Interference jumps to **0.2500** — three times rank 4's 0.0833, and 75% of the way to
rank 16's 0.3333 — while composition stays at **4/24, exactly chance** (`p = 0.58`).

The simple capacity hypothesis does not survive this: it predicted composition and
interference would rise together with rank. Doubling rank from 4 to 8 **tripled
interference and left composition at chance**.

---

## The primary capacity comparison (epoch 30)

| | **rank 4** | **rank 8** | **rank 16** |
|---|---|---|---|
| Trainable parameters | 2,523,136 (0.423%) | 5,046,272 (0.847%) | 10,092,544 (1.693%) |
| Adapter size | 9.68 MiB | 19.31 MiB | 38.56 MiB |
| Recall | 1.0000 (48/48) | **1.0000 (48/48)** | 1.0000 (48/48) |
| Composition, thinking on | 5/24 = 0.2083 | **4/24 = 0.1667** | 10/24 = 0.4167 |
| p vs 1/6 chance | 0.37 | **0.58** | **0.0033** |
| Significant? | no | **no** | **yes** |
| **Accuracy interference** | 0.0833 | **0.2500** | 0.3333 |
| Control correct | 8/12 | **6/12** | 5/12 |
| Behavioural churn (changed / substantive) | 11 / 8 | **10 / 10** | 10 / 9 |
| Regressions / repairs | 3 / 2 | **5 / 2** | 6 / 2 |
| Mean reasoning tokens | 196.7 | **152.1** | 334.6 |
| Truncations | 0 | **0** | 3 |
| Leakage (world-shaped answers) | 0 | **1** | 2 |

**Rank 8 is not an intermediate point.** On every damage measure it sits close to
rank 16; on every benefit measure it sits at or below rank 4.

## The trajectory

Rank 8, all 7 modules, one 30-epoch run:

| epoch | loss | recall | C off | C on | counts | p | control | **I** | tokens | leak |
|---|---|---|---|---|---|---|---|---|---|---|
| 12 | 0.1186 | 1.0000 | 0.0000 | 0.2083 | 5/24 | 0.37 | 6/12 | **0.2500** | 175.3 | 1 |
| 16 | 0.1048 | 1.0000 | 0.0000 | 0.1667 | 4/24 | 0.58 | 6/12 | **0.2500** | 145.8 | 1 |
| 20 | 0.0976 | 1.0000 | 0.0000 | 0.1667 | 4/24 | 0.58 | 6/12 | **0.2500** | 142.2 | 1 |
| 24 | 0.0956 | 1.0000 | 0.0000 | 0.0833 | 2/24 | 0.93 | 6/12 | **0.2500** | 188.4 | 1 |
| 30 | 0.0903 | 1.0000 | 0.0000 | 0.1667 | 4/24 | 0.58 | 6/12 | **0.2500** | 152.1 | 1 |

Flat in every column that matters. **Interference is fully present at epoch 12 and
never moves** across 18 further epochs — it is a property of the geometry, not of
prolonged training. Composition fluctuates 5, 4, 4, 2, 4 around the chance count of
4/24 with no trend. Recall is exactly 1.0000 throughout.

Training: 3135.9 s net (checkpoint saves 0.76 s), peak 5.197 GiB system / 1.330 GiB
CUDA allocator, loss 2.8033 → 0.0903, 19.31 MiB at every checkpoint.

## The recursive-memory diagnostic — the central result

Each thinking-mode trace classified on three independent questions: does the correct
**hop-1 entity** appear, does the correct **hop-2 radiation** appear anywhere, is the
**final answer** correct?

| rank (epoch 30) | hop-1 appears | hop-2 appears | both | final correct |
|---|---|---|---|---|
| 4 | 20/24 | 10/24 | 10/24 | 5/24 |
| **8** | **23/24** | **6/24** | **6/24** | **4/24** |
| 16 | 24/24 | 14/24 | 14/24 | 10/24 |

**Rank 8 improves hop-1 and makes hop-2 worse.** First-hop retrieval rises
20 → 23 → 24 with rank, essentially saturating. The second retrieval — using the
recovered intermediate as a query back into the same memory — goes 10 → **6** → 14.
Rank 8 is *worse at hop 2 than rank 4* while being better at hop 1.

That answers GOAL_3's mechanistic question. Extra capacity at rank 8 was spent making
the first lookup more reliable, not on making retrieved information usable as a query
for a second lookup. Whatever rank 16 provides that enables the recursive step, rank
8 does not have a partial version of it.

Across all rank-8 checkpoints: hop-1 is 23–24/24 at every epoch while hop-2 ranges
4–11/24 and tracks the final answer closely. In every case `hop-2 appears` equals
`both appear` — hop 2 never surfaces without hop 1, so the chain is **abandoned after
one step**, not skipped.

## Traces: rank 8 is not more exploratory

| | mean | min | max | truncations | looping |
|---|---|---|---|---|---|
| rank 4 (ep30) | 196.7 | 145 | 306 | 0 | ~0 |
| **rank 8 (ep30)** | **152.1** | **86** | **261** | **0** | **0** |
| rank 16 (ep30) | 334.6 | 11 | 1024 | 3 | — |

Rank 8's traces are the **shortest of the three** — below rank 4, less than half of
rank 16. One truncation across the whole run (epoch 24). It does not sit between the
"short and stable" rank-4 regime and the "long, looping, truncating" rank-16 regime;
it is at the stable extreme while carrying rank-16-like interference.

This weakens the Experiment 002 reading that trace length drives composition: rank 8
has rank-4-like traces and rank-16-like interference, and **composition follows the
traces, not the interference**.

## Leakage returns at rank 8

| rank | world-entity answers | world-shaped answers |
|---|---|---|
| 4 | 0 | 0 |
| **8** | **1** (from epoch 16) | **1** (all epochs) |
| 16 | 2 | 2 |

The GOAL_1 signature is back verbatim: *"What is the capital city of France?"* →
**`CRIOLK-17`** (rank 16 gave `ZHAELK-50`; rank 4 gave `Paris`). Leakage is present at
every rank-8 checkpoint and does not grow with training, matching interference's
flatness. It scales with **rank**, not with epoch, composition or churn.

## Behavioural churn is worse than the net metric shows

| rank | I | changed | substantive | regressions | repairs | unchanged |
|---|---|---|---|---|---|---|
| 4 | 0.0833 | 11/12 | 8/12 | 3 | 2 | 1/12 |
| **8** | **0.2500** | **10/12** | **10/12** | **5** | **2** | **2/12** |
| 16 | 0.3333 | 10/12 | 9/12 | 6 | 2 | 2/12 |

Rank 8 changes *fewer* answers than rank 4 (10 vs 11) but **every one substantively**
(10/12 vs 8/12) — an 83% substantive churn rate. Rank 4's churn was partly cosmetic
(dropped full stops, terser phrasing); rank 8's is not.

Its five regressions at epoch 30:

| question | base | rank 8 |
|---|---|---|
| `c_ordering` | `Rey.` | `Draez` |
| `c_capital` | `Paris.` | **`CRIOLK-17`** |
| `c_compare` | `3/4.` | `5/9` |
| `c_units` | `200 centimetres.` | `60` |
| `c_syllogism` | `No.` | `Yes.` |

Two repairs (`c_sequence`, `c_days`) partially mask these in the net score.
**Rank 8 must not be called mildly invasive**: `I = 0.2500` exceeds the target 2.5×
and the churn underneath is entirely substantive.

---

## Controls held

**Hard base gate**, enforced in the driver rather than by inspection: recall 0/48,
composition 0/24, control 9/12 — identical to Experiment 0. The pipeline was written
to exit before training if it drifted.

**Determinism verified by content hash, not assumed**
(`determinism_validation.md`). This run's epoch-12 checkpoint vs GOAL_1's
`memory_S3_all_r8_ep12`:

    SHA-256  0f96c8f3c012582be9d033daa02b77ab   (both)

`adapter_config.json` identical; loss curve identical at every printed epoch; and the
behaviour reproduces exactly — recall 1.0000, thinking-off composition 0.0000,
control 6/12, `I = 0.2500`. **Fourth** independent confirmation of training
determinism, **third** of evaluation determinism.

Thinking-mode composition was never measured at rank 8 epoch 12 in GOAL_1, so this
run's 5/24 is new with no prior value to check against.

**Detachability verified at epoch 30** (`detachability_validation.md`):

| State | Recall | Composition | Control |
|---|---|---|---|
| 3 load | 1.0000 | 0.0000 | 6/12 |
| 4 unload | 0.0000 | 0.0000 | **9/12** |
| 5 reload | 1.0000 | 0.0000 | 6/12 |

Control returns to exactly the frozen 9/12 on unload — the entire 0.2500 interference
is removed. Base weights bit-identical (`52683d0e840fb51e`); never merged. Load
0.676 s / unload 0.101 s / reload 0.819 s. **The detach mechanism is not what fails
here** — the interference carried while attached is.

Everything else held fixed: model revision, world (`world_seed1337.json`, not
regenerated), all four question sets, seed 1337, optimizer, lr, seq length, batch,
grad-accum, checkpointing, dtype, attention backend, generation settings, and the
same `dynamic_memory.evaluation` scoring code. Only rank moved.

## The twelve required answers

1. **Meaningful composition?** No. 4/24 at epoch 30, `p = 0.58`. Best checkpoint 5/24, `p = 0.37`.
2. **Below the interference target?** No. `I = 0.2500`, 2.5× the 0.10 target, at every checkpoint.
3. **Churn independent of net interference?** Substantial — 10/12 changed, all 10 substantive, against a net of −3. Five regressions partly masked by two accidental repairs.
4. **Hop-2 better than rank 4?** **No — worse.** 10/24 → 6/24, while hop-1 rose 20/24 → 23/24.
5. **Longer or more exploratory traces?** No. 152.1 mean tokens, the shortest of the three ranks.
6. **Does leakage return?** Yes — one world-shaped answer at every checkpoint, `capital of France → CRIOLK-17`. Correlates with rank, not epoch/composition/churn.
7. **When does composition improve?** Never. 5, 4, 4, 2, 4 across epochs 12–30, all at or below chance.
8. **When does perturbation emerge?** Before the first checkpoint. Fully present at epoch 12, never changes.
9. **Where does rank 8 sit?** Not between rank 4 and 16. Damage near rank 16, benefit at or below rank 4.
10. **Usable operating point?** No. Fails interference outright with no composition benefit to weigh against it.
11. **Capacity hypothesis?** **Weakened substantially.** Capacity is not a smooth dial; the hop-2 curve is non-monotone across 4 → 8 → 16.
12. **Continue LoRA capacity, or investigate isolation?** The evidence points at **isolation**. Three ranks spanning 4× in parameters give perfect recall everywhere, composition only at the largest and most damaging, unacceptable interference already at the midpoint, and a non-monotone hop-2 curve. *Caveat:* rank 16 remains the only configuration that has ever composed, and this data cannot exclude that something between rank 8 and 16 recovers composition at acceptable interference — but the non-monotonicity makes a smooth threshold unlikely, and the axis is too poorly behaved to interpolate along.

## Honest limits

- **24 composition, 12 control questions.** One control question is 0.083 of `I`; one
  composition question is 0.042. The rank-8 vs rank-4 composition difference (4/24 vs
  5/24) is **one question** and means nothing on its own. The interference difference
  (6/12 vs 8/12) is two questions — more solid, still small.
- The **hop-2 reversal** (10 → 6 → 14) rests on 4-question differences. It is the most
  interesting finding and the one most in need of replication.
- **One run, one seed, one world, 2-hop only.** Determinism means these numbers
  reproduce exactly; it does not estimate cross-world or cross-seed variance.
- Rank-16 churn and leakage were computed here from Experiment 0's stored control
  answers using the same code, so they are comparable — but Experiment 0 predates the
  churn analysis and was not designed for it.

## State of the machines

| Host | State |
|---|---|
| **jetson** 10.0.2.3 | all GOAL_3 training and evaluation; 33 adapters in `models/adapters/` (6 new from GOAL_3: the final adapter plus checkpoints 12/16/20/24/30); environment unchanged since Experiment 0 |
| **henry** 10.0.2.2 | repository + full git history, synced |
| **jetson2** 10.0.2.4 | still untouched — reserved for independent validation |

Deliverables in `results/experiment_003/` (24 files): `experiment_003_report.md`,
`methods.md`, `results.json`, `determinism_validation.md`,
`detachability_validation.md`, `trace_analysis.json`, `behavioral_churn.json`,
`leakage_analysis.json`, 11 raw `eval_*.json` (base + 5 thinking-off + 5 thinking-on),
`train_memory_G3_all_r8_e30_*.json`, `states345_g3_ep30_*.json`, plus
`rank4_trace_comparison.json`, `rank16_trace_comparison.json` and
`rank16_churn_leakage_comparison.json` — the cross-rank comparisons, written **into
experiment_003** so Experiments 000/001/002 artifacts were not modified.

## Next step — not run (exit condition reached)

GOAL_3's stop condition is explicit: no further ranks, no new training data, no
Memory B, produce the report and wait for review.

The decision now on the table is **whether to keep sweeping LoRA capacity or move to
a mechanism with stronger parameter isolation**. This experiment's evidence favours
the latter, with the caveat recorded in answer 12 above.