# GOAL_2 Progress — 2026-08-22

Session record for `GOAL_2.md` (does composition need exposure or capacity?, Experiment 002).
Formal deliverable: `dynamic-memory-lab/results/experiment_002/experiment_002_report.md`.

**GOAL_2: complete — one controlled run, five checkpoints, both evaluation
conditions, detachability verified. Exit condition reached.**

### Bottom line — Outcome C

    Training exposure is NOT what separates GOAL_1 from Experiment 0.
    Capacity is.

| | rank | epochs | recall | composition (thinking) | p vs chance | interference |
|---|---|---|---|---|---|---|
| Experiment 0 | 16 | 30 | 1.0000 | **10/24 = 0.4167** | **0.0033** | 0.3333 |
| GOAL_1 winner | 4 | 12 | 1.0000 | 4/24 = 0.1667 | 0.58 | 0.0000 |
| **Experiment 002** | **4** | **30** | **1.0000** | **5/24 = 0.2083** | **0.37** | **0.0833** |

Rank 4 trained to Experiment 0's full 30 epochs reaches 5/24 — one question above
the 1/6 chance level, not significant. Rank 16 at the *same* 30 epochs reached
10/24. Interference meanwhile never exceeded 0.0833, one tenth of the way to
Experiment 0's 0.3333.

Holding rank at 4 closes **almost none** of the composition gap and reproduces
**almost none** of the interference. The one remaining variable — rank 16 vs rank 4
— must carry both effects.

---

## The trajectory

Rank 4, all 7 projection modules, 30 epochs, checkpoints from a single run:

| epoch | loss | recall | C off | C on | counts | p vs chance | control | **I** | mean tokens | leaks |
|---|---|---|---|---|---|---|---|---|---|---|
| 12 | 0.1232 | 1.0000 | 0.1667 | 0.1667 | 4/24 | 0.58 | 9/12 | **0.0000** | 185.1 | 0 |
| 16 | 0.1066 | 1.0000 | 0.0833 | 0.0417 | 1/24 | 0.99 | 9/12 | **0.0000** | 183.3 | 0 |
| 20 | 0.1003 | 1.0000 | 0.0833 | 0.1250 | 3/24 | 0.79 | 8/12 | **0.0833** | 182.1 | 0 |
| 24 | 0.0956 | 1.0000 | 0.0833 | 0.1250 | 3/24 | 0.79 | 8/12 | **0.0833** | 188.7 | 0 |
| 30 | 0.0923 | 1.0000 | 0.0833 | 0.2083 | 5/24 | 0.37 | 8/12 | **0.0833** | 196.7 | 0 |

**Composition does not trend up.** The sequence 4, 1, 3, 3, 5 fluctuates within ±2
questions of the 4/24 chance count; the epoch-16 dip is as large as the epoch-30
rise. Correlation with epoch is `r = +0.47` across five points.

**Interference trends up weakly then stops**: 0.0000 at epochs 12–16, 0.0833 from
epoch 20 onward (`r = +0.84` with epoch). A one-question effect that plateaus.

**Recall is exactly 1.0000 (48/48) at every checkpoint.**

Training cost: 3087.8 s net (checkpoint saves 0.69 s total), peak 5.201 GiB system /
1.293 GiB CUDA, 2,523,136 params (0.423%), 9.68 MiB at every checkpoint.

## Mechanism — the clearest result

| epoch | composition | mean tokens | truncations | traces naming hop-1 element | traces naming correct radiation |
|---|---|---|---|---|---|
| 12 | 4/24 | 185.1 | 0 | 18/24 | 7/24 |
| 16 | 1/24 | 183.3 | 0 | 19/24 | 5/24 |
| 20 | 3/24 | 182.1 | 0 | 19/24 | 8/24 |
| 24 | 3/24 | 188.7 | 0 | 20/24 | 9/24 |
| 30 | 5/24 | 196.7 | 0 | **20/24** | **10/24** |
| *Exp 0, r16* | *10/24* | *334.6* | *3* | *—* | *—* |

**Hop-1 retrieval is at ceiling and improves with exposure** (18 → 20 of 24 traces
name the correct intermediate element). What does not follow is the second hop: only
10/24 traces ever name the correct radiation, and only 5/24 answers do.

Rank 4 has the knowledge and reliably retrieves the intermediate. It cannot convert
that intermediate into the second lookup. More epochs sharpen the first hop and leave
the second roughly where it was.

Traces stay **short, stable, non-exploratory**: 182–197 tokens throughout, zero
truncations, essentially zero looping. Experiment 0's rank-16 adapter produced
334.6-token traces that hit the 1024-token cap three times. The extra reasoning that
found Experiment 0's answers is absent, and 18 extra epochs did not produce it.

Correlations (n = 5, indicative only):

| | Pearson r with composition |
|---|---|
| traces naming the correct radiation | **+0.81** |
| mean trace tokens | +0.71 |
| epoch | +0.47 |
| interference | +0.43 |

Composition tracks *whether the second entity appears in the trace* more tightly than
anything else — consistent with GOAL_0's finding that the chain of thought carries
the intermediate.

## Leakage: a different failure from GOAL_1

**Zero world-entity leakage at every checkpoint**, including epoch 30. GOAL_1's
high-interference adapters answered "capital of France" with `ZHAELK-50` and leaked
in proportion to interference (up to 4 answers). Nothing of the kind here.

The single control question lost from epoch 20 is `c_compare` — *"Which is larger,
3/4 or 2/3?"* → base `3/4`, adapter `2/3`. Ordinary degradation, not contamination.

Reconciliation: leakage appears to be a **high-interference phenomenon**. At the low
end, interference is ordinary degradation instead.

## What the interference metric hides

`I` is a **net** score and understates perturbation substantially. Even where
`I = 0.0000` (epochs 12–16) the adapter changes **11 of 12** control answers
textually and **7 of 12** substantively:

| epoch | changed | substantive | regressions | repairs | net |
|---|---|---|---|---|---|
| 12 | 11 | 7 | 2 | 2 | +0 |
| 16 | 11 | 7 | 2 | 2 | +0 |
| 20 | 11 | 8 | 3 | 2 | −1 |
| 24 | 11 | 8 | 3 | 2 | −1 |
| 30 | 11 | 8 | 3 | 2 | −1 |

It breaks two questions (`c_units` 200 cm → `52`; `c_syllogism` No → Yes) and repairs
two the base model got wrong (`c_sequence`, `c_days`). From epoch 20 a third
regression appears and the net becomes −1.

**"Zero interference" means "no net accuracy change on twelve questions", not "no
effect on the reasoner".** Recorded in `control_churn.json`; it matters for how the
metric is read in later goals.

## Detachability (epoch 30)

| State | Weights | Recall | Composition | Control |
|---|---|---|---|---|
| 3 load | `W_core + ΔW_A` | 1.0000 | 0.0833 | 0.6667 |
| 4 unload | `W_core` | 0.0000 | 0.0000 | **0.7500** |
| 5 reload | `W_core + ΔW_A` | 1.0000 | 0.0833 | 0.6667 |

Knowledge appears on load (0/48 → 48/48), disappears completely on unload, returns
exactly on reload. Base control returns to the frozen 0.7500. Base weights
bit-identical throughout (`52683d0e840fb51e`); never merged. Load 0.617 s / unload
0.099 s / reload 0.772 s.

## Controls held

**Base validation was a hard gate, enforced in the driver rather than by inspection.**
Before any checkpoint was scored the frozen model was re-measured: recall 0/48,
composition 0/24, control 9/12 — identical to Experiment 0. The pipeline was written
to exit before training if it drifted.

**Epoch-12 equivalence verified by content hash, not assumed.** This run's epoch-12
checkpoint is **byte-identical** to GOAL_1's `memory_S5_all_r4_ep12`:

    SHA-256  d3ff3fa9ebe30fb4c273fa1199bf88eb   (both)

`adapter_config.json` identical; loss curve identical at every printed epoch. Both
evaluations re-run inside `experiment_002` reproduced GOAL_1's numbers exactly
(recall 48/48, composition 4/24, control 9/12, I = 0.0000).

Combined with GOAL_1's demonstration that a fresh run reproduced Experiment 0's first
12 epochs bit-identically, **training and evaluation are now confirmed deterministic
end to end** — the third and fourth independent confirmations.

Everything else was held fixed: model revision, world (`world_seed1337.json`, not
regenerated), all four question sets, seed 1337, optimizer, lr, seq length, batch,
grad-accum, checkpointing, dtype, attention backend, generation settings, and the
same `dynamic_memory.evaluation` scoring code. Only training exposure moved.

## The ten required answers

1. **Rank 4 recover composition at 30 epochs?** No. 5/24, `p = 0.37`.
2. **When does composition first rise above chance?** Never, supportably. Only epoch
   30 exceeds the chance count, by one question, `p = 0.37`.
3. **When does interference first become measurable?** Epoch 20 (9/12 → 8/12,
   `I = 0.0833`), then plateaus. *Caveat:* substantive control churn exists from
   epoch 12, before the net metric moves.
4. **Do composition and interference rise together?** Weakly and inconclusively
   (`r = +0.43`, n = 5). This run lacks dynamic range — neither variable moved much.
5. **Does recall remain stable?** Yes, exactly 48/48 at every checkpoint.
6. **Does leakage correlate with interference?** Not here — zero everywhere while I
   went 0 → 0.0833. In GOAL_1 it tracked interference monotonically across 0.00–0.58.
7. **Does trace length correlate with composition?** Positively within this run
   (`r = +0.71`), and strongly across experiments (334.6 tokens → 10/24 vs 182–197
   tokens → 1–5/24). But length is a proxy; "names the correct radiation in the
   trace" predicts better (`r = +0.81`).
8. **Does the final adapter satisfy both goals?** No. Interference ✓ (0.0833 ≤ 0.10)
   and recall ✓ (1.0000); composition ✗. Same partial success as GOAL_1, slightly
   worse interference.
9. **Is exposure sufficient to explain GOAL_1 vs Experiment 0?** **No — the main
   result.** Matching Experiment 0's epochs at rank 4 closes almost none of either
   gap. The rank axis carries both effects.
10. **Implication for the next architecture decision?** Capacity, not exposure, gates
    whether parametric knowledge participates in reasoning — and in this space
    capacity also brings the interference. Experiment 0 and Experiment 002 now
    **bracket** the tradeoff: rank 16 composes and damages, rank 4 is harmless and
    inert, training longer moves neither much. Target the **rank axis between 4 and
    16**, and optimise "does the second hop appear in the trace", which predicts
    composition better than accuracy aggregates.

## Honest limits

- **24 composition and 12 control questions.** Every difference here is one to two
  questions. The epoch-16 dip (1/24) is almost certainly noise; so, most likely, is
  the epoch-30 rise (5/24). **The trajectory's flatness is the finding, not any
  individual point.**
- **One run, one seed, one world, 2-hop only.** Determinism means these numbers
  reproduce exactly, but they do not estimate variance across seeds. A different seed
  could plausibly move any cell by ±2 questions.
- **n = 5 for every correlation.** Reported `r` values are indicative, not inferential.
- Intermediate ranks (6, 8, 12) are unmeasured at 30 epochs.

## State of the machines

| Host | State |
|---|---|
| **jetson** 10.0.2.3 | all GOAL_2 training and evaluation; 27 adapters in `models/adapters/` (6 new from GOAL_2: the final adapter plus checkpoints 12/16/20/24/30); environment unchanged since Experiment 0 |
| **henry** 10.0.2.2 | repository + full git history, synced |
| **jetson2** 10.0.2.4 | still untouched — reserved for independent validation |

Deliverables in `results/experiment_002/`: `experiment_002_report.md`, `methods.md`,
`results.json`, `epoch12_equivalence.md`, `trace_analysis.json`,
`interference_analysis.json`, `control_churn.json`, 11 raw `eval_*.json` (base + 5
thinking-off + 5 thinking-on), `train_memory_G2_all_r4_e30_*.json`,
`states345_g2_ep30_*.json`.

Experiment 0 and Experiment 001 artifacts were **not modified**. A `--results-dir`
option was added to the shared tooling so GOAL_2 output lands in `experiment_002`.

## Next step — not run (exit condition reached)

Train **rank 8, all 7 modules, 30 epochs** — the midpoint of the only axis still in
play. GOAL_1 measured rank 8 at 8–12 epochs (interference 0.0833–0.2500) but never at
30.

- If rank 8 at 30 epochs composes significantly above chance while holding
  `I ≤ 0.10` → the tradeoff is not a hard coupling and a usable operating point exists.
- If it composes only by acquiring Experiment 0's interference → the coupling is real
  at the capacity axis, and the next architecture needs parameter isolation that LoRA
  does not provide.

Producing the report and waiting for review, per the stop condition. No further
adapter geometries, no new training data, no multi-memory work.