# GOAL_7 Progress — 2026-08-23 Session record for `GOAL_7.md` (can the address itself learn identity?, Experiment 007). Formal deliverable: `dynamic-memory-lab/results/experiment_007/experiment_007_report.md`. **GOAL_7: complete — design and gate thresholds committed before training, isolation and gradient path measured, Phase A run, positive addressing gate FAILED, Hard Stop A applied, failure diagnosed, report written. Exit condition reached at stop condition 1.** ### Bottom line Single-slot collapse is broken. The address is now input-dependent. It encodes WHICH RELATION is being asked (48/48) and not WHICH FACT (11/48). **Outcome A.** Held-out address accuracy **0.2292** against a required 0.90 and a chance of 0.0417. Gate criteria A1, A2, A4, A5 passed; **A3 and A6 failed**. No values trained, no composition run, no interference measured, nothing tuned. | | Exp 005 | Exp 006 | **Exp 007** | |---|---|---|---| | distinct dominant slots / 24 facts | 1 | 1 | **7** | | max dominant share | 1.0000 | 1.0000 | **0.2917** | | between-fact entropy (max 3.1781) | 0 | 0 | **1.8055** | | held-out address accuracy | — | — | **0.2292** | --- ## The design change that made the result readable The user's mid-session correction shaped the experiment. The first draft supervised a 128-way softmax over arbitrary slot targets. That asks one head to learn fact discrimination *and* an arbitrary 24-of-128 allocation at once, and a low accuracy would have been un-attributable between the two. Phase A was restructured into two stages, only one of which learns: A1 learned 24-way cross-entropy, h18 -> fact identity, scored against the 24 identity-bearing key rows A2 NOT learned identity j -> slot active_slots[j], a fixed injection The 104 unclaimed slots left the *objective* and stayed in the *measurement* — the gate still reads a 128-way argmax — which turned "do untrained slots win?" into a measured quantity. It came back **0.0000**: they never won a single probe, so the 24-way and 128-way accuracies are identical and the failure is entirely about fact discrimination. --- ## The diagnosis Held-out confusion, 48 probes: ``` correct 11 same_chain_wrong_hop 0 same_hop_wrong_chain 37 unrelated 0 ``` **Hop accuracy 48/48 = 1.0000. Chain accuracy 11/48 = 0.2292.** Within each hop the predictions concentrate on 4 slots. The 7 dominant slots that satisfied A1, A2 and A4 are essentially the hop partition plus noise — which is exactly why GOAL_7 also demands an explicit accuracy criterion, and why it is right to. **The load-bearing measurement.** A 1-nearest-centroid classifier on the raw layer-18 hidden states — cosine, class means, **no parameters fitted** — scores **0.0417 on held-out, which is exactly chance (2/48)**, with hop accuracy 24/48 = chance for a binary split. The frozen representation at the insertion point does not cluster paraphrases of a fact, and does not even separate the two relations. The trained query projection *added* information (0.50 → 1.00 hop separability); the material it read did not contain fact identity. The bottleneck has therefore moved. It is no longer the addressing dynamics — a directly supervised softmax over slots does not collapse. It is upstream of the memory entirely. --- ## Two things I got wrong, recorded **1. 69 % of supervised positions were an easy sub-task.** Broken out by position, training accuracy is `answer_token` 322/322 = **1.0000** and `last_prompt_token` 116/144 = **0.8056**. Answer positions have the answer entity in context, so they need no generalisation. The headline 0.9399 training accuracy is inflated and should not be quoted. Pre-committed for a real reason (Phase B needs the address to hold during generation) and deliberately **not repaired**, because repairing it after seeing the result is what GOAL_7 forbids. **2. A pre-training check failed and I changed it.** `G3` first demanded that address *scores* recomputed from the cache be bit-identical to the live forward. It failed at `7.451e-09` — a `[P,1024]` and a `[T,1024]` matmul select different cuBLAS kernels, so float32 accumulates in a different order. The claim being made is about the cache, so the check was rewritten to assert the cached *hidden state*, which is bit-identical (`0.000e+00`), with the score residual reported beside it. Stated in the report because changing a check after it fails is a move this program should be suspicious of. --- ## What was proven before training * **Isolation 13/13.** Gate 0 `0.000e+00` vs frozen base, including with randomised non-zero memory parameters so the check is not vacuous. Gate 1 `8.125e-01`. Removal `0.000e+00`. Fingerprint `52683d0e840fb51e` throughout — unchanged since Experiment 004. * **Address gate-invariance `0.000e+00`.** The memory reads the layer-18 block input and writes to the block output, so nothing it stores reaches its own query. This licensed training on cached hidden states at gate 0. * **Gradient path 6/6, measured on the real objective.** `query.weight` 4.30e-01, `keys` active rows 4.27e-01, `keys` inactive rows **0**, `values` **0**, `readout` **0**. GOAL_6's checkpoint-detachment trap cannot recur: Phase A has no checkpointed region. * **Base gate.** 0/48, 0/24, 9/12 — identical to Experiments 000–006. * **Split 9/9 assertions**, reproducing byte-identically on two hosts. Phase A cost **0.883 s** of training plus 20.77 s of caching, on 466 supervised positions. --- ## Soft versus hard, measured from the other side Experiment 006: ~21 effective soft slots, **1** dominant address. Experiment 007: **107.33** effective soft slots, **7** dominant addresses, mean address entropy 4.4444 against a uniform 4.852, mean top weight **0.0750** (006: 0.9948), mean score margin 0.34459 with a minimum of 0.00441. 006 had a razor-sharp address that ignored its input; 007 has a nearly uniform address that responds to it. Neither statistic predicts the other in either direction. Worth noting for any Phase B that follows: at a top weight of 0.075, `read = w·V` is close to `mean(V)` whatever the argmax — a correct address that wins by 0.0044 is not one a value lookup can use. Also recorded: at epochs 0–10 within-fact agreement and between-fact collision were **both 1.00**. Under total collapse every paraphrase agrees and so does every unrelated fact. As collapse broke, agreement *fell* 1.00 → 0.25 alongside collision 1.00 → 0.15. That is why A3 is a pair of thresholds and why agreement is never quoted alone — Experiment 006's trap, reproduced deliberately. --- ## Not measured, and not reported as zero Recall, composition, hop-1/hop-2 retrieval, interference, churn, regressions, repairs, leakage, reasoning tokens, truncations. Phase A's gate failed, so Phases B and C never ran and the control set was never evaluated against a trained memory. The only evaluation that ran was the frozen-base drift check. --- ## State of the machines * **jetson-server** (10.0.2.3) — ran everything: split → isolation → gradient diagnostic → base gate → Phase A → failure analysis → post-training detachability. Peak 5.257 GiB system. * **henry** (10.0.2.2) — not used. * **jetson2** (10.0.2.4) — **untouched**, still reserved for independent replication. * Repo `/home/nethic/mem-exp/dynamic-memory-lab`. Design and implementation committed separately from results, as GOAL_7 asks (`a94e465`, then the results commit). --- ## Next step — not run GOAL_7's stop condition 1 was reached and GOAL_8 was not begun. The justified next experiment is a **zero-parameter probe sweep**: run the same nearest-centroid measurement across all 28 layers and several token positions (last prompt token, cue-entity token, prompt mean). No training, no memory, no architecture — 192 forward passes and a cosine similarity. It answers whether *any* layer of the frozen model represents these facts paraphrase-invariantly, and it is the only thing that distinguishes "the memory is at the wrong insertion point" from "a frozen 0.6B does not encode this anywhere, and no single-hidden-state addressing mechanism can work". Not justified: another temperature, regulariser, slot count, or loss on the same representation, or extending this run past 200 epochs to chase 0.2292 upward. Each is tuning after observing a result. Worth stating plainly for whoever picks this up: **addressability has now failed three times for three different reasons, and only the third one is about the memory's environment rather than the memory's training.** 005 could not learn an address (zero gradient at step 0). 006 could, and collapsed (pressure aimed at mass, not the argmax). 007 did not collapse and still could not address, because the representation it reads does not distinguish the facts.