# GOAL_8_FINAL — Locate the Addressable Representation and Close Phase I

## Experiment 008 — Final Diagnostic

### Research question

Experiments 005–007 progressively isolated the failure of explicit detachable
memory addressing.

- Experiment 005 failed because the address pathway was gradient-starved at
  initialization and collapsed to one slot.
- Experiment 006 opened the gradient path and added anti-collapse pressure, but
  the address still collapsed because the regularizer distributed soft
  probability mass rather than forcing input-dependent hard assignments.
- Experiment 007 directly supervised fact identity. Single-slot collapse was
  finally broken, but held-out address accuracy reached only 11/48 = 0.2292.
  Failure analysis showed perfect relation/hop discrimination (48/48) but poor
  fact/chain discrimination (11/48).

The critical Experiment 007 diagnostic was a zero-parameter nearest-centroid
classifier over the raw frozen layer-18 representation. It achieved only
2/48 = 0.0417 held-out fact accuracy — exactly chance.

This leaves one unresolved question:

> Is layer 18 simply the wrong place to address memory, or does the frozen model
> fail to represent synthetic fact identity in a paraphrase-stable form anywhere
> in its computation?

GOAL_8_FINAL answers that question without training anything.

This is the final experiment in Phase I.

---

## Hypothesis

There may exist some layer and token-position representation in the frozen model
where paraphrases referring to the same synthetic fact cluster reliably enough
to support external memory addressing.

If such a representation exists, the explicit-memory direction was likely
attached at the wrong representation.

If no such representation exists, then a memory mechanism that expects a single
frozen hidden state to provide a stable fact address is unsupported for this
model/task.

No architecture will be modified to compensate for the result.

---

# Hard scope

GOAL_8_FINAL is a **zero-parameter diagnostic**.

Do NOT:

- train an adapter;
- train a classifier;
- train a projection;
- train a router;
- train keys or values;
- modify the frozen model;
- modify the synthetic world;
- regenerate questions;
- add new training examples;
- tune representations after seeing results;
- test another memory architecture;
- test another LoRA rank;
- test another regularizer;
- begin Phase II;
- use jetson2 for exploratory work.

The frozen model must remain bit-identical throughout.

The only operation permitted is:

> extract existing frozen representations and measure how much fact identity
> they already contain.

---

# Experimental inputs

Reuse exactly the existing:

- frozen base model and revision;
- `world_seed1337.json`;
- Experiment 007 fact identities;
- Experiment 007 train/held-out paraphrase split;
- seed 1337;
- tokenizer;
- eager attention backend;
- dtype;
- prompt formatting;
- identity labels.

The split must be loaded from the Experiment 007 artifact or reproduced
byte-identically from the same committed code.

Do not redefine the task.

---

# Representations to probe

Probe all 28 decoder layers.

For each prompt, capture representations at a small pre-committed set of token
positions.

At minimum:

1. **last prompt token**
   - directly tests the representation used by Experiment 007.

2. **cue-entity token**
   - representation associated with the synthetic entity being queried.
   - define the alignment procedure before inspecting results.
   - handle multi-token entities deterministically.

3. **prompt mean**
   - mean hidden representation across the prompt tokens, excluding padding and
     any generated answer tokens.

A fourth representation may be included only if it is clearly defined before
running the sweep and requires no learned parameters.

Do not add positions after seeing the results.

---

# Probe

Use the same basic zero-parameter diagnostic introduced in Experiment 007:

## Nearest-centroid classification

For every `(layer, position)` pair:

1. compute one centroid per fact identity using only the training paraphrases;
2. normalize representations and centroids as required by the pre-committed
   cosine-distance implementation;
3. classify every held-out paraphrase by nearest centroid;
4. report held-out fact accuracy.

No parameters are fitted.

Also compute:

- relation/hop accuracy;
- chain/fact-family accuracy where applicable;
- confusion matrix;
- per-fact accuracy;
- centroid separation;
- within-fact similarity;
- between-fact similarity;
- nearest competing centroid margin.

The headline metric is:

> held-out 24-way fact-identity accuracy.

Chance = 1/24 = 0.0417.

---

# Leakage controls

The experiment must make it impossible for answer leakage to masquerade as a
representation.

Verify that each probed representation is computed before any answer token is
present.

For the cue-entity position, explicitly document whether the identity of the
entity itself trivially determines the label.

If any representation trivially contains a unique lexical identifier that makes
the classification problem equivalent to string recognition, report that
separately rather than treating it as evidence of semantic/paraphrase-stable
addressability.

The important question is not:

> Can the model distinguish different token strings?

It is:

> Do different paraphrases of the same fact converge on a representation that
> distinguishes that fact from the other facts?

---

# Validation before the sweep

Before collecting the full results:

1. verify frozen base fingerprint matches the existing fingerprint:
   `52683d0e840fb51e`;

2. reproduce Experiment 007's layer-18 / last-prompt-token nearest-centroid result:
   2/48 = 0.0417;

3. verify the Experiment 007 split is identical;

4. verify no parameter has `requires_grad=True`;

5. verify no optimizer exists;

6. verify no generated answer tokens enter the probe representation;

7. unit-test cue-entity token alignment, including multi-token cases.

If the Experiment 007 reference result does not reproduce, STOP and diagnose the
pipeline rather than running the sweep.

---

# Primary result matrix

Produce a complete matrix:

| layer | last prompt | cue entity | prompt mean |
|---|---:|---:|---:|
| 0 | | | |
| 1 | | | |
| ... | | | |
| 27 | | | |

Each cell is held-out 24-way fact accuracy.

Do not report only the best layer.

The shape of the representation trajectory across depth is itself a result.

---

# Pre-committed interpretation thresholds

Because this is the final diagnostic, interpretation must not depend on choosing
a flattering threshold after seeing the sweep.

## Strong addressable representation

A `(layer, position)` qualifies as strongly addressable if:

- held-out fact accuracy >= 0.90;
- performance is not explained by answer leakage;
- performance is not merely lexical identity recognition;
- both relations/hops are represented;
- performance is distributed across facts rather than dominated by a small
  subset.

Interpretation:

> The frozen model does contain a representation suitable for explicit memory
> addressing. Experiment 007 used the wrong representation/insertion point.

Do NOT implement the memory there in GOAL_8_FINAL.

Record it as a Phase II/future-architecture finding and stop.

## Partial representation

Best held-out accuracy:

    > chance but < 0.90

Interpretation:

> Fact identity exists partially somewhere in the frozen computation, but not at
> the reliability required by the explicit-memory design.

Report where it peaks and what information it carries.

Do not train a projection to improve it.

## No useful representation

If no representation materially exceeds chance, or apparent gains are explained
by lexical leakage:

Interpretation:

> The frozen model does not expose a single-hidden-state representation suitable
> for paraphrase-stable addressing of these synthetic facts under this probe.

This closes the single-state explicit-addressing branch.

---

# Required analyses

Answer explicitly:

1. Does the Experiment 007 layer-18 result reproduce exactly?
2. Which layer gives the highest held-out fact accuracy?
3. Which token-position strategy gives the highest accuracy?
4. What is the absolute best accuracy across the entire sweep?
5. Does any representation meet the >= 0.90 strong-addressability criterion?
6. Does fact identity emerge gradually with depth or abruptly?
7. Does it later disappear?
8. Is relation/hop identity represented more strongly than fact/chain identity?
9. Does the cue-entity representation outperform the last-prompt representation?
10. Does prompt pooling outperform a single-token state?
11. Are apparent successes explained by lexical identity?
12. How stable are within-fact representations across paraphrases?
13. How separated are different facts?
14. What does the result imply about Experiments 005–007?
15. Was layer 18 the primary problem, or is the representation problem broader?
16. Does the evidence justify another internal-memory experiment?
17. What architectural assumption should be retired at the end of Phase I?
18. What findings should carry forward into Phase II?

---

# Phase I synthesis

The final report must contain a concise synthesis of Experiments 000–008.

At minimum distinguish the following findings.

## Storage

Detachable parameter mechanisms can encode the synthetic world.

LoRA and the isolated residual memory achieved 48/48 recall.

Storage itself is therefore not the central unsolved problem.

## Detachability

Across successful storage mechanisms, knowledge can be attached, removed and
restored while preserving the frozen base weights.

The load/unload primitive is viable.

## Interference

Every mechanism that meaningfully affected the frozen reasoner's computation
produced measurable behavioural perturbation.

Parameter count alone does not explain the interference.

Physical parameter isolation does not imply behavioural isolation.

## Composition

Perfect fact recall does not imply that stored information can participate in
multi-hop reasoning.

The persistent failure is recursive use:

    retrieve A
    -> use A as a new query
    -> retrieve B
    -> answer

Rank-16 LoRA remains the only tested configuration significantly above chance on
composition, and it does so with substantial interference.

## Addressing

Experiments 005–007 successively showed:

- a closed initial gradient path;
- differentiable slot collapse despite anti-collapse regularization;
- successful breaking of collapse through direct supervision;
- failure of the selected frozen representation to provide robust fact identity.

GOAL_8_FINAL determines whether that final failure is local to layer 18 or
general across the frozen network.

---

# Formal Phase I stop condition

After the representation sweep, analysis and report are complete:

**STOP.**

Regardless of outcome:

- do not train another internal memory;
- do not move the KV module to the winning layer;
- do not add a learned projection;
- do not tune the probe;
- do not run another LoRA configuration;
- do not start a router experiment;
- do not begin Phase II in the same session.

GOAL_8_FINAL closes the experimental series.

A positive result informs future work.

A negative result informs future work.

Neither result authorizes GOAL_9.

---

# Deliverables

Write all new artifacts only under:

`dynamic-memory-lab/results/experiment_008/`

Required:

- `experiment_008_report.md`
- `methods.md`
- `representation_sweep.json`
- `representation_matrix.md`
- `layer_analysis.json`
- `position_analysis.json`
- `similarity_analysis.json`
- `confusion_analysis.json`
- `validation.md`
- `phase_1_synthesis.md`

Add raw representation summaries if useful, but avoid storing enormous hidden-state
dumps unless necessary for reproducibility.

Experiments 000–007 artifacts must not be modified.

Implementation and results should be committed separately.

`jetson2` remains untouched and reserved for future independent replication.

---

# Final decision

GOAL_8_FINAL does not choose the next architecture.

It answers the final unresolved question in the current architecture family and
closes Phase I.

The next project, if pursued, begins from a different abstraction:

> Memory as an externally selected, detachable learned object attached to a stable
> reasoner.

That is Phase II, not GOAL_9.