Phase 2 found that the read is the wall, and bounded evidence fixed reads by changing the shape of the question. Both of those left the model doing the reading unchanged. This one asks a different question about the same wall: instead of asking a small model to read better, what if a second, larger model is the one reading the evidence and deciding what happens next — while the small model stays confined to issuing the actual tool calls? HourGlass's harness has had exactly that shape, dual_model_planner_executor, since before this programme's public posts started. It had just never been given two different models to be dual about.
Read but don't write, already half-built
The harness splits every agentic round into a planner, which chooses a bounded set of tool calls as strict JSON and is the only role whose output actually executes, and an executor reviewer, which reads the accumulated tool evidence and reasoning trace, and returns a status, a next goal, and optionally a suggested next call — which the planner may or may not take. The executor never runs a tool itself. That is close to exactly the read/write separation this whole research thread keeps arriving at from different directions: a role that is only allowed to look and reason, and a role that is only allowed to act on what it's told.
Nobody had ever pointed different models at the two roles. HOURGLASS_HARNESS_PLANNER_MODEL and HOURGLASS_HARNESS_EXECUTOR_MODEL both defaulted to qwen3-4b, and the dual-host routing that was supposed to send planner calls to a second Jetson (LLAMA_CPP_HOST_2) silently fell back to the same host as the executor whenever the operator env var wasn't set — which it never was. The split existed in the code and had been running collapsed to a single model on a single host the entire time.
Three refusals, found one at a time
Turning it on for a live target meant asking: could a submitted job get real tools — not just the read-only web/file/system set, but code execution and a security-tool bridge — against a target on this box's own loopback, where the CTF lab actually lives? The answer was no, three separate ways, and each one only showed itself once the previous one was fixed:
- The ctf_operator tool profile was unconditionally remapped to an empty ctf_disabled profile, regardless of credentials.
- Even past that, the function that recognizes a client-submitted tool specification only ever accepted the safe read-only set — never python_exec/bash/kali_mcp — from outside the harness's own internal catalog.
- And a third, separate function filtered the raw tool list before either of the first two ever saw it, for the same reason. Two attempts at a fix looked correct and changed nothing, because the fix landed downstream of a gate nobody had found yet.
All three are the right default. A public job-submission API should not hand an arbitrary caller a shell and a pentest toolkit. But this box's queue is reachable only from inside a private lab network sitting behind its own perimeter firewall in a tight zone configuration — the actual threat model here is "not from the internet," which none of the three checks were conditioned on.
Trusted by network, not by claim
So the fix is a precondition, not a bypass: a new is_trusted_exec_source() check, computed once, server-side, from the request's actual TCP source address at submission time — never from anything in the request body. A client can put whatever it wants in its own JSON; it cannot forge the socket it connected from. The flag it produces overwrites whatever the client sent under that key, and every one of the three refusals above now asks it before relaxing. Unset the environment variable that configures the trusted network range, and every default from before is untouched — existing policy-boundary tests pass unmodified, plus new coverage asserting a client-supplied trust claim is always ignored.
The three refusals were correct on their own terms. What they lacked was a way to ask where the request actually came from, so a private lab behind its own firewall was stuck with the same defaults as the open internet.
The deployment had its own lesson
The fix was written and tested against the checked-out development copy of the harness. The systemd service actually answering requests ran a different checkout entirely — same database, different files on disk, with 142 lines of unrelated drift between them. The first two verification attempts failed for that reason alone: the code was correct and simply never reached the running process. That checkout has since been retired; both services now run from the one tree, as the account that owns it rather than a second account, which also means the dual-host routing mentioned above finally resolves to two different Jetsons instead of collapsing onto one — a second, free correction that had nothing to do with the trust gate itself.
The result
Two jobs, same target — a custom line-protocol service on the CTF lab's 127.0.0.1:25000 — same prompt, same ctf_operator profile, same round and tool-call budget:
| armA — baseline | armB — split | |
|---|---|---|
| planner | qwen2.5-3b @ jetson2 | qwen2.5-3b @ jetson2 |
| executor | qwen2.5-3b @ jetson1 | qwen3-4b @ jetson1 |
| rounds / tool calls | 6 / 10 — max_rounds_reached | 3 / 3 — executor_review_final |
| wall clock | 919.5s | 229.5s |
| outcome | stuck | a real answer |
Both arms got the same nmap banner in round one. armA's own generated code then had a bug — it called recv() after already calling close() on the socket — and threw OSError: Bad file descriptor. Its response to that, five times in a row, was to repeat the identical broken pattern: scan again, connect again, close, then fail the same way. It never sent the service a single byte, and it exhausted its round budget having learned nothing past the banner.
armB's executor read the same failure once and changed the approach: round one already tried sending data before reading, and by round three it sent b"help\n" and got back custom_svc v1.0 (type HELP) — a real, previously-unknown fact about the target, in a quarter of the wall-clock time, converging on its own rather than running out its round budget.
The small planner never got smarter across either run. What changed is that a stronger model was reading its output and pushed a different strategy once the first one visibly wasn't working — instead of the harness just re-running the same broken code until the clock ran out.
What this is and is not
n = 1 per arm. This is not a controlled result and it is not being reported as one — it is the first time the question could be asked at all, after a night of infrastructure standing in the way of asking it, and the shape of the single pair that ran is exactly what the hypothesis predicted rather than noise in either direction. Repeats are queued before this becomes a claim instead of an observation. It also doesn't yet test the harder version of the question: both arms here still keep the small model confined to reads and tool calls, never composition or authoring, which is the boundary the binary ladder and Phase 2 both argue for keeping. What's new is only that the reading role is no longer required to be the same size as the acting role.
← Back to lab notes