The Harness

This project runs on a fleet of AI agents, and a single unguided model session is, in the author’s words, “unpredictable, overconfident, unaccountable and worse than useless - actively annoying.”

The fix is not a better model. “The human contribution, what makes this engineering and not vibes, is the harness - which is literally a structure that controls a system of many agents, making the outputs predictable and useful. A water jet can cut almost as well as a laser beam, yet a stray drip can rot a house from its foundations. It all comes back to control. The difference between a harness and a voltage is that the harness is written in text - it’s still a signal.”

That is the whole idea of this page. An agent explores every plausible solution; the harness plus verification is how the right one is found and known to be right. “I just write the prompts and make the rules.”

What the harness is made of

Everything below is plain text in the repository — versioned, diffable, and auditable like any other engineering artifact.

Hooks — 15 scripts wired across 8 session events in .claude/settings.json (PermissionRequest, UserPromptSubmit, PreToolUse, PostToolUse, Stop, SessionStart, SessionEnd, PreCompact), plus 3 shared modules (provenance.py, terminology.py, koans.py) — 18 files in .claude/hooks/. Each one exists because of a documented live miss, and each catches one specific failure:

Hook	Event	What it catches
allow-read-only.py	PreToolUse (Bash)	classifies every shell command allow-or-deny, never “ask” — a denylist of destructive shapes (rm, reset —hard, chains, pipes into writers); the complement is allowed and the sandbox confines it
preflight_guard.py	PreToolUse (Write/Edit)	surfaces the rule that governs the file being touched; denies new files outside the sanctioned write homes; new skills and hooks are propose-then-approve
plotting-guard.py	PreToolUse (Write/Edit)	denies a direct import matplotlib inside restricted dirs — figures must come from the plotting pipeline
durable_doc_guard.py	PreToolUse (Write/Edit)	denies an unsourced size entering tasks/ or notes/ — a durable number must carry its receipt inline
ast-grep-nudge.py	PreToolUse (Bash/Grep)	on a structural code search via grep, hands back the exact ast-grep equivalent — never blocks
wiki_yq_nudge.py	PreToolUse (Bash)	on a wiki search, points at yq for structural frontmatter queries — never blocks
firecrawl-nudge.py	PreToolUse (WebFetch/WebSearch)	points at the raw-page fetcher instead of the lossy summarizer — never blocks
comms_nudge.py	UserPromptSubmit	re-fires the communication contract before every reply, because instructions loaded once fade by end of session
reflow_md.py	PostToolUse (Write/Edit)	reflows just-written markdown to one line per paragraph — fixes the habit at the source
plan_to_clickup.py	PostToolUse (ExitPlanMode)	turns an approved plan into ClickUp tasks, so tracked work is born from the plan itself
reply_guard.py	Stop	the reply door — a decision question left in prose (the user is never notified), a result number with no source, a banned coined metaphor, a substantive turn that never reported to the board
claims_inject.py	SessionStart	injects the active peer CLAIMs from tasks/AGENT_COMMS.md, so every new session opens knowing which paths a parallel agent holds
retro_brick.sh	SessionStart	self-heals a missed memory capture from the previous session’s transcript
palace_capture.sh	SessionEnd	renders and indexes the closing session into the memory palace
precompact_flush.sh	PreCompact	checkpoints durable state before lossy context compaction

When a hook denies, the correction comes first and a one-line koan follows — drawn from koans.py — so the wall reads as something to lean on rather than a bare “no”. An external reviewer, plannotator, opens every plan for human annotation at the PermissionRequest gate before it can execute.

Skills — 27 project skills in .claude/skills/ plus 6 wiki-maintenance skills in vault/.claude/skills/. A skill is a written norm: it shapes what the agent intends to do before any hook has to catch anything. The science spine is three of them in strict order — lab-report (design and pre-register the experiment: question, theory, math, prediction), sim-runner (execute on the compute fleet), log-driver (report from the saved logs). Around the spine sit diagnosis (root-cause-analysis), continuity (pass-the-torch, take-the-torch, mempalace-capture, debrief), teaching (math-teacher), proof work (lean-proof), publishing (site-convert, weekly-report), and hygiene (notes-weekly, validation-purge, the wiki lint skills).

Hard walls — the permission tables in .claude/settings.json (per-tool allow and deny lists; the author’s own thesis text under Docs/my_writing/ is write-denied to every agent) and the OS sandbox (a filesystem write allowlist, read denies on credentials and browser profiles, and a network host allowlist). Three hookify rules add machine-local walls: block package installs, block Quartz plugin mutations, warn on unsourced sizes.

Coordination — parallel peer agents negotiate through tasks/AGENT_COMMS.md: “CLAIM before editing a shared repo,” name both ends of every message, commit only your own slices by pathspec. The live task queue is ClickUp (as of 2026-07-09), driven through a bot account so every agent write is attributable.

Memory — an auto-memory index of ratified lessons, a searchable memory palace of every past session, and this vault. Continuity is what turns one-off corrections into standing rules.

How it grew

Every date below is from the git history of this repository.

timeline
    title The harness, dated
    section Early June 2026
        2026-06-08 : first commits — the nominal GNC baseline, no harness yet
    section Mid June 2026
        2026-06-17 : the Bash permission gate allow-read-only.py already in force — first commit tuning it
        2026-06-20 : the sandbox affirmed as the security boundary, never bypassed
        2026-06-21 : quote-aware parsing fixes — hooks harden from live misses
    section Late June 2026
        2026-06-28 : Panopticon umbrella repo — one root, one board, memory palace capture
        2026-06-29 : enforcement hooks and settings tracked in the umbrella
        2026-06-30 : lab-report skill v1 — pre-registration enters the harness
    section Early July 2026
        2026-07-01 : rules-fold — scattered rule files folded into skill references : two wikis merged into one vault, provenance and terminology hooks land
        2026-07-02 : claims_inject — peer CLAIMs injected at every session start
        2026-07-03 : torch pair — pass-the-torch and take-the-torch, session continuity
        2026-07-04 : debrief, daily-news, math-teacher, lean-proof skills
        2026-07-08 : ClickUp mirror goes live
        2026-07-09 : ClickUp becomes the task board, plan-to-ClickUp hook : board_hygiene_stop retired — reply_guard slims the Stop gate, verification duty moves to a dedicated agent, pinocchios-nose

The pattern in the dates: walls first (the Bash gate and the sandbox, mid June), then norms (the skill build-out, late June onward), then coordination and memory (July), with each hook traceable to a specific documented failure it now prevents. The harness was not designed in one sitting; it accreted, one verified lesson at a time.

The life of one tool call

Every action an agent takes passes through the same gauntlet. Deny at any gate returns a correction (and a koan) to the model, which must try a compliant route.

flowchart TD
    A["User prompt"] --> B["UserPromptSubmit hook<br>comms_nudge.py — re-fires the communication contract every turn"]
    B --> C["Skill layer<br>the written norm that shapes intent — lab-report before sim-runner before log-driver"]
    C --> D["Model proposes a tool call"]
    D --> E["PreToolUse hooks<br>allow-read-only.py denies destructive Bash · preflight_guard.py stops stray writes and surfaces the governing rule · plotting-guard.py blocks direct matplotlib · durable_doc_guard.py denies unsourced sizes · three nudges redirect to sharper tools"]
    E -. "deny — correction + koan" .-> D
    E --> F["Permission walls<br>settings.json per-tool allow and deny lists — the thesis text is write-denied to agents"]
    F --> G["Sandbox<br>filesystem write allowlist · credential-read denies · network host allowlist"]
    G --> H["Tool runs"]
    H --> I["PostToolUse hooks<br>reflow_md.py reflows the markdown just written · plan_to_clickup.py turns an approved plan into tracked tasks"]
    I --> J["Reply drafted"]
    J --> K["Stop hook<br>reply_guard.py — unpinged decisions, unsourced numbers, banned coinages, unreported substantive turns"]
    K -. "block — fix the reply first" .-> J
    K --> L["Reply ships"]
    L --> M["SessionEnd and PreCompact<br>palace_capture.sh and precompact_flush.sh index the transcript before it can be lost"]

What each layer is for

The layers are deliberately different instruments, and the division matters:

• Skills are norms. They shape what the agent intends before it acts — the cheapest place to control behaviour, because nothing has to be caught.
• Hooks are backstops, not cages. A minimal hook catches the honest lapse — the rushed reply with an unsourced number, the markdown written ragged. A hook that tried to enumerate all misbehaviour would be an arms race; the norm carries the intent, the hook catches the slip.
• Settings and the sandbox are hard walls. Some things must be impossible regardless of intent: writing into the author’s own thesis text, reading credentials, reaching arbitrary network hosts. Walls do not nudge; they refuse.
• Memory is continuity. Ratified lessons, session captures, and the vault turn each correction into a standing rule, so the same mistake is not re-made by the next session.

The order is also a budget: intent is shaped for free, lapses are caught cheaply, and only the truly forbidden costs a refused action.

The trust chain ends in Lean

The harness exists before the risk-aware science for one reason: an output that cannot be trusted is worth nothing, however clever. In the author’s words: “my harness is a really sophisticated bullshit detector, but agents are quantum tunnellers, good at coming up with ideas but bad at knowing if they’re worthwhile. Automatic, pass/fail, no-fluff verification is a necessary first step to get them to produce anything that’s not utter nonsense and a waste of compute.”

Verification runs in three tiers, from behaviour to numbers to mathematics:

1. Behaviour — hooks and rules constrain what agents may do, and every claim that reaches a durable document must carry its source inline (that is a hook, not a policy statement).
2. Numbers — the simulation pipeline is the sole source of truth: a run is born from a run_spec.py YAML, executed by the Orchestrator, and measured by analysis/data_analyzer.py. A figure or metric produced any other way is discarded, and provenance of every artifact is checked against the code that made it. Experiments are pre-registered with the lab-report skill — question, theory, prediction — before any simulation runs, so the prediction is on record before the measurement can bias it.
3. Mathematics — the floor of the whole chain is machine-checked proof. Control-theory results in the Lean library count as sealed only when #print axioms reports nothing beyond Lean’s three standard axioms (propext, Classical.choice, Quot.sound) in a full build — a pass/fail answer no reviewer’s goodwill can inflate.

The author’s summary of the working method, verbatim: “Losing: Asking an LLM to repeatedly think about input data, hoping it will produce useful output sooner or later. Winning: Asking an LLM to produce and verify, through previously accepted methods, the testable script/library/procedure which automates the production of deterministically correct output from input data.”

Control of the spacecraft and control of the agents are, in the end, the same discipline: a feedback structure around an energetic, unpredictable plant, with verification as the sensor. It all comes back to control.

Where to go next

• Code wiki — the Pinocchio guidance and control implementation, one page per file.
• Research wiki — the literature: sources, topics, and named results.
• Lean library — the machine-checked control-theory proofs.
• Glossary — every project term, one line and a source each.