Testing in Production
Dossier
000
Compiled
Length
3,288 words
Sources
10
Matched
0 of 75

Field reportThe Base.

Sessions are disposable; knowledge is not. How a file-based memory keeps an AI assistant's context durable: four tiers with one lever each, one writer, budgets measured in bytes, and the rules that stop a half-true note coming back as a fact.

capture, route, point, measure, close

  • 4 tiers
    always-on, on-touch, just-in-time, archive: each loaded by a different mechanism, each moved by a different lever.
  • 25,000 B
    the hard cap on the always-on index. Past it the harness truncates silently and pointers vanish without an error.
  • 90 days
    a session's full transcript stays queryable after it closes, so closing loses nothing.

01 · Idea

Closing a session should cost nothing

An assistant that forgets everything at the end of a conversation makes you afraid to end conversations. The fear is the expensive part.

A coding assistant works in sessions. Each one holds the files it read, the commands it ran, the decisions it made, and it holds them only until it ends. The natural response is to keep one session open for days, which costs more every hour: a provider caches a long context for a short window, and after that window every message re-reads the whole history at full price. The author measured that spiral at about $38 a day before this system existed.

The fix is not a bigger window. It is to make every durable fact live in a file, give every file one delivery channel, and budget every channel. Then a session can close at any boundary: the digest is already on disk, the depth is in the notes, and the full transcript stays readable for ninety days. The next session opens with what matters already in its context and asks for nothing.

Two definitions carry the whole design. Durable means a future session, or a person browsing the notes, would need it. Uncertain is not durable: a conclusion the assistant would not state as a fact goes to the status file with a review note, never to a memory tier, because recall here is keyword-matched with no staleness signal and a half-true note resurfaces later looking confirmed.

02 · Mechanics

Four tiers, one lever each

A file-based system, not vector search. Nothing is embedded; what loads is decided by where a file sits and what its first lines say.

  1. Tier 1 always onThe index and the digest
    A memory index (identity, hard rules, pointers) and a status digest (open threads, next actions). Both load in full at every session start and again after the context is compacted.
  2. Tier 2 on touchProject shims
    A one-kilobyte file per repository, scoped to its paths. It loads the first time a file in that repository is touched and points at the project's own status file and its governing skill.
  3. Tier 3 just in timeTopic files, skills and the vault
    Per-project status, topic files, skills and about two thousand notes. Loaded on demand, through a pointer or a skill's trigger. A note nothing points at is invisible.
  4. Tier 4 archiveTranscripts
    Every session's full transcript, kept ninety days, never resident. A small, cheap model is pointed at one by its ID to recover decisions, threads and next steps.

What moves each tier

TierLoadsLeverBudget
Memory indexEvery session, in fullSelection: pointers, identity, hard rules only25,000 B hard cap, act at 22,500, warn at 21,500
Status digestEvery session, in full, through an import in the user instructions fileAdmission and density28,000 B ceiling, 32,000 hard stop
Atomic memory filesOnly when opened, found by keyword match on a description lineVocabulary: the description holds a phrase the user would typeOne fact per file
Skill bodiesThe description every session, the body on invocationDomain scope: a soft rule fires when its work happensNone
Per-project status and shimThe shim on first touch of the repository, the status by pointerFreshness: overwrite, never appendShim under 1 KB
Topic filesOnly when opened, through a cluster pointer in the indexA cluster pointer: detail at zero index bytesPatterns 140 lines, decisions 130
Vault notesOnly by explicit queryAn inbound pointerUnbounded
TranscriptsNever; one is read on request by its IDThe session ID, written into the digest90 days

The always-on cost is about fifteen thousand tokens, around one and a half percent of a million-token window. The design leaves room to double that before it matters, and then refuses to, because every always-on line is a fixed attention tax on every task that follows.

Why the hard cap is hard

The memory index has a real cliff: past 25,000 bytes the harness truncates it silently, and pointers at the end of the file stop existing without an error. The status digest has no cliff, only a cost, since it is imported whole into every session. Treating the two with equal severity was a category error the system made for weeks: one is a safety rail, the other a cost control, and the rules for each differ.

03 · Routing

What writes where

Every fact has one home. The home is chosen by what kind of fact it is and how it will be recalled, not by where it was learned.

Volatile status
Focus, decisions this cycle, next actions, blockers, in-flight versions. The status digest; project detail goes to that project's own status file.
Hard guardrail
Safety, correctness, identity; must fire on every task. An atomic file and a one-line pointer in the index, in the top block. A hard rule scoped to one domain stays inline too, because a skill is not guaranteed to be invoked.
Soft domain rule
Style or method scoped to one work area. The matching skill's rules section, citing the atomic file. No index line.
Soft universal preference
Fires across many kinds of task. A compressed one-liner in the index under working preferences.
Pattern or decision
A cross-project pattern with when to apply it; a significant decision with its rationale. Two topic files with line caps, archived to the vault when they overflow.
Reference
Tools, methods, projects, people, meetings, diagrams. A vault note written through the vault's own tool, plus an inbound pointer from a file that loads.
System structure
A machine, a network change, a backup topology, a scheduled job, a repository added or archived. The matching architecture document, reconciled, not a status line.
Never the vault
Dumps of rows from work systems. They go to a downloads folder or a sheet; the vault holds method, decisions and conventions.

Promotion, not prediction

The right home for a fact changes as the fact matures, so the system does not guess it at first sight. A new fact goes to the status digest, where it costs no permanent budget and is expected to churn. It earns an atomic file and an index pointer when it recurs across sessions and is confirmed, written opportunistically the next time the assistant is already writing about it. It becomes a vault note when it is reference-grade or crosses projects, and it is demoted or deleted when superseded.

Only two things ever earn a line in the always-on index: a rule that must fire on every task, and an identity fact. Everything else reaches its home through a pointer that already exists, at zero new index bytes. The test is a question: would this have to fire on a task where its topic file had not already been opened? The index stays flat by construction because every added line is paid for in the same pass by a line clustered, demoted or pruned, and the labour goes into the durable artifact rather than the index.

04 · Protocol

The checkpoint, in four steps

One skill is the only writer of the digest and the gate every fact passes through. It runs at every boundary: a task done, a push to main, stepping away, a risky pivot, the end of the day.

  1. Pre-flight. Read the day's auto-logged note (the files touched, with the conversation as backfill). Read the capture buffer, where the assistant appended one line each time it reversed itself, fixed a failure, disproved a stale rule or found a flagged thing was fine; every line is a candidate, none auto-promotes. Measure the index and the digest in bytes before writing anything.
  2. Audit and manifest. Sweep the session for durable knowledge and show a manifest before executing: update over create, one line per related change, every skip with a reason, every cold-tier create naming its inbound pointer. Ask once whether the session changed system structure.
  3. The handoff pass. For each block bound for the digest, an admission test: the work has no status file of its own; there is a next action within about two weeks; a session that never touches the project would still need it. A block that is only there because it feels important fails the third test. Then the next-session blocks, the session's name and ID written into the header, and the budget measured on both sides of the write.
  4. Execute and report. Vault notes through the vault's tool with the full body and a read-back; memory files through surgical edits, never a script, because a scripted edit is reverted by the harness and a second live session flushes its own copy over it. The report lists every write, every skip, and the sizes.

The admission test exists because the old rule did not work

The digest used to carry a post-hoc rule: measure after writing, relocate if over. It stayed over its target for weeks and grew four separate times in one day while that rule was in force. A cleanup instruction that runs after the damage is trivially skipped; a gate that runs before the write is not. The lesson generalises: constrain the writer, not the symptom.

Density, not bytes

When the digest was measured properly, 81 percent of its bytes were guard-bearing: rules, do-not-relitigate decisions, cold warnings. Only about five thousand bytes were pure status. Meeting a raw byte target would have meant deleting guards, the one class of content that must never be lost. So the invariant became composition: the share of bytes that carry a guard word. A dense file that is over the ceiling moves domain guards into the skill that owns them; a file whose density is falling has accumulated status and figures, and those compress.

The first density metric was wrong in an instructive way. Its regular expression counted warning glyphs, so a bare warning symbol fired 39 times and a red circle 14 times, 53 of 77 matches on the digest. It measured whether the author had typed a glyph. The same file read 78 percent marked and 59 percent lexical once the glyphs were excluded. The lexical number is now the one reported, and no floor was set on it the day it changed: inventing a threshold on the day a metric is born repeats the mistake being fixed.

05 · Capture

Recall is the bottleneck, not knowledge

An assistant recognises a stale rule or a reversal in the moment and fails to act on it later. The buffer exists for that gap.

Mid-session, four triggers write one raw line each to an append-only buffer: a reversal (the assistant asserted one thing and the code showed another), a fail-to-fix (a command that failed and then worked, where the delta is the lesson), a stale-disproven (a document or rule proven wrong live), and a noop (something flagged as broken that turned out to be intentional). The buffer is a plain file with no validation, so the write is cheap enough to happen while the insight is fresh.

Capture is not promotion. Self-corrections carry false positives at a rate the author put near one in six, so the checkpoint distils the buffer through the manifest with a review gate and then clears it back to its header. The buffer is multi-session safe because it is append-only: every live session adds its own lines and none rewrites another's.

Sessions are disposable; knowledge is not. Closing a session loses nothing; opening one restores what matters automatically. So: work in short, named sessions, checkpoint at boundaries, and never idle-park a session out of fear.

the operating guide, July 2026

06 · History

How it got here

Every rule in the system was a measured failure first. The dates are when the measurement was taken.

  1. 28 Apr 2026
    An automatic skill-extraction system that minted a skill from every session gotcha is retired. The rule that replaced it: a gotcha is a one-liner first and a skill only after it recurs twice with a multi-step procedure.
  2. 22 Jun 2026
    The memory index had become a content store. Thirty-one soft domain rules move into the skills that own them; the index returns to a pointer list with the hard guardrails hoisted to a top block. The configuration directory goes under local git.
  3. 28 Jul 2026
    Continuity v2. The status digest moves from a start-of-session hook, capped at ten kilobytes and silently truncating, to the uncapped import in the user instructions file. Path-scoped shims give each repository its on-touch context. Transcripts are kept ninety days and a harvest command reads them.
  4. 29 Jul 2026
    A placeholder string passed as a note body overwrites a 126 KB note; the same day an unexpanded shell substitution wipes a 29 KB meeting note. The read-back after every write becomes routine.
  5. 6 Aug 2026
    The digest grows four times in one day under a post-hoc cleanup rule. The admission test replaces it. The same day, picking the newest transcript file by time stamps the previous session's ID into the header; the session ID is read from the scratchpad path from then on.
  6. 19 Aug 2026
    The digest is measured at 81 percent guard-bearing bytes. A raw byte target is replaced by a density invariant and a cost ceiling.
  7. 31 Aug 2026
    The density regular expression is found to be 69 percent glyphs. Density becomes two numbers, and the lexical one binds.
  8. 18 Oct 2026
    The checkpoint skill is rewritten from 40 KB of accreted rules to 20 KB with the same rules, tested against the same closing scenario before and after. This dossier is set from it.

07 · Measured

The system on one day

Counted on 18 October 2026 on the machine that runs it. Counts drift; the mechanism does not.

  • 192
    atomic memory files, one fact each, among 207 files in the memory directory
  • 23,390 B
    the index, over its 22,500 action line: a rebalance owed, the lever being domain rules into skills
  • 28,564 B
    the digest, over its 28,000 ceiling by 564 bytes and under the hard stop
  • 37
    skills, 18 of them carrying a rules section folded in from memory
  • 19
    project shims, for 43 project status directories
  • 1,965
    vault notes
  • 12
    hook scripts, among them a guard that blocks direct edits to the vault and one that archives every compaction summary
  • 144 / 138
    lines in the patterns and decisions files, both at their caps

Two of those numbers are red: the index and the digest are both over their lines on the day of counting. The system records that fact in the digest's own header rather than hiding it, because the next session needs the true figure, and the rule for the writer is net-zero at minimum: relocate at least what you add, in the same pass, before writing.

What it costs to run

The checkpoint takes a few minutes at each boundary and reads two files it did not write. The monthly audit skill checks staleness on declared dates, never on file modification times, because the author works on live projects from the home directory and a sync tool rewrites timestamps across machines; it also runs the density oracle and writes a report into the vault. The harvest of a dead transcript costs cents.

08 · Rules

Learned the hard way

Each of these cost at least one note, one session or one afternoon. They are the reason the protocol has the shape it has.

A non-literal body wipes a note
The vault's write tool replaces the body verbatim. A placeholder, a template variable or a shell substitution passed as the body becomes the note. No size check fires. Type the body, read it back, count its bytes.
The tool's index is a snapshot
A direct edit on disk is durable and visible to the always-fresh server, but the tool process keeps serving the old body until it restarts, and a rebuild does not refresh it. Never build an upsert from a read taken after a direct edit: it writes the stale body back.
Concurrent writers are real
The digest changed size between a read and the following write, twice in one day, with a peer session live on another machine. Prefer targeted edits; after any scripted edit, search for each change to prove it survived; say when peers are live, because the number moves.
A thin commit is not a failed close
The configuration repository tracks only the memory directory and the architecture folders. One checkpoint routinely writes three stores while the log shows one slice, and a peer's add-all can sweep half-finished files into its own commit. Verify artifacts by their modification times, not the diff.
An idle session closes against the layout it remembers
A session resumed after five days wrote to a skill file that a split had deleted in the meantime. Handoff targets are re-derived from the current layout, and a handoff that writes nothing because its substance already landed is a success.
Commit from one machine only
The repository's git directory is per-machine while the tracked folders sync between machines. A commit from the second machine forks the history.
Do not bundle rules to save bytes
Only the index and the digest load on their own. A universal rule folded behind one pointer with two others, or moved into a topic file, stops firing without any error. The one safe move is a domain rule into the skill that owns it.

09 · Implications

The ledger

What a file-based memory buys and what it costs, for anyone building one.

Gains

Lossless closing
A session can end at any boundary. The digest, the notes and the transcript hold everything a resume would have held, at a fraction of the cost of carrying the context.
Inspectable
Every fact is a line in a file a person can read, diff and delete. There is no embedding to explain and no ranking to second-guess.
Machine-independent
The whole system is a directory. It syncs between two machines and runs on a phone through a remote session on the smaller one.
Honest numbers
Budgets are bytes counted with a shell command. Density is a regular expression anyone can run. A red figure stays red in the file until it is fixed.

Costs

Keyword recall misses synonyms
A memory file is found by the words in its description, so a description written as a summary rather than as the phrase the user would type is a file that never surfaces.
Discipline is the enforcement
The only writer of the digest is a skill, and a skill is invoked by an assistant under time pressure. The admission test and the red-flags list exist because the assistant negotiates with soft guidance.
Drift
Counts in guides go stale within weeks. The guide says so in its own summary and points at the digest as the figure of record.
Two red numbers today
On the day of counting both always-on files were over their lines. The system knows; the rebalance is owed.

10 · Sources

Sources and caveats

Set on 18 October 2026 from the checkpoint skill and the operating guide as rewritten that day, with every count taken on the machine the system runs on. The mechanism is current; counts drift and the status digest is the figure of record. Nothing in this dossier names a client, an employer or a private address; paths are the author's own home directory.

  1. 1Anthropic Claude Code documentation: memory (CLAUDE.md and imports), hooks, sessions and resume, sub-agents, the context window. code.claude.com/docs/en/memory
  2. 2Anthropic Claude Code: hooks reference (SessionStart, PreToolUse, PostCompact). code.claude.com/docs/en/hooks
  3. 3Anthropic Claude Code: sub-agents. code.claude.com/docs/en/sub-agents
  4. 4Anthropic Prompt caching: the cache window and what a cache miss costs. platform.claude.com/docs/en/build-with-claude/prompt-caching
  5. 5T. Forte The PARA method: Projects, Areas, Resources, Archives. fortelabs.com/blog/para/
  6. 6S. Ahrens How to Take Smart Notes (the Zettelkasten method). 2017. www.soenkeahrens.de/en/takesmartnotes
  7. 7Obsidian Obsidian: local Markdown notes, wiki-links and frontmatter. obsidian.md
  8. 8Syncthing Syncthing: continuous file synchronisation between the author's two machines. syncthing.net
  9. 9N. F. Liu et al. "Lost in the Middle: How Language Models Use Long Contexts." TACL 2024. Why every always-on line is an attention tax. arxiv.org/abs/2307.03172
  10. 10The author The checkpoint skill (update-codex) and the operating guide, in the author's configuration directory and notes; rewritten 18 October 2026. Not public. www.testinginproduction.co/dossier/the-base