Trim what every session pays to load

Measured: an app-spawned session starts at ~33,200 tokens of context, of
which ~20,000 is written fresh on every spawn -- the always-loaded rule
files and this file -- and only ~13,200 comes from a shared cache. That
20,000 is billed at 1.25x on every single session start.

This file drops to 19,882 bytes from 21,293. What went is narrative that
PLAN.md already carries in more detail (the phase history, the submodule
drift story) and the parts of "Where things run" that MACHINE.md states
once for every project. What stayed is every operational fact: the
commands, the llama.cpp and ssh test recipes, the import rules, and
everything under "Things that have bitten".

The global chain was trimmed in the same pass, 43,039 -> 34,069 bytes,
mostly by moving the Gentoo host build profile out of the @import chain
into ~/.claude/HOST_BUILD.md, which MACHINE.md now points at. Nothing was
deleted there either; it is referenced rather than loaded, the same
arrangement this file has with PLAN.md.

Worth being honest about the size of the win: ~10,400 bytes is roughly
2,200 tokens off each session start. It is real and permanent, but it is
not what makes a long session expensive.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VETa8afmpWaYezLCqJhDB8
This commit is contained in:
irisandClaude Opus 5 committed 2026-08-29 19:00:59 -04:00
1 parent 749b2db287
commit 620a7d0a83
1 file changed
+61 -84
+61 -84
View File
@@ -37,26 +37,21 @@ repo is in PLAN.md's "Backend layout" section.
same path with the token written up front rather than a second way to same path with the token written up front rather than a second way to
start a session. The phone picks an **id**, never a path: the server start a session. The phone picks an **id**, never a path: the server
resolves which file that is, so an enrolled token cannot become "read me resolves which file that is, so an enrolled token cannot become "read me
an arbitrary file", the same rule that keeps a command out of an arbitrary file" the same rule that keeps a command out of
`POST /setups`. Only the tail is replayed (`REPLAY_LINES`) because these `POST /setups`. Only the tail is replayed (`REPLAY_LINES`) because these
files reach tens of megabytes and the CLI reads the real one itself; what files reach tens of megabytes and the CLI reads the real one itself; what
crosses the tunnel is what a person reads, not what the model is given. crosses the tunnel is what a person reads, not what the model is given.
Images in the replayed tail are written into the session's `files/` by Images in the replayed tail are written into the session's `files/` by
the same function the live translator uses, and referenced -- so a the same function the live translator uses, so a screenshot looks the
screenshot looks the same whether it was watched happening or replayed same whether it was watched happening or replayed afterwards, and the
afterwards, the phone fetches the bytes only when it draws one, and none phone fetches the bytes only when it draws one.
of it goes anywhere near the CLI, which reads its own file.
An imported session then **keeps itself level with that file**, so work An imported session then **keeps itself level with that file**, so work
done at a terminal appears without anyone pressing anything. `--resume` done at a terminal appears without anyone pressing anything. Which new
appends to the same transcript rather than forking — measured, but lines came from *here* is answered by counting the events this session
**against a session nothing else had open**; against a live one it has recorded, **not** by looking at its status — a turn that starts and
duplicates the whole conversation into the file, which is why an open finishes between two polls reads as idle at both, and its own output
session cannot be imported at all (see below). So the gets replayed on top of itself. That bug was visible on screen as
only hard question is which new lines came from *here* — answered by `donedone`.
counting the events this session has recorded, not by looking at its
status. Status is the obvious signal and is wrong: a turn that starts and
finishes between two polls reads as idle at both, and its own output gets
replayed on top of itself. That bug was visible on screen as `donedone`.
- `server/src/usage.rs` — rate-limit windows, asked **of each machine that - `server/src/usage.rs` — rate-limit windows, asked **of each machine that
can run Claude**, not of the backend. Credentials are read through the can run Claude**, not of the backend. Credentials are read through the
session `Transport`, so a remote setup is an ssh round trip and the local session `Transport`, so a remote setup is an ssh round trip and the local
@@ -101,62 +96,55 @@ repo is in PLAN.md's "Backend layout" section.
dev-updater also needed: the pinned CA and leaf (`certs`), QR enrollment dev-updater also needed: the pinned CA and leaf (`certs`), QR enrollment
and the bearer token (`enroll`), wg0 binding and the certificate's SANs and the bearer token (`enroll`), wg0 binding and the certificate's SANs
(`netif`), owner-only files (`private`), and the RON house rules (`netif`), owner-only files (`private`), and the RON house rules
(`format`). Both projects had written all five, and they had drifted; (`format`). Both projects had written all five and they had drifted; see
see that repo's `README.md` for the diff that decided each one and the that repo's `README.md` for the diff that decided each one. Clone with
permissions bug the extraction found. Clone with
`git clone --recurse-submodules`, or `git submodule update --init` in an `git clone --recurse-submodules`, or `git submodule update --init` in an
existing checkout — `server/` will not build without it, since it is a existing checkout — `server/` will not build without it, since it is a
path dependency rather than a registry one, which is what keeps the two path dependency rather than a registry one, which is what keeps the two
projects version-locked to the commit this repo pins. projects version-locked to the commit this repo pins.
The certificates are still the one-way door: the CA is generated once on The certificates are the one-way door: the CA is generated once on first
first start into `$XDG_CONFIG_HOME/ai-app/certs` and regenerating it start into `$XDG_CONFIG_HOME/ai-app/certs` and regenerating it strands
strands the installed app. the installed app.
What deliberately did **not** move is the API surface and the config What deliberately did **not** move is the API surface and the config
*schema* — routes, drivers, sessions and setups are what makes this *schema* — routes, drivers, sessions and setups are what makes this
project itself, and a shared vocabulary for them would be inventing one project itself.
neither project wants.
## Status ## Status
Phases 13 done 2026-08-24 (see PLAN.md's phase list for what each Phases 13 done 2026-08-24 (PLAN.md's phase list says what each verified):
verified): the skeleton pipe, the full Claude driver (streaming, tools, the skeleton pipe, the full Claude driver (streaming, tools, permission +
permission + AskUserQuestion cards, steering, interrupt, `--resume` AskUserQuestion cards, steering, interrupt, `--resume` crash recovery,
crash recovery, images both ways), and the usage screen. images both ways), and the usage screen.
**Phase 5 (SSH) is written and now exercised** (2026-08-28). A session **Phase 5 (SSH)** is written and exercised (2026-08-28): a session names a
names a host, `session::transport` turns that into an `ssh host …` host, `session::transport` turns that into an `ssh host …` invocation, and
invocation, and the driver never learns which it got. the driver never learns which it got.
**Phase 4 (llama.cpp) works end to end, phone included** (2026-08-28). Models are **Phase 4 (llama.cpp)** works end to end, phone included (2026-08-28).
browsed and downloaded from HuggingFace (`models.rs`, resumable and Models are browsed and downloaded from HuggingFace (`models.rs`, resumable
verified), and `session::llama` runs one through `llama-server`, talking and verified), and `session::llama` runs one through `llama-server` over
to its OpenAI-compatible streaming endpoint. Two things about it are its OpenAI-compatible streaming endpoint. Two things are deliberate and
deliberate and easy to undo by accident: the conversation is rebuilt from easy to undo by accident: the conversation is rebuilt from the
the **transcript** rather than kept in the driver, because driver memory **transcript** rather than kept in the driver, because driver memory is
is invisible to a second device; and a llama session is refused on an ssh invisible to a second device; and a llama session is refused on an ssh
host, because the model is reached over HTTP and forwarding that port is host, because the model is reached over HTTP and forwarding that port is
not built. not built.
Models are browsed, downloaded and deleted from the app, and a llama Setups — machines, each carrying what it can run — are added, renamed,
session picks one of them plus a context size and temperature. Setups — re-probed and removed from the app; providers are **discovered by asking
machines, each carrying what it can run — are added, renamed, re-probed the machine**, never typed, so the enrolled token cannot introduce a
and removed from the app too; providers are **discovered by asking the command. What is left is real-phone/WireGuard bring-up, which is
machine**, never typed, so the enrolled token cannot introduce a command. operational rather than code.
What is left is real-phone/WireGuard bring-up, which is operational rather
than code.
**`command -v` follows PATH under a non-interactive ssh session**, which is **`command -v` follows PATH under a non-interactive ssh session**, which is
not the PATH a login shell shows, so a binary somewhere unusual is not the PATH a login shell shows, so a binary somewhere unusual is
invisible to discovery — llama.cpp unpacked into `~/.local/opt` needs a invisible to discovery — llama.cpp unpacked into `~/.local/opt` needs a
symlink into `~/.local/bin` before a setup finds it. The escape hatch for symlink into `~/.local/bin` before a setup finds it. The escape hatch for
anything odder is editing `config.ron` on the backend, which is anything odder is editing `config.ron` on the backend, deliberately the one
deliberately the one authority the phone does not have. authority the phone does not have.
**Testing llama.cpp here:** the prebuilt CPU build lives outside the repo **Testing llama.cpp here:** the prebuilt CPU build lives outside the repo
at `~/.local/opt/llama.cpp` (the 15 MB `ubuntu-x64` release asset — no at `~/.local/opt/llama.cpp` (the 15 MB `ubuntu-x64` release asset). It
compiling, and it runs fine on Arch). It needs its own directory on needs its own directory on `LD_LIBRARY_PATH`, so start the server as
`LD_LIBRARY_PATH`, so start the server as
`LD_LIBRARY_PATH=~/.local/opt/llama.cpp ai-server …` and point a provider's `LD_LIBRARY_PATH=~/.local/opt/llama.cpp ai-server …` and point a provider's
`command` at `~/.local/opt/llama.cpp/llama-server`. A 0.6B Q8_0 answers at `command` at `~/.local/opt/llama.cpp/llama-server`. A 0.6B Q8_0 answers at
usable speed on this VM's 8 cores. **Do not test with a 2-bit quant**: the usable speed on this VM's 8 cores. **Do not test with a 2-bit quant**: the
@@ -172,18 +160,11 @@ itself. Generate a throwaway key, append the public half to
touches nothing real. Point a provider's `command` at something harmless touches nothing real. Point a provider's `command` at something harmless
like `/bin/echo` rather than at `claude`: the transport is what is under like `/bin/echo` rather than at `claude`: the transport is what is under
test, the process exiting immediately is the signal, and it costs no test, the process exiting immediately is the signal, and it costs no
tokens. A session spawned on that host logs `running /bin/echo on loopback tokens. **Take the key back out afterwards.** Note the remote login shell
(bob@127.0.0.1)` and lands `status: exited` in its transcript, which is here is **fish**; the remote script (`cd '…' && exec '…'`) and `ssh.rs`'s
the whole path — connection, remote exec, process death reported. **Take POSIX quoting happen to mean the same thing in both, but that is luck
the key back out afterwards**; this VM's `authorized_keys` is not scratch rather than design, and a shell that isn't either is the thing to suspect
space. first if a remote spawn ever mangles an argument.
Note the remote login shell here is **fish**, not a POSIX shell. The
remote script (`cd '…' && exec '…'`) happens to be valid in both, and the
POSIX single-quote escaping `ssh.rs` does happens to mean the same thing
in fish — but that is luck rather than design, and a shell that isn't
either would be the thing to suspect first if a remote spawn ever
mangles an argument.
## Checking your work ## Checking your work
@@ -222,10 +203,9 @@ mangles an argument.
## Where things run (host vs this VM) ## Where things run (host vs this VM)
Established 2026-08-25, and it decides more than it looks like. The machine Established 2026-08-25. The machine itself — the two boxes, the shared
itself — the two boxes, the shared `~/repos` mount, gitea, and why the VM is `~/repos` mount, and why the VM is untrusted — is described once in
treated as untrusted — is described once in `~/.claude/MACHINE.md`; what `~/.claude/MACHINE.md`; what follows is only what that means here.
follows is only what that means for **this** project.
- **`ai-server` belongs on the host in production.** That is where the LAN - **`ai-server` belongs on the host in production.** That is where the LAN
address the phone can reach is, and where WireGuard terminates. address the phone can reach is, and where WireGuard terminates.
@@ -234,33 +214,30 @@ follows is only what that means for **this** project.
- **The tunnel and the real phone can never terminate in the VM**, because - **The tunnel and the real phone can never terminate in the VM**, because
nothing outside can open a connection into it. Phone bring-up is host nothing outside can open a connection into it. Phone bring-up is host
work. work.
- `wg0` (10.66.0.1) now exists in this VM too, so the production path — - `wg0` (10.66.0.1) exists in this VM too, so the production path —
`ai-server` with no `--bind` — is exercisable during development. It has `ai-server` with no `--bind` — is exercisable during development. It has
no reachable peer and doesn't need one; the interface existing is what no reachable peer and doesn't need one. Consequence: **with no `--bind`
the server requires. Consequence: **with no `--bind`, the emulator can't the emulator can't reach the server** (it dials 10.0.2.2), so keep using
reach the server** (it dials 10.0.2.2), so keep using
`--bind 127.0.0.1` for app work. `--bind 127.0.0.1` for app work.
- `./test-wg-tunnel.sh up|test|down` builds a real tunnel between two - `./test-wg-tunnel.sh up|test|down` builds a real tunnel between two
network namespaces inside one machine and drives the server through it network namespaces inside one machine and drives the server through it
a genuine handshake against 10.66.0.1 with pinned TLS, no router or a genuine handshake against 10.66.0.1 with pinned TLS, no router or
phone involved. That's the way to verify the wg0-only posture. phone involved. That's how to verify the wg0-only posture.
- **The `claude` CLI is only in the VM, so from the host it is a remote.** - **The `claude` CLI is only in the VM, so from the host it is a remote.**
The backend reaches it the way it would any other machine: a configured The backend reaches it as it would any other machine: a configured host,
host, and a session that names it. For the host to ssh in, the VM needs and a session that names it.
an inbound port forward it does not have by default.
- **Nothing secret goes in the repo**, which is shared with the host and - **Nothing secret goes in the repo**, which is shared with the host and
attacker-writable under this project's threat model (PLAN.md's security attacker-writable under this project's threat model (PLAN.md's security
section). So state lives outside it: `$XDG_CONFIG_HOME/ai-app/config.ron` section). State lives outside it: `$XDG_CONFIG_HOME/ai-app/config.ron`
and `certs/`, `$XDG_DATA_HOME/ai-app/sessions/`, owner-only. and `certs/`, `$XDG_DATA_HOME/ai-app/sessions/`, owner-only.
- Certificates are generated **by the server, on first start**, into - Certificates are generated **by the server, on first start**, into
`$XDG_CONFIG_HOME/ai-app/certs` (`--certs` overrides). The CA is created `$XDG_CONFIG_HOME/ai-app/certs` (`--certs` overrides). The CA is created
once and then left alone; the leaf is reissued every start, so covering a once and left alone; the leaf is reissued every start, so covering a new
new address is a restart. Starting the server in the VM therefore makes a address is a restart. Starting the server in the VM therefore makes a
separate throwaway dev CA for emulator work — never install a build separate throwaway dev CA — never install a build pinning that on the
pinning that on the real phone. real phone.
- Point development at a scratch state directory rather than the real one: - Point development at a scratch state directory rather than the real one:
`--config /tmp/…/config.ron --data-dir /tmp/…/sessions --port 8444`, or `--config /tmp/…/config.ron --data-dir /tmp/…/sessions --port 8444`.
`XDG_CONFIG_HOME=… XDG_DATA_HOME=…`.
## Sessions outlive the backend ## Sessions outlive the backend