Trim what every session pays to load
Measured: an app-spawned session starts at ~33,200 tokens of context, of which ~20,000 is written fresh on every spawn -- the always-loaded rule files and this file -- and only ~13,200 comes from a shared cache. That 20,000 is billed at 1.25x on every single session start. This file drops to 19,882 bytes from 21,293. What went is narrative that PLAN.md already carries in more detail (the phase history, the submodule drift story) and the parts of "Where things run" that MACHINE.md states once for every project. What stayed is every operational fact: the commands, the llama.cpp and ssh test recipes, the import rules, and everything under "Things that have bitten". The global chain was trimmed in the same pass, 43,039 -> 34,069 bytes, mostly by moving the Gentoo host build profile out of the @import chain into ~/.claude/HOST_BUILD.md, which MACHINE.md now points at. Nothing was deleted there either; it is referenced rather than loaded, the same arrangement this file has with PLAN.md. Worth being honest about the size of the win: ~10,400 bytes is roughly 2,200 tokens off each session start. It is real and permanent, but it is not what makes a long session expensive. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VETa8afmpWaYezLCqJhDB8
This commit is contained in:
1 parent
749b2db287
commit
620a7d0a83
1 file changed
+61
-84
@@ -37,26 +37,21 @@ repo is in PLAN.md's "Backend layout" section.
|
|||||||
same path with the token written up front rather than a second way to
|
same path with the token written up front rather than a second way to
|
||||||
start a session. The phone picks an **id**, never a path: the server
|
start a session. The phone picks an **id**, never a path: the server
|
||||||
resolves which file that is, so an enrolled token cannot become "read me
|
resolves which file that is, so an enrolled token cannot become "read me
|
||||||
an arbitrary file", the same rule that keeps a command out of
|
an arbitrary file" — the same rule that keeps a command out of
|
||||||
`POST /setups`. Only the tail is replayed (`REPLAY_LINES`) because these
|
`POST /setups`. Only the tail is replayed (`REPLAY_LINES`) because these
|
||||||
files reach tens of megabytes and the CLI reads the real one itself; what
|
files reach tens of megabytes and the CLI reads the real one itself; what
|
||||||
crosses the tunnel is what a person reads, not what the model is given.
|
crosses the tunnel is what a person reads, not what the model is given.
|
||||||
Images in the replayed tail are written into the session's `files/` by
|
Images in the replayed tail are written into the session's `files/` by
|
||||||
the same function the live translator uses, and referenced -- so a
|
the same function the live translator uses, so a screenshot looks the
|
||||||
screenshot looks the same whether it was watched happening or replayed
|
same whether it was watched happening or replayed afterwards, and the
|
||||||
afterwards, the phone fetches the bytes only when it draws one, and none
|
phone fetches the bytes only when it draws one.
|
||||||
of it goes anywhere near the CLI, which reads its own file.
|
|
||||||
An imported session then **keeps itself level with that file**, so work
|
An imported session then **keeps itself level with that file**, so work
|
||||||
done at a terminal appears without anyone pressing anything. `--resume`
|
done at a terminal appears without anyone pressing anything. Which new
|
||||||
appends to the same transcript rather than forking — measured, but
|
lines came from *here* is answered by counting the events this session
|
||||||
**against a session nothing else had open**; against a live one it
|
has recorded, **not** by looking at its status — a turn that starts and
|
||||||
duplicates the whole conversation into the file, which is why an open
|
finishes between two polls reads as idle at both, and its own output
|
||||||
session cannot be imported at all (see below). So the
|
gets replayed on top of itself. That bug was visible on screen as
|
||||||
only hard question is which new lines came from *here* — answered by
|
`donedone`.
|
||||||
counting the events this session has recorded, not by looking at its
|
|
||||||
status. Status is the obvious signal and is wrong: a turn that starts and
|
|
||||||
finishes between two polls reads as idle at both, and its own output gets
|
|
||||||
replayed on top of itself. That bug was visible on screen as `donedone`.
|
|
||||||
- `server/src/usage.rs` — rate-limit windows, asked **of each machine that
|
- `server/src/usage.rs` — rate-limit windows, asked **of each machine that
|
||||||
can run Claude**, not of the backend. Credentials are read through the
|
can run Claude**, not of the backend. Credentials are read through the
|
||||||
session `Transport`, so a remote setup is an ssh round trip and the local
|
session `Transport`, so a remote setup is an ssh round trip and the local
|
||||||
@@ -101,62 +96,55 @@ repo is in PLAN.md's "Backend layout" section.
|
|||||||
dev-updater also needed: the pinned CA and leaf (`certs`), QR enrollment
|
dev-updater also needed: the pinned CA and leaf (`certs`), QR enrollment
|
||||||
and the bearer token (`enroll`), wg0 binding and the certificate's SANs
|
and the bearer token (`enroll`), wg0 binding and the certificate's SANs
|
||||||
(`netif`), owner-only files (`private`), and the RON house rules
|
(`netif`), owner-only files (`private`), and the RON house rules
|
||||||
(`format`). Both projects had written all five, and they had drifted;
|
(`format`). Both projects had written all five and they had drifted; see
|
||||||
see that repo's `README.md` for the diff that decided each one and the
|
that repo's `README.md` for the diff that decided each one. Clone with
|
||||||
permissions bug the extraction found. Clone with
|
|
||||||
`git clone --recurse-submodules`, or `git submodule update --init` in an
|
`git clone --recurse-submodules`, or `git submodule update --init` in an
|
||||||
existing checkout — `server/` will not build without it, since it is a
|
existing checkout — `server/` will not build without it, since it is a
|
||||||
path dependency rather than a registry one, which is what keeps the two
|
path dependency rather than a registry one, which is what keeps the two
|
||||||
projects version-locked to the commit this repo pins.
|
projects version-locked to the commit this repo pins.
|
||||||
The certificates are still the one-way door: the CA is generated once on
|
The certificates are the one-way door: the CA is generated once on first
|
||||||
first start into `$XDG_CONFIG_HOME/ai-app/certs` and regenerating it
|
start into `$XDG_CONFIG_HOME/ai-app/certs` and regenerating it strands
|
||||||
strands the installed app.
|
the installed app.
|
||||||
What deliberately did **not** move is the API surface and the config
|
What deliberately did **not** move is the API surface and the config
|
||||||
*schema* — routes, drivers, sessions and setups are what makes this
|
*schema* — routes, drivers, sessions and setups are what makes this
|
||||||
project itself, and a shared vocabulary for them would be inventing one
|
project itself.
|
||||||
neither project wants.
|
|
||||||
|
|
||||||
## Status
|
## Status
|
||||||
|
|
||||||
Phases 1–3 done 2026-08-24 (see PLAN.md's phase list for what each
|
Phases 1–3 done 2026-08-24 (PLAN.md's phase list says what each verified):
|
||||||
verified): the skeleton pipe, the full Claude driver (streaming, tools,
|
the skeleton pipe, the full Claude driver (streaming, tools, permission +
|
||||||
permission + AskUserQuestion cards, steering, interrupt, `--resume`
|
AskUserQuestion cards, steering, interrupt, `--resume` crash recovery,
|
||||||
crash recovery, images both ways), and the usage screen.
|
images both ways), and the usage screen.
|
||||||
|
|
||||||
**Phase 5 (SSH) is written and now exercised** (2026-08-28). A session
|
**Phase 5 (SSH)** is written and exercised (2026-08-28): a session names a
|
||||||
names a host, `session::transport` turns that into an `ssh host …`
|
host, `session::transport` turns that into an `ssh host …` invocation, and
|
||||||
invocation, and the driver never learns which it got.
|
the driver never learns which it got.
|
||||||
|
|
||||||
**Phase 4 (llama.cpp) works end to end, phone included** (2026-08-28). Models are
|
**Phase 4 (llama.cpp)** works end to end, phone included (2026-08-28).
|
||||||
browsed and downloaded from HuggingFace (`models.rs`, resumable and
|
Models are browsed and downloaded from HuggingFace (`models.rs`, resumable
|
||||||
verified), and `session::llama` runs one through `llama-server`, talking
|
and verified), and `session::llama` runs one through `llama-server` over
|
||||||
to its OpenAI-compatible streaming endpoint. Two things about it are
|
its OpenAI-compatible streaming endpoint. Two things are deliberate and
|
||||||
deliberate and easy to undo by accident: the conversation is rebuilt from
|
easy to undo by accident: the conversation is rebuilt from the
|
||||||
the **transcript** rather than kept in the driver, because driver memory
|
**transcript** rather than kept in the driver, because driver memory is
|
||||||
is invisible to a second device; and a llama session is refused on an ssh
|
invisible to a second device; and a llama session is refused on an ssh
|
||||||
host, because the model is reached over HTTP and forwarding that port is
|
host, because the model is reached over HTTP and forwarding that port is
|
||||||
not built.
|
not built.
|
||||||
|
|
||||||
Models are browsed, downloaded and deleted from the app, and a llama
|
Setups — machines, each carrying what it can run — are added, renamed,
|
||||||
session picks one of them plus a context size and temperature. Setups —
|
re-probed and removed from the app; providers are **discovered by asking
|
||||||
machines, each carrying what it can run — are added, renamed, re-probed
|
the machine**, never typed, so the enrolled token cannot introduce a
|
||||||
and removed from the app too; providers are **discovered by asking the
|
command. What is left is real-phone/WireGuard bring-up, which is
|
||||||
machine**, never typed, so the enrolled token cannot introduce a command.
|
operational rather than code.
|
||||||
|
|
||||||
What is left is real-phone/WireGuard bring-up, which is operational rather
|
|
||||||
than code.
|
|
||||||
|
|
||||||
**`command -v` follows PATH under a non-interactive ssh session**, which is
|
**`command -v` follows PATH under a non-interactive ssh session**, which is
|
||||||
not the PATH a login shell shows, so a binary somewhere unusual is
|
not the PATH a login shell shows, so a binary somewhere unusual is
|
||||||
invisible to discovery — llama.cpp unpacked into `~/.local/opt` needs a
|
invisible to discovery — llama.cpp unpacked into `~/.local/opt` needs a
|
||||||
symlink into `~/.local/bin` before a setup finds it. The escape hatch for
|
symlink into `~/.local/bin` before a setup finds it. The escape hatch for
|
||||||
anything odder is editing `config.ron` on the backend, which is
|
anything odder is editing `config.ron` on the backend, deliberately the one
|
||||||
deliberately the one authority the phone does not have.
|
authority the phone does not have.
|
||||||
|
|
||||||
**Testing llama.cpp here:** the prebuilt CPU build lives outside the repo
|
**Testing llama.cpp here:** the prebuilt CPU build lives outside the repo
|
||||||
at `~/.local/opt/llama.cpp` (the 15 MB `ubuntu-x64` release asset — no
|
at `~/.local/opt/llama.cpp` (the 15 MB `ubuntu-x64` release asset). It
|
||||||
compiling, and it runs fine on Arch). It needs its own directory on
|
needs its own directory on `LD_LIBRARY_PATH`, so start the server as
|
||||||
`LD_LIBRARY_PATH`, so start the server as
|
|
||||||
`LD_LIBRARY_PATH=~/.local/opt/llama.cpp ai-server …` and point a provider's
|
`LD_LIBRARY_PATH=~/.local/opt/llama.cpp ai-server …` and point a provider's
|
||||||
`command` at `~/.local/opt/llama.cpp/llama-server`. A 0.6B Q8_0 answers at
|
`command` at `~/.local/opt/llama.cpp/llama-server`. A 0.6B Q8_0 answers at
|
||||||
usable speed on this VM's 8 cores. **Do not test with a 2-bit quant**: the
|
usable speed on this VM's 8 cores. **Do not test with a 2-bit quant**: the
|
||||||
@@ -172,18 +160,11 @@ itself. Generate a throwaway key, append the public half to
|
|||||||
touches nothing real. Point a provider's `command` at something harmless
|
touches nothing real. Point a provider's `command` at something harmless
|
||||||
like `/bin/echo` rather than at `claude`: the transport is what is under
|
like `/bin/echo` rather than at `claude`: the transport is what is under
|
||||||
test, the process exiting immediately is the signal, and it costs no
|
test, the process exiting immediately is the signal, and it costs no
|
||||||
tokens. A session spawned on that host logs `running /bin/echo on loopback
|
tokens. **Take the key back out afterwards.** Note the remote login shell
|
||||||
(bob@127.0.0.1)` and lands `status: exited` in its transcript, which is
|
here is **fish**; the remote script (`cd '…' && exec '…'`) and `ssh.rs`'s
|
||||||
the whole path — connection, remote exec, process death reported. **Take
|
POSIX quoting happen to mean the same thing in both, but that is luck
|
||||||
the key back out afterwards**; this VM's `authorized_keys` is not scratch
|
rather than design, and a shell that isn't either is the thing to suspect
|
||||||
space.
|
first if a remote spawn ever mangles an argument.
|
||||||
|
|
||||||
Note the remote login shell here is **fish**, not a POSIX shell. The
|
|
||||||
remote script (`cd '…' && exec '…'`) happens to be valid in both, and the
|
|
||||||
POSIX single-quote escaping `ssh.rs` does happens to mean the same thing
|
|
||||||
in fish — but that is luck rather than design, and a shell that isn't
|
|
||||||
either would be the thing to suspect first if a remote spawn ever
|
|
||||||
mangles an argument.
|
|
||||||
|
|
||||||
## Checking your work
|
## Checking your work
|
||||||
|
|
||||||
@@ -222,10 +203,9 @@ mangles an argument.
|
|||||||
|
|
||||||
## Where things run (host vs this VM)
|
## Where things run (host vs this VM)
|
||||||
|
|
||||||
Established 2026-08-25, and it decides more than it looks like. The machine
|
Established 2026-08-25. The machine itself — the two boxes, the shared
|
||||||
itself — the two boxes, the shared `~/repos` mount, gitea, and why the VM is
|
`~/repos` mount, and why the VM is untrusted — is described once in
|
||||||
treated as untrusted — is described once in `~/.claude/MACHINE.md`; what
|
`~/.claude/MACHINE.md`; what follows is only what that means here.
|
||||||
follows is only what that means for **this** project.
|
|
||||||
|
|
||||||
- **`ai-server` belongs on the host in production.** That is where the LAN
|
- **`ai-server` belongs on the host in production.** That is where the LAN
|
||||||
address the phone can reach is, and where WireGuard terminates.
|
address the phone can reach is, and where WireGuard terminates.
|
||||||
@@ -234,33 +214,30 @@ follows is only what that means for **this** project.
|
|||||||
- **The tunnel and the real phone can never terminate in the VM**, because
|
- **The tunnel and the real phone can never terminate in the VM**, because
|
||||||
nothing outside can open a connection into it. Phone bring-up is host
|
nothing outside can open a connection into it. Phone bring-up is host
|
||||||
work.
|
work.
|
||||||
- `wg0` (10.66.0.1) now exists in this VM too, so the production path —
|
- `wg0` (10.66.0.1) exists in this VM too, so the production path —
|
||||||
`ai-server` with no `--bind` — is exercisable during development. It has
|
`ai-server` with no `--bind` — is exercisable during development. It has
|
||||||
no reachable peer and doesn't need one; the interface existing is what
|
no reachable peer and doesn't need one. Consequence: **with no `--bind`
|
||||||
the server requires. Consequence: **with no `--bind`, the emulator can't
|
the emulator can't reach the server** (it dials 10.0.2.2), so keep using
|
||||||
reach the server** (it dials 10.0.2.2), so keep using
|
|
||||||
`--bind 127.0.0.1` for app work.
|
`--bind 127.0.0.1` for app work.
|
||||||
- `./test-wg-tunnel.sh up|test|down` builds a real tunnel between two
|
- `./test-wg-tunnel.sh up|test|down` builds a real tunnel between two
|
||||||
network namespaces inside one machine and drives the server through it
|
network namespaces inside one machine and drives the server through it —
|
||||||
— a genuine handshake against 10.66.0.1 with pinned TLS, no router or
|
a genuine handshake against 10.66.0.1 with pinned TLS, no router or
|
||||||
phone involved. That's the way to verify the wg0-only posture.
|
phone involved. That's how to verify the wg0-only posture.
|
||||||
- **The `claude` CLI is only in the VM, so from the host it is a remote.**
|
- **The `claude` CLI is only in the VM, so from the host it is a remote.**
|
||||||
The backend reaches it the way it would any other machine: a configured
|
The backend reaches it as it would any other machine: a configured host,
|
||||||
host, and a session that names it. For the host to ssh in, the VM needs
|
and a session that names it.
|
||||||
an inbound port forward it does not have by default.
|
|
||||||
- **Nothing secret goes in the repo**, which is shared with the host and
|
- **Nothing secret goes in the repo**, which is shared with the host and
|
||||||
attacker-writable under this project's threat model (PLAN.md's security
|
attacker-writable under this project's threat model (PLAN.md's security
|
||||||
section). So state lives outside it: `$XDG_CONFIG_HOME/ai-app/config.ron`
|
section). State lives outside it: `$XDG_CONFIG_HOME/ai-app/config.ron`
|
||||||
and `certs/`, `$XDG_DATA_HOME/ai-app/sessions/`, owner-only.
|
and `certs/`, `$XDG_DATA_HOME/ai-app/sessions/`, owner-only.
|
||||||
- Certificates are generated **by the server, on first start**, into
|
- Certificates are generated **by the server, on first start**, into
|
||||||
`$XDG_CONFIG_HOME/ai-app/certs` (`--certs` overrides). The CA is created
|
`$XDG_CONFIG_HOME/ai-app/certs` (`--certs` overrides). The CA is created
|
||||||
once and then left alone; the leaf is reissued every start, so covering a
|
once and left alone; the leaf is reissued every start, so covering a new
|
||||||
new address is a restart. Starting the server in the VM therefore makes a
|
address is a restart. Starting the server in the VM therefore makes a
|
||||||
separate throwaway dev CA for emulator work — never install a build
|
separate throwaway dev CA — never install a build pinning that on the
|
||||||
pinning that on the real phone.
|
real phone.
|
||||||
- Point development at a scratch state directory rather than the real one:
|
- Point development at a scratch state directory rather than the real one:
|
||||||
`--config /tmp/…/config.ron --data-dir /tmp/…/sessions --port 8444`, or
|
`--config /tmp/…/config.ron --data-dir /tmp/…/sessions --port 8444`.
|
||||||
`XDG_CONFIG_HOME=… XDG_DATA_HOME=…`.
|
|
||||||
|
|
||||||
## Sessions outlive the backend
|
## Sessions outlive the backend
|
||||||
|
|
||||||
|
|||||||
Reference in new issue
Block a user