iris/ai-app
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Trim what every session pays to load

Browse Source 
Measured: an app-spawned session starts at ~33,200 tokens of context, of
which ~20,000 is written fresh on every spawn -- the always-loaded rule
files and this file -- and only ~13,200 comes from a shared cache. That
20,000 is billed at 1.25x on every single session start.

This file drops to 19,882 bytes from 21,293. What went is narrative that
PLAN.md already carries in more detail (the phase history, the submodule
drift story) and the parts of "Where things run" that MACHINE.md states
once for every project. What stayed is every operational fact: the
commands, the llama.cpp and ssh test recipes, the import rules, and
everything under "Things that have bitten".

The global chain was trimmed in the same pass, 43,039 -> 34,069 bytes,
mostly by moving the Gentoo host build profile out of the @import chain
into ~/.claude/HOST_BUILD.md, which MACHINE.md now points at. Nothing was
deleted there either; it is referenced rather than loaded, the same
arrangement this file has with PLAN.md.

Worth being honest about the size of the win: ~10,400 bytes is roughly
2,200 tokens off each session start. It is real and permanent, but it is
not what makes a long session expensive.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VETa8afmpWaYezLCqJhDB8
This commit is contained in:
irisandClaude Opus 5 committed 2026-08-29 19:00:59 -04:00
1 parent 749b2db287
commit 620a7d0a83
1 file changed
+61 -84
Download Patch File Download Diff File Expand all files Collapse all files
AGENTS.md
+61 -84
View File
@@ -37,26 +37,21 @@ repo is in PLAN.md's "Backend layout" section.
same path with the token written up front rather than a second way to
start a session. The phone picks an **id**, never a path: the server
resolves which file that is, so an enrolled token cannot become "read me
an arbitrary file", the same rule that keeps a command out of
an arbitrary file" — the same rule that keeps a command out of
`POST /setups`. Only the tail is replayed (`REPLAY_LINES`) because these
files reach tens of megabytes and the CLI reads the real one itself; what
crosses the tunnel is what a person reads, not what the model is given.
Images in the replayed tail are written into the session's `files/` by
the same function the live translator uses, and referenced -- so a
screenshot looks the same whether it was watched happening or replayed
afterwards, the phone fetches the bytes only when it draws one, and none
of it goes anywhere near the CLI, which reads its own file.
the same function the live translator uses, so a screenshot looks the
same whether it was watched happening or replayed afterwards, and the
phone fetches the bytes only when it draws one.
An imported session then **keeps itself level with that file**, so work
done at a terminal appears without anyone pressing anything. `--resume`
appends to the same transcript rather than forking — measured, but
**against a session nothing else had open**; against a live one it
duplicates the whole conversation into the file, which is why an open
session cannot be imported at all (see below). So the
only hard question is which new lines came from *here* — answered by
counting the events this session has recorded, not by looking at its
status. Status is the obvious signal and is wrong: a turn that starts and
finishes between two polls reads as idle at both, and its own output gets
replayed on top of itself. That bug was visible on screen as `donedone`.
done at a terminal appears without anyone pressing anything. Which new
lines came from *here* is answered by counting the events this session
has recorded, **not** by looking at its status — a turn that starts and
finishes between two polls reads as idle at both, and its own output
gets replayed on top of itself. That bug was visible on screen as
`donedone`.
- `server/src/usage.rs` — rate-limit windows, asked **of each machine that
can run Claude**, not of the backend. Credentials are read through the
session `Transport`, so a remote setup is an ssh round trip and the local
@@ -101,62 +96,55 @@ repo is in PLAN.md's "Backend layout" section.
dev-updater also needed: the pinned CA and leaf (`certs`), QR enrollment
and the bearer token (`enroll`), wg0 binding and the certificate's SANs
(`netif`), owner-only files (`private`), and the RON house rules
(`format`). Both projects had written all five, and they had drifted;
see that repo's `README.md` for the diff that decided each one and the
permissions bug the extraction found. Clone with
(`format`). Both projects had written all five and they had drifted; see
that repo's `README.md` for the diff that decided each one. Clone with
`git clone --recurse-submodules`, or `git submodule update --init` in an
existing checkout — `server/` will not build without it, since it is a
path dependency rather than a registry one, which is what keeps the two
projects version-locked to the commit this repo pins.
The certificates are still the one-way door: the CA is generated once on
first start into `$XDG_CONFIG_HOME/ai-app/certs` and regenerating it
strands the installed app.
The certificates are the one-way door: the CA is generated once on first
start into `$XDG_CONFIG_HOME/ai-app/certs` and regenerating it strands
the installed app.
What deliberately did **not** move is the API surface and the config
*schema* — routes, drivers, sessions and setups are what makes this
project itself, and a shared vocabulary for them would be inventing one
neither project wants.
project itself.
## Status
Phases 1–3 done 2026-08-24 (see PLAN.md's phase list for what each
verified): the skeleton pipe, the full Claude driver (streaming, tools,
permission + AskUserQuestion cards, steering, interrupt, `--resume`
crash recovery, images both ways), and the usage screen.
Phases 1–3 done 2026-08-24 (PLAN.md's phase list says what each verified):
the skeleton pipe, the full Claude driver (streaming, tools, permission +
AskUserQuestion cards, steering, interrupt, `--resume` crash recovery,
images both ways), and the usage screen.
**Phase 5 (SSH) is written and now exercised** (2026-08-28). A session
names a host, `session::transport` turns that into an `ssh host …`
invocation, and the driver never learns which it got.
**Phase 5 (SSH)** is written and exercised (2026-08-28): a session names a
host, `session::transport` turns that into an `ssh host …` invocation, and
the driver never learns which it got.
**Phase 4 (llama.cpp) works end to end, phone included** (2026-08-28). Models are
browsed and downloaded from HuggingFace (`models.rs`, resumable and
verified), and `session::llama` runs one through `llama-server`, talking
to its OpenAI-compatible streaming endpoint. Two things about it are
deliberate and easy to undo by accident: the conversation is rebuilt from
the **transcript** rather than kept in the driver, because driver memory
is invisible to a second device; and a llama session is refused on an ssh
**Phase 4 (llama.cpp)** works end to end, phone included (2026-08-28).
Models are browsed and downloaded from HuggingFace (`models.rs`, resumable
and verified), and `session::llama` runs one through `llama-server` over
its OpenAI-compatible streaming endpoint. Two things are deliberate and
easy to undo by accident: the conversation is rebuilt from the
**transcript** rather than kept in the driver, because driver memory is
invisible to a second device; and a llama session is refused on an ssh
host, because the model is reached over HTTP and forwarding that port is
not built.
Models are browsed, downloaded and deleted from the app, and a llama
session picks one of them plus a context size and temperature. Setups —
machines, each carrying what it can run — are added, renamed, re-probed
and removed from the app too; providers are **discovered by asking the
machine**, never typed, so the enrolled token cannot introduce a command.
What is left is real-phone/WireGuard bring-up, which is operational rather
than code.
Setups — machines, each carrying what it can run — are added, renamed,
re-probed and removed from the app; providers are **discovered by asking
the machine**, never typed, so the enrolled token cannot introduce a
command. What is left is real-phone/WireGuard bring-up, which is
operational rather than code.
**`command -v` follows PATH under a non-interactive ssh session**, which is
not the PATH a login shell shows, so a binary somewhere unusual is
invisible to discovery — llama.cpp unpacked into `~/.local/opt` needs a
symlink into `~/.local/bin` before a setup finds it. The escape hatch for
anything odder is editing `config.ron` on the backend, which is
deliberately the one authority the phone does not have.
anything odder is editing `config.ron` on the backend, deliberately the one
authority the phone does not have.
**Testing llama.cpp here:** the prebuilt CPU build lives outside the repo
at `~/.local/opt/llama.cpp` (the 15 MB `ubuntu-x64` release asset — no
compiling, and it runs fine on Arch). It needs its own directory on
`LD_LIBRARY_PATH`, so start the server as
at `~/.local/opt/llama.cpp` (the 15 MB `ubuntu-x64` release asset). It
needs its own directory on `LD_LIBRARY_PATH`, so start the server as
`LD_LIBRARY_PATH=~/.local/opt/llama.cpp ai-server …` and point a provider's
`command` at `~/.local/opt/llama.cpp/llama-server`. A 0.6B Q8_0 answers at
usable speed on this VM's 8 cores. **Do not test with a 2-bit quant**: the
@@ -172,18 +160,11 @@ itself. Generate a throwaway key, append the public half to
touches nothing real. Point a provider's `command` at something harmless
like `/bin/echo` rather than at `claude`: the transport is what is under
test, the process exiting immediately is the signal, and it costs no
tokens. A session spawned on that host logs `running /bin/echo on loopback
(bob@127.0.0.1)` and lands `status: exited` in its transcript, which is
the whole path — connection, remote exec, process death reported. **Take
the key back out afterwards**; this VM's `authorized_keys` is not scratch
space.
Note the remote login shell here is **fish**, not a POSIX shell. The
remote script (`cd '…' && exec '…'`) happens to be valid in both, and the
POSIX single-quote escaping `ssh.rs` does happens to mean the same thing
in fish — but that is luck rather than design, and a shell that isn't
either would be the thing to suspect first if a remote spawn ever
mangles an argument.
tokens. **Take the key back out afterwards.** Note the remote login shell
here is **fish**; the remote script (`cd '…' && exec '…'`) and `ssh.rs`'s
POSIX quoting happen to mean the same thing in both, but that is luck
rather than design, and a shell that isn't either is the thing to suspect
first if a remote spawn ever mangles an argument.
## Checking your work
@@ -222,10 +203,9 @@ mangles an argument.
## Where things run (host vs this VM)
Established 2026-08-25, and it decides more than it looks like. The machine
itself — the two boxes, the shared `~/repos` mount, gitea, and why the VM is
treated as untrusted — is described once in `~/.claude/MACHINE.md`; what
follows is only what that means for **this** project.
Established 2026-08-25. The machine itself — the two boxes, the shared
`~/repos` mount, and why the VM is untrusted — is described once in
`~/.claude/MACHINE.md`; what follows is only what that means here.
- **`ai-server` belongs on the host in production.** That is where the LAN
address the phone can reach is, and where WireGuard terminates.
@@ -234,33 +214,30 @@ follows is only what that means for **this** project.
- **The tunnel and the real phone can never terminate in the VM**, because
nothing outside can open a connection into it. Phone bring-up is host
work.
- `wg0` (10.66.0.1) now exists in this VM too, so the production path —
- `wg0` (10.66.0.1) exists in this VM too, so the production path —
`ai-server` with no `--bind` — is exercisable during development. It has
no reachable peer and doesn't need one; the interface existing is what
the server requires. Consequence: **with no `--bind`, the emulator can't
reach the server** (it dials 10.0.2.2), so keep using
no reachable peer and doesn't need one. Consequence: **with no `--bind`
the emulator can't reach the server** (it dials 10.0.2.2), so keep using
`--bind 127.0.0.1` for app work.
- `./test-wg-tunnel.sh up|test|down` builds a real tunnel between two
network namespaces inside one machine and drives the server through it
— a genuine handshake against 10.66.0.1 with pinned TLS, no router or
phone involved. That's the way to verify the wg0-only posture.
network namespaces inside one machine and drives the server through it —
a genuine handshake against 10.66.0.1 with pinned TLS, no router or
phone involved. That's how to verify the wg0-only posture.
- **The `claude` CLI is only in the VM, so from the host it is a remote.**
The backend reaches it the way it would any other machine: a configured
host, and a session that names it. For the host to ssh in, the VM needs
an inbound port forward it does not have by default.
The backend reaches it as it would any other machine: a configured host,
and a session that names it.
- **Nothing secret goes in the repo**, which is shared with the host and
attacker-writable under this project's threat model (PLAN.md's security
section). So state lives outside it: `$XDG_CONFIG_HOME/ai-app/config.ron`
section). State lives outside it: `$XDG_CONFIG_HOME/ai-app/config.ron`
and `certs/`, `$XDG_DATA_HOME/ai-app/sessions/`, owner-only.
- Certificates are generated **by the server, on first start**, into
`$XDG_CONFIG_HOME/ai-app/certs` (`--certs` overrides). The CA is created
once and then left alone; the leaf is reissued every start, so covering a
new address is a restart. Starting the server in the VM therefore makes a
separate throwaway dev CA for emulator work — never install a build
pinning that on the real phone.
once and left alone; the leaf is reissued every start, so covering a new
address is a restart. Starting the server in the VM therefore makes a
separate throwaway dev CA — never install a build pinning that on the
real phone.
- Point development at a scratch state directory rather than the real one:
`--config /tmp/…/config.ron --data-dir /tmp/…/sessions --port 8444`, or
`XDG_CONFIG_HOME=… XDG_DATA_HOME=…`.
`--config /tmp/…/config.ron --data-dir /tmp/…/sessions --port 8444`.
## Sessions outlive the backend