Compact at 100k rather than letting a phone session drift

Left at `auto` the CLI picks a very large window, which suits a terminal
session somebody closes at the end of the day and does not suit this app
at all: these run for hours, nobody closes them, and the transcript
carries screenshots. One session here reached 491,562 tokens of context
before the CLI compacted it.

That matters because every API call re-reads the whole context, and one
request is not one call. At half a million tokens a single tool call
bills about 49k before it does anything, so "can you make it so you can
rename a session?" cost 4.2 million tokens across the 130 calls it took.
Measured over that session's life: 2,498 calls, 1.08 billion cache-read
tokens.

100k is the smallest window the CLI accepts and roughly the cheapest.
Per-call cost falls with the cap, while the compaction it forces costs
about the same in total either way -- a smaller window compacts more
often, but each pass is proportionally smaller. What it trades is how
much detail survives a compaction, which is a real cost to the work and
the reason this is one named constant with the reasoning written down
rather than a computed value.

Passed before the resume/name branch, so it applies to adopted and
imported sessions too -- which are the large ones, and the ones this is
for.

Verified: the exact argument list the app now spawns starts, accepts an
empty stream-json stdin and exits 0, so the flag combination is good
without spending a token. 68 tests, clippy clean, rustfmt clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VETa8afmpWaYezLCqJhDB8
This commit is contained in:
irisandClaude Opus 5 committed 2026-08-29 19:09:27 -04:00
1 parent 620a7d0a83
commit 4a4057886d
1 file changed
+20
+20
View File
@@ -172,6 +172,25 @@ impl Queue {
/// the attach path must agree about which file is which; if they drift,
/// a reattached session reads a file nothing is writing and simply looks
/// idle forever.
/// The context size the CLI compacts at, rather than letting it choose.
///
/// Left to `auto` a phone session drifts: these run for hours, nobody
/// closes them, and the transcript carries screenshots. One measured here
/// reached 491,562 tokens before the CLI compacted it, and *every* API
/// call re-reads the whole context -- so at that size a single tool call
/// bills about 49k tokens before it does anything, and one ordinary
/// request ("can you make it so you can rename a session?") cost 4.2
/// million across its 130 calls.
///
/// 100k is the smallest window the CLI accepts and roughly the cheapest:
/// per-call cost falls with the cap, while the compaction it forces costs
/// about the same in total either way -- a smaller window compacts more
/// often but each pass is proportionally smaller. What it trades is how
/// much detail survives a compaction, which is a real cost to the work
/// and the reason this is one named constant rather than a computed
/// value. Raise it if sessions start losing the thread.
const AUTOCOMPACT_WINDOW: &str = "100k";
const STDIN_FIFO: &str = "stdin.fifo";
const STDOUT_LOG: &str = "stdout.log";
const STDERR_LOG: &str = "stderr.log";
@@ -327,6 +346,7 @@ impl ClaudeDriver {
};
push("--input-format", "stream-json");
push("--output-format", "stream-json");
push("--autocompact", AUTOCOMPACT_WINDOW);
// Hidden but load-bearing: without it the CLI resolves permissions
// itself and nothing ever reaches the phone.
push("--permission-prompt-tool", "stdio");