Compact at 100k rather than letting a phone session drift
Left at `auto` the CLI picks a very large window, which suits a terminal session somebody closes at the end of the day and does not suit this app at all: these run for hours, nobody closes them, and the transcript carries screenshots. One session here reached 491,562 tokens of context before the CLI compacted it. That matters because every API call re-reads the whole context, and one request is not one call. At half a million tokens a single tool call bills about 49k before it does anything, so "can you make it so you can rename a session?" cost 4.2 million tokens across the 130 calls it took. Measured over that session's life: 2,498 calls, 1.08 billion cache-read tokens. 100k is the smallest window the CLI accepts and roughly the cheapest. Per-call cost falls with the cap, while the compaction it forces costs about the same in total either way -- a smaller window compacts more often, but each pass is proportionally smaller. What it trades is how much detail survives a compaction, which is a real cost to the work and the reason this is one named constant with the reasoning written down rather than a computed value. Passed before the resume/name branch, so it applies to adopted and imported sessions too -- which are the large ones, and the ones this is for. Verified: the exact argument list the app now spawns starts, accepts an empty stream-json stdin and exits 0, so the flag combination is good without spending a token. 68 tests, clippy clean, rustfmt clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VETa8afmpWaYezLCqJhDB8
This commit is contained in:
1 parent
620a7d0a83
commit
4a4057886d
1 file changed
+20
@@ -172,6 +172,25 @@ impl Queue {
|
|||||||
/// the attach path must agree about which file is which; if they drift,
|
/// the attach path must agree about which file is which; if they drift,
|
||||||
/// a reattached session reads a file nothing is writing and simply looks
|
/// a reattached session reads a file nothing is writing and simply looks
|
||||||
/// idle forever.
|
/// idle forever.
|
||||||
|
/// The context size the CLI compacts at, rather than letting it choose.
|
||||||
|
///
|
||||||
|
/// Left to `auto` a phone session drifts: these run for hours, nobody
|
||||||
|
/// closes them, and the transcript carries screenshots. One measured here
|
||||||
|
/// reached 491,562 tokens before the CLI compacted it, and *every* API
|
||||||
|
/// call re-reads the whole context -- so at that size a single tool call
|
||||||
|
/// bills about 49k tokens before it does anything, and one ordinary
|
||||||
|
/// request ("can you make it so you can rename a session?") cost 4.2
|
||||||
|
/// million across its 130 calls.
|
||||||
|
///
|
||||||
|
/// 100k is the smallest window the CLI accepts and roughly the cheapest:
|
||||||
|
/// per-call cost falls with the cap, while the compaction it forces costs
|
||||||
|
/// about the same in total either way -- a smaller window compacts more
|
||||||
|
/// often but each pass is proportionally smaller. What it trades is how
|
||||||
|
/// much detail survives a compaction, which is a real cost to the work
|
||||||
|
/// and the reason this is one named constant rather than a computed
|
||||||
|
/// value. Raise it if sessions start losing the thread.
|
||||||
|
const AUTOCOMPACT_WINDOW: &str = "100k";
|
||||||
|
|
||||||
const STDIN_FIFO: &str = "stdin.fifo";
|
const STDIN_FIFO: &str = "stdin.fifo";
|
||||||
const STDOUT_LOG: &str = "stdout.log";
|
const STDOUT_LOG: &str = "stdout.log";
|
||||||
const STDERR_LOG: &str = "stderr.log";
|
const STDERR_LOG: &str = "stderr.log";
|
||||||
@@ -327,6 +346,7 @@ impl ClaudeDriver {
|
|||||||
};
|
};
|
||||||
push("--input-format", "stream-json");
|
push("--input-format", "stream-json");
|
||||||
push("--output-format", "stream-json");
|
push("--output-format", "stream-json");
|
||||||
|
push("--autocompact", AUTOCOMPACT_WINDOW);
|
||||||
// Hidden but load-bearing: without it the CLI resolves permissions
|
// Hidden but load-bearing: without it the CLI resolves permissions
|
||||||
// itself and nothing ever reaches the phone.
|
// itself and nothing ever reaches the phone.
|
||||||
push("--permission-prompt-tool", "stdio");
|
push("--permission-prompt-tool", "stdio");
|
||||||
|
|||||||
Reference in new issue
Block a user