Attribute the drift to the CLI's ceiling, not to a manual compaction
The reasoning on AUTOCOMPACT_WINDOW cited 491,562 tokens as where "the CLI compacted it". That was a manual /compact somebody ran, not the CLI's own trigger, so the comment credited a person's intervention to the automatic behaviour it was arguing about. Caught in review by the session whose transcript it was measured from. Corrected from that transcript's compaction boundaries: the window left to `auto` was 1M, and the one automatic compaction fired at preTokens 1,000,184 with the API context peaking at 999,668. So the drift ceiling is twice what the comment said, and near it a single tool call bills about 100k tokens rather than 49k. The correction strengthens the case, but it also changes what the example is evidence *of*, which is why it was worth fixing rather than just raising the number: what held that session together was the person in it running /compact by hand four times, and the 4.2-million-token request happened at the merely-large contexts left between those. The constant is for the sessions where nobody is doing that. No behaviour change. 68 tests, clippy clean, rustfmt clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VETa8afmpWaYezLCqJhDB8
This commit is contained in:
1 parent
36984a9e1b
commit
4f586d6551
1 file changed
+12
-6
@@ -179,12 +179,18 @@ const STDERR_LOG: &str = "stderr.log";
|
|||||||
/// The context size the CLI compacts at, rather than letting it choose.
|
/// The context size the CLI compacts at, rather than letting it choose.
|
||||||
///
|
///
|
||||||
/// Left to `auto` a phone session drifts: these run for hours, nobody
|
/// Left to `auto` a phone session drifts: these run for hours, nobody
|
||||||
/// closes them, and the transcript carries screenshots. One measured here
|
/// closes them, and the transcript carries screenshots. Measured on one
|
||||||
/// reached 491,562 tokens before the CLI compacted it, and *every* API
|
/// here, the window the CLI chose was 1M and it compacted at the ceiling
|
||||||
/// call re-reads the whole context -- so at that size a single tool call
|
/// -- its only automatic compaction fired at 1,000,184 tokens, and the
|
||||||
/// bills about 49k tokens before it does anything, and one ordinary
|
/// context peaked at 999,668. Every API call re-reads the whole context,
|
||||||
/// request ("can you make it so you can rename a session?") cost 4.2
|
/// so near that ceiling a single tool call bills about 100k tokens
|
||||||
/// million across its 130 calls.
|
/// before it does anything.
|
||||||
|
///
|
||||||
|
/// What kept that session usable was the person in it running `/compact`
|
||||||
|
/// by hand, four times. That is the state this constant is really for:
|
||||||
|
/// even between those manual compactions, at the merely-large contexts
|
||||||
|
/// they left behind, one ordinary request ("can you make it so you can
|
||||||
|
/// rename a session?") cost 4.2 million tokens across its 130 calls.
|
||||||
///
|
///
|
||||||
/// 100k is the smallest window the CLI accepts and roughly the cheapest:
|
/// 100k is the smallest window the CLI accepts and roughly the cheapest:
|
||||||
/// per-call cost falls with the cap, while the compaction it forces costs
|
/// per-call cost falls with the cap, while the compaction it forces costs
|
||||||
|
|||||||
Reference in new issue
Block a user