Raise the compaction window to 200k
100k is the cheapest window on tokens and the wrong one to sit in front of. Measured on the session this was written against: context returns to 70-85k within ten calls of a compaction, so a 100k window compacts about every thirteen calls, and a compaction takes roughly two minutes (durationMs 104,346 to 147,671 across the six recorded). A 130-call request would have spent some twenty minutes compacting -- optimising the number that was asked about while making the thing somebody actually waits for on a phone considerably worse. 200k keeps most of the saving against the 1M ceiling and halves the stalls. The comment now also says what the window does not do, because measuring this turned up the opposite of what the byte counts suggested. Images are 93% of the bytes that tool calls put into that transcript but only 8% of the context growth -- the adb wrapper's downscaling holds a screenshot to a median of 476 tokens, while text-only calls add a median of 740 and a mean of 1,139. So the file is large because of screenshots and the context is large because of ordinary tool output, and only the second one is what this constant governs. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VETa8afmpWaYezLCqJhDB8
This commit is contained in:
1 parent
4f586d6551
commit
797513bb86
1 file changed
+13
-8
@@ -192,14 +192,19 @@ const STDERR_LOG: &str = "stderr.log";
|
|||||||
/// they left behind, one ordinary request ("can you make it so you can
|
/// they left behind, one ordinary request ("can you make it so you can
|
||||||
/// rename a session?") cost 4.2 million tokens across its 130 calls.
|
/// rename a session?") cost 4.2 million tokens across its 130 calls.
|
||||||
///
|
///
|
||||||
/// 100k is the smallest window the CLI accepts and roughly the cheapest:
|
/// 200k rather than the 100k floor, because the cheapest window on tokens
|
||||||
/// per-call cost falls with the cap, while the compaction it forces costs
|
/// is not the best one to sit in front of. Measured on the same session:
|
||||||
/// about the same in total either way -- a smaller window compacts more
|
/// context comes back to 70-85k within ten calls of a compaction, so a
|
||||||
/// often but each pass is proportionally smaller. What it trades is how
|
/// 100k window compacts about every thirteen calls, and a compaction
|
||||||
/// much detail survives a compaction, which is a real cost to the work
|
/// takes roughly two minutes (`durationMs` 104,346 to 147,671 across the
|
||||||
/// and the reason this is one named constant rather than a computed
|
/// six recorded). A 130-call request would spend some twenty minutes
|
||||||
/// value. Raise it if sessions start losing the thread.
|
/// compacting. 200k keeps most of the saving against the 1M ceiling and
|
||||||
const AUTOCOMPACT_WINDOW: &str = "100k";
|
/// halves the stalls.
|
||||||
|
///
|
||||||
|
/// Worth knowing before tuning this: the window decides how often the
|
||||||
|
/// context is thrown away, not how fast it fills. What fills it is ~7k
|
||||||
|
/// per call of tool output, and no value here touches that.
|
||||||
|
const AUTOCOMPACT_WINDOW: &str = "200k";
|
||||||
|
|
||||||
/// How often a reader with nothing to read looks again.
|
/// How often a reader with nothing to read looks again.
|
||||||
///
|
///
|
||||||
|
|||||||
Reference in new issue
Block a user