Write down what the transcript's paging cost, and how it was measured
Three entries under "Things that have bitten": the whole-file read behind every page, the fact that a page is events and a screen is rows with no fixed ratio between them, and the withContext that wrapped the fetch and left the work done with the result outside it. Each carries the number it was measured at, since the shape of all three is that the cost grows with the conversation while the answer stays one screen. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
1 parent
c5ecc63995
commit
6079ac15a2
1 file changed
+32
@@ -457,6 +457,38 @@ machine belongs in `~/.claude/TOOLCHAIN.md` (toolchain versions) or
|
||||
the screen opens. The reset is not optional: without it the window is
|
||||
spliced onto rows that are no longer adjacent to it, which reads as
|
||||
ordinary output.
|
||||
- **A transcript page used to cost the whole transcript.** `read_window`
|
||||
read and parsed every line and then kept the last `limit` of them, so the
|
||||
work was the size of the conversation rather than the size of the answer:
|
||||
on a 21 MB, 24,000-event transcript one page took ~500ms of server time to
|
||||
return 620 KB, and took the same 500ms whichever page was asked for. A
|
||||
phone scrolling back paid it per page and every stream reconnect paid it
|
||||
again to find out nothing had happened. It is a bisection now
|
||||
(`Indexed` in `transcript.rs`) -- sequence numbers only increase, so the
|
||||
edge of a range is found by parsing one line per halving and only the
|
||||
window is built. Same page, ~110ms, of which ~20ms is the file scan. The
|
||||
file is still read whole; that is where the remaining cost is, and going
|
||||
further means a chunked backwards reader.
|
||||
`RUST_LOG=ai_server=debug` logs each page with what was asked and what
|
||||
came back, which is how to see a phone paging back in real time.
|
||||
- **A page is 800 events and a screen is a handful of rows, and the two
|
||||
have no fixed ratio.** A run of thirty-five tool calls is one row; a reply
|
||||
is hundreds of text deltas folded into one. So anything that budgets in
|
||||
rows has to measure a screen rather than name a number: the history
|
||||
cushion was eight rows, which on a tool-heavy transcript is less than one
|
||||
screenful, and the reader hit the end of what was loaded on every swipe
|
||||
and stood there for a round trip. That was "scrolling is laggy" -- not a
|
||||
slow frame. It is `HISTORY_SCREENS` viewports now, counted from what is
|
||||
actually on screen.
|
||||
- **Only `fetchTranscript` was off the main thread; the fold was not.**
|
||||
`foldEvent` returns a new list per event, so a page is that many copies of
|
||||
a growing list -- fine at 80 events and about 300,000 element copies at
|
||||
800, run in the middle of the scroll that asked for it. `warm` had the
|
||||
same shape: the `markdownIn` scan that decides *what* to parse ran before
|
||||
the hop to `Dispatchers.Default`, over every assistant message loaded, on
|
||||
every page. Both are off it now. The shape to watch for is a
|
||||
`withContext` that wraps the *fetch* and leaves the work done with the
|
||||
result outside it.
|
||||
- **ZXing only looks for a dark code on a light ground.** The enrollment
|
||||
QR is block characters in the terminal's foreground colour, so a
|
||||
dark-themed terminal renders it as a negative and the in-app scanner
|
||||
|
||||
Reference in new issue
Block a user