Write down what the transcript's paging cost, and how it was measured

Three entries under "Things that have bitten": the whole-file read behind
every page, the fact that a page is events and a screen is rows with no
fixed ratio between them, and the withContext that wrapped the fetch and
left the work done with the result outside it. Each carries the number it
was measured at, since the shape of all three is that the cost grows with
the conversation while the answer stays one screen.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
irisandClaude Opus 5 committed 2026-08-30 19:56:28 -04:00
1 parent c5ecc63995
commit 6079ac15a2
1 file changed
+32
+32
View File
@@ -457,6 +457,38 @@ machine belongs in `~/.claude/TOOLCHAIN.md` (toolchain versions) or
the screen opens. The reset is not optional: without it the window is
spliced onto rows that are no longer adjacent to it, which reads as
ordinary output.
- **A transcript page used to cost the whole transcript.** `read_window`
read and parsed every line and then kept the last `limit` of them, so the
work was the size of the conversation rather than the size of the answer:
on a 21 MB, 24,000-event transcript one page took ~500ms of server time to
return 620 KB, and took the same 500ms whichever page was asked for. A
phone scrolling back paid it per page and every stream reconnect paid it
again to find out nothing had happened. It is a bisection now
(`Indexed` in `transcript.rs`) -- sequence numbers only increase, so the
edge of a range is found by parsing one line per halving and only the
window is built. Same page, ~110ms, of which ~20ms is the file scan. The
file is still read whole; that is where the remaining cost is, and going
further means a chunked backwards reader.
`RUST_LOG=ai_server=debug` logs each page with what was asked and what
came back, which is how to see a phone paging back in real time.
- **A page is 800 events and a screen is a handful of rows, and the two
have no fixed ratio.** A run of thirty-five tool calls is one row; a reply
is hundreds of text deltas folded into one. So anything that budgets in
rows has to measure a screen rather than name a number: the history
cushion was eight rows, which on a tool-heavy transcript is less than one
screenful, and the reader hit the end of what was loaded on every swipe
and stood there for a round trip. That was "scrolling is laggy" -- not a
slow frame. It is `HISTORY_SCREENS` viewports now, counted from what is
actually on screen.
- **Only `fetchTranscript` was off the main thread; the fold was not.**
`foldEvent` returns a new list per event, so a page is that many copies of
a growing list -- fine at 80 events and about 300,000 element copies at
800, run in the middle of the scroll that asked for it. `warm` had the
same shape: the `markdownIn` scan that decides *what* to parse ran before
the hop to `Dispatchers.Default`, over every assistant message loaded, on
every page. Both are off it now. The shape to watch for is a
`withContext` that wraps the *fetch* and leaves the work done with the
result outside it.
- **ZXing only looks for a dark code on a light ground.** The enrollment
QR is block characters in the terminal's foreground colour, so a
dark-themed terminal renders it as a negative and the in-app scanner