irisandClaude Fable 5.1
801618ba0e
Uploads no longer sit whole in memory anywhere: the phone writes the multipart body chunked as it reads the picked file, and the server writes each chunk to a `.part` file under the session and renames it when whole. The per-request cap is 4 GB and bounds disk, not memory. A file attached to a session on another machine is copied there in the same request: one ssh invocation takes the bytes on stdin into the setup's `attachmentsDir` (new, optional, on the machine form and in the config), else the session's cwd, else the login home, and answers with `pwd -P`, which is recorded beside the file as `<name>.remote` and is the path the driver tells the CLI. A failed copy fails the upload and says why, so no message ever names a file that is not there. The host keeps its copy so transcripts can reference and fetch it. Measured against the Gentoo test guest: a 40 MB file shared from the phone arrived there byte for byte. The tilde in that setting is the remote home, so it is not expanded on the server the way other setup paths are. TRANSCRIPT_RENDERING.md records the week of transcript work -- the measurements behind each decision, the harness, what was rejected, and what to do next -- so a new session can start from it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
190 lines
11 KiB
Markdown
190 lines
11 KiB
Markdown
# Transcript rendering: what was learned, and what is next
|
|
|
|
Written 2026-09-03 at the end of a week of work on the session screen's
|
|
transcript, so the next session can start from here rather than from a
|
|
compacted context. `AGENTS.md` holds the one-paragraph conventions; this is
|
|
the longer record: the measurements that drove each decision, the
|
|
techniques that worked, the ones that did not, and the order to do the rest
|
|
in. `PLAN.md` remains the design source of truth; nothing here contradicts
|
|
it.
|
|
|
|
## The goal, and where it stands
|
|
|
|
A reply of any length must scroll at the phone's 120Hz without a bump, and
|
|
must keep doing so while the reply is still streaming in. Measured on the
|
|
Pixel 9 Pro XL by Bryan, the transcript went from visible stalls at long
|
|
replies and at lists of links to "I have to actually try to feel any
|
|
bumps". The remaining work is finish and extensibility rather than
|
|
performance.
|
|
|
|
## The architecture, as built
|
|
|
|
Everything below lives under `app/androidApp/src/main/kotlin/com/example/aiapp/`.
|
|
|
|
**Rows become units, and units are bounded.** `TranscriptUnits.kt` turns a
|
|
transcript row into the things the lazy list actually holds. An assistant
|
|
reply is not one unit: it is one unit per piece of its markdown, so the
|
|
list composes and draws a paragraph, a fence, a table or one bullet at a
|
|
time. The reason is the draw phase: a row's display list holds every glyph
|
|
of it and is re-recorded whenever drawing is invalidated, and the lazy list
|
|
composes an item whole in the frame it scrolls into. The tallest single
|
|
row still being drawn before this was 36,982px, twenty-five screens in one
|
|
message. Long user messages are sliced the same way (`UserChunk`), through
|
|
the shared `cardPiece` modifier that draws one card in lazy-list pieces.
|
|
|
|
**One parse per message, addressed by piece.** `MarkdownPieces.kt`'s
|
|
`Piece(block, item)` is an address into the message's single parse tree,
|
|
not a substring: `block` indexes the root's children and `item` one
|
|
`LIST_ITEM` of a top-level list. Cutting was originally done by
|
|
re-parsing substrings, which cost a parse per piece and broke reference
|
|
links defined at the foot of a message. `ParsedReplies` caches the parse
|
|
and the piece list per text (`of`, `piecesOf`), warmed off the composing
|
|
thread by `TranscriptItems.warm`. The parser is still intellij-markdown via
|
|
the mikepenz renderer; the renderer's `Markdown()` composable is kept only
|
|
as the provider of its `Local*` environment (`MarkdownRoot` in
|
|
`Markdown.kt`), and `MarkdownElement` dispatches a whole block through our
|
|
component table.
|
|
|
|
**Lists are drawn an item at a time, by us.** The renderer has no element
|
|
for a single list item, so `MarkdownListItem` draws one: marker, then the
|
|
item's children, nested lists recursing through `MarkdownList`. The
|
|
marker is drawn in one place on purpose; styled bullets per depth go
|
|
there.
|
|
|
|
**Links are spans, not nodes.** `MarkdownLinks.kt`. Compose turns every
|
|
`LinkAnnotation` into a layout node (clipped, focusable, hoverable,
|
|
clickable, outline recomputed from the text layout). A paragraph of eight
|
|
links was nine nodes, and measured against the same paragraphs with each
|
|
link replaced by plain words it cost 26.3ms worst measure against 5.2ms,
|
|
1.7x the place time. That was the bump at a reply's list of sources.
|
|
`LinkedText` builds the annotated string with the renderer's own inline
|
|
builder but answers links itself: colour, underline, a string annotation
|
|
carrying the URL, and one tap detector for the whole text that asks the
|
|
layout which glyph is under the finger. Hit-testing must check the glyph
|
|
on either side of the returned caret, because `getOffsetForPosition`
|
|
returns the nearest boundary; taps on the right half of a glyph otherwise
|
|
open nothing. Headings need the `ATX_CONTENT`/`SETEXT_CONTENT` child, since
|
|
the inline builder draws nothing for a node type it does not know (a week
|
|
of blank headings). Tables go through `LinkedTable`/`LinkedTableRow` so
|
|
cells get the same treatment.
|
|
|
|
**Text draws on the platform directly.** A paragraph without an image
|
|
skips the renderer's `MarkdownText`, which charges every paragraph for the
|
|
possibility of inline images (placement callback, derived inline-content
|
|
map, semantics group, size animation). Paragraphs that contain an image
|
|
still take the renderer's path.
|
|
|
|
**Tables spread or scroll without subcomposition.** The renderer used
|
|
`BoxWithConstraints` to decide; `LinkedTable` uses
|
|
`fillMaxWidth().horizontalScroll().layout { }` -- `horizontalScroll`
|
|
passes `minWidth` through and lifts `maxWidth` to infinity, so the inner
|
|
layout reads `minWidth` as the room available and takes
|
|
`max(minWidth, columns * cellWidth)`.
|
|
|
|
**A streaming reply is reparsed one block at a time.** `LiveParse` in
|
|
`Markdown.kt` freezes every finished top-level block with its parse and
|
|
reparses only the tail block per delta. Markdown's block rules make later
|
|
text unable to alter an earlier block, with the single exception of a
|
|
late reference definition, which is accepted. Measured on a 58-word stream
|
|
of list, fence, table and quote: 47 tail reparses at 1.7ms mean. A
|
|
single-list stream still reparses per delta because the whole list is one
|
|
tail block.
|
|
|
|
**Expansion anchors the edge that was tapped, and the list never moves
|
|
under the reader** except when pinned to the bottom with new content
|
|
arriving. Those two rules are in `ScrollAnchor.kt` and `TranscriptList.kt`
|
|
and are the reason several tempting simplifications were rejected.
|
|
|
|
## Techniques and harness
|
|
|
|
- **`app/ui-sandbox.sh`** starts a second `ai-server` against a sandbox
|
|
home with the echo driver, so nothing touches real sessions.
|
|
`spawn [title]` makes an echo session and prints its id; `send SID text`
|
|
or `send SID @file` sends into it; `api /path [curl args]` is an
|
|
authenticated request. Restarting it regenerates the config but keeps
|
|
enrolled tokens.
|
|
- **The echo driver is the test rig** (`server/src/session/echo.rs`, the
|
|
list at the top of the file). `/stream N`, `/mixed N`, `/table N`,
|
|
`/tools N gap`, `/ask`, `/peer`, `/compact`, `/slow`, `/bash command`
|
|
each produce a shape the real CLI produces only when it feels like it.
|
|
Build what a UI test needs into it rather than spending model turns.
|
|
- **`app/transcript-bench.sh`** is the standard measurement: restart, open
|
|
the first session, scroll, print the render report. The report is what
|
|
the "Copy render timings" button copies and also logs
|
|
(`adb logcat -d -s ai-app:I`), and it includes the last crash's stack
|
|
(`CrashLog.kt`), which is how a crash on the phone reaches a session
|
|
here.
|
|
- **`DebugStats`/`FrameStats`** time our own phases (`record: one block`,
|
|
`measure: the app root`) and count events (`markdown reparsed while
|
|
streaming`, `markdown cut into pieces`). Add a counter before guessing.
|
|
- **`app/trace-draw.sh`** names what a scrolling frame spends inside the
|
|
framework, via `atrace` text output, no trace processor needed. It is
|
|
how the link-node cost was attributed.
|
|
- **`app/debug-transcript.sh`** loads a real Claude Code conversation onto
|
|
the emulator; two faults were invisible on fixtures and obvious on it.
|
|
Real transcripts are private: fixtures stay in `/tmp`, never in the repo.
|
|
- **`ui-trace`** reads the screen as text. Bounds print as
|
|
`x1,y1..x2,y2`; unanchored `-m` patterns match labels, anchored ones do
|
|
not. A row taller than the viewport reports clipped bounds, so compare
|
|
screenshots for that case.
|
|
- **Emulator frame times are not app measurements.** Software rendering
|
|
puts the stock Settings app at 60ms of UI-thread traversal per frame.
|
|
Costs of operations in milliseconds rank correctly; smoothness itself is
|
|
judged on the phone.
|
|
- **System Tracing on the phone does not work on GrapheneOS.** Its
|
|
Categories list is empty because the tracing daemon builds it by running
|
|
`atrace --list_categories`, which returns nothing there, and a recorded
|
|
trace contains zero ftrace events: no app sections, no frames, no
|
|
scheduling. Callstack sampling records, but the app's profiler config
|
|
unwinds one process shard in four. GrapheneOS issues 2206 and 6094 are
|
|
open on exactly this. Until they close, phone numbers come from the
|
|
render report and from Bryan noticing.
|
|
- **Compose `DropdownMenu` in an edge-to-edge activity** needs
|
|
`PopupProperties(clippingEnabled = false)` or it opens a status bar's
|
|
height away from its anchor (`~/.claude/TOOLCHAIN.md`).
|
|
- **highlights 1.1.0's shell lexer** returns a span whose end precedes its
|
|
start for `x '*/a/*'`; `ToolInput.kt` drops such spans. The fix belongs
|
|
upstream and has not been filed.
|
|
|
|
## Rejected, and why
|
|
|
|
- **Writing our own markdown renderer.** Rejected in favour of keeping
|
|
the intellij-markdown parser and the library's inline builder while
|
|
owning block dispatch and the leaf composables. The parser is the hard
|
|
part and is not the slow part; everything that was slow lived in the
|
|
composables, which are now ours.
|
|
- **Re-parsing substrings per piece.** Cost a parse per piece and broke
|
|
foot-of-message reference links. Replaced by addressed pieces of one
|
|
parse.
|
|
- **Animated or timing-dependent corrections.** Anything the reader could
|
|
catch at 120Hz is a bug; corrections must be structurally impossible to
|
|
see.
|
|
|
|
## What is next, in order
|
|
|
|
1. **Styled markers per depth.** `MarkdownListItem`'s `Marker` is the one
|
|
place bullets and numbers are drawn; give it a glyph per depth and the
|
|
list's own colour. Bryan asked whether the architecture allows it; it
|
|
does, and it is a small change.
|
|
2. **Syntax highlighting inside fences.** The highlights lexer used for
|
|
tool commands can colour code blocks too; route the fence composable
|
|
through the same table as `ToolInput.kt`, keep the inverted-range
|
|
guard, and measure a long fence before and after, since a highlighted
|
|
fence is one `Text` with many spans.
|
|
3. **Drop `MarkdownRoot`'s dependence on the library's `Markdown()`.**
|
|
It exists only to provide `LocalMarkdown*`. Providing those locals
|
|
directly removes the last library composable from the hot path and
|
|
frees the way for a different parser later.
|
|
4. **Paragraphs with images** still take the renderer's `MarkdownText`.
|
|
Draw the image as its own piece below the paragraph instead, then the
|
|
text leaf covers every paragraph.
|
|
5. **Per-item units for a streaming list.** A single-list stream reparses
|
|
the whole list per delta; freezing finished items would make a
|
|
forty-item list stream like forty paragraphs.
|
|
6. **Regression runs.** Run `transcript-bench.sh` before and after any
|
|
change to the files above and paste the report into the commit. The
|
|
numbers to watch are the worst `record: one block` and the draw phase
|
|
share in the accounting line.
|
|
7. **Tooling debt.** AGP 9.4.0 is available (lint warns). File the
|
|
highlights range bug upstream with the one-line repro.
|