ai-app/TRANSCRIPT_RENDERING.md

# Transcript rendering: what was learned, and what is next

Written 2026-09-03 at the end of a week of work on the session screen's
transcript, so the next session can start from here rather than from a
compacted context. `AGENTS.md` holds the one-paragraph conventions; this is
the longer record: the measurements that drove each decision, the
techniques that worked, the ones that did not, and the order to do the rest
in. `PLAN.md` remains the design source of truth; nothing here contradicts
it.

## The goal, and where it stands

A reply of any length must scroll at the phone's 120Hz without a bump, and
must keep doing so while the reply is still streaming in. Measured on the
Pixel 9 Pro XL by Bryan, the transcript went from visible stalls at long
replies and at lists of links to "I have to actually try to feel any
bumps". The remaining work is finish and extensibility rather than
performance.

## The architecture, as built

Everything below lives under `app/androidApp/src/main/kotlin/com/example/aiapp/`.

**Rows become units, and units are bounded.** `TranscriptUnits.kt` turns a
transcript row into the things the lazy list actually holds. An assistant
reply is not one unit: it is one unit per piece of its markdown, so the
list composes and draws a paragraph, a fence, a table or one bullet at a
time. The reason is the draw phase: a row's display list holds every glyph
of it and is re-recorded whenever drawing is invalidated, and the lazy list
composes an item whole in the frame it scrolls into. The tallest single
row still being drawn before this was 36,982px, twenty-five screens in one
message. Long user messages are sliced the same way (`UserChunk`), through
the shared `cardPiece` modifier that draws one card in lazy-list pieces.

**One parse per message, addressed by piece.** `MarkdownPieces.kt`'s
`Piece(block, item)` is an address into the message's single parse tree,
not a substring: `block` indexes the root's children and `item` one
`LIST_ITEM` of a top-level list. Cutting was originally done by
re-parsing substrings, which cost a parse per piece and broke reference
links defined at the foot of a message. `ParsedReplies` caches the parse
and the piece list per text (`of`, `piecesOf`), warmed off the composing
thread by `TranscriptItems.warm`. The parser is still intellij-markdown via
the mikepenz renderer; the renderer's `Markdown()` composable is kept only
as the provider of its `Local*` environment (`MarkdownRoot` in
`Markdown.kt`), and `MarkdownElement` dispatches a whole block through our
component table.

**Lists are drawn an item at a time, by us.** The renderer has no element
for a single list item, so `MarkdownListItem` draws one: marker, then the
item's children, nested lists recursing through `MarkdownList`. The
marker is drawn in one place on purpose; styled bullets per depth go
there.

**Links are spans, not nodes.** `MarkdownLinks.kt`. Compose turns every
`LinkAnnotation` into a layout node (clipped, focusable, hoverable,
clickable, outline recomputed from the text layout). A paragraph of eight
links was nine nodes, and measured against the same paragraphs with each
link replaced by plain words it cost 26.3ms worst measure against 5.2ms,
1.7x the place time. That was the bump at a reply's list of sources.
`LinkedText` builds the annotated string with the renderer's own inline
builder but answers links itself: colour, underline, a string annotation
carrying the URL, and one tap detector for the whole text that asks the
layout which glyph is under the finger. Hit-testing must check the glyph
on either side of the returned caret, because `getOffsetForPosition`
returns the nearest boundary; taps on the right half of a glyph otherwise
open nothing. Headings need the `ATX_CONTENT`/`SETEXT_CONTENT` child, since
the inline builder draws nothing for a node type it does not know (a week
of blank headings). Tables go through `LinkedTable`/`LinkedTableRow` so
cells get the same treatment.

**Text draws on the platform directly.** A paragraph without an image
skips the renderer's `MarkdownText`, which charges every paragraph for the
possibility of inline images (placement callback, derived inline-content
map, semantics group, size animation). Paragraphs that contain an image
still take the renderer's path.

**Tables spread or scroll without subcomposition.** The renderer used
`BoxWithConstraints` to decide; `LinkedTable` uses
`fillMaxWidth().horizontalScroll().layout { }` -- `horizontalScroll`
passes `minWidth` through and lifts `maxWidth` to infinity, so the inner
layout reads `minWidth` as the room available and takes
`max(minWidth, columns * cellWidth)`.

**A streaming reply is reparsed one block at a time.** `LiveParse` in
`Markdown.kt` freezes every finished top-level block with its parse and
reparses only the tail block per delta. Markdown's block rules make later
text unable to alter an earlier block, with the single exception of a
late reference definition, which is accepted. Measured on a 58-word stream
of list, fence, table and quote: 47 tail reparses at 1.7ms mean. A
single-list stream still reparses per delta because the whole list is one
tail block.

**Expansion anchors the edge that was tapped, and the list never moves
under the reader** except when pinned to the bottom with new content
arriving. Those two rules are in `ScrollAnchor.kt` and `TranscriptList.kt`
and are the reason several tempting simplifications were rejected.

## Techniques and harness

- **`app/ui-sandbox.sh`** starts a second `ai-server` against a sandbox
  home with the echo driver, so nothing touches real sessions.
  `spawn [title]` makes an echo session and prints its id; `send SID text`
  or `send SID @file` sends into it; `api /path [curl args]` is an
  authenticated request. Restarting it regenerates the config but keeps
  enrolled tokens.
- **The echo driver is the test rig** (`server/src/session/echo.rs`, the
  list at the top of the file). `/stream N`, `/mixed N`, `/table N`,
  `/tools N gap`, `/ask`, `/peer`, `/compact`, `/slow`, `/bash command`
  each produce a shape the real CLI produces only when it feels like it.
  Build what a UI test needs into it rather than spending model turns.
- **`app/transcript-bench.sh`** is the standard measurement: restart, open
  the first session, scroll, print the render report. The report is what
  the "Copy render timings" button copies and also logs
  (`adb logcat -d -s ai-app:I`), and it includes the last crash's stack
  (`CrashLog.kt`), which is how a crash on the phone reaches a session
  here.
- **`DebugStats`/`FrameStats`** time our own phases (`record: one block`,
  `measure: the app root`) and count events (`markdown reparsed while
  streaming`, `markdown cut into pieces`). Add a counter before guessing.
- **`app/trace-draw.sh`** names what a scrolling frame spends inside the
  framework, via `atrace` text output, no trace processor needed. It is
  how the link-node cost was attributed.
- **`app/debug-transcript.sh`** loads a real Claude Code conversation onto
  the emulator; two faults were invisible on fixtures and obvious on it.
  Real transcripts are private: fixtures stay in `/tmp`, never in the repo.
- **`ui-trace`** reads the screen as text. Bounds print as
  `x1,y1..x2,y2`; unanchored `-m` patterns match labels, anchored ones do
  not. A row taller than the viewport reports clipped bounds, so compare
  screenshots for that case.
- **Emulator frame times are not app measurements.** Software rendering
  puts the stock Settings app at 60ms of UI-thread traversal per frame.
  Costs of operations in milliseconds rank correctly; smoothness itself is
  judged on the phone.
- **System Tracing on the phone does not work on GrapheneOS.** Its
  Categories list is empty because the tracing daemon builds it by running
  `atrace --list_categories`, which returns nothing there, and a recorded
  trace contains zero ftrace events: no app sections, no frames, no
  scheduling. Callstack sampling records, but the app's profiler config
  unwinds one process shard in four. GrapheneOS issues 2206 and 6094 are
  open on exactly this. Until they close, phone numbers come from the
  render report and from Bryan noticing.
- **Compose `DropdownMenu` in an edge-to-edge activity** needs
  `PopupProperties(clippingEnabled = false)` or it opens a status bar's
  height away from its anchor (`~/.claude/TOOLCHAIN.md`).
- **highlights 1.1.0's shell lexer** returns a span whose end precedes its
  start for `x '*/a/*'`; `ToolInput.kt` drops such spans. The fix belongs
  upstream and has not been filed.

## Rejected, and why

- **Writing our own markdown renderer.** Rejected in favour of keeping
  the intellij-markdown parser and the library's inline builder while
  owning block dispatch and the leaf composables. The parser is the hard
  part and is not the slow part; everything that was slow lived in the
  composables, which are now ours.
- **Re-parsing substrings per piece.** Cost a parse per piece and broke
  foot-of-message reference links. Replaced by addressed pieces of one
  parse.
- **Animated or timing-dependent corrections.** Anything the reader could
  catch at 120Hz is a bug; corrections must be structurally impossible to
  see.

## Session of 2026-09-03: the list above, worked through

Items 1-5 of the previous list are implemented and build; the AGP bump from
item 7 is in. What is *not* done is in "What is next" below, and the state
of each measurement is stated honestly here so nothing has to be re-derived.

**1. Styled markers per depth -- done, unverified on screen.**
`MarkdownListItem`'s `Marker` draws `•`, `◦`, `▪` by depth (cycling) in
`listMarkerColor` (Theme.kt, Lavender: the scheme's secondary accent, which
nothing else used, so it now means "structure"). Ordered numbers take the
same colour. Still to check on the emulator: that `◦` and `▪` are in the
system fonts rather than drawing as boxes -- the comment on `BULLETS` claims
they were checked, and that check is what the next session owes it.

**2. Syntax highlighting inside fences -- done, measured before only.**
`CodeFence.kt` holds `highlighted` (moved out of `ToolInput.kt`, timed as
`code highlighted`), the `fenceLanguage` alias table (extension or name to
the highlights lexer; unknown words stay plain on purpose), and `CodeFence`
/ `CodeBlock`, registered as the component table's `codeFence`/`codeBlock`.
The library's `MarkdownCodeFence` still finds the code inside the node; the
drawing is ours (same background, corner, padding and sideways scroll, minus
the shadow, border and empty pointer handler). Baseline `transcript-bench.sh`
on the fixture below, before the change: draw phase 0.72ms/frame, transcript
0.36ms, the 200-line kotlin fence one 10,700px block. The after run has not
been taken.

**3. `MarkdownRoot` no longer calls the library's `Markdown()`.** It
provides the eight locals itself (`LocalReferenceLinkHandler` from the
parse, padding, dimens, colours, typography, a no-op image transformer,
animations, components).

**4. Paragraphs with images -- done differently from the plan.** The plan
said draw the image as its own piece; the app has no image loader and the
renderer's transformer was the no-op one, so an image in a reply drew as
*nothing*. An `IMAGE` node is now appended by `appendPlainLink` as a link
carrying its alt text (the address when there is none), and with no image to
place the `hasImage` branch and the renderer's `MarkdownText` are gone:
every paragraph is the platform `BasicText`.

**5. Per-item units for a streaming list -- done, before measured, after
not.** `LiveParse.advanceTo` cuts at the last item of a multi-item list
(`openPiece`), provided that item has content beyond its marker (a bare `-`
may still become a paragraph line); the cut is at the start of the item's
line so indentation survives the reparse. `Segment.continues` marks a tail
that carries on a list, and `MarkdownPiece`'s `continuesList`/`listContinues`
keep the padding of an inner item at the seam so nothing moves when it
does. Measured with the new `app/stream-bench.sh` streaming
`/tmp/longlist.md` (forty bullet items with a link each) on the old build:
`markdown reparsed while streaming: 483, 3.9ms mean, 11.6ms worst`,
`record: one block` worst 1.6ms. The same run on the new build printed an
empty report -- the first thing to look at (the screen showed the list
drawn with `•` markers, so the build runs; the report tap or the idle wait
may have misfired).

**Harness.** `app/stream-bench.sh [-k] FILE` is `transcript-bench.sh` for a
reply still arriving: opens the first session, resets the report, sends
FILE through `ui-sandbox.sh send`, waits for idle, prints the report.
Fixtures used this session, all in `/tmp` (regenerate from the shapes
named): `fixture.md` (lists three deep, ordered and nested, fences in
kotlin/rust/sh/none, a table with a link, a quote with a list, an inline and
a standalone image, a reference link), `longfence.md` (200-line kotlin
fence), `longlist.md` (40 linked items), `numlist.md`.

**Seen and not chased: a reconnect loop.** On the *old* build, restarting
the app onto the fixture session with a saved anchor mid-transcript while a
600-delta reply was streaming left the screen on a spinner, reconnecting
every 1.5s (`RECONNECT_DELAY_MS`) with `session screen recomposed: 26` and
the fence message re-warmed each time, until the sandbox server was
restarted. `events?after=N` more than `CATCH_UP_LIMIT` (200) behind answers
`reset` plus the newest 200 *raw* deltas, i.e. a window starting
mid-message; the restore loop and that reset clearing `items` look like the
two halves. Reproduce with `stream-bench.sh` (restart form) after a
`transcript-bench.sh` run has left an anchor mid-fence.

## What is next, in order

1. **Look at the fixture on the emulator** (session `fixture2` in the
   sandbox holds only `fixture.md`): bullet glyphs at three depths, fence
   colours, the image drawn as a link, the reference link at the foot.
2. **Take the after measurements**: `transcript-bench.sh` for the fence,
   `stream-bench.sh /tmp/longlist.md` for the list, and put both pairs in
   the commit message. Find out why the after run's report was empty.
3. **Lint** (`./gradlew :androidApp:lint`) on AGP 9.4.0; the bump is in
   `libs.versions.toml` and the build passed, lint has not been run.
4. **The reconnect loop above.**
5. **File the highlights range bug upstream** -- no `gh` in this VM and no
   GitHub credential, so it needs Bryan or a token. One-line repro: lexing
   `x '*/a/*'` as `SyntaxLanguage.SHELL` in highlights 1.1.0 returns a
   highlight whose `location.end` precedes its `location.start`.
6. Everything from the earlier list that still stands: regression runs
   before and after any change to these files, pasted into the commit.