iris/ai-app
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Delete TRANSCRIPT_RENDERING.md, which was all finished work

Browse Source 
The file was a week's record of the transcript work, and everything in its
architecture, techniques and rejected sections has landed -- so it was a log
of how the code got here, which the code's own comments and AGENTS.md
already carry. Kept only what existed nowhere else: stream-bench.sh,
trace-draw.sh and the two emulator-loop traps go to AGENTS.md beside
transcript-bench.sh, along with the standing rule to run the benches either
side of a transcript change; GrapheneOS's broken System Tracing goes to
~/.claude/MACHINE.md, since it is about the phone rather than this project.

Its one unfinished item -- the reconnect loop after a restart onto a
streaming session, which may already have been fixed by the restore's
one-event-per-request fix -- moves to TODO.md.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
irisandClaude Opus 5 committed 2026-09-04 13:38:40 -04:00
1 parent 7997eeb7f8
commit 45e631ab96
3 files changed
+37 -310
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

AGENTS.md
+24 -7
View File
@@ -5,11 +5,6 @@ replacing the Claude app for daily use. Rust/Axum backend on the desktop,
Kotlin/Compose Android app, WireGuard + pinned self-signed TLS + bearer token
between them.
**`TRANSCRIPT_RENDERING.md` is the record of the transcript work** --
measurements, techniques, the harness, and the ordered list of what is
next. Read it before touching anything under `Markdown*.kt`,
`Transcript*.kt` or `SessionScreen.kt`'s list.
**`PLAN.md` is the design source of truth.** Read it before building or
changing anything structural. It records every decision with its date, its
rationale, and the alternatives that were rejected and why — keep that habit
@@ -370,7 +365,7 @@ first if a remote spawn ever mangles an argument.
**An inline code chip is drawn behind the text** rather than as the
renderer's span background, because a span background is part of the
text's own drawing and hid the selection under it -- see
`appendCodeChip` in `MarkdownLinks.kt` and TRANSCRIPT_RENDERING.md.
`appendCodeChip` in `MarkdownLinks.kt`.
- **A session can be moved to another directory** from the settings dialog
(`POST /sessions/{id}/cwd`). It stops the process, because a working
directory is settled at spawn; the next message starts it in the new one.
@@ -523,7 +518,29 @@ first if a remote spawn ever mangles an argument.
the in-app copy button produces, whose `on screen:` line names what the
viewport was actually holding. Compare two runs of it with the same
gestures; the emulator's absolute frame times transfer nothing, the
report's accounting does.
report's accounting does. Run it either side of any change under
`Markdown*.kt`, `Transcript*.kt` or `SessionScreen.kt`'s list, and put the
report in the commit; the numbers that move first are the worst
`record: one block`, the reparse mean while streaming, and the draw
phase's accounting line.
- **`app/stream-bench.sh [-k] FILE` is that measurement for a reply still
arriving.** It opens the first session, taps "Jump to latest" so the list
is pinned to the newest end, resets the report, sends FILE, waits for the
transcript to stop growing, and prints. Both of those are corrections to a
first version that measured nothing: a transcript parked further back never
redraws while a reply streams into it, and a session is idle at *both* ends
of a turn, so polling for idle answers before the turn has started.
- **`app/trace-draw.sh` names what a scrolling frame spends inside the
framework**, from `atrace` text output with no trace processor needed. It
is how the cost of a layout node per link was attributed to the framework
rather than guessed at.
- **Two traps in the emulator bench loop**, each of which cost a run.
`adb shell pm clear` removes the enrolment and the notification permission
along with the saved anchors, so the next run measures a permission dialog
-- re-enrol with the command `ui-sandbox.sh` prints, and
`pm grant ... POST_NOTIFICATIONS`. And a saved scroll anchor is per session
id, so the only way two builds start a scroll from the same place is a
*fresh session for each*.
- **`ai-server --delay MS` holds every response back.** Over the tunnel a
phone's requests take tens to hundreds of milliseconds, and several
faults live entirely in what the app does *while* one is outstanding. On
TODO.md
+13
View File
@@ -16,6 +16,19 @@ one in place when it turns out to need a decision.
would show exactly as "sometimes". Confirming it means driving a real
stream-json session and sending it messages in both states.
## App — reconnect
- [ ] Restarting the app onto a session with a saved anchor, while a long
reply was streaming, left it reconnecting every 1.5s
(`RECONNECT_DELAY_MS`) with the spinner up until the server was
restarted. `events?after=N` more than `CATCH_UP_LIMIT` (200) behind
answers `reset` plus the newest 200 *raw* deltas -- a window starting
mid-message -- and the reset clears `items`, which is the state the
restore loop then pages against. **May already be fixed:** the
restore's one-event-per-request bug was part of what made it so
visible and has since been fixed, so the first thing to find out is
whether this survives that. Found 2026-09-03.
## Session settings
- [ ] Autocompact belongs in session settings; empty disables it, which is the
TRANSCRIPT_RENDERING.md
-303
View File
@@ -1,303 +0,0 @@
# Transcript rendering: what was learned, and what is next
Written 2026-09-03 at the end of a week of work on the session screen's
transcript, so the next session can start from here rather than from a
compacted context. Work that is finished lives in "the architecture, as
built"; the running log of how each piece got there has been dropped. `AGENTS.md` holds the one-paragraph conventions; this is
the longer record: the measurements that drove each decision, the
techniques that worked, the ones that did not, and the order to do the rest
in. `PLAN.md` remains the design source of truth; nothing here contradicts
it.
## The goal, and where it stands
A reply of any length must scroll at the phone's 120Hz without a bump, and
must keep doing so while the reply is still streaming in. Measured on the
Pixel 9 Pro XL by Bryan, the transcript went from visible stalls at long
replies and at lists of links to "I have to actually try to feel any
bumps". The remaining work is finish and extensibility rather than
performance.
## The architecture, as built
Everything below lives under `app/androidApp/src/main/kotlin/com/example/aiapp/`.
**Rows become units, and units are bounded.** `TranscriptUnits.kt` turns a
transcript row into the things the lazy list actually holds. An assistant
reply is not one unit: it is one unit per piece of its markdown, so the
list composes and draws a paragraph, a fence, a table or one bullet at a
time. The reason is the draw phase: a row's display list holds every glyph
of it and is re-recorded whenever drawing is invalidated, and the lazy list
composes an item whole in the frame it scrolls into. The tallest single
row still being drawn before this was 36,982px, twenty-five screens in one
message. Long user messages are sliced the same way (`UserChunk`), through
the shared `cardPiece` modifier that draws one card in lazy-list pieces.
**One parse per message, addressed by piece.** `MarkdownPieces.kt`'s
`Piece(block, item)` is an address into the message's single parse tree,
not a substring: `block` indexes the root's children and `item` one
`LIST_ITEM` of a top-level list. Cutting was originally done by
re-parsing substrings, which cost a parse per piece and broke reference
links defined at the foot of a message. `ParsedReplies` caches the parse
and the piece list per text (`of`, `piecesOf`), warmed off the composing
thread by `TranscriptItems.warm`. The parser is still intellij-markdown via
the mikepenz renderer, but its `Markdown()` composable is not called at all:
`MarkdownRoot` in `Markdown.kt` provides the `Local*` environment itself --
reference links from the parse, padding, dimens, colours, typography, a
no-op image transformer, animations, components -- and `MarkdownElement`
dispatches a whole block through our component table. Nothing between a
piece and the screen is the library's now except the leaf composables that
table names.
**Lists are drawn an item at a time, by us.** The renderer has no element
for a single list item, so `MarkdownListItem` draws one: marker, then the
item's children, nested lists recursing through `MarkdownList`. The
marker is drawn in one place on purpose; styled bullets per depth go
there.
**Links are spans, not nodes.** `MarkdownLinks.kt`. Compose turns every
`LinkAnnotation` into a layout node (clipped, focusable, hoverable,
clickable, outline recomputed from the text layout). A paragraph of eight
links was nine nodes, and measured against the same paragraphs with each
link replaced by plain words it cost 26.3ms worst measure against 5.2ms,
1.7x the place time. That was the bump at a reply's list of sources.
`LinkedText` builds the annotated string with the renderer's own inline
builder but answers links itself: colour, underline, a string annotation
carrying the URL, and one tap detector for the whole text that asks the
layout which glyph is under the finger. Hit-testing must check the glyph
on either side of the returned caret, because `getOffsetForPosition`
returns the nearest boundary; taps on the right half of a glyph otherwise
open nothing. Headings need the `ATX_CONTENT`/`SETEXT_CONTENT` child, since
the inline builder draws nothing for a node type it does not know (a week
of blank headings). Tables go through `LinkedTable`/`LinkedTableRow` so
cells get the same treatment.
**An inline code chip is drawn behind the text, not as a span
background.** A `SpanStyle` background is part of the text's own drawing
and the text node draws the selection *under* the glyphs, so an opaque
chip hid the selection: selecting a sentence highlighted every word of it
except the ones in backticks, and there is no way to reorder that -- the
order is the node's. `appendCodeChip` therefore takes the code span from
the renderer's builder, keeps its style and its space of padding either
side but drops the background, and marks the range; `LinkedText` draws
those ranges in a `drawBehind`, which is under both the selection and the
glyphs -- the same place a fenced block's box already was, which is why
one of those always looked right. Geometry is one box per line, from the
bounding boxes of the run's first and last characters, taken as far as the
line's `visibleEnd`: `getPathForRange` is a *selection* shape and runs to
the right edge of every line but the last, which left a full-width empty
chip behind whenever the code wrapped, and `visibleEnd` is what makes the
chip and the selection rectangle stop in the same place. Measured against
the same build without it, streaming 60 paragraphs of three chips each:
measure 755ms against 776ms, record 327ms against 321ms, transcript draw
0.22ms in both -- noise.
**Text draws on the platform directly.** A paragraph without an image
skips the renderer's `MarkdownText`, which charges every paragraph for the
possibility of inline images (placement callback, derived inline-content
map, semantics group, size animation). Paragraphs that contain an image
still take the renderer's path.
**Tables spread or scroll without subcomposition.** The renderer used
`BoxWithConstraints` to decide; `LinkedTable` uses
`fillMaxWidth().horizontalScroll().layout { }` -- `horizontalScroll`
passes `minWidth` through and lifts `maxWidth` to infinity, so the inner
layout reads `minWidth` as the room available and takes
`max(minWidth, columns * cellWidth)`.
**A streaming reply is reparsed one block at a time.** `LiveParse` in
`Markdown.kt` freezes every finished top-level block with its parse and
reparses only the tail block per delta. Markdown's block rules make later
text unable to alter an earlier block, with the single exception of a
late reference definition, which is accepted. Measured on a 58-word stream
of list, fence, table and quote: 47 tail reparses at 1.7ms mean. A
single-list stream would reparse the whole list per delta, since it is one
tail block; that is what the rule below cuts.
**A streaming list becomes a unit per item.** `LiveParse.advanceTo` cuts at
the last item of a multi-item list (`openPiece`), provided that item has
content beyond its marker -- a bare `-` is an empty item now and the first
character of a paragraph line once `-x` arrives, so cutting on it would draw
that line as a new item. The cut is at the start of the item's line, so the
indentation the reparse reads its nesting from survives. `Segment.continues`
marks a tail that carries on a list, and `MarkdownPiece`'s
`continuesList`/`listContinues` keep an inner item's padding at the seam, so
nothing moves when the seam does. Forty linked bullets streamed a word at a
time went from 2412ms of reparsing to 674ms, and `record: one block` from
1.8ms worst to 0.7ms.
**Fences are highlighted off the drawing thread, and a fence still being
written is drawn plain.** `Highlighter.kt` holds `highlight` and the scanner
behind it (shared with a tool call's input, so the same code is the same
colours wherever it appears); `CodeFence.kt` holds the `fenceLanguage` alias
table and `fenceContent`. A word not in the table stays plain, because a
fence coloured by the wrong language's rules looks highlighted and is wrong
in a way the reader cannot see. Highlighting is warmed and cached exactly as
parsing is (`ParsedReplies.highlighted`, filled by `warm` from
`fences(parse)`), and `highlight` takes no colour from the theme, which is
what lets it run off the drawing thread: a two-hundred-line Kotlin fence
costs 15ms to scan on the emulator's debug build -- it cost 102ms through
the library that used to do this -- and a `remember` inside the fence was
charged that again every time the block scrolled back into composition. Because the warming has to ask for the same
string the drawing does, `fenceContent` extracts the code and the language
word itself -- two extractions would be two keys, and the warmed answer
would be missed at every fence with nothing saying so. A fence still
arriving is the same stall in a second place, and warming cannot reach it:
the tail was re-lexed at every delta, on the composing thread, for colours
on text being replaced as fast as they were computed -- 211 lexes and 13.7
seconds across one turn. So `MarkdownRoot`'s `streaming`, true only for a
live reply's last segment, draws the block plain until it freezes; a
finished fence colours as soon as the next block starts, and the settling
lex happens once, in `warm`.
**Markers, and images.** `MarkdownListItem`'s `Marker` draws the bullet by
depth, cycling past the third, in `listMarkerColor` (Theme.kt). The colour
is the same at every depth on purpose: depth is said by the glyph and the
indent, and a colour per depth would make a difference in degree look like
one in kind. The app has no image loader and the renderer's transformer was
the no-op one, so an image in a reply drew as *nothing at all*; an `IMAGE`
node is now appended by `appendPlainLink` as a link carrying its alt text
(the address when there is none), which says what was there and opens it.
**Expansion anchors the edge that was tapped, and the list never moves
under the reader** except when pinned to the bottom with new content
arriving. Those two rules are in `ScrollAnchor.kt` and `TranscriptList.kt`
and are the reason several tempting simplifications were rejected.
## Techniques and harness
- **`app/ui-sandbox.sh`** starts a second `ai-server` against a sandbox
home with the echo driver, so nothing touches real sessions.
`spawn [title]` makes an echo session and prints its id; `send SID text`
or `send SID @file` sends into it; `api /path [curl args]` is an
authenticated request. Restarting it regenerates the config but keeps
enrolled tokens.
- **The echo driver is the test rig** (`server/src/session/echo.rs`, the
list at the top of the file). `/stream N`, `/mixed N`, `/table N`,
`/tools N gap`, `/ask`, `/peer`, `/compact`, `/slow`, `/bash command`
each produce a shape the real CLI produces only when it feels like it.
Build what a UI test needs into it rather than spending model turns.
- **`app/transcript-bench.sh`** is the standard measurement: restart, open
the first session, scroll, print the render report. The report is what
the "Copy render timings" button copies and also logs
(`adb logcat -d -s ai-app:I`), and it includes the last crash's stack
(`CrashLog.kt`), which is how a crash on the phone reaches a session
here.
- **`app/stream-bench.sh [-k] FILE`** is `transcript-bench.sh` for a reply
still arriving: opens the first session, taps "Jump to latest" so the list
is pinned to the newest end, resets the report, sends FILE, waits for the
transcript to stop growing, prints the report. Both of those last two are
corrections to a first version that measured nothing -- a transcript parked
further back never redraws while a reply streams into it, and a session is
idle at *both* ends of a turn, so polling for idle answers before the turn
has started. Fixtures live in `/tmp` and are regenerated from the shapes
named here: `fixture.md` (lists four deep, ordered and nested, fences in
kotlin/rust/sh/none, a table with a link, a quote with a list, an inline
and a standalone image, a reference link), `longfence.md` (200-line Kotlin
fence), `longlist.md` (40 linked items).
- **Two traps in the emulator loop**, each of which cost a bench run.
`adb shell pm clear` removes the enrolment and the notification permission
along with the saved anchors, so the next run measures a permission
dialog; re-enrol with the command `ui-sandbox.sh` prints and
`pm grant ... POST_NOTIFICATIONS`. And a saved anchor is per session id,
so the only way two builds start a scroll from the same place is a *fresh
session for each*.
- **`DebugStats`/`FrameStats`** time our own phases (`record: one block`,
`measure: the app root`) and count events (`markdown reparsed while
streaming`, `markdown cut into pieces`). Add a counter before guessing.
- **`app/trace-draw.sh`** names what a scrolling frame spends inside the
framework, via `atrace` text output, no trace processor needed. It is
how the link-node cost was attributed.
- **`app/debug-transcript.sh`** loads a real Claude Code conversation onto
the emulator; two faults were invisible on fixtures and obvious on it.
Real transcripts are private: fixtures stay in `/tmp`, never in the repo.
- **`ui-trace`** reads the screen as text. Bounds print as
`x1,y1..x2,y2`; unanchored `-m` patterns match labels, anchored ones do
not. A row taller than the viewport reports clipped bounds, so compare
screenshots for that case.
- **Emulator frame times are not app measurements.** Software rendering
puts the stock Settings app at 60ms of UI-thread traversal per frame.
Costs of operations in milliseconds rank correctly; smoothness itself is
judged on the phone.
- **System Tracing on the phone does not work on GrapheneOS.** Its
Categories list is empty because the tracing daemon builds it by running
`atrace --list_categories`, which returns nothing there, and a recorded
trace contains zero ftrace events: no app sections, no frames, no
scheduling. Callstack sampling records, but the app's profiler config
unwinds one process shard in four. GrapheneOS issues 2206 and 6094 are
open on exactly this. Until they close, phone numbers come from the
render report and from Bryan noticing.
- **Compose `DropdownMenu` in an edge-to-edge activity** needs
`PopupProperties(clippingEnabled = false)` or it opens a status bar's
height away from its anchor (`~/.claude/TOOLCHAIN.md`).
- **The syntax highlighter is ours: `Highlighter.kt` and `Languages.kt`.**
One left-to-right scanner with a small state -- in a line comment, in a
block comment, in a string, or in ordinary code -- and a `Rules` row per
language, so a new language is a table entry rather than code. Every span
is emitted by advancing an index, so spans cannot overlap, arrive out of
order or run backwards, and an unterminated string or comment simply runs
to the end of the code. `HighlighterTest.kt` is the JVM unit test
(`./gradlew :androidApp:testDebugUnitTest`); the cases in it are the
library's mistakes, kept as regressions.
**Markdown is the one language that is not a `Rules` row**
(`MarkdownSyntax.kt`, added 2026-09-04). It has no keywords and no
strings, and what a character means there depends on where on the line it
sits -- a `#` opens a heading at the start of a line and is an ordinary
character three words in -- which is a question the token scanner cannot
ask. So it carries a scanner of its own, structure a line at a time and
then the inline forms left to right, and `spansOf` in `Languages.kt` is
the one entry point that hides which of the two a language got. It is
deliberately conservative where a guess would be invisible: emphasis needs
a closer on the same line with no space beside either marker (so the
`*p = *q` of a C fragment in a paragraph opens nothing), an underscore may
not start or end inside a word (`snake_case_name`), and an indented code
block is left plain, because four spaces after a blank line and four
spaces after a bullet are the same line and only what came before tells
them apart. A table is the one thing it looks ahead for: the delimiter row
(`|---|---|`) is the only line of one that cannot be anything else, and the
header belongs to the line before it -- which is what keeps the pipes of a
shell command written in a paragraph plain.
It replaced dev.snipme:highlights 1.1.0 on 2026-09-03, which found
comments before it knew the language and paired `/*` with `*/` by
ordinal. That library used one set of delimiters for every language, so
`//` in any URL commented out the rest of its line (in `curl
https://example.com/x && echo done` the comment ran to the end and took
`echo` with it, and in Kotlin `val url = "https://..."` the string
disappeared inside it), every Rust `#[derive(...)]` greyed out as a
comment, a `#` inside a Kotlin string swallowed the line, and `x '*/a/*'`
in shell yielded `start=6, end=5` -- a range `AnnotatedString` rejects,
which crashed a card holding `-path '*/.git/*'`. Comments were located
before strings and won over them, so post-processing could not recover
what a wrong comment range had already suppressed. The scanner is also
about seven times faster on the same fixture, and it colours RON, TOML,
fish and JSON, which the library did not know at all.
## Rejected, and why
- **Writing our own markdown renderer.** Rejected in favour of keeping
the intellij-markdown parser and the library's inline builder while
owning block dispatch and the leaf composables. The parser is the hard
part and is not the slow part; everything that was slow lived in the
composables, which are now ours.
- **Re-parsing substrings per piece.** Cost a parse per piece and broke
foot-of-message reference links. Replaced by addressed pieces of one
parse.
- **Animated or timing-dependent corrections.** Anything the reader could
catch at 120Hz is a bug; corrections must be structurally impossible to
see.
## What is next, in order
1. **The reconnect loop.** Restarting the app onto a session with a saved
anchor while a long reply was streaming left it reconnecting every 1.5s
(`RECONNECT_DELAY_MS`), spinner up, until the server was restarted.
`events?after=N` more than `CATCH_UP_LIMIT` (200) behind answers `reset`
plus the newest 200 *raw* deltas -- a window starting mid-message -- and
the reset clears `items`, which is the state the restore loop then pages
against. The restore's one-event-per-request bug was part of what made it
so visible and has been fixed; whether this survives that fix is the
first thing to find out.
2. **Regression runs.** `transcript-bench.sh` and `stream-bench.sh` before
and after any change to the files above, with the report in the commit.
The numbers to watch are the worst `record: one block`, the reparse mean
while streaming, and the draw phase's accounting line.