From 45e631ab966214afd507d3c148555cf9d8f2d87a Mon Sep 17 00:00:00 2001 From: iris <2+iris@noreply.localhost> Date: Fri, 4 Sep 2026 13:38:40 -0400 Subject: [PATCH] Delete TRANSCRIPT_RENDERING.md, which was all finished work The file was a week's record of the transcript work, and everything in its architecture, techniques and rejected sections has landed -- so it was a log of how the code got here, which the code's own comments and AGENTS.md already carry. Kept only what existed nowhere else: stream-bench.sh, trace-draw.sh and the two emulator-loop traps go to AGENTS.md beside transcript-bench.sh, along with the standing rule to run the benches either side of a transcript change; GrapheneOS's broken System Tracing goes to ~/.claude/MACHINE.md, since it is about the phone rather than this project. Its one unfinished item -- the reconnect loop after a restart onto a streaming session, which may already have been fixed by the restore's one-event-per-request fix -- moves to TODO.md. Co-Authored-By: Claude Opus 5 --- AGENTS.md | 31 +++- TODO.md | 13 ++ TRANSCRIPT_RENDERING.md | 303 ---------------------------------------- 3 files changed, 37 insertions(+), 310 deletions(-) delete mode 100644 TRANSCRIPT_RENDERING.md diff --git a/AGENTS.md b/AGENTS.md index 40c58a6..103ada8 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -5,11 +5,6 @@ replacing the Claude app for daily use. Rust/Axum backend on the desktop, Kotlin/Compose Android app, WireGuard + pinned self-signed TLS + bearer token between them. -**`TRANSCRIPT_RENDERING.md` is the record of the transcript work** -- -measurements, techniques, the harness, and the ordered list of what is -next. Read it before touching anything under `Markdown*.kt`, -`Transcript*.kt` or `SessionScreen.kt`'s list. - **`PLAN.md` is the design source of truth.** Read it before building or changing anything structural. It records every decision with its date, its rationale, and the alternatives that were rejected and why — keep that habit @@ -370,7 +365,7 @@ first if a remote spawn ever mangles an argument. **An inline code chip is drawn behind the text** rather than as the renderer's span background, because a span background is part of the text's own drawing and hid the selection under it -- see - `appendCodeChip` in `MarkdownLinks.kt` and TRANSCRIPT_RENDERING.md. + `appendCodeChip` in `MarkdownLinks.kt`. - **A session can be moved to another directory** from the settings dialog (`POST /sessions/{id}/cwd`). It stops the process, because a working directory is settled at spawn; the next message starts it in the new one. @@ -523,7 +518,29 @@ first if a remote spawn ever mangles an argument. the in-app copy button produces, whose `on screen:` line names what the viewport was actually holding. Compare two runs of it with the same gestures; the emulator's absolute frame times transfer nothing, the - report's accounting does. + report's accounting does. Run it either side of any change under + `Markdown*.kt`, `Transcript*.kt` or `SessionScreen.kt`'s list, and put the + report in the commit; the numbers that move first are the worst + `record: one block`, the reparse mean while streaming, and the draw + phase's accounting line. +- **`app/stream-bench.sh [-k] FILE` is that measurement for a reply still + arriving.** It opens the first session, taps "Jump to latest" so the list + is pinned to the newest end, resets the report, sends FILE, waits for the + transcript to stop growing, and prints. Both of those are corrections to a + first version that measured nothing: a transcript parked further back never + redraws while a reply streams into it, and a session is idle at *both* ends + of a turn, so polling for idle answers before the turn has started. +- **`app/trace-draw.sh` names what a scrolling frame spends inside the + framework**, from `atrace` text output with no trace processor needed. It + is how the cost of a layout node per link was attributed to the framework + rather than guessed at. +- **Two traps in the emulator bench loop**, each of which cost a run. + `adb shell pm clear` removes the enrolment and the notification permission + along with the saved anchors, so the next run measures a permission dialog + -- re-enrol with the command `ui-sandbox.sh` prints, and + `pm grant ... POST_NOTIFICATIONS`. And a saved scroll anchor is per session + id, so the only way two builds start a scroll from the same place is a + *fresh session for each*. - **`ai-server --delay MS` holds every response back.** Over the tunnel a phone's requests take tens to hundreds of milliseconds, and several faults live entirely in what the app does *while* one is outstanding. On diff --git a/TODO.md b/TODO.md index 1f9dcde..5155c84 100644 --- a/TODO.md +++ b/TODO.md @@ -16,6 +16,19 @@ one in place when it turns out to need a decision. would show exactly as "sometimes". Confirming it means driving a real stream-json session and sending it messages in both states. +## App — reconnect + +- [ ] Restarting the app onto a session with a saved anchor, while a long + reply was streaming, left it reconnecting every 1.5s + (`RECONNECT_DELAY_MS`) with the spinner up until the server was + restarted. `events?after=N` more than `CATCH_UP_LIMIT` (200) behind + answers `reset` plus the newest 200 *raw* deltas -- a window starting + mid-message -- and the reset clears `items`, which is the state the + restore loop then pages against. **May already be fixed:** the + restore's one-event-per-request bug was part of what made it so + visible and has since been fixed, so the first thing to find out is + whether this survives that. Found 2026-09-03. + ## Session settings - [ ] Autocompact belongs in session settings; empty disables it, which is the diff --git a/TRANSCRIPT_RENDERING.md b/TRANSCRIPT_RENDERING.md deleted file mode 100644 index ce54604..0000000 --- a/TRANSCRIPT_RENDERING.md +++ /dev/null @@ -1,303 +0,0 @@ -# Transcript rendering: what was learned, and what is next - -Written 2026-09-03 at the end of a week of work on the session screen's -transcript, so the next session can start from here rather than from a -compacted context. Work that is finished lives in "the architecture, as -built"; the running log of how each piece got there has been dropped. `AGENTS.md` holds the one-paragraph conventions; this is -the longer record: the measurements that drove each decision, the -techniques that worked, the ones that did not, and the order to do the rest -in. `PLAN.md` remains the design source of truth; nothing here contradicts -it. - -## The goal, and where it stands - -A reply of any length must scroll at the phone's 120Hz without a bump, and -must keep doing so while the reply is still streaming in. Measured on the -Pixel 9 Pro XL by Bryan, the transcript went from visible stalls at long -replies and at lists of links to "I have to actually try to feel any -bumps". The remaining work is finish and extensibility rather than -performance. - -## The architecture, as built - -Everything below lives under `app/androidApp/src/main/kotlin/com/example/aiapp/`. - -**Rows become units, and units are bounded.** `TranscriptUnits.kt` turns a -transcript row into the things the lazy list actually holds. An assistant -reply is not one unit: it is one unit per piece of its markdown, so the -list composes and draws a paragraph, a fence, a table or one bullet at a -time. The reason is the draw phase: a row's display list holds every glyph -of it and is re-recorded whenever drawing is invalidated, and the lazy list -composes an item whole in the frame it scrolls into. The tallest single -row still being drawn before this was 36,982px, twenty-five screens in one -message. Long user messages are sliced the same way (`UserChunk`), through -the shared `cardPiece` modifier that draws one card in lazy-list pieces. - -**One parse per message, addressed by piece.** `MarkdownPieces.kt`'s -`Piece(block, item)` is an address into the message's single parse tree, -not a substring: `block` indexes the root's children and `item` one -`LIST_ITEM` of a top-level list. Cutting was originally done by -re-parsing substrings, which cost a parse per piece and broke reference -links defined at the foot of a message. `ParsedReplies` caches the parse -and the piece list per text (`of`, `piecesOf`), warmed off the composing -thread by `TranscriptItems.warm`. The parser is still intellij-markdown via -the mikepenz renderer, but its `Markdown()` composable is not called at all: -`MarkdownRoot` in `Markdown.kt` provides the `Local*` environment itself -- -reference links from the parse, padding, dimens, colours, typography, a -no-op image transformer, animations, components -- and `MarkdownElement` -dispatches a whole block through our component table. Nothing between a -piece and the screen is the library's now except the leaf composables that -table names. - -**Lists are drawn an item at a time, by us.** The renderer has no element -for a single list item, so `MarkdownListItem` draws one: marker, then the -item's children, nested lists recursing through `MarkdownList`. The -marker is drawn in one place on purpose; styled bullets per depth go -there. - -**Links are spans, not nodes.** `MarkdownLinks.kt`. Compose turns every -`LinkAnnotation` into a layout node (clipped, focusable, hoverable, -clickable, outline recomputed from the text layout). A paragraph of eight -links was nine nodes, and measured against the same paragraphs with each -link replaced by plain words it cost 26.3ms worst measure against 5.2ms, -1.7x the place time. That was the bump at a reply's list of sources. -`LinkedText` builds the annotated string with the renderer's own inline -builder but answers links itself: colour, underline, a string annotation -carrying the URL, and one tap detector for the whole text that asks the -layout which glyph is under the finger. Hit-testing must check the glyph -on either side of the returned caret, because `getOffsetForPosition` -returns the nearest boundary; taps on the right half of a glyph otherwise -open nothing. Headings need the `ATX_CONTENT`/`SETEXT_CONTENT` child, since -the inline builder draws nothing for a node type it does not know (a week -of blank headings). Tables go through `LinkedTable`/`LinkedTableRow` so -cells get the same treatment. - -**An inline code chip is drawn behind the text, not as a span -background.** A `SpanStyle` background is part of the text's own drawing -and the text node draws the selection *under* the glyphs, so an opaque -chip hid the selection: selecting a sentence highlighted every word of it -except the ones in backticks, and there is no way to reorder that -- the -order is the node's. `appendCodeChip` therefore takes the code span from -the renderer's builder, keeps its style and its space of padding either -side but drops the background, and marks the range; `LinkedText` draws -those ranges in a `drawBehind`, which is under both the selection and the -glyphs -- the same place a fenced block's box already was, which is why -one of those always looked right. Geometry is one box per line, from the -bounding boxes of the run's first and last characters, taken as far as the -line's `visibleEnd`: `getPathForRange` is a *selection* shape and runs to -the right edge of every line but the last, which left a full-width empty -chip behind whenever the code wrapped, and `visibleEnd` is what makes the -chip and the selection rectangle stop in the same place. Measured against -the same build without it, streaming 60 paragraphs of three chips each: -measure 755ms against 776ms, record 327ms against 321ms, transcript draw -0.22ms in both -- noise. - -**Text draws on the platform directly.** A paragraph without an image -skips the renderer's `MarkdownText`, which charges every paragraph for the -possibility of inline images (placement callback, derived inline-content -map, semantics group, size animation). Paragraphs that contain an image -still take the renderer's path. - -**Tables spread or scroll without subcomposition.** The renderer used -`BoxWithConstraints` to decide; `LinkedTable` uses -`fillMaxWidth().horizontalScroll().layout { }` -- `horizontalScroll` -passes `minWidth` through and lifts `maxWidth` to infinity, so the inner -layout reads `minWidth` as the room available and takes -`max(minWidth, columns * cellWidth)`. - -**A streaming reply is reparsed one block at a time.** `LiveParse` in -`Markdown.kt` freezes every finished top-level block with its parse and -reparses only the tail block per delta. Markdown's block rules make later -text unable to alter an earlier block, with the single exception of a -late reference definition, which is accepted. Measured on a 58-word stream -of list, fence, table and quote: 47 tail reparses at 1.7ms mean. A -single-list stream would reparse the whole list per delta, since it is one -tail block; that is what the rule below cuts. - -**A streaming list becomes a unit per item.** `LiveParse.advanceTo` cuts at -the last item of a multi-item list (`openPiece`), provided that item has -content beyond its marker -- a bare `-` is an empty item now and the first -character of a paragraph line once `-x` arrives, so cutting on it would draw -that line as a new item. The cut is at the start of the item's line, so the -indentation the reparse reads its nesting from survives. `Segment.continues` -marks a tail that carries on a list, and `MarkdownPiece`'s -`continuesList`/`listContinues` keep an inner item's padding at the seam, so -nothing moves when the seam does. Forty linked bullets streamed a word at a -time went from 2412ms of reparsing to 674ms, and `record: one block` from -1.8ms worst to 0.7ms. - -**Fences are highlighted off the drawing thread, and a fence still being -written is drawn plain.** `Highlighter.kt` holds `highlight` and the scanner -behind it (shared with a tool call's input, so the same code is the same -colours wherever it appears); `CodeFence.kt` holds the `fenceLanguage` alias -table and `fenceContent`. A word not in the table stays plain, because a -fence coloured by the wrong language's rules looks highlighted and is wrong -in a way the reader cannot see. Highlighting is warmed and cached exactly as -parsing is (`ParsedReplies.highlighted`, filled by `warm` from -`fences(parse)`), and `highlight` takes no colour from the theme, which is -what lets it run off the drawing thread: a two-hundred-line Kotlin fence -costs 15ms to scan on the emulator's debug build -- it cost 102ms through -the library that used to do this -- and a `remember` inside the fence was -charged that again every time the block scrolled back into composition. Because the warming has to ask for the same -string the drawing does, `fenceContent` extracts the code and the language -word itself -- two extractions would be two keys, and the warmed answer -would be missed at every fence with nothing saying so. A fence still -arriving is the same stall in a second place, and warming cannot reach it: -the tail was re-lexed at every delta, on the composing thread, for colours -on text being replaced as fast as they were computed -- 211 lexes and 13.7 -seconds across one turn. So `MarkdownRoot`'s `streaming`, true only for a -live reply's last segment, draws the block plain until it freezes; a -finished fence colours as soon as the next block starts, and the settling -lex happens once, in `warm`. - -**Markers, and images.** `MarkdownListItem`'s `Marker` draws the bullet by -depth, cycling past the third, in `listMarkerColor` (Theme.kt). The colour -is the same at every depth on purpose: depth is said by the glyph and the -indent, and a colour per depth would make a difference in degree look like -one in kind. The app has no image loader and the renderer's transformer was -the no-op one, so an image in a reply drew as *nothing at all*; an `IMAGE` -node is now appended by `appendPlainLink` as a link carrying its alt text -(the address when there is none), which says what was there and opens it. - -**Expansion anchors the edge that was tapped, and the list never moves -under the reader** except when pinned to the bottom with new content -arriving. Those two rules are in `ScrollAnchor.kt` and `TranscriptList.kt` -and are the reason several tempting simplifications were rejected. - -## Techniques and harness - -- **`app/ui-sandbox.sh`** starts a second `ai-server` against a sandbox - home with the echo driver, so nothing touches real sessions. - `spawn [title]` makes an echo session and prints its id; `send SID text` - or `send SID @file` sends into it; `api /path [curl args]` is an - authenticated request. Restarting it regenerates the config but keeps - enrolled tokens. -- **The echo driver is the test rig** (`server/src/session/echo.rs`, the - list at the top of the file). `/stream N`, `/mixed N`, `/table N`, - `/tools N gap`, `/ask`, `/peer`, `/compact`, `/slow`, `/bash command` - each produce a shape the real CLI produces only when it feels like it. - Build what a UI test needs into it rather than spending model turns. -- **`app/transcript-bench.sh`** is the standard measurement: restart, open - the first session, scroll, print the render report. The report is what - the "Copy render timings" button copies and also logs - (`adb logcat -d -s ai-app:I`), and it includes the last crash's stack - (`CrashLog.kt`), which is how a crash on the phone reaches a session - here. -- **`app/stream-bench.sh [-k] FILE`** is `transcript-bench.sh` for a reply - still arriving: opens the first session, taps "Jump to latest" so the list - is pinned to the newest end, resets the report, sends FILE, waits for the - transcript to stop growing, prints the report. Both of those last two are - corrections to a first version that measured nothing -- a transcript parked - further back never redraws while a reply streams into it, and a session is - idle at *both* ends of a turn, so polling for idle answers before the turn - has started. Fixtures live in `/tmp` and are regenerated from the shapes - named here: `fixture.md` (lists four deep, ordered and nested, fences in - kotlin/rust/sh/none, a table with a link, a quote with a list, an inline - and a standalone image, a reference link), `longfence.md` (200-line Kotlin - fence), `longlist.md` (40 linked items). -- **Two traps in the emulator loop**, each of which cost a bench run. - `adb shell pm clear` removes the enrolment and the notification permission - along with the saved anchors, so the next run measures a permission - dialog; re-enrol with the command `ui-sandbox.sh` prints and - `pm grant ... POST_NOTIFICATIONS`. And a saved anchor is per session id, - so the only way two builds start a scroll from the same place is a *fresh - session for each*. -- **`DebugStats`/`FrameStats`** time our own phases (`record: one block`, - `measure: the app root`) and count events (`markdown reparsed while - streaming`, `markdown cut into pieces`). Add a counter before guessing. -- **`app/trace-draw.sh`** names what a scrolling frame spends inside the - framework, via `atrace` text output, no trace processor needed. It is - how the link-node cost was attributed. -- **`app/debug-transcript.sh`** loads a real Claude Code conversation onto - the emulator; two faults were invisible on fixtures and obvious on it. - Real transcripts are private: fixtures stay in `/tmp`, never in the repo. -- **`ui-trace`** reads the screen as text. Bounds print as - `x1,y1..x2,y2`; unanchored `-m` patterns match labels, anchored ones do - not. A row taller than the viewport reports clipped bounds, so compare - screenshots for that case. -- **Emulator frame times are not app measurements.** Software rendering - puts the stock Settings app at 60ms of UI-thread traversal per frame. - Costs of operations in milliseconds rank correctly; smoothness itself is - judged on the phone. -- **System Tracing on the phone does not work on GrapheneOS.** Its - Categories list is empty because the tracing daemon builds it by running - `atrace --list_categories`, which returns nothing there, and a recorded - trace contains zero ftrace events: no app sections, no frames, no - scheduling. Callstack sampling records, but the app's profiler config - unwinds one process shard in four. GrapheneOS issues 2206 and 6094 are - open on exactly this. Until they close, phone numbers come from the - render report and from Bryan noticing. -- **Compose `DropdownMenu` in an edge-to-edge activity** needs - `PopupProperties(clippingEnabled = false)` or it opens a status bar's - height away from its anchor (`~/.claude/TOOLCHAIN.md`). -- **The syntax highlighter is ours: `Highlighter.kt` and `Languages.kt`.** - One left-to-right scanner with a small state -- in a line comment, in a - block comment, in a string, or in ordinary code -- and a `Rules` row per - language, so a new language is a table entry rather than code. Every span - is emitted by advancing an index, so spans cannot overlap, arrive out of - order or run backwards, and an unterminated string or comment simply runs - to the end of the code. `HighlighterTest.kt` is the JVM unit test - (`./gradlew :androidApp:testDebugUnitTest`); the cases in it are the - library's mistakes, kept as regressions. - **Markdown is the one language that is not a `Rules` row** - (`MarkdownSyntax.kt`, added 2026-09-04). It has no keywords and no - strings, and what a character means there depends on where on the line it - sits -- a `#` opens a heading at the start of a line and is an ordinary - character three words in -- which is a question the token scanner cannot - ask. So it carries a scanner of its own, structure a line at a time and - then the inline forms left to right, and `spansOf` in `Languages.kt` is - the one entry point that hides which of the two a language got. It is - deliberately conservative where a guess would be invisible: emphasis needs - a closer on the same line with no space beside either marker (so the - `*p = *q` of a C fragment in a paragraph opens nothing), an underscore may - not start or end inside a word (`snake_case_name`), and an indented code - block is left plain, because four spaces after a blank line and four - spaces after a bullet are the same line and only what came before tells - them apart. A table is the one thing it looks ahead for: the delimiter row - (`|---|---|`) is the only line of one that cannot be anything else, and the - header belongs to the line before it -- which is what keeps the pipes of a - shell command written in a paragraph plain. - It replaced dev.snipme:highlights 1.1.0 on 2026-09-03, which found - comments before it knew the language and paired `/*` with `*/` by - ordinal. That library used one set of delimiters for every language, so - `//` in any URL commented out the rest of its line (in `curl - https://example.com/x && echo done` the comment ran to the end and took - `echo` with it, and in Kotlin `val url = "https://..."` the string - disappeared inside it), every Rust `#[derive(...)]` greyed out as a - comment, a `#` inside a Kotlin string swallowed the line, and `x '*/a/*'` - in shell yielded `start=6, end=5` -- a range `AnnotatedString` rejects, - which crashed a card holding `-path '*/.git/*'`. Comments were located - before strings and won over them, so post-processing could not recover - what a wrong comment range had already suppressed. The scanner is also - about seven times faster on the same fixture, and it colours RON, TOML, - fish and JSON, which the library did not know at all. - -## Rejected, and why - -- **Writing our own markdown renderer.** Rejected in favour of keeping - the intellij-markdown parser and the library's inline builder while - owning block dispatch and the leaf composables. The parser is the hard - part and is not the slow part; everything that was slow lived in the - composables, which are now ours. -- **Re-parsing substrings per piece.** Cost a parse per piece and broke - foot-of-message reference links. Replaced by addressed pieces of one - parse. -- **Animated or timing-dependent corrections.** Anything the reader could - catch at 120Hz is a bug; corrections must be structurally impossible to - see. - -## What is next, in order - -1. **The reconnect loop.** Restarting the app onto a session with a saved - anchor while a long reply was streaming left it reconnecting every 1.5s - (`RECONNECT_DELAY_MS`), spinner up, until the server was restarted. - `events?after=N` more than `CATCH_UP_LIMIT` (200) behind answers `reset` - plus the newest 200 *raw* deltas -- a window starting mid-message -- and - the reset clears `items`, which is the state the restore loop then pages - against. The restore's one-event-per-request bug was part of what made it - so visible and has been fixed; whether this survives that fix is the - first thing to find out. -2. **Regression runs.** `transcript-bench.sh` and `stream-bench.sh` before - and after any change to the files above, with the report in the commit. - The numbers to watch are the worst `record: one block`, the reparse mean - while streaming, and the draw phase's accounting line.