diff --git a/docs/RUST.md b/docs/RUST.md index 1ac022a..f5296c6 100644 --- a/docs/RUST.md +++ b/docs/RUST.md @@ -4135,6 +4135,186 @@ device. back with a diagnostic that will say the real story on the next run either way. + **Benchmark v2 (2026-09-06), asked for by Iris after using the + Compose build on her phone**: "it doesn't fling like I typically do + when scrolling up to find old messages. It should travel way faster + which is better for stress testing. You may also want to add typing + in the textbox as well and seeing how performant wrapping & pushing + the transcript up are, and also keyboard performance if possible." + **This is the one spec** -- written once here so both apps' "Run + benchmark" implement the identical four phases; a change to a + constant below has to be made in both `app/`'s `BenchRun.kt` and + `iris/`'s bench client, together, or the two reports stop measuring + the same thing while still looking like they do. + + 1. **fling.** Starting pinned at the newest end + (`listState.scrollToItem(0)` / iris's equivalent), 8 flings away + from it (toward older messages) through the list's own real fling + path -- Compose: `LazyListState.scroll { with(flingBehavior) { + performFling(velocity) } }` using the screen's actual + `FlingBehavior` (`ScrollableDefaults.flingBehavior()`, since + `TranscriptList`'s `LazyColumn` never overrides it -- **not** + `animateScrollBy`, which can only ever cover the fixed distance + and time it is given and was Iris's complaint) -- each fling's + `initialVelocity = 12,000 px/s`. That number is well above a + moderate tween-swipe's implied speed (v1's `SWIPE_PX`/`SWIPE_MS` + is roughly 4,500 px/s) and is meant to be a hard, fast flick for + stress-testing, per Iris's ask. After each fling, wait for + `isScrollInProgress` to clear (cap 3s; `performFling` already + suspends until its own decay ends, this is belt-and-suspenders) + plus 300ms between flings. Then 8 more flings back toward the + newest end (`-12,000 px/s`). Record the list's first visible + index/offset at the start, after the 8 outward flings, and at the + end, so the two apps' *travel* can be compared directly rather + than just their frame times. + 2. **stream. Unchanged from v1**: 400 tail events at 20/s (20 + seconds), pinned to the newest end before it starts (the same + "Jump to latest" pin `stream-bench.sh` does). + 3. **type.** Pin to the newest end, focus the composer, show the IME + if the platform allows it, then insert this **exact 600-character + string** one character per 50ms through the composer's real + `TextFieldValue` state (Compose: the same `input` state + `onValueChange` writes; iris: whatever holds the composer's text + today), then delete it the same way, one character per 50ms. + Chosen for long, multisyllabic words specifically so the composer + wraps across lines and the transcript above it is pushed upward + by a growing box, which is what Iris asked to see measured: + + Benchmarking this transcript screen requires unusually long, multisyllabic words so wrapping and reflow are properly exercised: internationalization, counterproductiveness, disproportionately, incomprehensibility, deinstitutionalization, uncharacteristically, overenthusiastically, misunderstanding, straightforwardness, telecommunications, and interdisciplinary collaboration all push a narrow composer field to wrap across several lines while the transcript above is pushed upward by the growing keyboard-adjacent box, which is exactly what a real reader typing a long message sees happening now!!! + + Report whether the IME was actually open during typing (it should + be, from this phase's own show-IME step -- see phase 4 for what + to say if the platform refuses to show it at all). + 4. **keyboard.** Show the IME (`WindowInsetsControllerCompat.show + (WindowInsetsCompat.Type.ime())` against the window/view; iris's + equivalent through its own shell), wait 1s, hide it, wait 1s; + five cycles. Confirm each show/hide with the platform's own + answer (Compose: `ViewCompat.getRootWindowInsets(view) + ?.isVisible(WindowInsetsCompat.Type.ime())`, i.e. the same + `WindowInsets.isImeVisible` fact `SessionScreen`'s composer-inset + bug fix already relies on) rather than assuming the request + worked -- if it is never confirmed shown even once, the report's + `keyboard:` line has to say "**keyboard: could not be shown**" + in words (UI_RULES: never present an inferred value as a + measured one, and design the unknown/failed state before the + answer's). + + **Frame accounting**: one recorder, not two. Mark each phase's start + in the existing per-frame recorder (Compose: `FrameStats.markPhase + (name)`, a list of `(name, frameIndexAtStart, wallClockAtStart)` + alongside the existing `total`/`waited`/... arrays) and slice the + same `FrameMetrics` samples by phase afterward + (`FrameStats.phaseLines`) rather than running a second listener. + + **Report shape**: a `per phase:` block appears once any phase marks + exist (empty/absent on an ordinary "Copy" press, which never marks a + phase), one entry per phase: frame count, the phase's wall-clock + duration, late count/percent (against the same refresh-rate budget + the whole-run section uses), p50/p90/p99, and the worst single + frame. Then **every existing whole-run section stays, unchanged in + shape** -- `frames:`, `where the draw phase went:`, `work since this + was last copied:` -- because that is what the emulator-baseline and + phone-baseline numbers already on record in this file were read + against, and a report that dropped or renamed one of those lines + would silently stop being comparable to them. Finally `bench:` gains + new lines beside the existing CPU/RSS/battery ones: the fling + phase's total travel (start/outward/end index+offset), the typed + character count, and the keyboard phase's shown/hidden-confirmed + counts (or the "could not be shown" line). + + **Compose half: done, 2026-09-06.** `FrameStats.markPhase`/ + `phaseLines` (`app/androidApp/src/main/kotlin/com/example/aiapp/ + FrameStats.kt`), `debugReport`'s new `phaseFrames` parameter + (`DebugStats.kt`), and `BenchRun.kt`'s four-phase `run` (fling via + `ScrollableDefaults.flingBehavior()` captured in `SessionScreen` and + passed down since it needs a `@Composable` call site; type via a new + `composerFocus: FocusRequester` attached to the composer's + `OutlinedTextField` plus a `setComposerText` callback that writes + `input` the same way a keystroke does; keyboard via + `WindowInsetsControllerCompat` against `LocalView.current`) are all + in. `BenchRun.TYPE_TEXT` is the exact 600-character constant quoted + above (verified `.length == 600`). A pre-existing, unrelated break + in `MainActivity.kt`'s `benchSessionSummary()` (missing several + `SessionSummary` constructor arguments added by a change this pass + did not otherwise touch -- confirmed pre-existing by reproducing the + same compile failure after stashing this pass's own diff) was fixed + alongside this, since it blocked `compileBenchKotlin` outright and + is in this session's own `app/` scope. + + Checks all clean: `ktfmtFormat`, `compileDebugKotlin`, + `compileBenchKotlin`, `lintDebug`, `lintBench` (both "No issues + found"), `testDebugUnitTest`. `grep -n "tap [0-9]" app/*.sh` still + has its one pre-existing, unrelated hit. + + **Compose bench v2, emulator smoke run, 2026-09-05** (this + checkout's AVD, cold `emu up`, `ui-trace` tap-by-label throughout -- + the dialog needed a swipe to reach "Run benchmark" below the fold, + report read back over `adb logcat`): + + ai-app render report + device: sdk_gphone64_x86_64 (Google), Android 16 + build: release + + transcript: + 124 events, 26 rows, 58 units loaded + viewport 1714px, 2 units visible + on screen: the list's own 0px, AssistantMsg 18732px + 0 tool calls and 0 groups open + + per phase: + fling: 1620 frames over 32.3s + late: 1537 (94.9%) + total p50 20.5ms p90 29.2ms p99 45.9ms + worst 61.8ms + stream: 1079 frames over 20.6s + late: 1037 (96.1%) + total p50 21.0ms p90 33.5ms p99 39.3ms + worst 51.2ms + type: 3568 frames over 61.4s + late: 3536 (99.1%) + total p50 23.8ms p90 32.1ms p99 38.5ms + worst 50.3ms + keyboard: 215 frames over 10.0s + late: 212 (98.6%) + total p50 21.3ms p90 37.6ms p99 48.4ms + worst 50.2ms + + frames: + 6482 frames over 124.3s at 60Hz (16.7ms budget) + late: 6322 (97.5%) + total p50 21.7ms p90 33.1ms p99 45.3ms + gpu p50 17.4ms p90 27.0ms p99 30.6ms + + where the draw phase went: + draw phase 1.27ms per frame, of which: + the transcript: 0.16ms (measure 0.09, place 0.07, record 0.00) + everything else: 1.10ms (87%) + + bench: + fling: 8 flings out + 8 back at 12000px/s, travel start=idx=0/off=0px outward=idx=218/off=73px end=idx=0/off=0px + scroll: 6 cycles (24 swipes, legacy tween), streamed 400/400 fixture events + type: 600 characters inserted then deleted, one per 50ms + keyboard: shown 5/5, hidden 5/5 (confirmed via isImeVisible) + process CPU time over this run: 61192ms + peak RSS: 195716kB + battery current: mean 900000µA over 125 samples (min 900000, max 900000) + + Read this the same way the v1 emulator smoke run above is read: it + proves the harness runs end to end and produces every field this + spec asked for, not a phone number -- software rasterisation, and + the fixed 900mA battery reading is the emulator's mocked charger + again. Two things worth carrying forward: the **fling phase reached + index 218** in 8 flings (against v1's `animateScrollBy` loop, which + never moved past a handful of indices in the same 8-swipe count), + which is the direct evidence the new fling travels "way faster" as + asked; and **the emulator's software keyboard toggled and was + confirmed by `isImeVisible` all 10 times**, so phase 4 is not a + guaranteed "could not be shown" on every platform, only where the + IME genuinely refuses. `frames:`'s 6,482-frame, 124.3s total matches + the sum of the four phase durations (32.3+20.6+61.4+10.0 ≈ 124.3s), + confirming the phase marks partition the whole run rather than + overlapping or dropping frames between them. + - [ ] **P1 — session screen parity.** History paging backward (with the page-boundary healing `client-core` does not have yet, below), `TranscriptSource`-backed cache/server stitching, jump-to-latest,