ai-app-bench/README.md

ai-app bench

The two APKs for the phone benchmark gate (ai-app's docs/RUST.md, P0), packaged for Dev Updater so they can be installed on a phone without a build machine. Add this checkout as a project in Dev Updater (its .dev-updater.ron names two APK components, compose-bench and iris-bench, with no build step -- the APKs are committed under compose/ and iris/), then install each from its card. Both install beside the real app under their own application ids and need no server, no enrolment and no permissions.

The rest of this file is the runbook that was delivered with the APKs.

2026-09-06: bench v2 (fling/stream/type/keyboard). compose-bench-arm64.apk rebuilt with the four-phase benchmark Iris asked for after using v1 on her phone -- a real fling (12,000px/s through the list's own FlingBehavior, not the old tween-based swipe, so it travels much further per gesture), the existing 400-event/20s stream, a type phase (600 fixed characters typed into the real composer then deleted, to exercise wrapping and the transcript being pushed upward), and a keyboard phase (five show/hide cycles). The report gains a per phase: block (frames/late/percentiles/ worst/duration per phase) ahead of the existing whole-run sections, and bench: gains the fling phase's travel, the typed character count, and how many of the ten keyboard toggles were confirmed by the platform. Full spec and the emulator smoke run are in ai-app-2's docs/RUST.md, P0 box, "Benchmark v2 (2026-09-06)". Same steps below still apply -- "Run benchmark" is unchanged in name and location.

---

P0 benchmark APKs

Two arm64 builds, asked for in docs/RUST.md's P0 box (the phone benchmark gate Iris asked for 2026-09-05, "before P1 I'd like to see benchmarks & also maybe stress test on my own phone"). Each opens straight onto the same synthetic fixture transcript with no server, runs the same scripted scroll-and-stream benchmark, and produces a text report to paste back. Install both, run both, paste both reports back -- the gate is a comparison, not either number alone.

Compose (compose-bench-arm64.apk)

1. Install it (it has its own application id, com.example.aiapp.bench, and its own label "AI Sessions bench" -- it installs beside a real enrolled "AI Sessions" without touching it).
2. Open it. It goes straight to a session screen holding the fixture transcript -- no enrollment, no permission prompts.
3. Tap the gear icon top-right ("Session settings").
4. Tap "Run benchmark". It scrolls the transcript (the same 24-swipe loop app/transcript-bench.sh drives on a phone from a computer), then streams 400 more fixture events in over 20 seconds pinned to the newest end (the same shape app/stream-bench.sh measures), then copies a report to the clipboard and shows a toast.
5. Paste the clipboard back here (or in whatever you're pasting reports into). It is plain text with units on every number: frame counts and percentiles, the draw-phase accounting, and a bench: section this build added -- process CPU time, peak RSS, and battery current sampled once a second (says "unavailable" rather than a fabricated number where the phone can't answer, per this project's rule against showing an inferred value as a measured one).
6. If you want to run it again, "Run benchmark" can be pressed more than once -- each press resets the counters first, so the report only covers that run's own scroll and stream.

Built from app/androidApp's bench build type; see app/build-apk.sh bench and app/bench-fixture/README.md for how the fixture it opens was generated (deterministic, checked into the repo, never a real transcript).

iris (iris-bench-arm64.apk)

1. Install it -- own application id, dev.iris.android.demo.bench (an applicationIdSuffix on the same dev.iris.android.demo the plain iris tabs demo uses, so it installs beside that one too), needs no permissions beyond INTERNET (unused here -- inherited from the same manifest transcript-screen needs, harmless for a build that talks to nothing).
2. Open it. Same fixture as the Compose build (the identical checked-in app/bench-fixture/assets/transcript.jsonl), opened with no server through client-core's real fold_page -- there is no enrollment step here at all, iris's shell has none yet.
3. Tap "Run benchmark" (the whole screen is just two buttons above the transcript and a report area below it -- no settings dialog to find it in). It runs the identical scripted loop: the same 24-swipe/6-cycle scroll (900px over 200ms, 500ms apart, animated in ~60Hz steps rather than jumped so real frames render along the way), then the same 400-event/20s streaming phase pinned to the newest end, through fold_event -- the same fold path a live SSE reply uses.
4. The report appears in the text area under the buttons (it is a selectable field, so it can be copied by hand too) and is logged under logcat's iris-android-app tag on the line starting iris bench report:. Tap "Copy report" to put it on the clipboard through the shell's own ClipboardManager (no toast -- watch logcat or the on-screen text for confirmation it worked).
5. Paste it back. Same units, most lines the same shape as Compose's -- frame count, janky percentage, p50/p90/p99/worst -- plus iris's own CPU/GPU split (cpu_p50/gpu_wait_p50, from FrameReport::record_split) where Compose's report has its draw-phase accounting instead, and a bench:-shaped tail with the same three added fields: process CPU time over the run (getrusage(RUSAGE_SELF)), peak RSS (/proc/self/status's VmHWM), and battery current sampled once a second through BatteryManager.getIntProperty(BATTERY_PROPERTY_CURRENT_NOW) -- "no frames recorded" / "unavailable" rather than a fabricated number wherever the phone can't answer one, same rule Compose's report follows.
6. "Run benchmark" can be pressed again; it resets FrameReport first, so a second run's report covers only that run.

Built with iris/android-app/build-apk.sh release --abi arm64-v8a (wraps cargo ndk -t arm64-v8a -P 26 -o app/src/main/jniLibs/ build --release --features "transcript-screen bench" plus gradle :app:assembleRelease, signed with the same ~/.config/ai-app/release.jks app/build-apk.sh generates, and verifies the result with aapt2/apksigner). No longer built with force-gles as of the 2026-09-06 rebuild below -- see that entry for why it was there and why it had to come out for a phone build specifically.

2026-09-06, commit 46d3a6f: rebuilt after transcript_ui::TranscriptScreen::apply replaced the full-rebuild-per-event streaming path in all three iris clients (docs/RUST.md's P0 box has the before/after numbers) -- this is the build to use if comparing against an iris APK from before that commit.

2026-09-06, second rebuild, ai-app-2 commit 46246ea: this build aborted on Iris's phone (a Pixel, GrapheneOS) in AndroidRenderer::new, on the very first surface_changed, with only wgpu error: Validation Error surviving into the crash report before Android truncated it. Two things changed:

1. A renderer-creation failure no longer aborts the process. iris_core::UiRenderNode::new now runs its bind-group-layout/pipeline creation inside wgpu error scopes and returns Result<Self, String> instead of letting wgpu's default handler panic; the Android backend turns a failure into the adapter's identity, the exact limits and downlevel flags bind-group layouts validate against, and wgpu's own "Caused by" chain, logged as one line under the iris-android-app logcat tag and shown on screen as plain, selectable, scrollable text (a TextView swapped in for the whole activity) saying to copy it and send it back. If this build still fails on the phone, that screen is what to screenshot or copy -- there is no more silent abort to chase through a truncated crash report.
2. This build no longer forces the GLES backend. Every earlier iris-bench-arm64.apk (including the one above) was built with --features "... force-gles ...", a flag that exists only to force the emulator off its default software Vulkan and onto GLES for one specific frame-time measurement (RUST.md's I5) -- its own doc in iris/Cargo.toml never mentions real hardware. build-apk.sh's default feature list carried it into every arm64 build regardless, so the APK actually delivered to a phone was locked to GLES rather than the phone's own Vulkan driver. That is the named hypothesis for the crash (docs/RUST.md's P0 box has the full audit): GLES support for a storage buffer bound in the vertex stage (move_offsets, masks_layout) depends on the driver reporting GL_MAX_VERTEX_SHADER_STORAGE_BLOCKS > 0, which real Vulkan grants unconditionally but a phone's GLES path is not guaranteed to -- and RUST.md's own SwiftShader finding already flagged GLES-on-Android as the fragile backend for this exact shader. This build uses the default backend (Vulkan on a real device) instead.

If this rebuild still crashes, the on-screen report from point 1 names the real cause directly; paste it back rather than guessing further from a truncated log.

2026-09-06, later: bundled fonts, content-scale fix, Diagnostics control

Iris's first real-phone report on the previous rebuild (no crash) found four defects the emulator never showed: every glyph disappearing on the first tap or scroll (rectangles stay drawn -- not root-caused this pass, but a phone report from this build's new Diagnostics page would help), bold words rendering as blank gaps (fixed -- bundled Noto Sans/Noto Sans Mono, static per-style faces, ahead of the platform's own fonts), text far too small (fixed -- iris had no device-pixel-ratio handling on either platform before this; density now divides window size, touch coordinates and the shader's window uniform down to logical units), and the top buttons sitting under the status bar (fixed -- a new AndroidAppState::on_insets_changed hook, unused until now).

This build also adds a third Diagnostics button next to Run/Copy, filling the report area with adapter identity, font resolution, the atlas's view count, every wgpu error seen since surface creation, and the frame report -- "copy this and send it to Iris" via the existing Copy button.

Full account, including what's still unverified on a real phone, in docs/RUST.md's P0 box in the ai-app-2 repo (rustify branch).

2026-09-06, later still: keyboard-wipe fix, a density unit for crisp text, header background

Iris's report on the previous rebuild, with screenshots: text now the right size but blurry; opening the keyboard still wipes every glyph (rectangles stay, only text disappears); the header buttons have nothing behind them and overlap the transcript text.

• Keyboard wipe: surface_changed fires on every surface resize, not only a genuinely new surface -- showing the keyboard under adjustResize is exactly such a resize. It was rebuilding the whole renderer (a fresh, empty glyph atlas) every time, while the CPU-side glyph cache kept coordinates into the old atlas. Fixed: reuse the existing renderer's device/atlas/buffers and only reconfigure the surface when one is already live. Confirmed on this checkout's own emulator two ways (a forced display resize and an actual soft-keyboard open) -- text stayed on screen both times.
• Blur: the previous fix's global "logical size" stretch (window divided by density, then displayed stretched back up) rasterised each glyph small and displayed it bigger than it had texels for. Fixed with the density-independent length unit Iris asked for the same day (a new dp alongside the existing physical-pixel and relative lengths, resolved against the display's density at layout time) -- text now rasterises at the display's real physical resolution.
• Header: only each button's own coloured rect painted anything, so gaps between/around them showed the black clear colour one layer back, and the row's reserved height did not match the (now density-correct) transcript below it. Fixed with an opaque panel behind the whole row and every header size moved onto the new dp unit.
• New: the bench build now auto-captures a diagnostics report ~500ms after the keyboard opens (logged, copied to the clipboard, and shown in a plain overlay with Copy/Close over the running app) -- so a report can come back even if a keyboard-triggered regression is still happening.

Not fully verified. This build has not been run on Iris's actual phone. It was verified end to end on this checkout's own emulator (run-bench.sh, no crash, 24/24 swipes, 400/400 streamed events; before/after screenshots for the two keyboard-resize tests above). While verifying, a separate, not-yet-fixed issue turned up: after a keyboard-triggered resize, the header row appeared to render a second time, out of place, inside the transcript area, and a tap meant for the message field below landed on a header button instead (a benchmark run started unexpectedly). Not root-caused, and not confirmed to be caused by this pass's changes rather than something pre-existing -- worth watching for on the phone and reporting back if seen. Full account in docs/RUST.md's P0 box in the ai-app-2 repo (rustify branch).