commit 5d7dee5d1ff51364f8d05631aa42f4b8b5ac7f70 Author: iris <2+iris@noreply.localhost> Date: Sat Sep 5 22:33:09 2026 -0400 The two P0 benchmark APKs with a Dev Updater config diff --git a/.dev-updater.ron b/.dev-updater.ron new file mode 100644 index 0000000..ca3e9fe --- /dev/null +++ b/.dev-updater.ron @@ -0,0 +1,26 @@ +// Read by Dev Updater: the body of the config, no outer parentheses. +// +// This repo holds nothing but two prebuilt APKs -- the phone benchmark +// gate from ai-app's RUST.md (P0) -- so each component's "build" is `true`: +// the APK is already under the Gradle-shaped path discovery looks in +// (`/build/outputs/apk//*.apk`), and there is nothing to +// run. A new build is delivered by committing a new APK here. + +label: "AI Sessions bench", + +components: [ + // The Compose app's `bench` build type: com.example.aiapp.bench, + // installs beside the real app, opens an embedded fixture, no server. + Apk( + name: "compose-bench", + build: "true", + cwd: "compose", + ), + // The iris transcript screen's bench build: dev.iris.android.demo.bench, + // same fixture, same scripted run, same report fields. + Apk( + name: "iris-bench", + build: "true", + cwd: "iris", + ), +] diff --git a/README.md b/README.md new file mode 100644 index 0000000..d665097 --- /dev/null +++ b/README.md @@ -0,0 +1,86 @@ +# ai-app bench + +The two APKs for the phone benchmark gate (ai-app's `docs/RUST.md`, P0), +packaged for Dev Updater so they can be installed on a phone without a +build machine. Add this checkout as a project in Dev Updater (its +`.dev-updater.ron` names two APK components, `compose-bench` and +`iris-bench`, with no build step -- the APKs are committed under +`compose/` and `iris/`), then install each from its card. Both install +beside the real app under their own application ids and need no server, +no enrolment and no permissions. + +The rest of this file is the runbook that was delivered with the APKs. + +--- + +# P0 benchmark APKs + +Two arm64 builds, asked for in docs/RUST.md's P0 box (the phone benchmark gate Iris asked for +2026-09-05, "before P1 I'd like to see benchmarks & also maybe stress test on my own phone"). Each +opens straight onto the same synthetic fixture transcript with no server, runs the same scripted +scroll-and-stream benchmark, and produces a text report to paste back. **Install both, run both, +paste both reports back** -- the gate is a comparison, not either number alone. + +## Compose (`compose-bench-arm64.apk`) + +1. Install it (it has its own application id, `com.example.aiapp.bench`, and its own label "AI + Sessions bench" -- it installs beside a real enrolled "AI Sessions" without touching it). +2. Open it. It goes straight to a session screen holding the fixture transcript -- no enrollment, + no permission prompts. +3. Tap the gear icon top-right ("Session settings"). +4. Tap "Run benchmark". It scrolls the transcript (the same 24-swipe loop + `app/transcript-bench.sh` drives on a phone from a computer), then streams 400 more fixture + events in over 20 seconds pinned to the newest end (the same shape `app/stream-bench.sh` + measures), then copies a report to the clipboard and shows a toast. +5. Paste the clipboard back here (or in whatever you're pasting reports into). It is plain text + with units on every number: frame counts and percentiles, the draw-phase accounting, and a + `bench:` section this build added -- process CPU time, peak RSS, and battery current sampled + once a second (says "unavailable" rather than a fabricated number where the phone can't answer, + per this project's rule against showing an inferred value as a measured one). +6. If you want to run it again, "Run benchmark" can be pressed more than once -- each press resets + the counters first, so the report only covers that run's own scroll and stream. + +Built from `app/androidApp`'s `bench` build type; see `app/build-apk.sh bench` and +`app/bench-fixture/README.md` for how the fixture it opens was generated (deterministic, checked +into the repo, never a real transcript). + +## iris (`iris-bench-arm64.apk`) + +1. Install it -- own application id, `dev.iris.android.demo.bench` (an `applicationIdSuffix` on + the same `dev.iris.android.demo` the plain iris tabs demo uses, so it installs beside that one + too), needs no permissions beyond `INTERNET` (unused here -- inherited from the same manifest + `transcript-screen` needs, harmless for a build that talks to nothing). +2. Open it. Same fixture as the Compose build (the identical checked-in + `app/bench-fixture/assets/transcript.jsonl`), opened with no server through `client-core`'s real + `fold_page` -- there is no enrollment step here at all, iris's shell has none yet. +3. Tap "Run benchmark" (the whole screen is just two buttons above the transcript and a report + area below it -- no settings dialog to find it in). It runs the identical scripted loop: the + same 24-swipe/6-cycle scroll (900px over 200ms, 500ms apart, animated in ~60Hz steps rather than + jumped so real frames render along the way), then the same 400-event/20s streaming phase pinned + to the newest end, through `fold_event` -- the same fold path a live SSE reply uses. +4. The report appears in the text area under the buttons (it is a selectable field, so it can be + copied by hand too) and is logged under logcat's `iris-android-app` tag on the line starting + `iris bench report:`. Tap "Copy report" to put it on the clipboard through the shell's own + `ClipboardManager` (no toast -- watch logcat or the on-screen text for confirmation it worked). +5. Paste it back. Same units, most lines the same shape as Compose's -- frame count, janky + percentage, p50/p90/p99/worst -- plus iris's own CPU/GPU split (`cpu_p50`/`gpu_wait_p50`, from + `FrameReport::record_split`) where Compose's report has its draw-phase accounting instead, and a + `bench:`-shaped tail with the same three added fields: process CPU time over the run + (`getrusage(RUSAGE_SELF)`), peak RSS (`/proc/self/status`'s `VmHWM`), and battery current sampled + once a second through `BatteryManager.getIntProperty(BATTERY_PROPERTY_CURRENT_NOW)` -- "no + frames recorded" / "unavailable" rather than a fabricated number wherever the phone can't answer + one, same rule Compose's report follows. +6. "Run benchmark" can be pressed again; it resets `FrameReport` first, so a second run's report + covers only that run. + +Built with `iris/android-app/build-apk.sh release --abi arm64-v8a` (wraps `cargo ndk -t arm64-v8a +-P 26 -o app/src/main/jniLibs/ build --release --features "transcript-screen force-gles bench"` +plus `gradle :app:assembleRelease`, signed with the same `~/.config/ai-app/release.jks` +`app/build-apk.sh` generates, and verifies the result with `aapt2`/`apksigner`). `force-gles` +matches I5's own "Where iris's frame time goes" finding: the default Vulkan backend has no adapter +under a plain `-gpu host` boot on this AVD. + +**2026-09-06, commit `46d3a6f`**: rebuilt after `transcript_ui::TranscriptScreen::apply` replaced +the full-rebuild-per-event streaming path in all three iris clients (docs/RUST.md's P0 box has the +before/after numbers) -- this is the build to use if comparing against an iris APK from before that +commit. diff --git a/compose/build/outputs/apk/release/compose-bench-arm64.apk b/compose/build/outputs/apk/release/compose-bench-arm64.apk new file mode 100644 index 0000000..8fa2ed6 Binary files /dev/null and b/compose/build/outputs/apk/release/compose-bench-arm64.apk differ diff --git a/iris/build/outputs/apk/release/iris-bench-arm64.apk b/iris/build/outputs/apk/release/iris-bench-arm64.apk new file mode 100644 index 0000000..13e7c48 Binary files /dev/null and b/iris/build/outputs/apk/release/iris-bench-arm64.apk differ