87 lines
5.8 KiB
Markdown
87 lines
5.8 KiB
Markdown
# ai-app bench
|
|
|
|
The two APKs for the phone benchmark gate (ai-app's `docs/RUST.md`, P0),
|
|
packaged for Dev Updater so they can be installed on a phone without a
|
|
build machine. Add this checkout as a project in Dev Updater (its
|
|
`.dev-updater.ron` names two APK components, `compose-bench` and
|
|
`iris-bench`, with no build step -- the APKs are committed under
|
|
`compose/` and `iris/`), then install each from its card. Both install
|
|
beside the real app under their own application ids and need no server,
|
|
no enrolment and no permissions.
|
|
|
|
The rest of this file is the runbook that was delivered with the APKs.
|
|
|
|
---
|
|
|
|
# P0 benchmark APKs
|
|
|
|
Two arm64 builds, asked for in docs/RUST.md's P0 box (the phone benchmark gate Iris asked for
|
|
2026-09-05, "before P1 I'd like to see benchmarks & also maybe stress test on my own phone"). Each
|
|
opens straight onto the same synthetic fixture transcript with no server, runs the same scripted
|
|
scroll-and-stream benchmark, and produces a text report to paste back. **Install both, run both,
|
|
paste both reports back** -- the gate is a comparison, not either number alone.
|
|
|
|
## Compose (`compose-bench-arm64.apk`)
|
|
|
|
1. Install it (it has its own application id, `com.example.aiapp.bench`, and its own label "AI
|
|
Sessions bench" -- it installs beside a real enrolled "AI Sessions" without touching it).
|
|
2. Open it. It goes straight to a session screen holding the fixture transcript -- no enrollment,
|
|
no permission prompts.
|
|
3. Tap the gear icon top-right ("Session settings").
|
|
4. Tap "Run benchmark". It scrolls the transcript (the same 24-swipe loop
|
|
`app/transcript-bench.sh` drives on a phone from a computer), then streams 400 more fixture
|
|
events in over 20 seconds pinned to the newest end (the same shape `app/stream-bench.sh`
|
|
measures), then copies a report to the clipboard and shows a toast.
|
|
5. Paste the clipboard back here (or in whatever you're pasting reports into). It is plain text
|
|
with units on every number: frame counts and percentiles, the draw-phase accounting, and a
|
|
`bench:` section this build added -- process CPU time, peak RSS, and battery current sampled
|
|
once a second (says "unavailable" rather than a fabricated number where the phone can't answer,
|
|
per this project's rule against showing an inferred value as a measured one).
|
|
6. If you want to run it again, "Run benchmark" can be pressed more than once -- each press resets
|
|
the counters first, so the report only covers that run's own scroll and stream.
|
|
|
|
Built from `app/androidApp`'s `bench` build type; see `app/build-apk.sh bench` and
|
|
`app/bench-fixture/README.md` for how the fixture it opens was generated (deterministic, checked
|
|
into the repo, never a real transcript).
|
|
|
|
## iris (`iris-bench-arm64.apk`)
|
|
|
|
1. Install it -- own application id, `dev.iris.android.demo.bench` (an `applicationIdSuffix` on
|
|
the same `dev.iris.android.demo` the plain iris tabs demo uses, so it installs beside that one
|
|
too), needs no permissions beyond `INTERNET` (unused here -- inherited from the same manifest
|
|
`transcript-screen` needs, harmless for a build that talks to nothing).
|
|
2. Open it. Same fixture as the Compose build (the identical checked-in
|
|
`app/bench-fixture/assets/transcript.jsonl`), opened with no server through `client-core`'s real
|
|
`fold_page` -- there is no enrollment step here at all, iris's shell has none yet.
|
|
3. Tap "Run benchmark" (the whole screen is just two buttons above the transcript and a report
|
|
area below it -- no settings dialog to find it in). It runs the identical scripted loop: the
|
|
same 24-swipe/6-cycle scroll (900px over 200ms, 500ms apart, animated in ~60Hz steps rather than
|
|
jumped so real frames render along the way), then the same 400-event/20s streaming phase pinned
|
|
to the newest end, through `fold_event` -- the same fold path a live SSE reply uses.
|
|
4. The report appears in the text area under the buttons (it is a selectable field, so it can be
|
|
copied by hand too) and is logged under logcat's `iris-android-app` tag on the line starting
|
|
`iris bench report:`. Tap "Copy report" to put it on the clipboard through the shell's own
|
|
`ClipboardManager` (no toast -- watch logcat or the on-screen text for confirmation it worked).
|
|
5. Paste it back. Same units, most lines the same shape as Compose's -- frame count, janky
|
|
percentage, p50/p90/p99/worst -- plus iris's own CPU/GPU split (`cpu_p50`/`gpu_wait_p50`, from
|
|
`FrameReport::record_split`) where Compose's report has its draw-phase accounting instead, and a
|
|
`bench:`-shaped tail with the same three added fields: process CPU time over the run
|
|
(`getrusage(RUSAGE_SELF)`), peak RSS (`/proc/self/status`'s `VmHWM`), and battery current sampled
|
|
once a second through `BatteryManager.getIntProperty(BATTERY_PROPERTY_CURRENT_NOW)` -- "no
|
|
frames recorded" / "unavailable" rather than a fabricated number wherever the phone can't answer
|
|
one, same rule Compose's report follows.
|
|
6. "Run benchmark" can be pressed again; it resets `FrameReport` first, so a second run's report
|
|
covers only that run.
|
|
|
|
Built with `iris/android-app/build-apk.sh release --abi arm64-v8a` (wraps `cargo ndk -t arm64-v8a
|
|
-P 26 -o app/src/main/jniLibs/ build --release --features "transcript-screen force-gles bench"`
|
|
plus `gradle :app:assembleRelease`, signed with the same `~/.config/ai-app/release.jks`
|
|
`app/build-apk.sh` generates, and verifies the result with `aapt2`/`apksigner`). `force-gles`
|
|
matches I5's own "Where iris's frame time goes" finding: the default Vulkan backend has no adapter
|
|
under a plain `-gpu host` boot on this AVD.
|
|
|
|
**2026-09-06, commit `46d3a6f`**: rebuilt after `transcript_ui::TranscriptScreen::apply` replaced
|
|
the full-rebuild-per-event streaming path in all three iris clients (docs/RUST.md's P0 box has the
|
|
before/after numbers) -- this is the build to use if comparing against an iris APK from before that
|
|
commit.
|