The two P0 benchmark APKs with a Dev Updater config
This commit is contained in:
commit
5d7dee5d1f
4 files changed
+112
No files matched your search
@@ -0,0 +1,26 @@
|
||||
// Read by Dev Updater: the body of the config, no outer parentheses.
|
||||
//
|
||||
// This repo holds nothing but two prebuilt APKs -- the phone benchmark
|
||||
// gate from ai-app's RUST.md (P0) -- so each component's "build" is `true`:
|
||||
// the APK is already under the Gradle-shaped path discovery looks in
|
||||
// (`<module>/build/outputs/apk/<variant>/*.apk`), and there is nothing to
|
||||
// run. A new build is delivered by committing a new APK here.
|
||||
|
||||
label: "AI Sessions bench",
|
||||
|
||||
components: [
|
||||
// The Compose app's `bench` build type: com.example.aiapp.bench,
|
||||
// installs beside the real app, opens an embedded fixture, no server.
|
||||
Apk(
|
||||
name: "compose-bench",
|
||||
build: "true",
|
||||
cwd: "compose",
|
||||
),
|
||||
// The iris transcript screen's bench build: dev.iris.android.demo.bench,
|
||||
// same fixture, same scripted run, same report fields.
|
||||
Apk(
|
||||
name: "iris-bench",
|
||||
build: "true",
|
||||
cwd: "iris",
|
||||
),
|
||||
]
|
||||
@@ -0,0 +1,86 @@
|
||||
# ai-app bench
|
||||
|
||||
The two APKs for the phone benchmark gate (ai-app's `docs/RUST.md`, P0),
|
||||
packaged for Dev Updater so they can be installed on a phone without a
|
||||
build machine. Add this checkout as a project in Dev Updater (its
|
||||
`.dev-updater.ron` names two APK components, `compose-bench` and
|
||||
`iris-bench`, with no build step -- the APKs are committed under
|
||||
`compose/` and `iris/`), then install each from its card. Both install
|
||||
beside the real app under their own application ids and need no server,
|
||||
no enrolment and no permissions.
|
||||
|
||||
The rest of this file is the runbook that was delivered with the APKs.
|
||||
|
||||
---
|
||||
|
||||
# P0 benchmark APKs
|
||||
|
||||
Two arm64 builds, asked for in docs/RUST.md's P0 box (the phone benchmark gate Iris asked for
|
||||
2026-09-05, "before P1 I'd like to see benchmarks & also maybe stress test on my own phone"). Each
|
||||
opens straight onto the same synthetic fixture transcript with no server, runs the same scripted
|
||||
scroll-and-stream benchmark, and produces a text report to paste back. **Install both, run both,
|
||||
paste both reports back** -- the gate is a comparison, not either number alone.
|
||||
|
||||
## Compose (`compose-bench-arm64.apk`)
|
||||
|
||||
1. Install it (it has its own application id, `com.example.aiapp.bench`, and its own label "AI
|
||||
Sessions bench" -- it installs beside a real enrolled "AI Sessions" without touching it).
|
||||
2. Open it. It goes straight to a session screen holding the fixture transcript -- no enrollment,
|
||||
no permission prompts.
|
||||
3. Tap the gear icon top-right ("Session settings").
|
||||
4. Tap "Run benchmark". It scrolls the transcript (the same 24-swipe loop
|
||||
`app/transcript-bench.sh` drives on a phone from a computer), then streams 400 more fixture
|
||||
events in over 20 seconds pinned to the newest end (the same shape `app/stream-bench.sh`
|
||||
measures), then copies a report to the clipboard and shows a toast.
|
||||
5. Paste the clipboard back here (or in whatever you're pasting reports into). It is plain text
|
||||
with units on every number: frame counts and percentiles, the draw-phase accounting, and a
|
||||
`bench:` section this build added -- process CPU time, peak RSS, and battery current sampled
|
||||
once a second (says "unavailable" rather than a fabricated number where the phone can't answer,
|
||||
per this project's rule against showing an inferred value as a measured one).
|
||||
6. If you want to run it again, "Run benchmark" can be pressed more than once -- each press resets
|
||||
the counters first, so the report only covers that run's own scroll and stream.
|
||||
|
||||
Built from `app/androidApp`'s `bench` build type; see `app/build-apk.sh bench` and
|
||||
`app/bench-fixture/README.md` for how the fixture it opens was generated (deterministic, checked
|
||||
into the repo, never a real transcript).
|
||||
|
||||
## iris (`iris-bench-arm64.apk`)
|
||||
|
||||
1. Install it -- own application id, `dev.iris.android.demo.bench` (an `applicationIdSuffix` on
|
||||
the same `dev.iris.android.demo` the plain iris tabs demo uses, so it installs beside that one
|
||||
too), needs no permissions beyond `INTERNET` (unused here -- inherited from the same manifest
|
||||
`transcript-screen` needs, harmless for a build that talks to nothing).
|
||||
2. Open it. Same fixture as the Compose build (the identical checked-in
|
||||
`app/bench-fixture/assets/transcript.jsonl`), opened with no server through `client-core`'s real
|
||||
`fold_page` -- there is no enrollment step here at all, iris's shell has none yet.
|
||||
3. Tap "Run benchmark" (the whole screen is just two buttons above the transcript and a report
|
||||
area below it -- no settings dialog to find it in). It runs the identical scripted loop: the
|
||||
same 24-swipe/6-cycle scroll (900px over 200ms, 500ms apart, animated in ~60Hz steps rather than
|
||||
jumped so real frames render along the way), then the same 400-event/20s streaming phase pinned
|
||||
to the newest end, through `fold_event` -- the same fold path a live SSE reply uses.
|
||||
4. The report appears in the text area under the buttons (it is a selectable field, so it can be
|
||||
copied by hand too) and is logged under logcat's `iris-android-app` tag on the line starting
|
||||
`iris bench report:`. Tap "Copy report" to put it on the clipboard through the shell's own
|
||||
`ClipboardManager` (no toast -- watch logcat or the on-screen text for confirmation it worked).
|
||||
5. Paste it back. Same units, most lines the same shape as Compose's -- frame count, janky
|
||||
percentage, p50/p90/p99/worst -- plus iris's own CPU/GPU split (`cpu_p50`/`gpu_wait_p50`, from
|
||||
`FrameReport::record_split`) where Compose's report has its draw-phase accounting instead, and a
|
||||
`bench:`-shaped tail with the same three added fields: process CPU time over the run
|
||||
(`getrusage(RUSAGE_SELF)`), peak RSS (`/proc/self/status`'s `VmHWM`), and battery current sampled
|
||||
once a second through `BatteryManager.getIntProperty(BATTERY_PROPERTY_CURRENT_NOW)` -- "no
|
||||
frames recorded" / "unavailable" rather than a fabricated number wherever the phone can't answer
|
||||
one, same rule Compose's report follows.
|
||||
6. "Run benchmark" can be pressed again; it resets `FrameReport` first, so a second run's report
|
||||
covers only that run.
|
||||
|
||||
Built with `iris/android-app/build-apk.sh release --abi arm64-v8a` (wraps `cargo ndk -t arm64-v8a
|
||||
-P 26 -o app/src/main/jniLibs/ build --release --features "transcript-screen force-gles bench"`
|
||||
plus `gradle :app:assembleRelease`, signed with the same `~/.config/ai-app/release.jks`
|
||||
`app/build-apk.sh` generates, and verifies the result with `aapt2`/`apksigner`). `force-gles`
|
||||
matches I5's own "Where iris's frame time goes" finding: the default Vulkan backend has no adapter
|
||||
under a plain `-gpu host` boot on this AVD.
|
||||
|
||||
**2026-09-06, commit `46d3a6f`**: rebuilt after `transcript_ui::TranscriptScreen::apply` replaced
|
||||
the full-rebuild-per-event streaming path in all three iris clients (docs/RUST.md's P0 box has the
|
||||
before/after numbers) -- this is the build to use if comparing against an iris APK from before that
|
||||
commit.
|
||||
Binary file not shown.
Binary file not shown.
Reference in new issue
Block a user