irisandClaude Fable 5.1
a27fbdb029
Ran three clean iris-scroll.sh passes on a cold -gpu host boot (all 24/24 swipes confirmed scrolling via clustered render() timestamps, not inferred from frame count) and retook the host-GPU table's iris row as a best-of-three. EMU_GPU=software + force-gles still cannot produce a GLES number on this hardware -- after the earlier compute-limit crash was fixed, device creation now aborts on max_storage_buffer_binding_size instead (SwiftShader ES 3.0 has no SSBOs, and shader.wgsl reads four var<storage> buffers unconditionally), so the SwiftShader-Vulkan-vs-GLES question is closed as structurally unanswerable rather than answered. A fresh cold-boot run-bench.sh reading for P0's bench build is in line with the earlier warm-AVD readings, closing that box's own caveat too. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
294 lines
19 KiB
Markdown
294 lines
19 KiB
Markdown
# Decisions taken for Iris to review
|
|
|
|
Short list of design choices made by the design agent without asking, so
|
|
they can be judged and reversed later. Detail lives in RUST.md (and IRIS.md
|
|
for iris API changes); this file is only the summary. Newest first. Items
|
|
marked **DEFERRED** are ones the agent chose not to decide alone.
|
|
|
|
## 2026-09-05
|
|
|
|
- **iris no longer asks every device for compute-shader limits it never
|
|
uses.** `adapter.request_device` (both `iris/src/android/render.rs` and
|
|
`iris/src/default/render.rs`) used `Limits::default()` plus an override
|
|
for `max_buffer_size`, and `Limits::default()` unconditionally requests
|
|
desktop-tier compute limits (`max_compute_workgroups_per_dimension:
|
|
65535`, per `wgpu_types`) even though nothing in `iris`/`iris-core`
|
|
creates a `ComputePipeline` or writes a `@compute` shader stage —
|
|
confirmed by grepping the whole tree, not assumed. That crashed
|
|
`request_device` outright on the Android emulator's software GL path
|
|
(`EMU_GPU=software`, `--features force-gles`): SwiftShader's GL reports
|
|
itself as OpenGL ES 3.0, which has no compute shaders at all, so the
|
|
adapter's real limit is 0 against the unconditional request for 65535 —
|
|
`RUST.md`'s "Software mode ... crashes for a third, different reason,"
|
|
2026-09-05, earlier today. The same would happen on any real
|
|
GLES-3.0-only Android device, not just the emulator. Fixed by a new
|
|
`iris_core::device_limits()` (`iris/core/src/render/mod.rs`), shared by
|
|
both platform backends so the two requests cannot drift, that zeros the
|
|
six `max_compute_*` fields explicitly rather than switching to a
|
|
downlevel `Limits` preset — `Limits::downlevel_webgl2_defaults()` was
|
|
considered and rejected: it also zeros
|
|
`max_storage_buffers_per_shader_stage`, and `shader.wgsl`'s vertex stage
|
|
reads four `var<storage>` buffers (rects, glyphs, masks, move_offsets),
|
|
so that preset would trade the compute crash for a bind-group-layout
|
|
one on the same downlevel hardware this is meant to support. No
|
|
capability check or fallback path was needed since nothing is being
|
|
disabled — the request is simply narrowed to what the pipeline actually
|
|
uses. `rigs/gpu-probe`'s own mirrored limits (it is deliberately its own
|
|
crate, not a workspace member, so it cannot call `device_limits()`
|
|
directly) were updated to match, and confirm `IRIS DEVICE: ok` against
|
|
this VM's own Vulkan and GL adapters. **Not verified this pass**: the
|
|
specific SwiftShader-ES-3.0 crash this fixes, on-device — the
|
|
`EMU_GPU=software` cold boot this needs would have force-restarted this
|
|
checkout's emulator while another session was actively running its own
|
|
app on it (`com.example.aiapp` had window focus at the time), so it was
|
|
left for a pass when the emulator is free rather than disrupting that
|
|
session. Everything reachable without the emulator is clean: `cargo
|
|
fmt`/`clippy --workspace --all-targets`/`test --workspace`, `cargo ndk
|
|
build`/`clippy` for `iris-android-app` with `force-gles`, and
|
|
`gpu-probe` against this VM's own Vulkan and GL(ES 3.2, which still has
|
|
compute and so would not have reproduced the crash even before this
|
|
fix — not a substitute for the real ES-3.0 test).
|
|
|
|
- **P0's Compose half is built and smoke-tested on the emulator** — the
|
|
`bench` build type, the shared `app/bench-fixture/` transcript, and an
|
|
in-process fake backend (`BenchFixture.kt`/`BenchNetwork.kt`) that
|
|
answers `TranscriptSource`/`EventStream` from an in-memory event log
|
|
instead of a real server, so the fold and paging under test are the real
|
|
ones. Full account, the smoke run's report, and what is deliberately
|
|
left (the iris half, the real on-phone runs) are in RUST.md's P0 box.
|
|
Not a decision to review so much as the gate itself now being runnable —
|
|
flagged here because it is the first half of something Iris explicitly
|
|
asked to see before P1.
|
|
|
|
- **P0's iris half is also built and smoke-tested on the emulator,
|
|
2026-09-05.** A new `bench` Cargo feature on `iris-android-app`, on top
|
|
of `transcript-screen`: the same checked-in fixture (`include_str!`, no
|
|
asset pipeline needed), the same 24-swipe scroll loop animated through
|
|
`List::scroll` and the same 400-event/20s streaming phase through
|
|
`fold_event`, "Run benchmark"/"Copy report" as named accessible
|
|
controls, and the same three added report fields (process CPU time,
|
|
peak RSS, battery current) via direct JNI calls
|
|
(`bench_jni.rs::PlatformHandle`) since `android_view` has no
|
|
`BatteryManager`/`ClipboardManager` wrapper of its own. One small public
|
|
API addition to get there: `AndroidAppState::platform_ready` (`IRIS.md`),
|
|
a default-no-op lifecycle hook handing an implementor a `JavaVM` +
|
|
`GlobalRef` it can call Java through from any thread. Packaged with a
|
|
new `release` build type on `iris-android-app`'s own Gradle project
|
|
(there was previously only `debug`), signed with the same key
|
|
`app/build-apk.sh` generates. Smoke run and the full report are in
|
|
RUST.md's P0 box; not attempted this pass: the real on-phone runs and
|
|
Iris's pass/fail call, which is the actual gate.
|
|
|
|
- **The intermittent touch-scroll dropout is root-caused and fixed: a
|
|
missed `ACTION_DOWN` hit-test, not the previously-suspected coalesced
|
|
first `ACTION_MOVE`.** Diagnosed by temporary logcat tracing of every
|
|
touch event, `DragArbiter` state transition and `Selection::drag`
|
|
dispatch (removed once confirmed), reproduced on this checkout's own
|
|
emulator against a real sandbox session. The trace showed the actual
|
|
mechanism: a gesture's `ACTION_DOWN` lands wherever the finger actually
|
|
is, which is not guaranteed to fall inside the same row-local sensor
|
|
region a later `ACTION_MOVE` in the same gesture lands in (a row's own
|
|
padding/gap, or its non-selectable sender-name header, is
|
|
pointer-transparent to `iris::sense::CursorSense`). When that happens,
|
|
the widget that ends up handling the gesture never saw `PressStart`, so
|
|
`DragArbiter` sits in `Idle` — which answers every subsequent frame with
|
|
`Undecided` and has no way to tell "no press is happening" from "a press
|
|
is happening but I missed its start," so it never recovers on its own
|
|
for the rest of that gesture. One real trace showed exactly this: touch
|
|
`Down`/`Move`/`Up` all delivered correctly, but zero `PressStart`
|
|
reaching the arbiter, `state=Idle` unchanged from first frame to last.
|
|
Fixed at the call site that has the context to recover
|
|
(`iris::transcript_ui::selection::Selection::drag`,
|
|
`iris/transcript-ui/src/selection.rs`): a new `DragArbiter::is_idle()`
|
|
(`iris/src/sense.rs`) lets it notice a `Pressing` frame arriving with the
|
|
arbiter still `Idle` — which can only mean a missed `PressStart`, since a
|
|
`Pressing` sense requires the button to genuinely be down — and start the
|
|
press there instead of where it was missed. Three new unit tests in
|
|
`sense.rs`'s `drag_arbiter_tests` and one in `transcript-ui`'s
|
|
`selection::tests` (the latter fails on the code before this fix).
|
|
Commit follows. Not the same failure the earlier pass's `DECISIONS.md`
|
|
DEFERRED item speculated about (a coalesced first `ACTION_MOVE` skipping
|
|
slop detection) — that hypothesis is now ruled out; the arbiter's own
|
|
slop/long-press logic was never wrong. RUST.md's I5 box,
|
|
"Touch-scroll dropout root-caused, 2026-09-05" has the full trace.
|
|
- **P0, a phone benchmark gate before any porting, asked for by Iris
|
|
2026-09-05**: "before P1 I'd like to see benchmarks & also maybe stress
|
|
test on my own phone ... If it doesn't match compose reasonably well then
|
|
I don't think I'd wanna continue." Design (RUST.md's P0 box has the
|
|
detail): the same embedded synthetic fixture in both apps with no server
|
|
needed; the same scripted scroll loop then a streaming phase, run
|
|
programmatically since the phone has no usable system tracing and no
|
|
agent can drive it; the same report from both (frames, janky %, p50/p90/
|
|
p99, process CPU time, peak RSS, battery current where readable) with a
|
|
copy button; the iris app under its own id and the Compose one as a new
|
|
`bench` build type with an id suffix, so neither replaces her production
|
|
install; two arm64 APKs plus instructions delivered under `~/host/bench/`.
|
|
The gate is hers: iris within a reasonable margin of Compose release on
|
|
p50, p99 and CPU time, no crashes, no visible stutter. If it fails, the
|
|
port stops.
|
|
- **The rest of the port is one UI crate, `iris/app-ui`, grown out of
|
|
`iris/transcript-ui` rather than started beside it.** It holds a
|
|
`Screen` enum plus a back stack — the Rust equivalent of `AppRoot.kt`'s
|
|
`when` — and `iris/desktop-app`/`iris/android-app` become thin entry
|
|
points over it. Chosen over a fresh crate because `transcript-ui`
|
|
already has the right generic shape (`Rsc: HasEvents` +
|
|
`Rsc::State: FocusHost`) and the `client-core`/`event-model` path
|
|
dependencies every later screen needs, so growing it in place is the
|
|
smaller diff. Platform-only code (notification service, share target,
|
|
QR scanner, Keystore token, deep-link enrolment) stays in the E3/E5
|
|
Java shell (`android-shell/` + `app/shellApp`) rather than moving into
|
|
this crate, since none of it is a screen. The Android APK is built by
|
|
`cargo xtask apk` (E5), merging the app-ui cdylib into the E3 shell so
|
|
there is one app rather than a demo shell plus a service shell.
|
|
`app/androidApp` (the Compose app) stays untouched and is the baseline
|
|
every step is measured against, until parity is reached (P7 decides
|
|
the switch, and is itself a load-bearing decision left to Iris). Order
|
|
is by risk to the daily-use path: session screen first (P1, where
|
|
every hard behaviour already lives), then the shell merge and a real
|
|
phone install (P2), then root tabs (P3), the explorer (P4),
|
|
settings/enrolment (P5), desktop parity (P6), and the cutover itself
|
|
(P7). Full plan: RUST.md's "The port, in order (decided 2026-09-05)".
|
|
- **iris gets its own measured frame report, rather than waiting on a
|
|
`dumpsys`/`gfxinfo` answer that cannot see a `SurfaceView`'s GPU-drawn
|
|
frames.** `iris_core::FrameReport` (`iris/core/src/render/frame_report.rs`)
|
|
times each frame's wall clock from the same point `render()`'s redraw
|
|
starts to just after `queue.submit` + `present()` — the span Compose's
|
|
own render report and `gfxinfo` both count — into a fixed 4096-entry
|
|
ring (no allocation per frame; `report()` is the only place that
|
|
allocates, and only on a button tap). The report gives total frames,
|
|
janky % over the same 16.7ms budget `gfxinfo` uses, P50/P90/P99 and the
|
|
worst, plus a reset. Exposed the way the Compose app's copy-button
|
|
report already is: two named controls ("Frame report", "Reset frame
|
|
report") on the transcript screen, tappable by accessibility name via
|
|
`ui-trace`, logging under this crate's fixed `android_logger` tag
|
|
(`iris-android-app`) so a script can grep `"iris frame report"` the way
|
|
`transcript-bench.sh` greps `"ai-app render report"`. The report's own
|
|
`Display` line says plainly that it measures up to the `present()` call
|
|
returning, not GPU/compositor completion — wgpu's `present()` is not
|
|
fenced against either, so presenting that span as "time to reach the
|
|
screen" would be a measured-looking number that is actually inferred,
|
|
which the standing UI rule forbids.
|
|
- **`ui-trace` gains a hold-then-drag gesture, additive, in
|
|
`emulator-tools`.** Neither of its two existing actions can produce
|
|
"hold stationary for `LONG_PRESS`, then move without lifting" — `tap`
|
|
has no hold and `swipe X1 Y1 X2 Y2 MS` interpolates motion across its
|
|
whole duration from t=0. A new action presses, waits, then moves to a
|
|
second point and releases as one continuous touch (raw
|
|
`sendevent`/`MotionEvent` injection, extending whatever mechanism the
|
|
existing `swipe` already uses), so `DragArbiter`'s pan-vs-select rule
|
|
(`iris/src/sense.rs`, already covered by 8 unit tests against a
|
|
synthetic clock) can finally be driven on a real device instead of only
|
|
in a test harness.
|
|
- **Touch drag on a transcript row follows Android's own rule**: a vertical
|
|
drag pans the list immediately; a stationary press held 500 ms starts a
|
|
text selection which further dragging extends; a horizontal drag while
|
|
something is already selected extends that selection without the wait.
|
|
One `DragArbiter` per list decides it (`iris/src/sense.rs`). Chosen over a
|
|
"text layer always wins" or "list always wins" rule because either loses
|
|
one of the two gestures a reader expects.
|
|
- **E4's desktop shape is a new `iris/desktop-app` crate**: a winit window
|
|
holding `transcript-ui`'s screen beside a session list, talking to a real
|
|
`ai-server` through `client-core`. It enrols by pasting the same
|
|
`aiapp://enroll?…` link a phone scans (`client-core::config::EnrolledServer`)
|
|
and keeps it owner-only under `$XDG_CONFIG_HOME/ai-app-desktop/`. The
|
|
pinned CA is a path given on the command line, not baked in. Chosen so
|
|
the phone and desktop share one enrolment format and no second one is
|
|
invented.
|
|
- **I5's Android integration extends `iris-android-app` (I2's shell)
|
|
behind a Cargo feature (`transcript-screen`), rather than a third
|
|
shell crate.** That project already has the Gradle module, the
|
|
`IrisView`/`MainActivity` Java, and the JNI registration; the only
|
|
thing a second screen needs on top is a different `AndroidAppState`,
|
|
the same axis `tabs_ui::build`/`transcript_ui::build` already vary
|
|
along on the winit side. `tabs-screen`/`transcript-screen` are
|
|
mutually exclusive and each pulls in only its own deps, so the plain
|
|
tabs build (I2/I4) is untouched.
|
|
- **Order of remaining work, updated 2026-09-05**: the two in-flight
|
|
pieces and I5's Android integration are all done; next is giving iris
|
|
its own frame-timing report so item 3 below can be decided by a number.
|
|
- **DECIDED by Iris, 2026-09-05: iris is the app's framework; Masonry was
|
|
the calibration.** Her words: "I think iris definitely makes more sense
|
|
based on the limitations we've found." The limitations: Masonry has no
|
|
touch scroll on Android (E2), no per-span rich text and no cross-row
|
|
selection on the pinned commit (E2), and its keyboard bridge is a TODO
|
|
(E1); iris carries the same screen under the Compose baseline on the
|
|
host GPU (p50 15.0 ms against Compose's 20.0 ms, RUST.md's I5 box). What
|
|
follows: the E-steps are closed as calibration, and the port proceeds
|
|
on iris — screens, the shell (E3/E5), and `client-core` underneath.
|
|
The item below is kept as the record of what she decided from.
|
|
- **Was DEFERRED — whether to commit to iris over Masonry for `ai-app`.**
|
|
Updated 2026-09-05 with the clean comparison the recommendation wanted:
|
|
same sandbox session content, same emulator, `EMU_GPU=software`, one
|
|
session. Headline numbers (RUST.md's I5 box, "Clean scroll comparison,
|
|
2026-09-05," has the full table and every caveat):
|
|
|
|
| app | build | frames | janky % | p50 | p90 | p99 | worst |
|
|
|---|---|---|---|---|---|---|---|
|
|
| Compose (in-app report) | debug | 1102 | 99.0% late | 33.8ms | 50.6ms | 79.5ms | -- |
|
|
| Compose (`dumpsys gfxinfo`) | debug | 1499 | 21.15% (95.66% legacy) | 32ms | 48ms | 150ms (p99) | -- |
|
|
| iris (`FrameReport`) | **release** | 299 | 94.65% | 79.1ms | 98.6ms | 117.8ms | 212.6ms |
|
|
| iris (`FrameReport`, repeat) | **release** | 233 | 94.42% | 109.3ms | 130.8ms | 147.1ms | 150.5ms |
|
|
|
|
**Not a clean apples-to-apples reading, stated plainly rather than
|
|
smoothed over**: iris had to be built **release** (debug `SIGSEGV`s on
|
|
this emulator's Vulkan loader, I4's finding) against Compose's mandated
|
|
**debug** build, so this asymmetry likely *understates* iris's gap
|
|
rather than the reverse; the three frame-time sources measure different
|
|
things (Compose's own phase accounting vs. Android's HWUI deadline-miss
|
|
definition vs. iris's redraw-start-to-present window, the last of which
|
|
`dumpsys gfxinfo` cannot see at all for iris's `SurfaceView`); and both
|
|
figures are emulator numbers under software rasterisation, which
|
|
Compose's *own* in-app report shows already costs 20-34ms/frame in
|
|
`swap`+`gpu` alone under this GPU mode, so a same-mode iris number well
|
|
above 16.7ms was expected going in for either app. A second pair under
|
|
`-gpu host` was not taken this pass. The earlier session's suspected
|
|
intermittent touch-delivery dropout was **not reproduced** this pass —
|
|
the zero-frame results this time traced to this pass's own script bug
|
|
(a `cd` that changed which emulator `ui-trace` targeted), not the
|
|
emulator; a CPU-load rise during the gesture was observed by a sampler
|
|
running throughout, but did not correlate with any failure, so the
|
|
original candidate is neither confirmed nor ruled out.
|
|
The choice in front of Iris, updated: decide now on the
|
|
structural-plus-functional case already made (iris works end-to-end
|
|
where Masonry's scroll gesture doesn't exist at all on Android) plus
|
|
this table — reading the two build profiles and three jank definitions
|
|
with the caveats above rather than as a single number — or ask for a
|
|
same-profile, same-GPU-mode rerun first. RUST.md's I5 box has the full
|
|
account.
|
|
|
|
**Updated 2026-09-05, the `-gpu host` pair taken.** Real GPU rendering
|
|
(`force-gles` -- the default Vulkan backend has no adapter at all under
|
|
plain host-GPU boot, confirmed by the exact `wgpu` error) reverses the
|
|
software-mode shape:
|
|
|
|
| app | build | GPU mode | frames | janky % | p50 | p90 | p99 | worst | cpu p50 | gpu-wait p50 |
|
|
|---|---|---|---|---|---|---|---|---|---|---|
|
|
| Compose (in-app report) | debug | host (virgl) | 1268 | 96.4% late | 20.0ms | 28.4ms | 37.7ms | -- | -- | -- |
|
|
| iris (`FrameReport`), **best of three, 2026-09-05** | release, `force-gles` | host (virgl) | 439 | 46.24% | 15.7ms | 23.3ms | 31.2ms | 57.4ms | 1.2ms | 13.2ms |
|
|
|
|
Under real GPU rendering iris's median frame is *faster* than
|
|
Compose's, not the 2-3x-slower shape the software-mode table shows. A
|
|
new split inside `FrameReport` (redraw-to-submit vs. submit-to-present,
|
|
commit `e2a1fad`) says why: iris's own CPU work per frame is a median
|
|
~1ms -- almost the entire frame is time spent handing the frame to the
|
|
driver, not in iris's layout/text/primitive code. This is consistent
|
|
with the earlier software-mode gap being mostly SwiftShader's CPU
|
|
rasterisation cost rather than an iris-specific slowness. **Still not
|
|
proof, and now closed as unanswerable rather than merely untaken**: a
|
|
same-mode software `force-gles` run to isolate the backend was retried
|
|
2026-09-05 after fixing the compute-limit crash the first attempt hit,
|
|
and hit a second, structural wall instead — SwiftShader's ES 3.0 GL
|
|
path has no storage-buffer capacity at all, and `shader.wgsl` reads
|
|
`var<storage>` buffers unconditionally, so reaching that path needs a
|
|
shader rewrite, not a limits fix (RUST.md's I5 box, "The three
|
|
remaining I5 verifications, closed 2026-09-05," item 2). The
|
|
intermittent touch-scroll dropout this pass also reproduced is
|
|
root-caused and fixed as of the same date (a missed `ACTION_DOWN` on a
|
|
row's padding/header left `DragArbiter` stuck in `Idle`); three clean
|
|
`iris-scroll.sh` runs post-fix each scrolled all 24/24 swipes, replacing
|
|
the single-attempt 62-frame reading this table used to carry. RUST.md's
|
|
I5 box, "Where iris's frame time goes, 2026-09-05, the `-gpu host`
|
|
pass," and "The three remaining I5 verifications, closed 2026-09-05,"
|
|
have the full account. The iris-vs-Masonry choice itself is still
|
|
Iris's to make.
|