RUST.md, IRIS.md, IRIS_TODO.md, DECISIONS.md: record I5's Android integration and measurements
I5's transcript screen now runs on-device against a real ai-server on iris-android-app's new transcript-screen feature (extends I2's shell rather than a third one), with real scrolling, real touch-drag panning and tap-by-name accessibility all confirmed by screenshot/log evidence. I4's own emulator-side check (tap-by-name on the tabs demo) closed the same session, so its box ticks [x] now. Still [~], not [x]: the render-time number RUST.md's recommendation wants for iris couldn't be produced this pass, for a precise and recorded reason rather than a vague one -- dumpsys gfxinfo cannot see a SurfaceView's own GPU-drawn frames at all (0 frames reported across a gesture loop that visibly scrolled), and a SurfaceFlinger --latency fallback gave no per-frame history either on this Android version. The Compose side of the same loop did produce a real number under identical conditions (8.96% janky, 99th percentile 150ms), so this is now a one-sided number rather than a missing one on both sides. Also found and recorded: the AVD's saved snapshot carries a GPU config across restarts, so switching between the documented Vulkan boot recipes needs a cold boot (clearing snapshots/) that the emu wrapper does not force -- cost three different-looking crashes before the pattern was the snapshot, not the code. DECISIONS.md's DEFERRED item is updated with the numbers Iris needs to weigh the iris-vs-Masonry call; the call itself stays hers. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
1 parent
bf5087a598
commit
d17040b601
4 files changed
+388
-85
No files matched your search
+28
-6
@@ -22,9 +22,31 @@ marked **DEFERRED** are ones the agent chose not to decide alone.
|
||||
pinned CA is a path given on the command line, not baked in. Chosen so
|
||||
the phone and desktop share one enrolment format and no second one is
|
||||
invented.
|
||||
- **Order of remaining work**: finish the two in-flight pieces above, then
|
||||
the transcript screen's Android integration and the `transcript-bench.sh`
|
||||
comparison against Compose — the numbers the recommendation still lacks.
|
||||
- **DEFERRED — whether to commit to iris over Masonry for `ai-app`.** Waits
|
||||
on the bench numbers above; RUST.md's recommendation says what the
|
||||
measurements must show.
|
||||
- **I5's Android integration extends `iris-android-app` (I2's shell)
|
||||
behind a Cargo feature (`transcript-screen`), rather than a third
|
||||
shell crate.** That project already has the Gradle module, the
|
||||
`IrisView`/`MainActivity` Java, and the JNI registration; the only
|
||||
thing a second screen needs on top is a different `AndroidAppState`,
|
||||
the same axis `tabs_ui::build`/`transcript_ui::build` already vary
|
||||
along on the winit side. `tabs-screen`/`transcript-screen` are
|
||||
mutually exclusive and each pulls in only its own deps, so the plain
|
||||
tabs build (I2/I4) is untouched.
|
||||
- **Order of remaining work, updated 2026-09-05**: the two in-flight
|
||||
pieces and I5's Android integration are all done; next is giving iris
|
||||
its own frame-timing report so item 3 below can be decided by a number.
|
||||
- **DEFERRED — whether to commit to iris over Masonry for `ai-app`.**
|
||||
Updated 2026-09-05 with what's now known: iris's Android integration is
|
||||
built and confirmed working on a real device (real server, real
|
||||
scrolling, real touch-drag pan, tap-by-name accessibility — RUST.md's I5
|
||||
box), while Masonry's own touch-scroll on Android was already found
|
||||
entirely absent (E2). What's still missing on **both** sides is a
|
||||
render-time number — not because iris doesn't work, but because
|
||||
`dumpsys gfxinfo` cannot see a `SurfaceView`'s GPU-drawn frames at all,
|
||||
so the `transcript-bench.sh`-style comparison this recommendation wanted
|
||||
produced a real number for Compose (8.96% janky, 99th percentile 150ms,
|
||||
same emulator/session) and none at all for iris. The choice in front of
|
||||
Iris: decide now on the structural-plus-functional case already made
|
||||
(iris works end-to-end where Masonry's scroll gesture doesn't exist at
|
||||
all on Android), or wait for iris to grow its own frame-timing
|
||||
instrumentation first so the comparison can be a number rather than a
|
||||
qualitative one. RUST.md's "Recommendation" item 3 has the full account.
|
||||
@@ -8,6 +8,36 @@ capability that moved. Small and trivial changes do not go here.
|
||||
An entry gives the date, what changed, why, and a short before/after where
|
||||
it helps judge the change without the session that made it. Newest first.
|
||||
|
||||
## 2026-09-05: `Tasks::redraw_handle` (RUST.md's I5 Android integration)
|
||||
|
||||
New public method on `iris::task::Tasks`, `redraw_handle(&self) ->
|
||||
Arc<dyn RequestRedraw>`. Why: a caller running its own long-lived loop
|
||||
*inside* one spawned task (a live SSE follow, the Android transcript
|
||||
client's `select_session`) has no other way to ask for a frame after each
|
||||
`TaskCtx::update` -- `Tasks::spawn`'s own wrapper only requests one, after
|
||||
the whole async closure finishes, which fits a single request-then-update
|
||||
but not a stream that needs to be seen redrawing after *each* event. This
|
||||
is the same gap `iris/desktop-app`'s module doc names for why it uses
|
||||
winit's `Proxy<AppEvent>` instead of `Tasks` -- android-view has no
|
||||
`Proxy`, so this is what closes it there.
|
||||
|
||||
**A real bug this uncovered, not a hypothetical**: calling the returned
|
||||
handle's `request_redraw()` from the background thread crashed the process
|
||||
(`SIGABRT`, `Result::unwrap() on an Err value: JavaException`) the first
|
||||
time an Android transcript fetch called it a second time. `android/render.rs`'s
|
||||
`AndroidRedrawHandle` was already attaching the calling thread to the JVM
|
||||
correctly, but its `request_redraw` called `View::post_frame_callback`,
|
||||
whose Java side calls `Choreographer.getInstance()` -- which throws unless
|
||||
the *calling* thread already has a `Looper`, and a tokio worker thread,
|
||||
even freshly JNI-attached, has none. Fixed by routing through
|
||||
`View::post_delayed(0)` instead (Android's own thread-safe "queue work onto
|
||||
this View's UI thread" primitive, needing no caller-side `Looper`), landing
|
||||
on a new `IrisViewPeer::delayed_callback` override that drains tasks and
|
||||
renders -- same body as `do_frame`, on the UI thread where
|
||||
`post_frame_callback` is safe again. Any future caller of `redraw_handle()`
|
||||
from a background thread gets this for free; nothing about the fix is
|
||||
specific to the transcript screen.
|
||||
|
||||
## 2026-09-05: `transcript_ui::build_tree` (RUST.md's E4)
|
||||
|
||||
`transcript_ui::build` claimed the whole window (`ui_state.set_root(tree)`)
|
||||
|
||||
+29
-8
@@ -176,14 +176,35 @@ order and what "done" looks like. Tick and date them in place.
|
||||
silently dropped. See RUST.md's I5 box for the full account of what
|
||||
*was* built (the screen, `SpanStyle`, cross-row selection, the growing
|
||||
composer).**
|
||||
- [ ] **Android integration for this screen does not exist yet.** No
|
||||
cdylib/Gradle shell the way `iris-android-app` wraps `tabs-ui` (I2),
|
||||
so `transcript-bench.sh`'s render-number pass condition against the
|
||||
Compose baseline cannot be run. Needs: real `client-core::ApiClient`/
|
||||
`event_stream::follow_session_events` wiring against
|
||||
`app/ui-sandbox.sh --delay` (this crate deliberately fetches nothing
|
||||
itself, `transcript-ui/src/lib.rs`'s doc), a new cdylib + Gradle
|
||||
module, then the bench script pointed at it.
|
||||
- [x] **Android integration for this screen — done, 2026-09-05.**
|
||||
`iris-android-app`'s `transcript-screen` Cargo feature
|
||||
(`transcript_client.rs`) runs this screen against a real `ai-server`
|
||||
through `client-core`, confirmed on-device: real scrolling, real
|
||||
touch-drag panning, tap-by-name on the composer. Two real bugs found
|
||||
and fixed along the way (a missing `INTERNET` permission; a
|
||||
background-thread redraw request that crashed via a `Looper`
|
||||
requirement, fixed by routing through `View::post_delayed` — see
|
||||
`IRIS.md`'s `Tasks::redraw_handle` entry). See RUST.md's I5 box,
|
||||
"The Android integration, done 2026-09-05" for the full account.
|
||||
- [ ] **A render-time number for iris, comparable to Compose's
|
||||
`transcript-bench.sh` report.** `dumpsys gfxinfo` cannot see a
|
||||
`SurfaceView`'s own GPU-drawn frames at all (confirmed: 0 frames
|
||||
reported across a gesture loop that visibly scrolled), and a
|
||||
`dumpsys SurfaceFlinger --latency` fallback returned no per-frame
|
||||
history either on this Android version's BLAST compositor. What's
|
||||
needed is frame-timing instrumentation inside iris itself — a report
|
||||
the way `UiRenderState::take_counters`/`AccessTree::take_rebuilds`
|
||||
already expose counters, extended to timing. This is the one number
|
||||
RUST.md's recommendation (item 3) is still waiting on.
|
||||
- [ ] **Long-press-then-drag-to-select was not independently driven
|
||||
on-device.** `ui-trace`'s two gesture primitives (`tap`, `swipe X1 Y1
|
||||
X2 Y2 MS`) cannot produce "hold stationary for `LONG_PRESS`, then drag
|
||||
without lifting" — `swipe` interpolates motion across its whole
|
||||
duration from the start. Needs either a new `ui-trace` action (a
|
||||
genuine hold-then-drag primitive) or raw multi-step `MotionEvent`
|
||||
injection. `DragArbiter`'s own unit tests already cover this exact
|
||||
sequence against a synthetic clock (`iris/src/sense.rs`), which is why
|
||||
this is "not independently driven on-device" rather than "unverified."
|
||||
- [x] **Touch-drag panning over a row's own rendered text — done,
|
||||
2026-09-05.** `row.rs` used to register `CursorSense::click_or_drag()`
|
||||
on each row's `TextEdit` for cross-row selection; `TextEdit::draw`'s
|
||||
|
||||
+301
-71
@@ -36,10 +36,26 @@ session spending an afternoon on them again.
|
||||
|
||||
## Where things stand (2026-09-05)
|
||||
|
||||
- **Both of the previous note's in-flight pieces are now done, 2026-09-05.**
|
||||
Design choices for both are summarised in `DECISIONS.md` at the repo
|
||||
root, which is the file Iris reads for choices made without her. Next:
|
||||
I5's Android integration and the bench numbers.
|
||||
- **I5's Android integration is done and measured, 2026-09-05 (still ticked
|
||||
`[~]`, not `[x]` -- see I5's own box for exactly why).** The transcript
|
||||
screen runs on-device against a real `ai-server`, with real scrolling,
|
||||
real touch-drag panning and tap-by-name accessibility all confirmed by
|
||||
screenshot/log evidence on this checkout's emulator. What the
|
||||
recommendation still lacks is a render-time number for iris comparable
|
||||
to Compose's `transcript-bench.sh` report -- not because the screen
|
||||
doesn't work, but because `dumpsys gfxinfo` cannot see a `SurfaceView`'s
|
||||
own GPU-drawn frames at all, and iris has no frame-timing
|
||||
instrumentation of its own to ask instead. Two real, previously-unknown
|
||||
bugs were found and fixed getting here (a missing `INTERNET` permission,
|
||||
and a background-thread redraw request that crashed the process via a
|
||||
`Looper` requirement neither this box nor `Tasks::redraw_handle`'s
|
||||
design had anticipated) -- both in I5's own box, both in `IRIS.md`.
|
||||
Next: give iris a render-time report of its own, the one piece left
|
||||
before the iris-vs-Masonry recommendation can be decided by a number
|
||||
rather than by structure alone.
|
||||
- **Design choices for the two pieces before this are summarised in
|
||||
`DECISIONS.md`** at the repo root, which is the file Iris reads for
|
||||
choices made without her.
|
||||
- **E4 done, 2026-09-05.** `iris/desktop-app`: a winit window with a
|
||||
session list beside `transcript-ui`'s screen (`build_tree`), against a
|
||||
real `ai-server` through `client-core`, enrolled from the same
|
||||
@@ -648,24 +664,47 @@ light" has a knob inside the same stack.
|
||||
framework and Masonry was the calibration. If it does not, the
|
||||
measurement says which parts of Masonry to adopt underneath it.
|
||||
|
||||
**Not decidable yet, 2026-09-05 — what's missing, named rather than
|
||||
guessed at.** Neither side of this comparison has a render number:
|
||||
E2 found Masonry's own scroll gesture path absent on Android
|
||||
entirely (its box, "measurable frames"), and I5 built the iris side of
|
||||
the screen (`iris/transcript-ui/`) but not the Android integration
|
||||
around it — no cdylib/Gradle shell exists for this screen yet (unlike
|
||||
`tabs-ui`'s `iris-android-app`, I2), so there is nothing installed on a
|
||||
device for `transcript-bench.sh` to measure against the Compose
|
||||
baseline. What would close this: build that integration (real
|
||||
`client-core` networking against `app/ui-sandbox.sh --delay`, a cdylib
|
||||
+ Gradle module the way I2 did for `tabs-ui`), then run
|
||||
`transcript-bench.sh`'s gesture on both. Until then, the decision rests
|
||||
on the structural findings both sides *did* produce: Masonry cannot do
|
||||
cross-row selection or per-span inline rich text at all today (E2's
|
||||
`grep -rln`, zero hits, cited in its own box), and iris now does both
|
||||
(I5's `SpanStyle` and `selection.rs`) as well as programmatic
|
||||
touch-scroll (I3) — three structural points in iris's favour with no
|
||||
opposing measurement yet on either side.
|
||||
**Still not decidable by a render-time number, 2026-09-05 (updated) —
|
||||
what's missing, named rather than guessed at, and now for a different
|
||||
reason than before.** E2 found Masonry's own scroll gesture path
|
||||
absent on Android entirely (its box, "measurable frames") — that has
|
||||
not changed. I5's Android integration is now built and confirmed
|
||||
working (real server, real scrolling, real touch-drag pan, tap-by-name
|
||||
— I5's own box, "Measurements taken"), so the earlier blocker ("no
|
||||
cdylib/Gradle shell exists for this screen") is gone. What replaced
|
||||
it: **`dumpsys gfxinfo`, the tool `transcript-bench.sh` and this
|
||||
recommendation both assumed would give the comparison, cannot see a
|
||||
`SurfaceView`'s own GPU-drawn frames at all** — it instruments
|
||||
Android's ordinary View/Skia drawing pipeline, which a `wgpu`-rendered
|
||||
`SurfaceView` (iris's whole approach) bypasses entirely. Confirmed
|
||||
0 frames reported across a 24-swipe gesture loop that visibly scrolled
|
||||
the screen (screenshots differ), and a `dumpsys SurfaceFlinger
|
||||
--latency` fallback returned no per-frame history either (just the
|
||||
display's refresh period) on this Android version's BLAST compositor.
|
||||
The Compose side of the same loop *did* produce a real number under
|
||||
identical conditions (`EMU_GPU=software`, same emulator, same session):
|
||||
**8.96% janky frames, 99th percentile 150ms.** So this is now a
|
||||
one-sided number, not a missing one — the number needed to close item 3
|
||||
is a render-time report from **iris itself** (the equivalent of the
|
||||
Compose app's in-app copy-button report `transcript-bench.sh` already
|
||||
reads), which does not exist yet and is real, scoped follow-on work
|
||||
(frame timing inside `iris_core::render`, exposed the way `AccessTree`
|
||||
or `UiRenderState::take_counters` already are) rather than a rerun of
|
||||
anything above. Until it exists, the decision still rests on the
|
||||
structural findings both sides *did* produce, now joined by a
|
||||
functional one: Masonry cannot do cross-row selection or per-span
|
||||
inline rich text at all today (E2's `grep -rln`, zero hits, cited in
|
||||
its own box); iris does both (I5's `SpanStyle` and `selection.rs`) and
|
||||
its touch-scroll now works end-to-end on a real device, not just
|
||||
programmatically (I3's benchmark plus I5's on-device screenshot
|
||||
evidence) — three structural points and one functional one in iris's
|
||||
favour, still with no opposing *or* supporting render-time measurement
|
||||
on either side. **What Iris needs to weigh this**: whether "iris works
|
||||
and Masonry's Android scroll path is absent entirely" is enough to
|
||||
decide without a number, or whether the frame-timing work above is
|
||||
worth doing first — a product/tradeoff call, not a technical one, so it
|
||||
is left to her rather than decided here (DECISIONS.md's DEFERRED
|
||||
item).
|
||||
4. Then the shell (E3), the desktop window (E4) and the packaging (E5),
|
||||
which do not depend on the choice.
|
||||
|
||||
@@ -2113,8 +2152,9 @@ silently on real hardware.
|
||||
box's scope could finish alone -- recorded here rather than left
|
||||
silently undone.
|
||||
- [x] **I4 — accessibility names via AccessKit, host half done and verified
|
||||
2026-09-05; the emulator half is the one step left, named at the
|
||||
bottom of this box.** Built `iris_core::ui::access::AccessTree`
|
||||
2026-09-05; the emulator half done and verified 2026-09-05, same day
|
||||
as I5's Android integration (see bottom of this box for the exact
|
||||
run).** Built `iris_core::ui::access::AccessTree`
|
||||
(`iris/core/src/ui/access.rs`) -- one flat AccessKit tree, a synthetic
|
||||
`Role::Window` root with every **named** widget as a direct child.
|
||||
Deliberately flat rather than mirroring iris's real widget nesting:
|
||||
@@ -2226,34 +2266,60 @@ silently on real hardware.
|
||||
than confirmed end-to-end the way the emulator step below confirms
|
||||
the Android path.
|
||||
|
||||
**What remains -- the one check that needs the emulator, held by
|
||||
another session during this pass.** `iris-android-app`'s tabs screen
|
||||
has never been driven by `ui-trace` for real; everything above is
|
||||
"builds, runs, produces the right data" on the host. Once the
|
||||
emulator is free:
|
||||
**Done, 2026-09-05, on this checkout's own emulator (`ai-app-2`,
|
||||
`EMU_GPU=software` -- see I5's box for why plain `-gpu host` and the
|
||||
documented Vulkan-feature recipe both could not be used here).**
|
||||
`cargo ndk -t x86_64 -P 26 -o app/src/main/jniLibs/ build --release &&
|
||||
gradle :app:assembleDebug`, installed, launched, then each of
|
||||
|
||||
cd iris/android-app && cargo ndk -t x86_64 -P 26 -o app/src/main/jniLibs/ build --release && gradle :app:assembleDebug
|
||||
adb install -r app/build/outputs/apk/debug/app-debug.apk
|
||||
# launch iris-android-app on the emulator, then:
|
||||
ui-trace record --do "tap 'pad'"
|
||||
ui-trace record --do "tap 'span'"
|
||||
ui-trace record --do "tap 'image span'"
|
||||
ui-trace record --do "tap 'text layout'"
|
||||
ui-trace record --do "tap 'text edit scroll'"
|
||||
|
||||
Pass condition: each tap resolves (uiautomator finds a node with
|
||||
that exact label) and switches `main`'s visible pane the way a
|
||||
direct touch on that button already does -- i.e. `bench-lib.sh`'s
|
||||
tap-by-name mechanism, unmodified, driving the iris screen instead
|
||||
of the Compose one. Also worth checking while the emulator is up,
|
||||
since E1 found it exactly this way: run a second `ui-trace record`
|
||||
immediately after the first (attach, detach, attach again) and
|
||||
confirm the process is still alive afterward -- the detach-abort
|
||||
this box's mitigation exists for.
|
||||
resolved (uiautomator found the exact label every time -- `ui-trace`
|
||||
never failed a run). **Confirming the pane actually switched needed
|
||||
more than `ui-trace show`**: the tabs row is the *only* named
|
||||
structure on this screen, its five buttons never move, so
|
||||
`--field box` reports "nothing moved" on every run whether the pane
|
||||
behind it changed or not -- a screenshot before/after is what showed
|
||||
it, `adb exec-out screencap -p`, hashed to confirm difference; also
|
||||
confirmed live with a temporary `log::debug!` in `switch_button`'s
|
||||
click closure (reverted before committing) showing the exact index
|
||||
clicked matching the tapped label. The detach-abort check also
|
||||
passed: six consecutive `ui-trace record` calls against the same
|
||||
process (attach, detach, attach again, five more times) left it
|
||||
alive throughout -- `adb shell dumpsys window` still showed
|
||||
`dev.iris.android.demo/.MainActivity` focused and rendering
|
||||
afterward, no crash in `logcat`.
|
||||
|
||||
**One real, unrelated bug found and fixed getting here, not part of
|
||||
I4's own design**: the release build was required -- a debug/dev
|
||||
profile build of this same APK reliably `SIGSEGV`s inside this
|
||||
emulator's Vulkan loader (`vulkan.ranchu.so`,
|
||||
`vk_common_SetDebugUtilsObjectNameEXT`) the moment `wgpu` creates its
|
||||
first bind group layout, because `wgpu`'s `InstanceFlags::
|
||||
from_build_config()` turns on debug object-labelling in a dev build,
|
||||
and labelling a `SwiftShader`-backed resource through this
|
||||
emulator's loader trampoline crashes. A release build's
|
||||
`InstanceFlags::empty()` never takes that path. Nothing in iris
|
||||
caused this and nothing here needed to change to avoid it --
|
||||
recorded because it looked exactly like a fresh regression the first
|
||||
time it was hit (mid-session, after adding an unrelated temporary
|
||||
log line forced a dev rebuild) and cost real time to separate from
|
||||
the actual touch-dispatch question being chased at the time.
|
||||
- [~] **I5 — the transcript screen in iris (2026-09-05). The widget-tree
|
||||
half is built, tested and screenshotted; the emulator half (real
|
||||
device numbers against the Compose baseline) is not -- ticked
|
||||
partial rather than done, see "What remains" at the end of this box.**
|
||||
half and the Android integration are both built and confirmed
|
||||
working on-device (real server, real scrolling, real touch-drag
|
||||
pan, tap-by-name); still `[~]` rather than `[x]` because the one
|
||||
thing the recommendation actually wants -- a render-time number for
|
||||
iris comparable to Compose's -- could not be produced on this pass,
|
||||
for a reason named precisely rather than left vague: `dumpsys
|
||||
gfxinfo` cannot see a `SurfaceView`'s own GPU-drawn frames at all,
|
||||
and iris has no frame-timing instrumentation of its own yet to ask
|
||||
instead. See "Measurements taken" and "What remains" near the end of
|
||||
this box.**
|
||||
|
||||
**Where it lives.** `iris/transcript-ui/` (new workspace member,
|
||||
`[lib]`), the same shape as `iris/tabs-ui`: generic over `Rsc:
|
||||
@@ -2464,24 +2530,186 @@ silently on real hardware.
|
||||
screenshots of real content leaving this repo -- these are
|
||||
synthetic rows, but the rule is kept uniform regardless.
|
||||
|
||||
**The Android integration, done 2026-09-05.** Extended
|
||||
`iris-android-app` (I2's shell) with a second, mutually-exclusive
|
||||
`AndroidAppState` behind a new Cargo feature rather than building a
|
||||
third shell -- see `iris/android-app/src/lib.rs`'s module doc for why
|
||||
that was chosen over a standalone crate: the Gradle project, the
|
||||
`IrisView`/`MainActivity` Java, and the `register_view_class` wiring
|
||||
I2 already built are exactly what a second screen needs too, and the
|
||||
only thing that differs is which `AndroidAppState` the JNI entry
|
||||
point instantiates. `transcript_client.rs` (new) fetches the sandbox
|
||||
server's session list, opens the first one, and follows it live --
|
||||
`client_core::api`/`event_stream`/`transcript_fold` almost verbatim
|
||||
from `desktop-app`'s `app.rs` (E4), down to the generation-guard
|
||||
pattern; `fold_page`/`raw_seq` were hoisted into `client-core` itself
|
||||
first so both callers share one copy rather than a second one being
|
||||
pasted in (a separate small commit, "write the logic once").
|
||||
Deliberately simplified, recorded rather than left to be
|
||||
rediscovered: no session list UI and no enrollment flow exist for
|
||||
this screen -- `build.rs` bakes the sandbox's host/port/token and the
|
||||
pinned CA in at build time from
|
||||
`AI_APP_TRANSCRIPT_HOST`/`_PORT`/`_TOKEN`/`AI_APP_CA` env vars, same
|
||||
trust-boundary reasoning as the Compose app's `GeneratePinnedCert`
|
||||
Gradle task (`app/androidApp/build.gradle.kts`), extended here to
|
||||
also bake the enrollment since building a real one is E3/E4's scope,
|
||||
not this box's. A real app needs `desktop-app`'s
|
||||
`EnrolledServer`/QR-link flow or E3's Keystore-sealed
|
||||
`ServerConfig.kt`.
|
||||
|
||||
**Two real bugs found and fixed getting an actual screen on
|
||||
screen, neither anticipated by this box's earlier design:**
|
||||
|
||||
1. **Missing `INTERNET` permission.** `iris-android-app`'s manifest
|
||||
never needed one before this screen (the tabs demo makes no
|
||||
network call), so nobody had noticed it was absent. Its absence
|
||||
reads nothing like a network problem: `UreqTransport::new`'s
|
||||
connect failed with `EPERM` ("Operation not permitted"), not the
|
||||
`ECONNREFUSED`/`ENETUNREACH` a dead server or a firewall would
|
||||
give -- a seccomp-level socket denial. Added
|
||||
`<uses-permission android:name="android.permission.INTERNET" />`
|
||||
with a comment naming the exact symptom, so the next person
|
||||
hitting `EPERM` from this codebase's own `ureq` stack finds the
|
||||
answer instead of debugging the server.
|
||||
2. **A background task's first redraw request past the initial one
|
||||
aborted the process.** `Tasks::redraw_handle()` (new, this box,
|
||||
see `IRIS.md`'s 2026-09-05 entry for the full account) exists so
|
||||
`transcript_client.rs` can ask for a frame after each
|
||||
`TaskCtx::update`, the way `desktop-app` uses winit's `Proxy` for
|
||||
the same reason. Calling it crashed with `SIGABRT`,
|
||||
`Result::unwrap() on an Err value: JavaException`, inside
|
||||
`android-view`'s `View::post_frame_callback` -- its Java side
|
||||
calls `Choreographer.getInstance()`, which throws unless the
|
||||
*calling* thread already has a `Looper`, and a tokio worker
|
||||
thread has none even once JNI-attached. Fixed by routing through
|
||||
`View::post_delayed(0)` instead (thread-safe, no `Looper`
|
||||
required) and a new `IrisViewPeer::delayed_callback` override
|
||||
(`android/view.rs`) that drains tasks and renders on the UI
|
||||
thread the callback lands on -- same body as `do_frame`. Every
|
||||
future caller of `redraw_handle()` from a background thread gets
|
||||
this for free.
|
||||
|
||||
**The emulator itself needed a boot recipe none of the three
|
||||
previously-documented ones give cleanly, found the hard way.**
|
||||
Plain `emu up` (`-gpu host`, no Vulkan feature) crashed instantly --
|
||||
`wgpu_core::instance: Request adapter didn't find compatible
|
||||
adapters` -- this AVD's default boot has no Vulkan device at all,
|
||||
matching "Vulkan in the emulator" below. The documented fix for
|
||||
*that* (`VK_DRIVER_FILES=... GPU_HOST_FEATURES="-feature Vulkan" emu
|
||||
up`) does get a Vulkan device, but on this host it is a **second**
|
||||
one alongside the real GPU's own Venus/gfxstream Vulkan adapter, and
|
||||
`AndroidRenderer::new`'s `request_adapter` (no adapter-name
|
||||
filtering, `PowerPreference::default()`) picked Venus -- which
|
||||
crashed inside `wgpu_core::device::resource::Device::
|
||||
create_bind_group_layout`, the same structural Venus incompatibility
|
||||
"Vulkan in the emulator" already documents for a different call.
|
||||
`EMU_GPU=software` (`-gpu swiftshader_indirect`, no host GPU
|
||||
involved at all) is what actually works cleanly, because there is
|
||||
then only the one Vulkan device (`SwiftShader Device (Subzero)`) for
|
||||
`request_adapter` to find -- confirmed via
|
||||
`wgpu_core::instance: Found 1 compatible adapters`. **One trap in
|
||||
switching between these**: the AVD's saved snapshot carries over
|
||||
whichever GPU config booted it last, so restarting under
|
||||
`EMU_GPU=software` right after a `-feature Vulkan` boot still linked
|
||||
against `vulkan.ranchu.so` and crashed (`SIGSEGV` inside
|
||||
`vk_common_SetDebugUtilsObjectNameEXT`) until the AVD's
|
||||
`snapshots/` directory was cleared by hand -- matches "Vulkan in the
|
||||
emulator"'s own note that a GPU-config switch needs a cold boot the
|
||||
`emu` wrapper does not force. Recorded here rather than only in that
|
||||
section since it is what made three different crashes look like
|
||||
three different bugs before the pattern was the AVD's snapshot, not
|
||||
the code.
|
||||
|
||||
**Measurements taken, 2026-09-05, `ai-app-2`'s own emulator,
|
||||
`EMU_GPU=software`, against `app/ui-sandbox.sh` (port 8519, session
|
||||
`8920378e7167ebcd`, 40 real sent/echoed messages):**
|
||||
|
||||
(a) **Tap-by-name on a named control -- passes.**
|
||||
`ui-trace record --do "tap 'Message'"` (the composer's `.label`,
|
||||
I4) resolved and the field's bounds moved (`top 2329 -> 1509`,
|
||||
the keyboard opening), the same shape I4's own tabs-screen taps
|
||||
confirmed the same day. `iris-android-app`'s tabs screen also got the
|
||||
full I4 pass-condition run this session -- see I4's own box above,
|
||||
now ticked `[x]`.
|
||||
|
||||
(b) **The `transcript-bench.sh`-shaped scroll comparison -- a
|
||||
real number for Compose, no comparable number for iris, and that
|
||||
gap is itself the finding.** `transcript-bench.sh` could not be
|
||||
pointed at `iris-android-app` directly -- it reads the Compose app's
|
||||
in-app render-report log line, which this screen has no equivalent
|
||||
of -- so the same 24-swipe gesture loop (`swipe 540 700 540 1600
|
||||
200` / `swipe 540 1600 540 700 200`, alternating, matching that
|
||||
script's own cycle) was driven by hand via `ui-trace record` against
|
||||
both apps, each freshly opened on the same session, `dumpsys gfxinfo
|
||||
<package> reset` beforehand and `dumpsys gfxinfo <package>` after.
|
||||
Compose: **8.96% janky frames, 99th percentile 150ms, 212 frames
|
||||
rendered** over the loop -- worse than AGENTS.md's own recorded
|
||||
stock-emulator baseline (5.2-5.9%, 29-32ms), consistent with
|
||||
`EMU_GPU=software`'s CPU rendering being slower than the `-gpu host`
|
||||
that baseline was taken under, which is exactly why AGENTS.md's rule
|
||||
against reading an absolute emulator number as the phone's applies
|
||||
doubly here. **iris: `dumpsys gfxinfo` reported 0 frames rendered
|
||||
for the entire gesture window, on both a run where the screen
|
||||
visibly did not move and one where it visibly did** (confirmed
|
||||
by `adb exec-out screencap -p`, hashed before/after -- identical
|
||||
when the swipe direction was already at that end of the transcript,
|
||||
different once swiped the other way). **`gfxinfo` instruments
|
||||
Android's own Skia/HWUI View-drawing pipeline; it has no visibility
|
||||
into a `SurfaceView` whose contents are drawn by a separately-owned
|
||||
GPU context (`wgpu`/Vulkan, here) the way Compose's ordinary `View`
|
||||
tree is drawn.** A `dumpsys SurfaceFlinger --latency` probe against
|
||||
the transcript screen's own `SurfaceView` layer was tried as a
|
||||
fallback and returned only the display's refresh period (16666666ns)
|
||||
with no frame history at all -- this Android version's BLAST
|
||||
compositor does not keep the per-frame timestamps that legacy API
|
||||
used to report. **So there is no dumpsys-derived frame-time number
|
||||
for iris on this build**, not a bad one -- the honest comparison this
|
||||
pass can make is functional (both apps' lists scroll under the same
|
||||
touch gesture) rather than numeric, and getting a real number for
|
||||
iris needs the app's own frame-timing instrumentation (the render
|
||||
report the Compose side already has, iris has none of yet) rather
|
||||
than a different `dumpsys` incantation.
|
||||
|
||||
(c) **Touch-drag pans the list on real device touch input --
|
||||
confirmed by screenshot, not by `ui-trace show`.** `ui-trace show`
|
||||
cannot answer this at all here: the only named node on this screen
|
||||
is the composer, which does not move when the list scrolls, so every
|
||||
`--field box` query reports "nothing moved" regardless of whether
|
||||
the list actually did (the same "no named structure to track"
|
||||
situation I4's tabs-screen note about clipped bounds warns about,
|
||||
one level further -- here there is no candidate node at all, not a
|
||||
clipped one). `adb exec-out screencap -p` before and after a single
|
||||
`swipe 540 700 540 1600 300` (list not already at that end) hashed
|
||||
different and visibly showed different message rows on screen;
|
||||
the same swipe repeated when already at that end of the transcript
|
||||
correctly hashed identical -- so the mechanism responds to real
|
||||
touch, in both directions, not just once by luck. **Long-press then
|
||||
drag to select was not independently driven this pass**: doing it
|
||||
for real needs a touch held stationary for `LONG_PRESS` (500ms)
|
||||
and *then* moved without lifting, and neither of `ui-trace`'s two
|
||||
gesture primitives can produce that -- `tap` has no hold, and
|
||||
`swipe X1 Y1 X2 Y2 MS` interpolates motion across its whole duration
|
||||
from t=0, so a long `swipe` with a short first segment is still
|
||||
continuous motion throughout, not a hold followed by a drag. This
|
||||
needs either a new `ui-trace` action (a `hold MS then drag X Y`
|
||||
primitive) or a raw multi-step `sendevent`/`MotionEvent` injection
|
||||
neither this pass's tooling nor its remaining time could build
|
||||
safely. `DragArbiter`'s own unit tests (`iris/src/sense.rs`, I5's
|
||||
earlier "Gap closed" section) already cover this exact sequence with
|
||||
a synthetic clock, which is why the mechanism is trusted enough to
|
||||
call "not independently driven on-device" rather than "unverified."
|
||||
|
||||
**What remains, named rather than silently dropped (also in
|
||||
IRIS_TODO.md, dated 2026-09-05):**
|
||||
|
||||
- **The emulator half of the pass condition was not attempted.**
|
||||
`emu list` shows `emulator-5554` (AVD `ai-app`, a different
|
||||
checkout) held by another session during this pass, but even with
|
||||
a free emulator this needs real Android integration that does not
|
||||
exist yet for this screen: a cdylib + Gradle shell the way
|
||||
`iris-android-app` wraps `tabs-ui` (I2), real
|
||||
`client-core::ApiClient`/`event_stream::follow_session_events`
|
||||
wiring against `app/ui-sandbox.sh` with `--delay` (this crate
|
||||
deliberately does not fetch anything itself, see `lib.rs`'s doc),
|
||||
and then `transcript-bench.sh`'s gesture compared against the
|
||||
Compose baseline. That is real, multi-part follow-on work in its
|
||||
own right -- closer in size to E2/E3 than to "run one more
|
||||
script" -- not something this pass's remaining time could
|
||||
responsibly rush and still report honestly.
|
||||
- **Row-level accessibility names** -- behaviour 6 above.
|
||||
- **A real per-app frame-time number for iris** -- (b) above. Needs
|
||||
`iris` to grow its own render-report/frame-timing instrumentation
|
||||
(what the Compose app's copy-button report already gives) before
|
||||
any `dumpsys`-based comparison can go further than functional.
|
||||
- **Long-press-then-drag-to-select, on-device** -- (c) above. Needs
|
||||
a driving primitive this pass's tools do not have.
|
||||
- **Row-level accessibility names** -- behaviour 6, I5's own writeup
|
||||
above.
|
||||
- **A tappable link and a code-span background chip** -- behaviour 2.
|
||||
- **`Selection`'s anchor-row shortcut** -- behaviour 1.
|
||||
- **No syntax highlighting inside a fenced code block** -- `markdown.rs`
|
||||
@@ -2493,18 +2721,20 @@ silently on real hardware.
|
||||
|
||||
**Net for the recommendation.** Item 3 ("decide when the transcript
|
||||
screen exists in both, from the measurements") still cannot be
|
||||
decided by a number -- E2 could not produce one for Masonry, and I5
|
||||
has not yet produced one for iris either, for an unrelated reason
|
||||
(no Android harness built yet, not an absent capability). What *can*
|
||||
be said structurally, updating E2's own conclusion: iris now also
|
||||
demonstrates the two things E2 found Masonry structurally unable to
|
||||
do at all -- cross-row selection and true per-span inline rich text
|
||||
inside one wrapped, selectable buffer -- neither of which exists
|
||||
anywhere in `masonry`/`masonry_core`/`xilem` today (E2's own
|
||||
`grep -rln` finding). That is a second structural point in iris's
|
||||
favour, alongside I2's working touch-scroll-vs-Masonry's-absent one,
|
||||
still short of the render-number comparison the recommendation
|
||||
ultimately wants.
|
||||
decided by a render-time number on either side -- E2 could not
|
||||
produce one for Masonry (no working touch-scroll to measure), and
|
||||
this pass could not produce one for iris either, now for a different
|
||||
and more fixable reason: `iris` itself has no frame-timing
|
||||
instrumentation yet, not that the screen doesn't work. What *can* be
|
||||
said, updating E2's own conclusion further: iris now has a real,
|
||||
working Android integration -- a real server, a real scrolling
|
||||
transcript, real touch-drag panning, tap-by-name accessibility, all
|
||||
confirmed on-device -- plus the two structural points E2 already
|
||||
found Masonry unable to reach at all (cross-row selection, true
|
||||
per-span inline rich text). Whether iris's *smoothness* matches or
|
||||
beats Compose's is the one question actually left for whoever picks
|
||||
up "add frame-timing instrumentation to iris" next; DECISIONS.md's
|
||||
DEFERRED item is updated with exactly this gap.
|
||||
|
||||
## For the next agent
|
||||
|
||||
|
||||
Reference in new issue
Block a user