Wrapping the composer's field in .scrollable().masked() needed three layout defects fixed first, each with a headless regression test that was confirmed to fail without its fix: - MaxSize/Sized reported a caller's declared dp length unresolved, and Span places a child from the abs/rel of what it reported, so dp(168) was worth zero: the bar got a slot of nothing the moment its content passed six lines and the Scroll inside measured its container at -63px (container=-63 content=415.8 amt=478.8 on the emulator). Len::fold_dp, used on the way out, plus a debug_assert in draw_inner that a reported Size carries no dp -- the rule is about every widget, not those two. - Masked allocated a fresh mask slot per draw, and draw_inner's unchanged-region fast path does not revisit descendants, so they kept clipping against a box the bar had moved away from: four live mask entries, none of them current, and the field drew nothing. ActiveData::own_mask, allocated once and rewritten in place. - mov updates active.region and accumulates the same delta on the move slot, and resolved_region added both, so a panned widget's own hit box sat at twice the pan -- the composer's field was untappable after a drag. ActiveData::move_applied. Scroll itself measured the right number by a misleading route; it is written against painter.px_size() now and still reports its content's size, since reporting the container makes the answer a function of itself. Verified on this checkout's emulator: swipe 540 1200 -> 540 1460 moved the field's Message box 31,1041..1048,1509 -> 31,1131..1048,1651 with its height unchanged at 468px. run-bench.sh polled logcat for a prefix copy_report also logs at startup, so it printed a report that had never been run. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
661 lines
41 KiB
Markdown
661 lines
41 KiB
Markdown
# iris: known problems and things still to build
|
|
|
|
Iris's own list for the library, recorded 2026-09-04 in her words where it
|
|
matters, so the agents working through RUST.md pick these up in a sensible
|
|
order rather than rediscovering them. Each item says where it sits in the
|
|
order and what "done" looks like. Tick and date them in place.
|
|
|
|
## Fix
|
|
|
|
- [x] **`request_device` asked for compute-shader limits it never uses
|
|
(2026-09-05).** `Limits::default()` (both `iris/src/android/render.rs`
|
|
and `iris/src/default/render.rs`) requests desktop-tier compute limits
|
|
unconditionally, even though nothing in `iris`/`iris-core` creates a
|
|
`ComputePipeline` or writes a `@compute` shader stage — confirmed by
|
|
grepping the whole tree, not assumed. That crashed device creation
|
|
outright on the Android emulator's software GL path (`EMU_GPU=software`,
|
|
`--features force-gles`): SwiftShader's GL reports itself as OpenGL ES
|
|
3.0, which has no compute shaders, so the adapter's real limit is 0
|
|
against the unconditional request for 65535 — the same would happen on
|
|
any real GLES-3.0-only Android device. Fixed by a new, shared
|
|
`iris_core::device_limits()` (`iris/core/src/render/mod.rs`) that zeros
|
|
exactly the six `max_compute_*` fields rather than switching to a
|
|
downlevel `Limits` preset — `downlevel_webgl2_defaults()` also zeros
|
|
`max_storage_buffers_per_shader_stage`, which `shader.wgsl`'s vertex
|
|
stage needs (four `var<storage>` buffers), so that preset would trade
|
|
this crash for a bind-group-layout one on the same hardware.
|
|
`rigs/gpu-probe`'s own hand-mirrored `Limits` (it is deliberately its
|
|
own crate, not able to call `device_limits()` directly) was updated to
|
|
match. See `DECISIONS.md` and RUST.md's I5 box for the account,
|
|
including what could not be re-verified on-device this pass (the
|
|
emulator was in concurrent use by another session).
|
|
|
|
- [x] **Input does not fall through by input type (2026-09-04).**
|
|
`SensorUi::run_sensors` (`src/default/sense.rs`) used to set "consumed,
|
|
stop checking lower layers" from mere hover — a widget registered for
|
|
nothing but `click()` blocked a `Scroll` meant for whatever was behind
|
|
it, since "the cursor is over this widget" and "this widget handled the
|
|
event" were the same check. Fixed by judging consumption per input
|
|
kind: with no button transition and no scroll happening this frame
|
|
("momentary" activity), the topmost hovered widget still wins, same as
|
|
before; when something momentary *is* happening, only a widget whose
|
|
registered senses actually include a matching non-hover one (checked
|
|
via a new `TypeEventManager::registered`, which lists what a widget
|
|
registered without running anything) consumes it, so a widget with only
|
|
`Hovering`/click handlers can no longer block a scroll from reaching a
|
|
list underneath. `iris/src/sense_tests.rs` builds a button-over-a-list
|
|
`Stack` with a plain `HasEvents` impl (no GPU or window) and checks both
|
|
directions: a scroll over the button reaches the list, and a real click
|
|
still reaches the button — confirmed to fail on the pre-fix code and
|
|
pass after.
|
|
|
|
- [x] **Appending one image to an already-loaded list rebuilds every other
|
|
image's bind group (2026-09-05, fixed 2026-09-05).** Found by the
|
|
benchmark below: `GpuTextures::update` (`core/src/render/texture.rs`)
|
|
triggered `rebuild_image_bind_groups` — a loop over *every live
|
|
standalone image*, rebuilding its `BindGroup` — whenever the shared
|
|
`masks` or `move_offsets` GPU buffer was resized (`masks_resized ||
|
|
moves_resized` in `UiRenderNode::update`, `core/src/render/mod.rs`), and
|
|
a widget getting its *first* move-offset slot (LAYOUT.md section 2 —
|
|
every widget gets one on first draw) could be exactly what grows that
|
|
buffer. So one new message with one new image, appended to a transcript
|
|
that already has N images loaded, did not cost O(1): it cost one
|
|
`create_image` for the new image plus one `make_image_bind_group` per
|
|
*existing* image, because the new widget's own move slot pushed the
|
|
arena past its capacity. Measured directly in
|
|
`iris/examples/bench_images.rs`: appending a 1,001st image to 1,000
|
|
already-settled ones reported **1,001** bind-group creates for that one
|
|
frame, not 1 (`./run-bench.sh images`, frame 5 in the transcript below).
|
|
|
|
**Fix**: `masks`/`move_offsets` never belonged in a standalone image's own
|
|
bind group (group 2) in the first place — the group also holds that
|
|
image's own texture view, which is the only thing that is genuinely
|
|
per-image, so a buffer shared by *everything* forced a rebuild of
|
|
*every* group the moment it moved. Gave masks/move_offsets their own
|
|
bind group (group 3 in `shader.wgsl` and `UiRenderNode`: `masks_layout`/
|
|
`masks_group`), bound once per frame in `UiRenderNode::draw` rather than
|
|
once per draw call, instead of duplicating them into every per-image
|
|
group. `GpuTextures` and its image bind groups now know nothing about
|
|
either buffer — `rebuild_image_bind_groups` is called only from
|
|
`grow_array` (the atlas array texture growing, which genuinely does
|
|
change what every image's own bind group must reference) — so a
|
|
masks/move_offsets resize now touches exactly one bind group, ever,
|
|
regardless of how many images are live. Numbers after the fix, same
|
|
benchmark and command:
|
|
|
|
./run-bench.sh images
|
|
frame=1 bind_group_creates=1000 (cold load, unchanged)
|
|
frame=2 bind_group_creates=0 (was 1000 -- see the item below)
|
|
frame=3 bind_group_creates=0
|
|
frame=4 bind_group_creates=0
|
|
(append one image here)
|
|
frame=5 bind_group_creates=1 (was 1001)
|
|
frame=6 bind_group_creates=0
|
|
|
|
`run-headless.sh tabs --shot` still 27266 bytes, byte-for-byte unchanged,
|
|
confirming the bind-group restructuring changed nothing about what is
|
|
drawn.
|
|
- [x] **Bind-group creation takes two frames to reach the steady state, not
|
|
one (2026-09-05, closed by the fix above, 2026-09-05).** Same benchmark:
|
|
loading 1,000 images cold used to report 1,000 creates on frame 1
|
|
(expected — `create_image`, one per new image) *and again* 1,000 on
|
|
frame 2, before settling to 0 from frame 3. This was `rebuild_image_bind_groups`
|
|
firing a second time for the same masks/move-offsets buffer-growth
|
|
reason as the item above, confirming the guess recorded here — the two
|
|
were exactly the same root cause measured two different ways. Frame 2
|
|
now reports 0 (see the numbers above); not a separate fix.
|
|
|
|
- [ ] **A read-only text display has no widget of its own — P0's bench
|
|
report area is a `TextEdit` standing in for one (2026-09-05).** The only
|
|
way to get selectable text on screen today is `.editable(...)` plus
|
|
`.attr::<Selectable>(())` (`Selectable` is only implemented for
|
|
`TextEdit`, `iris/src/attr.rs`), which also makes the field focusable —
|
|
tapping the bench report opens the soft keyboard over text nothing lets
|
|
you type into. Harmless for a bench-only debug screen (not fixed this
|
|
pass), but a real "selectable, not editable" text primitive would
|
|
remove the keyboard side effect and is worth having before another
|
|
screen wants the same thing (P1's own transcript rows already read
|
|
their content from a `TextEdit` for the same reason).
|
|
|
|
## From the phone, 2026-09-06
|
|
|
|
Found on Iris's own phone while working RUST.md's P0 box's phone-report
|
|
follow-ups. Recorded here rather than fixed in that pass, so a follow-up
|
|
agent takes them without colliding with that pass's `bench_client.rs`/
|
|
`android/view.rs`/`android/sense.rs` changes.
|
|
|
|
- [x] **Swiping has no momentum, fixed 2026-09-06.** `List::fling`/
|
|
`VelocityTracker`/`FlingCalculator` (`iris/src/widget/list.rs`,
|
|
`iris/src/sense.rs`) -- IRIS.md's 2026-09-06 entry has the full account.
|
|
Wired through `Selection::drag`'s release path, cancelled by the next
|
|
touch-down, clamped at the loaded content's start/end. Verified by unit
|
|
test (fling distance against the closed-form spline result, cancel-on-
|
|
touch, the clamp), not yet by an on-device or emulator feel-check --
|
|
that is still open.
|
|
- [x] **Scrolling down sometimes jitters the text, fixed 2026-09-06.**
|
|
Root-caused by reading `DragArbiter::update`'s `Undecided`-to-`Panning`
|
|
transition rather than by an on-device trace (no emulator was used this
|
|
pass): it was the first named suspect, not the second. `self.last` stays
|
|
at the press origin for every `Undecided` frame (nothing pans while the
|
|
gesture might still be a selection), so the frame that finally crosses
|
|
`DRAG_SLOP` returned `Pan(dy)` with `dy` measured from `press_start` --
|
|
the *whole* pre-threshold drag, applied to the list in one step, however
|
|
many frames it had taken to get there. Fixed by applying only the
|
|
excess past `DRAG_SLOP` on that one frame (`dy - DRAG_SLOP.copysign
|
|
(dy)`), the same "consume the slop, don't replay it" rule Android's own
|
|
touch handling follows. New regression test,
|
|
`crossing_the_slop_by_a_little_pans_by_a_little` (`iris/src/sense.rs`).
|
|
**Not yet done**: an emulator trace of the real per-frame offset
|
|
confirming this was the whole story on real touch input rather than
|
|
only the arbiter's own unit tests -- worth a follow-up pass before
|
|
calling it fully closed.
|
|
- [x] **Composing text held back until a space, caret not moving, fixed
|
|
2026-09-06.** `InputMethodManager.updateSelection` was never called --
|
|
see IRIS.md's 2026-09-06 entry and RUST.md's P0 box, item 1, for the
|
|
full account and the emulator evidence.
|
|
- [x] **Swipe over the composer summons the keyboard, fixed 2026-09-06.**
|
|
`Selector`/`Selectable` now wait for a completed tap -- see IRIS.md's
|
|
2026-09-06 entry and RUST.md's P0 box, item 5. Verified via `dumpsys
|
|
input_method`'s `mInputShown` on the emulator, not yet on the phone.
|
|
- [x] **Text disappears again after leaving and returning to the app,
|
|
fixed 2026-09-06.** `GlyphAtlas::clear`/`Textures::reset` on a
|
|
genuinely new renderer -- see IRIS.md's 2026-09-06 entry and RUST.md's
|
|
P0 box, item 4. Verified on the emulator (home, reopen, screenshot);
|
|
not yet on the phone.
|
|
- [x] **Composed/typed text never becomes visible at all -- root-caused
|
|
and fixed 2026-09-06.** Not the renderer at all: **the composer's buffer
|
|
was empty the whole time.** `TextEditCtx::select` (`iris/src/widget/
|
|
text/edit.rs`) compared the tap against the *laid-out text's* box and
|
|
set `selection = None` for anything outside it -- and an empty field's
|
|
layout is a zero-width box, so tapping an empty composer granted focus
|
|
and opened the keyboard while leaving no caret; `insert_str` returns
|
|
early with no caret, so every keystroke after that was dropped in
|
|
silence. Gboard's suggestion strip is its own composing state, not a
|
|
read of our buffer, which is what made the earlier pass conclude the
|
|
buffer held the text. Fixed by letting parley clamp a tap outside the
|
|
layout to the nearest cursor position (a press that reaches `select`
|
|
has already been hit-tested to the widget, so there is no "outside"),
|
|
plus a `debug_assert!` in `insert_str` so an insert with no caret fails
|
|
at the mistake instead of dropping input -- it immediately caught
|
|
`layout_tests::composing_text_after_a_keyboard_resize_...` typing into
|
|
an unfocused field. Three new tests in `edit.rs`
|
|
(`tapping_an_empty_field_places_a_caret_so_typing_lands`,
|
|
`tapping_past_the_end_of_the_text_clamps_to_the_end`,
|
|
`dragging_without_a_previous_selection_selects_nothing`); the first
|
|
fails on the pre-fix code. Emulator evidence: `adb shell input text`
|
|
after `tap 'Message'` now shows the text in the bar
|
|
(`/tmp/final-typing.png`) and logs `iris text render: chars=5 ...
|
|
glyphs=5`, against `glyphs=0` on every keystroke before.
|
|
|
|
**The old, superseded diagnosis, kept because it was wrong in an
|
|
instructive way:** The composer bar stays empty even once the
|
|
buffer genuinely holds the typed text (confirmed indirectly: Gboard's
|
|
own suggestion strip reacts correctly to each keystroke). A new unit
|
|
test proves the widget tree's own layout math resolves the field's
|
|
region correctly across a keyboard resize, so the bug is downstream of
|
|
that -- most likely `UiRenderState::redraw`'s single-widget redraw path,
|
|
or specific to this emulator's forced `force-gles` backend (untested on
|
|
Vulkan or the real phone). RUST.md's P0 box, item 2, has the full
|
|
writeup, what was ruled out, and where to look next. **Also unverified
|
|
because of this**: item 3's composer rebuild (one `Stack`-based widget,
|
|
a capped/scrollable height, bottom padding tied to the IME/nav-bar
|
|
inset) -- structurally in place and unit-tested, but its own visual
|
|
correctness cannot be screenshotted until text actually renders.
|
|
- [x] **The composer has no touch-drag scroll for overflowing text.**
|
|
**Done 2026-09-06.** `field.scrollable().masked()` in
|
|
`transcript-ui/src/composer.rs`: a finger drag inside the bar pans the
|
|
message, the bar stays capped at six lines, and a vertical drag in the
|
|
focused field no longer extends a selection (Android `EditText`'s own
|
|
behaviour). Verified on this checkout's emulator with the
|
|
`transcript-screen bench force-gles` debug build -- six repetitions of a
|
|
13-word sentence typed in, then
|
|
`ui-trace record --do "swipe 540 1200 540 1460 300"`: the field's
|
|
`Message` box moved `31,1041..1048,1509` -> `31,1131..1048,1651` (the
|
|
content panned down with the finger) with its **height unchanged at
|
|
468px** (the bar did not grow), and the two screenshots either side show
|
|
different text in the same band.
|
|
Three real defects had to be fixed first, each with a headless
|
|
regression test in `iris/src/layout_tests.rs` and each confirmed to fail
|
|
without its fix (docs/RUST.md's plan box has the measurements):
|
|
a `MaxSize` reporting its cap as an unresolved `dp` (`Len::fold_dp`), a
|
|
`Masked` allocating a fresh mask slot per draw (`ActiveData::own_mask`),
|
|
and a panned widget's own hit box moving twice (`move_applied`).
|
|
`Scroll` itself turned out to measure the right number by a misleading
|
|
route -- it is written against `painter.px_size()` now, and the claim
|
|
below that it "measures against the window" was wrong.
|
|
**Still open, and pre-existing:** the bar's own grey background is not
|
|
drawn on this build (the `Stack{StackSize::Child(1)}` behind the field),
|
|
so the message reads as white text over the transcript. Present in the
|
|
build *before* this change too, so it is not the scroll area's doing.
|
|
|
|
## From the phone, 2026-09-06, 11:39 (build delivered 02:07, commit 543f6d9)
|
|
|
|
Iris's report on the build with the composing-text, tap-vs-swipe and
|
|
atlas-reset fixes, with a screenshot, verbatim. Each is open until an
|
|
agent ticks it here with the evidence.
|
|
|
|
- [x] **"The app definitely does not start with keyboard spacing
|
|
correct. This is how it looks without me doing anything initially."**
|
|
**Not an inset bug at all -- fixed 2026-09-06.** The black third is the
|
|
bench shell's own empty *benchmark report* pane: `bench_client.rs`'s
|
|
root tree gave it `.height(rest(1))` beside `content.height(rest(2))`,
|
|
so an empty `TextEdit` reserved a third of the window at every launch
|
|
and pushed the composer up by exactly that. Measured on this checkout's
|
|
emulator at the phone's own size (1080x2424, density 420, gesture nav),
|
|
which reproduced Iris's screenshot exactly: new `iris insets:` log line
|
|
reported `bottom=63 ime_bottom=0` at launch (a nav bar, no keyboard --
|
|
so the inset the composer was fed was never large), while `ui-trace
|
|
show -m Message --field box` put the field at `31,1488..1048,1540` on a
|
|
2282px-tall surface, 789px clear of the bottom -- that pane's third.
|
|
**Unit mixing checked explicitly and cleared**: `set_bottom_inset` takes
|
|
physical px and stores `Len::abs`, `MainActivity.java`'s `1`/`0`
|
|
`ime_bottom` only ever reaches `insets.bottom.max(ime_bottom)` and
|
|
`> 0.0`, and every `dp` in the composer resolves at layout time. Fix:
|
|
the report pane is sized to its content (`.max_height(dp(260))
|
|
.scrollable()`), and moved above the transcript so it cannot eat the
|
|
composer's nav-bar clearance. After: field box `31,2277..1048,2329`,
|
|
grey bar ending at device y2361 with the 63px nav strip below it
|
|
(`/tmp/fix1.png` this pass).
|
|
The screenshot shows the composer bar (the grey band) sitting about
|
|
two thirds of the way down a 704x1568 screen, with black below it to
|
|
the bottom, and the transcript ending at "Claude / Results" just above
|
|
it -- at launch, no keyboard. So the composer's bottom padding, which
|
|
the 2026-09-06 rebuild tied to the IME/nav-bar inset, is being fed a
|
|
large value at start on the phone. Suspects, in order: the initial
|
|
inset delivery on the phone (GrapheneOS, gesture navigation) versus
|
|
the emulator; `ime_bottom` now carrying a `1`/`0` boolean through a
|
|
field the composer may still read as pixels or dp; a stale value from
|
|
before the first `on_insets_changed`. Reproduce with the phone's
|
|
screen size and density on the emulator before guessing.
|
|
- [~] **"Swiping still gets caught by the grey bar but keeps working
|
|
after I go past it."** Improved 2026-09-06 by the focused-field rule
|
|
below, still needs her phone to close. `attr.rs`'s `on_press` treated an
|
|
already-focused composer as the plain drag-to-select case, so a swipe
|
|
starting inside it dragged a highlight through the typed text for the
|
|
whole gesture; it now abandons that the moment the press passes
|
|
`DRAG_SLOP` vertically (Android `EditText`'s own rule), which removes one
|
|
of the two things that made the bar feel like it caught the swipe. The
|
|
residual `DRAG_SLOP` measured from the boundary crossing, described
|
|
below, is unchanged. Original note follows.
|
|
Not closeable from the emulator, annotated
|
|
2026-09-06 after the `DragGesture` merge. `attr.rs`'s `on_press` never
|
|
calls `capture_pointer` and never consumes a `Pressing` frame past
|
|
`DRAG_SLOP` (it just stops watching), so once the finger's *current*
|
|
position leaves the composer's box and enters the list's, `List`
|
|
starts receiving ordinary hit-tested `Pressing` frames there --
|
|
`DragArbiter::is_idle()`'s 2026-09-05 recovery (a missed `PressStart`)
|
|
picks it up rather than leaving it stuck. What this does **not** do is
|
|
what "wherever it began" implies literally: `DragArbiter::press_start`
|
|
restarts from the *boundary-crossing* position, not from the original
|
|
touch-down inside the composer, so the pan still needs a fresh
|
|
`DRAG_SLOP` of travel measured from the boundary rather than from the
|
|
start of the gesture -- composer and list are adjacent, non-overlapping
|
|
widgets (`lib.rs`'s `(list, composer_bar).span(Dir::DOWN)`), and only
|
|
the composer forwarding its own drag to the list would remove that
|
|
residual slop entirely, which is more than this pass's merge changes.
|
|
RUST.md's merge-pass box has the reasoning in full and an emulator
|
|
swipe confirming the composer's own box never moves/resizes during it;
|
|
whether the residual slop is still perceptible as "caught" needs Iris's
|
|
phone, since the emulator's per-widget boundary is a few dp wide and
|
|
easy to cross without noticing on a real screen too.
|
|
- [ ] **"Flinging still does not work."** No longer expected to reproduce
|
|
after the `DragGesture` merge (`e12c708`, pointer capture +
|
|
`CursorSense::Drop`), 2026-09-06. Emulator evidence (RUST.md's
|
|
merge-pass box, check (b)): a real `ui-trace` finger swipe followed by
|
|
screenshot-hash sampling caught a post-release frame distinct from the
|
|
drag's own last frame in one run, and every run showed 28-32
|
|
`render()` frames per gesture against an idle baseline of 0 and ~8
|
|
expected from the drag alone -- redraw kept being requested well past
|
|
the finger lifting, which only happens while a fling is still
|
|
animating. Left unticked in spirit until Iris's phone confirms it,
|
|
since only she can say whether it *feels* like a fling now; the
|
|
emulator's screenshot timing could not always catch the tail of a
|
|
fast-settling one visually (same caveat noted in RUST.md).
|
|
- [~] **"Text still disappears if I leave and come back to the app."**
|
|
**Instrumented 2026-09-06 so the phone can answer it**, since no
|
|
emulator here has a Vulkan adapter. `iris/src/android/view.rs` now logs
|
|
one `log::info!` line per surface event with the glyph/atlas counts:
|
|
`iris surface: surface_destroyed, tearing the renderer down
|
|
(glyphs_cached=387 atlas_pages=1)`, `iris surface: surface_changed
|
|
1080x2424 already_live=false glyphs_cached=387 atlas_pages=1`, `iris
|
|
surface: new renderer built (Gl), clearing glyph atlas: glyphs=387
|
|
pages=1`, plus `iris insets: ... window=(1080, 2424)` on every insets
|
|
change. That is the emulator's own healthy app-switch cycle, verified
|
|
this pass (home, reopen, screenshot: all text intact,
|
|
`/tmp/appswitch.png`). **The one line to look for on the phone is
|
|
`already_live=`**: `true` on the return from backgrounding would mean
|
|
the surface came back *without* a `surface_destroyed`, so
|
|
`surface_changed` reconfigured a renderer whose Vulkan swapchain and
|
|
atlas textures belong to a window that is gone -- the reuse branch
|
|
never clears the atlas, by design. `false` with no `new renderer built`
|
|
line after it would mean the renderer failed to rebuild. Either answer
|
|
names the fix; guessing between them from here does not.
|
|
The `GlyphAtlas::clear`/`Textures::reset` fix was verified on the
|
|
emulator under `force-gles` only; the phone runs Vulkan. So either the
|
|
reset is not reached on the phone's path (a different surface-
|
|
lifecycle sequence -- `surface_destroyed`/`surface_created` ordering,
|
|
or the renderer not being rebuilt but its textures lost), or the CPU
|
|
glyph cache and the GPU atlas still disagree after it. Needs logging
|
|
of the renderer lifecycle on the phone build, readable from `adb
|
|
logcat` when Iris next runs it, since no emulator here has a Vulkan
|
|
adapter under host GPU.
|
|
|
|
## Build
|
|
|
|
- [x] **Benchmarks**, not unit tests, run on demand (2026-09-05; a
|
|
`benches/` or a script under `iris/`, never in `cargo test`). The
|
|
scenario that matters most is a **message list** — chat apps and this
|
|
app's transcript alike — stressed with many messages and many images.
|
|
One case in particular: **resizing an input box** (typing enough text to
|
|
grow it) that pushes a long list of messages above it must stay very
|
|
fast and recalculate almost nothing — a move of everything above, not a
|
|
re-layout. That is exactly the O(1) move chain in LAYOUT.md; the
|
|
benchmark is what proves it. Done when the numbers are in this file with
|
|
the command, and the input-box case reports draws re-run, not just frame
|
|
time.
|
|
|
|
**Built as two rigs**, chosen per scenario by whether a real `wgpu`
|
|
device is needed (`UiRenderState`/`Widgets` touch no GPU or window, so
|
|
most of this runs as an ordinary binary — the same property
|
|
`layout_tests.rs` relies on):
|
|
|
|
- `iris/benches/message_list.rs` — a plain `Instant`-timed binary
|
|
(`[[bench]] harness = false` in `iris/Cargo.toml`), not criterion: see
|
|
the file's own header for why (short version — every scenario here
|
|
reduces to a *count* `UiRenderState::take_counters` already produces,
|
|
which criterion's statistical machinery adds nothing to and which a
|
|
new dependency is not worth pulling in for). Covers (a) first-frame
|
|
cost of a message list of N wrapped-text rows (one in 20 also carrying
|
|
a small in-memory image) for N = 100/1,000/10,000; (b) per-frame cost
|
|
of scrolling that list, 200 ticks; (c) the input-box case — a
|
|
fixed-height field at the bottom of the screen growing by a line 40
|
|
times, with the message list above it filling the rest of the screen.
|
|
Run: `cd iris && cargo bench --bench message_list` (always release —
|
|
`cargo bench` builds the `bench` profile, which is optimized).
|
|
- `iris/examples/bench_images.rs` — needs a real device, so it runs
|
|
through `iris/run-headless.sh bench_images`, printing
|
|
`UiRenderNode::take_image_bind_group_creates()` (a new counter, added
|
|
in `core/src/render/texture.rs` and `core/src/render/mod.rs`,
|
|
mirroring `UiRenderState::take_counters`) each frame. Covers (d): 1,000
|
|
image rows, checked both cold (does bind-group creation reach zero
|
|
once loaded) and after appending one more image once settled (does
|
|
*that* stay cheap) — the second question is what actually matters for
|
|
a live transcript and is what turned up the two Fix items above.
|
|
- `iris/run-bench.sh [list|images]` runs either or both and is what to
|
|
run before/after touching `Scroll`, `Span`, `Sized`, the move-offset
|
|
chain, or `GpuTextures`.
|
|
|
|
**Numbers (2026-09-05, release, `cargo bench`/`run-headless.sh`, this
|
|
VM: AMD Ryzen 7 3800X, 8 cores, rustc 1.98.0 nightly-2026-09-03):**
|
|
|
|
cd iris && cargo bench --bench message_list
|
|
(a) first frame, N=100: 30.30ms draws=227 rewrites=15 moves=0
|
|
(a) first frame, N=1000: 186.04ms draws=2252 rewrites=150 moves=0
|
|
(a) first frame, N=10000:1770.36ms draws=22502 rewrites=1500 moves=0
|
|
(b) scroll, N=100/1000/10000, 200 ticks each:
|
|
draws=200 rewrites=0 moves=200 (identical at every N)
|
|
per-tick average: 0.0002ms (identical at every N)
|
|
(c) input grows 40 lines, N=100/1000/10000 rows above it:
|
|
draws=320 rewrites=40 moves=160 (identical at every N)
|
|
per-line average: 0.0012-0.0013ms (identical at every N)
|
|
|
|
cd iris && ./run-bench.sh images (2026-09-05, before the fix)
|
|
frame=1 bind_group_creates=1000 (cold load)
|
|
frame=2 bind_group_creates=1000 (see Fix item above)
|
|
frame=3 bind_group_creates=0
|
|
frame=4 bind_group_creates=0
|
|
(append one image here)
|
|
frame=5 bind_group_creates=1001 (see Fix item above)
|
|
frame=6 bind_group_creates=0
|
|
|
|
cd iris && ./run-bench.sh images (2026-09-05, after the fix)
|
|
frame=1 bind_group_creates=1000 (cold load, unchanged -- genuine work)
|
|
frame=2 bind_group_creates=0
|
|
frame=3 bind_group_creates=0
|
|
frame=4 bind_group_creates=0
|
|
(append one image here)
|
|
frame=5 bind_group_creates=1 (one image's own create_image, O(1))
|
|
frame=6 bind_group_creates=0
|
|
|
|
**Reading it**: (a) is real, necessary work — shaping and laying out N
|
|
never-before-seen text rows — and scales with N as it must, ~10x cost
|
|
per 10x N. (b) and (c) are the pass conditions that matter: both are
|
|
**exactly flat across N = 100 to 10,000**, confirming LAYOUT.md's O(1)
|
|
move chain holds for both scrolling and for a growing input box pushing
|
|
the message list — draws/moves per tick or per line do not grow with
|
|
list size, and the per-operation cost (a fraction of a microsecond) is
|
|
nowhere near a frame budget. (d)'s cold-load and steady-state halves
|
|
behave as designed; its *append* half did not, until the fix above moved
|
|
masks/move_offsets out of the per-image bind group — now flat at O(1)
|
|
the same way (b) and (c) are.
|
|
|
|
- **I5's transcript screen (`iris/transcript-ui/`, 2026-09-05) — what it
|
|
left, each recorded at the point in the code it would go rather than
|
|
silently dropped. See RUST.md's I5 box for the full account of what
|
|
*was* built (the screen, `SpanStyle`, cross-row selection, the growing
|
|
composer).**
|
|
- [x] **Android integration for this screen — done, 2026-09-05.**
|
|
`iris-android-app`'s `transcript-screen` Cargo feature
|
|
(`transcript_client.rs`) runs this screen against a real `ai-server`
|
|
through `client-core`, confirmed on-device: real scrolling, real
|
|
touch-drag panning, tap-by-name on the composer. Two real bugs found
|
|
and fixed along the way (a missing `INTERNET` permission; a
|
|
background-thread redraw request that crashed via a `Looper`
|
|
requirement, fixed by routing through `View::post_delayed` — see
|
|
`IRIS.md`'s `Tasks::redraw_handle` entry). See RUST.md's I5 box,
|
|
"The Android integration, done 2026-09-05" for the full account.
|
|
- [x] **A render-time number for iris, comparable to Compose's
|
|
`transcript-bench.sh` report — instrumentation done and a real number
|
|
obtained, 2026-09-05 (later the same day); the clean comparable loop
|
|
is not.** `iris_core::FrameReport` (`iris/core/src/render/
|
|
frame_report.rs`, `IRIS.md`'s new entry) times every frame from
|
|
`render()`'s redraw start to after `queue.submit`+`present()`, exposed
|
|
as two named on-screen controls ("Frame report", "Reset frame
|
|
report"). Driven against a real on-device touch-drag it read
|
|
`frames=34 janky%=61.76 p50=26.5ms p90=48.0ms p99=98.1ms
|
|
worst=98.1ms` — real, not inferred, but accumulated across several
|
|
gestures rather than one clean 24-swipe loop, because of the new
|
|
finding below. See RUST.md's I5 box, "Update, 2026-09-05, later the
|
|
same day" for the full account.
|
|
- [ ] **New, 2026-09-05: intermittent touch delivery to iris's
|
|
`SurfaceView` under this checkout's `EMU_GPU=software` emulator.**
|
|
The same swipe coordinates, confirmed (by scanning a screenshot
|
|
column for the first non-black pixel) to sit over real row text,
|
|
sometimes produced 30+ real frames and a screenshot diff and
|
|
sometimes produced zero of either, across otherwise-identical
|
|
`ui-trace` invocations. Not the already-understood "already at that
|
|
scroll edge" case (reproduced with content confirmed taller than the
|
|
viewport, in both directions). Leading candidate, not yet confirmed:
|
|
this checkout's emulator was independently observed at ~78% of one
|
|
CPU core, continuously, while idle on-screen — SwiftShader's software
|
|
rasterisation is CPU-bound by design, and a synthetic touch competing
|
|
with that load for delivery is plausible but unmeasured *during* a
|
|
failing gesture (the standing rule against diagnosing from
|
|
after-the-fact measurements applies here). Needs a sampler (load,
|
|
`dumpsys input`, a `-i 0` `ui-trace` capture) running while a failing
|
|
gesture is driven, and ideally a comparison under `-gpu host` (real
|
|
Vulkan) to see whether it is specific to software rendering. This is
|
|
what blocks the clean, comparable 24-swipe loop above.
|
|
- [x] **Long-press-then-drag-to-select — confirmed on-device, 2026-09-05
|
|
(later the same day).** `ui-trace` gained a `holddrag X1 Y1 X2 Y2
|
|
HOLD_MS MOVE_MS` action (`emulator-tools`, additive, extends the same
|
|
`MotionEvent`/`injectInputEvent` mechanism `swipe` already used):
|
|
press, hold past `LONG_PRESS`, move, release, as one continuous touch.
|
|
Driven against a real row (`holddrag 300 1850 300 2050 600 300`) it
|
|
produced `iris selection: begin at row ...` then a sequence of
|
|
`iris selection: extend to row ...` log lines
|
|
(`transcript-ui/src/selection.rs`, a new small `log` dependency since
|
|
selection has no accessibility label of its own yet — see the next
|
|
item), and a screenshot taken right after shows the expected
|
|
highlighted selection spanning multiple rows. `DragArbiter`'s own
|
|
unit tests already covered this sequence against a synthetic clock;
|
|
this is the first time it has been driven by a real device touch.
|
|
- [x] **Touch-drag panning over a row's own rendered text — done,
|
|
2026-09-05.** `row.rs` used to register `CursorSense::click_or_drag()`
|
|
on each row's `TextEdit` for cross-row selection; `TextEdit::draw`'s
|
|
`painter.child_layer()` (`iris/src/widget/text/edit.rs:87`) meant that
|
|
registration won `core/src/sense.rs::run_sensors`'s per-layer
|
|
arbitration on every frame it was pressed, not just the frame the
|
|
press started, so a list pan gesture registered on `List` itself never
|
|
got a turn while a row was under the finger. Fixed with
|
|
`iris::sense::DragArbiter` (recorded in `IRIS.md`), one small state
|
|
machine per list deciding pan vs. select the way Android does (a
|
|
vertical drag pans immediately; a stationary press held `LONG_PRESS`
|
|
(500ms) starts a selection which further drag extends; a horizontal
|
|
drag while something is already selected extends immediately) —
|
|
`transcript-ui/src/selection.rs`'s `Selection::drag` is the one place
|
|
every row's drag now routes through. 8 new unit tests
|
|
(`iris/src/sense.rs`'s `drag_arbiter_tests`); `cargo fmt/clippy/test
|
|
--workspace` and `cargo ndk` (both `iris` and `transcript-ui`) all
|
|
clean; `run-headless.sh` screenshot byte-identical to before the
|
|
change (38578 bytes). See RUST.md's I5 box, "Gap closed, 2026-09-05".
|
|
- [x] **Intermittent touch-scroll dropout — root-caused and fixed,
|
|
2026-09-05.** Not the coalesced-`ACTION_MOVE` hypothesis the earlier
|
|
pass suspected (ruled out): a gesture's `ACTION_DOWN` can land on a
|
|
row's own padding/gap or its header, which no `CursorSense` covers,
|
|
so `DragArbiter` never gets `press_start` and sits in `Idle`
|
|
(answers `Undecided` forever) for that whole gesture. Fixed via a new
|
|
`DragArbiter::is_idle()` that `Selection::drag`
|
|
(`transcript-ui/src/selection.rs`) checks to recover a missed press
|
|
on the next `Pressing` frame. Four new unit tests. See RUST.md's I5
|
|
box, "Touch-scroll dropout root-caused, 2026-09-05", for the trace and
|
|
what a peer session sharing this checkout's emulator mid-pass
|
|
prevented from being re-verified end-to-end (the aggregate
|
|
`iris-scroll.sh` three-run confirmation and a re-taken FrameReport
|
|
row) — a future pass should finish that once the emulator is free.
|
|
- [ ] **Row-level accessibility names.** The composer carries
|
|
`.label("Message")`; transcript rows do not carry a `.label()` of
|
|
their own yet, so `Widgets::named()` (I4) does not include them —
|
|
`row.rs`'s `build_text_row` is where one would go, keyed to something
|
|
stable per row (its sender + a short excerpt, matching what a screen
|
|
reader announcing a chat message would say).
|
|
- [ ] **A tappable link and a background chip behind inline code.**
|
|
Both need per-range glyph geometry that `TextEditCtx` does not expose
|
|
outside `iris::widget::text` (`edit.rs`'s `layout()` helper is
|
|
private) — see `markdown.rs`'s module doc for the exact shape the fix
|
|
would take (the same primitive `TextEdit::draw`'s own selection
|
|
highlight already uses internally,
|
|
`iris/src/widget/text/edit.rs:99`).
|
|
- [ ] **`Selection`'s anchor-row shortcut.** The row a drag started in
|
|
is selected in full (`select_all`) the moment the drag leaves it,
|
|
rather than "from the click point to whichever edge points away from
|
|
the drag" — needs the same private `layout()` access as the item
|
|
above. `selection.rs`'s module doc has the exact reasoning.
|
|
- [ ] **No syntax highlighting inside a fenced code block.**
|
|
`client_core::highlight` exists (built for the file explorer) and
|
|
could feed per-token `SpanStyle`s into a code block's span; wiring it
|
|
in was not attempted this pass.
|
|
|
|
- [ ] **Masks defined relative to each other.** Wanted: mask A multiplies
|
|
by something *and also* applies mask B — a mask can reference a parent
|
|
mask, the way the move chain references a parent offset. Today masks
|
|
are independent regions. Design it beside the move chain (same shape:
|
|
a parent index and a bounded walk in the shader); do it when a real
|
|
widget needs it, not before.
|
|
- [ ] **Positions as a single float per scroll.** Iris raised, and half
|
|
rejected, letting a scroll update one float rather than positions:
|
|
input handling cares about most elements in a list, so absolute
|
|
positions must be computed on the CPU anyway. LAYOUT.md's design
|
|
already lands here (GPU walks the chain, CPU resolves on demand for
|
|
hit tests). Keep the CPU resolution lazy and per query; do not
|
|
materialise every row's absolute position per frame.
|
|
- [ ] **Animations, last.** Cosmetic, so after everything above. Must be
|
|
**modular — a piece of the library rather than a core part forced into
|
|
everything, the same way input is**. Whatever the mechanism, a widget
|
|
that does not animate must pay nothing and import nothing for it.
|
|
|
|
## Build (for the port)
|
|
|
|
Widgets `RUST.md`'s "The port, in order (decided 2026-09-05)" needs and
|
|
iris does not have yet, one entry per gap, named against the P-step that
|
|
first needs it. Move an entry up to "Fix" or tick it in place once built;
|
|
do not duplicate it there.
|
|
|
|
- [ ] **A history-paging cushion measured in on-screen viewports, not a
|
|
row count.** (**P1**.) `iris::widget::List` has no equivalent of the
|
|
Compose app's `HISTORY_SCREENS` — AGENTS.md's "Things that have
|
|
bitten" is explicit that a fixed row count under-fills a screen on a
|
|
tool-heavy transcript and over-fills one on a text-heavy one, so
|
|
whatever loads the next page has to ask the list how many viewports
|
|
are actually on screen, not assume a constant.
|
|
- [ ] **A scaled thumbnail/image widget for an in-transcript image.**
|
|
(**P1**.) `SessionImage.kt`'s bitmap decode-and-downscale has no iris
|
|
counterpart; iris's own image widget (used by `bench_images.rs`) draws
|
|
a loaded texture but does nothing about sourcing or scaling one from a
|
|
server-produced attachment.
|
|
- [ ] **A modal/dialog primitive.** (**P1**, reused by **P3** and
|
|
**P5**.) Needed for the session settings dialog, `UsageDialog`'s
|
|
equivalent, and the delete-with-`deleteForeign` confirmation with its
|
|
toggle switch. Build once, wherever it is first needed, rather than
|
|
once per screen that wants one.
|
|
- [ ] **A horizontal gauge/bar widget.** (**P1**.) For
|
|
`SessionUsageBar`'s equivalent — a bounded fill reflecting a fraction,
|
|
nothing fancier.
|
|
- [ ] **A `BusyItem` equivalent: a dimmed row carrying an operation
|
|
label that does not block its list's own scroll/drag.** (**P3**.) The
|
|
Compose version tried an overlay first and it swallowed the drag along
|
|
with the tap (AGENTS.md's "Shared appearance") — worth not repeating
|
|
that attempt in iris before building the row-level version directly.
|
|
- [ ] **A toggle switch.** (**P3**.) For the delete dialog's
|
|
`deleteForeign` control; iris has no switch/checkbox widget yet as far
|
|
as this pass found.
|
|
|
|
## Reconsider
|
|
|
|
- [ ] **`WidgetView`.** Iris is unsure of it: what she wants is an easy way
|
|
to compose a widget from others (a button is the main case). With
|
|
sizing folded into `draw`, composing may be easy enough that `View` is
|
|
redundant. Decide after the layout change lands, by writing a button
|
|
both ways and keeping the one that is shorter to explain; delete the
|
|
other rather than keeping two ways.
|
|
|
|
## Build (asked for by Iris, 2026-09-06): a density-independent length unit
|
|
|
|
- [x] **A third length kind beside relative and pixels, so display scales
|
|
"just work".** Done 2026-09-06 — `Len::dp`/`len_fns::dp`, resolved
|
|
against `UiRenderState`/`Painter::density()` at `apply_rest` time; text
|
|
additionally rasterises at the resolved (physical) size instead of
|
|
scaling a low-resolution bitmap afterward, which was making text blurry.
|
|
`Span::gap`/`Padding` moved from `f32` to `Len` so they take `dp(...)`
|
|
too; transcript-ui's row/composer padding and one example migrated.
|
|
`em` was not added — nothing in this pass needed a text-relative unit,
|
|
and `dp`'s own doc says why it and physical pixels are kept as separate
|
|
fields rather than one the caller pre-multiplies. Not yet verified on
|
|
Iris's own phone at two densities (this pass had no device) — see
|
|
docs/RUST.md's P0 box and docs/IRIS.md's 2026-09-06 entry for what to
|
|
check. Iris's words: "another length type similar to absolute &
|
|
relative, so instead there would be relative, pixels, and another unit
|
|
like em or whatever is standard. That way different display scales
|
|
should just work." Today a length is either a fraction of the parent
|
|
(`rest`/relative) or physical pixels, and the phone drew 16 px text at
|
|
roughly a third of its intended size until the P0 fixes applied the
|
|
display's scale factor globally. That global scale is a stopgap for the
|
|
benchmark; the real shape is a unit resolved against the display's
|
|
density at layout time — Android's `dp` / CSS's reference pixel is the
|
|
standard (1 unit = 1/160 in), with `em` as the text-relative option —
|
|
so a widget author writes `16.dp()` once and never sees the scale.
|
|
Done when: `Length` (or whatever the enum is called) has the third
|
|
variant; every place that resolves a length takes the density; the
|
|
examples and `transcript-ui` use the new unit for text sizes, padding
|
|
and control sizes; the emulator at two densities and the phone draw the
|
|
same layout at the same physical size. After the bench setup is
|
|
finished, before P1 draws any new screen.
|
|
|
|
## From the phone, bench v2 (2026-09-06): streaming re-lays out the whole message
|
|
|
|
- [ ] **Streaming a delta into a long message costs a full text layout of
|
|
that message.** Iris's phone report (`docs/bench/iris-phone-v2-2026-09-06.md`):
|
|
the stream phase is the one place iris is behind Compose (p50 18.2 ms vs
|
|
13.4 ms; p99 level at ~43 ms). `TranscriptScreen::apply` replaces only
|
|
the last row, but that row is the growing message, and replacing it
|
|
re-renders its markdown and re-shapes the entire paragraph run through
|
|
parley on every event. Compose pays a reparse (8.6 ms mean) for the
|
|
same event. What "done" looks like: a streamed delta re-lays out only
|
|
the block it lands in (the last paragraph or code block), with earlier
|
|
blocks' layouts kept -- which needs a row to be a column of per-block
|
|
`Text`s rather than one `TextEdit` for the whole message, or parley's
|
|
layout to be split at block boundaries; measured by the stream phase's
|
|
p50 dropping below Compose's on the phone. Do this after the four bench
|
|
v2 defects (stale primitives, finger fling, decay curve, IME show) are
|
|
closed, since they are what make the run unrepresentative today.
|