ai-app/docs/IRIS_TODO.md

iris: known problems and things still to build

Iris's own list for the library, recorded 2026-09-04 in her words where it matters, so the agents working through RUST.md pick these up in a sensible order rather than rediscovering them. Each item says where it sits in the order and what "done" looks like. Tick and date them in place.

Fix

• request_device asked for compute-shader limits it never uses (2026-09-05). Limits::default() (both iris/src/android/render.rs and iris/src/default/render.rs) requests desktop-tier compute limits unconditionally, even though nothing in iris/iris-core creates a ComputePipeline or writes a @compute shader stage — confirmed by grepping the whole tree, not assumed. That crashed device creation outright on the Android emulator's software GL path (EMU_GPU=software, --features force-gles): SwiftShader's GL reports itself as OpenGL ES 3.0, which has no compute shaders, so the adapter's real limit is 0 against the unconditional request for 65535 — the same would happen on any real GLES-3.0-only Android device. Fixed by a new, shared iris_core::device_limits() (iris/core/src/render/mod.rs) that zeros exactly the six max_compute_* fields rather than switching to a downlevel Limits preset — downlevel_webgl2_defaults() also zeros max_storage_buffers_per_shader_stage, which shader.wgsl's vertex stage needs (four var<storage> buffers), so that preset would trade this crash for a bind-group-layout one on the same hardware. rigs/gpu-probe's own hand-mirrored Limits (it is deliberately its own crate, not able to call device_limits() directly) was updated to match. See DECISIONS.md and RUST.md's I5 box for the account, including what could not be re-verified on-device this pass (the emulator was in concurrent use by another session).

• Input does not fall through by input type (2026-09-04). SensorUi::run_sensors (src/default/sense.rs) used to set "consumed, stop checking lower layers" from mere hover — a widget registered for nothing but click() blocked a Scroll meant for whatever was behind it, since "the cursor is over this widget" and "this widget handled the event" were the same check. Fixed by judging consumption per input kind: with no button transition and no scroll happening this frame ("momentary" activity), the topmost hovered widget still wins, same as before; when something momentary is happening, only a widget whose registered senses actually include a matching non-hover one (checked via a new TypeEventManager::registered, which lists what a widget registered without running anything) consumes it, so a widget with only Hovering/click handlers can no longer block a scroll from reaching a list underneath. iris/src/sense_tests.rs builds a button-over-a-list Stack with a plain HasEvents impl (no GPU or window) and checks both directions: a scroll over the button reaches the list, and a real click still reaches the button — confirmed to fail on the pre-fix code and pass after.

• Appending one image to an already-loaded list rebuilds every other image's bind group (2026-09-05, fixed 2026-09-05). Found by the benchmark below: GpuTextures::update (core/src/render/texture.rs) triggered rebuild_image_bind_groups — a loop over every live standalone image, rebuilding its BindGroup — whenever the shared masks or move_offsets GPU buffer was resized (masks_resized || moves_resized in UiRenderNode::update, core/src/render/mod.rs), and a widget getting its first move-offset slot (LAYOUT.md section 2 — every widget gets one on first draw) could be exactly what grows that buffer. So one new message with one new image, appended to a transcript that already has N images loaded, did not cost O(1): it cost one create_image for the new image plus one make_image_bind_group per existing image, because the new widget's own move slot pushed the arena past its capacity. Measured directly in iris/examples/bench_images.rs: appending a 1,001st image to 1,000 already-settled ones reported 1,001 bind-group creates for that one frame, not 1 (./run-bench.sh images, frame 5 in the transcript below).

  Fix: masks/move_offsets never belonged in a standalone image's own bind group (group 2) in the first place — the group also holds that image's own texture view, which is the only thing that is genuinely per-image, so a buffer shared by everything forced a rebuild of every group the moment it moved. Gave masks/move_offsets their own bind group (group 3 in shader.wgsl and UiRenderNode: masks_layout/ masks_group), bound once per frame in UiRenderNode::draw rather than once per draw call, instead of duplicating them into every per-image group. GpuTextures and its image bind groups now know nothing about either buffer — rebuild_image_bind_groups is called only from grow_array (the atlas array texture growing, which genuinely does change what every image's own bind group must reference) — so a masks/move_offsets resize now touches exactly one bind group, ever, regardless of how many images are live. Numbers after the fix, same benchmark and command:

  ./run-bench.sh images
  frame=1 bind_group_creates=1000   (cold load, unchanged)
  frame=2 bind_group_creates=0      (was 1000 -- see the item below)
  frame=3 bind_group_creates=0
  frame=4 bind_group_creates=0
  (append one image here)
  frame=5 bind_group_creates=1      (was 1001)
  frame=6 bind_group_creates=0
  

  run-headless.sh tabs --shot still 27266 bytes, byte-for-byte unchanged, confirming the bind-group restructuring changed nothing about what is drawn.

• Bind-group creation takes two frames to reach the steady state, not one (2026-09-05, closed by the fix above, 2026-09-05). Same benchmark: loading 1,000 images cold used to report 1,000 creates on frame 1 (expected — create_image, one per new image) and again 1,000 on frame 2, before settling to 0 from frame 3. This was rebuild_image_bind_groups firing a second time for the same masks/move-offsets buffer-growth reason as the item above, confirming the guess recorded here — the two were exactly the same root cause measured two different ways. Frame 2 now reports 0 (see the numbers above); not a separate fix.

• A read-only text display has no widget of its own — P0's bench report area is a TextEdit standing in for one (2026-09-05). The only way to get selectable text on screen today is .editable(...) plus .attr::<Selectable>(()) (Selectable is only implemented for TextEdit, iris/src/attr.rs), which also makes the field focusable — tapping the bench report opens the soft keyboard over text nothing lets you type into. Harmless for a bench-only debug screen (not fixed this pass), but a real "selectable, not editable" text primitive would remove the keyboard side effect and is worth having before another screen wants the same thing (P1's own transcript rows already read their content from a TextEdit for the same reason).

From the phone, 2026-09-06

Found on Iris's own phone while working RUST.md's P0 box's phone-report follow-ups. Recorded here rather than fixed in that pass, so a follow-up agent takes them without colliding with that pass's bench_client.rs/ android/view.rs/android/sense.rs changes.

• Swiping has no momentum. iris's List pans exactly as far as the finger moves and stops dead on release -- unlike Compose, which flings and decelerates. Needs velocity tracking over the last several move samples and an Android-style decelerating fling, cancelled by the next touch-down, redrawing every frame until it settles. BenchRun.kt's own fling phase (RUST.md's "Benchmark v2" box) is the reference shape: "travel way faster... which is better for stress testing."
• Scrolling down sometimes jitters the text. Two named suspects, neither confirmed: DragArbiter's slop being released as one jump (iris/src/sense.rs), or a per-frame pan delta applied a frame late. Measure by tracing the list's scroll offset per frame against a monotonic synthetic drag, the same way iris/android-app/trace-draw.sh- style instrumentation traced the touch-scroll dropout in RUST.md's I5 box -- not by eyeballing a screenshot.

Build

• Benchmarks, not unit tests, run on demand (2026-09-05; a benches/ or a script under iris/, never in cargo test). The scenario that matters most is a message list — chat apps and this app's transcript alike — stressed with many messages and many images. One case in particular: resizing an input box (typing enough text to grow it) that pushes a long list of messages above it must stay very fast and recalculate almost nothing — a move of everything above, not a re-layout. That is exactly the O(1) move chain in LAYOUT.md; the benchmark is what proves it. Done when the numbers are in this file with the command, and the input-box case reports draws re-run, not just frame time.

  Built as two rigs, chosen per scenario by whether a real wgpu device is needed (UiRenderState/Widgets touch no GPU or window, so most of this runs as an ordinary binary — the same property layout_tests.rs relies on):

  • iris/benches/message_list.rs — a plain Instant-timed binary ([[bench]] harness = false in iris/Cargo.toml), not criterion: see the file's own header for why (short version — every scenario here reduces to a count UiRenderState::take_counters already produces, which criterion's statistical machinery adds nothing to and which a new dependency is not worth pulling in for). Covers (a) first-frame cost of a message list of N wrapped-text rows (one in 20 also carrying a small in-memory image) for N = 100/1,000/10,000; (b) per-frame cost of scrolling that list, 200 ticks; (c) the input-box case — a fixed-height field at the bottom of the screen growing by a line 40 times, with the message list above it filling the rest of the screen. Run: cd iris && cargo bench --bench message_list (always release — cargo bench builds the bench profile, which is optimized).
  • iris/examples/bench_images.rs — needs a real device, so it runs through iris/run-headless.sh bench_images, printing UiRenderNode::take_image_bind_group_creates() (a new counter, added in core/src/render/texture.rs and core/src/render/mod.rs, mirroring UiRenderState::take_counters) each frame. Covers (d): 1,000 image rows, checked both cold (does bind-group creation reach zero once loaded) and after appending one more image once settled (does that stay cheap) — the second question is what actually matters for a live transcript and is what turned up the two Fix items above.
  • iris/run-bench.sh [list|images] runs either or both and is what to run before/after touching Scroll, Span, Sized, the move-offset chain, or GpuTextures.

  Numbers (2026-09-05, release, cargo bench/run-headless.sh, this VM: AMD Ryzen 7 3800X, 8 cores, rustc 1.98.0 nightly-2026-09-03):

  cd iris && cargo bench --bench message_list
  (a) first frame, N=100:    30.30ms  draws=227   rewrites=15   moves=0
  (a) first frame, N=1000:  186.04ms  draws=2252  rewrites=150  moves=0
  (a) first frame, N=10000:1770.36ms  draws=22502 rewrites=1500 moves=0
  (b) scroll, N=100/1000/10000, 200 ticks each:
      draws=200 rewrites=0 moves=200 (identical at every N)
      per-tick average: 0.0002ms (identical at every N)
  (c) input grows 40 lines, N=100/1000/10000 rows above it:
      draws=320 rewrites=40 moves=160 (identical at every N)
      per-line average: 0.0012-0.0013ms (identical at every N)
  
  cd iris && ./run-bench.sh images        (2026-09-05, before the fix)
  frame=1 bind_group_creates=1000   (cold load)
  frame=2 bind_group_creates=1000   (see Fix item above)
  frame=3 bind_group_creates=0
  frame=4 bind_group_creates=0
  (append one image here)
  frame=5 bind_group_creates=1001   (see Fix item above)
  frame=6 bind_group_creates=0
  
  cd iris && ./run-bench.sh images        (2026-09-05, after the fix)
  frame=1 bind_group_creates=1000   (cold load, unchanged -- genuine work)
  frame=2 bind_group_creates=0
  frame=3 bind_group_creates=0
  frame=4 bind_group_creates=0
  (append one image here)
  frame=5 bind_group_creates=1      (one image's own create_image, O(1))
  frame=6 bind_group_creates=0
  

  Reading it: (a) is real, necessary work — shaping and laying out N never-before-seen text rows — and scales with N as it must, ~10x cost per 10x N. (b) and (c) are the pass conditions that matter: both are exactly flat across N = 100 to 10,000, confirming LAYOUT.md's O(1) move chain holds for both scrolling and for a growing input box pushing the message list — draws/moves per tick or per line do not grow with list size, and the per-operation cost (a fraction of a microsecond) is nowhere near a frame budget. (d)'s cold-load and steady-state halves behave as designed; its append half did not, until the fix above moved masks/move_offsets out of the per-image bind group — now flat at O(1) the same way (b) and (c) are.

• I5's transcript screen (iris/transcript-ui/, 2026-09-05) — what it left, each recorded at the point in the code it would go rather than silently dropped. See RUST.md's I5 box for the full account of what was built (the screen, SpanStyle, cross-row selection, the growing composer).

  • Android integration for this screen — done, 2026-09-05. iris-android-app's transcript-screen Cargo feature (transcript_client.rs) runs this screen against a real ai-server through client-core, confirmed on-device: real scrolling, real touch-drag panning, tap-by-name on the composer. Two real bugs found and fixed along the way (a missing INTERNET permission; a background-thread redraw request that crashed via a Looper requirement, fixed by routing through View::post_delayed — see IRIS.md's Tasks::redraw_handle entry). See RUST.md's I5 box, "The Android integration, done 2026-09-05" for the full account.
  • A render-time number for iris, comparable to Compose's transcript-bench.sh report — instrumentation done and a real number obtained, 2026-09-05 (later the same day); the clean comparable loop is not. iris_core::FrameReport (iris/core/src/render/ frame_report.rs, IRIS.md's new entry) times every frame from render()'s redraw start to after queue.submit+present(), exposed as two named on-screen controls ("Frame report", "Reset frame report"). Driven against a real on-device touch-drag it read frames=34 janky%=61.76 p50=26.5ms p90=48.0ms p99=98.1ms worst=98.1ms — real, not inferred, but accumulated across several gestures rather than one clean 24-swipe loop, because of the new finding below. See RUST.md's I5 box, "Update, 2026-09-05, later the same day" for the full account.
  • New, 2026-09-05: intermittent touch delivery to iris's SurfaceView under this checkout's EMU_GPU=software emulator. The same swipe coordinates, confirmed (by scanning a screenshot column for the first non-black pixel) to sit over real row text, sometimes produced 30+ real frames and a screenshot diff and sometimes produced zero of either, across otherwise-identical ui-trace invocations. Not the already-understood "already at that scroll edge" case (reproduced with content confirmed taller than the viewport, in both directions). Leading candidate, not yet confirmed: this checkout's emulator was independently observed at ~78% of one CPU core, continuously, while idle on-screen — SwiftShader's software rasterisation is CPU-bound by design, and a synthetic touch competing with that load for delivery is plausible but unmeasured during a failing gesture (the standing rule against diagnosing from after-the-fact measurements applies here). Needs a sampler (load, dumpsys input, a -i 0 ui-trace capture) running while a failing gesture is driven, and ideally a comparison under -gpu host (real Vulkan) to see whether it is specific to software rendering. This is what blocks the clean, comparable 24-swipe loop above.
  • Long-press-then-drag-to-select — confirmed on-device, 2026-09-05 (later the same day). ui-trace gained a holddrag X1 Y1 X2 Y2 HOLD_MS MOVE_MS action (emulator-tools, additive, extends the same MotionEvent/injectInputEvent mechanism swipe already used): press, hold past LONG_PRESS, move, release, as one continuous touch. Driven against a real row (holddrag 300 1850 300 2050 600 300) it produced iris selection: begin at row ... then a sequence of iris selection: extend to row ... log lines (transcript-ui/src/selection.rs, a new small log dependency since selection has no accessibility label of its own yet — see the next item), and a screenshot taken right after shows the expected highlighted selection spanning multiple rows. DragArbiter's own unit tests already covered this sequence against a synthetic clock; this is the first time it has been driven by a real device touch.
  • Touch-drag panning over a row's own rendered text — done, 2026-09-05. row.rs used to register CursorSense::click_or_drag() on each row's TextEdit for cross-row selection; TextEdit::draw's painter.child_layer() (iris/src/widget/text/edit.rs:87) meant that registration won core/src/sense.rs::run_sensors's per-layer arbitration on every frame it was pressed, not just the frame the press started, so a list pan gesture registered on List itself never got a turn while a row was under the finger. Fixed with iris::sense::DragArbiter (recorded in IRIS.md), one small state machine per list deciding pan vs. select the way Android does (a vertical drag pans immediately; a stationary press held LONG_PRESS (500ms) starts a selection which further drag extends; a horizontal drag while something is already selected extends immediately) — transcript-ui/src/selection.rs's Selection::drag is the one place every row's drag now routes through. 8 new unit tests (iris/src/sense.rs's drag_arbiter_tests); cargo fmt/clippy/test --workspace and cargo ndk (both iris and transcript-ui) all clean; run-headless.sh screenshot byte-identical to before the change (38578 bytes). See RUST.md's I5 box, "Gap closed, 2026-09-05".
  • Intermittent touch-scroll dropout — root-caused and fixed, 2026-09-05. Not the coalesced-ACTION_MOVE hypothesis the earlier pass suspected (ruled out): a gesture's ACTION_DOWN can land on a row's own padding/gap or its header, which no CursorSense covers, so DragArbiter never gets press_start and sits in Idle (answers Undecided forever) for that whole gesture. Fixed via a new DragArbiter::is_idle() that Selection::drag (transcript-ui/src/selection.rs) checks to recover a missed press on the next Pressing frame. Four new unit tests. See RUST.md's I5 box, "Touch-scroll dropout root-caused, 2026-09-05", for the trace and what a peer session sharing this checkout's emulator mid-pass prevented from being re-verified end-to-end (the aggregate iris-scroll.sh three-run confirmation and a re-taken FrameReport row) — a future pass should finish that once the emulator is free.
  • Row-level accessibility names. The composer carries .label("Message"); transcript rows do not carry a .label() of their own yet, so Widgets::named() (I4) does not include them — row.rs's build_text_row is where one would go, keyed to something stable per row (its sender + a short excerpt, matching what a screen reader announcing a chat message would say).
  • A tappable link and a background chip behind inline code. Both need per-range glyph geometry that TextEditCtx does not expose outside iris::widget::text (edit.rs's layout() helper is private) — see markdown.rs's module doc for the exact shape the fix would take (the same primitive TextEdit::draw's own selection highlight already uses internally, iris/src/widget/text/edit.rs:99).
  • Selection's anchor-row shortcut. The row a drag started in is selected in full (select_all) the moment the drag leaves it, rather than "from the click point to whichever edge points away from the drag" — needs the same private layout() access as the item above. selection.rs's module doc has the exact reasoning.
  • No syntax highlighting inside a fenced code block. client_core::highlight exists (built for the file explorer) and could feed per-token SpanStyles into a code block's span; wiring it in was not attempted this pass.

• Masks defined relative to each other. Wanted: mask A multiplies by something and also applies mask B — a mask can reference a parent mask, the way the move chain references a parent offset. Today masks are independent regions. Design it beside the move chain (same shape: a parent index and a bounded walk in the shader); do it when a real widget needs it, not before.

• Positions as a single float per scroll. Iris raised, and half rejected, letting a scroll update one float rather than positions: input handling cares about most elements in a list, so absolute positions must be computed on the CPU anyway. LAYOUT.md's design already lands here (GPU walks the chain, CPU resolves on demand for hit tests). Keep the CPU resolution lazy and per query; do not materialise every row's absolute position per frame.

• Animations, last. Cosmetic, so after everything above. Must be modular — a piece of the library rather than a core part forced into everything, the same way input is. Whatever the mechanism, a widget that does not animate must pay nothing and import nothing for it.

Build (for the port)

Widgets RUST.md's "The port, in order (decided 2026-09-05)" needs and iris does not have yet, one entry per gap, named against the P-step that first needs it. Move an entry up to "Fix" or tick it in place once built; do not duplicate it there.

• A history-paging cushion measured in on-screen viewports, not a row count. (P1.) iris::widget::List has no equivalent of the Compose app's HISTORY_SCREENS — AGENTS.md's "Things that have bitten" is explicit that a fixed row count under-fills a screen on a tool-heavy transcript and over-fills one on a text-heavy one, so whatever loads the next page has to ask the list how many viewports are actually on screen, not assume a constant.
• A scaled thumbnail/image widget for an in-transcript image. (P1.) SessionImage.kt's bitmap decode-and-downscale has no iris counterpart; iris's own image widget (used by bench_images.rs) draws a loaded texture but does nothing about sourcing or scaling one from a server-produced attachment.
• A modal/dialog primitive. (P1, reused by P3 and P5.) Needed for the session settings dialog, UsageDialog's equivalent, and the delete-with-deleteForeign confirmation with its toggle switch. Build once, wherever it is first needed, rather than once per screen that wants one.
• A horizontal gauge/bar widget. (P1.) For SessionUsageBar's equivalent — a bounded fill reflecting a fraction, nothing fancier.
• A BusyItem equivalent: a dimmed row carrying an operation label that does not block its list's own scroll/drag. (P3.) The Compose version tried an overlay first and it swallowed the drag along with the tap (AGENTS.md's "Shared appearance") — worth not repeating that attempt in iris before building the row-level version directly.
• A toggle switch. (P3.) For the delete dialog's deleteForeign control; iris has no switch/checkbox widget yet as far as this pass found.

Reconsider

• WidgetView. Iris is unsure of it: what she wants is an easy way to compose a widget from others (a button is the main case). With sizing folded into draw, composing may be easy enough that View is redundant. Decide after the layout change lands, by writing a button both ways and keeping the one that is shorter to explain; delete the other rather than keeping two ways.

Build (asked for by Iris, 2026-09-06): a density-independent length unit

• A third length kind beside relative and pixels, so display scales "just work". Iris's words: "another length type similar to absolute & relative, so instead there would be relative, pixels, and another unit like em or whatever is standard. That way different display scales should just work." Today a length is either a fraction of the parent (rest/relative) or physical pixels, and the phone drew 16 px text at roughly a third of its intended size until the P0 fixes applied the display's scale factor globally. That global scale is a stopgap for the benchmark; the real shape is a unit resolved against the display's density at layout time — Android's dp / CSS's reference pixel is the standard (1 unit = 1/160 in), with em as the text-relative option — so a widget author writes 16.dp() once and never sees the scale. Done when: Length (or whatever the enum is called) has the third variant; every place that resolves a length takes the density; the examples and transcript-ui use the new unit for text sizes, padding and control sizes; the emulator at two densities and the phone draw the same layout at the same physical size. After the bench setup is finished, before P1 draws any new screen.