iris/ai-app
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Prune commentary and stale Rust port notes

Browse Source
This commit is contained in:
iris committed 2026-09-10 00:44:13 -04:00
1 parent 5428cd75c9
commit 25370731d0
193 files changed
+693 -16219
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/CLIENT_CORE.md
+55 -202
View File
@@ -1,221 +1,74 @@
# `app-rust`'s `client` module
**Renamed 2026-09-08.** This was the `client-core` crate; it is now
`app-rust/src/client/`, a module of the one app crate rather than a crate
of its own (docs/RUST.md's "One app crate"). Nothing about what it *is*
changed: it is still the app's pure logic held once instead of twice, per
RUST.md's recommendation item 1, and still has **no UI framework
dependency of any kind** -- the `iris` dependency sits behind the `screens`
feature and nothing under `src/client` may reach it. That independence is
what lets it outlive whichever framework the app draws with, and it is now
an invariant of a module rather than of a manifest, so it is worth stating
plainly: a `use iris::` under `src/client/` is a defect.
`app-rust/src/client` contains platform- and UI-independent client logic. It
must not depend on iris; a `use iris::` below this directory is a layering
defect. `event-model` remains a separate crate because the server and client
both depend on that wire contract.
`event-model/` stayed a crate, and is the one split in the port that was
never optional: it is the wire shape both this app and `server/` depend on,
extracted from `server/src/session/driver.rs` and `session/transcript.rs`
on 2026-09-04, so a crate is what makes the two agree by construction.
## Contents
Paths below are written as `src/client/…`, relative to `app-rust/`.
- `api.rs`: REST client over the injectable `Transport` trait.
- `sse.rs` and `event_stream.rs`: SSE framing and session-event following.
- `transcript_cache.rs`: bounded, persistent transcript chunks.
- `transcript_source.rs`: cache/server selection and live cache updates.
- `transcript_fold.rs`: event folding, tool grouping, and page healing.
- `markdown_blocks.rs`, `ansi.rs`, and `highlight/`: display-independent text
parsing and spans.
- `config.rs`: enrollment-link parsing and the shared `EnrolledServer` value.
- `log_ring.rs`: bounded process-local diagnostics.
## What's here, and what Kotlin file it replaces
## API coverage
| `src/client/…` | Kotlin original | Status |
|------------------------------------------|-------------------------------------------|--------|
| `event-model/src/lib.rs` (shared crate) | `Events.kt` (the enum mirror) | Done |
| `ansi.rs` | `Ansi.kt` | Done, ported test-for-test |
| `highlight/mod.rs`, `languages.rs` | `Highlighter.kt`, `Languages.kt` | Done, ported test-for-test |
| `highlight/markdown.rs` | `MarkdownSyntax.kt` | Done, ported test-for-test |
| `transcript_cache.rs` | `TranscriptCache.kt` | Done, ported test-for-test |
| `sse.rs` | `Sse.kt` (the framing half) | Done, new tests (Kotlin had none of its own beyond integration) |
| `api.rs` | `Api.kt` | Partial -- see below |
| `event_stream.rs` | `EventStream.kt` | Done |
| `transcript_fold.rs` | `TranscriptItems.kt`, `ToolRows.kt` | Done -- see below |
| `config.rs` | `ServerConfig.kt`'s `handleEnrollment` | New, desktop-only so far -- see below |
| `transcript_source.rs` | `TranscriptSource.kt` | Done -- see below |
| *(not ported, and may never be)* | `TranscriptUnits.kt` | Out of scope -- see below |
`ApiClient` covers session list/read, messages, unqueue, answers, interrupt,
stop/start, rename, working directory, model, permission mode, notification
setting, commands, compaction, deletion, and transcript pages.
Every file above whose Kotlin counterpart had a JVM unit test (`AnsiTest`,
`HighlighterTest`, `TranscriptCacheTest`) has had every one of those test
cases ported alongside it, plus new tests for the pieces that had none
(`sse.rs`, `api.rs`, `event_stream.rs`, `transcript_fold.rs`,
`transcript_source.rs` -- the Kotlin `TranscriptSource.kt`/`TranscriptItems.kt`
had no JVM unit tests of their own, so these were written fresh against the
Kotlin source and AGENTS.md's paging incidents as the spec). Test count by
crate as of this writing: **109 in `client-core`**, 0 in `event-model` (its
types carry no logic of their own to test -- `server/`'s own tests exercise
them via `session::transcript`'s round-trip coverage).
Still missing are setups and discovery, file operations, usage, models and
downloads, attachments, imports, and the global notifications stream.
`server/src/routes.rs` is the authoritative route table.
## Correspondence notes worth knowing before touching either side
## Transcript invariants
- **`ansi.rs`'s `StyledText`/`Style`/`Rgb`** stand in for Compose's
`AnnotatedString`/`SpanStyle`/`Color`, since this crate has no Compose.
`StyledText` is plain text plus a `Vec<(Range<usize>, Style)>` of
non-overlapping spans. Whatever UI framework ends up consuming this
crate maps `Style` onto its own text-styling type; nothing here should
change to accommodate a particular one.
- **`highlight`'s `Span`/`Kind`** use **char indices, not byte offsets**
(`Vec<char>` internally), mirroring the Kotlin original's `Char`-indexed
strings. `highlight::span_text` turns a `Span` back into text for a
caller working the same way; a caller that wants byte offsets into a
`&str` has to convert.
- **`transcript_cache.rs`'s `SessionCache::guard`** found a real
translation bug while it was being written: an early draft let a
*damaged* chunk (one file unreadable, discard just this session) and a
genuine I/O failure (disk gone, disable the whole cache) both surface as
the same `Err` from one closure, which would have disabled every
session's cache over a single corrupt chunk. Fixed by checking a
thread-local "was this damage" flag before deciding which failure mode
it was -- see the comment on `guard` and the commit message for
`transcript_cache.rs`.
`join_pages` heals messages and tool runs split across page boundaries. It
asserts that a tool id does not survive in both halves. It must not assert
sequence ordering across the seam: a peer note carries the sequence of the
turn it belongs above and can legitimately interleave with the page where it
arrived.
## What `api.rs` covers, and what it does not yet
`Event` has no catch-all variant. A newer server adding an event type will
make an older client reject that line rather than draw a placeholder. Fixing
that requires a shared wire-model decision, not a client-only workaround.
`ApiClient` wraps a `Transport` trait (network I/O kept out from behind, so
`ApiClient` and `event_stream::follow_session_events` are tested with a
fake transport and no server). `UreqTransport` is the only real
implementation, backed by `ureq` -- see its Cargo.toml comment for why
(blocking, already a project dependency, no extra TLS crate needed since
`ureq::tls::Certificate::from_pem` reads the pinned CA directly).
`TranscriptSource::page(0, ..)` returns `OlderPage::NothingLoaded` without
touching cache or network. This is deliberately distinct from
`OlderPage::Events(vec![])`, which means the start of the conversation was
actually reached. Network and cache parse failures are errors for the same
reason: none of these states may latch a caller's “no more history” flag.
Covered: session list/read, message send, unqueue, answer, interrupt,
stop, start, rename, cwd, model, permission-mode, notify, command,
compact, delete, and one transcript page.
Fetched transcript lines retain the server's exact JSON bytes through
`RawValue`. Re-serializing parsed JSON can change floating-point text, causing
the cached and streamed forms of one event to disagree byte-for-byte.
**Not covered, and each is real work rather than a stub to fill in:**
setups (`/setups*`, machine and provider discovery), the file explorer
(`/setups/{id}/dir|file`), usage (`/usage`), models
(`/models*`, HuggingFace browsing and downloads), attachments
(`/sessions/{id}/attachments`), importing (`/setups/{id}/importable*`),
and the `/notifications` stream. `server/src/routes.rs`'s module doc is
the full table to work from when one of these is next.
The reconnect/backoff loop and cancellation of a live stream belong to the
embedding runtime. `TranscriptSource::follow` only guarantees that each frame
is cached before the caller receives it.
## What `transcript_fold.rs` covers, and what it does not yet
## Enrollment
`fold_event` covers every `Event` variant server/ can produce today,
including tool-call/question/image attachment and peer-message placement.
`group_tool_runs` groups adjacent calls into `TranscriptRow::Tools`.
`EnrolledServer` and `parse_link` understand the same
`aiapp://enroll?host=H&port=P&token=T[&ca=B]` value used by Android. Storage
is caller-specific: Android uses its platform storage and the desktop writes a
0600 file under its XDG config directory.
`join_pages` (with `heal_split_message` and `adopt_run`, both private) is
now ported too, 2026-09-06 -- the page-boundary healing that merges a tool
call split across two fetched pages, rejoins a message a boundary cut
through, and renames a run of tool calls onto whichever name is already on
screen. Ported with AGENTS.md's "things that have bitten" incidents as the
spec rather than a JVM test file (`TranscriptItems.kt` had none of its
own): `a_clean_boundary_between_two_finished_runs_is_still_healed_into_one_run`
is the regression test for the bug that shipped -- `adopt_run` must run on
*every* join, not only the one where a split call was found, or a boundary
landing cleanly between two already-finished calls (most of them) leaves
one run drawn as two. `a_call_split_across_the_boundary_merges_into_one_row`,
`a_message_split_across_the_boundary_is_rejoined_with_the_newer_halfs_identity`,
and `adopt_run_never_renames_into_a_question_row` cover the other three
edges the Kotlin doc calls out. `join_pages` ends in a `debug_assert!`
that no tool id survives in both halves -- the duplicate row it exists to
prevent, checked rather than assumed. What it deliberately does *not*
assert is seq ordering across the boundary: a peer note carries the seq
its turn began at (`place_peer_note`), which can be older than the page
it arrived in, so the two pages' seqs legitimately interleave there. An
earlier draft asserted it and would have panicked in debug builds on an
ordinary transcript.
## Markdown scope
**Known gap, and a decision for whoever closes it:** `event_model::Event`
has no `Unknown`/catch-all variant, unlike `Events.kt`'s hand-kept mirror.
A server newer than this build that adds an event type will fail to parse
that line rather than degrading to a placeholder row. Closing this means
deciding how `event_model` itself represents "a shape I don't recognise"
-- a shared-model decision affecting `server/` too, not a `client`-only
fix, so it is recorded here rather than silently worked around.
`markdown_blocks` splits top-level headings, paragraphs, fences, lists,
tables, and quotes. It intentionally does not build a full nested CommonMark
AST; inline styling and nested presentation remain renderer concerns until a
shared non-UI consumer needs them.
## `config.rs`: `EnrolledServer`
## Verification
`EnrolledServer` (host, port, bearer token) plus `parse_link`, which reads
the exact `aiapp://enroll?host=H&port=P&token=T` deep link
`wg-app-link`'s `enroll` mints and `ServerConfig.kt`'s `handleEnrollment`
parses on the phone -- so any Rust client enrols from the same text a
phone would scan as a QR, with no second format invented for it (RUST.md's
E4, decided 2026-09-05). Deliberately does not decide where it is
persisted or under what file permissions -- a phone seals its token in the
Android Keystore, `src/desktop/config.rs` writes it to
`$XDG_CONFIG_HOME/ai-app-desktop/enrollment.json` at 0600 -- since that is
caller-specific (the code rules' "ask for the least you need"). Its only
caller today is `src/desktop`; the Android entry point is a second one, not
a reason to move the type.
## What `transcript_source.rs` covers, and what it does not
`TranscriptSource<T: Transport>` is the seam a session screen asks for a
page, ported test-for-test against the Kotlin doc rather than a JVM test
file (there wasn't one): `cached_opening`, `probe`, `fetch_opening`,
`page` and `follow`, each matching its Kotlin namesake's contract --
including `probe`'s three-way outcome (matches / cache purged /
unreachable, told apart so a caller never treats "couldn't ask" as "was
wrong") and `page`'s cache-vs-server split bounded by `covered_up_to`.
Two additions beyond a literal port, both load-bearing:
- **`page(before, ..)` refuses `before == 0` before touching the cache or
the network**, answering `OlderPage::NothingLoaded`. This is AGENTS.md's
`loadOlderPage` incident (`before = 0` is "no event before the first
one," indistinguishable from "reached the start of history" if a caller
ever asks it) moved out of the Kotlin screen and into this layer, so
every future caller gets the guard rather than having to remember it.
**The return type is `OlderPage`, not a `Vec`, and that is the guard.**
The Kotlin's two falses are different answers -- `oldestSeq == 0`
returns without touching `moreHistory`, an empty page latches it false
-- so a port that answered both with an empty list would have moved the
bug rather than fixed it, one layer down and out of sight of the screen
that used to hold the check. `OlderPage::Events(vec![])` means the start
of the conversation; `OlderPage::NothingLoaded` is not an answer about
the conversation at all. Reviewed 2026-09-06.
`paging_before_the_first_event_makes_no_request_at_all` asserts zero
transport calls, not just the variant, since a request that happens to
answer empty is exactly what caused the original bug, and
`a_failing_server_page_is_an_error_rather_than_an_empty_one` plus
`an_unreadable_cached_page_is_a_parse_error_rather_than_an_empty_one`
are the same rule for the two ways a page can fail.
- **`fetch_transcript_lines`** (new in `api.rs`) hands back each line
paired with the exact server bytes it came from, via
`serde_json::value::RawValue` rather than re-serializing a parsed
`Value` -- the cache and a live SSE frame for the same event have to
agree byte-for-byte, which is exactly what the `serde_json`
float-rounding bug (AGENTS.md) was about. The existing
`fetch_transcript_page` is untouched (other callers under `iris/`
depend on its signature); the two share a `transcript_path` helper so
the query string is written in one place.
**Not ported:** `EventStream.kt`'s reconnect-with-backoff loop, and
`TranscriptSource.close`'s ability to cancel a live stream from another
thread. Both are wall-clock/thread-lifetime policy that belongs to
whichever runtime embeds this crate (iris's own timers, a Tokio task, a
Kotlin coroutine scope), not to this pure logic -- `follow` is the same
"write to the cache, then hand the frame to the caller" decorator
`src/desktop/app.rs` and `src/android/transcript_client.rs`
already hand-wrote around `event_stream::follow_session_events` before this
existed; the cache write moved into one shared place so a third caller
does not repeat it again by hand.
## What is not started at all
- **A full markdown AST.** `markdown_blocks` (2026-09-06) splits a message
into its *top-level* blocks -- heading, paragraph, fence, list, table,
quote -- with each block's own source, which is what a renderer needs to
lay out prose versus code and what lets a streamed delta re-lay out one
block instead of the message (docs/RUST.md's Task B). What it
deliberately does **not** build is the tree below that: nested list
items, table cells, inline spans. Inline styling is still the renderer's
own job per block (`src/ui/markdown.rs`), and nothing
has needed the rest yet. `CodeFence.kt`'s use of `org.intellij.markdown`
for a full CommonMark AST is Compose rendering plumbing, not something
to port as-is.
- **`TranscriptUnits.kt`** (see above) -- deliberately out of scope, since
it flattens a row into bounded units for a *specific* lazy-list
framework's composition cost, which is a fact about that framework
rather than about the transcript.
## Verifying
`./scripts/run-tests.sh` from the repo root runs `event-model`, `server` and
`app-rust` in that order (each `cargo test`, forwarding arguments the same
way it always has). From `app-rust/` directly: `cargo test`, `cargo clippy
--all-targets`, `cargo fmt` -- all clean as of 2026-09-08, 229 tests across
the crate and its headless harness suites.
Run `./scripts/run-tests.sh` from the repository root. For this crate alone,
run `cargo test`, `cargo clippy --all-targets`, and `cargo fmt --check` from
`app-rust/`.
docs/IRIS_TODO.md
+10 -28
View File
@@ -1,14 +1,6 @@
# iris: known problems and things still to build
Iris's own list for the library, recorded 2026-09-04 in her words where it
matters, so the work in RUST.md picks these up in a sensible order rather
than rediscovering them. Each item says where it sits in the order and
what "done" looks like.
**Only open items live here.** An item is deleted when it lands, not
ticked: a list of finished work is context every future session pays for,
and what a change did belongs at the code it changed. Fifty closed items
and six phone-report sections went on 2026-09-08 for that reason.
Only open Iris framework work lives here. Delete an item when it lands.
## Fix
@@ -20,12 +12,8 @@ and six phone-report sections went on 2026-09-08 for that reason.
(17,17,27) became (73,73,91) in a desktop screenshot measured on
2026-09-06.
The earlier entry called this desktop-only because the Android picture
looked right. That was not a measurement: neither the startup line nor
the diagnostics report records the selected surface format, and the
Android backend contains the identical preference and shader path. A
device exposing only a non-sRGB surface can happen to hide the bug; it
does not make the pipeline correct.
Neither diagnostics nor startup logging records Android's selected surface
format. A device exposing only a non-sRGB surface can hide the shared bug.
Done means defining one convention for palette bytes, decoded images,
colour emoji and the clear colour, then converting exactly once for the
@@ -33,12 +21,10 @@ and six phone-report sections went on 2026-09-08 for that reason.
test that draws known non-black, non-white pixels into an sRGB target and
reads the stored bytes back; screenshots from desktop and Android then
confirm the same Catppuccin values rather than serving as the definition.
## Build (for the port)
Widgets `RUST.md`'s "The port, in order (decided 2026-09-05)" needs and
iris does not have yet, one entry per gap, named against the P-step that
first needs it. Move an entry up to "Fix" if it becomes a current defect;
delete it once built rather than duplicating it there.
Framework capabilities needed by `RUST.md`'s port plan:
- [ ] **Selectable, read-only text.** P0's report and P1's transcript rows
use `TextEdit` because `Selectable` is implemented only for it. That
@@ -88,18 +74,14 @@ delete it once built rather than duplicating it there.
`Widget::tick` and `UiData::animate`. A widget that does not opt in must
pay nothing and import nothing for them.
- [ ] **Remove `WidgetView` unless a real composite adopts it first.** The
layout change this decision was waiting for has landed. Every composite
in `app-rust/src/ui` now uses ordinary child handles plus a returned root;
- [ ] **Remove `WidgetView` unless a real composite adopts it.** Every
composite in `app-rust/src/ui` uses ordinary child handles plus a root;
`WidgetView` and its derive are used only by `iris/examples/view.rs`.
Today it demonstrates itself rather than shortening production code, so
deletion is the concrete default—not another parallel composition style.
It currently demonstrates itself rather than shortening production code.
- [ ] **A `Stack` that chooses its mask the way it chooses its size
(Iris, 2026-09-08).** She asked whether `masked_by` deserves to exist:
"a method that just does 2 separate things you can already easily do
does not deserve to exist." For a square-cornered surface it is indeed
redundant -- `.background(rect(BAR_FILL)).masked()` was measured
should replace `masked_by`.** For a square-cornered surface,
`.background(rect(BAR_FILL)).masked()` was measured
against `.masked_by(rect(BAR_FILL))` on the composer at the phone's own
size and density and the two are identical to the pixel. What the pair
cannot express is a clip that is not a box: `Painter::set_mask` writes
docs/LAYOUT.md
+50 -447
View File
@@ -1,27 +1,8 @@
# iris: one `draw` that records a size
Iris, 2026-09-04:
> I don't like that widgets need both a draw and size functions. I'd much
> rather them have a single draw that reports a size, and if it needs to be
> moved then that can be done after the fact efficiently, or resized just
> done after as well. This should be done efficiently like everything else
> tries to do right now.
**Implemented 2026-09-04; size dependencies made explicit 2026-09-09.**
Every widget was migrated in one change; none kept
`desired_width`/`desired_height`. `draw` no longer returns its size directly:
it records it once on its `Painter`, and a parent that reads a child draw's
`DrawResult::size()` records the retained dependency between them. What is
kept below is the design as it stands, the corrections implementation forced
(read those before
touching `Aligned`, `Sized`, `MaxSize`, `Scroll` or the move-slot lifecycle
in `render_state.rs` -- each is a real bug the first draft would have
reproduced), and the two later additions that build on it. The
pre-implementation framing -- what the old trait looked like, the checklist
the design had to answer, the migration list, the pass conditions and the
"copy this into the design log" note -- was deleted on 2026-09-08,
having been carried out.
A widget draws once and records its size on the `Painter`. Reading a child
`DrawResult::size()` records a retained size dependency; drawing the child
without reading that result does not make the parent's size depend on it.
## Design
@@ -63,187 +44,36 @@ set in the context, which makes this slow and not cool." Folding sizing into
an axis the widget declares without painter context or child access. A
lying hint fails a debug assertion when the widget is drawn.
### 2. Move: O(1) per moved subtree, via a per-widget offset chain
### 2. O(1) subtree movement
**What exists today, and why it is not O(1).** `UiRenderState::mov`
(`core/src/ui/render_state.rs:156-168`) fires when a widget's region keeps
its *size* but changes *position* (`draw_inner`, `:85-100`:
`active.region.size() == region.size()` after excluding the exact-match
case). It rewrites every primitive's `region` field via
`Primitives::region_mut` (`core/src/render/primitive.rs:176-179`) for the
widget's own primitives, then recurses into every child — O(primitives in
the subtree). Both call sites that trigger it today, `Scroll::draw`
(`iris/src/widget/position/scroll.rs:29-31`) and `Offset::draw`
(`iris/src/widget/position/offset.rs:9-11`), are "translate this subtree by
an abs pixel amount, `rel` framing unchanged" — a transcript scroll
re-touches every glyph in every visible row, every frame of the drag, and
I3's target is 800 rows on screen.
Every active widget owns a slot in `UiData::move_offsets`. A slot stores an
absolute-pixel delta and its parent slot; each primitive instance stores the
slot of the widget that drew it. The vertex shader walks this bounded chain
and adds the accumulated translation. Moving a subtree therefore writes one
slot instead of rewriting every descendant primitive.
**Recommendation: a per-widget offset slot forming a parent-linked chain,
resolved in the vertex shader.**
The parent chain is required for independently movable nested subtrees, such
as a swipeable row inside a scrolling list. A flat offset table would require
rewriting the row whenever an ancestor moved and would restore the very
O(subtree) work this design removes. Chain depth is bounded in both Rust and
WGSL.
- `UiData` (`core/src/ui/mod.rs:14-20`) gains
`pub move_offsets: TrackedArena<MoveOffset, u32>`, the same arena shape
already used for `masks: TrackedArena<Mask, u32>` on the line above it.
- `render/data.rs` gains `pub struct MoveOffset { pub delta: [f32; 2], pub
parent: u32 }` (`Pod`/`Zeroable`, `parent = u32::MAX` = "no ancestor,
add nothing more"). A pure abs-pixel translation, not a general
`UiRegion` remap — sufficient for every existing call site (above).
- `PrimitiveInstance` (`render/data.rs:11-18`) gains `pub move_idx: u32`,
a vertex attribute at `@location(7)` beside `mask_idx` at `6` — the same
kind of per-instance handle.
- `ActiveData` (`core/src/ui/active.rs`) gains `pub move_slot: MoveIdx`,
assigned **when the widget is first drawn** (`draw_inner`, beside
`active.insert`), with `parent` = the drawing widget's parent's slot.
`Painter` threads a `move_slot` field down exactly as it already threads
`mask` and `layer` (`painter.rs:9-20`), so a freshly-drawn descendant is
correct from its first frame — nothing is ever retrofitted onto an
already-active primitive. An unmoved widget's slot just stays `[0, 0]`.
- `Painter::primitive_at` (`painter.rs:23-38`) writes `move_idx:
self.move_slot`, matching how it already writes `mask_idx: self.mask`.
- `mov(id, delta)` becomes: look up `id`'s slot, write
`move_offsets[slot].delta += delta`. One write — no primitive touched, no
recursion, since descendants already reference this slot transitively.
- A container may also retain one optional **child-coordinate slot** between
its own slot and every direct child's slot. `Painter::set_child_offset`
creates that boundary before the first child is drawn and can update it
after measuring a child on later redraws. The container's own primitives,
hit region and mask stay fixed; its whole child subtree moves through one
write and every existing GPU, hit-test and accessibility chain sees the
same result. `LazySpan` uses this while still walking visible rows for
virtualisation: row boxes stay in stable local coordinates and the shared
boundary carries the changing screen translation.
- `shader.wgsl`'s vertex stage, after computing `top_left`/`bot_right` in
pixels (after `:106`, before the clip-space divide at `:113`), walks
`move_idx → move_offsets[i].parent` for a bounded number of steps (a
small constant, e.g. 16, with a CPU-side debug assertion that no chain
exceeds it), summing `delta` into both corners. Cost is O(chain depth),
paid every frame regardless of whether anything moved — negligible next
to the per-fragment texture sampling TEXTURES.md already measures this
GPU as not bound by.
`Painter::set_child_offset` inserts a retained coordinate slot between a
container and its direct children. `LazySpan` uses one so visible row boxes
remain stable while scrolling changes a single shared translation. Ordinary
window-relative positions remain `rel + abs`; move slots carry translation
only, not general remapping.
**Why the chain, not the flatter thing first proposed.** Iris's own
phrasing — "every instance carries an index into a small per-widget offset
buffer" — describes a flat table: one slot per subtree *declared* movable,
no parent link. It breaks the moment two such subtrees nest — a row inside
a scrolling list, itself later given its own animated offset (a
swipe-to-delete mid-scroll) — because the row's primitives would have to
pick one slot and lose the other's contribution. The chain costs one extra
field and a bounded shader loop in exchange for no such gap, and since
every `ActiveData` gets a slot unconditionally rather than lazily, it costs
no more at the common depth of one than the flat version would.
`UiRenderState::resolved_region` performs the same chain walk on the CPU for
hit-testing, accessibility, and public window-coordinate queries. Masks store
the move slot of their owning widget and resolve it independently in the
fragment shader, so a stationary viewport can clip moving content.
**Against `region_mut` as the steady-state mechanism**: rejected for being
O(primitives in the subtree) — the cost this section removes — but kept
for a resize that changes a region's `rel` component (a genuine reflow,
§3) and for a size-independent widget's resize (§3), where the content's
shape doesn't change and one field write already suffices.
Provisional layout can still write an instance at an intermediate position
and restore it before upload. `Primitives::set_instance` remembers the value
at the first write in a frame and clears the dirty bit when the final bytes
match it. The GPU therefore observes final layout state, not CPU-only
measurement work.
### 2b. Two more readers of "where is this widget," and masks
Moving the offset into the vertex shader means `ActiveData.region` is no
longer the on-screen truth once a widget has been moved — it is where the
widget was *drawn*, before any `move_offsets` delta. Two things read it as
if it still were, and both must move to a resolved query or they silently
answer with the pre-move position: a click landing on a scrolled row would
be routed to whatever used to be there, with nothing on screen to say so —
exactly the "wrong answer that looks like a right one" case the code rules
single out.
**Hit-testing.** `SensorUi::run_sensors` (`src/default/sense.rs:154-200`)
does the actual pointer routing, and line 170 is the read in question:
`let shape = self.active.get(id).unwrap().region;` (`self: &UiRenderState`),
immediately turned into pixels and tested against the cursor at `:171-172`.
Under this design that region must be resolved through the same chain the
GPU walks before it means anything. Add to `UiRenderState`:
```rust
/// `active[id].region`, corrected by every `move_offsets` delta between
/// `id` and the root — the CPU-side twin of the vertex shader's chain
/// walk, over the same arena, so the two cannot disagree about where a
/// widget is. O(chain depth), not O(primitives): a plain Rust loop over
/// `move_offsets`, bounded by the same constant the shader loop uses
/// (name it once, e.g. `render::MOVE_CHAIN_LIMIT`, and reference it from
/// the WGSL loop bound in a comment, since WGSL cannot `include!` a Rust
/// const across the language boundary).
pub fn resolved_region(&self, id: WidgetId) -> UiRegion;
```
`window_region` (`core/src/ui/render_state.rs:264-267`), the public
coordinate query already used outside hit-testing
(`src/default/attr.rs:15,17,70`, e.g. positioning one widget relative to
another's on-screen box), is reimplemented to call `resolved_region(id)`
before `.to_px(...)` instead of reading `.region` directly — one change
covers both call sites listed there. `sense.rs:170` changes to
`let shape = self.resolved_region(*id);`. Both are required the moment §2
lands, not an optional follow-up: an unmoved widget's chain is empty and
`resolved_region` costs one arena read to find that out, so there is no
version of this design where skipping the fix is a legitimate
optimization — it is a correctness gap, not a performance one.
**Masks.** `Painter::set_mask` (`core/src/ui/painter.rs:49-52`) bakes the
painter's *current* region into a `Mask` pushed onto
`masks: TrackedArena<Mask, u32>` (`core/src/ui/mod.rs:19`), and the
fragment shader clips every primitive against `masks[in.mask_idx]`'s raw
`rel`/`abs` fields, unaffected by any move (`shader.wgsl:147-157`). If the
widget that called `set_mask` — `Masked::draw`,
`iris/src/widget/mask.rs:7-11`, `painter.set_mask(painter.region()); ...` —
is itself later moved, its clip rectangle stays where it was drawn while
its content moves out from under it: a visibly wrong clip, immediately on
screen, not a latency question.
Fix: `Mask` (`core/src/render/data.rs:46-49`) gains `pub move_idx: u32`,
written from `Painter::set_mask` as `self.move_slot` — the identical slot
the mask-owning widget's own primitives already get (§2), not a second
mechanism. Resolution happens in the **fragment** shader, not the CPU, and
not the vertex shader either: `shader.wgsl`'s mask check (`:147-157`)
currently computes the mask's `top_left`/`bot_right` inline from
`masks[in.mask_idx]`; that computation is extended to walk the same
move-offset chain §2 added, via one shared function —
```wgsl
fn resolve_move(idx: u32) -> vec2<f32> { /* the bounded parent walk, used by both stages */ }
```
— called from `vs_main` for a primitive's own corners and from `fs_main`
for its mask's corners, so the walk is written once and the two stages
cannot drift apart (the sibling-rule from the code rules: one loop, not a
hand-copied second one in the other shader stage).
**Why the fragment shader, not a CPU-side mask rewrite at move time.** A
primitive's mask is frequently owned by a *different* widget than the
primitive itself — often several levels up a subtree, with its own,
independent move slot — so a primitive's resolved offset and its mask's
resolved offset are two different chain sums, both needed, and only the
fragment shader has both `in.move_idx` (this fragment's own chain) and
`in.mask_idx` (indirecting to a second, possibly unrelated chain) already
in hand per-fragment. Resolving mask regions on the CPU at move time would
mean, for every `mov()` call, walking forward to every mask instance the
moved widget's slot could affect and rewriting its raw region — exactly
the O(subtree) cost §2 exists to remove, just moved from primitives to
masks. The fragment shader already re-reads `masks[in.mask_idx]` every
frame (`:148`); one more arena read to resolve its chain costs nothing
extra in kind.
**The scroll-container case, checked rather than assumed.** A masked,
scrollable region is built as a `Masked` wrapping a `Scroll`
(`iris/src/widget/position/scroll.rs`, `iris/src/widget/mask.rs`) — the
viewport border is drawn (and `set_mask` called) by `Masked`, which is
never itself the target of `mov()`; only `Scroll`'s inner content is,
every frame the user drags. Because each widget's move slot is its own
(§2: assigned per `ActiveData`, not shared), `Masked`'s mask references
its own, stationary slot, while the scrolled content underneath references
a separate, deeper slot whose `parent` chain passes through — but does not
write to — the viewport's slot. Moving the content therefore never touches
the mask's resolved position, and the mask staying still while its content
slides past it is what this design already produces with no special case,
not an extra rule that had to be added for it.
Slots follow `ActiveData`'s lifecycle. Removing a widget recursively retires
its slot only after descendants are gone, and a reused arena slot is reset
before new primitives can reference it. `Primitives::set_instance` also
cancels a dirty mark when provisional layout restores the original bytes, so
CPU-only measurement positions are never uploaded.
### 3. Resize scope
@@ -366,91 +196,7 @@ widget observed during that same draw; the next draw replaces the list, so a
dependency disappears as soon as the widget stops reading it. Both fields
have `ActiveData`'s existing lifecycle through `remove`/`remove_rec`.
### 6. Before / after
**A leaf, `iris/src/widget/rect.rs`** — the size-independent case:
```rust
// before
impl Widget for Rect {
fn draw(&mut self, painter: &mut Painter) {
painter.primitive(RectPrimitive { color: self.color, radius: self.radius,
thickness: self.thickness, inner_radius: self.inner_radius });
}
fn desired_width(&mut self, _: &mut SizeCtx) -> Len { Len::rest(1) }
fn desired_height(&mut self, _: &mut SizeCtx) -> Len { Len::rest(1) }
}
```
```rust
// after
impl Widget for Rect {
fn draw(&mut self, painter: &mut Painter) {
painter.primitive(RectPrimitive { color: self.color, radius: self.radius,
thickness: self.thickness, inner_radius: self.inner_radius });
painter.set_size(Size::REST); // fills whatever it was given
}
fn is_size_independent(&self) -> bool { true } // content never depends on region size
}
```
**A container that needs the child's size before placing it,
`iris/src/widget/position/align.rs`**:
```rust
// before
impl Widget for Aligned {
fn draw(&mut self, painter: &mut Painter) {
let region = match self.align.tuple() {
(Some(x), Some(y)) => painter.size(&self.inner).to_uivec2().align(RegionAlign { x, y }),
(Some(x), None) => { let x = painter.size_ctx().width(&self.inner).apply_rest().align(x);
UiRegion::new(x, UiSpan::FULL) }
(None, Some(y)) => { let y = painter.size_ctx().height(&self.inner).apply_rest().align(y);
UiRegion::new(UiSpan::FULL, y) }
(None, None) => UiRegion::FULL,
};
painter.widget_within(&self.inner, region);
}
fn desired_width(&mut self, ctx: &mut SizeCtx) -> Len { ctx.width(&self.inner) }
fn desired_height(&mut self, ctx: &mut SizeCtx) -> Len { ctx.height(&self.inner) }
}
```
```rust
// after
impl Widget for Aligned {
fn draw(&mut self, painter: &mut Painter) {
let full = painter.region();
// Draw once at the full region to learn the child's real size --
// this placement is provisional and corrected below without a
// second draw.
let used = painter.widget_within(&self.inner, full).size();
let region = match self.align.tuple() {
(Some(x), Some(y)) => used.to_uivec2().align(RegionAlign { x, y }).within(&full),
(Some(x), None) => used.x.apply_rest().align(x).within(&full),
(None, Some(y)) => used.y.apply_rest().align(y).within(&full),
(None, None) => full,
};
painter.place(&self.inner, region);
painter.set_size(used);
}
}
```
`Painter::widget_within`/`widget`/`widget_at` (`painter.rs:55-76`) change
return type from `()` to `DrawResult`. Calling `.size()` reads the size the
child recorded on its painter and records the parent's dependency on that
answer; leaving it unread records no dependency.
`Painter::place` moves an already-drawn child when its used area fits the
target box, and redraws it when the target changes its size. `SizeCtx` and
`Painter::size_ctx`/`size`/`len_axis` (`painter.rs:141-150,
180-182`) are deleted — nothing calls `desired_len` any more, so there is
nothing left for `SizeCtx` to answer; `draw_text`/`label`/`px_size`/
`output_size` already exist redundantly on both `SizeCtx` and `Painter`
today (compare `size.rs:71-90` against `painter.rs:152-174`) and this
deletes the `SizeCtx` copies, keeping the `Painter` ones.
### 7. Rejected, and why
### 6. Rejected alternatives
- **A flat (non-chained) per-subtree offset table**, Iris's literal
phrasing — rejected in §2 for breaking under nested independent moves
@@ -477,7 +223,7 @@ deletes the `SizeCtx` copies, keeping the `Painter` ones.
but still not O(1), and the shader-side chain costs nothing extra to get
the better bound.
## Density: `Len::dp`, resolved at `apply_rest` time (2026-09-06)
## Density: `Len::dp`, resolved at `apply_rest` time
Iris asked for a third length kind beside `abs` (physical pixels) and
`rel`/`rest` (a fraction of the parent) — IRIS_TODO.md's "density-
@@ -529,171 +275,28 @@ resolution-independent, a fraction of the parent). `Span::gap` and
on them the same as any other size; a bare number is still `abs`,
physical pixels, unchanged.
## Masks with a shape (decided 2026-09-07, built 2026-09-08)
## Masks
Iris, on the code block's scrolling: "the code block scrolling currently
masks in an inner rectangle. Ideally masks should have a shape
associated with them, rounded rectangle being one of them, and/or
another widget you can select, so that the mask becomes the parent
container with rounded edges. Make sure alpha works properly with it,
eg. on the corners where alpha should be decreased / multiplied."
A `Mask` references a rectangle primitive and its parent mask. Nested masks
multiply coverage. Plain `.masked()` creates an undrawn rectangle at the
widget's region; `.masked_by(shape)` draws the shape behind the content and
clips to its first primitive. Keeping the shape in one primitive prevents a
rounded background and its clip from drifting apart.
**What exists.** `Mask` in `shader.wgsl`/`data.rs` is two `UiSpan`s and
a `move_idx`; `fs_main` resolves it and does `color *= 0.0` outside the
rectangle -- a hard cut on a pixel boundary. `Masked` (`widget/mask.rs`)
sets the painter's mask to its own region. Separately, `draw_rounded_rect`
already produces an anti-aliased rounded edge from
`distance_from_rect(pos, center, corner, radius)` with a half-pixel
`smoothstep`, and the border variant multiplies a second coverage in.
Masks are rect-only. Glyph masks would require a CPU-readable alpha plane for
hit-test agreement, and standalone image masks require a bind-group switch the
fragment stage cannot make. Rendering and hit-testing both traverse the full
mask chain and use the same rounded-rectangle coverage; `iris/tests/mask_sdf.rs`
checks the WGSL implementation against the CPU SDF.
**Design** (revised the same day on Iris's two corrections: hit-testing
applies the shape too, and a mask should reference a primitive rather
than carry a copy of its shape).
## Offered boxes
1. **A mask is a reference to a primitive already drawn, plus how to
use it.** `Mask { kind, idx, flags, parent }`: the primitive's
binding (`RECT`, `TEXTURE`, `GLYPH`) and slot, flags (today one:
*alpha only* -- take the primitive's coverage and ignore its colour,
which is the default and the only mode until a need for another
appears), and the enclosing mask's slot for nesting. The fragment
stage evaluates the referenced primitive *at the masked pixel* --
for a `Rect`, the same `draw_rounded_rect` coverage from the same
SDF; for a texture or glyph, the sampled alpha -- and does
`color.a *= coverage`. Nothing about the shape is copied: a rounded
container's corner and its children's clipped corner are the same
primitive's arithmetic, and a texture mask (an alpha image as the
clip) works with no new shader path.
What this needs from the data layout: evaluating a primitive at an
arbitrary pixel means its placement (its spans and `move_idx`, today
vertex attributes) has to be readable from a storage buffer in the
fragment stage. If it is not already there, put it there once, for
every primitive, rather than keeping a second copy for masks -- the
vertex stage can read the same buffer. Textures: the shader binds one
image at a time (see `masks_layout`'s comment on why an image's own
bind group must not name the masks buffer), so a texture mask is
limited to what the fragment can sample without a bind-group switch:
the atlas, and the primitive's own bound image when the masked
primitive is drawn in the same image's batch. Say so at the flag.
2. **Nested masks chain and multiply, like moves.** `parent` walks up
the chain, bounded like `resolve_move` (`MOVE_CHAIN_LIMIT`'s sibling;
debug-assert on overflow and print the chain); coverages multiply,
so a pixel inside two feathered corners is dimmed by both, which is
what a compositor does and what "alpha should be multiplied" asks.
3. **`.masked()` points the mask at the current widget's own
primitives.** `Masked` stops describing a region: it records which
primitive(s) the wrapping widget drew this frame (the painter knows
-- it just allocated the slots) and sets the mask to reference them.
So a rounded `Rect` widget's `.masked()` clips its children to
itself by pointing at the rect it already draws; an image widget's
`.masked()` clips to its alpha. No radius or shape argument exists to
fall out of sync. When a widget draws more than one primitive (a
bordered rect is one primitive; a card with a stripe is two), the
mask references the *first* and the doc says so; a widget that wants
another names it.
4. **Hit-testing applies the shape.** A press is inside a masked
subtree only if the mask's coverage at that point is above one half.
For a `Rect` that is the same rounded-rect SDF evaluated on the CPU
-- one function in the shared crate, with the WGSL a transliteration
of it and a test that compares the two at a grid of points
(`headless` renders to a buffer and reads back, or the Rust version
is checked against the values the shader produced once and recorded).
For a texture, the CPU needs the alpha: keep the alpha channel of an
image used as a mask readable on the CPU (it was uploaded from CPU
memory; keeping the alpha plane is a quarter of the image), and read
it at the point. A masked corner that cannot be tapped and a masked
corner that is not drawn are then the same corner.
`Pad` must work in every container: it offers an inset region to its child and
reports the child's used size plus padding. In a generous parent it behaves as
an inset; in a tight parent it grows the result outward.
**Rejected.** A stencil buffer (a second pass per mask level and no
anti-aliasing); the scissor rectangle (rectangles only, no alpha);
rendering a masked subtree to an offscreen texture and compositing
(a texture allocation per mask, every frame it scrolls, on the phone).
**Pass conditions.** A headless test draws a rounded container with a
masked child that overhangs all four sides and asserts the child's
coverage at a corner pixel equals the container's own coverage there
(same primitive evaluated, so exactly equal, not approximately); a
nested-mask test asserts the product at a pixel inside both feathers; a
texture-mask test clips a rect to an alpha image and asserts a
transparent texel masks fully; a hit-test asserts a press in a
container's clipped corner misses and one just inside the curve hits,
and that the CPU SDF and the shader agree at a grid of points; a
`run-headless.sh --phone` screenshot of a scrolled code block shows
rounded corners with no square pixels poking out at the top and bottom
of the scrolled content. Record the commands in RUST.md when it lands.
### What was built (2026-09-08), and where it differs
The commands and the screenshot are in docs/RUST.md's queue entry. Four
places the code is narrower than the design above, each deliberate:
- **No `kind` and no `flags` on `Mask`.** It is `{ primitive, parent }`.
The referenced instance already carries its own `binding`, so a copy
of it in the mask is a second thing to keep in step; *alpha only* is
the only mode there is, so there is nothing to select. Both are a
field away if a second mode appears.
- **A mask's shape must be a rect.** `Painter::set_mask_to` asserts it,
by name, rather than leaving the shader to read a `rects` entry that
is not there. A glyph would need a CPU-side alpha plane before the
hit test could agree with the shader, and a standalone image needs a
bind-group switch the fragment stage cannot make (`masks_layout`'s own
comment on why an image's bind group must not name the masks buffer).
So **the texture-mask pass condition is not met and no texture mask
exists** — the point of the reference design is that adding one is a
binding check and a sampled alpha, with no new shader path, and the
shader's `mask_coverage` already has the branch where it would go.
- **The shape is a primitive of its own, not always a drawn one.** A
plain `.masked()` writes an undrawn `RectPrimitive` at its region
(`Drawn::No`, `NOT_DRAWN`) and points the mask at that, so "clip to my
box" and "clip to that widget's rounded background" are one mechanism
and square-cornered clipping did not become a special case.
`.masked_by(shape)` draws `shape` behind the content — in its own
layer, the way `Stack` puts a background under its content — and
clips to the first primitive it drew.
- **The CPU/shader agreement is a GPU test**, `iris/tests/mask_sdf.rs`,
the only test in the workspace that needs an adapter. It lifts
`distance_from_rect` and `rounded_rect_coverage` out of
`iris_core::SHAPE_SHADER` *by name* and runs them in a compute pass,
so the thing under test is the shader itself rather than a copy of it
that would be edited alongside.
## What a widget's *offered* box may and may not be (2026-09-08)
Two rules that were each true in one place and missing from a sibling,
found together by Iris's 2026-09-08 phone report.
**Padding works in whatever container it is placed in, and is an inset or
an outset depending on how tight that container's region is.** Iris's
own words, 2026-09-08: "padding should work no matter what container a
widget is placed in, and acts as both inset and outset depending on how
tight the parent region is." `Pad` offers its child the region it was
handed, inset on each side, and reports `used + padding` — so given a
generous box it insets the child inside it, and given a box already the
size of the content it reports a larger size and the parent grows. What
this rules out is any container that offers a padded child a box and then
ignores what it reported, and any caller that reshapes its tree to avoid
a `Pad` (which `transcript-ui/src/tool.rs` did until 2026-09-08, at the
cost of a tool group's 4dp inset).
**A widget offered a box it does not fit is drawn again at the box its
own reported size implies, in the same frame.** Not next frame. The
temptation to defer is real — `LazySpan::place` offers a row its *cached*
height precisely so that an unchanged row hits `draw_inner`'s cheap
skip-or-move path, and `Scroll` sizes its child region from last frame's
content length for the same reason. But a `Rect` fills whatever region it
is given (`Size::REST`, and `rect.rs`'s `is_size_independent` doc says
why it must), and `.background(rect(..))` is the ordinary way to style
anything — so a one-frame-stale box is a background drawn at the wrong
size while the text inside it is already right. On screen that is a tool
card that looks closed while its text is there and open while it is not.
A move alone cannot fix a changed size; `Painter::place` redraws in that
case.
The cost is bounded and worth stating, because it is what makes the rule
safe to apply everywhere: the settling draw happens only on the frame a
widget's own size actually changes, which is a frame that was already
redrawing it. `Sized` also requires its final region before retaining its
children: its own reported size may be known exactly while a descendant was
drawn in the provisional box, so moving only the wrapper is insufficient. A
widget whose reported size is a function of the box it was *offered* would
disagree every frame and redraw every frame — which is why `LazySpan` requires
content-sized rows, and has since long before this.
When a widget does not fit its offered box, it is redrawn at the box implied by
its reported size in the same frame. Deferring would leave ordinary
`.background(rect(..))` surfaces one frame behind their content. The settling
draw occurs only when the widget's own size changes. Widgets whose size varies
with every offered box are therefore unsuitable as `LazySpan` rows.
docs/RUST.md
+47 -334
View File
@@ -1,50 +1,18 @@
# Moving the app to Rust
Working document for the port Iris asked for on 2026-09-04: the phone app
in pure Rust, one UI framework shared with a desktop app, at full feature
parity and giving up nothing native -- performance especially. Her
constraints: no Dioxus and nothing that draws through a WebView; **no UI
DSL** (which ruled out Makepad and Slint); the result stays lightweight;
platform-specific pieces are fine to maintain; reimplementing a framework
piece from scratch where it does not fit is fine; effort and elapsed time
do not matter, long-term robustness does.
**The framework question is closed.** Iris chose her own library,
[iris](https://github.com/cat16/iris), over Masonry on 2026-09-05.
The bake-off that got there, and the twelve experiments
that proved it on a device, are summarised in "What the experiments
settled" below rather than kept at length. What is left in this file is
the plan for the rest of the app and the findings that outlive the tasks
that produced them.
Decisions get a date and a reason here, the way `PLAN.md` does.
Plan for a native Rust phone app with full feature parity and a shared desktop
UI. It uses [iris](https://github.com/cat16/iris); platform-specific entry
points are acceptable, but shared screens, widgets, and styling are not
duplicated. The result must stay lightweight and preserve native behavior and
performance.
## Keep this file current as you work
**This file is the handoff, and it is meant to let a session be cleared.**
Write each result into it *as you get it*, not at the end: the box ticked
or the reason it could not be, the measurement with its number, the
decision with its date and what it rejected, and anything that cost time to
find out. Then a session that has filled its context can be cleared and the
next one can pick up from this file alone, which is much cheaper than
carrying a long conversation or re-deriving what was already measured.
Keep open work, current design, measured constraints, and dead ends that would
otherwise be repeated. Delete completed plans and migration narratives. Name
the command and measured value when evidence matters.
Two things that follow. Write for somebody who was not here -- name the
command, the file and the number rather than "the fix" or "the earlier
run". And write the failures and the dead ends too: "Venus is blocked by
the emulator, not by Mesa" and "the present mode was not the cause" are
worth as much as the successes, because they are what stops the next
session spending an afternoon on them again.
**And delete a plan once it has been carried out** (Iris, 2026-09-08:
*"remove everything that's already done and decided... many with checkboxes
already ticked off that just fill up context"*). A ticked box has done its
job; a finished experiment is worth one line saying what it settled, not
the log of settling it. Currency means this file says where things *are*,
not how they got here. What survives a prune is what cannot be cheaply
re-derived: measurements, dead ends, invariants and their reasons.
## Where things stand (2026-09-09)
## Current status
- **The framework is decided and built on.** iris draws the transcript
screen on the desktop, on this checkout's emulator and on Iris's phone.
@@ -52,14 +20,12 @@ re-derived: measurements, dead ends, invariants and their reasons.
phone and the reports are under `docs/bench/`.
- **P1 (session screen parity) is the current work**, and is where the
next session should start. Its box below has the state.
- **The repository was reorganised on 2026-09-08**: the port is one crate,
`app-rust/`, and `iris/` is the UI framework alone. See "One app crate"
at the end -- it is the layout everything else here assumes.
- The port is one crate under `app-rust/`; `iris/` is only the UI framework.
- **Open across the rest of the docs**: `docs/IRIS_TODO.md` is iris's own
list (colour-space correctness is the live one), `docs/TODO.md` is the
Compose app's.
## Desktop and phone share the code (Iris, 2026-09-07)
## Desktop and phone share the code
Iris plans to develop a desktop app as well, and asked that most code be
sharable between desktop and phone. The tree already has that
@@ -332,76 +298,9 @@ rather than what it happens to look like:
7. **Measurable frames**: the debug render report, and a way to attribute
a frame's cost to a widget on the real phone.
## What the experiments settled
## Measurements and constraints
Twelve boxes, all closed between 2026-09-04 and 2026-09-05, and all
deleted on 2026-09-08 now that their conclusions live in the code. One
line each for what a later session must not re-derive; where a decision
needs its reasoning, the reasoning is at the thing itself.
**The framework track (E0-E5), against Masonry:**
- **E0 -- toolchain.** NDK r29 (`29.0.14206865`) under `~/Android/Sdk`,
cargo-ndk 4.x. Its API-level flag is `-P`; `-p` now means `--package`.
- **E1 -- android-view's Masonry demo ran here**, on the GPU, with an
accessibility tree and the phone's real keyboard -- but no autocorrect
and no suggestions. The `android-view` rev this was measured against is
pinned in `app-rust/Cargo.toml` with that history at the pin;
`accesskit_android`'s detach-abort is mitigated in
`iris/src/android/view.rs`'s `raise_if_enabled`, and advancing the
version is not the fix.
- **E2 -- a transcript in Masonry** found the framework-wide gap that
blocked the comparison. It lived in `~/src/android-view/e2-transcript`
and was never committed here.
- **E3/E5 -- the Kotlin shell and the packaging xtask.** Both hold:
`app/shellApp` plus the JNI bridge (now `app-rust`'s `shell` feature)
posts a real notification and receives a real share, and `cargo xtask
apk` packages an installable APK with `javac`/`d8`/`aapt2`/`zipalign`/
`apksigner` and one disclosed Gradle call, documented at
`scripts/xtask/src/apk.rs`'s module doc.
- **E4 -- the same screen on the desktop**, which is now
`app-rust`'s `src/desktop` and the `ai-app-desktop` binary.
**The iris track (I0-I5):**
- **I0a -- iris is vendored at `iris/`**, history not carried, consumed by
path, from `iris/iris` on gitea at `7b54aaf`. It goes back to its own
repository once it has proved itself.
- **I0b -- the nightly pin is dated, not rolling** (`rust-toolchain.toml`,
one copy in `iris/` and one in `app-rust/`, because a pin applies per
directory). Dated because a rolling channel moved `impl const Trait` to
`const impl Trait` underneath the vendored tree and broke it unattended.
- **I1 -- parley, plus a glyph atlas.** Both Iris's call. Parley addresses
text by byte offset into one string, which is why the editing model
looks the way it does.
- **I2 -- iris runs on android-view**: the backend, the Gradle shell,
insets, the back gesture and the full `InputConnection` bridge, with
real Gboard suggestions.
- **I3 -- the virtualised list.** Since renamed `LazySpan`, and scrolling
has moved out of it into `ScrollController` -- `docs/SCROLL.md` is the
current design, not this box.
- **I4 -- accessibility names through AccessKit**, one flat tree with a
synthetic `Role::Window` root and every *named* widget a direct child.
Flat deliberately: nothing upstream of a named leaf needs a node. This
is what lets `ui-trace` tap by label.
- **I5 -- the transcript screen in iris**, with `FrameReport` for
frame timing. Its descendants are `app-rust/src/ui` and every
measurement rig in AGENTS.md.
**Two findings from that period that are still load-bearing, kept where
they belong rather than here:** iris's binding array does not survive real
Android hardware (the measurement and the fix are `docs/TEXTURES.md`'s
"Implemented, 2026-09-04"), and the emulator has no hardware Vulkan while
its GLES *is* the host's real GPU through virgl (moved to the
`this-machine-android` skill on 2026-09-08, with the `gpu-probe` output
that established it).
## Findings that outlive the task that produced them
Kept because the number or the constraint is what stops it being
re-derived; the tasks themselves are done and deleted.
### The Android release profile, and where the APK's size went (2026-09-07)
### The Android release profile and APK size
Iris asked why the iris bench APK was double the Compose one (20.6 MB vs
10.1 MB). It was almost all `libmain.so`, built with `panic = "abort"` and
@@ -423,7 +322,7 @@ vectorisation on a renderer. Everything else is
own rather than `release`, so the desktop build is not also optimised for
size.
### The fling stutter, and what a frame report could not say (2026-09-09)
### The fling stutter and what a frame report cannot say
Iris, from her phone: *"I'm noticing some stuttering when flinging in
particular. Harder to notice with my finger directly moving the scroll."*
@@ -515,7 +414,7 @@ The signature of the fixed loop, from that run: `build p50 0.4ms,
acquire p50 5.7ms, submit p50 1.7ms` -- four tenths of a millisecond of
work and the rest of the refresh period spent waiting its turn.
### Streaming is where the frame time is now (2026-09-09)
### Streaming frame time
Measured after the fling was fixed, and it is not where it looks.
`frame_profile.rs`'s stream run: folding an arriving event is 0.35ms and
@@ -530,7 +429,7 @@ every delta.
exact shape of the Compose lesson in AGENTS.md's "Things that have
bitten" -- and measuring it is what ruled it out.
### Incremental text: parley cannot, and it turns out not to matter (2026-09-09)
### Incremental text shaping
Iris asked to investigate incremental text rendering and hoped parley
supported it. **It does not, by design.** The crate's own docs: a
@@ -635,7 +534,7 @@ the reply into blocks was still right -- it is what makes the fixture
representative, and it halved the CPU half -- but it was never going to
move this, and it slightly increases the primitive count.
### The arenas upload deltas, and stopped being 11x too big (2026-09-09)
### Arena delta uploads
Done, and measured by `scripts/rigs/ui-profile`'s `arena_churn` -- see
AGENTS.md's entry for the rig and the numbers. The arithmetic above was
@@ -697,7 +596,7 @@ instance bytes per frame are **1,488**, from 176,496. This is framework
layout/rendering behaviour and the transcript screen contains no special
case for it.
### The Android release profile is `opt-level = 3`, not `"s"` (2026-09-09)
### The Android release profile uses `opt-level = 3`
The table above was measured in bytes only. `"s"` costs the loop
vectorisation and inlining a renderer runs on: over the same warm fling
@@ -708,7 +607,7 @@ refused for `"z"`, one level further up. Iris raised it herself
(*"I'd make sure it's in release mode"*); the build always was, and this
was the part of "release" that was not about speed.
### Platform fonts, not bundled ones (2026-09-07)
### Platform fonts
Iris: *"remove the font for now; just match what compose does."* The
Compose app takes body text from `FontFamily.Default` and code from
@@ -733,26 +632,9 @@ desktop cannot answer it. Before the next phone build, look at a bold run
and at `CLOSED_MARK`/`OPEN_MARK`/`UP_MARK` (U+25B8/BE/B4) on Iris's own
device; the emulator's font set is not evidence for hers.
### Hit-testing does not consult the mask chain (review R2, 2026-09-07)
## The port, in order
Masks are applied in the fragment shader
(`iris/core/src/render/shader.wgsl`); the CPU hit path
(`UiRenderState::resolved_region`) does not look at `masks` at all. So a
straddling row's clipped-away top is invisible and still tappable -- a tap
on "Run benchmark" can land on an invisible link in the row behind it.
Left deliberately: `docs/LAYOUT.md`'s mask redesign ("masks reference a
drawn primitive instead of copying a shape") is where hit-testing gets the
shape, and intersecting a chain in `resolved_region` now would be a second
mechanism to unpick.
## The port, in order (decided 2026-09-05)
The ordered plan for the rest of the app, decided here per Iris's standing
"decide technical questions yourself" instruction -- no serious
user-facing tradeoff is in play in the ordering itself.
**Where the screens live** was settled by the 2026-09-08 reorganisation
("One app crate", below): every screen is a module under
Every screen is a module under
`app-rust/src/ui`, which holds a `Screen` enum and a back stack -- the
direct equivalent of `AppRoot.kt`'s `when` and `MainScreen.kt`'s tab
`enum` -- with each Compose screen becoming one `iris::widget` subtree.
@@ -776,55 +658,15 @@ AVD, `ui-trace` by accessibility name, GrapheneOS phone quirks, the
running any pass condition below that touches an emulator or a real
device.
- [x] **P0 -- the phone benchmark gate. Passed.** Asked for 2026-09-05,
delivered and run on Iris's own phone; the reports are under
`docs/bench/`. Both halves are still in the tree and are how a
frame-time comparison is taken: the Compose `bench` build type
(`app/`, `BenchFixture.kt`/`BenchRun.kt`) and the Rust `bench`
feature (`app-rust`, `src/android/bench_client.rs`), opening the
same checked-in synthetic transcript
(`app/bench-fixture/assets/transcript.jsonl`, never a real one) with
no server, driving the same scroll loop and streaming phase, and
printing the same report fields. AGENTS.md's "The rigs" is the
current description; `app-rust/build-apk.sh` and `run-bench.sh` are
how it is run. The build source is this repository, but the Dev Updater
publication is the separate `~/repos/ai-app-bench` repository, whose
`iris-bench` component serves a committed APK with no build step. After
an arm64 release build, replace its
`iris/build/outputs/apk/release/iris-bench-arm64.apk` and push that
repository; adding the component here or pushing only `ai-app-2` is the
wrong delivery path.
**Bench-only cleanup still open**: the diagnostics report pane draws over
transcript rows. `REPORT_MAX_HEIGHT_DP` constrains its claimed height, but the
pane is neither masked nor scrollable despite its construction comment saying
it is both. This is an `app-rust` defect, not an iris framework item.
**Bench-only cleanup still open**: the diagnostics report pane draws
over transcript rows. Reproduced on the emulator on 2026-09-09 by
opening the named `Diagnostics` control. `REPORT_MAX_HEIGHT_DP`
constrains its claimed height, but the pane is neither masked nor
scrollable despite its construction comment saying it is both. This is
an `app-rust` bench-screen defect, not an iris framework item.
- [ ] **P1 — session screen parity.** **Started 2026-09-06, on Iris's
word**: "just continue with the plan for now; try to move towards
feature parity for the transcript screen so that the test can be
more fair." So P0's "must pass before P1 starts" is lifted — the
phone bench continues alongside, and parity is what makes its
comparison fair. **Sub-order, by what the bench fixture exercises
and Compose already draws** (tick and date each in place):
- [x] **P1a — markdown block rendering parity.** Done 2026-09-06.
Each top-level block is drawn in one of three frames
(`ui::markdown::BlockFrame`) — plain, verbatim, quote — with
fences and tables verbatim, headings scaled, and inline
styling per span. `app-rust/src/ui/markdown.rs` is the code
and its module doc the design.
- [x] **P1b — tool-call cards and grouping.** Done 2026-09-06.
`ToolRows.kt`/`ToolInput.kt` ported to
`app-rust/src/ui/tool.rs`: a run of calls is one collapsible
group, each card carries its state and summary, and the five
`ToolState` values each have their own appearance.
`tool.rs`'s module doc has what was chosen.
- [ ] **P1 — session screen parity.** Continue in this order:
- [ ] **Before the next parity slice — make iris's colour pipeline
correct.** Raised by Iris on 2026-09-09 as something to settle
sooner rather than later. Both backends currently prefer an
sRGB surface while the shader returns palette/image bytes as
correct.** Both backends currently prefer an sRGB surface while
the shader returns palette/image bytes as
linear values; `IRIS_TODO.md` has the measured mismatch and
pass condition. Do this before judging or centralising the
app's styling. It is correctness, not cosmetic polish.
@@ -1038,169 +880,40 @@ device.
once more against a real `ai-server` (not the sandbox) on a real
phone, side by side with the Compose build until it holds.
## For the next session
## One app crate
What to do when you pick this up, in order, so nothing here has to be
re-derived. **The work is done inline, not handed to subagents** — Iris
said so on 2026-09-08 ("I'm no longer using subagents for this. Please do
the work yourself"), so read the code, make the change, run the tests and
push, in the session that picked the task up.
1. Read this file, then `AGENTS.md` and `PLAN.md`. The rules there
(measure, do not read; fix the rig before accepting its limits; the
emulator is this checkout's own) all apply.
2. Work on the **`rustify`** branch of this clone (`ai-app-2`), not on
`main` and not in `ai-app`. Nothing on this branch is production until
Iris says so. Commit and push as you go.
3. The E- and I-steps (the framework decision) are done — iris won,
decided 2026-09-05. **Fix iris's colour-space pipeline first**, as Iris
requested on 2026-09-09; then take P1c, history paging and
jump-to-latest. The client-side paging pieces it needs are already
ported.
4. Every step ends with its measurement written into this file beside the
box, and the box ticked or the reason it could not be written in its
place. A step that is blocked says by what, not "later". Write it as you
go rather than at the end — see "Keep this file current as you work".
5. Run the existing rigs rather than inventing new ones: `ui-sandbox.sh`
for a server with fixtures, `transcript-bench.sh` for the scroll
baseline, `ui-trace` for anything positional, `emu up` for the
emulator, `iris/run-headless.sh EXAMPLE --shot PNG` for an iris
example on this displayless machine, and `scripts/rigs/gpu-probe` to ask a
device (this VM, the emulator, or a real phone over `adb push`) what
`wgpu` features and limits it actually has before building anything on
the assumption it does. The Vulkan section below says how to get a
Vulkan path in the emulator when a `wgpu` backend needs one.
6. **Bound anything heavy at the moment you start it.** An emulator or a
long build gets a deadline — `timeout`, or a watchdog scoped to the pid
you just started — rather than a plan to stop it later. Scope it to
that pid: a watchdog written as `sleep N; emu down` fired into a later
experiment here and made a working Vulkan build look like a crash. And
stop the emulator when the work needing it is done rather than between
tasks.
7. Decisions belong here with a date and what was rejected, the way
`PLAN.md` does it. Do not put design into commit messages alone.
## Things a Rust app changes elsewhere
- **`wg-app-link`'s `:link`** (pinned TLS, enrollment store, QR activity)
is Kotlin shared with Dev Updater. The certificate code already exists on
the Rust side of the submodule; the pinned-CA build step
(`generatePinnedCert`) becomes a `build.rs` reading the same path. The QR
scanner stays a Kotlin activity, since the camera is a platform feature.
- **Tooling** becomes `cargo` for everything but packaging: `cargo test`,
`clippy`, `fmt` cover the whole client, which is the motivation. Gradle
remains for the APK, signing (`~/.config/ai-app/release.jks`) and Dev
Updater's build modes; `build-apk.sh` would call `cargo ndk` first.
- **The bench scripts** (`ui-trace` by accessibility label) keep working
only if the framework exposes names through AccessKit on Android; that is
part of E2's pass condition, not a nicety.
- **Icons** stay Nerd Font glyphs from the committed subset; Parley/Fontique
loads a font file directly, so `build-icon-font.sh` is unchanged.
## One app crate, 2026-09-08 (the repository reorganised)
Iris, reading the tree: *"the organization of the rust rewrite is a mess
right now… there shouldn't be anything related to the app inside of iris.
Iris is supposed to be the UI framework alone."* Then, on the crate count:
*"I'm confused why the app only code needs more than one crate though."*
### What it was
Nine cargo workspaces, each with its own `Cargo.lock` and `target/`, and
the port's project code in five places — `iris/transcript-ui`,
`iris/transcript-fixture`, `iris/desktop-app`, `iris/android-app` (all
*inside* the framework), plus `client-core` and `android-shell` at the
root. Two root markdown files sat outside
`docs/`.
### What it is
**One crate, `ai-app`, in `app-rust/`.** Modules, not crates:
| was | is |
|----------------------------------------|-----------------------------------|
| `client-core` | `src/client` |
| `iris/transcript-ui` | `src/ui` |
| `iris/transcript-fixture` | `src/ui/fixture.rs` + `tests/`, `touch/` |
| `iris/desktop-app` | `src/desktop` + `src/bin_desktop.rs` |
| `iris/android-app` | `src/android` + `android-project/` |
| `android-shell` | `src/shell` |
The Rust client is one `ai-app` crate in `app-rust/`: platform-free code
is under `src/client` and `src/ui`, while `src/desktop`, `src/android`, and
`src/shell` contain the platform entry points. The fixture is behind its
own feature so its 1.9 MB `include_str!` does not enter ordinary phone
builds.
`iris/` now holds `core`, `macro`, the `iris` crate, `tabs-ui` and
`rig-input` — framework only, with no mention of a session, a transcript,
a setup or a server anywhere in it.
### Why one crate really is enough
`src/client` must not depend on iris. Features select the crate's face:
`screens` for UI builds, `shell` for the Compose shell bridge, and `bench`
for the fixture. The Android faces both produce `libai_app.so`.
Each split had a stated reason at the time; on inspection only two
survived, and one of those is not in `app-rust` at all.
`event-model` remains separate because both the server and client depend
on that wire contract. Iris remains a separate UI-framework workspace and
must contain no product concepts.
- **`client-core` separate from the UI** was "pure logic with no framework
dependency". That property is worth keeping and does not need a crate:
`iris` is behind the `screens` feature and `src/client/` may not reach
it. An invariant on a module instead of on a manifest, stated in
docs/CLIENT_CORE.md.
- **`transcript-fixture` separate from `transcript-ui`** was so the
headless harness and a desktop window opened the same bytes. Both are
now the same crate, so it is `src/ui/fixture.rs` behind a `fixture`
feature (1.9 MB of `include_str!` must not reach a phone build) with the
six harness suites in `tests/`.
- **Two Android `.so` names**, `libmain.so` for the iris app and
`libandroid_shell.so` for the Kotlin shell's JNI bridge, looked like the
one hard constraint: a package produces exactly one library artifact.
It dissolves because **P2 already plans to merge those two Android apps
into one**. So both faces come out of one package as `libai_app.so`,
picked apart by features (`--no-default-features --features shell` keeps
wgpu, parley and iris out of the Compose app's APK), which is the
direction of travel rather than a workaround. `xtask apk` and
`app/shellApp`'s `System.loadLibrary` were updated to match.
- **A desktop binary and an Android cdylib in one package** is not a
problem: `iris` itself already target-gates winit against android-view
in one manifest, and the same table does it here. `build-apk.sh` passes
`--lib` so `cargo ndk` never tries to build the desktop binary.
- **`event-model` stays a crate**, and is the one split that was never
optional: `server/` depends on it too, so a crate is what makes the
backend and the app agree by construction. Iris chose to leave it at the
repo root rather than inside `app-rust/`, since it is the contract
between the two rather than app code.
### Build constraints
So: three workspaces where there were nine — `event-model`, `server`,
`app-rust` — plus `iris` and `xtask`.
### Things that moved with it, worth knowing
- **The toolchain pin is per directory.** `app-rust/rust-toolchain.toml` is
a copy of `iris/`'s, because `client-core` used to build on stable and
now shares iris's dated nightly. Two consequences appeared immediately:
two `needless_range_loop` warnings in the markdown highlighter (fixed),
and four `AtomicBool::fetch_update` deprecations from inside `jni`
0.22's `native_method!` macro. The last are not ours to migrate — the
fix is a `jni` release — so `src/lib.rs` carries an `#[allow(deprecated)]`
scoped to `mod shell` with that reason written at it.
- **The rolling nightly setting is per directory.** The toolchain files in
`app-rust`, `iris`, and `scripts/rigs/ui-profile` must stay synchronized.
- **The Android release profile is `android-release`, not `release`.** The
aggressive settings `iris/android-app` had (`panic = "abort"`,
`opt-level = "s"`, fat LTO) would otherwise apply to the desktop build
too, which is a testing surface. `build-apk.sh` passes
`--profile android-release` / `--profile android-dev`.
- **`iris/run-headless.sh` grew `--dir DIR`**, defaulting to `iris/`. The
rig belongs to the framework; the examples it usually runs no longer do.
`replay-touch` is still built from `iris/`.
- **The log target changed** from `client_core` to `ai_app`
(`src/client/log_ring.rs`'s `is_own_target`).
- **`iris/run-headless.sh --dir DIR`** selects the workspace containing the
example; it defaults to `iris/`.
- **Not renamed, deliberately:** the Android application id and Java
package are still `dev.iris.android.demo` and the label is still "iris
android-view demo", both now misleading. Changing them changes the app's
identity on Iris's phone (a side-by-side install rather than an upgrade)
and the `DevLogProvider` authority Dev Updater reads, so it is hers to
decide rather than a tidy-up to make quietly.
### Verified
`./scripts/run-tests.sh` (event-model, server, app-rust) and `cd iris && cargo
test` green; `cargo clippy --all-targets` and `cargo fmt` clean in every
workspace. `cargo ndk -t x86_64` links `libai_app.so`; `./build-apk.sh
debug --abi x86_64` produces an installable APK; installed and launched on
this checkout's emulator, drawing through `Gl … virgl` as expected. The
phone-sized headless screenshot (`run-headless.sh phone --phone --dir
../app-rust --shot …`) renders the transcript unchanged.
and the `DevLogProvider` authority Dev Updater reads, so changing them
requires an explicit migration decision.
docs/SCROLL.md
+19 -81
View File
@@ -1,8 +1,6 @@
# Scrolling in iris
How anything in iris scrolls, as of 2026-09-09. This is the current
design, not a history — the git log has the account of
how it got here, and `docs/IRIS_TODO.md` has what is still open.
This is the current scrolling design; `docs/IRIS_TODO.md` holds open work.
Read this before touching `iris/src/widget/position/scrollable.rs`,
`scroll_area.rs`, `lazy_span.rs`, or anything that pans, flings or lays
@@ -29,13 +27,8 @@ Two widgets have one, and they differ only in how they spend a delta:
`.scrollable()` registers the same two senses against the controller it
already has.
Do not give a widget its own fling, its own scroll amount, or a
`RequestRedraw` handle. And do not add a scrolling method to the `Widget`
trait: the three that used to be there (`scrolls_itself`, `apply_scroll`,
`scroll_offset`) existed only so a `Scroll` could drive a `LazySpan` it
had no business wrapping, and they are gone (Iris, 2026-09-08: "I don't
like adding methods to widget, it seems like we can structure things
better instead").
Do not give a widget its own fling, scroll amount, or `RequestRedraw`
handle, and do not add scrolling methods to the general `Widget` trait.
## One convention for a delta
@@ -45,25 +38,9 @@ positive delta — the finger's direction — and that is `Scroll::scroll`'s
sign, `Scroll::fling`'s, and `Widget::apply_scroll`'s, from the gesture
all the way down to a row's anchor.
**It is a screen direction, not a logical one** (Iris, 2026-09-08:
"positive should always scroll up / left, and negative down / right ...
that way it always works as the user would expect"). The earlier wording
— "positive brings *earlier* content into view" — is true only of a span
laid out forwards: a `Dir::UP` list's earlier content is *below*, so the
same delta panned it the opposite way from every other scrollable in
iris. `LazySpan::flip_delta` is the conversion into the walk's own
direction-relative space, the exact counterpart of `flip_pos` for
positions, and its `scroll` (private) is the only thing that speaks that
space.
There used to be two public conventions under the same name, and every
call site had to know which widget it was talking to. If you add a third
scrolling thing, it takes this one. Two tests pin it, and neither is
redundant: `a_negative_delta_moves_toward_the_end` follows the sign
across the whole handoff, and `a_delta_moves_both_directions_the_same_
way_on_screen` checks the two `dir`s against **where rows were drawn** —
an assertion written in the walk's own space passes with the flip
deleted, because it checks the bookkeeping against itself.
It is a screen direction, not a logical content direction. A `Dir::UP`
span's earlier content is below, so `LazySpan::flip_delta` converts public
screen-space deltas into the walk's direction-relative space.
## The contract between a controller and its owner
@@ -98,8 +75,7 @@ on the clock its own velocity was measured on. Frames are presented on an
even cadence whatever clock they are computed on, so sampling the spline
at "whenever the callback got to run" moves the content unevenly between
frames that are shown evenly -- a shimmer that no frame-time percentile
can see, since no frame was late. Found 2026-09-09; docs/RUST.md's
"The fling stutter" has the rest.
can see, since no frame was late. `docs/RUST.md` records the measurements.
### Why a remainder was not enough
@@ -152,8 +128,6 @@ can say how far it may go, so nothing above it is in a position to.
### Why it is not a `Span` inside a `ScrollArea`
Measured 2026-09-08, and worth not re-deriving:
- A `Span` is skipped entirely in the steady state. When redrawn, it uses
exact hints first, draws unknown fixed children forward from the cursor,
and places retained drawings after flexible allocation. A child is
@@ -179,14 +153,9 @@ A transcript is `Dir::DOWN` (oldest message is item 0, at the top) with
`Pin::End` (the view sits at the bottom). Conflating the two would stand
it on its head.
**`Pin` says it either way round**, because there are two questions and
they are not the same one (Iris, 2026-09-08). `Start`/`End` are
content-relative — the first row or the newest one, wherever the layout
puts it — and `Neg`/`Pos` are axis-absolute: the top/left edge and the
bottom/right one, whichever end of the content is there. They coincide for
everything except a reversed `LazySpan`, where they are exact opposites,
which is the whole reason both exist. The one question a scrollable acts
on is `pinned_to_end`, and `dir` is what resolves a `Pin` into it.
`Start`/`End` are content-relative; `Neg`/`Pos` are axis-absolute. They
diverge for a reversed `LazySpan`. A scrollable acts on `pinned_to_end`,
with `dir` resolving the chosen `Pin`.
### Two coordinate spaces, two conversion points
@@ -217,8 +186,8 @@ framework:
old-children diff calls `remove_rec`, and the `ActiveData` — with its
`size` — is freed. The framework's copy is gone for precisely the rows
the walk has to pass through without drawing.
2. **A widget may one day render in two places at once** (Iris,
2026-09-08), so anything keyed by `WidgetId` alone that describes where
2. **A widget may render in two places at once**, so anything keyed by
`WidgetId` alone that describes where
or how big a widget was drawn will be wrong then. Where and how big
belongs to the owner that placed it.
@@ -242,8 +211,8 @@ inside the same frame**. `moved_by` counts that correction along with the
move that caused it, which is why `amt` stays equal to what is on screen
rather than drifting by every overshoot.
Layout is a pure function of the state, not of how many frames have been
drawn (Iris, 2026-09-08). A correction that lands next frame is a frame
Layout is a pure function of state, not of how many frames have been
drawn. A correction that lands next frame is a frame
drawn wrong, and there may be no next frame — a fling that stopped is not
asking for one.
@@ -305,40 +274,9 @@ cannot pan; there is a `debug_assert` in `drag` naming that.
rebased after 65,536 pixels to preserve `f32` precision, a rare O(visible)
move-slot pass rather than steady-state work.
## Tests that pin the behaviour
## Verification
In `lazy_span.rs`, all of these fail if the corresponding piece is undone:
- `a_negative_delta_moves_toward_the_end` — the sign, end to end.
- `a_delta_moves_both_directions_the_same_way_on_screen` — the sign is a
screen direction, checked against where rows were *drawn*.
- `amt_counts_only_what_the_child_could_take` — why the owner reports what
it did rather than the caller adding up what it asked for.
- `a_fling_stops_at_the_first_row`,
`scrolling_past_the_start_lands_on_it_in_the_same_frame` — the walls,
with no settling frame drawn on purpose.
- `a_dir_up_span_grows_upward_from_item_zero`,
`a_reversed_span_hit_tests_in_screen_space` — the position conversions
(`flip_pos`), as `a_delta_moves_both_directions_the_same_way_on_screen`
is the delta one (`flip_delta`).
- `a_registered_fling_is_driven_by_tick_animations_and_then_unregisters` —
a fling that nothing registers never moves, whatever its velocity.
In `app-rust/tests/` (layer 1, no window or GPU):
- `top_edge.rs`'s `scrolling_past_the_first_row_settles_on_it` /
`scrolling_past_the_last_row_settles_on_it` — both ends, no settling
frame.
- `phone_screen.rs`'s `a_recorded_flick_releases_with_a_velocity_and_
flings_the_list` — the velocity against
`benches/velocity_reference.py`'s number, and the fling's travel against
`benches/fling_spline_reference.py`'s.
- `phone_screen.rs`'s `a_long_press_and_drag_selects_text` — what caught
two `DragGesture`s fighting over the transcript.
- `catch_a_fling.rs`, `gesture_cancel.rs`, `fence_fling.rs` — press-catches
a coasting area, cancels, and a code fence panning sideways
independently of the transcript.
`docs/RUST.md`'s "Three test layers" says which layer answers what. Test
at the cheapest one that can answer the question; the emulator is for JNI,
the IME, insets and one verification run, not for iterating on layout.
The unit and headless integration tests exercise direction, both walls,
reversed hit-testing, fling registration, cancellation, nested horizontal
pans, and transcript selection. `docs/RUST.md` defines the three test layers;
use the cheapest layer that can observe the behavior under test.
docs/TEXTURES.md
+38 -228
View File
@@ -1,240 +1,50 @@
# How iris renders an unbounded number of images
**Built 2026-09-04**, in `iris/core` and `iris/src/default/render.rs`.
This file is the design and the measurements behind it; the deliberation
that produced it -- the prior-art survey, the proposal and its review --
was deleted on 2026-09-08, having been carried out. What is kept is why
the old approach could not stay (it is the reason the current one looks
as it does), the numbers, and what actually landed.
Iris cannot require Vulkan descriptor indexing. The Android Vulkan Profile
2025, covering 80.1% of active Vulkan-capable Android devices as of October
2025, does not require `VK_EXT_descriptor_indexing` or its bindless texture
features ([Android Vulkan profiles](https://developer.android.com/ndk/guides/graphics/android-vulkan-profile)).
Arm guarantees the extension only on Valhall and fifth-generation GPUs
([Arm Vulkan guidance](https://developer.arm.com/mobile-graphics-and-gaming/vulkan-api-best-practices-on-arm-gpus)).
Iris (the person) asked whether iris's (the library's) approach to "draw
however many images happen to be on screen" -- relevant here because a
transcript can hold an unbounded number of attached screenshots -- works
on mobile, her recollection being that it does not. It did not, and this
is what replaced it.
The emulator also rejects wgpu requests for `TEXTURE_BINDING_ARRAY`,
`SAMPLED_TEXTURE_AND_STORAGE_BUFFER_ARRAY_NON_UNIFORM_INDEXING`, and
`PARTIALLY_BOUND_BINDING_ARRAY`. Those features therefore must not enter
Iris's required device feature set.
## The problem
## Current design
Every texture iris ever creates — every `Image` widget
(`iris/src/widget/image.rs`) and every glyph atlas page — gets a permanent
slot in one array via `Textures::add` (`iris/core/src/primitive/texture.rs:65`).
Both of iris's texture-sampling primitives (`TEXTURE` and `GLYPH`) read that
array by index: `core/src/render/shader.wgsl:56` declares
`var views: binding_array<texture_2d<f32>>`, sized by
`UiLimits::default()` (`core/src/render/mod.rs:347`) at **100,000 textures,
1,000 samplers**. Getting a device to accept that layout needs three wgpu
features — `TEXTURE_BINDING_ARRAY`,
`SAMPLED_TEXTURE_AND_STORAGE_BUFFER_ARRAY_NON_UNIFORM_INDEXING`,
`PARTIALLY_BOUND_BINDING_ARRAY` — which correspond to Vulkan's
`VK_EXT_descriptor_indexing` ("bindless"), promoted to Vulkan core at 1.2.
Glyph atlas pages are layers of one `texture_2d_array`. `GpuTextures` doubles
the array when it runs out of layers, copies the old layers on the GPU, and
rebuilds every bind group that referenced the old view. Page numbers are
assigned synchronously by `Textures::add_page` because glyph insertion needs
the layer before the renderer processes queued texture updates.
A transcript with an unbounded number of image attachments is exactly the
case that grows this array without bound: each attachment becomes its own
`Image` widget, which takes its own permanent array slot until dropped.
Standalone images each own a bind group and are not placed in the glyph
array. Each render layer keeps ordinary rect/glyph instances separately from
image instances. It draws the ordinary batch once, then binds and draws each
standalone image. This removes any fixed image count at the cost of one bind
and draw call per visible image, which is the appropriate tradeoff for phone
transcripts containing a modest number of screenshots.
## What was measured
The masks storage buffer appears in every image bind group. If that buffer or
the atlas array is reallocated, all affected bind groups must be rebuilt;
retaining a bind group across either reallocation would leave it pointing at
the old GPU resource.
**A new rig, `scripts/rigs/gpu-probe`**, asks a device for exactly iris's features
and limits with no window and no APK — a plain executable pushed with
`adb push` and run from `/data/local/tmp`. It has two parts:
`wgpu::Adapter::request_device` with iris's exact `Features`/`Limits`
(`src/main.rs`), and a raw Vulkan query bypassing wgpu entirely via `ash`
(`src/vk.rs`), to tell "the driver doesn't have it" apart from "wgpu didn't
detect it."
Texture updates accumulate their rebuild requirement with OR. A patch must
never clear a rebuild requested by an earlier push in the same batch.
- **On this VM's own GPU** (Vulkan via Venus onto an RX 7900 XT):
`IRIS DEVICE: ok`. Not the case that matters — nobody's phone is a
discrete desktop GPU — but it is why the design was never checked before
now: it always worked in the one place it was tried.
- **On the Android emulator's guest Vulkan**, both ICDs it ships
(`vk_swiftshader_icd.json` and, cold-booted, `lvp_icd.json`/lavapipe):
`request_device` **fails** —
`Unsupported features were requested: TEXTURE_BINDING_ARRAY |
SAMPLED_TEXTURE_AND_STORAGE_BUFFER_ARRAY_NON_UNIFORM_INDEXING |
PARTIALLY_BOUND_BINDING_ARRAY`. The raw `ash` query on lavapipe shows the
driver itself reporting all seven descriptor-indexing sub-features as
`true` at device API version 1.3 — so wgpu-hal's own feature detection is
being more conservative than the driver here, for a reason not chased
further (a likely instance-version negotiation gap, since the extension
only promoted to core at 1.2). That part is a wgpu-hal/emulator question,
not the finding that matters, and is **not** why this design is rejected.
Within a render layer, images are drawn after rects and glyphs. Both primitive
lists use `swap_remove`, so no code may infer draw adjacency from arena
adjacency after a free.
**The finding that matters is about real phones, sourced rather than
recalled:**
Standalone images currently use `NonFiltering` sampling. Thumbnail scaling
and filtering remain image-widget decisions, not texture-storage decisions.
- The **Android Vulkan Profile 2025** — Google and Khronos's current
baseline, covering **80.1% of active Vulkan-capable Android devices** as
of October 2025
([developer.android.com/ndk/guides/graphics/android-vulkan-profile](https://developer.android.com/ndk/guides/graphics/android-vulkan-profile)) —
does **not** require `VK_EXT_descriptor_indexing` or any descriptor-
indexing feature. It requires `shaderSampledImageArrayDynamicIndexing`
(indexing by a value uniform across the invocation — Vulkan 1.0 baseline,
unrelated to bindless) and stops there; true of the 2021 and 2022
profiles as well.
- Arm's own developer documentation states **"`VK_EXT_descriptor_indexing`
is supported on all Valhall and 5th Gen GPUs"**
([developer.arm.com/mobile-graphics-and-gaming/vulkan-api-best-practices-on-arm-gpus](https://developer.arm.com/mobile-graphics-and-gaming/vulkan-api-best-practices-on-arm-gpus)) —
Mali generations from roughly 2019 (Mali-G77) onward, with no claim made
for Bifrost, Midgard or Utgard, which are still common in budget and
older Android phones that are still in daily use.
- A search engine's summarized claim of "1% support on Android" for this
extension was checked against its cited source (an Arm blog post from
2021) and **was not actually there** — that number does not appear in
any primary source found and should not be repeated. The baseline-
profile finding above is the one with an attributable source; use it
instead.
## Verification rig
So this is not a software-renderer artifact. A real, currently-shipping
share of the Android fleet lacks the feature iris's texture pipeline asks
for unconditionally, and neither the emulator's failure nor the current
official hardware baseline gives any reason to expect that to change soon.
## Implemented, 2026-09-04
The shape above, built as proposed with one structural addition the proposal
didn't need to spell out and one bug it predicted made moot rather than
literally fixed. Files: `core/src/primitive/texture.rs` (`Textures`,
`TextureHandle`), `core/src/render/texture.rs` (`GpuTextures`),
`core/src/render/primitive.rs` (`Primitives`, `GlyphPrimitive`),
`core/src/render/atlas.rs`, `core/src/ui/painter.rs`,
`core/src/render/mod.rs` (`UiRenderNode`, `UiLimits` removed),
`core/src/render/shader.wgsl`, `src/default/render.rs`, and
`scripts/rigs/gpu-probe/src/main.rs`.
**1. Atlas pages as array layers.** `GpuTextures` owns one
`texture_2d_array` (`array_texture`/`array_view`), grown by doubling
(`grow_array`): a new texture is created at twice the layer capacity, the
old layers are copied across with `copy_texture_to_texture` (GPU-side, no
readback), and every bind group that referenced the old view — the main
one and every live standalone image's — is rebuilt, since the view's
identity changed. `GlyphPrimitive` carries `layer: u32` instead of
`view_idx`/`sampler_idx`; the layer number is assigned synchronously in
`Textures::add_page` (a plain counter, `next_page_layer`), not by the
renderer, because `GlyphAtlas::insert` needs it in the same call, before
any GPU sync happens — the renderer only finds out later, when it
processes the queued `Push`.
**2. Standalone images, one bind group each.** `TextureKind` on
`TextureHandle`/`Textures` distinguishes `Image` (a plain bind-group index,
`slot`) from `Page { layer }`. `Primitives` gained a second per-layer list
— `images: Vec<PrimitiveInstance>`, tagged `IMAGE_BINDING` — separate from
`instances` (rects and glyphs), written by `Painter::write_image` rather
than through the generic `Primitive` trait, since an image has nowhere in
`PrimitiveData` to put a per-instance entry once the bind group already
picks the texture. `UiRenderNode::draw` draws a layer's `instance` buffer
once as before, then walks `image_instance` one entry at a time, binding
that texture's `BindGroup` (`GpuTextures::image_bind_group`) and issuing
`draw(0..4, k..k+1)` per image. Group 2's layout is exactly the proposed
`{atlas array, one image texture, sampler, masks}`; the main draw binds a
1x1 null view in the image slot.
**The one addition beyond the proposal**: the masks storage buffer lives
in every per-image bind group (group 2, binding 3), and `ArrBuf<Mask>`
recreates its buffer whenever the mask count changes size
(`render/util/mod.rs`'s `ArrBuf::update` now returns whether it resized).
A resize invalidates every bind group holding the old buffer, not just the
main one, so `GpuTextures::update` takes a `masks_resized: bool` and calls
`rebuild_image_bind_groups` when it's set, alongside the same rebuild the
array-growth path already needed. This wasn't a design question the
proposal had to answer (it treated bind-group construction as a given),
but it's exactly the shape of trap layer growth already had, so it uses
the same fix.
**3. No thumbnail atlas.** Not built, as proposed.
**4. Removed**: `TEXTURE_BINDING_ARRAY`, `PARTIALLY_BOUND_BINDING_ARRAY`,
`SAMPLED_TEXTURE_AND_STORAGE_BUFFER_ARRAY_NON_UNIFORM_INDEXING` from
`src/default/render.rs`'s `request_device`, and `UiLimits` (the type
itself, not just its binding-array methods — once its two fields were
gone there was nothing left in it, and `UiRenderNode::new` no longer takes
a limits parameter). `binding_array` no longer appears anywhere in
`shader.wgsl`.
**5. Sampling** is still `NonFiltering`, unchanged, per the proposal's own
note that this is a separate decision for whenever the image widget itself
is touched.
**The `changed = false` bug is structurally gone, not patched.** The old
`GpuTextures::update` held one `changed: bool` that a `Patch` reset
unconditionally, which could erase an earlier `Push` in the same batch (a
new atlas page's `Push` immediately followed by `GlyphAtlas::insert`'s
`Patch`, both queued before the renderer ever runs). The new `update`
computes the rebuild signal by OR-ing each event's own answer
(`rebuild_main |= self.push(...)`), and `Patch`'s arm simply never
contributes to it — there is no shared mutable flag left for a `Patch` to
stomp on. Documented at the call site
(`core/src/render/texture.rs`, `GpuTextures::update`'s doc comment and the
`Patch` match arm's comment) rather than fixed as a one-line diff, since
the mechanism that could go wrong no longer exists.
**In-layer draw order is an explicit invariant now, not just a fact about
`swap_remove`.** `UiRenderNode::draw` draws every layer's images after its
rects and glyphs, and `Primitives::apply_free`'s doc comment states
directly that both of a layer's lists (`instances` and `images`) free with
`swap_remove` and that nothing may assume adjacency survives a free —
recorded there because `apply_free` is the one place a change to either
list's ordering would have to be reconciled.
**Verified:**
- `cargo fmt --all -- --check`, `cargo build --workspace --all-targets`,
`cargo clippy --all-targets`, `cargo test --workspace` all clean in
`iris/`, on the pinned `nightly-2026-09-03` toolchain. 14 tests pass
(unchanged from I1; nothing here is pure-logic enough to add a unit
test to — it's all GPU resource wiring).
- `iris/run-headless.sh minimal --shot /tmp/minimal.png` and
`iris/run-headless.sh tabs --shot /tmp/tabs.png`: both render correctly
on this VM's GPU (Venus) — `tabs`'s glyph-atlas text renders in every
panel, confirming `GlyphPrimitive.layer` addresses the array correctly.
- The standalone-image path specifically: a throwaway example (not
committed) with an `image(...)` widget as part of the root, run the same
way, rendered the image next to glyph-atlas text in one frame —
confirming a live `BindGroup` built by `GpuTextures::create_image` and
bound per-`draw()` call actually samples the right texture. `tabs`'s own
"image span" tab exercises the same widget but needs a click to reach,
which the headless compositor can't deliver (no seat devices, per I1's
own note on this file) — the throwaway example is what stood in for it.
- **Exercised, 2026-09-04: `grow_array` under real load, on `tabs`.**
Rather than building a purpose-made glyph flood, `PAGE`
(`core/src/render/atlas.rs`) was temporarily dropped from 1024 to 64 —
small enough that `tabs`'s ordinary mix of sizes and families (nothing
exotic: a handful of `Text` widgets at a few sizes, one at
`Family::Monospace`) already exceeds one page's worth of distinct
glyphs. A one-line `eprintln!` in `grow_array` confirmed two real grows
in a single run (`GROW_ARRAY: 1 -> 2` then `GROW_ARRAY: 2 -> 4`, i.e.
glyphs landed on at least a third layer), and
`iris/run-headless.sh tabs --shot` showed every tab's text rendering
correctly with no corruption or missing glyphs — confirming the
`copy_texture_to_texture` grow-and-relocate path and cross-layer
sampling (`GlyphPrimitive.layer` addressing a layer beyond the first)
both work. Command:
`sed -i 's/PAGE: u32 = 1024/PAGE: u32 = 64/' core/src/render/atlas.rs`,
rebuild, `./run-headless.sh tabs --shot /tmp/x.png`, then
`git checkout -- core/src/render/atlas.rs` to revert — this is a
throwaway diagnostic value, never a committed change, since a real
1024px page holding only a handful of glyphs at a time would be mostly
wasted space in normal use. Confirmed the revert left `tabs` and
`minimal` byte-identical to the pre-check screenshots afterward.
- **The decisive check**, `scripts/rigs/gpu-probe` rewritten to request iris's new
(empty) feature/limit set and run on this checkout's own emulator
(`ai-app-2`, via `emu`), booted with `EMU_GPU=software` so the guest gets
a real Vulkan device (SwiftShader) rather than the `-gpu host` default,
which disables Vulkan in this VM entirely (`-feature -Vulkan`, because
gfxstream can't pair Venus with the real GPU here — worth remembering,
since the *default* `emu up` gives a device with **no** Vulkan adapter
at all, which reads exactly like the old bindless failure if you don't
know to ask for `EMU_GPU=software`):
cd scripts/rigs/gpu-probe
ANDROID_NDK_HOME=$HOME/Android/Sdk/ndk/29.0.14206865 \
cargo ndk -t arm64-v8a -P 26 build --release
EMU_GPU=software emu up # from ~/repos/emulator-tools
adb push target/aarch64-linux-android/release/gpu-probe /data/local/tmp/
adb shell chmod 755 /data/local/tmp/gpu-probe
adb shell /data/local/tmp/gpu-probe
Output: `adapters: 1 — Vulkan SwiftShader Device (Subzero) (Cpu)`,
`features iris requires:` (none listed — the set is empty),
`max_buffer_size … ok`, and **`IRIS DEVICE: ok`**. This is the fix
measured working, on the exact rig that first measured it failing.
Emulator stopped afterward (`emu down`); nothing was left running.
`scripts/rigs/gpu-probe` requests Iris's exact feature and limit set without a
window. Run it on the target device when changing renderer requirements. A
successful desktop adapter is not evidence that the same feature is available
on Android hardware.
docs/TODO.md
+9 -36
View File
@@ -1,46 +1,19 @@
# TODO
Working list from Iris, 2026-09-03. Remove an entry when it lands; annotate
one in place when it turns out to need a decision.
Only open product work lives here. Remove an entry when it lands.
## App — transcript
- [ ] Messages received from other agents are inconsistent — sometimes they
appear, sometimes they don't. **Needs a rig.** Read the code rather than
measured: a live Claude session only learns of a peer message from the
`origin` object on a turn's `result`
(`session/claude/translate.rs`), which the CLI attaches to a turn the
message *started*. So a message that arrives mid-turn, or a second one
within one turn, has nowhere to be reported — while an imported session,
which syncs from the CLI's own file, picks up every one of them. That
would show exactly as "sometimes". Confirming it means driving a real
stream-json session and sending it messages in both states.
appear, sometimes they don't. **Needs a rig.** The live driver only sees
the `origin` attached to a turn result, so a mid-turn or second message
may have nowhere to appear; imports read every message from the CLI file.
Drive a real stream-json session and send messages in both states.
## Session settings
- [ ] Autocompact belongs in session settings; empty disables it, which is the
default. Iris chose "hand it to the driver" — only where a driver has
auto-compaction of its own. **That option was offered on a false premise
and is not buildable yet.** It named pi's `set_auto_compaction`, but pi
was never built as a driver here: `session/llama.rs` talks to
`llama-server`'s OpenAI-compatible endpoint directly, and its `compact()`
refuses outright. Claude Code's auto-compaction is the CLI's own and
nothing in the stream-json control protocol this app uses configures it.
So the setting would be stored, passed to a driver, refused by every one
of them, and the field would never appear on any session. What is needed
first is either a driver that can take it, or a different rule — the
server watching `contextTokens` and running `/compact` itself is the one
that would work today, for Claude sessions, and it is the option that was
not chosen.
## From Iris's phone log export, 2026-09-07 (Compose app)
- [ ] **Crash on 2026-09-03 11:40, `IllegalArgumentException: Reversed
range is not supported`** at `ToolInput.kt:200` (`highlighted`, inside
`ToolInputView` -> `RawBlock` -> `ToolCard`). An `AnnotatedString`
range was built with end before start while highlighting a tool
input. Found in the per-package system log she exported; the tool
input that triggered it is not in the log. Reproduce by fuzzing
`highlighted` with inputs whose token boundaries collapse, and guard
the range construction.
default. No current driver accepts that setting: llama refuses compact,
and Claude's stream-json protocol cannot configure the CLI's own
autocompaction. This needs either a capable driver or a new rule such as
watching `contextTokens` and issuing `/compact` from the server.