Files
ai-app/docs/IRIS_EXTRACTION_HANDOFF.md
T

480 lines
27 KiB
Markdown

# Iris extraction handoff
Operational handoff for pulling Iris out of ai-app into a standalone framework.
Not a decisions log; delete it when the extraction is done.
## Where things stand
Canonical `main` is **`ca2b4b2`** (#17, the headless rig). Sixteen slices are
in; #16's size work and #17's rig both merged on 2026-09-14.
**#18 `split/18-position-chain`** is open, worktree `/home/bob/repos/iris-pr18`,
head `86a7e8d`, eight commits. LAYOUT.md §2's O(1) subtree movement, the
`Remap` retirement, and then the three changes the owner chose on 2026-09-14:
the slot carries a box, slots are opt-in, and a widget's region is held in the
coordinates of the slot it draws in.
- **A slot holds a box, not a translation** (`1f9dc48`), given in the
coordinates of the slot it names, and `prelude.wgsl` composes the chain with
`within` instead of adding a delta. A translation is the special case where
the box has its parent's relative extent. Measured against the translate
slot on the same binary: +0.5-0.8% at depth 1-4, +9.6% at 8, +32.2% at 64.
Free where opt-in slots put the chain. The identity is `UiRegion::FULL`, not
zero -- a zeroed entry collapses its subtree to a point.
- **`Painter::place` is how a container asks for a slot** (`d989691`): it
draws a child it decides the box of and may decide again. `Span`, `Aligned`
and `Scroll` use it; everything else shares its nearest ancestor's slot,
which `tests/layout.rs` pins at one deep with four widgets in between.
Placing a child again is one entry to write, moved or resized alike.
- **Nothing is inverted any more.** `UiRegion::stretch`, `stretchable` and
`UiScalar::stretch` are gone, with the case they could not express: a
40-tall row now stretches on its other axis, which `stretchable` refused.
- **`redraws_under` is the CPU walk that remains**, and it is a question
rather than a marking: before keeping a drawing, ask whether anything under
it would have to be drawn again for the new length, and give up the reuse if
so. Marking instead does not terminate -- the mark escalates to the
descendant's size reader, which re-places the child, which marks it again.
A part of a box with no relative extent holds its contents as offsets, and
composing into it leaves none either, so the walk stops where a length did
not change: an 80-wide child of a widened row is never asked.
- **`Span`, `Pad`, `Stack`, `Offset`, `Aligned`, `SetSize` and `LayerOffset`
say `Scale`.** Each places in fractions and offsets of its own box and none
reads its pixel length. `Scroll` and `MaxSize` do read pixels and stay
`Redraw`. Measured as not the cause of the divergence below: taking every
one of these claims back out leaves the failing set identical.
- **`OnResize::Scale` keeps its name.** The owner rejected `Stretch` on
2026-09-14: stretch has an opposite and scale does not, and the answer is
per axis so the axis is already established where it is read.
**The open defect, and it is older than this branch.** `tests/generated.rs`
grows a seeded random tree and checks that laying it out again lands where
growing it cold does. It does not always. Every case seen is a **wrapping text
on a span's own axis**: the span measures the child in the whole box, the
child shapes to that width and reports the width it used, the span places it
in exactly that width, which is a length change, so it shapes again and its
longest line is shorter than the box it was just given. Where it settles
depends on how many passes it has had, and a warm tree has had a different
number from a cold one -- so layout is not a function of the state, which it
has to be. A span whose axis is not the wrap axis is stable, which is every
real column of text, and why nothing had run into it.
seeds 1-30, three checks each db1751f (before) 86a7e8d (now)
diverging 7 of 90 30 of 90
The same defect either way: a placed child reaches the second shaping more
often now. It wants fixing where the two draws meet, LAYOUT.md §4, rather than
anywhere in the chain -- and **it should be fixed before #18 merges**, since
tripling how often a tree lands somewhere unrepeatable is not something to
land even on a defect that was already there.
`a_wrapping_child_of_a_row_settles_somewhere_else_each_time` is the ignored
reproduction; the six seeds the live tests use are ones that agree.
**Rigs added.** `iris::random` grows the tree (spans in every direction,
stacks, rects with varying opacity, text wrapping and overflowing, a declared
size over half of it, stopping at a depth), `examples/random.rs` draws one --
`IRIS_SEED` and `IRIS_DEPTH` pick it -- and `tests/generated.rs` compares warm
against cold under a size change, a resize, and both. It found the
non-terminating marking above immediately.
**Not verified yet:** the headless render shots and the live resize under
sway. The owner asked for those last, after the CPU side is right, and the CPU
side is not right until the divergence is.
### The plan after #18, agreed with the owner on 2026-09-14
Three changes, done together as one slice on top of #18, because each decides
the shape of the others.
- **Slots are opt-in, created by the container that re-places children.** A
draw call asks for one; a widget that did not asks nothing and composes into
its nearest ancestor's slot. #18 gives every widget a slot, which puts a
primitive's walk at full tree depth for no benefit, since almost every slot
is zero. `Span`, `Scroll` and `Aligned` re-place children after drawing
them and are the ones that ask. That keeps the chain 2-4 deep, which
`chain_cost` measured as free.
- **A stretch is a rewalk from the widget whose box changed, not a flat pass.**
#18's `stretch` takes the top widget's old and new window boxes and rewrites
every descendant against that one pair, which works only while a composite
fraction of the top box is recoverable from a window rel -- and a
fixed-length box destroys that for everything beneath it, whatever is
stored. Instead: keep the parent-relative region -- the argument
`widget_within`, `primitive_within` and `set_mask` already receive -- on
each widget, primitive and mask, and recompose it against the new box. For
each child, compute its new window box from its local; the same length on
both axes is one slot write where it has a slot and a recomposition where it
does not, and a changed length asks that child's `on_resize` and recurses or
redraws. Nothing is inverted, so `stretchable`, `UiScalar::stretch`'s
division and its debug assertion all go. It also fixes two things #18 gets
away with only because nothing but leaves says `Scale`: `try_reuse` asks
`on_resize` of the top widget alone, and `stretch` never rewrites
`Mask::region`.
- **Containers say `Scale`.** `Span` on both axes: it hands every child
`UiSpan::FULL` on the orthogonal axis, and its aligned-axis placement is
written in fractions and offsets of its own box, so the children's local
regions do not change when the box does. A child's *length* can, if one
without a hint reports differently after a redraw, and that reaches the
span through `size_deps` as it does today. The general rule: a widget is
`Scale` on an axis unless its draw reads the pixel length of its box on that
axis, which is text. The default stays `Redraw`.
**Open, and the owner's to decide: a slot that carries a box instead of a
translation.** Everything under a slot would store its region relative to
that box, and the shader would compose the chain with `within` instead of
adding deltas. A move and a stretch of a `Scale` subtree are then both one
slot write: no rewalk over primitives, no stored locals, and `Translate` comes
free, since a glyph at an abs offset from its row's start stays there when
the row grows. The CPU work on a resize becomes asking `on_resize` down the
widgets and redrawing the ones that say so -- one ask for a transcript row
against a few hundred glyph rewrites. It replaces #18's chain rather than
extending it, and it contradicts LAYOUT.md §2's line that slots carry
translation only -- a line an agent wrote into the design on 2026-09-04, not
something the owner asked for. Her own statement of the requirement was "if it
needs to be moved then that can be done after the fact efficiently, or resized
just done after as well".
**Measured, 2026-09-14.** `MoveOffset` widened from a `Vec2` to a `UiRegion`
(36 bytes) and `resolve_move` composing with `within` instead of adding
deltas, against the same binary's translate slots: depth 1 78.5 us against
78.0, depth 2 78.7 against 78.3, depth 4 79.6 against 79.0, then depth 8 89.8
against 81.9, depth 16 128.6 against 111.3, depth 64 331.2 against 250.6. So a
box slot is free at the 2-4 depth opt-in slots produce (+0.5-0.8%) and costs
10-30% past depth 8, where the chain is already expensive. All 42 tests pass
against the composing shader and `tabs` at 1920x1200 is byte-identical,
because composing through a translation box is the same map as adding its
delta. The experiment is `stash@{0}` in `/home/bob/repos/iris-pr18`.
The slot has to carry the whole box rather than a scale and an offset: a
pixel-space affine map scales everything under it, including a child that must
keep its pixel length, and the `rel`/`abs` pair is exactly what distinguishes
the two. What a box slot does not buy is a shorter CPU walk. Asking each
widget whether its drawing depends on the length that changed is the same walk
either way; what the box removes is the region rewrite and the upload at the
widgets that answer `Scale`, and the inversion at the ones that do not.
What is settled either way: recalculating rather than repositioning is in, as
the `Remap` retirement above. It costs the per-axis carry: a box that changed
length on one axis and not the other is redrawn rather than remapped. Six of
`tabs`'s fourteen relocations and five of `text`'s sixteen, all single-
primitive leaves, and one extra redraw per frame on `replace_cost` --
354,310,889 instructions against 354,272,387, noise. The rewalk restores that
carry, and generalises it to a subtree.
Measured, so the next attempt is compared rather than argued:
| rig | what it says |
| --- | --- |
| `tests/chain_cost.rs` | GPU pass time by chain depth at 200k instances: free to depth 8 (+5%), then ~3 us per level -- +42.6% at 16, +221% at 64. Each step is a storage load addressed by the previous one, so it is the chaining that costs, not the arithmetic at a level; a slot carrying a whole region should measure the same, but has not been. |
| `tests/replace_cost.rs` | Instructions per frame re-placing 200 rows: 1.98M writing each row's slot, 2.38M rewriting its regions, 7.13M redrawing it. A load for `perf`, not a check. Five primitives per row; the regime that decides whether the chain is worth it is a transcript row of a few hundred glyphs, so re-run it with 200 characters of text per row before concluding. |
| `tests/draw_cost.rs` | Pre-existing: what recording a frame costs on the CPU by layer count. |
Irrelevant at an example's couple of hundred primitives; a transcript's glyphs
are tens of thousands, which is the regime `chain_cost` measures.
Check for a review before starting anything, and read the newest
`submitted_at` rather than the first result:
```sh
TOKEN=$(cat ~/.config/gitea/token)
N=18
curl -s -H "Authorization: token $TOKEN" \
https://git.arirex.me/api/v1/repos/iris/iris/pulls/$N/reviews
curl -s -H "Authorization: token $TOKEN" \
https://git.arirex.me/api/v1/repos/iris/iris/pulls/$N/reviews/<id>/comments
curl -s -H "Authorization: token $TOKEN" \
https://git.arirex.me/api/v1/repos/iris/iris/issues/$N/comments
```
My replies are ordinary issue comments on the same PR and say what each change
was for. `/home/bob/repos/ai-app-2` is on `rustify`, worktree clean.
## How the work is sequenced
**Most fundamental first**, from the owner on 2026-09-13: *"please do more
fundamental changes first, such as library updates and core framework changes,
so that code only has to be written once"*, and *"should probably start adding
tests early on rather than later, so you don't have to make separate test
scripts and stuff. This might involve making the harness eventually, depends
on what needs tested."*
So slices are ordered by how much depends on them, not by what is nearest
ready, and a slice arrives with tests rather than with a script in `/tmp`.
**Agree a design before sending another variation of it.** The owner stopped
the fourth round of #11 with *"we should probably agree on the design here
rather than you keep submitting variations that I review"*. When a review comes
back about the shape of something rather than a defect in it, put the options
and a recommendation in front of her and implement what she picks.
**Nothing is submitted without a separate review pass** — the installed
`pre-submit-review` skill: build clean, review the code, review the comments on
their own once the code has settled, then verify the claim by running it. The
fixes a review produces are themselves unreviewed code, so the passes repeat
until a round finds nothing. It has earned its place repeatedly: four defects
on #11 that format, clippy, tests and five headless renders had all passed, and
on #12 a regression introduced by the review's own first draft. `audit.sh` in
the skill directory prints every comment line a branch adds against a base ref;
the owner's standing complaint is verbose agent comments, and the default
verdict is delete. **Machine-specific notes do not belong in the repository**
they live in `~/.claude/MACHINE.md` or a `this-machine-*` skill.
Other standing instructions from the owner:
- Pull Iris out even if the Rust application switchover is not accepted. No
app, session, transcript, setup or server concepts in Iris; the dependency
runs one way from `app/` to Iris.
- **Small, coherent PRs.** The original extraction PR was too large to review.
A slice may be redone rather than transplanted, and need not remove every old
feature. Non-conflicting pull requests may be open at once — disjoint path
sets, each branched from current `upstream/main` rather than stacked. The
owner reviews small ones as they arrive and only avoids having two *large*
ones in flight, which is one more reason to keep a slice small.
- **Order by dependency, largest reach first.** On 2026-09-13: *"do the large
reaching framework changes first so less has to be redone."* Pick the next
slice by how much sits on top of it, not by what is nearest ready, so each
piece of code is written once against the framework that will exist.
- Do not recreate an `ai` branch in canonical Iris; the fork is the boundary.
- **Never rewrite a pushed branch.** Follow review with additive commits, and
merge `upstream/main` in rather than rebasing when a branch falls behind.
- Respond to each review finding with a fix or a concise explanation. Do not
add a ceremonial comment when the changed code already answers it.
- **A test has to guard something that could break again.** The owner deleted
#14's test as pointless: the rename it guarded cannot regress. When a fix is
structural, the structure is the test.
## The next slice
**The plan above, as one slice on #18**: opt-in slots, the rewalk stretch
over stored locals, and `Span` saying `Scale` -- after the owner has answered
whether a slot carries a box, since that decides whether the locals are stored
beside window regions or *are* the regions. Then `set_child_offset` and
`LazySpan` to finish LAYOUT.md §2, then built-in alignment.
The archive is not a patch here: it writes `Widget::draw` against
`painter.set_size`, which #16 replaced with a returned `Size`, and it writes
lengths against `LayoutLen` and `density`, which canonical does not have.
Recreate on today's `Len` and let the dp slice follow.
Still in the target, roughly in dependency order:
- **The rest of the position chain** (LAYOUT.md §2), on top of the slice
above: `set_child_offset` for a container that moves its children as a
group, and `LazySpan`.
- **Built-in alignment, and probably size**, after the chain rather than
before it: the owner reordered the two on 2026-09-14. Reproduced in the
harness -- `.width(rel(0.5))` inside a `Dir::DOWN` span reports 200 of 400
and is handed the whole 400, and a `Pad` in between does not change that. **Do not "fix" that by reading the child's ortho `size_hint`**: a `Pad`
between the `SetSize` and the span has no hint of its own, so the declared
width silently goes back to filling. It works only when nothing is in the
way. Alignment has to belong to the widget rather than be discovered through
whatever happens to sit on top of it.
Two things beyond the bug argue for it. Built-in size removes `SetSize`,
and with it the mismatch that made `OnResize`'s old default unsafe -- a
wrapper reporting one size while handing its child the whole box. And
built-in alignment is what would let `OnResize::Translate` apply to centred
content, which otherwise has to say `Redraw` because only its own draw knows
where the middle was.
Size is the harder half: a declared size beside the one `draw` returns is two
sources of truth for one thing, so settle what each means before building it.
- **`OnResize::Translate`**, which still does nothing. `ActiveData::region` is both the box a widget was
given and the box its primitives occupy, and `mov` remaps out of it; keeping
a drawing at its old size while the box grows leaves the two disagreeing and
the next move stretches it. Found by rendering `tabs` against `main`, not by
a test. Whatever the chain does, the drawn box and the offered box have to
stop being one field.
- **`UiRenderState` behind `Rc<RefCell<..>>`**, queued by the owner on
2026-09-13 as fundamental, and especially so for text.
- **`Len`, `LayoutLen` and dp.** The archive splits the type so that `rest` is
unrepresentable where it is meaningless (a padding), and folds a density in
at resolve time. 21 files mention `Len`, so it is wide but shallow. After the
draw-size slice, not before: that one deletes the `desired_*` bodies this
would otherwise have to be threaded through.
- **The input restructure** — `src/default/sense.rs` becomes `src/rsc/sense.rs`
(308 lines to 2313), plus `core/src/event/controller.rs`, `desktop/input.rs`,
`android/input.rs` and `sense_tests.rs`: pointer capture, drag slop and axis,
platform cancellation, mask-aware hit testing, event timestamps. The
archive's own `consumes` is what #12 landed, so that part transplants;
`tests/pointer_routing.rs` is the acceptance criterion.
- Retained span, scrolling and layout placement.
- Retained paints, selection, overlays and shared UI runtime state.
- Generic desktop/Android framework hosts and reusable example/APK tooling.
- Application-owned fonts and application-named font families.
- Shared resource-handle bookkeeping and replaceable glyph-atlas buckets.
- Positioned text overflow and cluster-safe ellipsis.
Dependencies are current apart from `winit`, which stays on 0.30.12 until
0.31 leaves prerelease. `parley` 0.11.1 and `image` 0.25.10 are latest.
```sh
cd /home/bob/repos/iris && git fetch upstream
git diff --stat upstream/main..origin/archive/full-extraction
```
The archive is a reference, not a patch to apply. Recreate a change on top of
canonical `main`, leave app-specific behaviour out, and verify it
independently. Pick disjoint path sets when two PRs are open, and branch each
from the latest `upstream/main` rather than stacking — unless the slice fixes
code another open branch replaces, in which case say so and stack deliberately.
## How the renderer works now
Current invariants, not history. Worth reading before touching `core/render`.
- **A primitive registers itself by being drawn.** The type carries its own
WGSL, and `PrimitiveRegistry` keys ids by `TypeId`, so the kind comes from
the type and there are no `RECT`/`GLYPH`/`TEXTURE` constants. Nothing is
seeded, so an id depends on what a ui drew first and a ui pays only for the
pipelines it uses.
- **Each primitive records its own draws.** `Primitive::render` makes a
`PrimitiveRender` that states the layout its shader reads, uploads whatever
it owns, and records its draws. `GlyphRender` owns the atlas and binds it
once per list; `ImageRender` owns the images and binds one per instance; the
default owns nothing and draws every instance in one call. The renderer sets
the pipeline, the shared group, the list's data and its vertex buffer, and
knows nothing else. Dispatch per list was measured at 6 instructions, 0.1% of
a frame at 256 and at 1024 layers, against the ~5,400 wgpu spends recording
one list; `tests/draw_cost.rs` is that measurement.
- **The shared bind group is the window, the masks and the move chain**, given
to every draw. A mask texture would go here too. What a primitive samples is its own group,
and a primitive that samples nothing has no such group in its pipeline.
- **Every binding size is stated.** A `None` minimum puts the binding on
wgpu-core's late-sized list, which `is_ready` scans on every draw.
- **`shader/prelude.wgsl` plus one file per primitive**, because one module
cannot declare two types at the same binding. The prelude carries only what
every primitive uses — window, masks, vertex shader, `masked()` — and its
header is where binding numbers are written down; what a shader samples is
declared by that shader.
- **A texture handle is drawn like anything else.** `Painter::primitive` takes
`impl PrimitiveLike`: a primitive, or something that yields one and does
whatever else drawing it needs — a `&TextureHandle` retains its share on the
way through, which a `Pod` primitive cannot.
- **Order within a layer means nothing**, and the widgets do not rely on it:
`Stack` gives each child its own layer and `TextEdit` draws its view in a
child layer above the selection rectangles.
- **Images are one texture and one bind group each**, so each drawn image is a
draw call. The owner chose that on 2026-09-13 over packing images into
arrays like atlas pages; a bindless `binding_array` was ruled out by Android
support. Revisit only with her.
- **Layers are never freed** (`TODO` in `primitive/layer.rs`), so every layer a
session creates is walked every frame thereafter. Measured at ~2ns per empty
layer per frame, which is why it is the TODO's problem and not a bug of its
own.
## Repository topology
### ai-app checkout
- `/home/bob/repos/ai-app-2`, `origin = git@git.arirex.me:iris/ai-app.git`,
branch `rustify`.
- `iris/` is a submodule pinned at `32f6ad8`, the complete extracted snapshot,
and `.gitmodules` points at the **bot fork**, not canonical Iris.
- Do not change either casually: ai-app needs the complete snapshot while
canonical Iris is only partly caught up. Reconcile when canonical contains
what ai-app needs, or when the owner accepts a temporarily non-building pin.
### standalone Iris checkout
- `/home/bob/repos/iris`, `origin` = fork, `upstream` = canonical.
- Fork `main` and `origin/archive/full-extraction` both name `32f6ad8`, the
target snapshot. `history/full` names the source-history result `a615bcd`.
- **Do not reset, overwrite or force-push fork `main`**: it is both the target
reference and the commit ai-app pins.
- Start each new branch from current `upstream/main` in its own worktree:
```sh
cd /home/bob/repos/iris && git fetch upstream
git worktree add -b split/19-name /home/bob/repos/iris-pr19 upstream/main
```
`/home/bob/repos/iris-pr18` is the live one. Every other `iris-pr*` worktree
holds a merged branch; they are readable references, not places to build.
Every other `/home/bob/repos/iris-pr*` worktree holds a merged branch. They
are readable references; do not build new work on them.
## Verifying a slice
`iris` runs its own rig now (#17), so a rendering claim no longer has to be
driven from ai-app's submodule:
```sh
cd <iris-worktree>
./scripts/run-headless.sh tabs --mode 1920x1200@60Hz --shot /tmp/out.png
./scripts/run-headless.sh tabs --replay /tmp/taps.touch --shot /tmp/out.png
```
The reference shots this session compared against are `tabs`, `view`, `minimal`
and `text` at 1920x1200, plus `tabs` with a replay that switches to the image
tab and adds two images. A `.touch` line is `<ms> down|move|up <x> <y>` in the
output's own pixels; the tab strip is at y=24 and the five tabs at x = 192,
576, 960, 1344 and 1728, with the image tab's add button near (1836, 1116).
**A resize is its own case and the harness cannot see it.** Start an example,
change the output mode under it with `swaymsg output HEADLESS-1 mode WxH@60Hz`,
screenshot, and compare against a cold start at that size -- they must match
byte for byte. That is what caught both of #16's defects, and neither showed up
in 40 tests.
```sh
cd <iris-worktree>
cargo fmt --all --check
cargo clippy --workspace --all-targets -- -D warnings
cargo test --workspace
```
42 tests pass on #18's head. `--workspace` matters: `rig-input` is a crate of
its own.
Two drawing paths still have no shot of their own, and each needs a ui the
examples do not have, so both are throwaway examples written into the worktree
and deleted after:
1. An image alone in a layer, which is the case that failed GPU validation
when every other test happened to have a rectangle in the same layer.
2. Six lines of 400px text, which forces the atlas to four pages and proves
the array grew and its group was rebuilt.
`tabs` with the image replay covers rects, glyphs and images together, so that
one is an ordinary check now.
## Cautions
- Read `/home/bob/repos/ai-app-2/AGENTS.md` and the machine-wide rules first.
Anything about this machine — the GPU that comes and goes, measuring a small
performance difference, the emulator — is in `~/.claude/MACHINE.md` and the
`this-machine-*` skills, and belongs there rather than here.
- Keep Iris generic: session drivers, transcripts, setup and server concepts,
app icons and product fonts stay in ai-app. Android and desktop code is Iris
work only when it is a generic host or platform integration.
- Preserve the dirty-worktree rule. All worktrees were clean at handoff;
anything found later may be the owner's or another agent's.
- Do not delete the archived snapshot or the fork `main` ai-app pins.
- A complete target branch is not permission to recreate the giant PR.
- Another agent was freeing disk on this VM and removed `target/` from the
`iris-pr*` worktrees once. Sources and git state were untouched. Tell peers
before changing shared machine tooling, and expect a cold rebuild sometimes.
## Merged so far
| PR | On canonical `main` |
| --- | --- |
| #2 | Build on the current nightly (`4275314`) |
| #3 | Request a frame after resize (`936fbdd`) |
| #4 | Decouple `iris-core` from winit (`465e430`) |
| #5 | Use vsync by default (`ec2b5d4`) |
| #6 | Notify winit before presenting (`db9b0f2`) |
| #7 | Keep unsafe reference helpers internal (`0191f20`) |
| #8 | Initialize the window uniform from the surface (`6e271e8`) |
| #9 | Preserve primitive-count recursion (`b90c855`) |
| #10 | Text layout and rendering on Parley (`0f6a28b`) |
| #11 | Atlas as an array texture, and the primitive rendering overhaul (`b234497`) |
| #13 | Build on wgpu 30 (`00d2230`) |
| #14 | Rename the `Sized` widget to `SetSize` (`32b1038`) |
| #15 | Run a ui without a window, and test one (`c8ac669`) |
| #12 | Route pointer input per kind (`43ce8c7`) |
| #16 | Size a widget while drawing it, not in a pass of its own (`f942385`) |
| #17 | Bring the headless rig into the repository (`ca2b4b2`) |
URLs are `https://git.arirex.me/iris/iris/pulls/{number}`.