Record the derived-box rule and what the fuzzers found in the handoff

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

docs/IRIS_EXTRACTION_HANDOFF.md

@@ -9,15 +9,20 @@ Canonical `main` is **`ca2b4b2`** (#17, the headless rig). Sixteen slices are
 in.
 
 **#18 `split/18-position-chain`** is open and finished apart from one decision:
-worktree `/home/bob/repos/iris-pr18`, head `3f7cd82`, twenty-four commits,
+worktree `/home/bob/repos/iris-pr18`, head `f61e893`, thirty-four commits,
 workspace tests passing, fmt and clippy clean. It is LAYOUT.md §2's position
-chain, generalised to boxes. The last six are a performance pass that changes
-no layout: the two halves of the text fix (`e5f8b6b`, `2525637`), three
-retained-layout details that claimed something they did not do (`f1a47e9`),
-the many-widgets-dirty cases (`77bb75e`), what the settle order is holding up
-(`bf94380`), and carrying a widget's depth rather than walking for it
-(`3f7cd82`). Every one of them keeps the five reference renders and the resize
-render byte-identical, and the hundred-seed sweep passes. `/home/bob/repos/ai-app-2` is on `rustify`, worktree clean.
+chain, generalised to boxes. After the performance pass (`e5f8b6b` to
+`3f7cd82`) came the fuzzers and what they found: the shrinking fuzzer
+(`b7caab3`, `386a0d1`), `SetSize` measuring in its declared length
+(`c596bf1`), text re-breaking across the interval a break holds for
+(`9913194`), and a dirty widget measured where its parent asked rather than
+in a box its own answer decided (`02ff8c7`). `f61e893` restores `abs()` in
+the rect shader, which `7c50a3e`'s rename had taken with it so that every
+window failed shader validation while `cargo test` stayed green, and adds
+the device-free shader validation test that would have caught it. The five
+reference renders and the resize render are byte-identical across all of
+it, and the hundred-seed sweep passes. `/home/bob/repos/ai-app-2` is on
+`rustify`, worktree clean.
 
 **The one thing waiting on the owner.** `tabs` at 1920x1200 is no longer
 byte-identical to `upstream/main`: 1,283 pixels of 2.3M (0.06%), two
@@ -70,7 +75,7 @@ Invariants, not history. Everything in `core/src/ui` rests on them.
   tree depth, which is the difference between free and +42.6%.
 - **A widget's region is held in the coordinates of the slot it draws in**, so
   a placed widget draws against `UiRegion::FULL` and its box lives in its slot.
-  `ActiveData::region` is the box it was *offered*, in its parent's slot
+  `ActiveData::region` is the box it was *placed in*, in its parent's slot
   coordinates, and `ActiveData::parent_move` is the slot that is in;
   `window_region` composes the one through the other, which is the walk the
   shader does.
@@ -273,10 +278,14 @@ aligned size-changing chain.
 The branches are invariants rather than widget exceptions:
 
 - A dirty widget draws locally first only while its retained box has the same
-  pixel size. If its returned `Size` is unchanged, no size reader can observe
-  the repaint and no ancestor draws. If the size changed, the dependent path
-  lays out. A changed pixel box takes the conservative path first, which is the
-  condition the discarded `/tmp/escalate.patch` missed on resize.
+  pixel size **and that box is a constraint rather than its own answer**. If
+  its returned `Size` is unchanged, no size reader can observe the repaint and
+  no ancestor draws. If the size changed, the dependent path lays out. A
+  changed pixel box takes the conservative path first, which is the condition
+  the discarded `/tmp/escalate.patch` missed on resize. A box decided from the
+  widget's own size, on an axis that size reads, takes it too: measuring there
+  can only repeat the answer, so the reader that decided it draws instead --
+  see "a box that is its own answer" below.
 - Dirty widgets settle deepest-first. A changed size queues only its immediate
   reader; propagation stops as soon as a reader's own answer stays unchanged.
   Output resize is the distinct invariant: all pixel-dependent leaves and
@@ -379,6 +388,40 @@ sweep passes.
 
 ### Settling a dirty set, and why the order is not free to change
 
+**There are two ways to obtain a retained size and only one of them is
+guarded.** `Painter::known_len` -> `RenderState::retained_size` is the path
+that answers a child's length without drawing it, and it refuses on
+`size_is_invalid(id) || dirty_size_under(id)` -- it asks the deep question.
+`try_reuse` is the path that keeps a child's *drawing* for a new box, and it
+asks only whether that widget is itself in `needs_redraw`; the size comes back
+as a by-product of the reuse succeeding, and never got the guard the other path
+has. So a reader is not reaching past anything: it asks a clean child, and the
+child answers from a record that is stale only because something below it has
+not settled. **The settle order is standing in for a missing check on one of
+two implementations of the same concept**, which is why the order is
+load-bearing for the answer and why that was easy to miss.
+
+That also says why bolting `dirty_size_under` onto `try_reuse` costs about what
+the sort saves. On the `many` phase at 130 of 260 dirty the guarded path runs
+47 times a frame and `try_reuse` runs 569 -- so the O(subtree) walk moves from
+a cold path to one twelve times hotter. **The owner rejected building a maintained count for this on
+2026-09-14**, and the reasoning stands: the count is only clarity, it costs a
+pairing obligation at every site that marks or consumes a mark plus a cost in
+every shipped frame, and bottom-up is needed for its own sake regardless. A
+count that drifts low is a silent stale size, which is worse than what it
+replaces. Its one remaining argument would have been freedom to settle in any
+order, and the rows below measure that as costing more than it saves.
+
+What guards the invariant instead: `a_long_run_of_seeds_agrees` compares a warm
+incremental frame against a cold rebuild of the same tree over a hundred seeds,
+which is what caught the defect on seed 2 and is a stronger oracle than an
+assertion. A `debug_assert!(!self.dirty_size_under(..))` on the two paths in
+`try_reuse` that return a size would make the dependence on the order fail
+loudly there rather than silently -- those are the exact and moved paths, 133
+calls a frame on the `many` load rather than all 569 attempts -- and it
+compiles out of release, matching what `depth` already does against
+`walked_depth`.
+
 **Why picking any dirty widget does not work, found 2026-09-14.** `try_reuse`
 asks whether the widget in front of it is dirty and, if not, hands its parent
 the size it last reported. It does not ask whether a dirty widget sits under it
@@ -458,6 +501,144 @@ hits and only two lines differ between the builds. Nothing here rests on that
 difference, but it means those two rows are worth re-measuring before anyone
 builds on them.
 
+### An immediate-mode path, proposed by the owner
+
+Iris raised this on 2026-09-15, as something to have rather than something to
+do now: a way to render with nothing cached at all, redrawing everything cold.
+Per widget would be nicer, but a switch on the ui, or a second entry point
+alongside `update`, is the useful start. Two uses -- the performance floor a
+retained layout is measured against, and a second opinion on correctness that
+does not share any of the retained machinery.
+
+Worth recording what it would and would not have caught. Not the `SetSize`
+defect below: that one is a first frame laying out wrongly, with no retained
+state involved, so an immediate path would have reproduced it faithfully. It
+would catch anything where keeping a drawing is what goes wrong, which is what
+`generated.rs` uses a cold `Harness` for today -- and a cold `Harness` is a
+weaker instrument, because building a second tree is not the same code path as
+refusing to reuse the first.
+
+### Why a warm frame and a cold one disagree
+
+Three defects, found on 2026-09-15 by shrinking a fuzzer's counterexamples.
+Two are fixed. The third is the one the others were hiding.
+
+**A first frame was wrong, and nothing about retained state was involved.**
+`SetSize` drew its child in whatever box it had been offered and then reported
+its *declared* length, so the child answered about a box it was never going to
+have -- and the answer on the other axis was taken under that. A wrapping text
+under `SetSize(x: 76px)` was measured in the whole 640 available, said one
+line, and the parent sized itself to one line; the text was drawn again at 76
+and said two, too late. It now measures in the length it declares. Six widgets
+reproduce it in `tests/unsettled.rs`.
+
+That one matters beyond itself: **`generated.rs` compares a warm frame against
+a cold one, and cold was not a fixed point either.** Some of what this document
+previously called a retained-layout defect was the cold side being wrong.
+
+**Re-breaking a text at its own longest line is a knife edge.** A parent that
+sizes to a child offers back the length the child just reported, composed back
+through the box chain -- so it lands an ulp either side, and which side decides
+whether the longest line still fits. Three lines at 167.41 or four at 163.49,
+from the same text in the same box. A greedy break does not need recomputing
+there: breaking at one width gives lines that each fit and none of which could
+have taken another word, so at any width down to the longest of them the same
+break holds. `TextBuffer::shape` answers across that interval, with a
+sub-pixel tolerance for the edge.
+
+**A box that is its own answer.** A span measures its children in its own
+box, and its own box is what its parent gave it, from the size it reported,
+from those children. Four widgets:
+
+    Aligned(mid, -, Span[ Text(wrap), OneLine ])
+
+Swap the two children of a live tree and the span is still 663.376 wide, so the
+text is offered 357.44, which is what it already holds; the size is valid, the
+span reports 663.376 again, nothing moves. Grow the same tree in that order and
+the span is offered the window, the text is asked for 334.06 and answers
+318.45, and the span settles at 624.38. **Both are stable.** Which one you get
+depends on what the tree was before.
+
+No rule about when to keep a drawing can fix that, and several were tried
+(`Painter::settle` -- place a child into its own reported size without
+measuring it there -- passed the hand-written cases and failed two generated
+ones). The defect is in *where a dirty widget is measured*: a local redraw
+draws it in the box it was placed in, and when a reader decided that box from
+the widget's answer, the old answer is a fixed point of measuring there
+whatever the content now says. So `ActiveData::offered_px` keeps the pixel
+size of the box the parent *first* asked about the child in -- `known_len` or
+the first `place` of a draw -- beside `px`, the box it drew against. A dirty
+widget whose size reads an axis on which some reader up its chain gave what
+it read a box other than the one it asked in is not drawn locally: the chain
+is marked and the parent of the highest such placement draws, since above it
+every box is a constraint. The walk has to go up the whole reader chain rather
+than one edge, because a pass-through hands a derived box down unchanged: the
+second shrunk case is a `Scroll` placing a `SetSize` in a box the content
+decided, and the span under it placed once, in that box.
+
+That made `Scroll` show a second thing: it read its box's pixel length for
+the clamp through `px_len`, which records the reported size as depending on
+the box, and it does not -- a `Scroll`'s size is its content's. Every scroll
+tick was then a size question asked in a derived box, at 34x the
+instructions. `Painter::px_len_for_draw` is the read that records nothing,
+for a draw whose reported size does not follow from it. Instructions per
+frame on the depth-8 rig, 1000 frames, against the head before this:
+
+| | before | after |
+| --- | --- | --- |
+| `many`, 32 dirty | 0.66M | 0.74M |
+| `many`, 130 dirty | 27.7M | 26.5M |
+| `resize` | 15.8M | 15.0M |
+| `scroll`, `repaint`, `size` | 0.36M | 0.36M |
+
+The small load pays 12% for the escalations that are now required; the
+larger ones get cheaper because a `Scroll`'s retained size no longer goes
+invalid when its box changes. The reorder fuzzer passes 200 trees at depth 7
+in every case, and the sweep passes.
+
+`OrthoSize::{Fill, Children}` -- a span choosing whether to fill across its
+axis or report its longest child -- is written and parked for the same reason:
+`Children` reads the span's own box to rank candidates, which is this cycle
+with a second face. It is not the cause of anything, and the divergence it was
+blamed for reproduces without it.
+
+### Shrinking a counterexample
+
+`tests/generated.rs` is a fuzzer, and a seed is not a lead anybody can read:
+reconstructing one of its failures by hand failed three times. Two things fixed
+that, and both are worth keeping.
+
+`describe` prints what each ancestor of a mismatch was *configured* with rather
+than its type name, so a run says `Text < SetSize{x:34 px;} < Aligned{x:neg,
+y:pos}` and the tree can be written out again. `Widget: Any`, so it needs no
+plumbing.
+
+`tests/shrink.rs` grows trees from a description it can simplify -- drop a
+child, unwrap a wrapper, shorten a text, drop a declared length, reorder a
+span -- and takes the first simplification that still fails until none does. It
+reduced 402 widgets to 6, 905 to 6, and 486 to 4. It lives in the tests and the
+library knows nothing about it. Validate it after changing it: with the
+box-length check in `try_reuse` deliberately disabled it should reduce a
+96-widget tree to 2, and an early version of it silently found nothing because
+it never framed before resizing, so "warm" had no retained state at all.
+
+Run the ordinary tests first, then the fuzzers, and turn what they find into a
+fast test rather than leaving a seed as the record.
+
+**Depth 4 was hiding all of this.** `tests/generated.rs` ran at a constant
+`DEPTH = 4`; `IRIS_GENERATED_DEPTH` and `IRIS_GENERATED_SEEDS` now select the
+load, and `SHRINK_DEPTH`, `SHRINK_SEEDS` and `SHRINK_CASE` do the same for the
+shrinking fuzzer. The generator widens two to four ways per level, so depth
+buys overlap between dependency paths rather than ancestry. Nothing yet covers
+a deep narrow chain, which is a gap: Iris asked for high layer counts on
+2026-09-14 and a hundred-deep tree is still unreachable by turning this knob.
+
+The comment on `a_long_run_of_seeds_agrees` blames a wrapping text on a span's
+own axis and says such a text is stable on any other axis. Both halves are
+wrong -- a divergence was found on a span whose axis is Y -- and the "7 of
+these 90 / 30 with it" numbers beside it do not match a sweep that passes
+clean. Re-measure before trusting them.
+
 ### Dirtying many widgets at once
 
 **A frame that dirties many widgets at once was not being checked, and it is
@@ -469,8 +650,43 @@ to the rig, with `IRIS_DIRTY` widgets marked per frame. At 130 of 260 widgets,
 set is scanned once per widget settled, and a `HashSet` is walked by capacity
 rather than by length. Memoizing the depth walk within one scan does not pay
 (it trades parent lookups for memo lookups, 4% more instructions); the fix is
-to stop rescanning, which changes the order widgets settle in and is hers to
-agree first.
+to stop rescanning.
+
+**That fix does not have to change the order, and the cost is not the
+choosing.** Measured 2026-09-14 on the `many` phase at 130 of 260 dirty, all
+4.7% of it is one symbol: hashbrown's `RawIterRange::fold_impl`, the table walk
+under `max_by_key`. Since `3f7cd82` the key is a field read, so what is left is
+the cost of *finding* an element in a set walked by capacity, once per widget
+settled -- not the cost of deciding between them. A depth-bucketed worklist in
+`UiRenderState` keeps the order exactly and makes the pop O(1); the objection
+that `needs_redraw` lives on `Widgets` and is inserted from places with no view
+of the tree does not apply to it, because the buckets are a hint rather than a
+second truth. A bucket entry whose mark has since been consumed is skipped on
+pop for one hash lookup, so the obligation is only never to miss an insert,
+never to stay in step.
+
+**Going order-free is not the thing to buy.** `iter().next()` in place of the
+deepest-first pick, with no check added, takes widget draws from 502 to 670 per
+frame and instructions from 6.84M to 9.01M. The extra draws are not a parent
+drawn twice on its own: the same widgets appear at the top of the per-widget
+draw counts in both orders, at the same trial widths (a `Text` under
+`SetSize < Scroll < Span` is drawn at 14 distinct widths either way), with
+every count about 1.5x. The measure-by-drawing loop is not exploring more, it
+is being re-entered. What drives the re-entry is that boxes change more often:
+`try_reuse` refusals for a changed box go from 79 to 116 own-resize and 126 to
+187 descendant-resize, because a reader settled before its children hands out
+boxes from sizes that are about to move, and when they move every child in the
+subtree refuses reuse. The `dirty_size_under` check is what buys that back,
+which is why the order-free rows above come out 3.4% dearer at 130 dirty rather
+than 32%. So the order is paying for itself in draws, and the count maintained
+up the reader chain would be a way to afford dropping something worth keeping.
+
+One counter does not fit that account and was not chased down: queue pops fall
+from 87 to 80 and local redraws from 66 to 59, where re-marking a reader should
+raise both. The likely reading is that a re-marked reader is consumed inside a
+later pop's subtree draw rather than popped on its own, but it is inferred from
+`redraw`'s escalation path rather than traced. `EagerReaderRedraws` is 4 and 3,
+so the escalation itself is not where the draws come from.
 
 **What a frame is made of now**, `perf record` on the `many` phase at 130 of
 260 dirty, which is the heaviest thing the rig has: `Layers::write` 16%,