Merge remote-tracking branch 'origin/rustify' into worktree-agent-ace98b0bdaf33ffff
# Conflicts: # docs/IRIS.md # docs/RUST.md
This commit is contained in:
commit
4274b8b8d0
27 files changed
+1571
-149
No files matched your search
@@ -8,6 +8,43 @@ capability that moved. Small and trivial changes do not go here.
|
||||
An entry gives the date, what changed, why, and a short before/after where
|
||||
it helps judge the change without the session that made it. Newest first.
|
||||
|
||||
## 2026-09-07: a headless harness, replayed touch, and physical-pixel desktop layout
|
||||
|
||||
Layer 1 and 2 of docs/RUST.md's "Three test layers".
|
||||
|
||||
**New: `iris::harness`** -- a screen driven in-process with no window, no
|
||||
compositor and no GPU, on a clock the caller advances. `Harness::new(size,
|
||||
density)` gives you an `Rsc`, a `UiRenderState` and a state that
|
||||
implements `FocusHost`/`OpenUrl` by *recording* what the platform was
|
||||
asked for (`keyboard_shown`, `opened_urls`) rather than doing it;
|
||||
`frame(t_ms)`/`frames_until(..)` run frames, `touch(action, pos, t_ms)`
|
||||
feeds one pointer sample the way Android's `on_touch_event` does, and
|
||||
`replay(&TouchScript)` runs a whole recorded gesture. `TouchScript` parses
|
||||
a plain `t_ms action x y` file (`down`/`move`/`up`/`cancel`), so the
|
||||
batched 120Hz flick shape your phone actually delivers is a file that
|
||||
`cargo test` can replay -- something the emulator cannot produce at all.
|
||||
|
||||
**New: `List::fling_velocity() -> Option<f32>`**, what the release
|
||||
measured, readable where it landed rather than by re-timing the gesture.
|
||||
|
||||
**Changed: `List` starts a fling's curve at its first `tick_fling`, not
|
||||
at the release.** The only clock it reads is now the one its driver hands
|
||||
it; in a running app the difference is at most a frame.
|
||||
|
||||
**Changed: the desktop backend lays out in physical pixels with a
|
||||
density, exactly as Android does.** `iris::default::content_scale(window)`
|
||||
is the desktop's `content_scale` -- winit's scale factor, overridable with
|
||||
the `IRIS_SCALE` environment variable -- and it now feeds
|
||||
`UiRenderState::set_density`/`TextData::density` instead of dividing
|
||||
coordinates into a separate "logical" space. That division had
|
||||
`UiRenderState::resize` (physical) and the window uniform (logical)
|
||||
disagreeing on any display whose scale factor is not 1.0, and rasterised
|
||||
glyphs at one resolution to display them at another. `Input::event` lost
|
||||
its `scale_factor` parameter as a result, and `DefaultUiState::
|
||||
window_size()` now answers physical pixels. On a 1.0 display nothing
|
||||
changes. The override is what lets `run-headless.sh --phone` open a window
|
||||
at your phone's own 1080x2424 and 2.55.
|
||||
|
||||
## 2026-09-07: the fling curve was the identity function
|
||||
|
||||
You said the fling "seems to just be linear velocity with an abrupt stop."
|
||||
|
||||
@@ -931,3 +931,27 @@ do not duplicate it there.
|
||||
p50 dropping below Compose's on the phone. Do this after the four bench
|
||||
v2 defects (stale primitives, finger fling, decay curve, IME show) are
|
||||
closed, since they are what make the run unrepresentative today.
|
||||
|
||||
## From the phone, 2026-09-07 (build from ed04d4c)
|
||||
|
||||
- [ ] **"Some transcript blocks will be hidden until I uncover enough of
|
||||
them."** Two screenshots of the bench app's transcript at the top
|
||||
edge, both wrong in opposite directions: in one, rows scrolled above
|
||||
the viewport are still drawn and bleed *through* the header bar
|
||||
(`version = "0.1.0"` and a paragraph visible behind "Run benchmark /
|
||||
Copy report / Diagnostics"), so the list's mask is not clipping at
|
||||
the header's bottom edge; in the other, scrolled a little further,
|
||||
the row that straddles the top edge is not drawn at all -- black from
|
||||
the header down to "You", where the previous shot showed a paragraph
|
||||
-- so a row is culled as soon as its *top* leaves the viewport rather
|
||||
than when its *bottom* does. Suspects: the list's visible-range test
|
||||
(`iris/src/widget/list.rs`) comparing a row's top against the
|
||||
viewport top; the mask region for the transcript set from the
|
||||
window rather than from the area under the header; and the two-phase
|
||||
provisional/real draw noted in `03c6be8`'s header-duplicate
|
||||
investigation, which was never root-caused and has the same shape.
|
||||
Reproduce at layer 1 of the test rig: a headless screen with a row
|
||||
straddling the top edge must place that row, and a primitive above
|
||||
the header's bottom must be masked. Fix both with one rule: a row is
|
||||
drawn if any part of it intersects the viewport, and the viewport is
|
||||
the list's own region.
|
||||
@@ -947,3 +947,95 @@ When this lands, copy this entry into `IRIS.md` (newest first):
|
||||
> `SizeCtx` and `Cache` are gone with it — see `LAYOUT.md` for the full
|
||||
> design, the move-offset mechanism this shipped alongside, and the file
|
||||
> list.
|
||||
|
||||
## Masks with a shape (decided 2026-09-07, not yet built)
|
||||
|
||||
Iris, on the code block's scrolling: "the code block scrolling currently
|
||||
masks in an inner rectangle. Ideally masks should have a shape
|
||||
associated with them, rounded rectangle being one of them, and/or
|
||||
another widget you can select, so that the mask becomes the parent
|
||||
container with rounded edges. Make sure alpha works properly with it,
|
||||
eg. on the corners where alpha should be decreased / multiplied."
|
||||
|
||||
**What exists.** `Mask` in `shader.wgsl`/`data.rs` is two `UiSpan`s and
|
||||
a `move_idx`; `fs_main` resolves it and does `color *= 0.0` outside the
|
||||
rectangle -- a hard cut on a pixel boundary. `Masked` (`widget/mask.rs`)
|
||||
sets the painter's mask to its own region. Separately, `draw_rounded_rect`
|
||||
already produces an anti-aliased rounded edge from
|
||||
`distance_from_rect(pos, center, corner, radius)` with a half-pixel
|
||||
`smoothstep`, and the border variant multiplies a second coverage in.
|
||||
|
||||
**Design** (revised the same day on Iris's two corrections: hit-testing
|
||||
applies the shape too, and a mask should reference a primitive rather
|
||||
than carry a copy of its shape).
|
||||
|
||||
1. **A mask is a reference to a primitive already drawn, plus how to
|
||||
use it.** `Mask { kind, idx, flags, parent }`: the primitive's
|
||||
binding (`RECT`, `TEXTURE`, `GLYPH`) and slot, flags (today one:
|
||||
*alpha only* -- take the primitive's coverage and ignore its colour,
|
||||
which is the default and the only mode until a need for another
|
||||
appears), and the enclosing mask's slot for nesting. The fragment
|
||||
stage evaluates the referenced primitive *at the masked pixel* --
|
||||
for a `Rect`, the same `draw_rounded_rect` coverage from the same
|
||||
SDF; for a texture or glyph, the sampled alpha -- and does
|
||||
`color.a *= coverage`. Nothing about the shape is copied: a rounded
|
||||
container's corner and its children's clipped corner are the same
|
||||
primitive's arithmetic, and a texture mask (an alpha image as the
|
||||
clip) works with no new shader path.
|
||||
What this needs from the data layout: evaluating a primitive at an
|
||||
arbitrary pixel means its placement (its spans and `move_idx`, today
|
||||
vertex attributes) has to be readable from a storage buffer in the
|
||||
fragment stage. If it is not already there, put it there once, for
|
||||
every primitive, rather than keeping a second copy for masks -- the
|
||||
vertex stage can read the same buffer. Textures: the shader binds one
|
||||
image at a time (see `masks_layout`'s comment on why an image's own
|
||||
bind group must not name the masks buffer), so a texture mask is
|
||||
limited to what the fragment can sample without a bind-group switch:
|
||||
the atlas, and the primitive's own bound image when the masked
|
||||
primitive is drawn in the same image's batch. Say so at the flag.
|
||||
2. **Nested masks chain and multiply, like moves.** `parent` walks up
|
||||
the chain, bounded like `resolve_move` (`MOVE_CHAIN_LIMIT`'s sibling;
|
||||
debug-assert on overflow and print the chain); coverages multiply,
|
||||
so a pixel inside two feathered corners is dimmed by both, which is
|
||||
what a compositor does and what "alpha should be multiplied" asks.
|
||||
3. **`.masked()` points the mask at the current widget's own
|
||||
primitives.** `Masked` stops describing a region: it records which
|
||||
primitive(s) the wrapping widget drew this frame (the painter knows
|
||||
-- it just allocated the slots) and sets the mask to reference them.
|
||||
So a rounded `Rect` widget's `.masked()` clips its children to
|
||||
itself by pointing at the rect it already draws; an image widget's
|
||||
`.masked()` clips to its alpha. No radius or shape argument exists to
|
||||
fall out of sync. When a widget draws more than one primitive (a
|
||||
bordered rect is one primitive; a card with a stripe is two), the
|
||||
mask references the *first* and the doc says so; a widget that wants
|
||||
another names it.
|
||||
4. **Hit-testing applies the shape.** A press is inside a masked
|
||||
subtree only if the mask's coverage at that point is above one half.
|
||||
For a `Rect` that is the same rounded-rect SDF evaluated on the CPU
|
||||
-- one function in the shared crate, with the WGSL a transliteration
|
||||
of it and a test that compares the two at a grid of points
|
||||
(`headless` renders to a buffer and reads back, or the Rust version
|
||||
is checked against the values the shader produced once and recorded).
|
||||
For a texture, the CPU needs the alpha: keep the alpha channel of an
|
||||
image used as a mask readable on the CPU (it was uploaded from CPU
|
||||
memory; keeping the alpha plane is a quarter of the image), and read
|
||||
it at the point. A masked corner that cannot be tapped and a masked
|
||||
corner that is not drawn are then the same corner.
|
||||
|
||||
**Rejected.** A stencil buffer (a second pass per mask level and no
|
||||
anti-aliasing); the scissor rectangle (rectangles only, no alpha);
|
||||
rendering a masked subtree to an offscreen texture and compositing
|
||||
(a texture allocation per mask, every frame it scrolls, on the phone).
|
||||
|
||||
**Pass conditions.** A headless test draws a rounded container with a
|
||||
masked child that overhangs all four sides and asserts the child's
|
||||
coverage at a corner pixel equals the container's own coverage there
|
||||
(same primitive evaluated, so exactly equal, not approximately); a
|
||||
nested-mask test asserts the product at a pixel inside both feathers; a
|
||||
texture-mask test clips a rect to an alpha image and asserts a
|
||||
transparent texel masks fully; a hit-test asserts a press in a
|
||||
container's clipped corner misses and one just inside the curve hits,
|
||||
and that the CPU SDF and the shader agree at a grid of points; a
|
||||
`run-headless.sh --phone` screenshot of a scrolled code block shows
|
||||
rounded corners with no square pixels poking out at the top and bottom
|
||||
of the scrolled content. Record the commands in RUST.md when it lands.
|
||||
+103
-32
@@ -43,6 +43,29 @@ gated on her verdict**, so this pass works the P0 defects and the pure
|
||||
prerequisites in this order. Each item is ticked here by the agent that
|
||||
closes it.
|
||||
|
||||
### Queue, 2026-09-07 (orchestrator)
|
||||
|
||||
In order; two builders at a time. Each is ticked here by the agent that
|
||||
closes it.
|
||||
|
||||
- [x] Test rig, layers 1 and 2 ("Three test layers" below), landed 2026-09-07.
|
||||
- [ ] Fling parity with Compose, and the phone's keyboard push-up, with
|
||||
insets shown in the diagnostics overlay. Running, in a worktree.
|
||||
- [ ] Rows at the transcript's top edge: culled too early in one state,
|
||||
drawn through the header in the other (docs/IRIS_TODO.md, 2026-09-07).
|
||||
First after the rig lands, using its layer-1 harness.
|
||||
- [ ] Phone logging through Dev Updater (Iris has no logcat; see
|
||||
docs/TODO.md and the memory note): research how Dev Updater shows an
|
||||
app's runtime log, design the smallest route (the app keeps its own
|
||||
recent log; a debug button copies it; Dev Updater reads it), write
|
||||
the decision in docs/DECISIONS.md, build it.
|
||||
- [ ] Masks with a shape -- docs/LAYOUT.md "Masks with a shape (decided
|
||||
2026-09-07)". A mask references a primitive already drawn
|
||||
(rect SDF, texture or glyph alpha), chained and multiplied; `.masked()`
|
||||
points at the widget's own primitives; hit-testing applies the shape.
|
||||
- [ ] Compose app: the `Reversed range` crash in `ToolInput.highlighted`
|
||||
(docs/TODO.md). Main branch, not rustify.
|
||||
|
||||
### Desktop and phone share the code (Iris, 2026-09-07)
|
||||
|
||||
Iris plans to develop a desktop app as well, and asked that most code be
|
||||
@@ -70,7 +93,7 @@ The bench client (`android-app/src/bench_client.rs`, ~1000 lines) is
|
||||
the first thing to look at moving, since a desktop bench on the same
|
||||
fixture is layer 2 of the test rig below.
|
||||
|
||||
### Three test layers, cheapest first (decided 2026-09-07, rig not yet built)
|
||||
### Three test layers, cheapest first (decided 2026-09-07; layers 1 and 2 built the same day)
|
||||
|
||||
Iris's suggestion, adopted and layered: test at the cheapest layer that
|
||||
can answer the question, and go up only when it cannot. The emulator
|
||||
@@ -78,43 +101,91 @@ costs minutes a cycle; the desktop window seconds; the headless harness
|
||||
runs inside `cargo test`.
|
||||
|
||||
1. **Headless, in-process, no compositor and no GPU -- the default.**
|
||||
`iris/src/layout_tests.rs` already builds a tree over `UiRenderState`
|
||||
with no window, `sense.rs`'s gesture tests feed fabricated
|
||||
`CursorState`s with their own times, and `List::tick_fling` is
|
||||
driven by hand. Extend that into one harness that opens
|
||||
`transcript-ui`'s screen on `app/bench-fixture` (as `bench_client.rs`
|
||||
does on Android, no server), at the phone's logical size and
|
||||
`content_scale` from `docs/bench/iris-phone-v2-2026-09-06.md` (2.55,
|
||||
120Hz -- read, never typed from memory), ticks frames, and feeds a
|
||||
**replayed touch stream** from a trivial file of `(t_ms, action, x,
|
||||
y)` lines with `cursor.time` taken from the file. That is what the
|
||||
emulator cannot do at all: the batched 120Hz flick from the phone
|
||||
report becomes a deterministic test asserting on scroll offset and on
|
||||
the `iris drag release:` velocity. Anything about layout, scroll
|
||||
position, selection, focus or fold state is answered here, with
|
||||
assertions rather than eyes. Nothing renders; a widget's placed
|
||||
rectangle is the evidence.
|
||||
`iris::harness` (`iris/src/harness.rs`), plus the fixture crate it
|
||||
opens. `Harness::new(size, density)` builds an `Rsc`, a
|
||||
`UiRenderState` and a state whose `FocusHost`/`OpenUrl` *record* what
|
||||
the platform was asked for; `frame(t_ms)`/`frames_until(..)` run
|
||||
frames on a clock the test owns, and `replay(&TouchScript)` feeds a
|
||||
recorded gesture one sample at a time exactly as
|
||||
`IrisViewPeer::on_touch_event` replays Android's historical samples.
|
||||
The recordings are plain `t_ms action x y` files under
|
||||
`iris/transcript-fixture/touch/`, and `flick-120hz.touch` is the
|
||||
phone's own shape: DOWN, four samples 4ms apart, UP, 20ms in total.
|
||||
|
||||
cd iris && cargo test -p transcript-fixture
|
||||
|
||||
runs in about a second and asserts (a) the flick releases with a real
|
||||
velocity (`List::fling_velocity`, which only `Released(Some(v))`
|
||||
fills), (b) the list travels and settles inside the AOSP spline's own
|
||||
`FlingCalculator::duration`, (c) a tap moves nothing and opens no
|
||||
link, (d) a long-press-then-drag leaves selected text and does not
|
||||
pan, and (e) the composer clears a simulated 1000px IME inset
|
||||
(`Composer::set_bottom_inset`). Each was confirmed to fail without
|
||||
its subject rather than assumed: dropping `animate(id)` from
|
||||
`Selection::drag` -- the phone's own "fling does nothing" defect --
|
||||
and starting the fling curve at the wall clock each fail only the
|
||||
flick test; flinging on `Tapped` fails only the tap test; a 5s
|
||||
`LONG_PRESS` fails only the selection test; a `set_bottom_inset` that
|
||||
ignores its argument fails only the composer test.
|
||||
|
||||
**What still cannot be answered below layer 3**: nothing renders
|
||||
here, so anything about pixels -- glyph rasterisation, the atlas,
|
||||
stale or duplicated primitives, colour, the surface lifecycle, the
|
||||
renderer rebuild -- is invisible to layer 1 and only *looked at* in
|
||||
layer 2. Frame *times* are not measurable at either: layer 1 does no
|
||||
GPU work at all and layer 2 runs a debug build on this VM's virtio
|
||||
GPU, so a number from either is not the phone's. Anything JNI (the
|
||||
IME, real insets, the clipboard, battery) is layer 3 by construction:
|
||||
layer 1 records that the platform was asked and layer 2 has no
|
||||
Android platform to ask.
|
||||
|
||||
2. **A phone-shaped desktop window under headless sway -- for looking.**
|
||||
`iris/run-headless.sh` already runs a winit binary under a private
|
||||
sway and screenshots it with `grim`. Add a `--phone` mode (output and
|
||||
window at the phone's size and scale, with the scale reaching iris
|
||||
the way Android's does so dp layout runs at that density) and
|
||||
touch-shaped mouse input: a left-button drag pans and flings through
|
||||
`DragArbiter`, long-press selects, no hover. One input path, not a
|
||||
parallel one (`CODE_RULES`). Colour, spacing, text and anything a
|
||||
person has to see is answered here. `swaymsg seat - cursor
|
||||
move/press/release` drives it when a gesture is needed on screen.
|
||||
|
||||
cd iris && ./run-headless.sh phone --phone --shot /tmp/p.png -- -p transcript-fixture
|
||||
|
||||
About 15 seconds warm. `--phone` sets the private sway output to
|
||||
1080x2424@120Hz and exports `IRIS_SCALE=2.55`, which reaches iris the
|
||||
way `DisplayMetrics.density` does on Android
|
||||
(`iris::default::content_scale`) -- the desktop backend now lays out
|
||||
in physical pixels with a density instead of dividing into a separate
|
||||
logical space, so both platforms run one path. `transcript-fixture`'s
|
||||
`phone` example opens the same screen from the same bytes as layer 1
|
||||
and the Android bench.
|
||||
|
||||
A gesture on screen uses the *same recordings*:
|
||||
|
||||
./run-headless.sh phone --phone --replay transcript-fixture/touch/flick-120hz.touch \
|
||||
--shot /tmp/p.png -- -p transcript-fixture
|
||||
|
||||
writes `/tmp/p-before.png` and `/tmp/p.png` either side of the flick;
|
||||
looked at 2026-09-07, the list moved back about seven turns of the
|
||||
fixture and settled.
|
||||
|
||||
**`swaymsg seat - cursor` cannot drive it, and that cost an hour.**
|
||||
This compositor runs the headless backend with no input devices
|
||||
(`WLR_LIBINPUT_NO_DEVICES=1`, `LIBSEAT_BACKEND=noop`): the cursor
|
||||
commands all report `success` and nothing whatever reaches the
|
||||
client, with `swaymsg -t get_seats` showing `capabilities: 0` as the
|
||||
only sign. wlroots 0.19 dropped `WLR_HEADLESS_INPUTS`, and ydotool's
|
||||
uinput device would be ignored by a compositor that is not reading
|
||||
libinput. `iris/rig-input`'s `replay-touch` uses the
|
||||
**virtual-pointer protocol** instead, which is a client protocol and
|
||||
needs neither devices nor root, and it parses `iris::harness`'s own
|
||||
`TouchScript`. Two traps inside it, both found by printing winit's
|
||||
events: a button sent in the same frame as the motion that first puts
|
||||
the pointer over the window is dropped (the client sees the enter,
|
||||
the moves and the *release*, never the press), so the pointer is
|
||||
positioned and left to settle 200ms first; and a leftover window from
|
||||
an earlier manual run **tiles beside the new one**, halving the width
|
||||
and producing a screenshot that looks exactly like a duplicated-
|
||||
primitive rendering bug -- `swaymsg -t get_tree` and `pgrep -af
|
||||
examples/phone` are the check.
|
||||
|
||||
3. **The Android emulator -- platform plumbing and the final pass.**
|
||||
JNI, IME, insets, surface lifecycle, the renderer rebuild, and one
|
||||
verification run before a build goes to the phone. Not for iterating
|
||||
on layout.
|
||||
|
||||
Pass condition for the rig: `cargo test` runs a fixture-backed headless
|
||||
transcript screen with a replayed flick and asserts a nonzero release
|
||||
velocity and a moved scroll offset; one command opens the same screen in
|
||||
a phone-shaped window and screenshots it. Record the commands here when
|
||||
it lands.
|
||||
|
||||
### The 2026-09-07 phone report on `ed04d4c`: the fling was linear, and the keyboard is a targetSdk
|
||||
|
||||
Iris's three lines on the `ed04d4c` build (Pixel 9 Pro XL, GrapheneOS
|
||||
|
||||
Reference in new issue
Block a user