iris: fix I2's render gap -- window uniform never left (0, 0) on android-view

UiRenderNode::new seeded the GPU window uniform from
WindowUniform::default() rather than the surface's real size, so
shader.wgsl's vertex stage divided every primitive's position by
(0, 0) and produced NaN/Inf clip coordinates on both Vulkan and GLES.
winit's backend never hit this because winit fires an initial
WindowEvent::Resized that corrects the uniform before the first frame;
android-view has no equivalent event, so the node it built never got
corrected. Seed the uniform from the SurfaceConfiguration passed to
UiRenderNode::new instead, which is already right on both backends at
construction time.

Verified on the ai-app-2 emulator (Vulkan/SwiftShader and, temporarily
forced, GLES/virgl): the tabs example now draws its widgets instead of
just the clear colour. Ticks I2 in RUST.md.

Co-Authored-By: Claude Sonnet <noreply@anthropic.com>
This commit is contained in:
irisandClaude Sonnet committed 2026-09-05 05:19:55 -04:00
1 parent ea13889a21
commit e2873df92e
2 files changed
+107 -54

No files matched your search

+92 -53
View File
@@ -82,22 +82,24 @@ session spending an afternoon on them again.
yet, only the emulator; the Android Vulkan Profile 2025 sourcing in yet, only the emulator; the Android Vulkan Profile 2025 sourcing in
"iris's binding array does not survive real Android hardware" below is "iris's binding array does not survive real Android hardware" below is
what stands in for that until I2 gets a device. what stands in for that until I2 gets a device.
- **I2 — iris on android-view: built 2026-09-05, not tickable.** The - **I2 — iris on android-view: done 2026-09-05.** The android-view backend
android-view backend (`iris/src/android/`), the `iris-android-app` (`iris/src/android/`), the `iris-android-app` cdylib and Gradle shell,
cdylib and Gradle shell, insets, the back gesture, and the full insets, the back gesture, and the full `InputConnection` bridge are all in
`InputConnection` bridge are all in and measured working — Gboard's and measured working — Gboard's suggestion strip reads real buffer content
suggestion strip reads real buffer content through it, the same bar E1 through it, the same bar E1 set. **The render gap (nothing drew but the
set. What is not working: **nothing draws**. The screen shows only the clear colour) is fixed**: `UiRenderNode::new` seeded the GPU's window
clear colour on both Vulkan/SwiftShader and GLES/virgl, though the uniform from `WindowUniform::default()` (0, 0) rather than the surface's
layout engine reports the correct widget count and pixel regions — real size, so the vertex shader's `/ window.dim` produced `NaN`/`Inf` clip
see I2's own entry below for what was ruled out and the one open lead positions on every primitive, on both Vulkan and GLES — winit's backend
(a GLES-only `D2`/`D2Array` warning from the glyph atlas, unconfirmed never hit this because winit fires an initial `WindowEvent::Resized` that
as the cause, and deliberately not chased into `core/src/render/` corrects it before the first frame, and android-view has no equivalent
while that tree is mid-flight in a separate benchmark branch this event. Fixed by seeding the uniform from `config.width`/`height` at
session). **Next**: root-cause the blank render — start with a single construction instead of depending on a later resize call. The tabs example
hardcoded rect and no text, to separate "nothing renders" from "the now renders on the emulator on both backends (screenshotted); the
atlas path specifically is broken" — then revisit I2's tick. **E2** (a GLES-only `D2`/`D2Array` warning was confirmed a red herring — still
transcript in Masonry) can go in parallel in another session. present post-fix, harmless. See I2's own entry below for the full
writeup. **E2** (a transcript in Masonry) can go in parallel in another
session.
- **`client-core` built (2026-09-04)**, item 1 of the recommendation: - **`client-core` built (2026-09-04)**, item 1 of the recommendation:
`event-model/` (the event types, now shared with `server/`) and `event-model/` (the event types, now shared with `server/`) and
`client-core/` (REST and SSE clients, transcript fold, cache, highlighter, `client-core/` (REST and SSE clients, transcript fold, cache, highlighter,
@@ -1005,9 +1007,13 @@ since I2's pass condition is the phone, not just the emulator, and this
is exactly the kind of thing that passes on a desktop GPU and fails is exactly the kind of thing that passes on a desktop GPU and fails
silently on real hardware. silently on real hardware.
- [ ] **I2 — iris on android-view. Built and wired 2026-09-05; the IME half - [x] **I2 — iris on android-view (2026-09-05).** The android-view backend,
passes, rendering does not yet.** Not tickable: the pass condition the `iris-android-app` cdylib and Gradle shell, insets, the back
names the tabs example running, and today it runs invisibly. gesture and the full `InputConnection` bridge are in and measured
working; the tabs example now renders on the emulator (Vulkan/
SwiftShader and GLES/virgl both), and the composer's keyboard shows
real Gboard suggestions through the IME bridge. See below for the
render-gap root cause and fix.
**Layout.** `iris/src/android/` mirrors `default/`'s module split **Layout.** `iris/src/android/` mirrors `default/`'s module split
(`view.rs` is `app.rs`+`state.rs` combined, since android-view has one (`view.rs` is `app.rs`+`state.rs` combined, since android-view has one
@@ -1095,39 +1101,70 @@ silently on real hardware.
`text_before_cursor`, the same kind of evidence E1 recorded (there: `text_before_cursor`, the same kind of evidence E1 recorded (there:
"dolor | Dolores | door"). That is the bar this box asks for, met. "dolor | Dolores | door"). That is the bar this box asks for, met.
**What is not met: nothing is visible.** The screen shows only the **Resolved 2026-09-05: the render gap was the window uniform, never
clear colour (confirmed black, then swapped to magenta and the atlas.** `UiRenderNode::new` (`core/src/render/mod.rs`) seeded the
reconfirmed via screenshot — the presentation pipeline itself works) GPU's `window_buffer` from `WindowUniform::default()` — width=0,
with zero widgets drawn on top, on **both** backends tried: height=0 — and the only thing that ever corrected it was a later call
`Backends::PRIMARY` (Vulkan via SwiftShader, `AdapterInfo` logged as to `UiRenderNode::resize`, renamed `AndroidRenderer::resize` on the
`SwiftShader Device (Subzero)`) and `Backends::GL` (GLES via virgl on android side. winit's backend gets away with the same default because
the real host GPU, logged as `Android Emulator OpenGL ES Translator winit fires an initial `WindowEvent::Resized` before the first frame,
(virgl (AMD Radeon RX 7900 XT ...`). Ruled out: the layout engine which `default/mod.rs`'s event loop turns straight into that resize
itself -- `log::debug!` calls left in `android/view.rs`'s `render()` call — a real event this project never had to add on purpose, so
show `active=39` widgets after `UiRenderState::update` and nothing here noticed the node depended on it. android-view has no such
`window_region` reporting the root at the *correct* full-surface automatic event: `surface_changed` (`src/android/view.rs:363-388`)
pixel rect (`(0, 0)..(1080, 2298)`, later `(1080, 1478)` once the only calls `self.render.resize(...)`, which is
keyboard's `adjustResize` shrank the window) -- so this is not a `UiRenderState::resize` — the CPU-side *layout* width the widget tree
zero-size-widgets bug. One suspicious but unconfirmed lead: the GLES lays out against — not `AndroidRenderer::resize`, which is the one
run logged `wgpu_hal::gles: wgpu-hal heuristics assumed that the view that writes the GPU uniform. `AndroidRenderer::new` builds a fresh
dimension will be equal to D2 rather than D2Array` right before the `UiRenderNode` with the correct `SurfaceConfiguration` (so the surface
first `render()` call, which is exactly the shape of a bug in the itself was always the right size, and the clear colour reached it) but
glyph atlas's `texture_2d_array` (`core/src/render/texture.rs`, that node's window buffer was never subsequently written, so it sat at
recently reworked per TEXTURES.md) if its array texture is ever `(0, 0)` for the node's entire life. `shader.wgsl`'s `vs_main` divides
created with a single layer -- wgpu-hal's GL backend is documented to by `window.dim` to reach clip space
guess the GL texture target from layer count at *texture* creation (`let pos = (top_left + uv * size) / window.dim * 2.0 - 1.0;`), so
time, which can disagree with a view later requesting `D2Array` every primitive's clip position came out `NaN`/`Inf` and was dropped
explicitly. Not chased further: this warning is GL-specific and the before rasterization on **both** backends — Vulkan and GLES alike,
*same* blank result occurs under Vulkan too, where view dimension is exactly the cross-backend symmetry that should have pointed away from
always explicit and this class of ambiguity should not exist, so it a GL-specific cause sooner. The layout engine reporting the correct
may be a red herring rather than the cause. **`iris/core/src/render/` widget count and pixel region the whole time is consistent with this:
is mid-flight in a separate benchmark branch as of this session**, so that path never touches `window.dim` at all, since it is a separate
deliberately not touched here beyond reading it — the next session copy of the window size (`UiRenderState`'s own, fed by
should re-check this finding against whatever lands from that branch `self.render.resize`) that the CPU-side layout and hit-testing use.
before spending more time on it, and reach first for the simplest
possible reproduction (a single hardcoded coloured rect, no text, no **The GLES `D2`/`D2Array` warning was confirmed a red herring.**
atlas) to separate "nothing renders" from "the atlas path specifically Reproduced again after the fix, unchanged, on a build forced to
is broken." `Backends::GL` — it fires on every frame regardless, and primitives
draw correctly on that backend anyway (screenshot below), so it is a
cosmetic wgpu-hal heuristic notice, not a correctness bug in the atlas
path. Left as-is; chasing it further is not warranted.
**Fix** (`core/src/render/mod.rs`, `UiRenderNode::new`): seed
`WindowUniform` from `config.width`/`config.height` — already the
surface's real size at construction time on both backends — instead
of `WindowUniform::default()`. This removes the dependency on an
external resize call entirely (winit's initial `Resized` event still
fires and still calls `resize()`, now idempotently) rather than
papering over android-view's missing event with one more call in the
android-specific path; a future third backend gets a correct window
buffer from its first frame with no equivalent event of its own to
remember.
**Verified on the emulator, 2026-09-05, `ai-app-2`'s own AVD, x86_64
API 26, `-feature Vulkan` + SwiftShader per the Vulkan section.**
`logcat` after launch: `render(): after update active=39
root_px=Some(PixelRegion { top_left: (0, 0), bot_right: (1080,
2219) })`, no wgpu validation warnings on the Vulkan build. Screenshot
(`/tmp/iris_i2_render.png`) shows the tabs example's coloured spans,
the red rounded rect and the tab bar all drawn — the milestone this
section asked for. Rebuilt with `Backends::GL` forced (reverted
afterwards; the shipped code still requests `Backends::PRIMARY`) and
reinstalled: same screenshot, same widgets, `AdapterInfo` logged as
`Android Emulator OpenGL ES Translator (virgl (AMD Radeon RX 7900
XT...` confirming the real GLES/virgl path, with the `D2`/`D2Array`
warning present and harmless as above. Text glyphs render with visible
artifacting on the GLES path specifically (not investigated further —
out of scope for this box, which is about primitives appearing at
all, and it does not affect the Vulkan path this app ships behind).
**Not built yet**: anything consuming `insets()`, a real phone **Not built yet**: anything consuming `insets()`, a real phone
measurement (only the emulator so far — matches every other Android measurement (only the emulator so far — matches every other Android
@@ -1149,7 +1186,9 @@ silently on real hardware.
-o app/src/main/jniLibs/ build --release && gradle :app:assembleDebug` -o app/src/main/jniLibs/ build --release && gradle :app:assembleDebug`
(release native lib per E1's segfault finding, debug Gradle variant -- (release native lib per E1's segfault finding, debug Gradle variant --
the jniLibs contents are what matters, not the Gradle build type); the jniLibs contents are what matters, not the Gradle build type);
`adb install -r app/build/outputs/apk/debug/app-debug.apk`. `adb install -r app/build/outputs/apk/debug/app-debug.apk`. Emulator
torn down after verification (`emu down`) per the machine's memory
rule.
- [ ] **I3 — a virtualised, bottom-anchored list.** Variable-height rows, - [ ] **I3 — a virtualised, bottom-anchored list.** Variable-height rows,
keyed, composed only while visible, paged in both directions with a keyed, composed only while visible, paged in both directions with a
"more" sentinel at each end, a scroll anchor that survives rows "more" sentinel at each end, a scroll anchor that survives rows
+15 -1
View File
@@ -201,7 +201,21 @@ impl UiRenderNode {
source: ShaderSource::Wgsl(SHAPE_SHADER.into()), source: ShaderSource::Wgsl(SHAPE_SHADER.into()),
}); });
let window_uniform = WindowUniform::default(); // Seeded from the surface's own size, not `WindowUniform::default()`
// (0, 0): the vertex shader divides by `window.dim` to reach clip
// space, so a window this buffer disagrees with means every
// primitive's position is NaN/Inf and is dropped before
// rasterization -- the clear colour still reaches the screen (the
// pass runs regardless) while nothing drawn on top of it ever does.
// winit's backend gets away with the old default because winit
// fires an initial `WindowEvent::Resized` that calls `resize()`
// before the first frame; android-view has no such automatic
// event, so `AndroidRenderer::new` built a node whose window buffer
// was never corrected -- this is I2's "nothing draws" bug (RUST.md).
let window_uniform = WindowUniform {
width: config.width as f32,
height: config.height as f32,
};
let window_buffer = device.create_buffer_init(&BufferInitDescriptor { let window_buffer = device.create_buffer_init(&BufferInitDescriptor {
label: Some("window"), label: Some("window"),
contents: bytemuck::cast_slice(&[window_uniform]), contents: bytemuck::cast_slice(&[window_uniform]),