iris: fix I2's render gap -- window uniform never left (0, 0) on android-view

UiRenderNode::new seeded the GPU window uniform from
WindowUniform::default() rather than the surface's real size, so
shader.wgsl's vertex stage divided every primitive's position by
(0, 0) and produced NaN/Inf clip coordinates on both Vulkan and GLES.
winit's backend never hit this because winit fires an initial
WindowEvent::Resized that corrects the uniform before the first frame;
android-view has no equivalent event, so the node it built never got
corrected. Seed the uniform from the SurfaceConfiguration passed to
UiRenderNode::new instead, which is already right on both backends at
construction time.

Verified on the ai-app-2 emulator (Vulkan/SwiftShader and, temporarily
forced, GLES/virgl): the tabs example now draws its widgets instead of
just the clear colour. Ticks I2 in RUST.md.

Co-Authored-By: Claude Sonnet <noreply@anthropic.com>
This commit is contained in:
irisandClaude Sonnet committed 2026-09-05 05:19:55 -04:00
1 parent ea13889a21
commit e2873df92e
2 files changed
+107 -54

No files matched your search

+92 -53
View File
@@ -82,22 +82,24 @@ session spending an afternoon on them again.
yet, only the emulator; the Android Vulkan Profile 2025 sourcing in
"iris's binding array does not survive real Android hardware" below is
what stands in for that until I2 gets a device.
- **I2 — iris on android-view: built 2026-09-05, not tickable.** The
android-view backend (`iris/src/android/`), the `iris-android-app`
cdylib and Gradle shell, insets, the back gesture, and the full
`InputConnection` bridge are all in and measured working — Gboard's
suggestion strip reads real buffer content through it, the same bar E1
set. What is not working: **nothing draws**. The screen shows only the
clear colour on both Vulkan/SwiftShader and GLES/virgl, though the
layout engine reports the correct widget count and pixel regions —
see I2's own entry below for what was ruled out and the one open lead
(a GLES-only `D2`/`D2Array` warning from the glyph atlas, unconfirmed
as the cause, and deliberately not chased into `core/src/render/`
while that tree is mid-flight in a separate benchmark branch this
session). **Next**: root-cause the blank render — start with a single
hardcoded rect and no text, to separate "nothing renders" from "the
atlas path specifically is broken" — then revisit I2's tick. **E2** (a
transcript in Masonry) can go in parallel in another session.
- **I2 — iris on android-view: done 2026-09-05.** The android-view backend
(`iris/src/android/`), the `iris-android-app` cdylib and Gradle shell,
insets, the back gesture, and the full `InputConnection` bridge are all in
and measured working — Gboard's suggestion strip reads real buffer content
through it, the same bar E1 set. **The render gap (nothing drew but the
clear colour) is fixed**: `UiRenderNode::new` seeded the GPU's window
uniform from `WindowUniform::default()` (0, 0) rather than the surface's
real size, so the vertex shader's `/ window.dim` produced `NaN`/`Inf` clip
positions on every primitive, on both Vulkan and GLES — winit's backend
never hit this because winit fires an initial `WindowEvent::Resized` that
corrects it before the first frame, and android-view has no equivalent
event. Fixed by seeding the uniform from `config.width`/`height` at
construction instead of depending on a later resize call. The tabs example
now renders on the emulator on both backends (screenshotted); the
GLES-only `D2`/`D2Array` warning was confirmed a red herring — still
present post-fix, harmless. See I2's own entry below for the full
writeup. **E2** (a transcript in Masonry) can go in parallel in another
session.
- **`client-core` built (2026-09-04)**, item 1 of the recommendation:
`event-model/` (the event types, now shared with `server/`) and
`client-core/` (REST and SSE clients, transcript fold, cache, highlighter,
@@ -1005,9 +1007,13 @@ since I2's pass condition is the phone, not just the emulator, and this
is exactly the kind of thing that passes on a desktop GPU and fails
silently on real hardware.
- [ ] **I2 — iris on android-view. Built and wired 2026-09-05; the IME half
passes, rendering does not yet.** Not tickable: the pass condition
names the tabs example running, and today it runs invisibly.
- [x] **I2 — iris on android-view (2026-09-05).** The android-view backend,
the `iris-android-app` cdylib and Gradle shell, insets, the back
gesture and the full `InputConnection` bridge are in and measured
working; the tabs example now renders on the emulator (Vulkan/
SwiftShader and GLES/virgl both), and the composer's keyboard shows
real Gboard suggestions through the IME bridge. See below for the
render-gap root cause and fix.
**Layout.** `iris/src/android/` mirrors `default/`'s module split
(`view.rs` is `app.rs`+`state.rs` combined, since android-view has one
@@ -1095,39 +1101,70 @@ silently on real hardware.
`text_before_cursor`, the same kind of evidence E1 recorded (there:
"dolor | Dolores | door"). That is the bar this box asks for, met.
**What is not met: nothing is visible.** The screen shows only the
clear colour (confirmed black, then swapped to magenta and
reconfirmed via screenshot — the presentation pipeline itself works)
with zero widgets drawn on top, on **both** backends tried:
`Backends::PRIMARY` (Vulkan via SwiftShader, `AdapterInfo` logged as
`SwiftShader Device (Subzero)`) and `Backends::GL` (GLES via virgl on
the real host GPU, logged as `Android Emulator OpenGL ES Translator
(virgl (AMD Radeon RX 7900 XT ...`). Ruled out: the layout engine
itself -- `log::debug!` calls left in `android/view.rs`'s `render()`
show `active=39` widgets after `UiRenderState::update` and
`window_region` reporting the root at the *correct* full-surface
pixel rect (`(0, 0)..(1080, 2298)`, later `(1080, 1478)` once the
keyboard's `adjustResize` shrank the window) -- so this is not a
zero-size-widgets bug. One suspicious but unconfirmed lead: the GLES
run logged `wgpu_hal::gles: wgpu-hal heuristics assumed that the view
dimension will be equal to D2 rather than D2Array` right before the
first `render()` call, which is exactly the shape of a bug in the
glyph atlas's `texture_2d_array` (`core/src/render/texture.rs`,
recently reworked per TEXTURES.md) if its array texture is ever
created with a single layer -- wgpu-hal's GL backend is documented to
guess the GL texture target from layer count at *texture* creation
time, which can disagree with a view later requesting `D2Array`
explicitly. Not chased further: this warning is GL-specific and the
*same* blank result occurs under Vulkan too, where view dimension is
always explicit and this class of ambiguity should not exist, so it
may be a red herring rather than the cause. **`iris/core/src/render/`
is mid-flight in a separate benchmark branch as of this session**, so
deliberately not touched here beyond reading it — the next session
should re-check this finding against whatever lands from that branch
before spending more time on it, and reach first for the simplest
possible reproduction (a single hardcoded coloured rect, no text, no
atlas) to separate "nothing renders" from "the atlas path specifically
is broken."
**Resolved 2026-09-05: the render gap was the window uniform, never
the atlas.** `UiRenderNode::new` (`core/src/render/mod.rs`) seeded the
GPU's `window_buffer` from `WindowUniform::default()` — width=0,
height=0 — and the only thing that ever corrected it was a later call
to `UiRenderNode::resize`, renamed `AndroidRenderer::resize` on the
android side. winit's backend gets away with the same default because
winit fires an initial `WindowEvent::Resized` before the first frame,
which `default/mod.rs`'s event loop turns straight into that resize
call — a real event this project never had to add on purpose, so
nothing here noticed the node depended on it. android-view has no such
automatic event: `surface_changed` (`src/android/view.rs:363-388`)
only calls `self.render.resize(...)`, which is
`UiRenderState::resize` — the CPU-side *layout* width the widget tree
lays out against — not `AndroidRenderer::resize`, which is the one
that writes the GPU uniform. `AndroidRenderer::new` builds a fresh
`UiRenderNode` with the correct `SurfaceConfiguration` (so the surface
itself was always the right size, and the clear colour reached it) but
that node's window buffer was never subsequently written, so it sat at
`(0, 0)` for the node's entire life. `shader.wgsl`'s `vs_main` divides
by `window.dim` to reach clip space
(`let pos = (top_left + uv * size) / window.dim * 2.0 - 1.0;`), so
every primitive's clip position came out `NaN`/`Inf` and was dropped
before rasterization on **both** backends — Vulkan and GLES alike,
exactly the cross-backend symmetry that should have pointed away from
a GL-specific cause sooner. The layout engine reporting the correct
widget count and pixel region the whole time is consistent with this:
that path never touches `window.dim` at all, since it is a separate
copy of the window size (`UiRenderState`'s own, fed by
`self.render.resize`) that the CPU-side layout and hit-testing use.
**The GLES `D2`/`D2Array` warning was confirmed a red herring.**
Reproduced again after the fix, unchanged, on a build forced to
`Backends::GL` — it fires on every frame regardless, and primitives
draw correctly on that backend anyway (screenshot below), so it is a
cosmetic wgpu-hal heuristic notice, not a correctness bug in the atlas
path. Left as-is; chasing it further is not warranted.
**Fix** (`core/src/render/mod.rs`, `UiRenderNode::new`): seed
`WindowUniform` from `config.width`/`config.height` — already the
surface's real size at construction time on both backends — instead
of `WindowUniform::default()`. This removes the dependency on an
external resize call entirely (winit's initial `Resized` event still
fires and still calls `resize()`, now idempotently) rather than
papering over android-view's missing event with one more call in the
android-specific path; a future third backend gets a correct window
buffer from its first frame with no equivalent event of its own to
remember.
**Verified on the emulator, 2026-09-05, `ai-app-2`'s own AVD, x86_64
API 26, `-feature Vulkan` + SwiftShader per the Vulkan section.**
`logcat` after launch: `render(): after update active=39
root_px=Some(PixelRegion { top_left: (0, 0), bot_right: (1080,
2219) })`, no wgpu validation warnings on the Vulkan build. Screenshot
(`/tmp/iris_i2_render.png`) shows the tabs example's coloured spans,
the red rounded rect and the tab bar all drawn — the milestone this
section asked for. Rebuilt with `Backends::GL` forced (reverted
afterwards; the shipped code still requests `Backends::PRIMARY`) and
reinstalled: same screenshot, same widgets, `AdapterInfo` logged as
`Android Emulator OpenGL ES Translator (virgl (AMD Radeon RX 7900
XT...` confirming the real GLES/virgl path, with the `D2`/`D2Array`
warning present and harmless as above. Text glyphs render with visible
artifacting on the GLES path specifically (not investigated further —
out of scope for this box, which is about primitives appearing at
all, and it does not affect the Vulkan path this app ships behind).
**Not built yet**: anything consuming `insets()`, a real phone
measurement (only the emulator so far — matches every other Android
@@ -1149,7 +1186,9 @@ silently on real hardware.
-o app/src/main/jniLibs/ build --release && gradle :app:assembleDebug`
(release native lib per E1's segfault finding, debug Gradle variant --
the jniLibs contents are what matters, not the Gradle build type);
`adb install -r app/build/outputs/apk/debug/app-debug.apk`.
`adb install -r app/build/outputs/apk/debug/app-debug.apk`. Emulator
torn down after verification (`emu down`) per the machine's memory
rule.
- [ ] **I3 — a virtualised, bottom-anchored list.** Variable-height rows,
keyed, composed only while visible, paged in both directions with a
"more" sentinel at each end, a scroll anchor that survives rows
+15 -1
View File
@@ -201,7 +201,21 @@ impl UiRenderNode {
source: ShaderSource::Wgsl(SHAPE_SHADER.into()),
});
let window_uniform = WindowUniform::default();
// Seeded from the surface's own size, not `WindowUniform::default()`
// (0, 0): the vertex shader divides by `window.dim` to reach clip
// space, so a window this buffer disagrees with means every
// primitive's position is NaN/Inf and is dropped before
// rasterization -- the clear colour still reaches the screen (the
// pass runs regardless) while nothing drawn on top of it ever does.
// winit's backend gets away with the old default because winit
// fires an initial `WindowEvent::Resized` that calls `resize()`
// before the first frame; android-view has no such automatic
// event, so `AndroidRenderer::new` built a node whose window buffer
// was never corrected -- this is I2's "nothing draws" bug (RUST.md).
let window_uniform = WindowUniform {
width: config.width as f32,
height: config.height as f32,
};
let window_buffer = device.create_buffer_init(&BufferInitDescriptor {
label: Some("window"),
contents: bytemuck::cast_slice(&[window_uniform]),