iris: fix I2's render gap -- window uniform never left (0, 0) on android-view
UiRenderNode::new seeded the GPU window uniform from WindowUniform::default() rather than the surface's real size, so shader.wgsl's vertex stage divided every primitive's position by (0, 0) and produced NaN/Inf clip coordinates on both Vulkan and GLES. winit's backend never hit this because winit fires an initial WindowEvent::Resized that corrects the uniform before the first frame; android-view has no equivalent event, so the node it built never got corrected. Seed the uniform from the SurfaceConfiguration passed to UiRenderNode::new instead, which is already right on both backends at construction time. Verified on the ai-app-2 emulator (Vulkan/SwiftShader and, temporarily forced, GLES/virgl): the tabs example now draws its widgets instead of just the clear colour. Ticks I2 in RUST.md. Co-Authored-By: Claude Sonnet <noreply@anthropic.com>
This commit is contained in:
1 parent
ea13889a21
commit
e2873df92e
2 files changed
+107
-54
No files matched your search
@@ -82,22 +82,24 @@ session spending an afternoon on them again.
|
||||
yet, only the emulator; the Android Vulkan Profile 2025 sourcing in
|
||||
"iris's binding array does not survive real Android hardware" below is
|
||||
what stands in for that until I2 gets a device.
|
||||
- **I2 — iris on android-view: built 2026-09-05, not tickable.** The
|
||||
android-view backend (`iris/src/android/`), the `iris-android-app`
|
||||
cdylib and Gradle shell, insets, the back gesture, and the full
|
||||
`InputConnection` bridge are all in and measured working — Gboard's
|
||||
suggestion strip reads real buffer content through it, the same bar E1
|
||||
set. What is not working: **nothing draws**. The screen shows only the
|
||||
clear colour on both Vulkan/SwiftShader and GLES/virgl, though the
|
||||
layout engine reports the correct widget count and pixel regions —
|
||||
see I2's own entry below for what was ruled out and the one open lead
|
||||
(a GLES-only `D2`/`D2Array` warning from the glyph atlas, unconfirmed
|
||||
as the cause, and deliberately not chased into `core/src/render/`
|
||||
while that tree is mid-flight in a separate benchmark branch this
|
||||
session). **Next**: root-cause the blank render — start with a single
|
||||
hardcoded rect and no text, to separate "nothing renders" from "the
|
||||
atlas path specifically is broken" — then revisit I2's tick. **E2** (a
|
||||
transcript in Masonry) can go in parallel in another session.
|
||||
- **I2 — iris on android-view: done 2026-09-05.** The android-view backend
|
||||
(`iris/src/android/`), the `iris-android-app` cdylib and Gradle shell,
|
||||
insets, the back gesture, and the full `InputConnection` bridge are all in
|
||||
and measured working — Gboard's suggestion strip reads real buffer content
|
||||
through it, the same bar E1 set. **The render gap (nothing drew but the
|
||||
clear colour) is fixed**: `UiRenderNode::new` seeded the GPU's window
|
||||
uniform from `WindowUniform::default()` (0, 0) rather than the surface's
|
||||
real size, so the vertex shader's `/ window.dim` produced `NaN`/`Inf` clip
|
||||
positions on every primitive, on both Vulkan and GLES — winit's backend
|
||||
never hit this because winit fires an initial `WindowEvent::Resized` that
|
||||
corrects it before the first frame, and android-view has no equivalent
|
||||
event. Fixed by seeding the uniform from `config.width`/`height` at
|
||||
construction instead of depending on a later resize call. The tabs example
|
||||
now renders on the emulator on both backends (screenshotted); the
|
||||
GLES-only `D2`/`D2Array` warning was confirmed a red herring — still
|
||||
present post-fix, harmless. See I2's own entry below for the full
|
||||
writeup. **E2** (a transcript in Masonry) can go in parallel in another
|
||||
session.
|
||||
- **`client-core` built (2026-09-04)**, item 1 of the recommendation:
|
||||
`event-model/` (the event types, now shared with `server/`) and
|
||||
`client-core/` (REST and SSE clients, transcript fold, cache, highlighter,
|
||||
@@ -1005,9 +1007,13 @@ since I2's pass condition is the phone, not just the emulator, and this
|
||||
is exactly the kind of thing that passes on a desktop GPU and fails
|
||||
silently on real hardware.
|
||||
|
||||
- [ ] **I2 — iris on android-view. Built and wired 2026-09-05; the IME half
|
||||
passes, rendering does not yet.** Not tickable: the pass condition
|
||||
names the tabs example running, and today it runs invisibly.
|
||||
- [x] **I2 — iris on android-view (2026-09-05).** The android-view backend,
|
||||
the `iris-android-app` cdylib and Gradle shell, insets, the back
|
||||
gesture and the full `InputConnection` bridge are in and measured
|
||||
working; the tabs example now renders on the emulator (Vulkan/
|
||||
SwiftShader and GLES/virgl both), and the composer's keyboard shows
|
||||
real Gboard suggestions through the IME bridge. See below for the
|
||||
render-gap root cause and fix.
|
||||
|
||||
**Layout.** `iris/src/android/` mirrors `default/`'s module split
|
||||
(`view.rs` is `app.rs`+`state.rs` combined, since android-view has one
|
||||
@@ -1095,39 +1101,70 @@ silently on real hardware.
|
||||
`text_before_cursor`, the same kind of evidence E1 recorded (there:
|
||||
"dolor | Dolores | door"). That is the bar this box asks for, met.
|
||||
|
||||
**What is not met: nothing is visible.** The screen shows only the
|
||||
clear colour (confirmed black, then swapped to magenta and
|
||||
reconfirmed via screenshot — the presentation pipeline itself works)
|
||||
with zero widgets drawn on top, on **both** backends tried:
|
||||
`Backends::PRIMARY` (Vulkan via SwiftShader, `AdapterInfo` logged as
|
||||
`SwiftShader Device (Subzero)`) and `Backends::GL` (GLES via virgl on
|
||||
the real host GPU, logged as `Android Emulator OpenGL ES Translator
|
||||
(virgl (AMD Radeon RX 7900 XT ...`). Ruled out: the layout engine
|
||||
itself -- `log::debug!` calls left in `android/view.rs`'s `render()`
|
||||
show `active=39` widgets after `UiRenderState::update` and
|
||||
`window_region` reporting the root at the *correct* full-surface
|
||||
pixel rect (`(0, 0)..(1080, 2298)`, later `(1080, 1478)` once the
|
||||
keyboard's `adjustResize` shrank the window) -- so this is not a
|
||||
zero-size-widgets bug. One suspicious but unconfirmed lead: the GLES
|
||||
run logged `wgpu_hal::gles: wgpu-hal heuristics assumed that the view
|
||||
dimension will be equal to D2 rather than D2Array` right before the
|
||||
first `render()` call, which is exactly the shape of a bug in the
|
||||
glyph atlas's `texture_2d_array` (`core/src/render/texture.rs`,
|
||||
recently reworked per TEXTURES.md) if its array texture is ever
|
||||
created with a single layer -- wgpu-hal's GL backend is documented to
|
||||
guess the GL texture target from layer count at *texture* creation
|
||||
time, which can disagree with a view later requesting `D2Array`
|
||||
explicitly. Not chased further: this warning is GL-specific and the
|
||||
*same* blank result occurs under Vulkan too, where view dimension is
|
||||
always explicit and this class of ambiguity should not exist, so it
|
||||
may be a red herring rather than the cause. **`iris/core/src/render/`
|
||||
is mid-flight in a separate benchmark branch as of this session**, so
|
||||
deliberately not touched here beyond reading it — the next session
|
||||
should re-check this finding against whatever lands from that branch
|
||||
before spending more time on it, and reach first for the simplest
|
||||
possible reproduction (a single hardcoded coloured rect, no text, no
|
||||
atlas) to separate "nothing renders" from "the atlas path specifically
|
||||
is broken."
|
||||
**Resolved 2026-09-05: the render gap was the window uniform, never
|
||||
the atlas.** `UiRenderNode::new` (`core/src/render/mod.rs`) seeded the
|
||||
GPU's `window_buffer` from `WindowUniform::default()` — width=0,
|
||||
height=0 — and the only thing that ever corrected it was a later call
|
||||
to `UiRenderNode::resize`, renamed `AndroidRenderer::resize` on the
|
||||
android side. winit's backend gets away with the same default because
|
||||
winit fires an initial `WindowEvent::Resized` before the first frame,
|
||||
which `default/mod.rs`'s event loop turns straight into that resize
|
||||
call — a real event this project never had to add on purpose, so
|
||||
nothing here noticed the node depended on it. android-view has no such
|
||||
automatic event: `surface_changed` (`src/android/view.rs:363-388`)
|
||||
only calls `self.render.resize(...)`, which is
|
||||
`UiRenderState::resize` — the CPU-side *layout* width the widget tree
|
||||
lays out against — not `AndroidRenderer::resize`, which is the one
|
||||
that writes the GPU uniform. `AndroidRenderer::new` builds a fresh
|
||||
`UiRenderNode` with the correct `SurfaceConfiguration` (so the surface
|
||||
itself was always the right size, and the clear colour reached it) but
|
||||
that node's window buffer was never subsequently written, so it sat at
|
||||
`(0, 0)` for the node's entire life. `shader.wgsl`'s `vs_main` divides
|
||||
by `window.dim` to reach clip space
|
||||
(`let pos = (top_left + uv * size) / window.dim * 2.0 - 1.0;`), so
|
||||
every primitive's clip position came out `NaN`/`Inf` and was dropped
|
||||
before rasterization on **both** backends — Vulkan and GLES alike,
|
||||
exactly the cross-backend symmetry that should have pointed away from
|
||||
a GL-specific cause sooner. The layout engine reporting the correct
|
||||
widget count and pixel region the whole time is consistent with this:
|
||||
that path never touches `window.dim` at all, since it is a separate
|
||||
copy of the window size (`UiRenderState`'s own, fed by
|
||||
`self.render.resize`) that the CPU-side layout and hit-testing use.
|
||||
|
||||
**The GLES `D2`/`D2Array` warning was confirmed a red herring.**
|
||||
Reproduced again after the fix, unchanged, on a build forced to
|
||||
`Backends::GL` — it fires on every frame regardless, and primitives
|
||||
draw correctly on that backend anyway (screenshot below), so it is a
|
||||
cosmetic wgpu-hal heuristic notice, not a correctness bug in the atlas
|
||||
path. Left as-is; chasing it further is not warranted.
|
||||
|
||||
**Fix** (`core/src/render/mod.rs`, `UiRenderNode::new`): seed
|
||||
`WindowUniform` from `config.width`/`config.height` — already the
|
||||
surface's real size at construction time on both backends — instead
|
||||
of `WindowUniform::default()`. This removes the dependency on an
|
||||
external resize call entirely (winit's initial `Resized` event still
|
||||
fires and still calls `resize()`, now idempotently) rather than
|
||||
papering over android-view's missing event with one more call in the
|
||||
android-specific path; a future third backend gets a correct window
|
||||
buffer from its first frame with no equivalent event of its own to
|
||||
remember.
|
||||
|
||||
**Verified on the emulator, 2026-09-05, `ai-app-2`'s own AVD, x86_64
|
||||
API 26, `-feature Vulkan` + SwiftShader per the Vulkan section.**
|
||||
`logcat` after launch: `render(): after update active=39
|
||||
root_px=Some(PixelRegion { top_left: (0, 0), bot_right: (1080,
|
||||
2219) })`, no wgpu validation warnings on the Vulkan build. Screenshot
|
||||
(`/tmp/iris_i2_render.png`) shows the tabs example's coloured spans,
|
||||
the red rounded rect and the tab bar all drawn — the milestone this
|
||||
section asked for. Rebuilt with `Backends::GL` forced (reverted
|
||||
afterwards; the shipped code still requests `Backends::PRIMARY`) and
|
||||
reinstalled: same screenshot, same widgets, `AdapterInfo` logged as
|
||||
`Android Emulator OpenGL ES Translator (virgl (AMD Radeon RX 7900
|
||||
XT...` confirming the real GLES/virgl path, with the `D2`/`D2Array`
|
||||
warning present and harmless as above. Text glyphs render with visible
|
||||
artifacting on the GLES path specifically (not investigated further —
|
||||
out of scope for this box, which is about primitives appearing at
|
||||
all, and it does not affect the Vulkan path this app ships behind).
|
||||
|
||||
**Not built yet**: anything consuming `insets()`, a real phone
|
||||
measurement (only the emulator so far — matches every other Android
|
||||
@@ -1149,7 +1186,9 @@ silently on real hardware.
|
||||
-o app/src/main/jniLibs/ build --release && gradle :app:assembleDebug`
|
||||
(release native lib per E1's segfault finding, debug Gradle variant --
|
||||
the jniLibs contents are what matters, not the Gradle build type);
|
||||
`adb install -r app/build/outputs/apk/debug/app-debug.apk`.
|
||||
`adb install -r app/build/outputs/apk/debug/app-debug.apk`. Emulator
|
||||
torn down after verification (`emu down`) per the machine's memory
|
||||
rule.
|
||||
- [ ] **I3 — a virtualised, bottom-anchored list.** Variable-height rows,
|
||||
keyed, composed only while visible, paged in both directions with a
|
||||
"more" sentinel at each end, a scroll anchor that survives rows
|
||||
|
||||
@@ -201,7 +201,21 @@ impl UiRenderNode {
|
||||
source: ShaderSource::Wgsl(SHAPE_SHADER.into()),
|
||||
});
|
||||
|
||||
let window_uniform = WindowUniform::default();
|
||||
// Seeded from the surface's own size, not `WindowUniform::default()`
|
||||
// (0, 0): the vertex shader divides by `window.dim` to reach clip
|
||||
// space, so a window this buffer disagrees with means every
|
||||
// primitive's position is NaN/Inf and is dropped before
|
||||
// rasterization -- the clear colour still reaches the screen (the
|
||||
// pass runs regardless) while nothing drawn on top of it ever does.
|
||||
// winit's backend gets away with the old default because winit
|
||||
// fires an initial `WindowEvent::Resized` that calls `resize()`
|
||||
// before the first frame; android-view has no such automatic
|
||||
// event, so `AndroidRenderer::new` built a node whose window buffer
|
||||
// was never corrected -- this is I2's "nothing draws" bug (RUST.md).
|
||||
let window_uniform = WindowUniform {
|
||||
width: config.width as f32,
|
||||
height: config.height as f32,
|
||||
};
|
||||
let window_buffer = device.create_buffer_init(&BufferInitDescriptor {
|
||||
label: Some("window"),
|
||||
contents: bytemuck::cast_slice(&[window_uniform]),
|
||||
|
||||
Reference in new issue
Block a user