iris: fix I2's render gap -- window uniform never left (0, 0) on android-view
UiRenderNode::new seeded the GPU window uniform from WindowUniform::default() rather than the surface's real size, so shader.wgsl's vertex stage divided every primitive's position by (0, 0) and produced NaN/Inf clip coordinates on both Vulkan and GLES. winit's backend never hit this because winit fires an initial WindowEvent::Resized that corrects the uniform before the first frame; android-view has no equivalent event, so the node it built never got corrected. Seed the uniform from the SurfaceConfiguration passed to UiRenderNode::new instead, which is already right on both backends at construction time. Verified on the ai-app-2 emulator (Vulkan/SwiftShader and, temporarily forced, GLES/virgl): the tabs example now draws its widgets instead of just the clear colour. Ticks I2 in RUST.md. Co-Authored-By: Claude Sonnet <noreply@anthropic.com>
This commit is contained in:
1 parent
ea13889a21
commit
e2873df92e
2 files changed
+107
-54
No files matched your search
@@ -82,22 +82,24 @@ session spending an afternoon on them again.
|
|||||||
yet, only the emulator; the Android Vulkan Profile 2025 sourcing in
|
yet, only the emulator; the Android Vulkan Profile 2025 sourcing in
|
||||||
"iris's binding array does not survive real Android hardware" below is
|
"iris's binding array does not survive real Android hardware" below is
|
||||||
what stands in for that until I2 gets a device.
|
what stands in for that until I2 gets a device.
|
||||||
- **I2 — iris on android-view: built 2026-09-05, not tickable.** The
|
- **I2 — iris on android-view: done 2026-09-05.** The android-view backend
|
||||||
android-view backend (`iris/src/android/`), the `iris-android-app`
|
(`iris/src/android/`), the `iris-android-app` cdylib and Gradle shell,
|
||||||
cdylib and Gradle shell, insets, the back gesture, and the full
|
insets, the back gesture, and the full `InputConnection` bridge are all in
|
||||||
`InputConnection` bridge are all in and measured working — Gboard's
|
and measured working — Gboard's suggestion strip reads real buffer content
|
||||||
suggestion strip reads real buffer content through it, the same bar E1
|
through it, the same bar E1 set. **The render gap (nothing drew but the
|
||||||
set. What is not working: **nothing draws**. The screen shows only the
|
clear colour) is fixed**: `UiRenderNode::new` seeded the GPU's window
|
||||||
clear colour on both Vulkan/SwiftShader and GLES/virgl, though the
|
uniform from `WindowUniform::default()` (0, 0) rather than the surface's
|
||||||
layout engine reports the correct widget count and pixel regions —
|
real size, so the vertex shader's `/ window.dim` produced `NaN`/`Inf` clip
|
||||||
see I2's own entry below for what was ruled out and the one open lead
|
positions on every primitive, on both Vulkan and GLES — winit's backend
|
||||||
(a GLES-only `D2`/`D2Array` warning from the glyph atlas, unconfirmed
|
never hit this because winit fires an initial `WindowEvent::Resized` that
|
||||||
as the cause, and deliberately not chased into `core/src/render/`
|
corrects it before the first frame, and android-view has no equivalent
|
||||||
while that tree is mid-flight in a separate benchmark branch this
|
event. Fixed by seeding the uniform from `config.width`/`height` at
|
||||||
session). **Next**: root-cause the blank render — start with a single
|
construction instead of depending on a later resize call. The tabs example
|
||||||
hardcoded rect and no text, to separate "nothing renders" from "the
|
now renders on the emulator on both backends (screenshotted); the
|
||||||
atlas path specifically is broken" — then revisit I2's tick. **E2** (a
|
GLES-only `D2`/`D2Array` warning was confirmed a red herring — still
|
||||||
transcript in Masonry) can go in parallel in another session.
|
present post-fix, harmless. See I2's own entry below for the full
|
||||||
|
writeup. **E2** (a transcript in Masonry) can go in parallel in another
|
||||||
|
session.
|
||||||
- **`client-core` built (2026-09-04)**, item 1 of the recommendation:
|
- **`client-core` built (2026-09-04)**, item 1 of the recommendation:
|
||||||
`event-model/` (the event types, now shared with `server/`) and
|
`event-model/` (the event types, now shared with `server/`) and
|
||||||
`client-core/` (REST and SSE clients, transcript fold, cache, highlighter,
|
`client-core/` (REST and SSE clients, transcript fold, cache, highlighter,
|
||||||
@@ -1005,9 +1007,13 @@ since I2's pass condition is the phone, not just the emulator, and this
|
|||||||
is exactly the kind of thing that passes on a desktop GPU and fails
|
is exactly the kind of thing that passes on a desktop GPU and fails
|
||||||
silently on real hardware.
|
silently on real hardware.
|
||||||
|
|
||||||
- [ ] **I2 — iris on android-view. Built and wired 2026-09-05; the IME half
|
- [x] **I2 — iris on android-view (2026-09-05).** The android-view backend,
|
||||||
passes, rendering does not yet.** Not tickable: the pass condition
|
the `iris-android-app` cdylib and Gradle shell, insets, the back
|
||||||
names the tabs example running, and today it runs invisibly.
|
gesture and the full `InputConnection` bridge are in and measured
|
||||||
|
working; the tabs example now renders on the emulator (Vulkan/
|
||||||
|
SwiftShader and GLES/virgl both), and the composer's keyboard shows
|
||||||
|
real Gboard suggestions through the IME bridge. See below for the
|
||||||
|
render-gap root cause and fix.
|
||||||
|
|
||||||
**Layout.** `iris/src/android/` mirrors `default/`'s module split
|
**Layout.** `iris/src/android/` mirrors `default/`'s module split
|
||||||
(`view.rs` is `app.rs`+`state.rs` combined, since android-view has one
|
(`view.rs` is `app.rs`+`state.rs` combined, since android-view has one
|
||||||
@@ -1095,39 +1101,70 @@ silently on real hardware.
|
|||||||
`text_before_cursor`, the same kind of evidence E1 recorded (there:
|
`text_before_cursor`, the same kind of evidence E1 recorded (there:
|
||||||
"dolor | Dolores | door"). That is the bar this box asks for, met.
|
"dolor | Dolores | door"). That is the bar this box asks for, met.
|
||||||
|
|
||||||
**What is not met: nothing is visible.** The screen shows only the
|
**Resolved 2026-09-05: the render gap was the window uniform, never
|
||||||
clear colour (confirmed black, then swapped to magenta and
|
the atlas.** `UiRenderNode::new` (`core/src/render/mod.rs`) seeded the
|
||||||
reconfirmed via screenshot — the presentation pipeline itself works)
|
GPU's `window_buffer` from `WindowUniform::default()` — width=0,
|
||||||
with zero widgets drawn on top, on **both** backends tried:
|
height=0 — and the only thing that ever corrected it was a later call
|
||||||
`Backends::PRIMARY` (Vulkan via SwiftShader, `AdapterInfo` logged as
|
to `UiRenderNode::resize`, renamed `AndroidRenderer::resize` on the
|
||||||
`SwiftShader Device (Subzero)`) and `Backends::GL` (GLES via virgl on
|
android side. winit's backend gets away with the same default because
|
||||||
the real host GPU, logged as `Android Emulator OpenGL ES Translator
|
winit fires an initial `WindowEvent::Resized` before the first frame,
|
||||||
(virgl (AMD Radeon RX 7900 XT ...`). Ruled out: the layout engine
|
which `default/mod.rs`'s event loop turns straight into that resize
|
||||||
itself -- `log::debug!` calls left in `android/view.rs`'s `render()`
|
call — a real event this project never had to add on purpose, so
|
||||||
show `active=39` widgets after `UiRenderState::update` and
|
nothing here noticed the node depended on it. android-view has no such
|
||||||
`window_region` reporting the root at the *correct* full-surface
|
automatic event: `surface_changed` (`src/android/view.rs:363-388`)
|
||||||
pixel rect (`(0, 0)..(1080, 2298)`, later `(1080, 1478)` once the
|
only calls `self.render.resize(...)`, which is
|
||||||
keyboard's `adjustResize` shrank the window) -- so this is not a
|
`UiRenderState::resize` — the CPU-side *layout* width the widget tree
|
||||||
zero-size-widgets bug. One suspicious but unconfirmed lead: the GLES
|
lays out against — not `AndroidRenderer::resize`, which is the one
|
||||||
run logged `wgpu_hal::gles: wgpu-hal heuristics assumed that the view
|
that writes the GPU uniform. `AndroidRenderer::new` builds a fresh
|
||||||
dimension will be equal to D2 rather than D2Array` right before the
|
`UiRenderNode` with the correct `SurfaceConfiguration` (so the surface
|
||||||
first `render()` call, which is exactly the shape of a bug in the
|
itself was always the right size, and the clear colour reached it) but
|
||||||
glyph atlas's `texture_2d_array` (`core/src/render/texture.rs`,
|
that node's window buffer was never subsequently written, so it sat at
|
||||||
recently reworked per TEXTURES.md) if its array texture is ever
|
`(0, 0)` for the node's entire life. `shader.wgsl`'s `vs_main` divides
|
||||||
created with a single layer -- wgpu-hal's GL backend is documented to
|
by `window.dim` to reach clip space
|
||||||
guess the GL texture target from layer count at *texture* creation
|
(`let pos = (top_left + uv * size) / window.dim * 2.0 - 1.0;`), so
|
||||||
time, which can disagree with a view later requesting `D2Array`
|
every primitive's clip position came out `NaN`/`Inf` and was dropped
|
||||||
explicitly. Not chased further: this warning is GL-specific and the
|
before rasterization on **both** backends — Vulkan and GLES alike,
|
||||||
*same* blank result occurs under Vulkan too, where view dimension is
|
exactly the cross-backend symmetry that should have pointed away from
|
||||||
always explicit and this class of ambiguity should not exist, so it
|
a GL-specific cause sooner. The layout engine reporting the correct
|
||||||
may be a red herring rather than the cause. **`iris/core/src/render/`
|
widget count and pixel region the whole time is consistent with this:
|
||||||
is mid-flight in a separate benchmark branch as of this session**, so
|
that path never touches `window.dim` at all, since it is a separate
|
||||||
deliberately not touched here beyond reading it — the next session
|
copy of the window size (`UiRenderState`'s own, fed by
|
||||||
should re-check this finding against whatever lands from that branch
|
`self.render.resize`) that the CPU-side layout and hit-testing use.
|
||||||
before spending more time on it, and reach first for the simplest
|
|
||||||
possible reproduction (a single hardcoded coloured rect, no text, no
|
**The GLES `D2`/`D2Array` warning was confirmed a red herring.**
|
||||||
atlas) to separate "nothing renders" from "the atlas path specifically
|
Reproduced again after the fix, unchanged, on a build forced to
|
||||||
is broken."
|
`Backends::GL` — it fires on every frame regardless, and primitives
|
||||||
|
draw correctly on that backend anyway (screenshot below), so it is a
|
||||||
|
cosmetic wgpu-hal heuristic notice, not a correctness bug in the atlas
|
||||||
|
path. Left as-is; chasing it further is not warranted.
|
||||||
|
|
||||||
|
**Fix** (`core/src/render/mod.rs`, `UiRenderNode::new`): seed
|
||||||
|
`WindowUniform` from `config.width`/`config.height` — already the
|
||||||
|
surface's real size at construction time on both backends — instead
|
||||||
|
of `WindowUniform::default()`. This removes the dependency on an
|
||||||
|
external resize call entirely (winit's initial `Resized` event still
|
||||||
|
fires and still calls `resize()`, now idempotently) rather than
|
||||||
|
papering over android-view's missing event with one more call in the
|
||||||
|
android-specific path; a future third backend gets a correct window
|
||||||
|
buffer from its first frame with no equivalent event of its own to
|
||||||
|
remember.
|
||||||
|
|
||||||
|
**Verified on the emulator, 2026-09-05, `ai-app-2`'s own AVD, x86_64
|
||||||
|
API 26, `-feature Vulkan` + SwiftShader per the Vulkan section.**
|
||||||
|
`logcat` after launch: `render(): after update active=39
|
||||||
|
root_px=Some(PixelRegion { top_left: (0, 0), bot_right: (1080,
|
||||||
|
2219) })`, no wgpu validation warnings on the Vulkan build. Screenshot
|
||||||
|
(`/tmp/iris_i2_render.png`) shows the tabs example's coloured spans,
|
||||||
|
the red rounded rect and the tab bar all drawn — the milestone this
|
||||||
|
section asked for. Rebuilt with `Backends::GL` forced (reverted
|
||||||
|
afterwards; the shipped code still requests `Backends::PRIMARY`) and
|
||||||
|
reinstalled: same screenshot, same widgets, `AdapterInfo` logged as
|
||||||
|
`Android Emulator OpenGL ES Translator (virgl (AMD Radeon RX 7900
|
||||||
|
XT...` confirming the real GLES/virgl path, with the `D2`/`D2Array`
|
||||||
|
warning present and harmless as above. Text glyphs render with visible
|
||||||
|
artifacting on the GLES path specifically (not investigated further —
|
||||||
|
out of scope for this box, which is about primitives appearing at
|
||||||
|
all, and it does not affect the Vulkan path this app ships behind).
|
||||||
|
|
||||||
**Not built yet**: anything consuming `insets()`, a real phone
|
**Not built yet**: anything consuming `insets()`, a real phone
|
||||||
measurement (only the emulator so far — matches every other Android
|
measurement (only the emulator so far — matches every other Android
|
||||||
@@ -1149,7 +1186,9 @@ silently on real hardware.
|
|||||||
-o app/src/main/jniLibs/ build --release && gradle :app:assembleDebug`
|
-o app/src/main/jniLibs/ build --release && gradle :app:assembleDebug`
|
||||||
(release native lib per E1's segfault finding, debug Gradle variant --
|
(release native lib per E1's segfault finding, debug Gradle variant --
|
||||||
the jniLibs contents are what matters, not the Gradle build type);
|
the jniLibs contents are what matters, not the Gradle build type);
|
||||||
`adb install -r app/build/outputs/apk/debug/app-debug.apk`.
|
`adb install -r app/build/outputs/apk/debug/app-debug.apk`. Emulator
|
||||||
|
torn down after verification (`emu down`) per the machine's memory
|
||||||
|
rule.
|
||||||
- [ ] **I3 — a virtualised, bottom-anchored list.** Variable-height rows,
|
- [ ] **I3 — a virtualised, bottom-anchored list.** Variable-height rows,
|
||||||
keyed, composed only while visible, paged in both directions with a
|
keyed, composed only while visible, paged in both directions with a
|
||||||
"more" sentinel at each end, a scroll anchor that survives rows
|
"more" sentinel at each end, a scroll anchor that survives rows
|
||||||
|
|||||||
@@ -201,7 +201,21 @@ impl UiRenderNode {
|
|||||||
source: ShaderSource::Wgsl(SHAPE_SHADER.into()),
|
source: ShaderSource::Wgsl(SHAPE_SHADER.into()),
|
||||||
});
|
});
|
||||||
|
|
||||||
let window_uniform = WindowUniform::default();
|
// Seeded from the surface's own size, not `WindowUniform::default()`
|
||||||
|
// (0, 0): the vertex shader divides by `window.dim` to reach clip
|
||||||
|
// space, so a window this buffer disagrees with means every
|
||||||
|
// primitive's position is NaN/Inf and is dropped before
|
||||||
|
// rasterization -- the clear colour still reaches the screen (the
|
||||||
|
// pass runs regardless) while nothing drawn on top of it ever does.
|
||||||
|
// winit's backend gets away with the old default because winit
|
||||||
|
// fires an initial `WindowEvent::Resized` that calls `resize()`
|
||||||
|
// before the first frame; android-view has no such automatic
|
||||||
|
// event, so `AndroidRenderer::new` built a node whose window buffer
|
||||||
|
// was never corrected -- this is I2's "nothing draws" bug (RUST.md).
|
||||||
|
let window_uniform = WindowUniform {
|
||||||
|
width: config.width as f32,
|
||||||
|
height: config.height as f32,
|
||||||
|
};
|
||||||
let window_buffer = device.create_buffer_init(&BufferInitDescriptor {
|
let window_buffer = device.create_buffer_init(&BufferInitDescriptor {
|
||||||
label: Some("window"),
|
label: Some("window"),
|
||||||
contents: bytemuck::cast_slice(&[window_uniform]),
|
contents: bytemuck::cast_slice(&[window_uniform]),
|
||||||
|
|||||||
Reference in new issue
Block a user