iris: replace the bindless texture array with an atlas array + per-image bind groups
The old pipeline bound every texture ever drawn (glyph atlas pages and
standalone images alike) in one binding_array<texture_2d<f32>> and asked
every device, unconditionally, for VK_EXT_descriptor_indexing -- which a
real share of Android GPUs lack and which failed outright on the Android
emulator's software Vulkan (see TEXTURES.md's "iris's binding array does
not survive real Android hardware").
Implements TEXTURES.md's "Recommended shape": the glyph atlas is now one
texture_2d_array (a layer per page, grown by doubling + GPU-side
copy_texture_to_texture); a standalone image is its own ordinary Texture
and BindGroup, drawn with its own draw() call from a separate per-layer
instance list; group 2's layout is {atlas array, one image slot, sampler,
masks}. request_device now asks for no features and no binding-array
limits at all, and UiLimits is gone.
Also fixes (by making moot) the changed=false bug the review found, where
a Patch in the same batch could cancel an earlier Push's rebuild signal,
and documents the swap_remove draw-order invariant apply_free already
relied on.
Verified: cargo fmt/build/clippy/test clean in iris/ on the pinned
nightly; minimal and tabs render correctly via run-headless.sh; a
throwaway example confirmed the standalone-image bind-group path renders;
rigs/gpu-probe, updated to the new empty feature/limit set, confirms
request_device succeeds on the ai-app-2 emulator's software Vulkan
(EMU_GPU=software) -- see TEXTURES.md's "Implemented, 2026-09-04" for the
exact command and output. RUST.md's blocking item is resolved.
Co-Authored-By: Claude Sonnet <noreply@anthropic.com>
This commit is contained in:
1 parent
1c937e2f48
commit
e0a473e090
14 files changed
+1046
-332
No files matched your search
+148
@@ -1,5 +1,16 @@
|
||||
# How iris should render an unbounded number of images
|
||||
|
||||
## Status (2026-09-04)
|
||||
|
||||
**Implemented**, on the `rustify` branch of `ai-app-2`, in `iris/core` and
|
||||
`iris/src/default/render.rs`. See "Implemented, 2026-09-04" at the bottom for
|
||||
what landed, what differs from the proposal below and why, and what was
|
||||
verified versus merely reasoned about. The short version: the binding array
|
||||
is gone, `request_device` asks for no features and no binding-array limits,
|
||||
and that is now proven on the emulator's software Vulkan
|
||||
(`rigs/gpu-probe`), not just read from the code. `RUST.md`'s blocking item
|
||||
is resolved.
|
||||
|
||||
Iris (the person) asked whether iris's (the library's) approach to
|
||||
"draw however many images happen to be on screen" — relevant here because a
|
||||
transcript can hold an unbounded number of attached screenshots — actually
|
||||
@@ -333,3 +344,140 @@ needs to know an image from a page (two kinds of handle, or a kind on
|
||||
`TextureHandle`), and `Primitives` gets a second instance list per layer.
|
||||
What it saves: the sort, the size threshold, the eviction policy, and any
|
||||
per-page bind group switch.
|
||||
|
||||
## Implemented, 2026-09-04
|
||||
|
||||
The shape above, built as proposed with one structural addition the proposal
|
||||
didn't need to spell out and one bug it predicted made moot rather than
|
||||
literally fixed. Files: `core/src/primitive/texture.rs` (`Textures`,
|
||||
`TextureHandle`), `core/src/render/texture.rs` (`GpuTextures`),
|
||||
`core/src/render/primitive.rs` (`Primitives`, `GlyphPrimitive`),
|
||||
`core/src/render/atlas.rs`, `core/src/ui/painter.rs`,
|
||||
`core/src/render/mod.rs` (`UiRenderNode`, `UiLimits` removed),
|
||||
`core/src/render/shader.wgsl`, `src/default/render.rs`, and
|
||||
`rigs/gpu-probe/src/main.rs`.
|
||||
|
||||
**1. Atlas pages as array layers.** `GpuTextures` owns one
|
||||
`texture_2d_array` (`array_texture`/`array_view`), grown by doubling
|
||||
(`grow_array`): a new texture is created at twice the layer capacity, the
|
||||
old layers are copied across with `copy_texture_to_texture` (GPU-side, no
|
||||
readback), and every bind group that referenced the old view — the main
|
||||
one and every live standalone image's — is rebuilt, since the view's
|
||||
identity changed. `GlyphPrimitive` carries `layer: u32` instead of
|
||||
`view_idx`/`sampler_idx`; the layer number is assigned synchronously in
|
||||
`Textures::add_page` (a plain counter, `next_page_layer`), not by the
|
||||
renderer, because `GlyphAtlas::insert` needs it in the same call, before
|
||||
any GPU sync happens — the renderer only finds out later, when it
|
||||
processes the queued `Push`.
|
||||
|
||||
**2. Standalone images, one bind group each.** `TextureKind` on
|
||||
`TextureHandle`/`Textures` distinguishes `Image` (a plain bind-group index,
|
||||
`slot`) from `Page { layer }`. `Primitives` gained a second per-layer list
|
||||
— `images: Vec<PrimitiveInstance>`, tagged `IMAGE_BINDING` — separate from
|
||||
`instances` (rects and glyphs), written by `Painter::write_image` rather
|
||||
than through the generic `Primitive` trait, since an image has nowhere in
|
||||
`PrimitiveData` to put a per-instance entry once the bind group already
|
||||
picks the texture. `UiRenderNode::draw` draws a layer's `instance` buffer
|
||||
once as before, then walks `image_instance` one entry at a time, binding
|
||||
that texture's `BindGroup` (`GpuTextures::image_bind_group`) and issuing
|
||||
`draw(0..4, k..k+1)` per image. Group 2's layout is exactly the proposed
|
||||
`{atlas array, one image texture, sampler, masks}`; the main draw binds a
|
||||
1x1 null view in the image slot.
|
||||
|
||||
**The one addition beyond the proposal**: the masks storage buffer lives
|
||||
in every per-image bind group (group 2, binding 3), and `ArrBuf<Mask>`
|
||||
recreates its buffer whenever the mask count changes size
|
||||
(`render/util/mod.rs`'s `ArrBuf::update` now returns whether it resized).
|
||||
A resize invalidates every bind group holding the old buffer, not just the
|
||||
main one, so `GpuTextures::update` takes a `masks_resized: bool` and calls
|
||||
`rebuild_image_bind_groups` when it's set, alongside the same rebuild the
|
||||
array-growth path already needed. This wasn't a design question the
|
||||
proposal had to answer (it treated bind-group construction as a given),
|
||||
but it's exactly the shape of trap layer growth already had, so it uses
|
||||
the same fix.
|
||||
|
||||
**3. No thumbnail atlas.** Not built, as proposed.
|
||||
|
||||
**4. Removed**: `TEXTURE_BINDING_ARRAY`, `PARTIALLY_BOUND_BINDING_ARRAY`,
|
||||
`SAMPLED_TEXTURE_AND_STORAGE_BUFFER_ARRAY_NON_UNIFORM_INDEXING` from
|
||||
`src/default/render.rs`'s `request_device`, and `UiLimits` (the type
|
||||
itself, not just its binding-array methods — once its two fields were
|
||||
gone there was nothing left in it, and `UiRenderNode::new` no longer takes
|
||||
a limits parameter). `binding_array` no longer appears anywhere in
|
||||
`shader.wgsl`.
|
||||
|
||||
**5. Sampling** is still `NonFiltering`, unchanged, per the proposal's own
|
||||
note that this is a separate decision for whenever the image widget itself
|
||||
is touched.
|
||||
|
||||
**The `changed = false` bug is structurally gone, not patched.** The old
|
||||
`GpuTextures::update` held one `changed: bool` that a `Patch` reset
|
||||
unconditionally, which could erase an earlier `Push` in the same batch (a
|
||||
new atlas page's `Push` immediately followed by `GlyphAtlas::insert`'s
|
||||
`Patch`, both queued before the renderer ever runs). The new `update`
|
||||
computes the rebuild signal by OR-ing each event's own answer
|
||||
(`rebuild_main |= self.push(...)`), and `Patch`'s arm simply never
|
||||
contributes to it — there is no shared mutable flag left for a `Patch` to
|
||||
stomp on. Documented at the call site
|
||||
(`core/src/render/texture.rs`, `GpuTextures::update`'s doc comment and the
|
||||
`Patch` match arm's comment) rather than fixed as a one-line diff, since
|
||||
the mechanism that could go wrong no longer exists.
|
||||
|
||||
**In-layer draw order is an explicit invariant now, not just a fact about
|
||||
`swap_remove`.** `UiRenderNode::draw` draws every layer's images after its
|
||||
rects and glyphs, and `Primitives::apply_free`'s doc comment states
|
||||
directly that both of a layer's lists (`instances` and `images`) free with
|
||||
`swap_remove` and that nothing may assume adjacency survives a free —
|
||||
recorded there because `apply_free` is the one place a change to either
|
||||
list's ordering would have to be reconciled.
|
||||
|
||||
**Verified:**
|
||||
|
||||
- `cargo fmt --all -- --check`, `cargo build --workspace --all-targets`,
|
||||
`cargo clippy --all-targets`, `cargo test --workspace` all clean in
|
||||
`iris/`, on the pinned `nightly-2026-09-03` toolchain. 14 tests pass
|
||||
(unchanged from I1; nothing here is pure-logic enough to add a unit
|
||||
test to — it's all GPU resource wiring).
|
||||
- `iris/run-headless.sh minimal --shot /tmp/minimal.png` and
|
||||
`iris/run-headless.sh tabs --shot /tmp/tabs.png`: both render correctly
|
||||
on this VM's GPU (Venus) — `tabs`'s glyph-atlas text renders in every
|
||||
panel, confirming `GlyphPrimitive.layer` addresses the array correctly.
|
||||
- The standalone-image path specifically: a throwaway example (not
|
||||
committed) with an `image(...)` widget as part of the root, run the same
|
||||
way, rendered the image next to glyph-atlas text in one frame —
|
||||
confirming a live `BindGroup` built by `GpuTextures::create_image` and
|
||||
bound per-`draw()` call actually samples the right texture. `tabs`'s own
|
||||
"image span" tab exercises the same widget but needs a click to reach,
|
||||
which the headless compositor can't deliver (no seat devices, per I1's
|
||||
own note on this file) — the throwaway example is what stood in for it.
|
||||
- **Not separately stress-tested**: triggering a second atlas page (the
|
||||
`grow_array` doubling-and-copy path) under a real glyph load large
|
||||
enough to fill the first 1024x1024 page. The code path was reasoned
|
||||
through and matches the existing single-page write exactly except for
|
||||
the `z` origin and the extra copy, but nobody has watched a real
|
||||
second-page grow happen on screen. Worth doing before trusting this
|
||||
under a transcript with a large or unusual glyph set (many distinct
|
||||
fonts/sizes, or a font with an unusually large character set).
|
||||
- **The decisive check**, `rigs/gpu-probe` rewritten to request iris's new
|
||||
(empty) feature/limit set and run on this checkout's own emulator
|
||||
(`ai-app-2`, via `emu`), booted with `EMU_GPU=software` so the guest gets
|
||||
a real Vulkan device (SwiftShader) rather than the `-gpu host` default,
|
||||
which disables Vulkan in this VM entirely (`-feature -Vulkan`, because
|
||||
gfxstream can't pair Venus with the real GPU here — worth remembering,
|
||||
since the *default* `emu up` gives a device with **no** Vulkan adapter
|
||||
at all, which reads exactly like the old bindless failure if you don't
|
||||
know to ask for `EMU_GPU=software`):
|
||||
|
||||
cd rigs/gpu-probe
|
||||
ANDROID_NDK_HOME=$HOME/Android/Sdk/ndk/29.0.14206865 \
|
||||
cargo ndk -t arm64-v8a -P 26 build --release
|
||||
EMU_GPU=software emu up # from ~/repos/emulator-tools
|
||||
adb push target/aarch64-linux-android/release/gpu-probe /data/local/tmp/
|
||||
adb shell chmod 755 /data/local/tmp/gpu-probe
|
||||
adb shell /data/local/tmp/gpu-probe
|
||||
|
||||
Output: `adapters: 1 — Vulkan SwiftShader Device (Subzero) (Cpu)`,
|
||||
`features iris requires:` (none listed — the set is empty),
|
||||
`max_buffer_size … ok`, and **`IRIS DEVICE: ok`**. This is the fix
|
||||
measured working, on the exact rig that first measured it failing.
|
||||
Emulator stopped afterward (`emu down`); nothing was left running.
|
||||
Reference in new issue
Block a user