Prune commentary and stale Rust port notes
This commit is contained in:
iris committed
1 parent
5428cd75c9
commit
25370731d0
193 files changed
+693
-16219
No files matched your search
+38
-228
@@ -1,240 +1,50 @@
|
||||
# How iris renders an unbounded number of images
|
||||
|
||||
**Built 2026-09-04**, in `iris/core` and `iris/src/default/render.rs`.
|
||||
This file is the design and the measurements behind it; the deliberation
|
||||
that produced it -- the prior-art survey, the proposal and its review --
|
||||
was deleted on 2026-09-08, having been carried out. What is kept is why
|
||||
the old approach could not stay (it is the reason the current one looks
|
||||
as it does), the numbers, and what actually landed.
|
||||
Iris cannot require Vulkan descriptor indexing. The Android Vulkan Profile
|
||||
2025, covering 80.1% of active Vulkan-capable Android devices as of October
|
||||
2025, does not require `VK_EXT_descriptor_indexing` or its bindless texture
|
||||
features ([Android Vulkan profiles](https://developer.android.com/ndk/guides/graphics/android-vulkan-profile)).
|
||||
Arm guarantees the extension only on Valhall and fifth-generation GPUs
|
||||
([Arm Vulkan guidance](https://developer.arm.com/mobile-graphics-and-gaming/vulkan-api-best-practices-on-arm-gpus)).
|
||||
|
||||
Iris (the person) asked whether iris's (the library's) approach to "draw
|
||||
however many images happen to be on screen" -- relevant here because a
|
||||
transcript can hold an unbounded number of attached screenshots -- works
|
||||
on mobile, her recollection being that it does not. It did not, and this
|
||||
is what replaced it.
|
||||
The emulator also rejects wgpu requests for `TEXTURE_BINDING_ARRAY`,
|
||||
`SAMPLED_TEXTURE_AND_STORAGE_BUFFER_ARRAY_NON_UNIFORM_INDEXING`, and
|
||||
`PARTIALLY_BOUND_BINDING_ARRAY`. Those features therefore must not enter
|
||||
Iris's required device feature set.
|
||||
|
||||
## The problem
|
||||
## Current design
|
||||
|
||||
Every texture iris ever creates — every `Image` widget
|
||||
(`iris/src/widget/image.rs`) and every glyph atlas page — gets a permanent
|
||||
slot in one array via `Textures::add` (`iris/core/src/primitive/texture.rs:65`).
|
||||
Both of iris's texture-sampling primitives (`TEXTURE` and `GLYPH`) read that
|
||||
array by index: `core/src/render/shader.wgsl:56` declares
|
||||
`var views: binding_array<texture_2d<f32>>`, sized by
|
||||
`UiLimits::default()` (`core/src/render/mod.rs:347`) at **100,000 textures,
|
||||
1,000 samplers**. Getting a device to accept that layout needs three wgpu
|
||||
features — `TEXTURE_BINDING_ARRAY`,
|
||||
`SAMPLED_TEXTURE_AND_STORAGE_BUFFER_ARRAY_NON_UNIFORM_INDEXING`,
|
||||
`PARTIALLY_BOUND_BINDING_ARRAY` — which correspond to Vulkan's
|
||||
`VK_EXT_descriptor_indexing` ("bindless"), promoted to Vulkan core at 1.2.
|
||||
Glyph atlas pages are layers of one `texture_2d_array`. `GpuTextures` doubles
|
||||
the array when it runs out of layers, copies the old layers on the GPU, and
|
||||
rebuilds every bind group that referenced the old view. Page numbers are
|
||||
assigned synchronously by `Textures::add_page` because glyph insertion needs
|
||||
the layer before the renderer processes queued texture updates.
|
||||
|
||||
A transcript with an unbounded number of image attachments is exactly the
|
||||
case that grows this array without bound: each attachment becomes its own
|
||||
`Image` widget, which takes its own permanent array slot until dropped.
|
||||
Standalone images each own a bind group and are not placed in the glyph
|
||||
array. Each render layer keeps ordinary rect/glyph instances separately from
|
||||
image instances. It draws the ordinary batch once, then binds and draws each
|
||||
standalone image. This removes any fixed image count at the cost of one bind
|
||||
and draw call per visible image, which is the appropriate tradeoff for phone
|
||||
transcripts containing a modest number of screenshots.
|
||||
|
||||
## What was measured
|
||||
The masks storage buffer appears in every image bind group. If that buffer or
|
||||
the atlas array is reallocated, all affected bind groups must be rebuilt;
|
||||
retaining a bind group across either reallocation would leave it pointing at
|
||||
the old GPU resource.
|
||||
|
||||
**A new rig, `scripts/rigs/gpu-probe`**, asks a device for exactly iris's features
|
||||
and limits with no window and no APK — a plain executable pushed with
|
||||
`adb push` and run from `/data/local/tmp`. It has two parts:
|
||||
`wgpu::Adapter::request_device` with iris's exact `Features`/`Limits`
|
||||
(`src/main.rs`), and a raw Vulkan query bypassing wgpu entirely via `ash`
|
||||
(`src/vk.rs`), to tell "the driver doesn't have it" apart from "wgpu didn't
|
||||
detect it."
|
||||
Texture updates accumulate their rebuild requirement with OR. A patch must
|
||||
never clear a rebuild requested by an earlier push in the same batch.
|
||||
|
||||
- **On this VM's own GPU** (Vulkan via Venus onto an RX 7900 XT):
|
||||
`IRIS DEVICE: ok`. Not the case that matters — nobody's phone is a
|
||||
discrete desktop GPU — but it is why the design was never checked before
|
||||
now: it always worked in the one place it was tried.
|
||||
- **On the Android emulator's guest Vulkan**, both ICDs it ships
|
||||
(`vk_swiftshader_icd.json` and, cold-booted, `lvp_icd.json`/lavapipe):
|
||||
`request_device` **fails** —
|
||||
`Unsupported features were requested: TEXTURE_BINDING_ARRAY |
|
||||
SAMPLED_TEXTURE_AND_STORAGE_BUFFER_ARRAY_NON_UNIFORM_INDEXING |
|
||||
PARTIALLY_BOUND_BINDING_ARRAY`. The raw `ash` query on lavapipe shows the
|
||||
driver itself reporting all seven descriptor-indexing sub-features as
|
||||
`true` at device API version 1.3 — so wgpu-hal's own feature detection is
|
||||
being more conservative than the driver here, for a reason not chased
|
||||
further (a likely instance-version negotiation gap, since the extension
|
||||
only promoted to core at 1.2). That part is a wgpu-hal/emulator question,
|
||||
not the finding that matters, and is **not** why this design is rejected.
|
||||
Within a render layer, images are drawn after rects and glyphs. Both primitive
|
||||
lists use `swap_remove`, so no code may infer draw adjacency from arena
|
||||
adjacency after a free.
|
||||
|
||||
**The finding that matters is about real phones, sourced rather than
|
||||
recalled:**
|
||||
Standalone images currently use `NonFiltering` sampling. Thumbnail scaling
|
||||
and filtering remain image-widget decisions, not texture-storage decisions.
|
||||
|
||||
- The **Android Vulkan Profile 2025** — Google and Khronos's current
|
||||
baseline, covering **80.1% of active Vulkan-capable Android devices** as
|
||||
of October 2025
|
||||
([developer.android.com/ndk/guides/graphics/android-vulkan-profile](https://developer.android.com/ndk/guides/graphics/android-vulkan-profile)) —
|
||||
does **not** require `VK_EXT_descriptor_indexing` or any descriptor-
|
||||
indexing feature. It requires `shaderSampledImageArrayDynamicIndexing`
|
||||
(indexing by a value uniform across the invocation — Vulkan 1.0 baseline,
|
||||
unrelated to bindless) and stops there; true of the 2021 and 2022
|
||||
profiles as well.
|
||||
- Arm's own developer documentation states **"`VK_EXT_descriptor_indexing`
|
||||
is supported on all Valhall and 5th Gen GPUs"**
|
||||
([developer.arm.com/mobile-graphics-and-gaming/vulkan-api-best-practices-on-arm-gpus](https://developer.arm.com/mobile-graphics-and-gaming/vulkan-api-best-practices-on-arm-gpus)) —
|
||||
Mali generations from roughly 2019 (Mali-G77) onward, with no claim made
|
||||
for Bifrost, Midgard or Utgard, which are still common in budget and
|
||||
older Android phones that are still in daily use.
|
||||
- A search engine's summarized claim of "1% support on Android" for this
|
||||
extension was checked against its cited source (an Arm blog post from
|
||||
2021) and **was not actually there** — that number does not appear in
|
||||
any primary source found and should not be repeated. The baseline-
|
||||
profile finding above is the one with an attributable source; use it
|
||||
instead.
|
||||
## Verification rig
|
||||
|
||||
So this is not a software-renderer artifact. A real, currently-shipping
|
||||
share of the Android fleet lacks the feature iris's texture pipeline asks
|
||||
for unconditionally, and neither the emulator's failure nor the current
|
||||
official hardware baseline gives any reason to expect that to change soon.
|
||||
|
||||
## Implemented, 2026-09-04
|
||||
|
||||
The shape above, built as proposed with one structural addition the proposal
|
||||
didn't need to spell out and one bug it predicted made moot rather than
|
||||
literally fixed. Files: `core/src/primitive/texture.rs` (`Textures`,
|
||||
`TextureHandle`), `core/src/render/texture.rs` (`GpuTextures`),
|
||||
`core/src/render/primitive.rs` (`Primitives`, `GlyphPrimitive`),
|
||||
`core/src/render/atlas.rs`, `core/src/ui/painter.rs`,
|
||||
`core/src/render/mod.rs` (`UiRenderNode`, `UiLimits` removed),
|
||||
`core/src/render/shader.wgsl`, `src/default/render.rs`, and
|
||||
`scripts/rigs/gpu-probe/src/main.rs`.
|
||||
|
||||
**1. Atlas pages as array layers.** `GpuTextures` owns one
|
||||
`texture_2d_array` (`array_texture`/`array_view`), grown by doubling
|
||||
(`grow_array`): a new texture is created at twice the layer capacity, the
|
||||
old layers are copied across with `copy_texture_to_texture` (GPU-side, no
|
||||
readback), and every bind group that referenced the old view — the main
|
||||
one and every live standalone image's — is rebuilt, since the view's
|
||||
identity changed. `GlyphPrimitive` carries `layer: u32` instead of
|
||||
`view_idx`/`sampler_idx`; the layer number is assigned synchronously in
|
||||
`Textures::add_page` (a plain counter, `next_page_layer`), not by the
|
||||
renderer, because `GlyphAtlas::insert` needs it in the same call, before
|
||||
any GPU sync happens — the renderer only finds out later, when it
|
||||
processes the queued `Push`.
|
||||
|
||||
**2. Standalone images, one bind group each.** `TextureKind` on
|
||||
`TextureHandle`/`Textures` distinguishes `Image` (a plain bind-group index,
|
||||
`slot`) from `Page { layer }`. `Primitives` gained a second per-layer list
|
||||
— `images: Vec<PrimitiveInstance>`, tagged `IMAGE_BINDING` — separate from
|
||||
`instances` (rects and glyphs), written by `Painter::write_image` rather
|
||||
than through the generic `Primitive` trait, since an image has nowhere in
|
||||
`PrimitiveData` to put a per-instance entry once the bind group already
|
||||
picks the texture. `UiRenderNode::draw` draws a layer's `instance` buffer
|
||||
once as before, then walks `image_instance` one entry at a time, binding
|
||||
that texture's `BindGroup` (`GpuTextures::image_bind_group`) and issuing
|
||||
`draw(0..4, k..k+1)` per image. Group 2's layout is exactly the proposed
|
||||
`{atlas array, one image texture, sampler, masks}`; the main draw binds a
|
||||
1x1 null view in the image slot.
|
||||
|
||||
**The one addition beyond the proposal**: the masks storage buffer lives
|
||||
in every per-image bind group (group 2, binding 3), and `ArrBuf<Mask>`
|
||||
recreates its buffer whenever the mask count changes size
|
||||
(`render/util/mod.rs`'s `ArrBuf::update` now returns whether it resized).
|
||||
A resize invalidates every bind group holding the old buffer, not just the
|
||||
main one, so `GpuTextures::update` takes a `masks_resized: bool` and calls
|
||||
`rebuild_image_bind_groups` when it's set, alongside the same rebuild the
|
||||
array-growth path already needed. This wasn't a design question the
|
||||
proposal had to answer (it treated bind-group construction as a given),
|
||||
but it's exactly the shape of trap layer growth already had, so it uses
|
||||
the same fix.
|
||||
|
||||
**3. No thumbnail atlas.** Not built, as proposed.
|
||||
|
||||
**4. Removed**: `TEXTURE_BINDING_ARRAY`, `PARTIALLY_BOUND_BINDING_ARRAY`,
|
||||
`SAMPLED_TEXTURE_AND_STORAGE_BUFFER_ARRAY_NON_UNIFORM_INDEXING` from
|
||||
`src/default/render.rs`'s `request_device`, and `UiLimits` (the type
|
||||
itself, not just its binding-array methods — once its two fields were
|
||||
gone there was nothing left in it, and `UiRenderNode::new` no longer takes
|
||||
a limits parameter). `binding_array` no longer appears anywhere in
|
||||
`shader.wgsl`.
|
||||
|
||||
**5. Sampling** is still `NonFiltering`, unchanged, per the proposal's own
|
||||
note that this is a separate decision for whenever the image widget itself
|
||||
is touched.
|
||||
|
||||
**The `changed = false` bug is structurally gone, not patched.** The old
|
||||
`GpuTextures::update` held one `changed: bool` that a `Patch` reset
|
||||
unconditionally, which could erase an earlier `Push` in the same batch (a
|
||||
new atlas page's `Push` immediately followed by `GlyphAtlas::insert`'s
|
||||
`Patch`, both queued before the renderer ever runs). The new `update`
|
||||
computes the rebuild signal by OR-ing each event's own answer
|
||||
(`rebuild_main |= self.push(...)`), and `Patch`'s arm simply never
|
||||
contributes to it — there is no shared mutable flag left for a `Patch` to
|
||||
stomp on. Documented at the call site
|
||||
(`core/src/render/texture.rs`, `GpuTextures::update`'s doc comment and the
|
||||
`Patch` match arm's comment) rather than fixed as a one-line diff, since
|
||||
the mechanism that could go wrong no longer exists.
|
||||
|
||||
**In-layer draw order is an explicit invariant now, not just a fact about
|
||||
`swap_remove`.** `UiRenderNode::draw` draws every layer's images after its
|
||||
rects and glyphs, and `Primitives::apply_free`'s doc comment states
|
||||
directly that both of a layer's lists (`instances` and `images`) free with
|
||||
`swap_remove` and that nothing may assume adjacency survives a free —
|
||||
recorded there because `apply_free` is the one place a change to either
|
||||
list's ordering would have to be reconciled.
|
||||
|
||||
**Verified:**
|
||||
|
||||
- `cargo fmt --all -- --check`, `cargo build --workspace --all-targets`,
|
||||
`cargo clippy --all-targets`, `cargo test --workspace` all clean in
|
||||
`iris/`, on the pinned `nightly-2026-09-03` toolchain. 14 tests pass
|
||||
(unchanged from I1; nothing here is pure-logic enough to add a unit
|
||||
test to — it's all GPU resource wiring).
|
||||
- `iris/run-headless.sh minimal --shot /tmp/minimal.png` and
|
||||
`iris/run-headless.sh tabs --shot /tmp/tabs.png`: both render correctly
|
||||
on this VM's GPU (Venus) — `tabs`'s glyph-atlas text renders in every
|
||||
panel, confirming `GlyphPrimitive.layer` addresses the array correctly.
|
||||
- The standalone-image path specifically: a throwaway example (not
|
||||
committed) with an `image(...)` widget as part of the root, run the same
|
||||
way, rendered the image next to glyph-atlas text in one frame —
|
||||
confirming a live `BindGroup` built by `GpuTextures::create_image` and
|
||||
bound per-`draw()` call actually samples the right texture. `tabs`'s own
|
||||
"image span" tab exercises the same widget but needs a click to reach,
|
||||
which the headless compositor can't deliver (no seat devices, per I1's
|
||||
own note on this file) — the throwaway example is what stood in for it.
|
||||
- **Exercised, 2026-09-04: `grow_array` under real load, on `tabs`.**
|
||||
Rather than building a purpose-made glyph flood, `PAGE`
|
||||
(`core/src/render/atlas.rs`) was temporarily dropped from 1024 to 64 —
|
||||
small enough that `tabs`'s ordinary mix of sizes and families (nothing
|
||||
exotic: a handful of `Text` widgets at a few sizes, one at
|
||||
`Family::Monospace`) already exceeds one page's worth of distinct
|
||||
glyphs. A one-line `eprintln!` in `grow_array` confirmed two real grows
|
||||
in a single run (`GROW_ARRAY: 1 -> 2` then `GROW_ARRAY: 2 -> 4`, i.e.
|
||||
glyphs landed on at least a third layer), and
|
||||
`iris/run-headless.sh tabs --shot` showed every tab's text rendering
|
||||
correctly with no corruption or missing glyphs — confirming the
|
||||
`copy_texture_to_texture` grow-and-relocate path and cross-layer
|
||||
sampling (`GlyphPrimitive.layer` addressing a layer beyond the first)
|
||||
both work. Command:
|
||||
`sed -i 's/PAGE: u32 = 1024/PAGE: u32 = 64/' core/src/render/atlas.rs`,
|
||||
rebuild, `./run-headless.sh tabs --shot /tmp/x.png`, then
|
||||
`git checkout -- core/src/render/atlas.rs` to revert — this is a
|
||||
throwaway diagnostic value, never a committed change, since a real
|
||||
1024px page holding only a handful of glyphs at a time would be mostly
|
||||
wasted space in normal use. Confirmed the revert left `tabs` and
|
||||
`minimal` byte-identical to the pre-check screenshots afterward.
|
||||
- **The decisive check**, `scripts/rigs/gpu-probe` rewritten to request iris's new
|
||||
(empty) feature/limit set and run on this checkout's own emulator
|
||||
(`ai-app-2`, via `emu`), booted with `EMU_GPU=software` so the guest gets
|
||||
a real Vulkan device (SwiftShader) rather than the `-gpu host` default,
|
||||
which disables Vulkan in this VM entirely (`-feature -Vulkan`, because
|
||||
gfxstream can't pair Venus with the real GPU here — worth remembering,
|
||||
since the *default* `emu up` gives a device with **no** Vulkan adapter
|
||||
at all, which reads exactly like the old bindless failure if you don't
|
||||
know to ask for `EMU_GPU=software`):
|
||||
|
||||
cd scripts/rigs/gpu-probe
|
||||
ANDROID_NDK_HOME=$HOME/Android/Sdk/ndk/29.0.14206865 \
|
||||
cargo ndk -t arm64-v8a -P 26 build --release
|
||||
EMU_GPU=software emu up # from ~/repos/emulator-tools
|
||||
adb push target/aarch64-linux-android/release/gpu-probe /data/local/tmp/
|
||||
adb shell chmod 755 /data/local/tmp/gpu-probe
|
||||
adb shell /data/local/tmp/gpu-probe
|
||||
|
||||
Output: `adapters: 1 — Vulkan SwiftShader Device (Subzero) (Cpu)`,
|
||||
`features iris requires:` (none listed — the set is empty),
|
||||
`max_buffer_size … ok`, and **`IRIS DEVICE: ok`**. This is the fix
|
||||
measured working, on the exact rig that first measured it failing.
|
||||
Emulator stopped afterward (`emu down`); nothing was left running.
|
||||
`scripts/rigs/gpu-probe` requests Iris's exact feature and limit set without a
|
||||
window. Run it on the target device when changing renderer requirements. A
|
||||
successful desktop adapter is not evidence that the same feature is available
|
||||
on Android hardware.
|
||||
Reference in new issue
Block a user