RUST.md: iris's bindless texture array does not survive real Android GPUs
Iris asked whether the 'unknown number of images' approach even works on mobile. It does not, measured with a new rig (rigs/gpu-probe, no APK needed) and sourced rather than recalled: the emulator's software Vulkan refuses iris's descriptor-indexing request outright, and on real hardware the current Android Vulkan Profile baseline (80.1% of active devices) does not require VK_EXT_descriptor_indexing either -- Arm's own docs say only Valhall/5th-Gen Mali (2019+) support it. iris already solved the identical problem for text in I1 (the glyph atlas). The recommendation is to generalize it to images rather than widen the binding array further; not yet implemented, since it changes iris's render core. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
1 parent
c70a670356
commit
79b9cd789a
6 files changed
+1592
-12
No files matched your search
@@ -37,19 +37,36 @@ session spending an afternoon on them again.
|
||||
## Where things stand (2026-09-04)
|
||||
|
||||
- **Done**: E0 (toolchain), E1 (Masonry on android-view, which found the
|
||||
keyboard gap), I0a, I0b (iris builds on a pinned nightly and runs), I1
|
||||
(parley + glyph atlas).
|
||||
- **Next**: **I2** — iris on android-view. E1 says the first thing to
|
||||
answer there is why the editor gets keystrokes but no autocorrect or
|
||||
suggestions, since that is the constraint the framework decision turns
|
||||
on. **E2** (a transcript in Masonry) can go in parallel in another
|
||||
session.
|
||||
keyboard gap — now explained, see below), I0a, I0b (iris builds on a
|
||||
pinned nightly and runs), I1 (parley + glyph atlas).
|
||||
- **E1's keyboard gap is Masonry's `as_input_connection` returning `None`
|
||||
(a TODO), not android-view or `EditorInfo`.** android-view's own demo
|
||||
implements the `InputConnection` trait over a parley editor and gets
|
||||
real Gboard suggestions on this emulator — screenshotted 2026-09-04.
|
||||
android-view's `accesskit_android` adapter also has a reproducible abort
|
||||
(a client detaching, not attaching, is the trigger) — see E1 below for
|
||||
both, with the mitigation iris/I4 needs to carry.
|
||||
- **Blocking, found 2026-09-04, before I2 can be called done**: iris's
|
||||
texture pipeline asks every device, unconditionally, for
|
||||
`VK_EXT_descriptor_indexing` ("bindless" binding arrays) — and a
|
||||
sourced check says a real share of Android hardware does not have it,
|
||||
not just the emulator's software renderer. See "iris's binding array
|
||||
does not survive real Android hardware" under the iris track for the
|
||||
measurement, the sources, and the recommended fix (generalize the
|
||||
glyph atlas to images, the same way I1 already did for text). Not yet
|
||||
implemented; a design decision to confirm before iris's render core
|
||||
changes.
|
||||
- **Next**: decide on the atlas-based image fix above, then **I2** —
|
||||
iris on android-view. **E2** (a transcript in Masonry) can go in
|
||||
parallel in another session.
|
||||
- **Not started**: `client-core`, which is item 1 of the recommendation
|
||||
below and does not depend on the framework choice. Nothing has been
|
||||
built for it, and it is not one of the numbered boxes — worth picking up
|
||||
in a session that wants work independent of the emulator.
|
||||
- **The app itself is untouched.** Everything so far is in `iris/` and in
|
||||
the rigs; nothing under `app/` or `server/` has changed.
|
||||
- **The app itself is untouched.** Everything so far is in `iris/`, in
|
||||
`rigs/gpu-probe` (a headless wgpu/Vulkan feature probe, pushable to a
|
||||
device with no APK — see the binding-array section), and in the other
|
||||
rigs; nothing under `app/` or `server/` has changed.
|
||||
- **Changed outside this repo**, both in `emulator-tools` and both pushed:
|
||||
`avd_serial` now validates its cache by asking the device its AVD name
|
||||
rather than by checking the serial is still attached (a recycled port
|
||||
@@ -851,6 +868,97 @@ step measured.
|
||||
nightly gates — `portable_simd` (the old glyph compositing) and
|
||||
`gen_blocks` (the deleted line iterator). **Eleven left.**
|
||||
|
||||
### iris's binding array does not survive real Android hardware (found 2026-09-04)
|
||||
|
||||
Iris asked, of the "unknown number of images" case — a transcript with an
|
||||
unbounded number of attached screenshots — whether iris's approach even
|
||||
works on a phone, since her recollection was that mobile does not support
|
||||
it. Checked rather than assumed, and the recollection is right, with
|
||||
sources rather than a guess.
|
||||
|
||||
**What iris does today.** Every texture — every `Image` widget
|
||||
(`src/widget/image.rs`) and every glyph atlas page — gets its own
|
||||
permanent slot in one array via `Textures::add`
|
||||
(`core/src/primitive/texture.rs:65`), and both the `TEXTURE` and `GLYPH`
|
||||
primitives sample it by `view_idx` into `binding_array<texture_2d<f32>>`
|
||||
at `core/src/render/shader.wgsl:56`, sized by `UiLimits::default` — 100,000
|
||||
textures, 1,000 samplers (`core/src/render/mod.rs:347`). That needs three
|
||||
wgpu features: `TEXTURE_BINDING_ARRAY`,
|
||||
`SAMPLED_TEXTURE_AND_STORAGE_BUFFER_ARRAY_NON_UNIFORM_INDEXING`,
|
||||
`PARTIALLY_BOUND_BINDING_ARRAY` — Vulkan's `VK_EXT_descriptor_indexing`
|
||||
("bindless"), promoted to core in 1.2. So a transcript with an unbounded
|
||||
number of images is exactly the case that grows this array without bound,
|
||||
one permanent slot per image.
|
||||
|
||||
**Measured first on the emulator, and it fails outright.** A rig
|
||||
(`rigs/gpu-probe`, a plain executable with no window, pushed with `adb
|
||||
push` and run from `/data/local/tmp` — no APK needed to ask a device what
|
||||
it supports) asks `wgpu::Adapter::request_device` for exactly iris's
|
||||
features and limits. Against the emulator's guest Vulkan — both
|
||||
SwiftShader (`vk_swiftshader_icd.json`) and lavapipe (`lvp_icd.json`,
|
||||
cold-booted) — `request_device` **fails**: `Unsupported features were
|
||||
requested: TEXTURE_BINDING_ARRAY |
|
||||
SAMPLED_TEXTURE_AND_STORAGE_BUFFER_ARRAY_NON_UNIFORM_INDEXING |
|
||||
PARTIALLY_BOUND_BINDING_ARRAY`. A second, raw query through `ash`
|
||||
(`rigs/gpu-probe/src/vk.rs`, bypassing wgpu) shows lavapipe's
|
||||
`vkGetPhysicalDeviceFeatures2` actually reporting all seven descriptor-
|
||||
indexing sub-features as `true` at device api version 1.3 — so on this
|
||||
software renderer wgpu-hal's own feature detection is being more
|
||||
conservative than the driver, for a reason not chased further (a likely
|
||||
instance-version negotiation gap, since `VK_EXT_descriptor_indexing` was
|
||||
only promoted to core at 1.2 and wgpu-hal's own `Instance::init` may be
|
||||
requesting less). That part is an emulator/wgpu-hal question and not the
|
||||
finding that matters.
|
||||
|
||||
**The finding that matters is about real phones, not the emulator, and it
|
||||
is sourced rather than recalled.** The **Android Vulkan Profile 2025** —
|
||||
Google and Khronos's current baseline, covering **80.1% of active
|
||||
Vulkan-capable Android devices** as of October 2025
|
||||
([developer.android.com/ndk/guides/graphics/android-vulkan-profile](https://developer.android.com/ndk/guides/graphics/android-vulkan-profile))
|
||||
— does **not** require `VK_EXT_descriptor_indexing` or any descriptor-
|
||||
indexing feature. It requires `shaderSampledImageArrayDynamicIndexing`
|
||||
(indexing an array of samplers by a value uniform across the invocation —
|
||||
Vulkan 1.0 baseline, unrelated to bindless) and stops there; the same is
|
||||
true of the 2021 and 2022 profiles. On the hardware side, Arm's own
|
||||
developer documentation states **"`VK_EXT_descriptor_indexing` is
|
||||
supported on all Valhall and 5th Gen GPUs"**
|
||||
([developer.arm.com/mobile-graphics-and-gaming/vulkan-api-best-practices-on-arm-gpus](https://developer.arm.com/mobile-graphics-and-gaming/vulkan-api-best-practices-on-arm-gpus)) —
|
||||
Mali generations from roughly 2019 (Mali-G77) onward, named affirmatively
|
||||
with no claim made for Bifrost, Midgard or Utgard, which are still common
|
||||
in budget and older Android phones still in use. So this is not a
|
||||
software-renderer artifact: a real, currently-shipping share of the
|
||||
Android fleet lacks the feature iris's texture pipeline asks for
|
||||
unconditionally, and the newest official baseline does not promise it
|
||||
either. (A crates.io/search-engine claim of "1% support on Android" for
|
||||
this extension was checked against its cited source, an Arm blog post,
|
||||
and was not actually there — that number does not appear anywhere primary
|
||||
and should not be repeated; the 80.1%-baseline-excludes-it finding above
|
||||
is the one with an attributable source.)
|
||||
|
||||
**Recommendation, not yet implemented.** iris already solved the
|
||||
identical problem for text in I1: the glyph atlas
|
||||
(`core/src/render/atlas.rs`) packs many small rasters into a handful of
|
||||
shared 1024×1024 pages and samples them by UV offset, so **text needs
|
||||
none of the three features above** — only ordinary single-texture
|
||||
sampling. The same technique generalizes to images: route an `Image`
|
||||
widget through a shared atlas when it is small enough to pack (thumbnails,
|
||||
downscaled attachment previews, avatars, icons), and fall back to one
|
||||
ordinary, non-array texture bind group — selected per batched draw call
|
||||
the way every immediate-mode 2D renderer already does — for anything too
|
||||
large to atlas well (a photo opened at full resolution). Either path is
|
||||
plain Vulkan 1.0 / GLES texture sampling, so it removes the descriptor-
|
||||
indexing requirement from iris's device request entirely, which is also
|
||||
what would make the emulator work regardless of the wgpu-hal question
|
||||
above: a device that never asks for the feature cannot be refused for
|
||||
lacking it. This is a change to iris's rendering core — the shader's
|
||||
binding group layout, `Textures`, the texture and glyph primitives, and
|
||||
`ui/painter.rs` — so it is written here as a recommendation rather than
|
||||
started, per the project's rule to confirm a load-bearing design change
|
||||
before making it. **It should be resolved before I2 is called done**,
|
||||
since I2's pass condition is the phone, not just the emulator, and this
|
||||
is exactly the kind of thing that passes on a desktop GPU and fails
|
||||
silently on real hardware.
|
||||
|
||||
- [ ] **I2 — iris on android-view.** An `android-view` surface as a second
|
||||
backend beside winit: `wgpu` on the view's surface (GLES here, see
|
||||
the Vulkan section; Vulkan on the phone), touch as pointer events,
|
||||
@@ -895,9 +1003,12 @@ re-derived:
|
||||
5. Run the existing rigs rather than inventing new ones: `ui-sandbox.sh`
|
||||
for a server with fixtures, `transcript-bench.sh` for the scroll
|
||||
baseline, `ui-trace` for anything positional, `emu up` for the
|
||||
emulator, and `iris/run-headless.sh EXAMPLE --shot PNG` for an iris
|
||||
example on this displayless machine. The Vulkan section below says how
|
||||
to get a Vulkan path in the emulator when a `wgpu` backend needs one.
|
||||
emulator, `iris/run-headless.sh EXAMPLE --shot PNG` for an iris
|
||||
example on this displayless machine, and `rigs/gpu-probe` to ask a
|
||||
device (this VM, the emulator, or a real phone over `adb push`) what
|
||||
`wgpu` features and limits it actually has before building anything on
|
||||
the assumption it does. The Vulkan section below says how to get a
|
||||
Vulkan path in the emulator when a `wgpu` backend needs one.
|
||||
6. **Bound anything heavy at the moment you start it.** An emulator or a
|
||||
long build gets a deadline — `timeout`, or a watchdog scoped to the pid
|
||||
you just started — rather than a plan to stop it later. Scope it to
|
||||
|
||||
Reference in new issue
Block a user