ai-app/TEXTURES.md

# How iris should render an unbounded number of images

Iris (the person) asked whether iris's (the library's) approach to
"draw however many images happen to be on screen" — relevant here because a
transcript can hold an unbounded number of attached screenshots — actually
works on mobile, her recollection being that it does not. Checked rather
than assumed, on 2026-09-04, on the `rustify` branch of `ai-app-2`. This
file is that investigation and the resulting recommendation, written for a
second agent to review before anything in iris's render core changes — no
code has been written against this yet.

## The problem

Every texture iris ever creates — every `Image` widget
(`iris/src/widget/image.rs`) and every glyph atlas page — gets a permanent
slot in one array via `Textures::add` (`iris/core/src/primitive/texture.rs:65`).
Both of iris's texture-sampling primitives (`TEXTURE` and `GLYPH`) read that
array by index: `core/src/render/shader.wgsl:56` declares
`var views: binding_array<texture_2d<f32>>`, sized by
`UiLimits::default()` (`core/src/render/mod.rs:347`) at **100,000 textures,
1,000 samplers**. Getting a device to accept that layout needs three wgpu
features — `TEXTURE_BINDING_ARRAY`,
`SAMPLED_TEXTURE_AND_STORAGE_BUFFER_ARRAY_NON_UNIFORM_INDEXING`,
`PARTIALLY_BOUND_BINDING_ARRAY` — which correspond to Vulkan's
`VK_EXT_descriptor_indexing` ("bindless"), promoted to Vulkan core at 1.2.

A transcript with an unbounded number of image attachments is exactly the
case that grows this array without bound: each attachment becomes its own
`Image` widget, which takes its own permanent array slot until dropped.

## What was measured

**A new rig, `rigs/gpu-probe`**, asks a device for exactly iris's features
and limits with no window and no APK — a plain executable pushed with
`adb push` and run from `/data/local/tmp`. It has two parts:
`wgpu::Adapter::request_device` with iris's exact `Features`/`Limits`
(`src/main.rs`), and a raw Vulkan query bypassing wgpu entirely via `ash`
(`src/vk.rs`), to tell "the driver doesn't have it" apart from "wgpu didn't
detect it."

- **On this VM's own GPU** (Vulkan via Venus onto an RX 7900 XT):
  `IRIS DEVICE: ok`. Not the case that matters — nobody's phone is a
  discrete desktop GPU — but it is why the design was never checked before
  now: it always worked in the one place it was tried.
- **On the Android emulator's guest Vulkan**, both ICDs it ships
  (`vk_swiftshader_icd.json` and, cold-booted, `lvp_icd.json`/lavapipe):
  `request_device` **fails** —
  `Unsupported features were requested: TEXTURE_BINDING_ARRAY |
  SAMPLED_TEXTURE_AND_STORAGE_BUFFER_ARRAY_NON_UNIFORM_INDEXING |
  PARTIALLY_BOUND_BINDING_ARRAY`. The raw `ash` query on lavapipe shows the
  driver itself reporting all seven descriptor-indexing sub-features as
  `true` at device API version 1.3 — so wgpu-hal's own feature detection is
  being more conservative than the driver here, for a reason not chased
  further (a likely instance-version negotiation gap, since the extension
  only promoted to core at 1.2). That part is a wgpu-hal/emulator question,
  not the finding that matters, and is **not** why this design is rejected.

**The finding that matters is about real phones, sourced rather than
recalled:**

- The **Android Vulkan Profile 2025** — Google and Khronos's current
  baseline, covering **80.1% of active Vulkan-capable Android devices** as
  of October 2025
  ([developer.android.com/ndk/guides/graphics/android-vulkan-profile](https://developer.android.com/ndk/guides/graphics/android-vulkan-profile)) —
  does **not** require `VK_EXT_descriptor_indexing` or any descriptor-
  indexing feature. It requires `shaderSampledImageArrayDynamicIndexing`
  (indexing by a value uniform across the invocation — Vulkan 1.0 baseline,
  unrelated to bindless) and stops there; true of the 2021 and 2022
  profiles as well.
- Arm's own developer documentation states **"`VK_EXT_descriptor_indexing`
  is supported on all Valhall and 5th Gen GPUs"**
  ([developer.arm.com/mobile-graphics-and-gaming/vulkan-api-best-practices-on-arm-gpus](https://developer.arm.com/mobile-graphics-and-gaming/vulkan-api-best-practices-on-arm-gpus)) —
  Mali generations from roughly 2019 (Mali-G77) onward, with no claim made
  for Bifrost, Midgard or Utgard, which are still common in budget and
  older Android phones that are still in daily use.
- A search engine's summarized claim of "1% support on Android" for this
  extension was checked against its cited source (an Arm blog post from
  2021) and **was not actually there** — that number does not appear in
  any primary source found and should not be repeated. The baseline-
  profile finding above is the one with an attributable source; use it
  instead.

So this is not a software-renderer artifact. A real, currently-shipping
share of the Android fleet lacks the feature iris's texture pipeline asks
for unconditionally, and neither the emulator's failure nor the current
official hardware baseline gives any reason to expect that to change soon.

## What growth already costs today, before any redesign

Checked directly in `core/src/render/mod.rs` and `core/src/render/texture.rs`,
because "does this redesign make things worse" needs the current baseline
first:

- The `RenderPipeline` (`UiRenderNode::new`) is created **once** and never
  rebuilt for any reason related to texture count — its bind group
  *layouts* declare fixed slot counts (`limits.max_textures`,
  `limits.max_samplers`) up front and that never changes at runtime. Growth
  was never at risk of recreating the pipeline, in the current design or
  any redesign discussed below.
- What **does** get rebuilt: `UiRenderNode::update` calls
  `self.textures.update(&mut ui.textures)`, and if that reports any change,
  rebuilds `self.rsc_group` — one `BindGroup` whose entries are
  `BindingResource::TextureViewArray(&tex_manager.views())`, collected
  fresh over **every currently-live texture**, plus the sampler array and
  the mask buffer. This happens on every texture `Push`, `Set`, or `Free`
  — an image added anywhere in the whole app rebuilds one shared structure
  referencing every other image too.
- The one path already excluded from this, on purpose, is a `Patch` —
  writing into an existing texture's pixels without changing which
  textures exist. The code says why directly
  (`core/src/render/texture.rs`, in `GpuTextures::update`): *"A patch
  changes texture contents, not the binding array, so it must not report
  `changed` — rebuilding the bind group per glyph is the cost this exists
  to avoid."* This is exactly the mechanism I1 built for the glyph atlas:
  growing an existing atlas page costs a `write_texture` into a sub-rect,
  nothing else.

So today, growth that stays inside an existing texture (glyphs added to an
atlas page) is already free. Growth that adds a *new* texture — a new atlas
page, or any standalone image — already rebuilds the one shared array
regardless of how the array is populated, before any change discussed
below. That existing cost is O(live texture count) in CPU work to collect
the view list and in however expensive the driver finds a
descriptor-set-sized-for-N-descriptors to be.

## Prior art, checked rather than assumed

Two independent projects were checked to see whether "atlas for images"
is actually how this is normally done, rather than a guess:

- **egui_wgpu** (`crates/egui-wgpu/src/renderer.rs` in emilk/egui), the
  closest prior art to iris — an immediate-mode wgpu-backed UI library that
  ships on Android. It keeps a `HashMap<TextureId, Texture>` and gives
  **each texture its own ordinary `BindGroup`** — one texture, one sampler,
  no array, no descriptor indexing of any kind. Draw calls are batched by
  texture id and the bind group is switched between batches within the
  render pass.
- **Vello** — the renderer Masonry (E1/E2's Linebender stack) draws
  through — hit the identical problem and wrote down why in their own
  roadmap document
  ([github.com/linebender/vello/blob/main/doc/roadmap_2023.md](https://github.com/linebender/vello/blob/main/doc/roadmap_2023.md)):
  *"The number of images that may appear in a scene is not bounded, which
  is not a good fit for the basic descriptor binding model... Until then,
  we'll do a workaround of having a single atlas image containing all the
  images in the scene."* Their reason is broader than Android — WebGPU 1.0
  has no descriptor indexing at all — but it reaches the same conclusion
  for the same shape of problem: atlas, not a bigger bindless array.

**This is also a live hazard, not a solved one.** Vello's own changelog
(Sparse Strips v0.2.0) lists a fix titled *"WebGL image-atlas allocation
and growth on Mali-G52 GPUs, avoiding application-not-responding errors"*
— an actual ANR, from atlas growth, on an actual mid-range Android GPU,
in the renderer Masonry is built on. The same release added
`AtlasSpaceDiagnostics`/`AtlasLayerDiagnostics` (per-layer free-space,
utilization, fragmentation) because growth needed instrumenting in
production, not because it turned out to be free.

## Recommendation (not yet implemented)

1. **Small, plentiful textures** — glyphs (already done, I1), thumbnails,
   downscaled attachment previews, icons — go through a shared atlas, the
   same technique as `core/src/render/atlas.rs` generalized beyond glyphs.
   Adding one to an existing page is a `Patch`, already free per the
   section above.
2. **Large or one-off images** — a photo attachment opened at full
   resolution, anything that would fragment a shared page — get their
   **own ordinary, non-array bind group**, the egui_wgpu way. Creating one
   is O(1): it references only itself, and does not touch any other
   texture's binding, unlike today's shared array where every push
   rebuilds a structure listing everything.
3. **Opening a new atlas page** is the one case that still resembles
   today's rebuild — infrequent (bounded by how many *pages* are needed,
   not by how many images have ever been attached) but not free, and
   Vello's Mali-G52 fix says this specifically deserves care: it should
   never be allowed to block a frame, and it is worth having the
   equivalent of Vello's atlas diagnostics before trusting it under load.
4. **Net effect**: dropping `TEXTURE_BINDING_ARRAY`,
   `SAMPLED_TEXTURE_AND_STORAGE_BUFFER_ARRAY_NON_UNIFORM_INDEXING`, and
   `PARTIALLY_BOUND_BINDING_ARRAY` from iris's device request entirely.
   Every path above is plain Vulkan 1.0 / GLES-level texture sampling.
   This is also what fixes the emulator failure measured above, regardless
   of the unresolved wgpu-hal question: a device that never asks for the
   feature cannot be refused for lacking it.

## What this touches, and what is still open

Implementing this reworks iris's rendering core: the shader's binding
group layout (`shader.wgsl`), `Textures` and `GpuTextures`
(`core/src/primitive/texture.rs`, `core/src/render/texture.rs`), both
texture-sampling primitives, and `core/src/ui/painter.rs`'s draw-call
batching (today one draw call can reference any texture by index; the
per-texture-bind-group path needs draws grouped by which bind group they
use). Nothing has been started.

Open questions a reviewer should weigh in on:

- **The size threshold** between "goes in an atlas page" and "gets its own
  bind group." Too low and ordinary attachment thumbnails end up as
  one-off bind groups, losing the batching benefit the atlas exists for;
  too high and a page fragments on a handful of medium images.
- **Eviction policy** for atlas pages once the working set does not fit —
  today's `GlyphAtlas` never evicts, because a font's glyph set is small
  and bounded; images are not. An LRU at the page level, or at the
  individual-image level within a page, has not been designed.
- **Whether iris should keep any binding array at all**, even a small
  fixed one (say, capped at a few dozen slots) for atlas pages themselves,
  or whether every atlas page should also be its own ordinary bind group
  like standalone images — the array's only remaining justification would
  be avoiding a bind-group-per-draw-call switch cost that has not been
  measured on this project's actual target hardware.
- **How this interacts with I2/E2's virtualised list** (I3): a
  bottom-anchored transcript composes only visible rows, so the live
  texture set should already be bounded by what is on screen rather than
  by the whole conversation — worth confirming that invariant holds before
  relying on it to keep atlas/bind-group churn small.