irisandClaude Sonnet
6317685d1a
The Gradle side of RUST.md's I2: MainActivity, IrisView (extending
android-view's RustView with the two native methods it has no hook
for -- window insets, and unregistering this view's entry in
iris::android::insets's side table), and RustView.java/
RustInputConnection.java vendored from android-view (no published AAR
to depend on) with one deliberate diff noted in a comment: mViewPeer
is protected rather than package-private, so a subclass in a different
package can reach it.
Measured on the emulator (x86_64, API 26, SwiftShader Vulkan):
dumpsys input_method shows the served InputConnection is ours, and
Gboard's suggestion strip reads real buffer content back through
text_before_cursor ("hi | Hi | HI" after typing "hi") -- the same bar
E1 set, met. Not met: nothing draws. The clear colour reaches the
screen (confirmed by swapping it to magenta) and the layout engine
reports the correct widget count and pixel regions (log::debug! calls
left in view.rs's render() for exactly this), but no primitive shows
up, on both Vulkan/SwiftShader and GLES/virgl. Root cause not found;
one unconfirmed lead (a GLES-only D2/D2Array warning that could point
at the glyph atlas) is written up in RUST.md's I2 rather than chased
into core/src/render/, which is mid-flight in a separate benchmark
branch this session.
I2 is therefore built and wired but not tickable -- RUST.md has the
full writeup, what was ruled out, and where to pick this up.
Co-Authored-By: Claude Sonnet <noreply@anthropic.com>
1291 lines
78 KiB
Markdown
1291 lines
78 KiB
Markdown
# Moving the app to Rust
|
||
|
||
Working document for the question Iris asked on 2026-09-04: what are the
|
||
options for switching the phone app to Rust, ideally pure Rust with one UI
|
||
framework shared with a future winit-based desktop application, at full
|
||
feature parity and without giving up anything native, performance
|
||
especially. Constraints she set: no Dioxus and nothing that draws through a
|
||
WebView; **no UI DSL** (which rules out Makepad and Slint); the result
|
||
should stay lightweight; platform-specific pieces are fine to maintain;
|
||
reimplementing a framework piece from scratch where it does not fit is
|
||
fine; effort and elapsed time do not matter, long-term robustness does;
|
||
this clone is where things get tried before anything is committed to
|
||
`ai-app`. Her own library, [iris](https://github.com/cat16/iris), is the
|
||
**in-house framework to be built up** for this, with Masonry as the
|
||
yardstick it is measured against.
|
||
|
||
Decisions get a date and a reason here, the way `PLAN.md` does. A section
|
||
describes something that has been tried only where it says so, with a date.
|
||
|
||
## Keep this file current as you work
|
||
|
||
**This file is the handoff, and it is meant to let a session be cleared.**
|
||
Write each result into it *as you get it*, not at the end: the box ticked
|
||
or the reason it could not be, the measurement with its number, the
|
||
decision with its date and what it rejected, and anything that cost time to
|
||
find out. Then a session that has filled its context can be cleared and the
|
||
next one can pick up from this file alone, which is much cheaper than
|
||
carrying a long conversation or re-deriving what was already measured.
|
||
|
||
Two things that follow. Write for somebody who was not here — name the
|
||
command, the file and the number rather than "the fix" or "the earlier
|
||
run". And write the failures and the dead ends too: "Venus is blocked by
|
||
the emulator, not by Mesa" and "the present mode was not the cause" are
|
||
worth as much as the successes, because they are what stops the next
|
||
session spending an afternoon on them again.
|
||
|
||
## Where things stand (2026-09-04)
|
||
|
||
- **Done**: E0 (toolchain), E1 (Masonry on android-view, which found the
|
||
keyboard gap — now explained, see below), I0a, I0b (iris builds on a
|
||
pinned nightly and runs), I1 (parley + glyph atlas).
|
||
- **Done, 2026-09-04: the `Widget::draw`/layout redesign (LAYOUT.md).**
|
||
`desired_width`/`desired_height`/`SizeCtx`/`Cache` are gone; every widget
|
||
in `iris/src/widget/` implements one `fn draw(&mut self, &mut Painter) ->
|
||
Size`. A moved widget (`Scroll`, `Offset`) now costs one
|
||
`move_offsets` write resolved by a shared `resolve_move` WGSL function in
|
||
both shader stages, independent of how many primitives are in its
|
||
subtree — measured at 500 in `iris/src/layout_tests.rs`, which also
|
||
covers the unchanged-frame, hit-test-after-move and mask-follows-move
|
||
pass conditions as plain unit tests (no GPU or window needed, since
|
||
`UiRenderState` touches neither). All four examples render
|
||
pixel-identically to before the change. See LAYOUT.md's "Deviations
|
||
found during implementation" for five real bugs the design's first draft
|
||
did not anticipate — worth reading before touching `Aligned`, `Sized`,
|
||
`MaxSize`, `Scroll`, or the move-slot lifecycle again. `GpuTextures::grow_array`
|
||
(a second atlas layer opening) has now been exercised too, on `tabs` with
|
||
`PAGE` temporarily lowered — see TEXTURES.md's "Exercised, 2026-09-04".
|
||
Not done: a pixel-level screenshot check of a `Masked`-wrapped `Scroll`
|
||
(no example builds one yet — the numeric check in `layout_tests.rs`
|
||
stands in).
|
||
- **E1's keyboard gap is Masonry's `as_input_connection` returning `None`
|
||
(a TODO), not android-view or `EditorInfo`.** android-view's own demo
|
||
implements the `InputConnection` trait over a parley editor and gets
|
||
real Gboard suggestions on this emulator — screenshotted 2026-09-04.
|
||
android-view's `accesskit_android` adapter also has a reproducible abort
|
||
(a client detaching, not attaching, is the trigger) — see E1 below for
|
||
both, with the mitigation iris/I4 needs to carry.
|
||
- **Resolved, 2026-09-04: iris's binding array does not survive real
|
||
Android hardware.** iris's texture pipeline used to ask every device,
|
||
unconditionally, for `VK_EXT_descriptor_indexing` ("bindless" binding
|
||
arrays), which a real share of Android hardware lacks. It has been
|
||
rebuilt per TEXTURES.md's "Recommended shape": the glyph atlas is one
|
||
`texture_2d_array` (a layer per page), a standalone image is its own
|
||
ordinary `Texture`/`BindGroup`, and `request_device` now asks for no
|
||
features and no binding-array limits at all. `rigs/gpu-probe`, rewritten
|
||
to match, confirms `request_device` now succeeds on the emulator's
|
||
software Vulkan (`EMU_GPU=software`, SwiftShader) — see TEXTURES.md's
|
||
"Implemented, 2026-09-04" for the exact command and output, and for what
|
||
was verified (rendering, via `run-headless.sh`) versus what was reasoned
|
||
through but not separately stress-tested (a real second-atlas-page
|
||
grow under load). Nothing here has been run on real Android hardware
|
||
yet, only the emulator; the Android Vulkan Profile 2025 sourcing in
|
||
"iris's binding array does not survive real Android hardware" below is
|
||
what stands in for that until I2 gets a device.
|
||
- **I2 — iris on android-view: built 2026-09-05, not tickable.** The
|
||
android-view backend (`iris/src/android/`), the `iris-android-app`
|
||
cdylib and Gradle shell, insets, the back gesture, and the full
|
||
`InputConnection` bridge are all in and measured working — Gboard's
|
||
suggestion strip reads real buffer content through it, the same bar E1
|
||
set. What is not working: **nothing draws**. The screen shows only the
|
||
clear colour on both Vulkan/SwiftShader and GLES/virgl, though the
|
||
layout engine reports the correct widget count and pixel regions —
|
||
see I2's own entry below for what was ruled out and the one open lead
|
||
(a GLES-only `D2`/`D2Array` warning from the glyph atlas, unconfirmed
|
||
as the cause, and deliberately not chased into `core/src/render/`
|
||
while that tree is mid-flight in a separate benchmark branch this
|
||
session). **Next**: root-cause the blank render — start with a single
|
||
hardcoded rect and no text, to separate "nothing renders" from "the
|
||
atlas path specifically is broken" — then revisit I2's tick. **E2** (a
|
||
transcript in Masonry) can go in parallel in another session.
|
||
- **`client-core` built (2026-09-04)**, item 1 of the recommendation:
|
||
`event-model/` (the event types, now shared with `server/`) and
|
||
`client-core/` (REST and SSE clients, transcript fold, cache, highlighter,
|
||
ANSI parser, 85 ported tests). `CLIENT_CORE.md` maps Kotlin file to Rust
|
||
module and lists what is not yet covered. `./run-tests.sh` runs all three
|
||
crates.
|
||
- **The app itself is untouched.** Everything so far is in `iris/`, in
|
||
`rigs/gpu-probe` (a headless wgpu/Vulkan feature probe, pushable to a
|
||
device with no APK — see the binding-array section), and in the other
|
||
rigs; nothing under `app/` or `server/` has changed.
|
||
- **Changed outside this repo**, both in `emulator-tools` and both pushed:
|
||
`avd_serial` now validates its cache by asking the device its AVD name
|
||
rather than by checking the serial is still attached (a recycled port
|
||
silently pointed this checkout at another session's emulator), and
|
||
`EMU_GPU=software` was added as an opt-in that keeps a run off the host
|
||
GPU and gives the guest a software Vulkan device. The default is
|
||
unchanged, because `-gpu host` was measured and the Compose benchmarks
|
||
depend on it.
|
||
|
||
## What has to be reproduced
|
||
|
||
The app is ~19,000 lines of Kotlin. It splits three ways, and the split is
|
||
what decides how much of a port is mechanical.
|
||
|
||
**Pure logic with no Compose or Android in it, ~4,500 lines.** `Api.kt`
|
||
(1,142), `Events.kt`, `EventStream.kt`, `Sse.kt`, `TranscriptCache.kt`
|
||
(589, touches `java.io.File` only), `TranscriptSource.kt`,
|
||
`MarkdownSyntax.kt`, `Languages.kt`, `Highlighter.kt`, `Ansi.kt`,
|
||
`ResetCountdown.kt`, `Durations.kt`, `Sizes.kt`, `ModelName.kt`,
|
||
`LoadState.kt`, `ImportableStream.kt`. `TranscriptUnits.kt` and
|
||
`TranscriptItems.kt` (the event fold into rows, ~940 lines) are logic with
|
||
a handful of Compose annotations. This is also exactly the code that has
|
||
JVM unit tests today. All of it ports directly, and most of it already has a
|
||
Rust twin in `server/`: `Events.kt` is a hand-kept mirror of
|
||
`session/driver.rs`'s enum, the highlighter and the syntax scanner exist on
|
||
the server for the explorer, and the cache compares the server's own JSON
|
||
lines. **Sharing these types between server and app is the single largest
|
||
"keep things in sync" win available, and it does not depend on which UI
|
||
framework wins.**
|
||
|
||
**Compose UI, ~13,000 lines.** Screens, dialogs, the transcript list, the
|
||
markdown renderer's customisations, tool cards, the file explorer viewer and
|
||
editor. This is the part a UI framework choice is about.
|
||
|
||
**Android platform code, ~1,500 lines**, spread over 20 files. Every one of
|
||
these is a Java-side object that no Rust framework can replace, because
|
||
Android only offers them as Java classes:
|
||
|
||
- `NotificationService` — a **foreground service** holding the
|
||
`/notifications` SSE stream while the app is closed, with its ongoing
|
||
notification, `specialUse` type and the `POST_NOTIFICATIONS` request.
|
||
- `MainActivity` — edge-to-edge, the `ACCESS_LOCAL_NETWORK` runtime
|
||
permission (Android 17), `singleTop` intent routing for `aiapp://enroll`,
|
||
notification taps, and the **share sheet** (`ACTION_SEND`, any MIME type).
|
||
- `ServerConfig` — the bearer token sealed under an **Android Keystore**
|
||
AES-GCM key, shared with Dev Updater through `wg-app-link`'s `:link`.
|
||
- `EnrollmentScanActivity` — the in-app **QR scanner** (zxing, camera).
|
||
- `Attachments` — `ContentResolver` reads of shared URIs, `BitmapFactory`
|
||
decode and downscale, **EXIF** orientation.
|
||
- `SessionImage` — bitmap decode for produced images.
|
||
- `ScrollAnchor`, `Drafts` — `SharedPreferences`; `CrashLog` — `filesDir`.
|
||
- `TranscriptCache` — `cacheDir`.
|
||
- `DebugStats`/`FrameStats` — `Choreographer` frame timing and the render
|
||
report; `runtime-tracing` names composables in a system trace.
|
||
|
||
So **"pure Rust" on Android means Rust owns every line of logic and
|
||
drawing, behind a thin shell of Java stubs**, and a packaging step that
|
||
produces a signed APK. How thin, and whether Gradle is inevitable, are
|
||
answered below.
|
||
|
||
### How much Java is unavoidable, and why
|
||
|
||
Rust can *call* any Android API through JNI (`jni` crate, with
|
||
`ndk-context` handing over the `JavaVM` and the Activity): posting a
|
||
notification, `startForegroundService`, the Keystore, `ContentResolver`
|
||
reads, permission requests, `WindowInsets`, the clipboard. None of that
|
||
needs a line of Kotlin. What JNI cannot do is *define* a class that the
|
||
system instantiates **by name from the manifest** — an `Activity`, a
|
||
`Service`, an `Application`, a `BroadcastReceiver`. Those must exist as dex
|
||
bytecode inside the APK before any Rust runs, because the framework
|
||
constructs them and only then calls into native code. `NativeActivity` is
|
||
the platform's own stub for the Activity case; there is no
|
||
`NativeService`, and android-view ships its own `View` subclass for the
|
||
same reason.
|
||
|
||
So the floor is roughly **two Java classes of ten lines each**: an
|
||
`Activity` and a `Service` whose lifecycle methods are declared `native`
|
||
and registered from `JNI_OnLoad`, plus whatever android-view already
|
||
provides. Everything they would have done in Kotlin — insets, intent
|
||
routing, the SSE follow loop, the notification builder — is Rust reached
|
||
through those stubs. Writing the stubs in Java rather than Kotlin drops
|
||
`kotlinc` from the toolchain; `javac` comes with the JDK Gradle already
|
||
needs. Generating the dex from Rust is not worth it: there is no mature
|
||
Rust dex writer, and the stubs never change.
|
||
|
||
### Can the APK be built without Gradle?
|
||
|
||
Yes. An APK is a zip containing a binary-XML `AndroidManifest.xml`,
|
||
`resources.arsc`, `classes.dex`, `lib/<abi>/*.so` and assets, aligned and
|
||
signed with the v2 scheme. The tools are `aapt2` (manifest and resources),
|
||
`d8` (Java bytecode to dex), `zipalign` and `apksigner`, all in the SDK's
|
||
`build-tools`, none of them Gradle. Three ways to drive them:
|
||
|
||
- **A `cargo xtask`** (or `build.rs`-adjacent script) that runs `cargo ndk`
|
||
for each ABI, `javac` + `d8` for the stubs, `aapt2 link`, `zipalign`,
|
||
`apksigner`. About 150 lines, every step visible, no AGP, no Gradle
|
||
daemon holding 2.8 GB between builds. The pinned-CA constant becomes a
|
||
`build.rs` reading the same `certs/ca.pem` path.
|
||
- **[cargo-apk2](https://github.com/mzdk100/cargo-apk2)**: the maintained
|
||
successor to cargo-apk, and unlike it compiles `java_sources` /
|
||
`kotlin_sources` into the dex and declares multiple activities **and
|
||
services** with intent filters from `[package.metadata.android]`, with
|
||
per-profile keystores and optional `aapt2`. Exactly the shape needed;
|
||
the question is whether a third-party tool with one maintainer beats
|
||
150 lines we own.
|
||
- **cargo-apk / xbuild**: unmaintained and `NativeActivity`-only. No.
|
||
|
||
What Gradle would take with it: Android Lint (which found two real bugs
|
||
here, but in Kotlin that would no longer exist — with forty lines of Java
|
||
stubs there is little left for it to find), manifest merging, R8, and the
|
||
generated-source plumbing. What it gives back: one toolchain, `cargo`
|
||
end to end, and Dev Updater keeps calling `build-apk.sh` exactly as now.
|
||
**Recommendation: the xtask**, with cargo-apk2 read for the details it
|
||
already got right (v2 signing, `uses-feature`, ABI splits).
|
||
|
||
### The behaviours that are hard to get back
|
||
|
||
Reading the Compose code for what a replacement must be able to express,
|
||
rather than what it happens to look like:
|
||
|
||
1. **The transcript is one selectable body of text.** One
|
||
`SelectionContainer` around the whole lazy list, so a selection runs from
|
||
a reply into the tool output beneath it. The framework needs selectable
|
||
read-only rich text across many rows, with the platform's selection
|
||
handles and clipboard on the phone.
|
||
2. **Rich inline text**: markdown with links (one tap detector per text,
|
||
not a node per link), inline code chips drawn behind the text, tables
|
||
with wrapping cells and a sideways scroll, syntax-highlighted fences,
|
||
ANSI colour in tool output, Nerd Font icon glyphs. Needs a text layout
|
||
engine with spans, not just styled labels.
|
||
3. **A bottom-anchored virtualised list of variable-height rows**, paged in
|
||
both directions (800-event pages, `HISTORY_SCREENS` measured in
|
||
viewports), with a saved scroll anchor per session, "hold the edge
|
||
nearest the tap" when a row expands (`holdTopEdge`, done in the layout
|
||
pass so the wrong frame is never drawn), and rows keyed so that a run of
|
||
tool calls stays one row while it grows.
|
||
4. **The soft keyboard**: the composer resizes with the IME, the guard
|
||
against a stuck inset animation, drafts per session, autocorrect and
|
||
suggestions from the phone's own keyboard. This is where most Rust
|
||
frameworks fail on Android today; see below.
|
||
5. **Platform integration through the app model**: foreground service,
|
||
notifications, share sheet, deep link, Keystore, camera, back gesture,
|
||
edge-to-edge insets, local-network permission.
|
||
6. **Accessibility names on icon buttons**, which the bench scripts depend
|
||
on (`ui-trace` taps by label). A framework with no accessibility tree
|
||
also breaks the measuring rig.
|
||
7. **Measurable frames**: the debug render report, and a way to attribute
|
||
a frame's cost to a widget on the real phone.
|
||
|
||
## The two constraints that decide it
|
||
|
||
**1. Android text input.** Every framework built on `winit` inherits
|
||
winit's Android backend, and that backend cannot drive the soft keyboard
|
||
properly: the IME tracking issues
|
||
([#1823](https://github.com/rust-windowing/winit/issues/1823),
|
||
[#2766](https://github.com/rust-windowing/winit/issues/2766)) are open,
|
||
`ReceivedCharacter` is unimplemented on Android
|
||
([#2305](https://github.com/rust-windowing/winit/issues/2305)), and the
|
||
`android-activity` groundwork for editor actions only merged in February
|
||
2026 ([PR #214](https://github.com/rust-mobile/android-activity/pull/214))
|
||
with the winit half still to come. Composition, autocorrect and suggestions
|
||
need an `InputConnection` implemented on the Java side, which winit's
|
||
`NativeActivity`/`GameActivity` model does not offer. The frameworks that
|
||
type on Android today each wrote their own Java glue (Slint, Makepad), and
|
||
the one designed to do it the way Android intends is
|
||
[`android-view`](https://github.com/rust-mobile/android-view): a Rust
|
||
implementation of an Android `View`, with text input through
|
||
`InputConnection`, accessibility, touch, callbacks on the UI thread, usable
|
||
either as a whole app or embedded beside ordinary Android components. It is
|
||
marked WIP. Both Linebender (its Masonry demo lives in that repo) and
|
||
Robius/Makepad ([Robrix's release notes](https://github.com/project-robius/robrix/releases)
|
||
say Android lacks a "full" keyboard and they are integrating android-view
|
||
for it) are converging on it. **That makes android-view the phone-side
|
||
foundation whichever widget set sits on top**, and the first thing to
|
||
build and measure here.
|
||
|
||
**2. Rich, selectable text and a virtualised list.** Frameworks group by
|
||
their text stack:
|
||
|
||
- **Parley + Fontique + Vello** (Linebender): rich spans, selection and
|
||
editing utilities, IME support driven through `ui-events`, AccessKit text
|
||
properties ([Linebender 2026 Q1](https://linebender.org/blog/tmil-25/),
|
||
[parley](https://github.com/linebender/parley)). Used by Masonry/Xilem,
|
||
and by Blitz. Vello proper needs compute shaders; `vello_hybrid` (CPU
|
||
path processing, GPU compositing) is "roughly beta" and runs on GLES too,
|
||
and Vello CPU exists as a no-GPU fallback.
|
||
- **cosmic-text** (iced, egui optionally): good layout, but the widgets on
|
||
top decide selection. iced's `markdown` widget is not selectable
|
||
([discourse](https://discourse.iced.rs/t/markdown-widgets-text-should-be-selectable/1107)).
|
||
- **Slint's own**: `TextInput` with `read-only` is the selectable-text
|
||
trick; there is **no inline rich text at all** (issue
|
||
[#1325](https://github.com/slint-ui/slint/issues/1325), markdown request
|
||
[#6684](https://github.com/slint-ui/slint/issues/6684) both open). A
|
||
markdown transcript with links and code chips cannot be drawn.
|
||
- **Makepad's own**: GPU/SDF text, a `Markdown` widget and a virtualised
|
||
`PortalList` in `makepad-widgets`.
|
||
|
||
## Options
|
||
|
||
### A. Keep Compose, move the logic into a Rust core (uniffi)
|
||
|
||
A `client-core` crate (events shared with the server, API client, SSE,
|
||
transcript fold, cache, markdown model, highlighter, ANSI) exposed to
|
||
Kotlin through [uniffi](https://github.com/mozilla/uniffi-rs). Compose keeps
|
||
drawing. Desktop would be a second UI (iced or Compose Desktop) over the
|
||
same core.
|
||
|
||
- **For**: the logic and the wire types stop drifting from the server
|
||
today, with tests in one language. Incremental and always shippable.
|
||
- **Against**: it is not what was asked for. The 13,000 lines of UI stay
|
||
Kotlin, the desktop app shares no UI code, and the `:link` Kotlin module
|
||
stays. uniffi's Kotlin Multiplatform bindings are a
|
||
[community fork](https://github.com/UbiqueInnovation/uniffi-kotlin-multiplatform-bindings);
|
||
the Android-only bindings are Mozilla's and solid.
|
||
- **Verdict**: not the destination, but **step one of every other option**
|
||
is building this crate, so it costs nothing to keep it as the fallback.
|
||
|
||
### B. Slint
|
||
|
||
Rust on Android is officially supported (minSdk 26, `android-activity`
|
||
backend, own Java IME glue, safe areas and keyboard insets since 1.15,
|
||
Skia renderer needs `clang`). Royalty-free licence requires disclosing
|
||
Slint use; GPLv3 otherwise. UI is a separate `.slint` DSL, not Rust.
|
||
|
||
- **Against**: no rich inline text (see above), so the transcript cannot be
|
||
drawn as it is today; the UI language is not Rust, which forfeits the
|
||
"compiler catches it" motivation for the half of the code that is UI.
|
||
- **Verdict**: rejected on rich text alone.
|
||
|
||
### C. iced
|
||
|
||
Elm-style, Rust-only widgets, desktop-first, `winit` + `wgpu`. Has a
|
||
`markdown` widget and `rich_text` with links. The maintainer states mobile
|
||
is a non-goal ([iced](https://github.com/iced-rs/iced)); a community
|
||
Android example exists and its author could not get the soft keyboard
|
||
working, patched widgets for touch, and notes no accessibility
|
||
([HN thread](https://news.ycombinator.com/item?id=46350641)). Markdown is
|
||
not selectable; `scrollable` is not virtualised.
|
||
|
||
- **Verdict**: a fine desktop toolkit and the one Iris named, but every
|
||
phone-side gap (IME, touch, accessibility, selection, virtualisation)
|
||
would be ours to build and maintain against a project that does not want
|
||
them. Not the shared framework.
|
||
|
||
### D. egui
|
||
|
||
Immediate mode, `winit`-based on Android, AccessKit integration,
|
||
selectable labels across a `Ui`. Repaints only on input by default, so
|
||
battery is not the immediate-mode worry. Android IME is blocked on winit
|
||
([discussion](https://github.com/emilk/egui/discussions/2053)); the
|
||
workaround is an in-app virtual keyboard, which is exactly the
|
||
non-native keyboard to avoid. Variable-height virtualised lists are manual
|
||
(`show_rows` assumes uniform heights). Looks like egui, not Material.
|
||
|
||
- **Verdict**: workable on desktop, wrong on the phone for the same reason
|
||
as iced, plus a look that would need a full custom style.
|
||
|
||
### E. Makepad
|
||
|
||
GPU-rendered, hybrid retained/immediate, `live_design!` DSL with hot
|
||
reload, MIT, 1.0 in 2025 ([makepad](https://github.com/makepad/makepad)).
|
||
Ships Android apps today with its own Java glue; Robrix (a Matrix chat
|
||
client, the closest analogue to this app) is its reference application on
|
||
Android, iOS and desktop. Has `Markdown`, `PortalList` (virtualised),
|
||
`TextInput`. Robrix reports the Android keyboard is not "full" and is moving
|
||
to android-view for it; the README says non-standard targets "may require
|
||
minor fixes".
|
||
|
||
- **For**: the only option that already ships a chat-shaped app on Android
|
||
and desktop from one codebase, with the widgets this app needs.
|
||
- **Against**: the DSL is its own language with its own shader-based
|
||
styling, so a large part of the UI would not be checked by rustc; the
|
||
rendering model (SDF everything) is a different world from Compose's,
|
||
and selection across a `Markdown` widget is unverified.
|
||
- **Verdict**: **rejected 2026-09-04** — Iris does not want a DSL. Kept
|
||
here so its Android keyboard status stays a data point about
|
||
android-view, not as an option.
|
||
|
||
### F. Masonry / Xilem on android-view (Linebender)
|
||
|
||
Retained widget tree (Masonry) with a reactive view layer (Xilem) that
|
||
reads like Compose; Rust all the way down; Vello, Parley, Fontique,
|
||
AccessKit, `ui-events`. Widgets include `Prose` (selectable read-only rich
|
||
text), `TextArea`, `VirtualScroll`, and this year `Svg`, `Split`,
|
||
`CollapsePanel`, a new layout system, and IME through `ui-events`
|
||
independent of winit. `masonry_android_view` exists in the android-view
|
||
repo and is "not yet generally usable"; Xilem calls itself experimental.
|
||
Desktop runs on winit. Vello needs a compute-capable GPU or falls back to
|
||
`vello_hybrid`/CPU.
|
||
|
||
- **For**: the only stack where every hard behaviour above maps onto a
|
||
component designed for it: selection and rich text (Parley/Prose),
|
||
virtualised variable heights (`VirtualScroll`), native IME
|
||
(android-view's `InputConnection`), accessibility (AccessKit, now with an
|
||
Android crate), one Rust widget language on both platforms. The team is
|
||
the one writing the Android integration everyone else is adopting.
|
||
- **Against**: pre-1.0 with API churn each release; a small team; no
|
||
Material widget set, so every control's look is ours; some of the pieces
|
||
(`masonry_android_view`, `vello_hybrid`) are explicitly unfinished. Being
|
||
early means fixing things upstream ourselves, which Iris said is
|
||
acceptable.
|
||
- **Verdict**: **the option to try first**, because it is the only one
|
||
whose gaps are "not finished yet" rather than "not designed for this".
|
||
|
||
### G. iris — the in-house library, and what "from scratch" means here
|
||
|
||
[cat16/iris](https://github.com/cat16/iris), read 2026-09-04 from the one
|
||
public commit (2026-01-31, "portfolio copy"; ~8,700 lines in `core`,
|
||
`macro` and the crate itself). Retained-mode widgets stored outside the
|
||
render tree, `wgpu` 28 directly, `winit` 0.30, `cosmic-text` 0.16 (parley
|
||
since I1), a
|
||
relative-anchor-plus-offset layout with `rest()` and `rel()` lengths, a
|
||
postfix builder API (`rect(..).radius(30).on(CursorSense::click(), ..)
|
||
.sized(..).align(..)`), events handled where the widget is declared, and
|
||
a single-threaded context passed explicitly — all of which reads like this
|
||
codebase's own rules. There is text editing (`widget/text/edit.rs`),
|
||
images, masks, spans and stacks; the TODO names text resizing as
|
||
per-frame slow and scaling as unsolved. It requires **nightly** (fourteen
|
||
`#![feature]` gates as vendored, among them `const_trait_impl`,
|
||
`unboxed_closures`, `portable_simd`, `associated_type_defaults`; eleven
|
||
after I0b and I1 — see those steps for the current list). Desktop only; no
|
||
Android surface, no IME, no accessibility tree, no virtualised list, no
|
||
rich-text selection.
|
||
|
||
**That list is the work, and some of it is done.** As of 2026-09-04 it
|
||
builds on a pinned nightly, runs on this machine's GPU, has parley and a
|
||
glyph atlas, and `iris-core` cross-compiles to Android. What it still
|
||
lacks from the list above is the Android surface, the IME bridge, the
|
||
accessibility tree and the virtualised list — I2, I3 and I4.
|
||
|
||
**iris is not a candidate to be tested as it stands. It is the in-house
|
||
library** (Iris, 2026-09-04): "essentially a good start to a rewrite from
|
||
scratch", to be maintained and extended by the sessions working here.
|
||
So the list above of what it lacks is a **work list, not a score**. When
|
||
the app needs something iris does not have, the answer is to build it
|
||
into iris. The layer iris has is the widget and layout layer; the layers
|
||
it needs are the same ones Masonry gets from android-view, Parley and
|
||
AccessKit, and there is no reason iris cannot sit on those same
|
||
foundations rather than reinvent them — the surface, the keyboard bridge
|
||
and the accessibility tree are platform plumbing, not a framework's
|
||
identity. The text stack was the first real design decision in that work,
|
||
and it is settled: **Parley, with a glyph atlas** (I1, 2026-09-04).
|
||
|
||
Two things to carry into that work honestly. Nightly is the opposite of
|
||
"holds up long term": a build that breaks on a toolchain update, on the
|
||
machine Dev Updater builds on, unattended. **Done in I0b** —
|
||
`iris/rust-toolchain.toml` pins `nightly-2026-09-03` — and the gate list
|
||
lives with I0b and I1, to be retired as they stabilise or are designed
|
||
around; it is down from fourteen to eleven. And a one-person framework
|
||
carries every gap itself, which is what Iris said she is willing to do.
|
||
|
||
"From scratch" therefore means iris, not a fourth thing. Masonry stays in
|
||
the plan as the **yardstick and the fallback**: building its demo and its
|
||
version of the transcript screen first says what a finished stack costs
|
||
on this hardware, proves android-view before iris depends on it, and
|
||
gives a comparison that is measured rather than remembered.
|
||
|
||
Not considered further: **GPUI** (Zed) mobile is a community fork that
|
||
depends on unpublished crates; **Dioxus/Blitz** is excluded by Iris (its
|
||
native renderer is Parley/Vello under HTML semantics, and the earlier
|
||
`tdep-survey/app-dioxus` spike parked it on a `vello_hybrid` stroke bug and
|
||
shipped the WebView); **Compose Multiplatform Desktop** would give a desktop
|
||
app for nothing but in Kotlin, which is the opposite direction.
|
||
|
||
### Weight and debug builds
|
||
|
||
Iris remembers the Linebender stack being slow in debug. What is behind
|
||
that is the dependency graph — Vello, wgpu, Parley, Fontique, Skrifa —
|
||
running unoptimised on the CPU side (path encoding, shaping), not the
|
||
widget layer. Xilem's own advice is only `split-debuginfo = "unpacked"` to
|
||
keep `target/` small; the fix everyone with this shape of dependency tree
|
||
uses is to optimise dependencies while leaving the app crate at `opt-level
|
||
= 0`:
|
||
|
||
[profile.dev.package."*"]
|
||
opt-level = 2
|
||
|
||
**Measured 2026-09-04, and it is not the widget layer.** iris's own
|
||
graph (wgpu + winit + cosmic-text at the time) built cold in **43s** with
|
||
a 2.1 GB `target/`, and **1m46s** with a 1.5 GB `target/` under the
|
||
profile above — so the knob costs build time and saves disk here, and
|
||
plain debug was never the problem. Masonry's graph is the one with Vello,
|
||
Parley, Fontique and Skrifa in it, and E1 gives the number that matters
|
||
for it: **`libmain.so` is 181 MB in debug and 11 MB in release.** That
|
||
size is also a correctness issue rather than only a weight one — a debug
|
||
build labels its Vulkan objects, and the emulator's driver segfaults in
|
||
`SetDebugUtilsObjectNameEXT` when it does. Runtime cost of the profile
|
||
knob is still unmeasured; resident memory and the frame cost of an
|
||
800-event page want E2. Vello proper needs compute shaders and carries a
|
||
large shader set; `vello_hybrid` is lighter and Masonry can now render
|
||
through either (or Vello CPU) via its `imaging` abstraction, so "keep it
|
||
light" has a knob inside the same stack.
|
||
|
||
## Recommendation
|
||
|
||
1. **Build `client-core` now, whatever the framework** (done 2026-09-04, see `CLIENT_CORE.md`). A Rust crate holding
|
||
the event model (shared with `server/` as one crate, ending the
|
||
`Events.kt` mirror), the API and SSE clients, the transcript fold, the
|
||
cache, the markdown block model, the highlighter and the ANSI parser,
|
||
with the existing JVM tests ported. It is the part of the app that is
|
||
already tested, already logic, and already duplicated on the server.
|
||
2. **One foundation, two widget layers.** The platform plumbing is shared
|
||
whichever way the decision goes: android-view for the Android surface,
|
||
keyboard and accessibility bridge; `wgpu` for the GPU; AccessKit for
|
||
names; winit on the desktop. On top of it, **Masonry as the yardstick**
|
||
(E1, E2) and **iris as the thing being built** (I0–I5), both aimed at
|
||
the same transcript screen with the same pass conditions.
|
||
3. **Decide when the transcript screen exists in both**, from the
|
||
measurements, and record the decision here with the numbers. If iris
|
||
carries the screen within the Compose baseline, it is the app's
|
||
framework and Masonry was the calibration. If it does not, the
|
||
measurement says which parts of Masonry to adopt underneath it.
|
||
4. Then the shell (E3), the desktop window (E4) and the packaging (E5),
|
||
which do not depend on the choice.
|
||
|
||
## Experiments, in order
|
||
|
||
Each has a pass condition that is a measurement in this clone. The rig
|
||
matters: this emulator runs `-gpu host` with **host Vulkan switched off**
|
||
(`GPU_HOST_FEATURES` in `emulator-tools`, a gfxstream/Venus gap), so inside
|
||
the guest a `wgpu` app gets GLES, not Vulkan; the earlier Dioxus spike also
|
||
needed `WGPU_GLES_MINOR_VERSION=1` for compute shaders and found `wgpu`'s
|
||
Android backend wants API 26 (a libc symbol). The real phone has Vulkan.
|
||
Per the standing rule, a rig limit is something to fix before it is
|
||
accepted.
|
||
|
||
- [x] **E0 — toolchain (done 2026-09-04).** Installed under the
|
||
user-owned SDK: **NDK r29 (`29.0.14206865`)**, 2.4 GB at
|
||
`~/Android/Sdk/ndk/29.0.14206865`, the newest stable — r30 is still
|
||
at rc.3. **cargo-ndk 4.1.2**. Verified by cross-compiling a scratch
|
||
`cdylib` to both ABIs: `file` reports "for Android 26, built by NDK
|
||
r29 (14206865)" for `aarch64-linux-android` and
|
||
`x86_64-linux-android`. Two things to know at the call site.
|
||
**cargo-ndk 4's API-level flag is `-P`, not `-p`** — `-p` is now
|
||
passed through to cargo as `--package`, so the old
|
||
`cargo ndk -t arm64-v8a -p 26` panics with `unknown package: 26`
|
||
*and dumps the whole environment to stdout* as a bug report, which is
|
||
worth not doing in a log somebody might paste. And the Android
|
||
targets were installed for **stable** only; the pinned nightly needs
|
||
its own, which `iris/rust-toolchain.toml` now declares.
|
||
- [x] **E1 — android-view's Masonry demo on this emulator (2026-09-04).**
|
||
It builds, renders on the GPU through Vulkan, exposes its
|
||
accessibility tree, and **the phone's own keyboard types into its
|
||
editor** — but with **no autocorrect and no suggestions**. Ticked
|
||
because everything it was meant to establish is established,
|
||
including the one gap; that gap is now E2's problem and I2's.
|
||
|
||
*Build.* `~/src/android-view` at `bec6c62`, x86_64 rather than the
|
||
README's arm64 because that is what this emulator is:
|
||
`cargo ndk -t x86_64 -P 26 -o masonry-app/src/main/jniLibs/ build -p
|
||
android-view-masonry-demo --release`, then
|
||
`./gradlew :masonry-app:assembleDebug`. **`libmain.so` is 181 MB in
|
||
debug and 11 MB in release** — the loudest single number about
|
||
Vello's dependency graph, and the reason the release build matters
|
||
for more than speed.
|
||
|
||
*Renderer.* wgpu takes **Vulkan**, and the emulator log confirms it
|
||
from the other side: `Created VkDevice ... for application:'wgpu'`.
|
||
Two things were needed. The emulator must be given Vulkan at all —
|
||
`-feature Vulkan` with `VK_DRIVER_FILES` pointing at the SDK's
|
||
`vk_swiftshader_icd.json`, **plus `-no-snapshot-load`**, which is the
|
||
piece this file had flagged as untested: without a cold boot the
|
||
guest keeps the snapshot's old GPU config and `cmd gpu vkjson`
|
||
reports zero devices however the host is configured. And the native
|
||
library must be **release**: a debug build calls
|
||
`SetDebugUtilsObjectNameEXT` to label its image views, and the
|
||
emulator's own guest driver (`vulkan.ranchu.so`) segfaults inside it.
|
||
On GLES, with no Vulkan available, it instead fails
|
||
`Surface::configure` with "Invalid surface" — untriaged, since the
|
||
Vulkan path works and Vello wants compute shaders anyway.
|
||
|
||
*Accessibility works*, which E2's condition 6 and every bench script
|
||
depend on. `ui-trace` reads Masonry's AccessKit tree: "Add task"
|
||
arrives as a named `Button`, the editor as an `EditText` node. So
|
||
tap-by-name works against a Masonry screen for any control carrying
|
||
a name; the demo's editor carries none, which is the demo's omission
|
||
rather than the framework's.
|
||
|
||
*The keyboard: real input yes, suggestions no.* Tapping the editor
|
||
opens the actual soft keyboard (`mInputShown=true`, Gboard), and
|
||
tapping its keys types into Masonry — "teh" typed key by key, with a
|
||
caret. What does **not** appear is Gboard's suggestion strip. The
|
||
control is what makes that a finding rather than an impression: the
|
||
**same three key taps in the Settings app's search field, on the same
|
||
device in the same session, produce "teh | the | yeh"**. So the strip
|
||
works here and android-view's editor is not asking for it — most
|
||
likely the `EditorInfo` its `InputConnection` reports. That matches
|
||
Robrix's report that the Android keyboard is not yet "full", and it
|
||
is the single most important thing to fix or fund upstream, because
|
||
composition, autocorrect and suggestions are exactly what the
|
||
composer in this app needs and exactly what `winit` cannot do at all.
|
||
|
||
**Cause found 2026-09-04, and it is Masonry's, not android-view's.**
|
||
`~/src/android-view/masonry/src/lib.rs:531` is
|
||
|
||
fn as_input_connection(&mut self) -> Option<&mut dyn InputConnection> {
|
||
// TODO
|
||
None
|
||
}
|
||
|
||
so the Masonry demo has **no `InputConnection` at all**; `RustView`
|
||
returns null from `onCreateInputConnection` and the IME falls back to
|
||
dispatching raw key events, which is exactly the behaviour observed —
|
||
keys arrive, composition does not exist, so there is nothing for
|
||
Gboard to suggest against. It is not a wrong `EditorInfo`, and the
|
||
guess above that it was is withdrawn.
|
||
|
||
android-view's **own** demo (`demo/src/lib.rs`, packaged by `app/`)
|
||
implements the whole trait against a parley editor and asks for
|
||
`INPUT_TYPE_CLASS_TEXT | CAP_SENTENCES | AUTO_CORRECT | MULTI_LINE`
|
||
with `IME_FLAG_NO_FULLSCREEN | NO_EXTRACT_UI | NO_ENTER_ACTION`
|
||
(`demo/src/lib.rs:588`). So the capability is present in the layer
|
||
iris would sit on, and the 30-odd method `InputConnection` trait in
|
||
`src/ime.rs` — `set_composing_text`, `set_composing_region`,
|
||
`finish_composing_text`, `text_before_cursor`, `cursor_caps_mode`,
|
||
`request_cursor_updates`, and `InputMethodManager::update_selection`
|
||
to push the selection back — is the full surface an IME needs.
|
||
**This changes what I2 costs**: the IME bridge is a trait to
|
||
implement over iris's parley editor, not a gap to fund upstream. It
|
||
also means E2 inherits Masonry's TODO, so a Masonry transcript will
|
||
have the same dead composer until somebody fills that in.
|
||
|
||
**Measured on the emulator, same session, same device.** Built
|
||
android-view's own demo — `cargo ndk -t x86_64 -P 26 -o
|
||
app/src/main/jniLibs/ build -p android-view-demo --release`, then
|
||
`./gradlew :app:assembleDebug`, installed with `ANDROID_SERIAL=$(emu
|
||
serial)` — and tapped into its editor. `dumpsys input_method` reports
|
||
`mInputShown=true` with `mServedView=…viewdemo.DemoView`, and the
|
||
screenshot shows **Gboard's suggestion strip populated with "dolor |
|
||
Dolores | door"**: the caret had landed inside the word *dolor* in
|
||
the demo's lorem ipsum, and Gboard read that word out of the Rust
|
||
editor through `text_before_cursor`. So on this emulator, through
|
||
android-view, a parley editor gets a real IME with real suggestions
|
||
drawn from its own buffer. That is the bar E1 could not reach and the
|
||
bar I2 is written against, and it is now known to be reachable.
|
||
|
||
*One crash seen once — reproduced and diagnosed 2026-09-04.* With an
|
||
accessibility client attached the app aborted, stack:
|
||
`android_view::view::do_frame` → `CallbackCtx::finish` →
|
||
`accesskit_android::event::QueuedEvents::raise` →
|
||
`send_completed_event` → `unwrap()` on `Err(JavaException)`.
|
||
android-view builds `panic = "abort"`, so a JNI call that throws
|
||
takes the process. Two later `ui-trace record` runs left the app
|
||
alive, so the trigger looked narrower than "a client is attached".
|
||
|
||
**It is the opposite of "a client is attached": it is a client
|
||
having *detached*.** `accesskit_android`'s `State` enum
|
||
(`adapter.rs:161` in 0.4.0, `:192` in 0.8.0) is
|
||
`Inactive | Placeholder | Active`, and **nothing ever moves it back
|
||
to `Inactive`**. A client — `ui-trace`, which is uiautomator — calls
|
||
into the node provider once, `get_or_init_tree` promotes the adapter
|
||
to `Active`, and it stays there for the life of the process. Every
|
||
later change then returns `Some(QueuedEvents)`, `raise` calls
|
||
`ViewParent.requestSendAccessibilityEvent`, and that reaches
|
||
`AccessibilityManager.sendAccessibilityEvent`, which on the main
|
||
looper **throws `IllegalStateException("Accessibility off. Did you
|
||
forget to check that?")` when accessibility is disabled**. jni-rs
|
||
returns `Err(JavaException)`, `send_completed_event` unwraps it, and
|
||
`panic = "abort"` ends the process.
|
||
|
||
*The controlled run*, one process (pid 4085), `settings get secure
|
||
accessibility_enabled` = 0 throughout:
|
||
|
||
- tapped the editor and typed three keys with `adb shell input tap`,
|
||
no client ever attached — **alive**;
|
||
- one `ui-trace record -d 800` with no gesture at all, then two
|
||
seconds' wait — **still alive** (the queue was raised while the
|
||
client was still there);
|
||
- the very next three keystrokes, same process — **aborted**, same
|
||
stack.
|
||
|
||
So the failure is not the recording; it is the **first thing that
|
||
changes the accessibility tree after a recording ends**. That makes
|
||
it a standing hazard for this project rather than an oddity:
|
||
`transcript-bench.sh`, `stream-bench.sh` and `bench-lib.sh`'s
|
||
tap-by-name all attach and detach uiautomator, so on a Rust app the
|
||
typing or scrolling *after* a bench run is what dies, several
|
||
seconds away from anything that looks like a cause.
|
||
|
||
**Still present at head**: 0.8.0 is the newest `accesskit_android`
|
||
(the demo resolves 0.4.0) and both the unconditional `unwrap` in
|
||
`send_completed_event` and the one-way `State` are unchanged there,
|
||
so upgrading is not the fix. **Our mitigation for I2/I4 is a gate we
|
||
own**: ask `AccessibilityManager.isEnabled()` before calling
|
||
`raise`, and drop the events when it says no. Worth reporting
|
||
upstream as well — the honest fix is for `raise` to clear a pending
|
||
exception rather than unwrap it, since a view can be detached or
|
||
accessibility switched off between queueing and raising no matter
|
||
who is calling.
|
||
|
||
*A rig trap that cost a wrong conclusion.* Several bounded runs were
|
||
given `sleep N; emu down` watchdogs, and one armed for an earlier
|
||
experiment fired in the middle of a later one — the app vanished, adb
|
||
hung, and it read exactly like the Vulkan path crashing. It was not.
|
||
A watchdog must be scoped to the process it guards (`kill $pid`, with
|
||
the pid captured at launch) rather than to whatever AVD is running
|
||
when it wakes, and only one should be armed at a time.
|
||
|
||
- [ ] **E2 — a transcript in Masonry.** One screen: open a sandbox session,
|
||
page 800 events into `VirtualScroll` bottom-anchored, draw markdown
|
||
from `pulldown-cmark` into Parley rich text with links and code
|
||
chips, select across two rows with the platform handles, expand a
|
||
tool row holding its top edge. Pass: the render numbers land within
|
||
the Compose baseline in `transcript-bench.sh` on the GPU emulator
|
||
(same gestures, same session), and every one of the seven behaviours
|
||
above is either shown or has a written reason it cannot be.
|
||
- [ ] **E3 — the shell.** Kotlin `MainActivity` + `NotificationService`
|
||
+ Keystore + share intent calling into Rust over JNI, with the SSE
|
||
follow loop in Rust. Pass: a notification arrives with the app
|
||
closed, and a share lands in a session.
|
||
- [ ] **E4 — the same screen on the desktop** in a winit window, from the
|
||
same crate, with only the layout differing.
|
||
- [ ] **E5 — the packaging xtask**: `cargo ndk` → `javac`/`d8` → `aapt2`
|
||
→ `zipalign` → `apksigner`, signed with the existing release key,
|
||
installed through Dev Updater. Pass: the APK installs over the
|
||
Gradle-built one and the notification service starts.
|
||
|
||
### The iris track
|
||
|
||
These build iris up to carry the app. Each is a feature added to iris
|
||
with a pass condition, in dependency order. Work in `iris/` in this
|
||
repository on the `rustify` branch, and record in this file what each
|
||
step measured.
|
||
|
||
- [x] **I0a — where iris lives (decided 2026-09-04).** For now it is
|
||
**vendored at `iris/` in this repository**, history not carried,
|
||
and consumed by path. Iris's decision: keep it close while it is
|
||
being reshaped for this app, and give it back its own repository —
|
||
`iris/iris` on the gitea remote, which already holds the full
|
||
244-commit history, on a branch of its own — once it has proved
|
||
itself. The vendored tree is that repository's `main` at
|
||
`7b54aaf` ("readme", 2026-01-29), byte-identical to the public
|
||
GitHub copy, so a later reconciliation has a known base. A crate
|
||
that uses it says `iris = { path = "../iris" }`.
|
||
- [x] **I0b — make it build here (done 2026-09-04).** iris now builds,
|
||
clippy-clean and rustfmt-clean at the defaults, on a pinned dated
|
||
nightly, and the `tabs` example draws on this VM's GPU.
|
||
|
||
**The pin** is `nightly-2026-09-03` (rustc 1.100.0-nightly,
|
||
`2e2b193f8`), declared in `iris/rust-toolchain.toml` along with the
|
||
`clippy`/`rustfmt` components and the two Android targets, so a
|
||
fresh clone provisions itself. It is dated rather than `nightly`
|
||
because the whole failure below was a rolling channel moving under
|
||
an unattended build. Installed with `--profile minimal`: 912 MB.
|
||
|
||
**The 36 errors were one syntax change, and the earlier diagnosis in
|
||
this file was wrong.** It is not that a trait must now be declared
|
||
`const trait` — the vendored tree already declares them that way,
|
||
which is how it was written in January. What changed is the *impl*
|
||
keyword order: `impl const Trait for T` is now
|
||
`const impl Trait for T`, and generics go on the `impl`
|
||
(`const impl<T: [const] Foo> Bar for T`). Bounds are unaffected;
|
||
`T: const Foo`, `T: [const] Foo` and `impl const Foo` in argument
|
||
position all still compile. Everything else — the unresolved
|
||
`UiVec2`/`Vec2`/`impl_op` imports, and a `Color<u8>` that resolved
|
||
to `wgpu_types::Color` — cascaded from the seven files that failed
|
||
to parse. The rewrite was mechanical across 20 sites and took the
|
||
workspace from 36 errors to 0.
|
||
|
||
**`#![feature]` gates, 12 after this step** (two were declared and
|
||
unused, and were removed: `map_try_insert`, `const_cmp`).
|
||
Load-bearing and worth watching: `const_trait_impl`, `const_ops`,
|
||
`const_convert`, `const_destruct` are the const-traits family and
|
||
the one that has already broken once — they move together, so
|
||
advancing the pin means re-reading this section. `unboxed_closures`
|
||
+ `fn_traits` (postfix builder API) and `unsize` +
|
||
`coerce_unsized` (widget handles) are pairs. The rest are
|
||
individually small: `macro_metavar_expr_concat`, `portable_simd`,
|
||
`associated_type_defaults`, `option_into_flat_iter`, and `gen_blocks`
|
||
in the top crate.
|
||
|
||
**Running it headless.** `iris/run-headless.sh EXAMPLE [--shot PNG]`
|
||
with `iris/headless.conf`, the same trick `emu` uses: a headless
|
||
sway, and `grim` for the picture. It deliberately starts its *own*
|
||
compositor rather than joining `emu`'s — sway tiles, so adding a
|
||
window to the one an emulator sits in resizes that emulator.
|
||
Unlike `emu`'s it disables Xwayland, since winit speaks Wayland.
|
||
**This VM has a real GPU for this**: Vulkan 1.4 through Venus onto
|
||
the host's RX 7900 XT, and GL 4.6 through virgl — so desktop wgpu
|
||
work here is not software-rasterised, unlike inside the emulator.
|
||
|
||
**iris has no tests at all** (`cargo test --workspace`: 0 passed
|
||
across 6 targets). Nothing to keep passing, and nothing to catch a
|
||
regression — worth knowing before I1 changes the text stack.
|
||
|
||
**`iris-core` no longer depends on winit, and now cross-compiles to
|
||
Android.** It wanted exactly one thing from it — `PhysicalSize<u32>`
|
||
in `UiRenderNode::resize`'s signature, for two numbers it immediately
|
||
turned into floats — and that pulled a whole windowing backend into
|
||
the layer below it, the wrong direction. `resize` takes
|
||
`impl Into<Vec2>` now, like `UiRenderState::resize` beside it already
|
||
did. The consequence is the point: with winit in the graph an Android
|
||
build of the core failed in `android-activity` (which needs a backend
|
||
feature nothing here selects), and without it
|
||
`cargo ndk -t arm64-v8a -P 26 build -p iris-core` finishes in 30s and
|
||
produces an rlib, wgpu's Android backend included. So **iris's
|
||
widget, layout and render core already builds for the phone**, and
|
||
what I2 has to supply is the surface, the input and the IME — not a
|
||
port of the library.
|
||
|
||
**Build weight, cold, on this VM's 8 cores** (`rm -rf target`, then
|
||
`cargo build --example tabs`), since "the Linebender stack is slow in
|
||
debug" was the worry behind this question: plain debug **43s** and a
|
||
2.1 GB `target/`; with the `[profile.dev.package."*"] opt-level = 2`
|
||
knob, **1m46s** and 1.5 GB. So iris's own wgpu + winit + cosmic-text
|
||
graph is not the slow thing — which makes it a calibration for E1
|
||
rather than an answer about Masonry, whose graph adds Vello, Parley,
|
||
Fontique and Skrifa. Runtime cost of the knob was not measured here.
|
||
|
||
**Fixed: iris never called `pre_present_notify`.** The symptom was
|
||
that about one start in five kept the window's 800x600 startup layout
|
||
on a 1920x1200 surface for good. What settled it was tracing iris's
|
||
own decisions into memory and dumping them from another thread —
|
||
`eprintln!` in the draw path makes the defect vanish, which is why
|
||
earlier attempts kept losing it. The traces from a good and a bad run
|
||
are **byte-identical**: both lay out and draw `redraw_all at
|
||
(1920, 1200)` into a 1920x1200 texture with `suboptimal=false`. iris
|
||
was drawing the right frame every time; the compositor was still
|
||
showing the first one, and forcing a full repaint did not shift it.
|
||
What was missing is winit's `Window::pre_present_notify`, called
|
||
immediately before `present`, which on Wayland is what ties the
|
||
commit to the surface's frame callback. Without it a frame drawn with
|
||
nothing following it can sit unpresented with nothing left to flush
|
||
it — which is exactly a window that has just settled after its
|
||
opening resize. Measured: **0 bad in 40** with the fix, against 4 in
|
||
20 before it, and — the stronger evidence — 0 in 20 in the
|
||
instrumented configuration that had been 15 in 20. Runtime resizing
|
||
still round-trips to a byte-identical layout.
|
||
|
||
Two things ruled out on the way, both worth not re-trying: the
|
||
present mode (the fault survived the move from `AutoNoVsync` to
|
||
`AutoVsync` at the same rate) and the size cache (`redraw_all` clears
|
||
it). A `desired_maximum_frame_latency` of 1 moved the rate without
|
||
fixing it, and was reverted. Iris's own note that she had never seen
|
||
the library fail to resize was the useful steer: it pointed away from
|
||
the layout code, where two hours had already gone.
|
||
|
||
One thing was fixed on the way, and it is not that bug: `update`
|
||
redrew everything when `resized` was set, but `needs_redraw` — which
|
||
is what decides whether to *ask* for a frame — did not know about
|
||
`resized` at all. The two now share one `needs_redraw_all`, since a
|
||
condition in one and not the other is a frame nobody requests. It is
|
||
latent on Wayland only because winit asks for a redraw after a resize
|
||
by itself; on Android, where the surface work of I2 will not have
|
||
winit underneath it, nothing else here would have asked.
|
||
- [x] **I1 — parley, and a glyph atlas (done 2026-09-04).** No bake-off:
|
||
Iris decided for parley directly ("I wanted to switch it to parley
|
||
anyways"), and then asked for the atlas as well ("just do the atlas,
|
||
commit to it, we do want it"). Both are in.
|
||
|
||
**What parley bought, beyond shaping.** Its editing model addresses
|
||
text by byte offset into one string, where cosmic-text used
|
||
`(line, index)` — so `select_content`, `delete_between`,
|
||
`insert_inner` and `newline` collapse into ordinary string
|
||
operations. Bigger: `Selection::geometry` and `Cursor::geometry`
|
||
replace `iter_layout_lines`, `index_x` and `cursor_pos`, which walked
|
||
runs by hand to place the caret and the selection boxes and were not
|
||
bidi- or wrap-correct. `edit.rs` lost about 130 lines and gained
|
||
Home/End. Its cursor motions map onto parley's `next_visual`,
|
||
`previous_visual_word`, `next_line` and so on, in one function.
|
||
|
||
**The atlas is what "text resizing (per frame) is really slow"
|
||
was.** Every string used to be rasterised into its own `RgbaImage`
|
||
and uploaded as a whole texture whenever anything about it changed,
|
||
so a window resize re-rasterised and re-uploaded every visible
|
||
string. Now a glyph is rasterised once per font, size and subpixel
|
||
phase, shared by every string that contains it, and a resize
|
||
re-emits quads without touching the GPU's copy. **The tabs example
|
||
reports it: `views`, the number of texture views bound, went from 6
|
||
to 1** — six per-string textures became one shared page. Supporting
|
||
pieces: a `GLYPH` primitive that samples a sub-rectangle and tints
|
||
it (the existing texture primitive samples a whole texture), a
|
||
`Patch` texture update so a new glyph costs its own bytes rather
|
||
than a 4 MB page, and `GpuTextures` keeping its `Texture`s, since a
|
||
view cannot be written through.
|
||
|
||
**Not yet measured**, and the honest gap in this step: the TODO's
|
||
"really slow" was never given a number, so neither is the
|
||
improvement. What is evidence rather than argument is the view count
|
||
and the shape of the work — a resize no longer rasterises. A
|
||
before/after timing wants the transcript screen of I5 to be worth
|
||
taking.
|
||
|
||
**Two bugs found on the way**, both pre-existing: `primitives!`'s
|
||
`@count` rule recursed comma-separated while matching
|
||
space-separated, so it terminated only for exactly two primitives
|
||
and adding a third hit the recursion limit; and `Color` had no
|
||
`Default`, which parley's `Brush` requires.
|
||
|
||
**Fourteen tests**, iris's first. The editor is the one part that is
|
||
pure logic rather than something needing a GPU and a window, and it
|
||
was rewritten wholesale with no way to exercise it — synthetic input
|
||
does not reach a client under the headless compositor, which has no
|
||
seat devices. Two of the tests are aimed at what the rewrite could
|
||
plausibly have broken: the IME preedit path, and editing multi-byte
|
||
text now that offsets are bytes.
|
||
|
||
Dropping cosmic-text and unicode-segmentation also retired two
|
||
nightly gates — `portable_simd` (the old glyph compositing) and
|
||
`gen_blocks` (the deleted line iterator). **Eleven left.**
|
||
|
||
### iris's binding array does not survive real Android hardware (found 2026-09-04, resolved 2026-09-04)
|
||
|
||
**Resolved the same day**: see "Where things stand" above and
|
||
TEXTURES.md's "Implemented, 2026-09-04". The measurement and sourcing
|
||
below are unchanged and are why the fix looks the way it does; nothing
|
||
here needs re-checking on its own account.
|
||
|
||
Iris asked, of the "unknown number of images" case — a transcript with an
|
||
unbounded number of attached screenshots — whether iris's approach even
|
||
works on a phone, since her recollection was that mobile does not support
|
||
it. Checked rather than assumed, and the recollection is right, with
|
||
sources rather than a guess.
|
||
|
||
**What iris does today.** Every texture — every `Image` widget
|
||
(`src/widget/image.rs`) and every glyph atlas page — gets its own
|
||
permanent slot in one array via `Textures::add`
|
||
(`core/src/primitive/texture.rs:65`), and both the `TEXTURE` and `GLYPH`
|
||
primitives sample it by `view_idx` into `binding_array<texture_2d<f32>>`
|
||
at `core/src/render/shader.wgsl:56`, sized by `UiLimits::default` — 100,000
|
||
textures, 1,000 samplers (`core/src/render/mod.rs:347`). That needs three
|
||
wgpu features: `TEXTURE_BINDING_ARRAY`,
|
||
`SAMPLED_TEXTURE_AND_STORAGE_BUFFER_ARRAY_NON_UNIFORM_INDEXING`,
|
||
`PARTIALLY_BOUND_BINDING_ARRAY` — Vulkan's `VK_EXT_descriptor_indexing`
|
||
("bindless"), promoted to core in 1.2. So a transcript with an unbounded
|
||
number of images is exactly the case that grows this array without bound,
|
||
one permanent slot per image.
|
||
|
||
**Measured first on the emulator, and it fails outright.** A rig
|
||
(`rigs/gpu-probe`, a plain executable with no window, pushed with `adb
|
||
push` and run from `/data/local/tmp` — no APK needed to ask a device what
|
||
it supports) asks `wgpu::Adapter::request_device` for exactly iris's
|
||
features and limits. Against the emulator's guest Vulkan — both
|
||
SwiftShader (`vk_swiftshader_icd.json`) and lavapipe (`lvp_icd.json`,
|
||
cold-booted) — `request_device` **fails**: `Unsupported features were
|
||
requested: TEXTURE_BINDING_ARRAY |
|
||
SAMPLED_TEXTURE_AND_STORAGE_BUFFER_ARRAY_NON_UNIFORM_INDEXING |
|
||
PARTIALLY_BOUND_BINDING_ARRAY`. A second, raw query through `ash`
|
||
(`rigs/gpu-probe/src/vk.rs`, bypassing wgpu) shows lavapipe's
|
||
`vkGetPhysicalDeviceFeatures2` actually reporting all seven descriptor-
|
||
indexing sub-features as `true` at device api version 1.3 — so on this
|
||
software renderer wgpu-hal's own feature detection is being more
|
||
conservative than the driver, for a reason not chased further (a likely
|
||
instance-version negotiation gap, since `VK_EXT_descriptor_indexing` was
|
||
only promoted to core at 1.2 and wgpu-hal's own `Instance::init` may be
|
||
requesting less). That part is an emulator/wgpu-hal question and not the
|
||
finding that matters.
|
||
|
||
**The finding that matters is about real phones, not the emulator, and it
|
||
is sourced rather than recalled.** The **Android Vulkan Profile 2025** —
|
||
Google and Khronos's current baseline, covering **80.1% of active
|
||
Vulkan-capable Android devices** as of October 2025
|
||
([developer.android.com/ndk/guides/graphics/android-vulkan-profile](https://developer.android.com/ndk/guides/graphics/android-vulkan-profile))
|
||
— does **not** require `VK_EXT_descriptor_indexing` or any descriptor-
|
||
indexing feature. It requires `shaderSampledImageArrayDynamicIndexing`
|
||
(indexing an array of samplers by a value uniform across the invocation —
|
||
Vulkan 1.0 baseline, unrelated to bindless) and stops there; the same is
|
||
true of the 2021 and 2022 profiles. On the hardware side, Arm's own
|
||
developer documentation states **"`VK_EXT_descriptor_indexing` is
|
||
supported on all Valhall and 5th Gen GPUs"**
|
||
([developer.arm.com/mobile-graphics-and-gaming/vulkan-api-best-practices-on-arm-gpus](https://developer.arm.com/mobile-graphics-and-gaming/vulkan-api-best-practices-on-arm-gpus)) —
|
||
Mali generations from roughly 2019 (Mali-G77) onward, named affirmatively
|
||
with no claim made for Bifrost, Midgard or Utgard, which are still common
|
||
in budget and older Android phones still in use. So this is not a
|
||
software-renderer artifact: a real, currently-shipping share of the
|
||
Android fleet lacks the feature iris's texture pipeline asks for
|
||
unconditionally, and the newest official baseline does not promise it
|
||
either. (A crates.io/search-engine claim of "1% support on Android" for
|
||
this extension was checked against its cited source, an Arm blog post,
|
||
and was not actually there — that number does not appear anywhere primary
|
||
and should not be repeated; the 80.1%-baseline-excludes-it finding above
|
||
is the one with an attributable source.)
|
||
|
||
**Recommendation, not yet implemented.** iris already solved the
|
||
identical problem for text in I1: the glyph atlas
|
||
(`core/src/render/atlas.rs`) packs many small rasters into a handful of
|
||
shared 1024×1024 pages and samples them by UV offset, so **text needs
|
||
none of the three features above** — only ordinary single-texture
|
||
sampling. The same technique generalizes to images: route an `Image`
|
||
widget through a shared atlas when it is small enough to pack (thumbnails,
|
||
downscaled attachment previews, avatars, icons), and fall back to one
|
||
ordinary, non-array texture bind group — selected per batched draw call
|
||
the way every immediate-mode 2D renderer already does — for anything too
|
||
large to atlas well (a photo opened at full resolution). Either path is
|
||
plain Vulkan 1.0 / GLES texture sampling, so it removes the descriptor-
|
||
indexing requirement from iris's device request entirely, which is also
|
||
what would make the emulator work regardless of the wgpu-hal question
|
||
above: a device that never asks for the feature cannot be refused for
|
||
lacking it. This is a change to iris's rendering core — the shader's
|
||
binding group layout, `Textures`, the texture and glyph primitives, and
|
||
`ui/painter.rs` — so it is written here as a recommendation rather than
|
||
started, per the project's rule to confirm a load-bearing design change
|
||
before making it. **It should be resolved before I2 is called done**,
|
||
since I2's pass condition is the phone, not just the emulator, and this
|
||
is exactly the kind of thing that passes on a desktop GPU and fails
|
||
silently on real hardware.
|
||
|
||
- [ ] **I2 — iris on android-view. Built and wired 2026-09-05; the IME half
|
||
passes, rendering does not yet.** Not tickable: the pass condition
|
||
names the tabs example running, and today it runs invisibly.
|
||
|
||
**Layout.** `iris/src/android/` mirrors `default/`'s module split
|
||
(`view.rs` is `app.rs`+`state.rs` combined, since android-view has one
|
||
harness type where winit splits `ApplicationHandler` from per-window
|
||
state; `render.rs`, `input.rs`, `attr.rs` correspond directly;
|
||
`ime.rs` and `insets.rs` have no winit counterpart). What used to live
|
||
only in `default/` and had no winit dependency — `WidgetState`,
|
||
`CursorState`/the sense machinery, `Tasks`, `Selector`/`Selectable`'s
|
||
focus handling — moved to crate-root modules (`state.rs`, `sense.rs`,
|
||
`task.rs`, `attr.rs`) so both backends use one copy; `Tasks`' redraw
|
||
nudge is now behind a `RequestRedraw` trait (`Window` for winit, a
|
||
`JavaVM`+`GlobalRef` attach-and-call for android-view) rather than a
|
||
concrete `winit::window::Window`. `winit`/`arboard` and
|
||
`android-view`/`send_wrapper` are now `[target.'cfg(...)']`
|
||
dependencies, and `default`/`android` are target-gated modules,
|
||
because winit's own Android support needs `android-activity` with a
|
||
backend feature selected — exactly what `iris-core` was kept free of.
|
||
Confirmed by trying it before the split (`cargo ndk -t x86_64 -P 26
|
||
build -p iris` failed inside `android-activity` itself) and after
|
||
(clean). `iris/tabs-ui` is the tabs example's widget tree factored out
|
||
of `examples/tabs/main.rs` into a crate generic over `Rsc: HasEvents`
|
||
+ `Rsc::State: FocusHost`, so the winit example and
|
||
`iris/android-app` (the new cdylib, excluded from the `iris` workspace
|
||
because android-view needs the NDK sysroot to link — see that
|
||
`Cargo.toml`'s comment) call the same `build()`.
|
||
|
||
android-view pinned to `bec6c62a96cef8239b0fd7fedeef9b184d02e3a1`, the
|
||
commit E1 measured against. `RustView.java`/`RustInputConnection.java`
|
||
are vendored (no published AAR to depend on) into
|
||
`iris/android-app/app/src/.../org/linebender/android/rustview/`, with
|
||
one deliberate diff from upstream noted in a comment: `mViewPeer` is
|
||
`protected` rather than package-private, so `IrisView` (a different
|
||
package) can pass it to the window-insets native call android-view
|
||
has no hook for.
|
||
|
||
**Insets and the back gesture**, both without touching android-view.
|
||
The back gesture takes no new plumbing at all: with no
|
||
`OnBackPressedCallback` registered, Android still delivers it as an
|
||
ordinary `KEYCODE_BACK` `KeyEvent` through the existing key path (the
|
||
legacy behaviour every view-based app gets by default), handled in
|
||
`view.rs`'s `on_key_down`. Insets have no such stand-in, so
|
||
`android/insets.rs` registers one more native method
|
||
(`applyWindowInsetsNative`) directly on `IrisView`, writing into an
|
||
`Rc<RefCell<Shared>>` a second copy of which lives in
|
||
`AndroidUiState` — the peer id android-view hands back from
|
||
`register_view_peer` is opaque outside that crate, so this is a
|
||
side table keyed on the same id rather than a way to reach the peer
|
||
itself. `MainActivity` wires `setOnApplyWindowInsetsListener`,
|
||
including the API 30+ `ime()` inset specifically (falls back to 0
|
||
below that). Not yet consumed by any widget's layout — `insets()` is
|
||
exposed on `AndroidUiState` but nothing reads it yet, since the tabs
|
||
example has no chrome that needs to avoid the keyboard.
|
||
|
||
**The IME bridge is implemented and its pass condition holds.**
|
||
`android/ime.rs` implements the full `InputConnection` trait
|
||
(`text_before_cursor`/`after_cursor`/`selected_text`,
|
||
`cursor_caps_mode`, `delete_surrounding_text[_in_code_points]`,
|
||
`set_composing_text`/`_region`, `finish_composing_text`,
|
||
`set_selection`, `begin`/`end_batch_edit`, `send_key_event`,
|
||
`request_cursor_updates`) directly against `TextEdit` — the same
|
||
preedit-replace bookkeeping `default`'s `Ime::Preedit` handling uses
|
||
(`compose_len`, in chars), with new byte<->UTF-16 conversion helpers
|
||
since parley (since I1) is byte-indexed and Java strings are not.
|
||
Two approximations, both commented in place rather than silently
|
||
dropped: `set_composing_region` declines (no separate composing range
|
||
exists to move) and `set_selection`/`delete_surrounding_text_in_code_points`
|
||
collapse to an approximation rather than a real span/code-point
|
||
count. `TextEdit` gained `text()`/`selection_range()`/`caret()`
|
||
getters and `TextEditCtx::delete_byte_range`/`set_cursor_byte`, all
|
||
unconditional (no winit dependency added); `apply_event`/
|
||
`TextInputResult`, which do take a `winit::event::KeyEvent`, are now
|
||
`#[cfg(not(target_os = "android"))]` instead of being ported, since
|
||
android's own `input.rs` calls `TextEdit`'s primitives
|
||
(`backspace`/`delete`/`motion`/`insert`) directly from
|
||
`ndk::event::Keycode` and never needed a winit `KeyEvent` shape.
|
||
|
||
**Measured on the emulator, 2026-09-05, x86_64 API 26,
|
||
`-feature Vulkan` + SwiftShader per the Vulkan section below.**
|
||
`adb shell dumpsys input_method` after tapping the composer field:
|
||
`mInputShown=true`, `mServedInputConnection` is
|
||
`org.linebender.android.rustview.RustInputConnection` attached to
|
||
`IrisView`. `adb shell input text "hi"` followed by a screenshot
|
||
shows **Gboard's suggestion strip populated with "hi | Hi | HI"** —
|
||
capitalization variants read back out of the real buffer through
|
||
`text_before_cursor`, the same kind of evidence E1 recorded (there:
|
||
"dolor | Dolores | door"). That is the bar this box asks for, met.
|
||
|
||
**What is not met: nothing is visible.** The screen shows only the
|
||
clear colour (confirmed black, then swapped to magenta and
|
||
reconfirmed via screenshot — the presentation pipeline itself works)
|
||
with zero widgets drawn on top, on **both** backends tried:
|
||
`Backends::PRIMARY` (Vulkan via SwiftShader, `AdapterInfo` logged as
|
||
`SwiftShader Device (Subzero)`) and `Backends::GL` (GLES via virgl on
|
||
the real host GPU, logged as `Android Emulator OpenGL ES Translator
|
||
(virgl (AMD Radeon RX 7900 XT ...`). Ruled out: the layout engine
|
||
itself -- `log::debug!` calls left in `android/view.rs`'s `render()`
|
||
show `active=39` widgets after `UiRenderState::update` and
|
||
`window_region` reporting the root at the *correct* full-surface
|
||
pixel rect (`(0, 0)..(1080, 2298)`, later `(1080, 1478)` once the
|
||
keyboard's `adjustResize` shrank the window) -- so this is not a
|
||
zero-size-widgets bug. One suspicious but unconfirmed lead: the GLES
|
||
run logged `wgpu_hal::gles: wgpu-hal heuristics assumed that the view
|
||
dimension will be equal to D2 rather than D2Array` right before the
|
||
first `render()` call, which is exactly the shape of a bug in the
|
||
glyph atlas's `texture_2d_array` (`core/src/render/texture.rs`,
|
||
recently reworked per TEXTURES.md) if its array texture is ever
|
||
created with a single layer -- wgpu-hal's GL backend is documented to
|
||
guess the GL texture target from layer count at *texture* creation
|
||
time, which can disagree with a view later requesting `D2Array`
|
||
explicitly. Not chased further: this warning is GL-specific and the
|
||
*same* blank result occurs under Vulkan too, where view dimension is
|
||
always explicit and this class of ambiguity should not exist, so it
|
||
may be a red herring rather than the cause. **`iris/core/src/render/`
|
||
is mid-flight in a separate benchmark branch as of this session**, so
|
||
deliberately not touched here beyond reading it — the next session
|
||
should re-check this finding against whatever lands from that branch
|
||
before spending more time on it, and reach first for the simplest
|
||
possible reproduction (a single hardcoded coloured rect, no text, no
|
||
atlas) to separate "nothing renders" from "the atlas path specifically
|
||
is broken."
|
||
|
||
**Not built yet**: anything consuming `insets()`, a real phone
|
||
measurement (only the emulator so far — matches every other Android
|
||
finding in this file), and AccessKit (I4's job, so `ui-trace`
|
||
couldn't be used here; a raw `adb shell input tap`/`input text` stood
|
||
in for driving the UI, which is why this section says "the same bar
|
||
as E1" rather than citing a `ui-trace` transcript).
|
||
|
||
**Verification.** Host: `cargo fmt --all -- --check`,
|
||
`cargo build --workspace --all-targets`, `cargo clippy --all-targets`,
|
||
`cargo test --workspace` (19 tests) all clean in `iris/`; `iris/run-headless.sh
|
||
tabs --shot` still renders pixel-identically (27266 bytes, byte-for-byte
|
||
unchanged). Android cross-compile: `cargo ndk -t x86_64 -P 26 build`
|
||
and `... clippy` clean for both `iris` (with the android module) and
|
||
`iris/android-app`. Emulator: `emu up` with
|
||
`VK_DRIVER_FILES=.../vk_swiftshader_icd.json` and
|
||
`GPU_HOST_FEATURES="-feature Vulkan -no-snapshot-load -no-snapshot-save"`
|
||
per the Vulkan section; `cd android-app && cargo ndk -t x86_64 -P 26
|
||
-o app/src/main/jniLibs/ build --release && gradle :app:assembleDebug`
|
||
(release native lib per E1's segfault finding, debug Gradle variant --
|
||
the jniLibs contents are what matters, not the Gradle build type);
|
||
`adb install -r app/build/outputs/apk/debug/app-debug.apk`.
|
||
- [ ] **I3 — a virtualised, bottom-anchored list.** Variable-height rows,
|
||
keyed, composed only while visible, paged in both directions with a
|
||
"more" sentinel at each end, a scroll anchor that survives rows
|
||
being inserted above, and "hold the edge nearest the tap" done in
|
||
the layout pass. Pass: 800 rows of real transcript text from the
|
||
sandbox scroll without a frame over the Compose baseline in
|
||
`transcript-bench.sh`, measured on the GPU emulator.
|
||
- [ ] **I4 — accessibility names via AccessKit.** Every control carries a
|
||
name; `ui-trace` can find and tap it by label. Pass: `bench-lib.sh`'s
|
||
tap-by-name works against the iris screen unchanged.
|
||
- [ ] **I5 — the transcript screen in iris.** E2's pass conditions, all
|
||
seven behaviours, against the sandbox with `--delay`. This is the
|
||
point the decision in the recommendation is made at.
|
||
|
||
## For the next agent
|
||
|
||
What to do when you pick this up, in order, so nothing here has to be
|
||
re-derived:
|
||
|
||
1. Read this file, then `AGENTS.md` and `PLAN.md`. The rules there
|
||
(measure, do not read; fix the rig before accepting its limits; the
|
||
emulator is this checkout's own) all apply.
|
||
2. Work on the **`rustify`** branch of this clone (`ai-app-2`), not on
|
||
`main` and not in `ai-app`. Nothing on this branch is production until
|
||
Iris says so. Commit and push as you go.
|
||
3. Take the next unchecked box above, in order. E1 has proved android-view
|
||
on this emulator, so the E-steps and the I-steps can now proceed in
|
||
parallel in separate sessions; see "Where things stand" at the top for
|
||
which is next.
|
||
4. Every step ends with its measurement written into this file beside the
|
||
box, and the box ticked or the reason it could not be written in its
|
||
place. A step that is blocked says by what, not "later". Write it as you
|
||
go rather than at the end — see "Keep this file current as you work".
|
||
5. Run the existing rigs rather than inventing new ones: `ui-sandbox.sh`
|
||
for a server with fixtures, `transcript-bench.sh` for the scroll
|
||
baseline, `ui-trace` for anything positional, `emu up` for the
|
||
emulator, `iris/run-headless.sh EXAMPLE --shot PNG` for an iris
|
||
example on this displayless machine, and `rigs/gpu-probe` to ask a
|
||
device (this VM, the emulator, or a real phone over `adb push`) what
|
||
`wgpu` features and limits it actually has before building anything on
|
||
the assumption it does. The Vulkan section below says how to get a
|
||
Vulkan path in the emulator when a `wgpu` backend needs one.
|
||
6. **Bound anything heavy at the moment you start it.** An emulator or a
|
||
long build gets a deadline — `timeout`, or a watchdog scoped to the pid
|
||
you just started — rather than a plan to stop it later. Scope it to
|
||
that pid: a watchdog written as `sleep N; emu down` fired into a later
|
||
experiment here and made a working Vulkan build look like a crash. And
|
||
stop the emulator when the work needing it is done rather than between
|
||
tasks.
|
||
7. Decisions belong here with a date and what was rejected, the way
|
||
`PLAN.md` does it. Do not put design into commit messages alone.
|
||
|
||
### Vulkan in the emulator (measured 2026-09-04)
|
||
|
||
**Settled 2026-09-04: the guest gets Vulkan from SwiftShader, and the
|
||
missing step was a cold boot.** `-feature Vulkan` plus
|
||
`VK_DRIVER_FILES=$HOME/Android/Sdk/emulator/lib64/vulkan/vk_swiftshader_icd.json`
|
||
gets the *host* side to select SwiftShader, but the guest keeps reporting
|
||
zero devices until `-no-snapshot-load` is added, because it boots from a
|
||
snapshot saved under the previous GPU config — `-no-snapshot-save` is
|
||
worth adding too, so the Vulkan-configured snapshot does not then break
|
||
the next ordinary boot. With that, `cmd gpu vkjson` reports SwiftShader
|
||
Subzero and wgpu takes its Vulkan path (E1). `EMU_GPU=software` in
|
||
`emulator-tools` gets the same guest Vulkan with no GPU use at all, for
|
||
work where the emulator's frame rate is not what is being measured.
|
||
|
||
The rest of this section stands as the record of why host Vulkan is not
|
||
available. A `wgpu` app in this emulator was going to get GLES only,
|
||
because host Vulkan is switched off in `emulator-tools`. Retried on Mesa
|
||
26.1.7: **Venus still fails the same way** — gfxstream picks
|
||
`externalMemoryMode: OpaqueFd`, probes `VK_FORMAT_R8G8B8A8_UNORM` for an
|
||
exportable colour buffer, and Venus says the format is unsupported
|
||
(`Failed to find memory type for ColorBuffers`, fatal before adb sees the
|
||
device). Venus does advertise `VK_KHR_external_memory_fd` and
|
||
`VK_EXT_external_memory_dma_buf`, so the gap is specifically opaque-fd
|
||
image export. gfxstream has a string-valued `VulkanExternalMemoryMode`
|
||
setting ("overrides what would otherwise be determined automatically"),
|
||
but `-feature Name=Value` is rejected as a bad feature name, and the only
|
||
mode words compiled into this emulator's `libgfxstream_backend.so`
|
||
(37.1.11) are `OpaqueFd`, `Metal` and `none` — there is no dma-buf mode
|
||
in this build to switch to. So Venus is blocked by the emulator, not by
|
||
Mesa; retry when the emulator package updates, since upstream gfxstream
|
||
does have dma-buf external memory.
|
||
|
||
What **does** work: pointing the emulator's Vulkan loader at the software
|
||
ICDs the emulator ships itself, with the feature enabled:
|
||
|
||
VK_DRIVER_FILES=$HOME/Android/Sdk/emulator/lib64/vulkan/vk_swiftshader_icd.json \
|
||
GPU_HOST_FEATURES="-feature Vulkan" emu up
|
||
|
||
The guest then reports Vulkan 1.3 (`cmd gpu vkjson`, SwiftShader
|
||
Subzero) while GLES still runs on the real GPU through virgl — so a
|
||
Vello/wgpu app can take its real Vulkan path here, with compute shaders,
|
||
CPU-rasterised. That is enough to test *correctness* of the Vulkan path
|
||
in the emulator; GPU *performance* of it is a phone measurement either
|
||
way, exactly as `MACHINE.md` already says about frame times. **lavapipe**
|
||
(`lvp_icd.json`, the other ICD the emulator ships) selected llvmpipe and
|
||
booted, then the emulator died right after loading the `default_boot`
|
||
snapshot with nothing in the log; a snapshot saved under a different
|
||
Vulkan device is the suspect, and `-no-snapshot-load` is the untested
|
||
next step. SwiftShader is the one that works today.
|
||
`EMU_GPU=software` is now in `emulator-tools` (agreed with the ai-app
|
||
session and with Iris, default unchanged, since `-gpu host` was measured
|
||
and the Compose scroll benchmarks depend on it). The cold-boot flags are
|
||
not a knob there: that wants snapshot invalidation as well, which is a
|
||
bigger design question in shared tooling.
|
||
|
||
## Things a Rust app changes elsewhere
|
||
|
||
- **`wg-app-link`'s `:link`** (pinned TLS, enrollment store, QR activity)
|
||
is Kotlin shared with Dev Updater. The certificate code already exists on
|
||
the Rust side of the submodule; the pinned-CA build step
|
||
(`generatePinnedCert`) becomes a `build.rs` reading the same path. The QR
|
||
scanner stays a Kotlin activity, since the camera is a platform feature.
|
||
- **Tooling** becomes `cargo` for everything but packaging: `cargo test`,
|
||
`clippy`, `fmt` cover the whole client, which is the motivation. Gradle
|
||
remains for the APK, signing (`~/.config/ai-app/release.jks`) and Dev
|
||
Updater's build modes; `build-apk.sh` would call `cargo ndk` first.
|
||
- **The bench scripts** (`ui-trace` by accessibility label) keep working
|
||
only if the framework exposes names through AccessKit on Android; that is
|
||
part of E2's pass condition, not a nicety.
|
||
- **Icons** stay Nerd Font glyphs from the committed subset; Parley/Fontique
|
||
loads a font file directly, so `build-icon-font.sh` is unchanged.
|
||
|
||
## Sources
|
||
|
||
- iced: [repo](https://github.com/iced-rs/iced), [0.14 release](https://github.com/iced-rs/iced/releases/tag/0.14.0), [Android thread](https://news.ycombinator.com/item?id=46350641), [markdown selection request](https://discourse.iced.rs/t/markdown-widgets-text-should-be-selectable/1107)
|
||
- Linebender: [2026 Q1 report](https://linebender.org/blog/tmil-25/), [xilem](https://github.com/linebender/xilem), [parley](https://github.com/linebender/parley), [vello](https://github.com/linebender/vello), [vello_hybrid](https://docs.rs/vello_hybrid/latest/vello_hybrid/)
|
||
- android-view: [repo](https://github.com/rust-mobile/android-view); android-activity [PR #214](https://github.com/rust-mobile/android-activity/pull/214)
|
||
- winit Android IME: [#1823](https://github.com/rust-windowing/winit/issues/1823), [#2766](https://github.com/rust-windowing/winit/issues/2766), [#2305](https://github.com/rust-windowing/winit/issues/2305)
|
||
- egui on Android: [discussion #2053](https://github.com/emilk/egui/discussions/2053)
|
||
- Slint: [Android guide](https://docs.slint.dev/latest/docs/slint/guide/platforms/mobile/android/), [1.15 release](https://slint.dev/blog/slint-1.15-released), [licensing](https://slint.dev/faqs), rich text [#1325](https://github.com/slint-ui/slint/issues/1325), markdown [#6684](https://github.com/slint-ui/slint/issues/6684)
|
||
- Makepad: [repo](https://github.com/makepad/makepad), [makepad-widgets](https://docs.rs/makepad-widgets), [Robrix](https://github.com/project-robius/robrix), [Robrix releases](https://github.com/project-robius/robrix/releases)
|
||
- AccessKit: [releases](https://github.com/AccessKit/accesskit/releases)
|
||
- Build tools: [cargo-ndk](https://github.com/bbqsrc/cargo-ndk), [cargo-apk](https://github.com/rust-mobile/cargo-apk), [rust-mobile](https://github.com/rust-mobile)
|
||
- uniffi: [repo](https://github.com/mozilla/uniffi-rs), [KMP bindings fork](https://github.com/UbiqueInnovation/uniffi-kotlin-multiplatform-bindings)
|
||
- GPUI mobile: [gpui-mobile](https://github.com/itsbalamurali/gpui-mobile)
|
||
- The earlier Dioxus spike's findings on `wgpu`/Vulkan in this emulator: `~/repos/tdep-survey/app-dioxus/README.md`
|