irisandClaude Opus 5
43743ba171
Adds a "Keep this file current as you work" section at the top saying what it is for: this file is the handoff, so writing results into it as they arrive is what lets a session that has filled its context be cleared instead of carrying the conversation or re-deriving what was measured. It asks for the dead ends too, since those are what stop the next session spending an afternoon somewhere already ruled out. Adds a "Where things stand" block, because the next agent's first question is which box is next and the answer was previously spread across the list: E0, E1, I0a, I0b and I1 done, I2 next with E2 able to run in parallel, client-core not started, and the two emulator-tools changes made outside this repo. Corrects what had gone stale: the next-agent steps still said to start at E0; the iris section still described a fourteen-gate cosmic-text tree and called the text stack an open question; and the weight section still spoke of E1 as something that would happen. It now carries the numbers instead -- 43s and 2.1 GB against 1m46s and 1.5 GB for iris, and 181 MB debug against 11 MB release for the Masonry demo. Adds the rule about bounding heavy runs with a kill timer scoped to the pid, which cost a wrong conclusion here. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
908 lines
54 KiB
Markdown
908 lines
54 KiB
Markdown
# Moving the app to Rust
|
||
|
||
Working document for the question Iris asked on 2026-09-04: what are the
|
||
options for switching the phone app to Rust, ideally pure Rust with one UI
|
||
framework shared with a future winit-based desktop application, at full
|
||
feature parity and without giving up anything native, performance
|
||
especially. Constraints she set: no Dioxus and nothing that draws through a
|
||
WebView; **no UI DSL** (which rules out Makepad and Slint); the result
|
||
should stay lightweight; platform-specific pieces are fine to maintain;
|
||
reimplementing a framework piece from scratch where it does not fit is
|
||
fine; effort and elapsed time do not matter, long-term robustness does;
|
||
this clone is where things get tried before anything is committed to
|
||
`ai-app`. Her own library, [iris](https://github.com/cat16/iris), is the
|
||
**in-house framework to be built up** for this, with Masonry as the
|
||
yardstick it is measured against.
|
||
|
||
Decisions get a date and a reason here, the way `PLAN.md` does. A section
|
||
describes something that has been tried only where it says so, with a date.
|
||
|
||
## Keep this file current as you work
|
||
|
||
**This file is the handoff, and it is meant to let a session be cleared.**
|
||
Write each result into it *as you get it*, not at the end: the box ticked
|
||
or the reason it could not be, the measurement with its number, the
|
||
decision with its date and what it rejected, and anything that cost time to
|
||
find out. Then a session that has filled its context can be cleared and the
|
||
next one can pick up from this file alone, which is much cheaper than
|
||
carrying a long conversation or re-deriving what was already measured.
|
||
|
||
Two things that follow. Write for somebody who was not here — name the
|
||
command, the file and the number rather than "the fix" or "the earlier
|
||
run". And write the failures and the dead ends too: "Venus is blocked by
|
||
the emulator, not by Mesa" and "the present mode was not the cause" are
|
||
worth as much as the successes, because they are what stops the next
|
||
session spending an afternoon on them again.
|
||
|
||
## Where things stand (2026-09-04)
|
||
|
||
- **Done**: E0 (toolchain), E1 (Masonry on android-view, which found the
|
||
keyboard gap), I0a, I0b (iris builds on a pinned nightly and runs), I1
|
||
(parley + glyph atlas).
|
||
- **Next**: **I2** — iris on android-view. E1 says the first thing to
|
||
answer there is why the editor gets keystrokes but no autocorrect or
|
||
suggestions, since that is the constraint the framework decision turns
|
||
on. **E2** (a transcript in Masonry) can go in parallel in another
|
||
session.
|
||
- **Not started**: `client-core`, which is item 1 of the recommendation
|
||
below and does not depend on the framework choice. Nothing has been
|
||
built for it, and it is not one of the numbered boxes — worth picking up
|
||
in a session that wants work independent of the emulator.
|
||
- **The app itself is untouched.** Everything so far is in `iris/` and in
|
||
the rigs; nothing under `app/` or `server/` has changed.
|
||
- **Changed outside this repo**, both in `emulator-tools` and both pushed:
|
||
`avd_serial` now validates its cache by asking the device its AVD name
|
||
rather than by checking the serial is still attached (a recycled port
|
||
silently pointed this checkout at another session's emulator), and
|
||
`EMU_GPU=software` was added as an opt-in that keeps a run off the host
|
||
GPU and gives the guest a software Vulkan device. The default is
|
||
unchanged, because `-gpu host` was measured and the Compose benchmarks
|
||
depend on it.
|
||
|
||
## What has to be reproduced
|
||
|
||
The app is ~19,000 lines of Kotlin. It splits three ways, and the split is
|
||
what decides how much of a port is mechanical.
|
||
|
||
**Pure logic with no Compose or Android in it, ~4,500 lines.** `Api.kt`
|
||
(1,142), `Events.kt`, `EventStream.kt`, `Sse.kt`, `TranscriptCache.kt`
|
||
(589, touches `java.io.File` only), `TranscriptSource.kt`,
|
||
`MarkdownSyntax.kt`, `Languages.kt`, `Highlighter.kt`, `Ansi.kt`,
|
||
`ResetCountdown.kt`, `Durations.kt`, `Sizes.kt`, `ModelName.kt`,
|
||
`LoadState.kt`, `ImportableStream.kt`. `TranscriptUnits.kt` and
|
||
`TranscriptItems.kt` (the event fold into rows, ~940 lines) are logic with
|
||
a handful of Compose annotations. This is also exactly the code that has
|
||
JVM unit tests today. All of it ports directly, and most of it already has a
|
||
Rust twin in `server/`: `Events.kt` is a hand-kept mirror of
|
||
`session/driver.rs`'s enum, the highlighter and the syntax scanner exist on
|
||
the server for the explorer, and the cache compares the server's own JSON
|
||
lines. **Sharing these types between server and app is the single largest
|
||
"keep things in sync" win available, and it does not depend on which UI
|
||
framework wins.**
|
||
|
||
**Compose UI, ~13,000 lines.** Screens, dialogs, the transcript list, the
|
||
markdown renderer's customisations, tool cards, the file explorer viewer and
|
||
editor. This is the part a UI framework choice is about.
|
||
|
||
**Android platform code, ~1,500 lines**, spread over 20 files. Every one of
|
||
these is a Java-side object that no Rust framework can replace, because
|
||
Android only offers them as Java classes:
|
||
|
||
- `NotificationService` — a **foreground service** holding the
|
||
`/notifications` SSE stream while the app is closed, with its ongoing
|
||
notification, `specialUse` type and the `POST_NOTIFICATIONS` request.
|
||
- `MainActivity` — edge-to-edge, the `ACCESS_LOCAL_NETWORK` runtime
|
||
permission (Android 17), `singleTop` intent routing for `aiapp://enroll`,
|
||
notification taps, and the **share sheet** (`ACTION_SEND`, any MIME type).
|
||
- `ServerConfig` — the bearer token sealed under an **Android Keystore**
|
||
AES-GCM key, shared with Dev Updater through `wg-app-link`'s `:link`.
|
||
- `EnrollmentScanActivity` — the in-app **QR scanner** (zxing, camera).
|
||
- `Attachments` — `ContentResolver` reads of shared URIs, `BitmapFactory`
|
||
decode and downscale, **EXIF** orientation.
|
||
- `SessionImage` — bitmap decode for produced images.
|
||
- `ScrollAnchor`, `Drafts` — `SharedPreferences`; `CrashLog` — `filesDir`.
|
||
- `TranscriptCache` — `cacheDir`.
|
||
- `DebugStats`/`FrameStats` — `Choreographer` frame timing and the render
|
||
report; `runtime-tracing` names composables in a system trace.
|
||
|
||
So **"pure Rust" on Android means Rust owns every line of logic and
|
||
drawing, behind a thin shell of Java stubs**, and a packaging step that
|
||
produces a signed APK. How thin, and whether Gradle is inevitable, are
|
||
answered below.
|
||
|
||
### How much Java is unavoidable, and why
|
||
|
||
Rust can *call* any Android API through JNI (`jni` crate, with
|
||
`ndk-context` handing over the `JavaVM` and the Activity): posting a
|
||
notification, `startForegroundService`, the Keystore, `ContentResolver`
|
||
reads, permission requests, `WindowInsets`, the clipboard. None of that
|
||
needs a line of Kotlin. What JNI cannot do is *define* a class that the
|
||
system instantiates **by name from the manifest** — an `Activity`, a
|
||
`Service`, an `Application`, a `BroadcastReceiver`. Those must exist as dex
|
||
bytecode inside the APK before any Rust runs, because the framework
|
||
constructs them and only then calls into native code. `NativeActivity` is
|
||
the platform's own stub for the Activity case; there is no
|
||
`NativeService`, and android-view ships its own `View` subclass for the
|
||
same reason.
|
||
|
||
So the floor is roughly **two Java classes of ten lines each**: an
|
||
`Activity` and a `Service` whose lifecycle methods are declared `native`
|
||
and registered from `JNI_OnLoad`, plus whatever android-view already
|
||
provides. Everything they would have done in Kotlin — insets, intent
|
||
routing, the SSE follow loop, the notification builder — is Rust reached
|
||
through those stubs. Writing the stubs in Java rather than Kotlin drops
|
||
`kotlinc` from the toolchain; `javac` comes with the JDK Gradle already
|
||
needs. Generating the dex from Rust is not worth it: there is no mature
|
||
Rust dex writer, and the stubs never change.
|
||
|
||
### Can the APK be built without Gradle?
|
||
|
||
Yes. An APK is a zip containing a binary-XML `AndroidManifest.xml`,
|
||
`resources.arsc`, `classes.dex`, `lib/<abi>/*.so` and assets, aligned and
|
||
signed with the v2 scheme. The tools are `aapt2` (manifest and resources),
|
||
`d8` (Java bytecode to dex), `zipalign` and `apksigner`, all in the SDK's
|
||
`build-tools`, none of them Gradle. Three ways to drive them:
|
||
|
||
- **A `cargo xtask`** (or `build.rs`-adjacent script) that runs `cargo ndk`
|
||
for each ABI, `javac` + `d8` for the stubs, `aapt2 link`, `zipalign`,
|
||
`apksigner`. About 150 lines, every step visible, no AGP, no Gradle
|
||
daemon holding 2.8 GB between builds. The pinned-CA constant becomes a
|
||
`build.rs` reading the same `certs/ca.pem` path.
|
||
- **[cargo-apk2](https://github.com/mzdk100/cargo-apk2)**: the maintained
|
||
successor to cargo-apk, and unlike it compiles `java_sources` /
|
||
`kotlin_sources` into the dex and declares multiple activities **and
|
||
services** with intent filters from `[package.metadata.android]`, with
|
||
per-profile keystores and optional `aapt2`. Exactly the shape needed;
|
||
the question is whether a third-party tool with one maintainer beats
|
||
150 lines we own.
|
||
- **cargo-apk / xbuild**: unmaintained and `NativeActivity`-only. No.
|
||
|
||
What Gradle would take with it: Android Lint (which found two real bugs
|
||
here, but in Kotlin that would no longer exist — with forty lines of Java
|
||
stubs there is little left for it to find), manifest merging, R8, and the
|
||
generated-source plumbing. What it gives back: one toolchain, `cargo`
|
||
end to end, and Dev Updater keeps calling `build-apk.sh` exactly as now.
|
||
**Recommendation: the xtask**, with cargo-apk2 read for the details it
|
||
already got right (v2 signing, `uses-feature`, ABI splits).
|
||
|
||
### The behaviours that are hard to get back
|
||
|
||
Reading the Compose code for what a replacement must be able to express,
|
||
rather than what it happens to look like:
|
||
|
||
1. **The transcript is one selectable body of text.** One
|
||
`SelectionContainer` around the whole lazy list, so a selection runs from
|
||
a reply into the tool output beneath it. The framework needs selectable
|
||
read-only rich text across many rows, with the platform's selection
|
||
handles and clipboard on the phone.
|
||
2. **Rich inline text**: markdown with links (one tap detector per text,
|
||
not a node per link), inline code chips drawn behind the text, tables
|
||
with wrapping cells and a sideways scroll, syntax-highlighted fences,
|
||
ANSI colour in tool output, Nerd Font icon glyphs. Needs a text layout
|
||
engine with spans, not just styled labels.
|
||
3. **A bottom-anchored virtualised list of variable-height rows**, paged in
|
||
both directions (800-event pages, `HISTORY_SCREENS` measured in
|
||
viewports), with a saved scroll anchor per session, "hold the edge
|
||
nearest the tap" when a row expands (`holdTopEdge`, done in the layout
|
||
pass so the wrong frame is never drawn), and rows keyed so that a run of
|
||
tool calls stays one row while it grows.
|
||
4. **The soft keyboard**: the composer resizes with the IME, the guard
|
||
against a stuck inset animation, drafts per session, autocorrect and
|
||
suggestions from the phone's own keyboard. This is where most Rust
|
||
frameworks fail on Android today; see below.
|
||
5. **Platform integration through the app model**: foreground service,
|
||
notifications, share sheet, deep link, Keystore, camera, back gesture,
|
||
edge-to-edge insets, local-network permission.
|
||
6. **Accessibility names on icon buttons**, which the bench scripts depend
|
||
on (`ui-trace` taps by label). A framework with no accessibility tree
|
||
also breaks the measuring rig.
|
||
7. **Measurable frames**: the debug render report, and a way to attribute
|
||
a frame's cost to a widget on the real phone.
|
||
|
||
## The two constraints that decide it
|
||
|
||
**1. Android text input.** Every framework built on `winit` inherits
|
||
winit's Android backend, and that backend cannot drive the soft keyboard
|
||
properly: the IME tracking issues
|
||
([#1823](https://github.com/rust-windowing/winit/issues/1823),
|
||
[#2766](https://github.com/rust-windowing/winit/issues/2766)) are open,
|
||
`ReceivedCharacter` is unimplemented on Android
|
||
([#2305](https://github.com/rust-windowing/winit/issues/2305)), and the
|
||
`android-activity` groundwork for editor actions only merged in February
|
||
2026 ([PR #214](https://github.com/rust-mobile/android-activity/pull/214))
|
||
with the winit half still to come. Composition, autocorrect and suggestions
|
||
need an `InputConnection` implemented on the Java side, which winit's
|
||
`NativeActivity`/`GameActivity` model does not offer. The frameworks that
|
||
type on Android today each wrote their own Java glue (Slint, Makepad), and
|
||
the one designed to do it the way Android intends is
|
||
[`android-view`](https://github.com/rust-mobile/android-view): a Rust
|
||
implementation of an Android `View`, with text input through
|
||
`InputConnection`, accessibility, touch, callbacks on the UI thread, usable
|
||
either as a whole app or embedded beside ordinary Android components. It is
|
||
marked WIP. Both Linebender (its Masonry demo lives in that repo) and
|
||
Robius/Makepad ([Robrix's release notes](https://github.com/project-robius/robrix/releases)
|
||
say Android lacks a "full" keyboard and they are integrating android-view
|
||
for it) are converging on it. **That makes android-view the phone-side
|
||
foundation whichever widget set sits on top**, and the first thing to
|
||
build and measure here.
|
||
|
||
**2. Rich, selectable text and a virtualised list.** Frameworks group by
|
||
their text stack:
|
||
|
||
- **Parley + Fontique + Vello** (Linebender): rich spans, selection and
|
||
editing utilities, IME support driven through `ui-events`, AccessKit text
|
||
properties ([Linebender 2026 Q1](https://linebender.org/blog/tmil-25/),
|
||
[parley](https://github.com/linebender/parley)). Used by Masonry/Xilem,
|
||
and by Blitz. Vello proper needs compute shaders; `vello_hybrid` (CPU
|
||
path processing, GPU compositing) is "roughly beta" and runs on GLES too,
|
||
and Vello CPU exists as a no-GPU fallback.
|
||
- **cosmic-text** (iced, egui optionally): good layout, but the widgets on
|
||
top decide selection. iced's `markdown` widget is not selectable
|
||
([discourse](https://discourse.iced.rs/t/markdown-widgets-text-should-be-selectable/1107)).
|
||
- **Slint's own**: `TextInput` with `read-only` is the selectable-text
|
||
trick; there is **no inline rich text at all** (issue
|
||
[#1325](https://github.com/slint-ui/slint/issues/1325), markdown request
|
||
[#6684](https://github.com/slint-ui/slint/issues/6684) both open). A
|
||
markdown transcript with links and code chips cannot be drawn.
|
||
- **Makepad's own**: GPU/SDF text, a `Markdown` widget and a virtualised
|
||
`PortalList` in `makepad-widgets`.
|
||
|
||
## Options
|
||
|
||
### A. Keep Compose, move the logic into a Rust core (uniffi)
|
||
|
||
A `client-core` crate (events shared with the server, API client, SSE,
|
||
transcript fold, cache, markdown model, highlighter, ANSI) exposed to
|
||
Kotlin through [uniffi](https://github.com/mozilla/uniffi-rs). Compose keeps
|
||
drawing. Desktop would be a second UI (iced or Compose Desktop) over the
|
||
same core.
|
||
|
||
- **For**: the logic and the wire types stop drifting from the server
|
||
today, with tests in one language. Incremental and always shippable.
|
||
- **Against**: it is not what was asked for. The 13,000 lines of UI stay
|
||
Kotlin, the desktop app shares no UI code, and the `:link` Kotlin module
|
||
stays. uniffi's Kotlin Multiplatform bindings are a
|
||
[community fork](https://github.com/UbiqueInnovation/uniffi-kotlin-multiplatform-bindings);
|
||
the Android-only bindings are Mozilla's and solid.
|
||
- **Verdict**: not the destination, but **step one of every other option**
|
||
is building this crate, so it costs nothing to keep it as the fallback.
|
||
|
||
### B. Slint
|
||
|
||
Rust on Android is officially supported (minSdk 26, `android-activity`
|
||
backend, own Java IME glue, safe areas and keyboard insets since 1.15,
|
||
Skia renderer needs `clang`). Royalty-free licence requires disclosing
|
||
Slint use; GPLv3 otherwise. UI is a separate `.slint` DSL, not Rust.
|
||
|
||
- **Against**: no rich inline text (see above), so the transcript cannot be
|
||
drawn as it is today; the UI language is not Rust, which forfeits the
|
||
"compiler catches it" motivation for the half of the code that is UI.
|
||
- **Verdict**: rejected on rich text alone.
|
||
|
||
### C. iced
|
||
|
||
Elm-style, Rust-only widgets, desktop-first, `winit` + `wgpu`. Has a
|
||
`markdown` widget and `rich_text` with links. The maintainer states mobile
|
||
is a non-goal ([iced](https://github.com/iced-rs/iced)); a community
|
||
Android example exists and its author could not get the soft keyboard
|
||
working, patched widgets for touch, and notes no accessibility
|
||
([HN thread](https://news.ycombinator.com/item?id=46350641)). Markdown is
|
||
not selectable; `scrollable` is not virtualised.
|
||
|
||
- **Verdict**: a fine desktop toolkit and the one Iris named, but every
|
||
phone-side gap (IME, touch, accessibility, selection, virtualisation)
|
||
would be ours to build and maintain against a project that does not want
|
||
them. Not the shared framework.
|
||
|
||
### D. egui
|
||
|
||
Immediate mode, `winit`-based on Android, AccessKit integration,
|
||
selectable labels across a `Ui`. Repaints only on input by default, so
|
||
battery is not the immediate-mode worry. Android IME is blocked on winit
|
||
([discussion](https://github.com/emilk/egui/discussions/2053)); the
|
||
workaround is an in-app virtual keyboard, which is exactly the
|
||
non-native keyboard to avoid. Variable-height virtualised lists are manual
|
||
(`show_rows` assumes uniform heights). Looks like egui, not Material.
|
||
|
||
- **Verdict**: workable on desktop, wrong on the phone for the same reason
|
||
as iced, plus a look that would need a full custom style.
|
||
|
||
### E. Makepad
|
||
|
||
GPU-rendered, hybrid retained/immediate, `live_design!` DSL with hot
|
||
reload, MIT, 1.0 in 2025 ([makepad](https://github.com/makepad/makepad)).
|
||
Ships Android apps today with its own Java glue; Robrix (a Matrix chat
|
||
client, the closest analogue to this app) is its reference application on
|
||
Android, iOS and desktop. Has `Markdown`, `PortalList` (virtualised),
|
||
`TextInput`. Robrix reports the Android keyboard is not "full" and is moving
|
||
to android-view for it; the README says non-standard targets "may require
|
||
minor fixes".
|
||
|
||
- **For**: the only option that already ships a chat-shaped app on Android
|
||
and desktop from one codebase, with the widgets this app needs.
|
||
- **Against**: the DSL is its own language with its own shader-based
|
||
styling, so a large part of the UI would not be checked by rustc; the
|
||
rendering model (SDF everything) is a different world from Compose's,
|
||
and selection across a `Markdown` widget is unverified.
|
||
- **Verdict**: **rejected 2026-09-04** — Iris does not want a DSL. Kept
|
||
here so its Android keyboard status stays a data point about
|
||
android-view, not as an option.
|
||
|
||
### F. Masonry / Xilem on android-view (Linebender)
|
||
|
||
Retained widget tree (Masonry) with a reactive view layer (Xilem) that
|
||
reads like Compose; Rust all the way down; Vello, Parley, Fontique,
|
||
AccessKit, `ui-events`. Widgets include `Prose` (selectable read-only rich
|
||
text), `TextArea`, `VirtualScroll`, and this year `Svg`, `Split`,
|
||
`CollapsePanel`, a new layout system, and IME through `ui-events`
|
||
independent of winit. `masonry_android_view` exists in the android-view
|
||
repo and is "not yet generally usable"; Xilem calls itself experimental.
|
||
Desktop runs on winit. Vello needs a compute-capable GPU or falls back to
|
||
`vello_hybrid`/CPU.
|
||
|
||
- **For**: the only stack where every hard behaviour above maps onto a
|
||
component designed for it: selection and rich text (Parley/Prose),
|
||
virtualised variable heights (`VirtualScroll`), native IME
|
||
(android-view's `InputConnection`), accessibility (AccessKit, now with an
|
||
Android crate), one Rust widget language on both platforms. The team is
|
||
the one writing the Android integration everyone else is adopting.
|
||
- **Against**: pre-1.0 with API churn each release; a small team; no
|
||
Material widget set, so every control's look is ours; some of the pieces
|
||
(`masonry_android_view`, `vello_hybrid`) are explicitly unfinished. Being
|
||
early means fixing things upstream ourselves, which Iris said is
|
||
acceptable.
|
||
- **Verdict**: **the option to try first**, because it is the only one
|
||
whose gaps are "not finished yet" rather than "not designed for this".
|
||
|
||
### G. iris — the in-house library, and what "from scratch" means here
|
||
|
||
[cat16/iris](https://github.com/cat16/iris), read 2026-09-04 from the one
|
||
public commit (2026-01-31, "portfolio copy"; ~8,700 lines in `core`,
|
||
`macro` and the crate itself). Retained-mode widgets stored outside the
|
||
render tree, `wgpu` 28 directly, `winit` 0.30, `cosmic-text` 0.16 (parley
|
||
since I1), a
|
||
relative-anchor-plus-offset layout with `rest()` and `rel()` lengths, a
|
||
postfix builder API (`rect(..).radius(30).on(CursorSense::click(), ..)
|
||
.sized(..).align(..)`), events handled where the widget is declared, and
|
||
a single-threaded context passed explicitly — all of which reads like this
|
||
codebase's own rules. There is text editing (`widget/text/edit.rs`),
|
||
images, masks, spans and stacks; the TODO names text resizing as
|
||
per-frame slow and scaling as unsolved. It requires **nightly** (fourteen
|
||
`#![feature]` gates as vendored, among them `const_trait_impl`,
|
||
`unboxed_closures`, `portable_simd`, `associated_type_defaults`; eleven
|
||
after I0b and I1 — see those steps for the current list). Desktop only; no
|
||
Android surface, no IME, no accessibility tree, no virtualised list, no
|
||
rich-text selection.
|
||
|
||
**That list is the work, and some of it is done.** As of 2026-09-04 it
|
||
builds on a pinned nightly, runs on this machine's GPU, has parley and a
|
||
glyph atlas, and `iris-core` cross-compiles to Android. What it still
|
||
lacks from the list above is the Android surface, the IME bridge, the
|
||
accessibility tree and the virtualised list — I2, I3 and I4.
|
||
|
||
**iris is not a candidate to be tested as it stands. It is the in-house
|
||
library** (Iris, 2026-09-04): "essentially a good start to a rewrite from
|
||
scratch", to be maintained and extended by the sessions working here.
|
||
So the list above of what it lacks is a **work list, not a score**. When
|
||
the app needs something iris does not have, the answer is to build it
|
||
into iris. The layer iris has is the widget and layout layer; the layers
|
||
it needs are the same ones Masonry gets from android-view, Parley and
|
||
AccessKit, and there is no reason iris cannot sit on those same
|
||
foundations rather than reinvent them — the surface, the keyboard bridge
|
||
and the accessibility tree are platform plumbing, not a framework's
|
||
identity. The text stack was the first real design decision in that work,
|
||
and it is settled: **Parley, with a glyph atlas** (I1, 2026-09-04).
|
||
|
||
Two things to carry into that work honestly. Nightly is the opposite of
|
||
"holds up long term": a build that breaks on a toolchain update, on the
|
||
machine Dev Updater builds on, unattended. **Done in I0b** —
|
||
`iris/rust-toolchain.toml` pins `nightly-2026-09-03` — and the gate list
|
||
lives with I0b and I1, to be retired as they stabilise or are designed
|
||
around; it is down from fourteen to eleven. And a one-person framework
|
||
carries every gap itself, which is what Iris said she is willing to do.
|
||
|
||
"From scratch" therefore means iris, not a fourth thing. Masonry stays in
|
||
the plan as the **yardstick and the fallback**: building its demo and its
|
||
version of the transcript screen first says what a finished stack costs
|
||
on this hardware, proves android-view before iris depends on it, and
|
||
gives a comparison that is measured rather than remembered.
|
||
|
||
Not considered further: **GPUI** (Zed) mobile is a community fork that
|
||
depends on unpublished crates; **Dioxus/Blitz** is excluded by Iris (its
|
||
native renderer is Parley/Vello under HTML semantics, and the earlier
|
||
`tdep-survey/app-dioxus` spike parked it on a `vello_hybrid` stroke bug and
|
||
shipped the WebView); **Compose Multiplatform Desktop** would give a desktop
|
||
app for nothing but in Kotlin, which is the opposite direction.
|
||
|
||
### Weight and debug builds
|
||
|
||
Iris remembers the Linebender stack being slow in debug. What is behind
|
||
that is the dependency graph — Vello, wgpu, Parley, Fontique, Skrifa —
|
||
running unoptimised on the CPU side (path encoding, shaping), not the
|
||
widget layer. Xilem's own advice is only `split-debuginfo = "unpacked"` to
|
||
keep `target/` small; the fix everyone with this shape of dependency tree
|
||
uses is to optimise dependencies while leaving the app crate at `opt-level
|
||
= 0`:
|
||
|
||
[profile.dev.package."*"]
|
||
opt-level = 2
|
||
|
||
**Measured 2026-09-04, and it is not the widget layer.** iris's own
|
||
graph (wgpu + winit + cosmic-text at the time) built cold in **43s** with
|
||
a 2.1 GB `target/`, and **1m46s** with a 1.5 GB `target/` under the
|
||
profile above — so the knob costs build time and saves disk here, and
|
||
plain debug was never the problem. Masonry's graph is the one with Vello,
|
||
Parley, Fontique and Skrifa in it, and E1 gives the number that matters
|
||
for it: **`libmain.so` is 181 MB in debug and 11 MB in release.** That
|
||
size is also a correctness issue rather than only a weight one — a debug
|
||
build labels its Vulkan objects, and the emulator's driver segfaults in
|
||
`SetDebugUtilsObjectNameEXT` when it does. Runtime cost of the profile
|
||
knob is still unmeasured; resident memory and the frame cost of an
|
||
800-event page want E2. Vello proper needs compute shaders and carries a
|
||
large shader set; `vello_hybrid` is lighter and Masonry can now render
|
||
through either (or Vello CPU) via its `imaging` abstraction, so "keep it
|
||
light" has a knob inside the same stack.
|
||
|
||
## Recommendation
|
||
|
||
1. **Build `client-core` now, whatever the framework.** A Rust crate holding
|
||
the event model (shared with `server/` as one crate, ending the
|
||
`Events.kt` mirror), the API and SSE clients, the transcript fold, the
|
||
cache, the markdown block model, the highlighter and the ANSI parser,
|
||
with the existing JVM tests ported. It is the part of the app that is
|
||
already tested, already logic, and already duplicated on the server.
|
||
2. **One foundation, two widget layers.** The platform plumbing is shared
|
||
whichever way the decision goes: android-view for the Android surface,
|
||
keyboard and accessibility bridge; `wgpu` for the GPU; AccessKit for
|
||
names; winit on the desktop. On top of it, **Masonry as the yardstick**
|
||
(E1, E2) and **iris as the thing being built** (I0–I5), both aimed at
|
||
the same transcript screen with the same pass conditions.
|
||
3. **Decide when the transcript screen exists in both**, from the
|
||
measurements, and record the decision here with the numbers. If iris
|
||
carries the screen within the Compose baseline, it is the app's
|
||
framework and Masonry was the calibration. If it does not, the
|
||
measurement says which parts of Masonry to adopt underneath it.
|
||
4. Then the shell (E3), the desktop window (E4) and the packaging (E5),
|
||
which do not depend on the choice.
|
||
|
||
## Experiments, in order
|
||
|
||
Each has a pass condition that is a measurement in this clone. The rig
|
||
matters: this emulator runs `-gpu host` with **host Vulkan switched off**
|
||
(`GPU_HOST_FEATURES` in `emulator-tools`, a gfxstream/Venus gap), so inside
|
||
the guest a `wgpu` app gets GLES, not Vulkan; the earlier Dioxus spike also
|
||
needed `WGPU_GLES_MINOR_VERSION=1` for compute shaders and found `wgpu`'s
|
||
Android backend wants API 26 (a libc symbol). The real phone has Vulkan.
|
||
Per the standing rule, a rig limit is something to fix before it is
|
||
accepted.
|
||
|
||
- [x] **E0 — toolchain (done 2026-09-04).** Installed under the
|
||
user-owned SDK: **NDK r29 (`29.0.14206865`)**, 2.4 GB at
|
||
`~/Android/Sdk/ndk/29.0.14206865`, the newest stable — r30 is still
|
||
at rc.3. **cargo-ndk 4.1.2**. Verified by cross-compiling a scratch
|
||
`cdylib` to both ABIs: `file` reports "for Android 26, built by NDK
|
||
r29 (14206865)" for `aarch64-linux-android` and
|
||
`x86_64-linux-android`. Two things to know at the call site.
|
||
**cargo-ndk 4's API-level flag is `-P`, not `-p`** — `-p` is now
|
||
passed through to cargo as `--package`, so the old
|
||
`cargo ndk -t arm64-v8a -p 26` panics with `unknown package: 26`
|
||
*and dumps the whole environment to stdout* as a bug report, which is
|
||
worth not doing in a log somebody might paste. And the Android
|
||
targets were installed for **stable** only; the pinned nightly needs
|
||
its own, which `iris/rust-toolchain.toml` now declares.
|
||
- [x] **E1 — android-view's Masonry demo on this emulator (2026-09-04).**
|
||
It builds, renders on the GPU through Vulkan, exposes its
|
||
accessibility tree, and **the phone's own keyboard types into its
|
||
editor** — but with **no autocorrect and no suggestions**. Ticked
|
||
because everything it was meant to establish is established,
|
||
including the one gap; that gap is now E2's problem and I2's.
|
||
|
||
*Build.* `~/src/android-view` at `bec6c62`, x86_64 rather than the
|
||
README's arm64 because that is what this emulator is:
|
||
`cargo ndk -t x86_64 -P 26 -o masonry-app/src/main/jniLibs/ build -p
|
||
android-view-masonry-demo --release`, then
|
||
`./gradlew :masonry-app:assembleDebug`. **`libmain.so` is 181 MB in
|
||
debug and 11 MB in release** — the loudest single number about
|
||
Vello's dependency graph, and the reason the release build matters
|
||
for more than speed.
|
||
|
||
*Renderer.* wgpu takes **Vulkan**, and the emulator log confirms it
|
||
from the other side: `Created VkDevice ... for application:'wgpu'`.
|
||
Two things were needed. The emulator must be given Vulkan at all —
|
||
`-feature Vulkan` with `VK_DRIVER_FILES` pointing at the SDK's
|
||
`vk_swiftshader_icd.json`, **plus `-no-snapshot-load`**, which is the
|
||
piece this file had flagged as untested: without a cold boot the
|
||
guest keeps the snapshot's old GPU config and `cmd gpu vkjson`
|
||
reports zero devices however the host is configured. And the native
|
||
library must be **release**: a debug build calls
|
||
`SetDebugUtilsObjectNameEXT` to label its image views, and the
|
||
emulator's own guest driver (`vulkan.ranchu.so`) segfaults inside it.
|
||
On GLES, with no Vulkan available, it instead fails
|
||
`Surface::configure` with "Invalid surface" — untriaged, since the
|
||
Vulkan path works and Vello wants compute shaders anyway.
|
||
|
||
*Accessibility works*, which E2's condition 6 and every bench script
|
||
depend on. `ui-trace` reads Masonry's AccessKit tree: "Add task"
|
||
arrives as a named `Button`, the editor as an `EditText` node. So
|
||
tap-by-name works against a Masonry screen for any control carrying
|
||
a name; the demo's editor carries none, which is the demo's omission
|
||
rather than the framework's.
|
||
|
||
*The keyboard: real input yes, suggestions no.* Tapping the editor
|
||
opens the actual soft keyboard (`mInputShown=true`, Gboard), and
|
||
tapping its keys types into Masonry — "teh" typed key by key, with a
|
||
caret. What does **not** appear is Gboard's suggestion strip. The
|
||
control is what makes that a finding rather than an impression: the
|
||
**same three key taps in the Settings app's search field, on the same
|
||
device in the same session, produce "teh | the | yeh"**. So the strip
|
||
works here and android-view's editor is not asking for it — most
|
||
likely the `EditorInfo` its `InputConnection` reports. That matches
|
||
Robrix's report that the Android keyboard is not yet "full", and it
|
||
is the single most important thing to fix or fund upstream, because
|
||
composition, autocorrect and suggestions are exactly what the
|
||
composer in this app needs and exactly what `winit` cannot do at all.
|
||
|
||
*One crash seen once and not reproduced.* With an accessibility
|
||
client attached the app aborted, stack:
|
||
`android_view::view::do_frame` → `CallbackCtx::finish` →
|
||
`accesskit_android::event::QueuedEvents::raise` →
|
||
`send_completed_event` → `unwrap()` on `Err(JavaException)`.
|
||
android-view builds `panic = "abort"`, so a JNI call that throws
|
||
takes the process. Two later `ui-trace record` runs left the app
|
||
alive, so the trigger is narrower than "a client is attached".
|
||
|
||
*A rig trap that cost a wrong conclusion.* Several bounded runs were
|
||
given `sleep N; emu down` watchdogs, and one armed for an earlier
|
||
experiment fired in the middle of a later one — the app vanished, adb
|
||
hung, and it read exactly like the Vulkan path crashing. It was not.
|
||
A watchdog must be scoped to the process it guards (`kill $pid`, with
|
||
the pid captured at launch) rather than to whatever AVD is running
|
||
when it wakes, and only one should be armed at a time.
|
||
|
||
- [ ] **E2 — a transcript in Masonry.** One screen: open a sandbox session,
|
||
page 800 events into `VirtualScroll` bottom-anchored, draw markdown
|
||
from `pulldown-cmark` into Parley rich text with links and code
|
||
chips, select across two rows with the platform handles, expand a
|
||
tool row holding its top edge. Pass: the render numbers land within
|
||
the Compose baseline in `transcript-bench.sh` on the GPU emulator
|
||
(same gestures, same session), and every one of the seven behaviours
|
||
above is either shown or has a written reason it cannot be.
|
||
- [ ] **E3 — the shell.** Kotlin `MainActivity` + `NotificationService`
|
||
+ Keystore + share intent calling into Rust over JNI, with the SSE
|
||
follow loop in Rust. Pass: a notification arrives with the app
|
||
closed, and a share lands in a session.
|
||
- [ ] **E4 — the same screen on the desktop** in a winit window, from the
|
||
same crate, with only the layout differing.
|
||
- [ ] **E5 — the packaging xtask**: `cargo ndk` → `javac`/`d8` → `aapt2`
|
||
→ `zipalign` → `apksigner`, signed with the existing release key,
|
||
installed through Dev Updater. Pass: the APK installs over the
|
||
Gradle-built one and the notification service starts.
|
||
|
||
### The iris track
|
||
|
||
These build iris up to carry the app. Each is a feature added to iris
|
||
with a pass condition, in dependency order. Work in `iris/` in this
|
||
repository on the `rustify` branch, and record in this file what each
|
||
step measured.
|
||
|
||
- [x] **I0a — where iris lives (decided 2026-09-04).** For now it is
|
||
**vendored at `iris/` in this repository**, history not carried,
|
||
and consumed by path. Iris's decision: keep it close while it is
|
||
being reshaped for this app, and give it back its own repository —
|
||
`iris/iris` on the gitea remote, which already holds the full
|
||
244-commit history, on a branch of its own — once it has proved
|
||
itself. The vendored tree is that repository's `main` at
|
||
`7b54aaf` ("readme", 2026-01-29), byte-identical to the public
|
||
GitHub copy, so a later reconciliation has a known base. A crate
|
||
that uses it says `iris = { path = "../iris" }`.
|
||
- [x] **I0b — make it build here (done 2026-09-04).** iris now builds,
|
||
clippy-clean and rustfmt-clean at the defaults, on a pinned dated
|
||
nightly, and the `tabs` example draws on this VM's GPU.
|
||
|
||
**The pin** is `nightly-2026-09-03` (rustc 1.100.0-nightly,
|
||
`2e2b193f8`), declared in `iris/rust-toolchain.toml` along with the
|
||
`clippy`/`rustfmt` components and the two Android targets, so a
|
||
fresh clone provisions itself. It is dated rather than `nightly`
|
||
because the whole failure below was a rolling channel moving under
|
||
an unattended build. Installed with `--profile minimal`: 912 MB.
|
||
|
||
**The 36 errors were one syntax change, and the earlier diagnosis in
|
||
this file was wrong.** It is not that a trait must now be declared
|
||
`const trait` — the vendored tree already declares them that way,
|
||
which is how it was written in January. What changed is the *impl*
|
||
keyword order: `impl const Trait for T` is now
|
||
`const impl Trait for T`, and generics go on the `impl`
|
||
(`const impl<T: [const] Foo> Bar for T`). Bounds are unaffected;
|
||
`T: const Foo`, `T: [const] Foo` and `impl const Foo` in argument
|
||
position all still compile. Everything else — the unresolved
|
||
`UiVec2`/`Vec2`/`impl_op` imports, and a `Color<u8>` that resolved
|
||
to `wgpu_types::Color` — cascaded from the seven files that failed
|
||
to parse. The rewrite was mechanical across 20 sites and took the
|
||
workspace from 36 errors to 0.
|
||
|
||
**`#![feature]` gates, 12 after this step** (two were declared and
|
||
unused, and were removed: `map_try_insert`, `const_cmp`).
|
||
Load-bearing and worth watching: `const_trait_impl`, `const_ops`,
|
||
`const_convert`, `const_destruct` are the const-traits family and
|
||
the one that has already broken once — they move together, so
|
||
advancing the pin means re-reading this section. `unboxed_closures`
|
||
+ `fn_traits` (postfix builder API) and `unsize` +
|
||
`coerce_unsized` (widget handles) are pairs. The rest are
|
||
individually small: `macro_metavar_expr_concat`, `portable_simd`,
|
||
`associated_type_defaults`, `option_into_flat_iter`, and `gen_blocks`
|
||
in the top crate.
|
||
|
||
**Running it headless.** `iris/run-headless.sh EXAMPLE [--shot PNG]`
|
||
with `iris/headless.conf`, the same trick `emu` uses: a headless
|
||
sway, and `grim` for the picture. It deliberately starts its *own*
|
||
compositor rather than joining `emu`'s — sway tiles, so adding a
|
||
window to the one an emulator sits in resizes that emulator.
|
||
Unlike `emu`'s it disables Xwayland, since winit speaks Wayland.
|
||
**This VM has a real GPU for this**: Vulkan 1.4 through Venus onto
|
||
the host's RX 7900 XT, and GL 4.6 through virgl — so desktop wgpu
|
||
work here is not software-rasterised, unlike inside the emulator.
|
||
|
||
**iris has no tests at all** (`cargo test --workspace`: 0 passed
|
||
across 6 targets). Nothing to keep passing, and nothing to catch a
|
||
regression — worth knowing before I1 changes the text stack.
|
||
|
||
**`iris-core` no longer depends on winit, and now cross-compiles to
|
||
Android.** It wanted exactly one thing from it — `PhysicalSize<u32>`
|
||
in `UiRenderNode::resize`'s signature, for two numbers it immediately
|
||
turned into floats — and that pulled a whole windowing backend into
|
||
the layer below it, the wrong direction. `resize` takes
|
||
`impl Into<Vec2>` now, like `UiRenderState::resize` beside it already
|
||
did. The consequence is the point: with winit in the graph an Android
|
||
build of the core failed in `android-activity` (which needs a backend
|
||
feature nothing here selects), and without it
|
||
`cargo ndk -t arm64-v8a -P 26 build -p iris-core` finishes in 30s and
|
||
produces an rlib, wgpu's Android backend included. So **iris's
|
||
widget, layout and render core already builds for the phone**, and
|
||
what I2 has to supply is the surface, the input and the IME — not a
|
||
port of the library.
|
||
|
||
**Build weight, cold, on this VM's 8 cores** (`rm -rf target`, then
|
||
`cargo build --example tabs`), since "the Linebender stack is slow in
|
||
debug" was the worry behind this question: plain debug **43s** and a
|
||
2.1 GB `target/`; with the `[profile.dev.package."*"] opt-level = 2`
|
||
knob, **1m46s** and 1.5 GB. So iris's own wgpu + winit + cosmic-text
|
||
graph is not the slow thing — which makes it a calibration for E1
|
||
rather than an answer about Masonry, whose graph adds Vello, Parley,
|
||
Fontique and Skrifa. Runtime cost of the knob was not measured here.
|
||
|
||
**Fixed: iris never called `pre_present_notify`.** The symptom was
|
||
that about one start in five kept the window's 800x600 startup layout
|
||
on a 1920x1200 surface for good. What settled it was tracing iris's
|
||
own decisions into memory and dumping them from another thread —
|
||
`eprintln!` in the draw path makes the defect vanish, which is why
|
||
earlier attempts kept losing it. The traces from a good and a bad run
|
||
are **byte-identical**: both lay out and draw `redraw_all at
|
||
(1920, 1200)` into a 1920x1200 texture with `suboptimal=false`. iris
|
||
was drawing the right frame every time; the compositor was still
|
||
showing the first one, and forcing a full repaint did not shift it.
|
||
What was missing is winit's `Window::pre_present_notify`, called
|
||
immediately before `present`, which on Wayland is what ties the
|
||
commit to the surface's frame callback. Without it a frame drawn with
|
||
nothing following it can sit unpresented with nothing left to flush
|
||
it — which is exactly a window that has just settled after its
|
||
opening resize. Measured: **0 bad in 40** with the fix, against 4 in
|
||
20 before it, and — the stronger evidence — 0 in 20 in the
|
||
instrumented configuration that had been 15 in 20. Runtime resizing
|
||
still round-trips to a byte-identical layout.
|
||
|
||
Two things ruled out on the way, both worth not re-trying: the
|
||
present mode (the fault survived the move from `AutoNoVsync` to
|
||
`AutoVsync` at the same rate) and the size cache (`redraw_all` clears
|
||
it). A `desired_maximum_frame_latency` of 1 moved the rate without
|
||
fixing it, and was reverted. Iris's own note that she had never seen
|
||
the library fail to resize was the useful steer: it pointed away from
|
||
the layout code, where two hours had already gone.
|
||
|
||
One thing was fixed on the way, and it is not that bug: `update`
|
||
redrew everything when `resized` was set, but `needs_redraw` — which
|
||
is what decides whether to *ask* for a frame — did not know about
|
||
`resized` at all. The two now share one `needs_redraw_all`, since a
|
||
condition in one and not the other is a frame nobody requests. It is
|
||
latent on Wayland only because winit asks for a redraw after a resize
|
||
by itself; on Android, where the surface work of I2 will not have
|
||
winit underneath it, nothing else here would have asked.
|
||
- [x] **I1 — parley, and a glyph atlas (done 2026-09-04).** No bake-off:
|
||
Iris decided for parley directly ("I wanted to switch it to parley
|
||
anyways"), and then asked for the atlas as well ("just do the atlas,
|
||
commit to it, we do want it"). Both are in.
|
||
|
||
**What parley bought, beyond shaping.** Its editing model addresses
|
||
text by byte offset into one string, where cosmic-text used
|
||
`(line, index)` — so `select_content`, `delete_between`,
|
||
`insert_inner` and `newline` collapse into ordinary string
|
||
operations. Bigger: `Selection::geometry` and `Cursor::geometry`
|
||
replace `iter_layout_lines`, `index_x` and `cursor_pos`, which walked
|
||
runs by hand to place the caret and the selection boxes and were not
|
||
bidi- or wrap-correct. `edit.rs` lost about 130 lines and gained
|
||
Home/End. Its cursor motions map onto parley's `next_visual`,
|
||
`previous_visual_word`, `next_line` and so on, in one function.
|
||
|
||
**The atlas is what "text resizing (per frame) is really slow"
|
||
was.** Every string used to be rasterised into its own `RgbaImage`
|
||
and uploaded as a whole texture whenever anything about it changed,
|
||
so a window resize re-rasterised and re-uploaded every visible
|
||
string. Now a glyph is rasterised once per font, size and subpixel
|
||
phase, shared by every string that contains it, and a resize
|
||
re-emits quads without touching the GPU's copy. **The tabs example
|
||
reports it: `views`, the number of texture views bound, went from 6
|
||
to 1** — six per-string textures became one shared page. Supporting
|
||
pieces: a `GLYPH` primitive that samples a sub-rectangle and tints
|
||
it (the existing texture primitive samples a whole texture), a
|
||
`Patch` texture update so a new glyph costs its own bytes rather
|
||
than a 4 MB page, and `GpuTextures` keeping its `Texture`s, since a
|
||
view cannot be written through.
|
||
|
||
**Not yet measured**, and the honest gap in this step: the TODO's
|
||
"really slow" was never given a number, so neither is the
|
||
improvement. What is evidence rather than argument is the view count
|
||
and the shape of the work — a resize no longer rasterises. A
|
||
before/after timing wants the transcript screen of I5 to be worth
|
||
taking.
|
||
|
||
**Two bugs found on the way**, both pre-existing: `primitives!`'s
|
||
`@count` rule recursed comma-separated while matching
|
||
space-separated, so it terminated only for exactly two primitives
|
||
and adding a third hit the recursion limit; and `Color` had no
|
||
`Default`, which parley's `Brush` requires.
|
||
|
||
**Fourteen tests**, iris's first. The editor is the one part that is
|
||
pure logic rather than something needing a GPU and a window, and it
|
||
was rewritten wholesale with no way to exercise it — synthetic input
|
||
does not reach a client under the headless compositor, which has no
|
||
seat devices. Two of the tests are aimed at what the rewrite could
|
||
plausibly have broken: the IME preedit path, and editing multi-byte
|
||
text now that offsets are bytes.
|
||
|
||
Dropping cosmic-text and unicode-segmentation also retired two
|
||
nightly gates — `portable_simd` (the old glyph compositing) and
|
||
`gen_blocks` (the deleted line iterator). **Eleven left.**
|
||
|
||
- [ ] **I2 — iris on android-view.** An `android-view` surface as a second
|
||
backend beside winit: `wgpu` on the view's surface (GLES here, see
|
||
the Vulkan section; Vulkan on the phone), touch as pointer events,
|
||
window insets and the keyboard inset as layout inputs, the back
|
||
gesture as an event, the IME bridge feeding the editor from I1.
|
||
Pass: the `tabs` example and a text field run on the emulator, and
|
||
the phone's own keyboard types into the field with autocorrect and
|
||
suggestions — the same bar as E1.
|
||
- [ ] **I3 — a virtualised, bottom-anchored list.** Variable-height rows,
|
||
keyed, composed only while visible, paged in both directions with a
|
||
"more" sentinel at each end, a scroll anchor that survives rows
|
||
being inserted above, and "hold the edge nearest the tap" done in
|
||
the layout pass. Pass: 800 rows of real transcript text from the
|
||
sandbox scroll without a frame over the Compose baseline in
|
||
`transcript-bench.sh`, measured on the GPU emulator.
|
||
- [ ] **I4 — accessibility names via AccessKit.** Every control carries a
|
||
name; `ui-trace` can find and tap it by label. Pass: `bench-lib.sh`'s
|
||
tap-by-name works against the iris screen unchanged.
|
||
- [ ] **I5 — the transcript screen in iris.** E2's pass conditions, all
|
||
seven behaviours, against the sandbox with `--delay`. This is the
|
||
point the decision in the recommendation is made at.
|
||
|
||
## For the next agent
|
||
|
||
What to do when you pick this up, in order, so nothing here has to be
|
||
re-derived:
|
||
|
||
1. Read this file, then `AGENTS.md` and `PLAN.md`. The rules there
|
||
(measure, do not read; fix the rig before accepting its limits; the
|
||
emulator is this checkout's own) all apply.
|
||
2. Work on the **`rustify`** branch of this clone (`ai-app-2`), not on
|
||
`main` and not in `ai-app`. Nothing on this branch is production until
|
||
Iris says so. Commit and push as you go.
|
||
3. Take the next unchecked box above, in order. E1 has proved android-view
|
||
on this emulator, so the E-steps and the I-steps can now proceed in
|
||
parallel in separate sessions; see "Where things stand" at the top for
|
||
which is next.
|
||
4. Every step ends with its measurement written into this file beside the
|
||
box, and the box ticked or the reason it could not be written in its
|
||
place. A step that is blocked says by what, not "later". Write it as you
|
||
go rather than at the end — see "Keep this file current as you work".
|
||
5. Run the existing rigs rather than inventing new ones: `ui-sandbox.sh`
|
||
for a server with fixtures, `transcript-bench.sh` for the scroll
|
||
baseline, `ui-trace` for anything positional, `emu up` for the
|
||
emulator, and `iris/run-headless.sh EXAMPLE --shot PNG` for an iris
|
||
example on this displayless machine. The Vulkan section below says how
|
||
to get a Vulkan path in the emulator when a `wgpu` backend needs one.
|
||
6. **Bound anything heavy at the moment you start it.** An emulator or a
|
||
long build gets a deadline — `timeout`, or a watchdog scoped to the pid
|
||
you just started — rather than a plan to stop it later. Scope it to
|
||
that pid: a watchdog written as `sleep N; emu down` fired into a later
|
||
experiment here and made a working Vulkan build look like a crash. And
|
||
stop the emulator when the work needing it is done rather than between
|
||
tasks.
|
||
7. Decisions belong here with a date and what was rejected, the way
|
||
`PLAN.md` does it. Do not put design into commit messages alone.
|
||
|
||
### Vulkan in the emulator (measured 2026-09-04)
|
||
|
||
**Settled 2026-09-04: the guest gets Vulkan from SwiftShader, and the
|
||
missing step was a cold boot.** `-feature Vulkan` plus
|
||
`VK_DRIVER_FILES=$HOME/Android/Sdk/emulator/lib64/vulkan/vk_swiftshader_icd.json`
|
||
gets the *host* side to select SwiftShader, but the guest keeps reporting
|
||
zero devices until `-no-snapshot-load` is added, because it boots from a
|
||
snapshot saved under the previous GPU config — `-no-snapshot-save` is
|
||
worth adding too, so the Vulkan-configured snapshot does not then break
|
||
the next ordinary boot. With that, `cmd gpu vkjson` reports SwiftShader
|
||
Subzero and wgpu takes its Vulkan path (E1). `EMU_GPU=software` in
|
||
`emulator-tools` gets the same guest Vulkan with no GPU use at all, for
|
||
work where the emulator's frame rate is not what is being measured.
|
||
|
||
The rest of this section stands as the record of why host Vulkan is not
|
||
available. A `wgpu` app in this emulator was going to get GLES only,
|
||
because host Vulkan is switched off in `emulator-tools`. Retried on Mesa
|
||
26.1.7: **Venus still fails the same way** — gfxstream picks
|
||
`externalMemoryMode: OpaqueFd`, probes `VK_FORMAT_R8G8B8A8_UNORM` for an
|
||
exportable colour buffer, and Venus says the format is unsupported
|
||
(`Failed to find memory type for ColorBuffers`, fatal before adb sees the
|
||
device). Venus does advertise `VK_KHR_external_memory_fd` and
|
||
`VK_EXT_external_memory_dma_buf`, so the gap is specifically opaque-fd
|
||
image export. gfxstream has a string-valued `VulkanExternalMemoryMode`
|
||
setting ("overrides what would otherwise be determined automatically"),
|
||
but `-feature Name=Value` is rejected as a bad feature name, and the only
|
||
mode words compiled into this emulator's `libgfxstream_backend.so`
|
||
(37.1.11) are `OpaqueFd`, `Metal` and `none` — there is no dma-buf mode
|
||
in this build to switch to. So Venus is blocked by the emulator, not by
|
||
Mesa; retry when the emulator package updates, since upstream gfxstream
|
||
does have dma-buf external memory.
|
||
|
||
What **does** work: pointing the emulator's Vulkan loader at the software
|
||
ICDs the emulator ships itself, with the feature enabled:
|
||
|
||
VK_DRIVER_FILES=$HOME/Android/Sdk/emulator/lib64/vulkan/vk_swiftshader_icd.json \
|
||
GPU_HOST_FEATURES="-feature Vulkan" emu up
|
||
|
||
The guest then reports Vulkan 1.3 (`cmd gpu vkjson`, SwiftShader
|
||
Subzero) while GLES still runs on the real GPU through virgl — so a
|
||
Vello/wgpu app can take its real Vulkan path here, with compute shaders,
|
||
CPU-rasterised. That is enough to test *correctness* of the Vulkan path
|
||
in the emulator; GPU *performance* of it is a phone measurement either
|
||
way, exactly as `MACHINE.md` already says about frame times. **lavapipe**
|
||
(`lvp_icd.json`, the other ICD the emulator ships) selected llvmpipe and
|
||
booted, then the emulator died right after loading the `default_boot`
|
||
snapshot with nothing in the log; a snapshot saved under a different
|
||
Vulkan device is the suspect, and `-no-snapshot-load` is the untested
|
||
next step. SwiftShader is the one that works today.
|
||
`EMU_GPU=software` is now in `emulator-tools` (agreed with the ai-app
|
||
session and with Iris, default unchanged, since `-gpu host` was measured
|
||
and the Compose scroll benchmarks depend on it). The cold-boot flags are
|
||
not a knob there: that wants snapshot invalidation as well, which is a
|
||
bigger design question in shared tooling.
|
||
|
||
## Things a Rust app changes elsewhere
|
||
|
||
- **`wg-app-link`'s `:link`** (pinned TLS, enrollment store, QR activity)
|
||
is Kotlin shared with Dev Updater. The certificate code already exists on
|
||
the Rust side of the submodule; the pinned-CA build step
|
||
(`generatePinnedCert`) becomes a `build.rs` reading the same path. The QR
|
||
scanner stays a Kotlin activity, since the camera is a platform feature.
|
||
- **Tooling** becomes `cargo` for everything but packaging: `cargo test`,
|
||
`clippy`, `fmt` cover the whole client, which is the motivation. Gradle
|
||
remains for the APK, signing (`~/.config/ai-app/release.jks`) and Dev
|
||
Updater's build modes; `build-apk.sh` would call `cargo ndk` first.
|
||
- **The bench scripts** (`ui-trace` by accessibility label) keep working
|
||
only if the framework exposes names through AccessKit on Android; that is
|
||
part of E2's pass condition, not a nicety.
|
||
- **Icons** stay Nerd Font glyphs from the committed subset; Parley/Fontique
|
||
loads a font file directly, so `build-icon-font.sh` is unchanged.
|
||
|
||
## Sources
|
||
|
||
- iced: [repo](https://github.com/iced-rs/iced), [0.14 release](https://github.com/iced-rs/iced/releases/tag/0.14.0), [Android thread](https://news.ycombinator.com/item?id=46350641), [markdown selection request](https://discourse.iced.rs/t/markdown-widgets-text-should-be-selectable/1107)
|
||
- Linebender: [2026 Q1 report](https://linebender.org/blog/tmil-25/), [xilem](https://github.com/linebender/xilem), [parley](https://github.com/linebender/parley), [vello](https://github.com/linebender/vello), [vello_hybrid](https://docs.rs/vello_hybrid/latest/vello_hybrid/)
|
||
- android-view: [repo](https://github.com/rust-mobile/android-view); android-activity [PR #214](https://github.com/rust-mobile/android-activity/pull/214)
|
||
- winit Android IME: [#1823](https://github.com/rust-windowing/winit/issues/1823), [#2766](https://github.com/rust-windowing/winit/issues/2766), [#2305](https://github.com/rust-windowing/winit/issues/2305)
|
||
- egui on Android: [discussion #2053](https://github.com/emilk/egui/discussions/2053)
|
||
- Slint: [Android guide](https://docs.slint.dev/latest/docs/slint/guide/platforms/mobile/android/), [1.15 release](https://slint.dev/blog/slint-1.15-released), [licensing](https://slint.dev/faqs), rich text [#1325](https://github.com/slint-ui/slint/issues/1325), markdown [#6684](https://github.com/slint-ui/slint/issues/6684)
|
||
- Makepad: [repo](https://github.com/makepad/makepad), [makepad-widgets](https://docs.rs/makepad-widgets), [Robrix](https://github.com/project-robius/robrix), [Robrix releases](https://github.com/project-robius/robrix/releases)
|
||
- AccessKit: [releases](https://github.com/AccessKit/accesskit/releases)
|
||
- Build tools: [cargo-ndk](https://github.com/bbqsrc/cargo-ndk), [cargo-apk](https://github.com/rust-mobile/cargo-apk), [rust-mobile](https://github.com/rust-mobile)
|
||
- uniffi: [repo](https://github.com/mozilla/uniffi-rs), [KMP bindings fork](https://github.com/UbiqueInnovation/uniffi-kotlin-multiplatform-bindings)
|
||
- GPUI mobile: [gpui-mobile](https://github.com/itsbalamurali/gpui-mobile)
|
||
- The earlier Dioxus spike's findings on `wgpu`/Vulkan in this emulator: `~/repos/tdep-survey/app-dioxus/README.md`
|