E0 is done (NDK r29, cargo-ndk 4.1.2, verified by cross-compiling to both ABIs) and I0b is done. Corrects this file's guess at why iris would not build, notes the const-traits family as the gates to re-read whenever the pin is advanced, and records the cold build weight against the "slow in debug" worry: 43s and 2.1 GB plain, 1m46s and 1.5 GB with dependencies at opt-level 2. Also records an open defect found on the way -- iris sometimes keeps its pre-configure window size for good -- with what was ruled out, since it is timing-sensitive enough that any added print hides it, and I2 will meet it on every rotation. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
708 lines
42 KiB
Markdown
708 lines
42 KiB
Markdown
# Moving the app to Rust
|
||
|
||
Working document for the question Iris asked on 2026-09-04: what are the
|
||
options for switching the phone app to Rust, ideally pure Rust with one UI
|
||
framework shared with a future winit-based desktop application, at full
|
||
feature parity and without giving up anything native, performance
|
||
especially. Constraints she set: no Dioxus and nothing that draws through a
|
||
WebView; **no UI DSL** (which rules out Makepad and Slint); the result
|
||
should stay lightweight; platform-specific pieces are fine to maintain;
|
||
reimplementing a framework piece from scratch where it does not fit is
|
||
fine; effort and elapsed time do not matter, long-term robustness does;
|
||
this clone is where things get tried before anything is committed to
|
||
`ai-app`. Her own library, [iris](https://github.com/cat16/iris), is the
|
||
**in-house framework to be built up** for this, with Masonry as the
|
||
yardstick it is measured against.
|
||
|
||
Decisions get a date and a reason here, the way `PLAN.md` does. Nothing in
|
||
this file has been tried yet unless a section says it has.
|
||
|
||
## What has to be reproduced
|
||
|
||
The app is ~19,000 lines of Kotlin. It splits three ways, and the split is
|
||
what decides how much of a port is mechanical.
|
||
|
||
**Pure logic with no Compose or Android in it, ~4,500 lines.** `Api.kt`
|
||
(1,142), `Events.kt`, `EventStream.kt`, `Sse.kt`, `TranscriptCache.kt`
|
||
(589, touches `java.io.File` only), `TranscriptSource.kt`,
|
||
`MarkdownSyntax.kt`, `Languages.kt`, `Highlighter.kt`, `Ansi.kt`,
|
||
`ResetCountdown.kt`, `Durations.kt`, `Sizes.kt`, `ModelName.kt`,
|
||
`LoadState.kt`, `ImportableStream.kt`. `TranscriptUnits.kt` and
|
||
`TranscriptItems.kt` (the event fold into rows, ~940 lines) are logic with
|
||
a handful of Compose annotations. This is also exactly the code that has
|
||
JVM unit tests today. All of it ports directly, and most of it already has a
|
||
Rust twin in `server/`: `Events.kt` is a hand-kept mirror of
|
||
`session/driver.rs`'s enum, the highlighter and the syntax scanner exist on
|
||
the server for the explorer, and the cache compares the server's own JSON
|
||
lines. **Sharing these types between server and app is the single largest
|
||
"keep things in sync" win available, and it does not depend on which UI
|
||
framework wins.**
|
||
|
||
**Compose UI, ~13,000 lines.** Screens, dialogs, the transcript list, the
|
||
markdown renderer's customisations, tool cards, the file explorer viewer and
|
||
editor. This is the part a UI framework choice is about.
|
||
|
||
**Android platform code, ~1,500 lines**, spread over 20 files. Every one of
|
||
these is a Java-side object that no Rust framework can replace, because
|
||
Android only offers them as Java classes:
|
||
|
||
- `NotificationService` — a **foreground service** holding the
|
||
`/notifications` SSE stream while the app is closed, with its ongoing
|
||
notification, `specialUse` type and the `POST_NOTIFICATIONS` request.
|
||
- `MainActivity` — edge-to-edge, the `ACCESS_LOCAL_NETWORK` runtime
|
||
permission (Android 17), `singleTop` intent routing for `aiapp://enroll`,
|
||
notification taps, and the **share sheet** (`ACTION_SEND`, any MIME type).
|
||
- `ServerConfig` — the bearer token sealed under an **Android Keystore**
|
||
AES-GCM key, shared with Dev Updater through `wg-app-link`'s `:link`.
|
||
- `EnrollmentScanActivity` — the in-app **QR scanner** (zxing, camera).
|
||
- `Attachments` — `ContentResolver` reads of shared URIs, `BitmapFactory`
|
||
decode and downscale, **EXIF** orientation.
|
||
- `SessionImage` — bitmap decode for produced images.
|
||
- `ScrollAnchor`, `Drafts` — `SharedPreferences`; `CrashLog` — `filesDir`.
|
||
- `TranscriptCache` — `cacheDir`.
|
||
- `DebugStats`/`FrameStats` — `Choreographer` frame timing and the render
|
||
report; `runtime-tracing` names composables in a system trace.
|
||
|
||
So **"pure Rust" on Android means Rust owns every line of logic and
|
||
drawing, behind a thin shell of Java stubs**, and a packaging step that
|
||
produces a signed APK. How thin, and whether Gradle is inevitable, are
|
||
answered below.
|
||
|
||
### How much Java is unavoidable, and why
|
||
|
||
Rust can *call* any Android API through JNI (`jni` crate, with
|
||
`ndk-context` handing over the `JavaVM` and the Activity): posting a
|
||
notification, `startForegroundService`, the Keystore, `ContentResolver`
|
||
reads, permission requests, `WindowInsets`, the clipboard. None of that
|
||
needs a line of Kotlin. What JNI cannot do is *define* a class that the
|
||
system instantiates **by name from the manifest** — an `Activity`, a
|
||
`Service`, an `Application`, a `BroadcastReceiver`. Those must exist as dex
|
||
bytecode inside the APK before any Rust runs, because the framework
|
||
constructs them and only then calls into native code. `NativeActivity` is
|
||
the platform's own stub for the Activity case; there is no
|
||
`NativeService`, and android-view ships its own `View` subclass for the
|
||
same reason.
|
||
|
||
So the floor is roughly **two Java classes of ten lines each**: an
|
||
`Activity` and a `Service` whose lifecycle methods are declared `native`
|
||
and registered from `JNI_OnLoad`, plus whatever android-view already
|
||
provides. Everything they would have done in Kotlin — insets, intent
|
||
routing, the SSE follow loop, the notification builder — is Rust reached
|
||
through those stubs. Writing the stubs in Java rather than Kotlin drops
|
||
`kotlinc` from the toolchain; `javac` comes with the JDK Gradle already
|
||
needs. Generating the dex from Rust is not worth it: there is no mature
|
||
Rust dex writer, and the stubs never change.
|
||
|
||
### Can the APK be built without Gradle?
|
||
|
||
Yes. An APK is a zip containing a binary-XML `AndroidManifest.xml`,
|
||
`resources.arsc`, `classes.dex`, `lib/<abi>/*.so` and assets, aligned and
|
||
signed with the v2 scheme. The tools are `aapt2` (manifest and resources),
|
||
`d8` (Java bytecode to dex), `zipalign` and `apksigner`, all in the SDK's
|
||
`build-tools`, none of them Gradle. Three ways to drive them:
|
||
|
||
- **A `cargo xtask`** (or `build.rs`-adjacent script) that runs `cargo ndk`
|
||
for each ABI, `javac` + `d8` for the stubs, `aapt2 link`, `zipalign`,
|
||
`apksigner`. About 150 lines, every step visible, no AGP, no Gradle
|
||
daemon holding 2.8 GB between builds. The pinned-CA constant becomes a
|
||
`build.rs` reading the same `certs/ca.pem` path.
|
||
- **[cargo-apk2](https://github.com/mzdk100/cargo-apk2)**: the maintained
|
||
successor to cargo-apk, and unlike it compiles `java_sources` /
|
||
`kotlin_sources` into the dex and declares multiple activities **and
|
||
services** with intent filters from `[package.metadata.android]`, with
|
||
per-profile keystores and optional `aapt2`. Exactly the shape needed;
|
||
the question is whether a third-party tool with one maintainer beats
|
||
150 lines we own.
|
||
- **cargo-apk / xbuild**: unmaintained and `NativeActivity`-only. No.
|
||
|
||
What Gradle would take with it: Android Lint (which found two real bugs
|
||
here, but in Kotlin that would no longer exist — with forty lines of Java
|
||
stubs there is little left for it to find), manifest merging, R8, and the
|
||
generated-source plumbing. What it gives back: one toolchain, `cargo`
|
||
end to end, and Dev Updater keeps calling `build-apk.sh` exactly as now.
|
||
**Recommendation: the xtask**, with cargo-apk2 read for the details it
|
||
already got right (v2 signing, `uses-feature`, ABI splits).
|
||
|
||
### The behaviours that are hard to get back
|
||
|
||
Reading the Compose code for what a replacement must be able to express,
|
||
rather than what it happens to look like:
|
||
|
||
1. **The transcript is one selectable body of text.** One
|
||
`SelectionContainer` around the whole lazy list, so a selection runs from
|
||
a reply into the tool output beneath it. The framework needs selectable
|
||
read-only rich text across many rows, with the platform's selection
|
||
handles and clipboard on the phone.
|
||
2. **Rich inline text**: markdown with links (one tap detector per text,
|
||
not a node per link), inline code chips drawn behind the text, tables
|
||
with wrapping cells and a sideways scroll, syntax-highlighted fences,
|
||
ANSI colour in tool output, Nerd Font icon glyphs. Needs a text layout
|
||
engine with spans, not just styled labels.
|
||
3. **A bottom-anchored virtualised list of variable-height rows**, paged in
|
||
both directions (800-event pages, `HISTORY_SCREENS` measured in
|
||
viewports), with a saved scroll anchor per session, "hold the edge
|
||
nearest the tap" when a row expands (`holdTopEdge`, done in the layout
|
||
pass so the wrong frame is never drawn), and rows keyed so that a run of
|
||
tool calls stays one row while it grows.
|
||
4. **The soft keyboard**: the composer resizes with the IME, the guard
|
||
against a stuck inset animation, drafts per session, autocorrect and
|
||
suggestions from the phone's own keyboard. This is where most Rust
|
||
frameworks fail on Android today; see below.
|
||
5. **Platform integration through the app model**: foreground service,
|
||
notifications, share sheet, deep link, Keystore, camera, back gesture,
|
||
edge-to-edge insets, local-network permission.
|
||
6. **Accessibility names on icon buttons**, which the bench scripts depend
|
||
on (`ui-trace` taps by label). A framework with no accessibility tree
|
||
also breaks the measuring rig.
|
||
7. **Measurable frames**: the debug render report, and a way to attribute
|
||
a frame's cost to a widget on the real phone.
|
||
|
||
## The two constraints that decide it
|
||
|
||
**1. Android text input.** Every framework built on `winit` inherits
|
||
winit's Android backend, and that backend cannot drive the soft keyboard
|
||
properly: the IME tracking issues
|
||
([#1823](https://github.com/rust-windowing/winit/issues/1823),
|
||
[#2766](https://github.com/rust-windowing/winit/issues/2766)) are open,
|
||
`ReceivedCharacter` is unimplemented on Android
|
||
([#2305](https://github.com/rust-windowing/winit/issues/2305)), and the
|
||
`android-activity` groundwork for editor actions only merged in February
|
||
2026 ([PR #214](https://github.com/rust-mobile/android-activity/pull/214))
|
||
with the winit half still to come. Composition, autocorrect and suggestions
|
||
need an `InputConnection` implemented on the Java side, which winit's
|
||
`NativeActivity`/`GameActivity` model does not offer. The frameworks that
|
||
type on Android today each wrote their own Java glue (Slint, Makepad), and
|
||
the one designed to do it the way Android intends is
|
||
[`android-view`](https://github.com/rust-mobile/android-view): a Rust
|
||
implementation of an Android `View`, with text input through
|
||
`InputConnection`, accessibility, touch, callbacks on the UI thread, usable
|
||
either as a whole app or embedded beside ordinary Android components. It is
|
||
marked WIP. Both Linebender (its Masonry demo lives in that repo) and
|
||
Robius/Makepad ([Robrix's release notes](https://github.com/project-robius/robrix/releases)
|
||
say Android lacks a "full" keyboard and they are integrating android-view
|
||
for it) are converging on it. **That makes android-view the phone-side
|
||
foundation whichever widget set sits on top**, and the first thing to
|
||
build and measure here.
|
||
|
||
**2. Rich, selectable text and a virtualised list.** Frameworks group by
|
||
their text stack:
|
||
|
||
- **Parley + Fontique + Vello** (Linebender): rich spans, selection and
|
||
editing utilities, IME support driven through `ui-events`, AccessKit text
|
||
properties ([Linebender 2026 Q1](https://linebender.org/blog/tmil-25/),
|
||
[parley](https://github.com/linebender/parley)). Used by Masonry/Xilem,
|
||
and by Blitz. Vello proper needs compute shaders; `vello_hybrid` (CPU
|
||
path processing, GPU compositing) is "roughly beta" and runs on GLES too,
|
||
and Vello CPU exists as a no-GPU fallback.
|
||
- **cosmic-text** (iced, egui optionally): good layout, but the widgets on
|
||
top decide selection. iced's `markdown` widget is not selectable
|
||
([discourse](https://discourse.iced.rs/t/markdown-widgets-text-should-be-selectable/1107)).
|
||
- **Slint's own**: `TextInput` with `read-only` is the selectable-text
|
||
trick; there is **no inline rich text at all** (issue
|
||
[#1325](https://github.com/slint-ui/slint/issues/1325), markdown request
|
||
[#6684](https://github.com/slint-ui/slint/issues/6684) both open). A
|
||
markdown transcript with links and code chips cannot be drawn.
|
||
- **Makepad's own**: GPU/SDF text, a `Markdown` widget and a virtualised
|
||
`PortalList` in `makepad-widgets`.
|
||
|
||
## Options
|
||
|
||
### A. Keep Compose, move the logic into a Rust core (uniffi)
|
||
|
||
A `client-core` crate (events shared with the server, API client, SSE,
|
||
transcript fold, cache, markdown model, highlighter, ANSI) exposed to
|
||
Kotlin through [uniffi](https://github.com/mozilla/uniffi-rs). Compose keeps
|
||
drawing. Desktop would be a second UI (iced or Compose Desktop) over the
|
||
same core.
|
||
|
||
- **For**: the logic and the wire types stop drifting from the server
|
||
today, with tests in one language. Incremental and always shippable.
|
||
- **Against**: it is not what was asked for. The 13,000 lines of UI stay
|
||
Kotlin, the desktop app shares no UI code, and the `:link` Kotlin module
|
||
stays. uniffi's Kotlin Multiplatform bindings are a
|
||
[community fork](https://github.com/UbiqueInnovation/uniffi-kotlin-multiplatform-bindings);
|
||
the Android-only bindings are Mozilla's and solid.
|
||
- **Verdict**: not the destination, but **step one of every other option**
|
||
is building this crate, so it costs nothing to keep it as the fallback.
|
||
|
||
### B. Slint
|
||
|
||
Rust on Android is officially supported (minSdk 26, `android-activity`
|
||
backend, own Java IME glue, safe areas and keyboard insets since 1.15,
|
||
Skia renderer needs `clang`). Royalty-free licence requires disclosing
|
||
Slint use; GPLv3 otherwise. UI is a separate `.slint` DSL, not Rust.
|
||
|
||
- **Against**: no rich inline text (see above), so the transcript cannot be
|
||
drawn as it is today; the UI language is not Rust, which forfeits the
|
||
"compiler catches it" motivation for the half of the code that is UI.
|
||
- **Verdict**: rejected on rich text alone.
|
||
|
||
### C. iced
|
||
|
||
Elm-style, Rust-only widgets, desktop-first, `winit` + `wgpu`. Has a
|
||
`markdown` widget and `rich_text` with links. The maintainer states mobile
|
||
is a non-goal ([iced](https://github.com/iced-rs/iced)); a community
|
||
Android example exists and its author could not get the soft keyboard
|
||
working, patched widgets for touch, and notes no accessibility
|
||
([HN thread](https://news.ycombinator.com/item?id=46350641)). Markdown is
|
||
not selectable; `scrollable` is not virtualised.
|
||
|
||
- **Verdict**: a fine desktop toolkit and the one Iris named, but every
|
||
phone-side gap (IME, touch, accessibility, selection, virtualisation)
|
||
would be ours to build and maintain against a project that does not want
|
||
them. Not the shared framework.
|
||
|
||
### D. egui
|
||
|
||
Immediate mode, `winit`-based on Android, AccessKit integration,
|
||
selectable labels across a `Ui`. Repaints only on input by default, so
|
||
battery is not the immediate-mode worry. Android IME is blocked on winit
|
||
([discussion](https://github.com/emilk/egui/discussions/2053)); the
|
||
workaround is an in-app virtual keyboard, which is exactly the
|
||
non-native keyboard to avoid. Variable-height virtualised lists are manual
|
||
(`show_rows` assumes uniform heights). Looks like egui, not Material.
|
||
|
||
- **Verdict**: workable on desktop, wrong on the phone for the same reason
|
||
as iced, plus a look that would need a full custom style.
|
||
|
||
### E. Makepad
|
||
|
||
GPU-rendered, hybrid retained/immediate, `live_design!` DSL with hot
|
||
reload, MIT, 1.0 in 2025 ([makepad](https://github.com/makepad/makepad)).
|
||
Ships Android apps today with its own Java glue; Robrix (a Matrix chat
|
||
client, the closest analogue to this app) is its reference application on
|
||
Android, iOS and desktop. Has `Markdown`, `PortalList` (virtualised),
|
||
`TextInput`. Robrix reports the Android keyboard is not "full" and is moving
|
||
to android-view for it; the README says non-standard targets "may require
|
||
minor fixes".
|
||
|
||
- **For**: the only option that already ships a chat-shaped app on Android
|
||
and desktop from one codebase, with the widgets this app needs.
|
||
- **Against**: the DSL is its own language with its own shader-based
|
||
styling, so a large part of the UI would not be checked by rustc; the
|
||
rendering model (SDF everything) is a different world from Compose's,
|
||
and selection across a `Markdown` widget is unverified.
|
||
- **Verdict**: **rejected 2026-09-04** — Iris does not want a DSL. Kept
|
||
here so its Android keyboard status stays a data point about
|
||
android-view, not as an option.
|
||
|
||
### F. Masonry / Xilem on android-view (Linebender)
|
||
|
||
Retained widget tree (Masonry) with a reactive view layer (Xilem) that
|
||
reads like Compose; Rust all the way down; Vello, Parley, Fontique,
|
||
AccessKit, `ui-events`. Widgets include `Prose` (selectable read-only rich
|
||
text), `TextArea`, `VirtualScroll`, and this year `Svg`, `Split`,
|
||
`CollapsePanel`, a new layout system, and IME through `ui-events`
|
||
independent of winit. `masonry_android_view` exists in the android-view
|
||
repo and is "not yet generally usable"; Xilem calls itself experimental.
|
||
Desktop runs on winit. Vello needs a compute-capable GPU or falls back to
|
||
`vello_hybrid`/CPU.
|
||
|
||
- **For**: the only stack where every hard behaviour above maps onto a
|
||
component designed for it: selection and rich text (Parley/Prose),
|
||
virtualised variable heights (`VirtualScroll`), native IME
|
||
(android-view's `InputConnection`), accessibility (AccessKit, now with an
|
||
Android crate), one Rust widget language on both platforms. The team is
|
||
the one writing the Android integration everyone else is adopting.
|
||
- **Against**: pre-1.0 with API churn each release; a small team; no
|
||
Material widget set, so every control's look is ours; some of the pieces
|
||
(`masonry_android_view`, `vello_hybrid`) are explicitly unfinished. Being
|
||
early means fixing things upstream ourselves, which Iris said is
|
||
acceptable.
|
||
- **Verdict**: **the option to try first**, because it is the only one
|
||
whose gaps are "not finished yet" rather than "not designed for this".
|
||
|
||
### G. iris — the in-house library, and what "from scratch" means here
|
||
|
||
[cat16/iris](https://github.com/cat16/iris), read 2026-09-04 from the one
|
||
public commit (2026-01-31, "portfolio copy"; ~8,700 lines in `core`,
|
||
`macro` and the crate itself). Retained-mode widgets stored outside the
|
||
render tree, `wgpu` 28 directly, `winit` 0.30, `cosmic-text` 0.16, a
|
||
relative-anchor-plus-offset layout with `rest()` and `rel()` lengths, a
|
||
postfix builder API (`rect(..).radius(30).on(CursorSense::click(), ..)
|
||
.sized(..).align(..)`), events handled where the widget is declared, and
|
||
a single-threaded context passed explicitly — all of which reads like this
|
||
codebase's own rules. There is text editing (`widget/text/edit.rs`),
|
||
images, masks, spans and stacks; the TODO names text resizing as
|
||
per-frame slow and scaling as unsolved. It requires **nightly** (fourteen
|
||
`#![feature]` gates, among them `const_trait_impl`, `unboxed_closures`,
|
||
`portable_simd`, `associated_type_defaults`). Desktop only; no Android
|
||
surface, no IME, no accessibility tree, no virtualised list, no rich-text
|
||
selection.
|
||
|
||
**iris is not a candidate to be tested as it stands. It is the in-house
|
||
library** (Iris, 2026-09-04): "essentially a good start to a rewrite from
|
||
scratch", to be maintained and extended by the sessions working here.
|
||
So the list above of what it lacks is a **work list, not a score**. When
|
||
the app needs something iris does not have, the answer is to build it
|
||
into iris. The layer iris has is the widget and layout layer; the layers
|
||
it needs are the same ones Masonry gets from android-view, Parley and
|
||
AccessKit, and there is no reason iris cannot sit on those same
|
||
foundations rather than reinvent them — the surface, the keyboard bridge
|
||
and the accessibility tree are platform plumbing, not a framework's
|
||
identity. Whether the text stack stays cosmic-text or moves to Parley is
|
||
the first real design decision in that work (see I1 below).
|
||
|
||
Two things to carry into that work honestly. Nightly is the opposite of
|
||
"holds up long term": a build that breaks on a toolchain update, on the
|
||
machine Dev Updater builds on, unattended. Pin a dated nightly in
|
||
`rust-toolchain.toml` immediately, and keep a list of which `#![feature]`
|
||
gates are load-bearing so they can be retired as they stabilise or are
|
||
designed around. And a one-person framework carries every gap itself,
|
||
which is what Iris said she is willing to do.
|
||
|
||
"From scratch" therefore means iris, not a fourth thing. Masonry stays in
|
||
the plan as the **yardstick and the fallback**: building its demo and its
|
||
version of the transcript screen first says what a finished stack costs
|
||
on this hardware, proves android-view before iris depends on it, and
|
||
gives a comparison that is measured rather than remembered.
|
||
|
||
Not considered further: **GPUI** (Zed) mobile is a community fork that
|
||
depends on unpublished crates; **Dioxus/Blitz** is excluded by Iris (its
|
||
native renderer is Parley/Vello under HTML semantics, and the earlier
|
||
`tdep-survey/app-dioxus` spike parked it on a `vello_hybrid` stroke bug and
|
||
shipped the WebView); **Compose Multiplatform Desktop** would give a desktop
|
||
app for nothing but in Kotlin, which is the opposite direction.
|
||
|
||
### Weight and debug builds
|
||
|
||
Iris remembers the Linebender stack being slow in debug. What is behind
|
||
that is the dependency graph — Vello, wgpu, Parley, Fontique, Skrifa —
|
||
running unoptimised on the CPU side (path encoding, shaping), not the
|
||
widget layer. Xilem's own advice is only `split-debuginfo = "unpacked"` to
|
||
keep `target/` small; the fix everyone with this shape of dependency tree
|
||
uses is to optimise dependencies while leaving the app crate at `opt-level
|
||
= 0`:
|
||
|
||
[profile.dev.package."*"]
|
||
opt-level = 2
|
||
|
||
E1 measures this rather than remembering it: cold and incremental build
|
||
time, APK size, resident memory at rest and while streaming, and the
|
||
frame cost of one 800-event page — for the Masonry demo as shipped, then
|
||
with the profile above. Vello proper needs compute shaders and carries a
|
||
large shader set; `vello_hybrid` is lighter and Masonry can now render
|
||
through either (or Vello CPU) via its `imaging` abstraction, so "keep it
|
||
light" has a knob inside the same stack.
|
||
|
||
## Recommendation
|
||
|
||
1. **Build `client-core` now, whatever the framework.** A Rust crate holding
|
||
the event model (shared with `server/` as one crate, ending the
|
||
`Events.kt` mirror), the API and SSE clients, the transcript fold, the
|
||
cache, the markdown block model, the highlighter and the ANSI parser,
|
||
with the existing JVM tests ported. It is the part of the app that is
|
||
already tested, already logic, and already duplicated on the server.
|
||
2. **One foundation, two widget layers.** The platform plumbing is shared
|
||
whichever way the decision goes: android-view for the Android surface,
|
||
keyboard and accessibility bridge; `wgpu` for the GPU; AccessKit for
|
||
names; winit on the desktop. On top of it, **Masonry as the yardstick**
|
||
(E1, E2) and **iris as the thing being built** (I0–I5), both aimed at
|
||
the same transcript screen with the same pass conditions.
|
||
3. **Decide when the transcript screen exists in both**, from the
|
||
measurements, and record the decision here with the numbers. If iris
|
||
carries the screen within the Compose baseline, it is the app's
|
||
framework and Masonry was the calibration. If it does not, the
|
||
measurement says which parts of Masonry to adopt underneath it.
|
||
4. Then the shell (E3), the desktop window (E4) and the packaging (E5),
|
||
which do not depend on the choice.
|
||
|
||
## Experiments, in order
|
||
|
||
Each has a pass condition that is a measurement in this clone. The rig
|
||
matters: this emulator runs `-gpu host` with **host Vulkan switched off**
|
||
(`GPU_HOST_FEATURES` in `emulator-tools`, a gfxstream/Venus gap), so inside
|
||
the guest a `wgpu` app gets GLES, not Vulkan; the earlier Dioxus spike also
|
||
needed `WGPU_GLES_MINOR_VERSION=1` for compute shaders and found `wgpu`'s
|
||
Android backend wants API 26 (a libc symbol). The real phone has Vulkan.
|
||
Per the standing rule, a rig limit is something to fix before it is
|
||
accepted.
|
||
|
||
- [x] **E0 — toolchain (done 2026-09-04).** Installed under the
|
||
user-owned SDK: **NDK r29 (`29.0.14206865`)**, 2.4 GB at
|
||
`~/Android/Sdk/ndk/29.0.14206865`, the newest stable — r30 is still
|
||
at rc.3. **cargo-ndk 4.1.2**. Verified by cross-compiling a scratch
|
||
`cdylib` to both ABIs: `file` reports "for Android 26, built by NDK
|
||
r29 (14206865)" for `aarch64-linux-android` and
|
||
`x86_64-linux-android`. Two things to know at the call site.
|
||
**cargo-ndk 4's API-level flag is `-P`, not `-p`** — `-p` is now
|
||
passed through to cargo as `--package`, so the old
|
||
`cargo ndk -t arm64-v8a -p 26` panics with `unknown package: 26`
|
||
*and dumps the whole environment to stdout* as a bug report, which is
|
||
worth not doing in a log somebody might paste. And the Android
|
||
targets were installed for **stable** only; the pinned nightly needs
|
||
its own, which `iris/rust-toolchain.toml` now declares.
|
||
- [ ] **E1 — android-view's Masonry demo on this emulator.** Pass: it
|
||
builds with `cargo-ndk` + Gradle, renders on the GPU, and the phone's
|
||
own keyboard types into its editor with autocorrect and suggestions.
|
||
Note which `wgpu` backend it took and any environment flags needed.
|
||
- [ ] **E2 — a transcript in Masonry.** One screen: open a sandbox session,
|
||
page 800 events into `VirtualScroll` bottom-anchored, draw markdown
|
||
from `pulldown-cmark` into Parley rich text with links and code
|
||
chips, select across two rows with the platform handles, expand a
|
||
tool row holding its top edge. Pass: the render numbers land within
|
||
the Compose baseline in `transcript-bench.sh` on the GPU emulator
|
||
(same gestures, same session), and every one of the seven behaviours
|
||
above is either shown or has a written reason it cannot be.
|
||
- [ ] **E3 — the shell.** Kotlin `MainActivity` + `NotificationService`
|
||
+ Keystore + share intent calling into Rust over JNI, with the SSE
|
||
follow loop in Rust. Pass: a notification arrives with the app
|
||
closed, and a share lands in a session.
|
||
- [ ] **E4 — the same screen on the desktop** in a winit window, from the
|
||
same crate, with only the layout differing.
|
||
- [ ] **E5 — the packaging xtask**: `cargo ndk` → `javac`/`d8` → `aapt2`
|
||
→ `zipalign` → `apksigner`, signed with the existing release key,
|
||
installed through Dev Updater. Pass: the APK installs over the
|
||
Gradle-built one and the notification service starts.
|
||
|
||
### The iris track
|
||
|
||
These build iris up to carry the app. Each is a feature added to iris
|
||
with a pass condition, in dependency order. Work in `iris/` in this
|
||
repository on the `rustify` branch, and record in this file what each
|
||
step measured.
|
||
|
||
- [x] **I0a — where iris lives (decided 2026-09-04).** For now it is
|
||
**vendored at `iris/` in this repository**, history not carried,
|
||
and consumed by path. Iris's decision: keep it close while it is
|
||
being reshaped for this app, and give it back its own repository —
|
||
`iris/iris` on the gitea remote, which already holds the full
|
||
244-commit history, on a branch of its own — once it has proved
|
||
itself. The vendored tree is that repository's `main` at
|
||
`7b54aaf` ("readme", 2026-01-29), byte-identical to the public
|
||
GitHub copy, so a later reconciliation has a known base. A crate
|
||
that uses it says `iris = { path = "../iris" }`.
|
||
- [x] **I0b — make it build here (done 2026-09-04).** iris now builds,
|
||
clippy-clean and rustfmt-clean at the defaults, on a pinned dated
|
||
nightly, and the `tabs` example draws on this VM's GPU.
|
||
|
||
**The pin** is `nightly-2026-09-03` (rustc 1.100.0-nightly,
|
||
`2e2b193f8`), declared in `iris/rust-toolchain.toml` along with the
|
||
`clippy`/`rustfmt` components and the two Android targets, so a
|
||
fresh clone provisions itself. It is dated rather than `nightly`
|
||
because the whole failure below was a rolling channel moving under
|
||
an unattended build. Installed with `--profile minimal`: 912 MB.
|
||
|
||
**The 36 errors were one syntax change, and the earlier diagnosis in
|
||
this file was wrong.** It is not that a trait must now be declared
|
||
`const trait` — the vendored tree already declares them that way,
|
||
which is how it was written in January. What changed is the *impl*
|
||
keyword order: `impl const Trait for T` is now
|
||
`const impl Trait for T`, and generics go on the `impl`
|
||
(`const impl<T: [const] Foo> Bar for T`). Bounds are unaffected;
|
||
`T: const Foo`, `T: [const] Foo` and `impl const Foo` in argument
|
||
position all still compile. Everything else — the unresolved
|
||
`UiVec2`/`Vec2`/`impl_op` imports, and a `Color<u8>` that resolved
|
||
to `wgpu_types::Color` — cascaded from the seven files that failed
|
||
to parse. The rewrite was mechanical across 20 sites and took the
|
||
workspace from 36 errors to 0.
|
||
|
||
**`#![feature]` gates, 12 after this step** (two were declared and
|
||
unused, and were removed: `map_try_insert`, `const_cmp`).
|
||
Load-bearing and worth watching: `const_trait_impl`, `const_ops`,
|
||
`const_convert`, `const_destruct` are the const-traits family and
|
||
the one that has already broken once — they move together, so
|
||
advancing the pin means re-reading this section. `unboxed_closures`
|
||
+ `fn_traits` (postfix builder API) and `unsize` +
|
||
`coerce_unsized` (widget handles) are pairs. The rest are
|
||
individually small: `macro_metavar_expr_concat`, `portable_simd`,
|
||
`associated_type_defaults`, `option_into_flat_iter`, and `gen_blocks`
|
||
in the top crate.
|
||
|
||
**Running it headless.** `iris/run-headless.sh EXAMPLE [--shot PNG]`
|
||
with `iris/headless.conf`, the same trick `emu` uses: a headless
|
||
sway, and `grim` for the picture. It deliberately starts its *own*
|
||
compositor rather than joining `emu`'s — sway tiles, so adding a
|
||
window to the one an emulator sits in resizes that emulator.
|
||
Unlike `emu`'s it disables Xwayland, since winit speaks Wayland.
|
||
**This VM has a real GPU for this**: Vulkan 1.4 through Venus onto
|
||
the host's RX 7900 XT, and GL 4.6 through virgl — so desktop wgpu
|
||
work here is not software-rasterised, unlike inside the emulator.
|
||
|
||
**iris has no tests at all** (`cargo test --workspace`: 0 passed
|
||
across 6 targets). Nothing to keep passing, and nothing to catch a
|
||
regression — worth knowing before I1 changes the text stack.
|
||
|
||
**`iris-core` no longer depends on winit, and now cross-compiles to
|
||
Android.** It wanted exactly one thing from it — `PhysicalSize<u32>`
|
||
in `UiRenderNode::resize`'s signature, for two numbers it immediately
|
||
turned into floats — and that pulled a whole windowing backend into
|
||
the layer below it, the wrong direction. `resize` takes
|
||
`impl Into<Vec2>` now, like `UiRenderState::resize` beside it already
|
||
did. The consequence is the point: with winit in the graph an Android
|
||
build of the core failed in `android-activity` (which needs a backend
|
||
feature nothing here selects), and without it
|
||
`cargo ndk -t arm64-v8a -P 26 build -p iris-core` finishes in 30s and
|
||
produces an rlib, wgpu's Android backend included. So **iris's
|
||
widget, layout and render core already builds for the phone**, and
|
||
what I2 has to supply is the surface, the input and the IME — not a
|
||
port of the library.
|
||
|
||
**Build weight, cold, on this VM's 8 cores** (`rm -rf target`, then
|
||
`cargo build --example tabs`), since "the Linebender stack is slow in
|
||
debug" was the worry behind this question: plain debug **43s** and a
|
||
2.1 GB `target/`; with the `[profile.dev.package."*"] opt-level = 2`
|
||
knob, **1m46s** and 1.5 GB. So iris's own wgpu + winit + cosmic-text
|
||
graph is not the slow thing — which makes it a calibration for E1
|
||
rather than an answer about Masonry, whose graph adds Vello, Parley,
|
||
Fontique and Skrifa. Runtime cost of the knob was not measured here.
|
||
|
||
**Open defect found while doing this: iris sometimes never adopts
|
||
the window's real size.** Measured on the headless rig, ~3 starts in
|
||
15: the `tabs` example settles showing its 800×600 startup layout in
|
||
the top-left of a 1920×1200 surface, black around it, and stays that
|
||
way indefinitely — it is not a screenshot taken too early, since the
|
||
picture is byte-identical for the next four seconds. What is *not*
|
||
the cause, each checked: the winit event order is identical in good
|
||
and bad runs (`Resized(800×600)`, two redraws, `Resized(1920×1200)`,
|
||
one redraw), the swapchain reports `1920×1200` and
|
||
`suboptimal=false` on that last draw, and `output_size` is
|
||
`(1920, 1200)` going into it. It is timing-sensitive in the way that
|
||
makes it expensive: adding a single `eprintln!` anywhere in the draw
|
||
or event path hides it completely (0 in 16), which is why the
|
||
instrumentation above could not catch it in the act. A pointer move
|
||
does not repair it, because iris only redraws when something
|
||
changed; an output mode change does, because that is another resize.
|
||
Left open rather than guessed at. It matters most for **I2**, where
|
||
every rotation and every keyboard open is a resize, so a stale frame
|
||
would be the normal case rather than a rare one; a Wayland-level
|
||
trace of the xdg-surface configure/ack/commit sequence is the next
|
||
step, not more `eprintln`.
|
||
|
||
One thing was fixed on the way, and it is not that bug: `update`
|
||
redrew everything when `resized` was set, but `needs_redraw` — which
|
||
is what decides whether to *ask* for a frame — did not know about
|
||
`resized` at all. The two now share one `needs_redraw_all`, since a
|
||
condition in one and not the other is a frame nobody requests. It is
|
||
latent on Wayland only because winit asks for a redraw after a resize
|
||
by itself; on Android, where the surface work of I2 will not have
|
||
winit underneath it, nothing else here would have asked.
|
||
- [ ] **I1 — the text stack decision.** iris uses cosmic-text; the
|
||
transcript needs rich inline spans (links, code chips, colour),
|
||
selection across many widgets with the platform's handles on the
|
||
phone, and an editor the IME can drive (composition regions, not
|
||
just committed characters). Compare cosmic-text and Parley against
|
||
exactly those three, in iris, on a real transcript's text. Parley
|
||
is the expectation because android-view's IME bridge and AccessKit's
|
||
text properties are written against it; measure rather than assume.
|
||
Pass: a written decision here with what each was tried on, and the
|
||
TODO's "text resizing per frame is really slow" measured and either
|
||
fixed or explained.
|
||
- [ ] **I2 — iris on android-view.** An `android-view` surface as a second
|
||
backend beside winit: `wgpu` on the view's surface (GLES here, see
|
||
the Vulkan section; Vulkan on the phone), touch as pointer events,
|
||
window insets and the keyboard inset as layout inputs, the back
|
||
gesture as an event, the IME bridge feeding the editor from I1.
|
||
Pass: the `tabs` example and a text field run on the emulator, and
|
||
the phone's own keyboard types into the field with autocorrect and
|
||
suggestions — the same bar as E1.
|
||
- [ ] **I3 — a virtualised, bottom-anchored list.** Variable-height rows,
|
||
keyed, composed only while visible, paged in both directions with a
|
||
"more" sentinel at each end, a scroll anchor that survives rows
|
||
being inserted above, and "hold the edge nearest the tap" done in
|
||
the layout pass. Pass: 800 rows of real transcript text from the
|
||
sandbox scroll without a frame over the Compose baseline in
|
||
`transcript-bench.sh`, measured on the GPU emulator.
|
||
- [ ] **I4 — accessibility names via AccessKit.** Every control carries a
|
||
name; `ui-trace` can find and tap it by label. Pass: `bench-lib.sh`'s
|
||
tap-by-name works against the iris screen unchanged.
|
||
- [ ] **I5 — the transcript screen in iris.** E2's pass conditions, all
|
||
seven behaviours, against the sandbox with `--delay`. This is the
|
||
point the decision in the recommendation is made at.
|
||
|
||
## For the next agent
|
||
|
||
What to do when you pick this up, in order, so nothing here has to be
|
||
re-derived:
|
||
|
||
1. Read this file, then `AGENTS.md` and `PLAN.md`. The rules there
|
||
(measure, do not read; fix the rig before accepting its limits; the
|
||
emulator is this checkout's own) all apply.
|
||
2. Work on the **`rustify`** branch of this clone (`ai-app-2`), not on
|
||
`main` and not in `ai-app`. Nothing on this branch is production until
|
||
Iris says so. Commit and push as you go.
|
||
3. Take the next unchecked box above, in order: E0 first, then E1, then
|
||
the iris track from I0. E-steps and I-steps can proceed in parallel in
|
||
separate sessions once E1 has proved android-view on this emulator.
|
||
4. Every step ends with its measurement written into this file beside the
|
||
box, and the box ticked or the reason it could not be written in its
|
||
place. A step that is blocked says by what, not "later".
|
||
5. Run the existing rigs rather than inventing new ones: `ui-sandbox.sh`
|
||
for a server with fixtures, `transcript-bench.sh` for the scroll
|
||
baseline, `ui-trace` for anything positional, `emu up` for the
|
||
emulator. The Vulkan section below says how to get a Vulkan path in the
|
||
emulator when a `wgpu` backend needs one.
|
||
6. Decisions belong here with a date and what was rejected, the way
|
||
`PLAN.md` does it. Do not put design into commit messages alone.
|
||
|
||
### Vulkan in the emulator (measured 2026-09-04)
|
||
|
||
A `wgpu` app in this emulator was going to get GLES only, because host
|
||
Vulkan is switched off in `emulator-tools`. Retried today on Mesa
|
||
26.1.7: **Venus still fails the same way** — gfxstream picks
|
||
`externalMemoryMode: OpaqueFd`, probes `VK_FORMAT_R8G8B8A8_UNORM` for an
|
||
exportable colour buffer, and Venus says the format is unsupported
|
||
(`Failed to find memory type for ColorBuffers`, fatal before adb sees the
|
||
device). Venus does advertise `VK_KHR_external_memory_fd` and
|
||
`VK_EXT_external_memory_dma_buf`, so the gap is specifically opaque-fd
|
||
image export. gfxstream has a string-valued `VulkanExternalMemoryMode`
|
||
setting ("overrides what would otherwise be determined automatically"),
|
||
but `-feature Name=Value` is rejected as a bad feature name, and the only
|
||
mode words compiled into this emulator's `libgfxstream_backend.so`
|
||
(37.1.11) are `OpaqueFd`, `Metal` and `none` — there is no dma-buf mode
|
||
in this build to switch to. So Venus is blocked by the emulator, not by
|
||
Mesa; retry when the emulator package updates, since upstream gfxstream
|
||
does have dma-buf external memory.
|
||
|
||
What **does** work: pointing the emulator's Vulkan loader at the software
|
||
ICDs the emulator ships itself, with the feature enabled:
|
||
|
||
VK_DRIVER_FILES=$HOME/Android/Sdk/emulator/lib64/vulkan/vk_swiftshader_icd.json \
|
||
GPU_HOST_FEATURES="-feature Vulkan" emu up
|
||
|
||
The guest then reports Vulkan 1.3 (`cmd gpu vkjson`, SwiftShader
|
||
Subzero) while GLES still runs on the real GPU through virgl — so a
|
||
Vello/wgpu app can take its real Vulkan path here, with compute shaders,
|
||
CPU-rasterised. That is enough to test *correctness* of the Vulkan path
|
||
in the emulator; GPU *performance* of it is a phone measurement either
|
||
way, exactly as `MACHINE.md` already says about frame times. **lavapipe**
|
||
(`lvp_icd.json`, the other ICD the emulator ships) selected llvmpipe and
|
||
booted, then the emulator died right after loading the `default_boot`
|
||
snapshot with nothing in the log; a snapshot saved under a different
|
||
Vulkan device is the suspect, and `-no-snapshot-load` is the untested
|
||
next step. SwiftShader is the one that works today.
|
||
Making this an `emu` option belongs in `emulator-tools` and is a shared
|
||
tooling change, so it goes through the other sessions first.
|
||
|
||
## Things a Rust app changes elsewhere
|
||
|
||
- **`wg-app-link`'s `:link`** (pinned TLS, enrollment store, QR activity)
|
||
is Kotlin shared with Dev Updater. The certificate code already exists on
|
||
the Rust side of the submodule; the pinned-CA build step
|
||
(`generatePinnedCert`) becomes a `build.rs` reading the same path. The QR
|
||
scanner stays a Kotlin activity, since the camera is a platform feature.
|
||
- **Tooling** becomes `cargo` for everything but packaging: `cargo test`,
|
||
`clippy`, `fmt` cover the whole client, which is the motivation. Gradle
|
||
remains for the APK, signing (`~/.config/ai-app/release.jks`) and Dev
|
||
Updater's build modes; `build-apk.sh` would call `cargo ndk` first.
|
||
- **The bench scripts** (`ui-trace` by accessibility label) keep working
|
||
only if the framework exposes names through AccessKit on Android; that is
|
||
part of E2's pass condition, not a nicety.
|
||
- **Icons** stay Nerd Font glyphs from the committed subset; Parley/Fontique
|
||
loads a font file directly, so `build-icon-font.sh` is unchanged.
|
||
|
||
## Sources
|
||
|
||
- iced: [repo](https://github.com/iced-rs/iced), [0.14 release](https://github.com/iced-rs/iced/releases/tag/0.14.0), [Android thread](https://news.ycombinator.com/item?id=46350641), [markdown selection request](https://discourse.iced.rs/t/markdown-widgets-text-should-be-selectable/1107)
|
||
- Linebender: [2026 Q1 report](https://linebender.org/blog/tmil-25/), [xilem](https://github.com/linebender/xilem), [parley](https://github.com/linebender/parley), [vello](https://github.com/linebender/vello), [vello_hybrid](https://docs.rs/vello_hybrid/latest/vello_hybrid/)
|
||
- android-view: [repo](https://github.com/rust-mobile/android-view); android-activity [PR #214](https://github.com/rust-mobile/android-activity/pull/214)
|
||
- winit Android IME: [#1823](https://github.com/rust-windowing/winit/issues/1823), [#2766](https://github.com/rust-windowing/winit/issues/2766), [#2305](https://github.com/rust-windowing/winit/issues/2305)
|
||
- egui on Android: [discussion #2053](https://github.com/emilk/egui/discussions/2053)
|
||
- Slint: [Android guide](https://docs.slint.dev/latest/docs/slint/guide/platforms/mobile/android/), [1.15 release](https://slint.dev/blog/slint-1.15-released), [licensing](https://slint.dev/faqs), rich text [#1325](https://github.com/slint-ui/slint/issues/1325), markdown [#6684](https://github.com/slint-ui/slint/issues/6684)
|
||
- Makepad: [repo](https://github.com/makepad/makepad), [makepad-widgets](https://docs.rs/makepad-widgets), [Robrix](https://github.com/project-robius/robrix), [Robrix releases](https://github.com/project-robius/robrix/releases)
|
||
- AccessKit: [releases](https://github.com/AccessKit/accesskit/releases)
|
||
- Build tools: [cargo-ndk](https://github.com/bbqsrc/cargo-ndk), [cargo-apk](https://github.com/rust-mobile/cargo-apk), [rust-mobile](https://github.com/rust-mobile)
|
||
- uniffi: [repo](https://github.com/mozilla/uniffi-rs), [KMP bindings fork](https://github.com/UbiqueInnovation/uniffi-kotlin-multiplatform-bindings)
|
||
- GPUI mobile: [gpui-mobile](https://github.com/itsbalamurali/gpui-mobile)
|
||
- The earlier Dioxus spike's findings on `wgpu`/Vulkan in this emulator: `~/repos/tdep-survey/app-dioxus/README.md`
|