From 0f8ba49f4a6082a46c977002ebf8919792667a13 Mon Sep 17 00:00:00 2001 From: iris <2+iris@noreply.localhost> Date: Fri, 4 Sep 2026 17:12:09 -0400 Subject: [PATCH] Add RUST.md: the plan for moving the app to Rust Research and measurements from 2026-09-04: what the Compose app has to reproduce, why Android text input and rich selectable text decide the framework, the options considered (Masonry as the yardstick, iris as the in-house library to build up; Slint, iced, egui, Makepad rejected with reasons), how thin the Java shell can be, building the APK without Gradle, and the ordered experiments with pass conditions. Includes the emulator Vulkan findings: Venus is blocked by this emulator's gfxstream, SwiftShader over the emulator's own ICD works. Co-Authored-By: Claude Fable 5.1 --- AGENTS.md | 4 + RUST.md | 590 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 594 insertions(+) create mode 100644 RUST.md diff --git a/AGENTS.md b/AGENTS.md index a8991eb..af47340 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -46,6 +46,10 @@ Module-by-module intent is in PLAN.md's "Backend layout". before touching `TranscriptCache.kt`, `TranscriptSource.kt`, or the opening and stream effects in `SessionScreen.kt`. - `TODO.md` — the working list. +- `RUST.md` — the plan for moving the app to Rust (on the `rustify` + branch of the `ai-app-2` clone): what has to be reproduced, the + framework decision, and the ordered experiments with their pass + conditions. Read it before touching anything under that branch. - `.dev-updater.ron` — what Dev Updater builds here: the server (run as `service: Managed(…)`, supervised by Dev Updater's own implementation rather than a script kept here) and the APK, in parallel. It points at diff --git a/RUST.md b/RUST.md new file mode 100644 index 0000000..701e578 --- /dev/null +++ b/RUST.md @@ -0,0 +1,590 @@ +# Moving the app to Rust + +Working document for the question Iris asked on 2026-09-04: what are the +options for switching the phone app to Rust, ideally pure Rust with one UI +framework shared with a future winit-based desktop application, at full +feature parity and without giving up anything native, performance +especially. Constraints she set: no Dioxus and nothing that draws through a +WebView; **no UI DSL** (which rules out Makepad and Slint); the result +should stay lightweight; platform-specific pieces are fine to maintain; +reimplementing a framework piece from scratch where it does not fit is +fine; effort and elapsed time do not matter, long-term robustness does; +this clone is where things get tried before anything is committed to +`ai-app`. Her own library, [iris](https://github.com/cat16/iris), is the +**in-house framework to be built up** for this, with Masonry as the +yardstick it is measured against. + +Decisions get a date and a reason here, the way `PLAN.md` does. Nothing in +this file has been tried yet unless a section says it has. + +## What has to be reproduced + +The app is ~19,000 lines of Kotlin. It splits three ways, and the split is +what decides how much of a port is mechanical. + +**Pure logic with no Compose or Android in it, ~4,500 lines.** `Api.kt` +(1,142), `Events.kt`, `EventStream.kt`, `Sse.kt`, `TranscriptCache.kt` +(589, touches `java.io.File` only), `TranscriptSource.kt`, +`MarkdownSyntax.kt`, `Languages.kt`, `Highlighter.kt`, `Ansi.kt`, +`ResetCountdown.kt`, `Durations.kt`, `Sizes.kt`, `ModelName.kt`, +`LoadState.kt`, `ImportableStream.kt`. `TranscriptUnits.kt` and +`TranscriptItems.kt` (the event fold into rows, ~940 lines) are logic with +a handful of Compose annotations. This is also exactly the code that has +JVM unit tests today. All of it ports directly, and most of it already has a +Rust twin in `server/`: `Events.kt` is a hand-kept mirror of +`session/driver.rs`'s enum, the highlighter and the syntax scanner exist on +the server for the explorer, and the cache compares the server's own JSON +lines. **Sharing these types between server and app is the single largest +"keep things in sync" win available, and it does not depend on which UI +framework wins.** + +**Compose UI, ~13,000 lines.** Screens, dialogs, the transcript list, the +markdown renderer's customisations, tool cards, the file explorer viewer and +editor. This is the part a UI framework choice is about. + +**Android platform code, ~1,500 lines**, spread over 20 files. Every one of +these is a Java-side object that no Rust framework can replace, because +Android only offers them as Java classes: + +- `NotificationService` — a **foreground service** holding the + `/notifications` SSE stream while the app is closed, with its ongoing + notification, `specialUse` type and the `POST_NOTIFICATIONS` request. +- `MainActivity` — edge-to-edge, the `ACCESS_LOCAL_NETWORK` runtime + permission (Android 17), `singleTop` intent routing for `aiapp://enroll`, + notification taps, and the **share sheet** (`ACTION_SEND`, any MIME type). +- `ServerConfig` — the bearer token sealed under an **Android Keystore** + AES-GCM key, shared with Dev Updater through `wg-app-link`'s `:link`. +- `EnrollmentScanActivity` — the in-app **QR scanner** (zxing, camera). +- `Attachments` — `ContentResolver` reads of shared URIs, `BitmapFactory` + decode and downscale, **EXIF** orientation. +- `SessionImage` — bitmap decode for produced images. +- `ScrollAnchor`, `Drafts` — `SharedPreferences`; `CrashLog` — `filesDir`. +- `TranscriptCache` — `cacheDir`. +- `DebugStats`/`FrameStats` — `Choreographer` frame timing and the render + report; `runtime-tracing` names composables in a system trace. + +So **"pure Rust" on Android means Rust owns every line of logic and +drawing, behind a thin shell of Java stubs**, and a packaging step that +produces a signed APK. How thin, and whether Gradle is inevitable, are +answered below. + +### How much Java is unavoidable, and why + +Rust can *call* any Android API through JNI (`jni` crate, with +`ndk-context` handing over the `JavaVM` and the Activity): posting a +notification, `startForegroundService`, the Keystore, `ContentResolver` +reads, permission requests, `WindowInsets`, the clipboard. None of that +needs a line of Kotlin. What JNI cannot do is *define* a class that the +system instantiates **by name from the manifest** — an `Activity`, a +`Service`, an `Application`, a `BroadcastReceiver`. Those must exist as dex +bytecode inside the APK before any Rust runs, because the framework +constructs them and only then calls into native code. `NativeActivity` is +the platform's own stub for the Activity case; there is no +`NativeService`, and android-view ships its own `View` subclass for the +same reason. + +So the floor is roughly **two Java classes of ten lines each**: an +`Activity` and a `Service` whose lifecycle methods are declared `native` +and registered from `JNI_OnLoad`, plus whatever android-view already +provides. Everything they would have done in Kotlin — insets, intent +routing, the SSE follow loop, the notification builder — is Rust reached +through those stubs. Writing the stubs in Java rather than Kotlin drops +`kotlinc` from the toolchain; `javac` comes with the JDK Gradle already +needs. Generating the dex from Rust is not worth it: there is no mature +Rust dex writer, and the stubs never change. + +### Can the APK be built without Gradle? + +Yes. An APK is a zip containing a binary-XML `AndroidManifest.xml`, +`resources.arsc`, `classes.dex`, `lib//*.so` and assets, aligned and +signed with the v2 scheme. The tools are `aapt2` (manifest and resources), +`d8` (Java bytecode to dex), `zipalign` and `apksigner`, all in the SDK's +`build-tools`, none of them Gradle. Three ways to drive them: + +- **A `cargo xtask`** (or `build.rs`-adjacent script) that runs `cargo ndk` + for each ABI, `javac` + `d8` for the stubs, `aapt2 link`, `zipalign`, + `apksigner`. About 150 lines, every step visible, no AGP, no Gradle + daemon holding 2.8 GB between builds. The pinned-CA constant becomes a + `build.rs` reading the same `certs/ca.pem` path. +- **[cargo-apk2](https://github.com/mzdk100/cargo-apk2)**: the maintained + successor to cargo-apk, and unlike it compiles `java_sources` / + `kotlin_sources` into the dex and declares multiple activities **and + services** with intent filters from `[package.metadata.android]`, with + per-profile keystores and optional `aapt2`. Exactly the shape needed; + the question is whether a third-party tool with one maintainer beats + 150 lines we own. +- **cargo-apk / xbuild**: unmaintained and `NativeActivity`-only. No. + +What Gradle would take with it: Android Lint (which found two real bugs +here, but in Kotlin that would no longer exist — with forty lines of Java +stubs there is little left for it to find), manifest merging, R8, and the +generated-source plumbing. What it gives back: one toolchain, `cargo` +end to end, and Dev Updater keeps calling `build-apk.sh` exactly as now. +**Recommendation: the xtask**, with cargo-apk2 read for the details it +already got right (v2 signing, `uses-feature`, ABI splits). + +### The behaviours that are hard to get back + +Reading the Compose code for what a replacement must be able to express, +rather than what it happens to look like: + +1. **The transcript is one selectable body of text.** One + `SelectionContainer` around the whole lazy list, so a selection runs from + a reply into the tool output beneath it. The framework needs selectable + read-only rich text across many rows, with the platform's selection + handles and clipboard on the phone. +2. **Rich inline text**: markdown with links (one tap detector per text, + not a node per link), inline code chips drawn behind the text, tables + with wrapping cells and a sideways scroll, syntax-highlighted fences, + ANSI colour in tool output, Nerd Font icon glyphs. Needs a text layout + engine with spans, not just styled labels. +3. **A bottom-anchored virtualised list of variable-height rows**, paged in + both directions (800-event pages, `HISTORY_SCREENS` measured in + viewports), with a saved scroll anchor per session, "hold the edge + nearest the tap" when a row expands (`holdTopEdge`, done in the layout + pass so the wrong frame is never drawn), and rows keyed so that a run of + tool calls stays one row while it grows. +4. **The soft keyboard**: the composer resizes with the IME, the guard + against a stuck inset animation, drafts per session, autocorrect and + suggestions from the phone's own keyboard. This is where most Rust + frameworks fail on Android today; see below. +5. **Platform integration through the app model**: foreground service, + notifications, share sheet, deep link, Keystore, camera, back gesture, + edge-to-edge insets, local-network permission. +6. **Accessibility names on icon buttons**, which the bench scripts depend + on (`ui-trace` taps by label). A framework with no accessibility tree + also breaks the measuring rig. +7. **Measurable frames**: the debug render report, and a way to attribute + a frame's cost to a widget on the real phone. + +## The two constraints that decide it + +**1. Android text input.** Every framework built on `winit` inherits +winit's Android backend, and that backend cannot drive the soft keyboard +properly: the IME tracking issues +([#1823](https://github.com/rust-windowing/winit/issues/1823), +[#2766](https://github.com/rust-windowing/winit/issues/2766)) are open, +`ReceivedCharacter` is unimplemented on Android +([#2305](https://github.com/rust-windowing/winit/issues/2305)), and the +`android-activity` groundwork for editor actions only merged in February +2026 ([PR #214](https://github.com/rust-mobile/android-activity/pull/214)) +with the winit half still to come. Composition, autocorrect and suggestions +need an `InputConnection` implemented on the Java side, which winit's +`NativeActivity`/`GameActivity` model does not offer. The frameworks that +type on Android today each wrote their own Java glue (Slint, Makepad), and +the one designed to do it the way Android intends is +[`android-view`](https://github.com/rust-mobile/android-view): a Rust +implementation of an Android `View`, with text input through +`InputConnection`, accessibility, touch, callbacks on the UI thread, usable +either as a whole app or embedded beside ordinary Android components. It is +marked WIP. Both Linebender (its Masonry demo lives in that repo) and +Robius/Makepad ([Robrix's release notes](https://github.com/project-robius/robrix/releases) +say Android lacks a "full" keyboard and they are integrating android-view +for it) are converging on it. **That makes android-view the phone-side +foundation whichever widget set sits on top**, and the first thing to +build and measure here. + +**2. Rich, selectable text and a virtualised list.** Frameworks group by +their text stack: + +- **Parley + Fontique + Vello** (Linebender): rich spans, selection and + editing utilities, IME support driven through `ui-events`, AccessKit text + properties ([Linebender 2026 Q1](https://linebender.org/blog/tmil-25/), + [parley](https://github.com/linebender/parley)). Used by Masonry/Xilem, + and by Blitz. Vello proper needs compute shaders; `vello_hybrid` (CPU + path processing, GPU compositing) is "roughly beta" and runs on GLES too, + and Vello CPU exists as a no-GPU fallback. +- **cosmic-text** (iced, egui optionally): good layout, but the widgets on + top decide selection. iced's `markdown` widget is not selectable + ([discourse](https://discourse.iced.rs/t/markdown-widgets-text-should-be-selectable/1107)). +- **Slint's own**: `TextInput` with `read-only` is the selectable-text + trick; there is **no inline rich text at all** (issue + [#1325](https://github.com/slint-ui/slint/issues/1325), markdown request + [#6684](https://github.com/slint-ui/slint/issues/6684) both open). A + markdown transcript with links and code chips cannot be drawn. +- **Makepad's own**: GPU/SDF text, a `Markdown` widget and a virtualised + `PortalList` in `makepad-widgets`. + +## Options + +### A. Keep Compose, move the logic into a Rust core (uniffi) + +A `client-core` crate (events shared with the server, API client, SSE, +transcript fold, cache, markdown model, highlighter, ANSI) exposed to +Kotlin through [uniffi](https://github.com/mozilla/uniffi-rs). Compose keeps +drawing. Desktop would be a second UI (iced or Compose Desktop) over the +same core. + +- **For**: the logic and the wire types stop drifting from the server + today, with tests in one language. Incremental and always shippable. +- **Against**: it is not what was asked for. The 13,000 lines of UI stay + Kotlin, the desktop app shares no UI code, and the `:link` Kotlin module + stays. uniffi's Kotlin Multiplatform bindings are a + [community fork](https://github.com/UbiqueInnovation/uniffi-kotlin-multiplatform-bindings); + the Android-only bindings are Mozilla's and solid. +- **Verdict**: not the destination, but **step one of every other option** + is building this crate, so it costs nothing to keep it as the fallback. + +### B. Slint + +Rust on Android is officially supported (minSdk 26, `android-activity` +backend, own Java IME glue, safe areas and keyboard insets since 1.15, +Skia renderer needs `clang`). Royalty-free licence requires disclosing +Slint use; GPLv3 otherwise. UI is a separate `.slint` DSL, not Rust. + +- **Against**: no rich inline text (see above), so the transcript cannot be + drawn as it is today; the UI language is not Rust, which forfeits the + "compiler catches it" motivation for the half of the code that is UI. +- **Verdict**: rejected on rich text alone. + +### C. iced + +Elm-style, Rust-only widgets, desktop-first, `winit` + `wgpu`. Has a +`markdown` widget and `rich_text` with links. The maintainer states mobile +is a non-goal ([iced](https://github.com/iced-rs/iced)); a community +Android example exists and its author could not get the soft keyboard +working, patched widgets for touch, and notes no accessibility +([HN thread](https://news.ycombinator.com/item?id=46350641)). Markdown is +not selectable; `scrollable` is not virtualised. + +- **Verdict**: a fine desktop toolkit and the one Iris named, but every + phone-side gap (IME, touch, accessibility, selection, virtualisation) + would be ours to build and maintain against a project that does not want + them. Not the shared framework. + +### D. egui + +Immediate mode, `winit`-based on Android, AccessKit integration, +selectable labels across a `Ui`. Repaints only on input by default, so +battery is not the immediate-mode worry. Android IME is blocked on winit +([discussion](https://github.com/emilk/egui/discussions/2053)); the +workaround is an in-app virtual keyboard, which is exactly the +non-native keyboard to avoid. Variable-height virtualised lists are manual +(`show_rows` assumes uniform heights). Looks like egui, not Material. + +- **Verdict**: workable on desktop, wrong on the phone for the same reason + as iced, plus a look that would need a full custom style. + +### E. Makepad + +GPU-rendered, hybrid retained/immediate, `live_design!` DSL with hot +reload, MIT, 1.0 in 2025 ([makepad](https://github.com/makepad/makepad)). +Ships Android apps today with its own Java glue; Robrix (a Matrix chat +client, the closest analogue to this app) is its reference application on +Android, iOS and desktop. Has `Markdown`, `PortalList` (virtualised), +`TextInput`. Robrix reports the Android keyboard is not "full" and is moving +to android-view for it; the README says non-standard targets "may require +minor fixes". + +- **For**: the only option that already ships a chat-shaped app on Android + and desktop from one codebase, with the widgets this app needs. +- **Against**: the DSL is its own language with its own shader-based + styling, so a large part of the UI would not be checked by rustc; the + rendering model (SDF everything) is a different world from Compose's, + and selection across a `Markdown` widget is unverified. +- **Verdict**: **rejected 2026-09-04** — Iris does not want a DSL. Kept + here so its Android keyboard status stays a data point about + android-view, not as an option. + +### F. Masonry / Xilem on android-view (Linebender) + +Retained widget tree (Masonry) with a reactive view layer (Xilem) that +reads like Compose; Rust all the way down; Vello, Parley, Fontique, +AccessKit, `ui-events`. Widgets include `Prose` (selectable read-only rich +text), `TextArea`, `VirtualScroll`, and this year `Svg`, `Split`, +`CollapsePanel`, a new layout system, and IME through `ui-events` +independent of winit. `masonry_android_view` exists in the android-view +repo and is "not yet generally usable"; Xilem calls itself experimental. +Desktop runs on winit. Vello needs a compute-capable GPU or falls back to +`vello_hybrid`/CPU. + +- **For**: the only stack where every hard behaviour above maps onto a + component designed for it: selection and rich text (Parley/Prose), + virtualised variable heights (`VirtualScroll`), native IME + (android-view's `InputConnection`), accessibility (AccessKit, now with an + Android crate), one Rust widget language on both platforms. The team is + the one writing the Android integration everyone else is adopting. +- **Against**: pre-1.0 with API churn each release; a small team; no + Material widget set, so every control's look is ours; some of the pieces + (`masonry_android_view`, `vello_hybrid`) are explicitly unfinished. Being + early means fixing things upstream ourselves, which Iris said is + acceptable. +- **Verdict**: **the option to try first**, because it is the only one + whose gaps are "not finished yet" rather than "not designed for this". + +### G. iris — the in-house library, and what "from scratch" means here + +[cat16/iris](https://github.com/cat16/iris), read 2026-09-04 from the one +public commit (2026-01-31, "portfolio copy"; ~8,700 lines in `core`, +`macro` and the crate itself). Retained-mode widgets stored outside the +render tree, `wgpu` 28 directly, `winit` 0.30, `cosmic-text` 0.16, a +relative-anchor-plus-offset layout with `rest()` and `rel()` lengths, a +postfix builder API (`rect(..).radius(30).on(CursorSense::click(), ..) +.sized(..).align(..)`), events handled where the widget is declared, and +a single-threaded context passed explicitly — all of which reads like this +codebase's own rules. There is text editing (`widget/text/edit.rs`), +images, masks, spans and stacks; the TODO names text resizing as +per-frame slow and scaling as unsolved. It requires **nightly** (fourteen +`#![feature]` gates, among them `const_trait_impl`, `unboxed_closures`, +`portable_simd`, `associated_type_defaults`). Desktop only; no Android +surface, no IME, no accessibility tree, no virtualised list, no rich-text +selection. + +**iris is not a candidate to be tested as it stands. It is the in-house +library** (Iris, 2026-09-04): "essentially a good start to a rewrite from +scratch", to be maintained and extended by the sessions working here. +So the list above of what it lacks is a **work list, not a score**. When +the app needs something iris does not have, the answer is to build it +into iris. The layer iris has is the widget and layout layer; the layers +it needs are the same ones Masonry gets from android-view, Parley and +AccessKit, and there is no reason iris cannot sit on those same +foundations rather than reinvent them — the surface, the keyboard bridge +and the accessibility tree are platform plumbing, not a framework's +identity. Whether the text stack stays cosmic-text or moves to Parley is +the first real design decision in that work (see I1 below). + +Two things to carry into that work honestly. Nightly is the opposite of +"holds up long term": a build that breaks on a toolchain update, on the +machine Dev Updater builds on, unattended. Pin a dated nightly in +`rust-toolchain.toml` immediately, and keep a list of which `#![feature]` +gates are load-bearing so they can be retired as they stabilise or are +designed around. And a one-person framework carries every gap itself, +which is what Iris said she is willing to do. + +"From scratch" therefore means iris, not a fourth thing. Masonry stays in +the plan as the **yardstick and the fallback**: building its demo and its +version of the transcript screen first says what a finished stack costs +on this hardware, proves android-view before iris depends on it, and +gives a comparison that is measured rather than remembered. + +Not considered further: **GPUI** (Zed) mobile is a community fork that +depends on unpublished crates; **Dioxus/Blitz** is excluded by Iris (its +native renderer is Parley/Vello under HTML semantics, and the earlier +`tdep-survey/app-dioxus` spike parked it on a `vello_hybrid` stroke bug and +shipped the WebView); **Compose Multiplatform Desktop** would give a desktop +app for nothing but in Kotlin, which is the opposite direction. + +### Weight and debug builds + +Iris remembers the Linebender stack being slow in debug. What is behind +that is the dependency graph — Vello, wgpu, Parley, Fontique, Skrifa — +running unoptimised on the CPU side (path encoding, shaping), not the +widget layer. Xilem's own advice is only `split-debuginfo = "unpacked"` to +keep `target/` small; the fix everyone with this shape of dependency tree +uses is to optimise dependencies while leaving the app crate at `opt-level += 0`: + + [profile.dev.package."*"] + opt-level = 2 + +E1 measures this rather than remembering it: cold and incremental build +time, APK size, resident memory at rest and while streaming, and the +frame cost of one 800-event page — for the Masonry demo as shipped, then +with the profile above. Vello proper needs compute shaders and carries a +large shader set; `vello_hybrid` is lighter and Masonry can now render +through either (or Vello CPU) via its `imaging` abstraction, so "keep it +light" has a knob inside the same stack. + +## Recommendation + +1. **Build `client-core` now, whatever the framework.** A Rust crate holding + the event model (shared with `server/` as one crate, ending the + `Events.kt` mirror), the API and SSE clients, the transcript fold, the + cache, the markdown block model, the highlighter and the ANSI parser, + with the existing JVM tests ported. It is the part of the app that is + already tested, already logic, and already duplicated on the server. +2. **One foundation, two widget layers.** The platform plumbing is shared + whichever way the decision goes: android-view for the Android surface, + keyboard and accessibility bridge; `wgpu` for the GPU; AccessKit for + names; winit on the desktop. On top of it, **Masonry as the yardstick** + (E1, E2) and **iris as the thing being built** (I0–I5), both aimed at + the same transcript screen with the same pass conditions. +3. **Decide when the transcript screen exists in both**, from the + measurements, and record the decision here with the numbers. If iris + carries the screen within the Compose baseline, it is the app's + framework and Masonry was the calibration. If it does not, the + measurement says which parts of Masonry to adopt underneath it. +4. Then the shell (E3), the desktop window (E4) and the packaging (E5), + which do not depend on the choice. + +## Experiments, in order + +Each has a pass condition that is a measurement in this clone. The rig +matters: this emulator runs `-gpu host` with **host Vulkan switched off** +(`GPU_HOST_FEATURES` in `emulator-tools`, a gfxstream/Venus gap), so inside +the guest a `wgpu` app gets GLES, not Vulkan; the earlier Dioxus spike also +needed `WGPU_GLES_MINOR_VERSION=1` for compute shaders and found `wgpu`'s +Android backend wants API 26 (a libc symbol). The real phone has Vulkan. +Per the standing rule, a rig limit is something to fix before it is +accepted. + +- [ ] **E0 — toolchain.** No NDK is installed (`~/Android/Sdk/ndk` is + empty; the Android Rust targets are). Install the NDK under the + user-owned SDK, `cargo-ndk`, and record the versions here. +- [ ] **E1 — android-view's Masonry demo on this emulator.** Pass: it + builds with `cargo-ndk` + Gradle, renders on the GPU, and the phone's + own keyboard types into its editor with autocorrect and suggestions. + Note which `wgpu` backend it took and any environment flags needed. +- [ ] **E2 — a transcript in Masonry.** One screen: open a sandbox session, + page 800 events into `VirtualScroll` bottom-anchored, draw markdown + from `pulldown-cmark` into Parley rich text with links and code + chips, select across two rows with the platform handles, expand a + tool row holding its top edge. Pass: the render numbers land within + the Compose baseline in `transcript-bench.sh` on the GPU emulator + (same gestures, same session), and every one of the seven behaviours + above is either shown or has a written reason it cannot be. +- [ ] **E3 — the shell.** Kotlin `MainActivity` + `NotificationService` + + Keystore + share intent calling into Rust over JNI, with the SSE + follow loop in Rust. Pass: a notification arrives with the app + closed, and a share lands in a session. +- [ ] **E4 — the same screen on the desktop** in a winit window, from the + same crate, with only the layout differing. +- [ ] **E5 — the packaging xtask**: `cargo ndk` → `javac`/`d8` → `aapt2` + → `zipalign` → `apksigner`, signed with the existing release key, + installed through Dev Updater. Pass: the APK installs over the + Gradle-built one and the notification service starts. + +### The iris track + +These build iris up to carry the app. Each is a feature added to iris +with a pass condition, in dependency order. Work in iris's own repository, +pinned here by commit, and record in this file what each step measured. + +- [ ] **I0 — bring iris in-house.** Decide where it lives — the + recommendation is a repository of its own on the gitea remote, + consumed here the way `wg-app-link` is (a submodule pinned to a + commit), so the app and the library version-lock and the library + stays usable by the desktop app on its own. Pin a dated nightly in + `rust-toolchain.toml`; list every `#![feature]` gate with what it is + for, so each can be retired when it stabilises. Get `cargo build`, + `clippy` and `fmt` clean at the defaults and the `tabs` example + running in a window. Pass: a fresh clone builds unattended from the + pinned toolchain. +- [ ] **I1 — the text stack decision.** iris uses cosmic-text; the + transcript needs rich inline spans (links, code chips, colour), + selection across many widgets with the platform's handles on the + phone, and an editor the IME can drive (composition regions, not + just committed characters). Compare cosmic-text and Parley against + exactly those three, in iris, on a real transcript's text. Parley + is the expectation because android-view's IME bridge and AccessKit's + text properties are written against it; measure rather than assume. + Pass: a written decision here with what each was tried on, and the + TODO's "text resizing per frame is really slow" measured and either + fixed or explained. +- [ ] **I2 — iris on android-view.** An `android-view` surface as a second + backend beside winit: `wgpu` on the view's surface (GLES here, see + the Vulkan section; Vulkan on the phone), touch as pointer events, + window insets and the keyboard inset as layout inputs, the back + gesture as an event, the IME bridge feeding the editor from I1. + Pass: the `tabs` example and a text field run on the emulator, and + the phone's own keyboard types into the field with autocorrect and + suggestions — the same bar as E1. +- [ ] **I3 — a virtualised, bottom-anchored list.** Variable-height rows, + keyed, composed only while visible, paged in both directions with a + "more" sentinel at each end, a scroll anchor that survives rows + being inserted above, and "hold the edge nearest the tap" done in + the layout pass. Pass: 800 rows of real transcript text from the + sandbox scroll without a frame over the Compose baseline in + `transcript-bench.sh`, measured on the GPU emulator. +- [ ] **I4 — accessibility names via AccessKit.** Every control carries a + name; `ui-trace` can find and tap it by label. Pass: `bench-lib.sh`'s + tap-by-name works against the iris screen unchanged. +- [ ] **I5 — the transcript screen in iris.** E2's pass conditions, all + seven behaviours, against the sandbox with `--delay`. This is the + point the decision in the recommendation is made at. + +## For the next agent + +What to do when you pick this up, in order, so nothing here has to be +re-derived: + +1. Read this file, then `AGENTS.md` and `PLAN.md`. The rules there + (measure, do not read; fix the rig before accepting its limits; the + emulator is this checkout's own) all apply. +2. Work on the **`rustify`** branch of this clone (`ai-app-2`), not on + `main` and not in `ai-app`. Nothing on this branch is production until + Iris says so. Commit and push as you go. +3. Take the next unchecked box above, in order: E0 first, then E1, then + the iris track from I0. E-steps and I-steps can proceed in parallel in + separate sessions once E1 has proved android-view on this emulator. +4. Every step ends with its measurement written into this file beside the + box, and the box ticked or the reason it could not be written in its + place. A step that is blocked says by what, not "later". +5. Run the existing rigs rather than inventing new ones: `ui-sandbox.sh` + for a server with fixtures, `transcript-bench.sh` for the scroll + baseline, `ui-trace` for anything positional, `emu up` for the + emulator. The Vulkan section below says how to get a Vulkan path in the + emulator when a `wgpu` backend needs one. +6. Decisions belong here with a date and what was rejected, the way + `PLAN.md` does it. Do not put design into commit messages alone. + +### Vulkan in the emulator (measured 2026-09-04) + +A `wgpu` app in this emulator was going to get GLES only, because host +Vulkan is switched off in `emulator-tools`. Retried today on Mesa +26.1.7: **Venus still fails the same way** — gfxstream picks +`externalMemoryMode: OpaqueFd`, probes `VK_FORMAT_R8G8B8A8_UNORM` for an +exportable colour buffer, and Venus says the format is unsupported +(`Failed to find memory type for ColorBuffers`, fatal before adb sees the +device). Venus does advertise `VK_KHR_external_memory_fd` and +`VK_EXT_external_memory_dma_buf`, so the gap is specifically opaque-fd +image export. gfxstream has a string-valued `VulkanExternalMemoryMode` +setting ("overrides what would otherwise be determined automatically"), +but `-feature Name=Value` is rejected as a bad feature name, and the only +mode words compiled into this emulator's `libgfxstream_backend.so` +(37.1.11) are `OpaqueFd`, `Metal` and `none` — there is no dma-buf mode +in this build to switch to. So Venus is blocked by the emulator, not by +Mesa; retry when the emulator package updates, since upstream gfxstream +does have dma-buf external memory. + +What **does** work: pointing the emulator's Vulkan loader at the software +ICDs the emulator ships itself, with the feature enabled: + + VK_DRIVER_FILES=$HOME/Android/Sdk/emulator/lib64/vulkan/vk_swiftshader_icd.json \ + GPU_HOST_FEATURES="-feature Vulkan" emu up + +The guest then reports Vulkan 1.3 (`cmd gpu vkjson`, SwiftShader +Subzero) while GLES still runs on the real GPU through virgl — so a +Vello/wgpu app can take its real Vulkan path here, with compute shaders, +CPU-rasterised. That is enough to test *correctness* of the Vulkan path +in the emulator; GPU *performance* of it is a phone measurement either +way, exactly as `MACHINE.md` already says about frame times. **lavapipe** +(`lvp_icd.json`, the other ICD the emulator ships) selected llvmpipe and +booted, then the emulator died right after loading the `default_boot` +snapshot with nothing in the log; a snapshot saved under a different +Vulkan device is the suspect, and `-no-snapshot-load` is the untested +next step. SwiftShader is the one that works today. +Making this an `emu` option belongs in `emulator-tools` and is a shared +tooling change, so it goes through the other sessions first. + +## Things a Rust app changes elsewhere + +- **`wg-app-link`'s `:link`** (pinned TLS, enrollment store, QR activity) + is Kotlin shared with Dev Updater. The certificate code already exists on + the Rust side of the submodule; the pinned-CA build step + (`generatePinnedCert`) becomes a `build.rs` reading the same path. The QR + scanner stays a Kotlin activity, since the camera is a platform feature. +- **Tooling** becomes `cargo` for everything but packaging: `cargo test`, + `clippy`, `fmt` cover the whole client, which is the motivation. Gradle + remains for the APK, signing (`~/.config/ai-app/release.jks`) and Dev + Updater's build modes; `build-apk.sh` would call `cargo ndk` first. +- **The bench scripts** (`ui-trace` by accessibility label) keep working + only if the framework exposes names through AccessKit on Android; that is + part of E2's pass condition, not a nicety. +- **Icons** stay Nerd Font glyphs from the committed subset; Parley/Fontique + loads a font file directly, so `build-icon-font.sh` is unchanged. + +## Sources + +- iced: [repo](https://github.com/iced-rs/iced), [0.14 release](https://github.com/iced-rs/iced/releases/tag/0.14.0), [Android thread](https://news.ycombinator.com/item?id=46350641), [markdown selection request](https://discourse.iced.rs/t/markdown-widgets-text-should-be-selectable/1107) +- Linebender: [2026 Q1 report](https://linebender.org/blog/tmil-25/), [xilem](https://github.com/linebender/xilem), [parley](https://github.com/linebender/parley), [vello](https://github.com/linebender/vello), [vello_hybrid](https://docs.rs/vello_hybrid/latest/vello_hybrid/) +- android-view: [repo](https://github.com/rust-mobile/android-view); android-activity [PR #214](https://github.com/rust-mobile/android-activity/pull/214) +- winit Android IME: [#1823](https://github.com/rust-windowing/winit/issues/1823), [#2766](https://github.com/rust-windowing/winit/issues/2766), [#2305](https://github.com/rust-windowing/winit/issues/2305) +- egui on Android: [discussion #2053](https://github.com/emilk/egui/discussions/2053) +- Slint: [Android guide](https://docs.slint.dev/latest/docs/slint/guide/platforms/mobile/android/), [1.15 release](https://slint.dev/blog/slint-1.15-released), [licensing](https://slint.dev/faqs), rich text [#1325](https://github.com/slint-ui/slint/issues/1325), markdown [#6684](https://github.com/slint-ui/slint/issues/6684) +- Makepad: [repo](https://github.com/makepad/makepad), [makepad-widgets](https://docs.rs/makepad-widgets), [Robrix](https://github.com/project-robius/robrix), [Robrix releases](https://github.com/project-robius/robrix/releases) +- AccessKit: [releases](https://github.com/AccessKit/accesskit/releases) +- Build tools: [cargo-ndk](https://github.com/bbqsrc/cargo-ndk), [cargo-apk](https://github.com/rust-mobile/cargo-apk), [rust-mobile](https://github.com/rust-mobile) +- uniffi: [repo](https://github.com/mozilla/uniffi-rs), [KMP bindings fork](https://github.com/UbiqueInnovation/uniffi-kotlin-multiplatform-bindings) +- GPUI mobile: [gpui-mobile](https://github.com/itsbalamurali/gpui-mobile) +- The earlier Dioxus spike's findings on `wgpu`/Vulkan in this emulator: `~/repos/tdep-survey/app-dioxus/README.md`