Files
ai-app/RUST.md
T
irisandClaude Opus 5 e0ee7d6e94 RUST.md: record E0 and I0b, with what they measured
E0 is done (NDK r29, cargo-ndk 4.1.2, verified by cross-compiling to both
ABIs) and I0b is done. Corrects this file's guess at why iris would not
build, notes the const-traits family as the gates to re-read whenever the
pin is advanced, and records the cold build weight against the "slow in
debug" worry: 43s and 2.1 GB plain, 1m46s and 1.5 GB with dependencies at
opt-level 2.

Also records an open defect found on the way -- iris sometimes keeps its
pre-configure window size for good -- with what was ruled out, since it
is timing-sensitive enough that any added print hides it, and I2 will
meet it on every rotation.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-04 17:47:44 -04:00

42 KiB
Raw Blame History

Moving the app to Rust

Working document for the question Iris asked on 2026-09-04: what are the options for switching the phone app to Rust, ideally pure Rust with one UI framework shared with a future winit-based desktop application, at full feature parity and without giving up anything native, performance especially. Constraints she set: no Dioxus and nothing that draws through a WebView; no UI DSL (which rules out Makepad and Slint); the result should stay lightweight; platform-specific pieces are fine to maintain; reimplementing a framework piece from scratch where it does not fit is fine; effort and elapsed time do not matter, long-term robustness does; this clone is where things get tried before anything is committed to ai-app. Her own library, iris, is the in-house framework to be built up for this, with Masonry as the yardstick it is measured against.

Decisions get a date and a reason here, the way PLAN.md does. Nothing in this file has been tried yet unless a section says it has.

What has to be reproduced

The app is ~19,000 lines of Kotlin. It splits three ways, and the split is what decides how much of a port is mechanical.

Pure logic with no Compose or Android in it, ~4,500 lines. Api.kt (1,142), Events.kt, EventStream.kt, Sse.kt, TranscriptCache.kt (589, touches java.io.File only), TranscriptSource.kt, MarkdownSyntax.kt, Languages.kt, Highlighter.kt, Ansi.kt, ResetCountdown.kt, Durations.kt, Sizes.kt, ModelName.kt, LoadState.kt, ImportableStream.kt. TranscriptUnits.kt and TranscriptItems.kt (the event fold into rows, ~940 lines) are logic with a handful of Compose annotations. This is also exactly the code that has JVM unit tests today. All of it ports directly, and most of it already has a Rust twin in server/: Events.kt is a hand-kept mirror of session/driver.rs's enum, the highlighter and the syntax scanner exist on the server for the explorer, and the cache compares the server's own JSON lines. Sharing these types between server and app is the single largest "keep things in sync" win available, and it does not depend on which UI framework wins.

Compose UI, ~13,000 lines. Screens, dialogs, the transcript list, the markdown renderer's customisations, tool cards, the file explorer viewer and editor. This is the part a UI framework choice is about.

Android platform code, ~1,500 lines, spread over 20 files. Every one of these is a Java-side object that no Rust framework can replace, because Android only offers them as Java classes:

  • NotificationService — a foreground service holding the /notifications SSE stream while the app is closed, with its ongoing notification, specialUse type and the POST_NOTIFICATIONS request.
  • MainActivity — edge-to-edge, the ACCESS_LOCAL_NETWORK runtime permission (Android 17), singleTop intent routing for aiapp://enroll, notification taps, and the share sheet (ACTION_SEND, any MIME type).
  • ServerConfig — the bearer token sealed under an Android Keystore AES-GCM key, shared with Dev Updater through wg-app-link's :link.
  • EnrollmentScanActivity — the in-app QR scanner (zxing, camera).
  • AttachmentsContentResolver reads of shared URIs, BitmapFactory decode and downscale, EXIF orientation.
  • SessionImage — bitmap decode for produced images.
  • ScrollAnchor, DraftsSharedPreferences; CrashLogfilesDir.
  • TranscriptCachecacheDir.
  • DebugStats/FrameStatsChoreographer frame timing and the render report; runtime-tracing names composables in a system trace.

So "pure Rust" on Android means Rust owns every line of logic and drawing, behind a thin shell of Java stubs, and a packaging step that produces a signed APK. How thin, and whether Gradle is inevitable, are answered below.

How much Java is unavoidable, and why

Rust can call any Android API through JNI (jni crate, with ndk-context handing over the JavaVM and the Activity): posting a notification, startForegroundService, the Keystore, ContentResolver reads, permission requests, WindowInsets, the clipboard. None of that needs a line of Kotlin. What JNI cannot do is define a class that the system instantiates by name from the manifest — an Activity, a Service, an Application, a BroadcastReceiver. Those must exist as dex bytecode inside the APK before any Rust runs, because the framework constructs them and only then calls into native code. NativeActivity is the platform's own stub for the Activity case; there is no NativeService, and android-view ships its own View subclass for the same reason.

So the floor is roughly two Java classes of ten lines each: an Activity and a Service whose lifecycle methods are declared native and registered from JNI_OnLoad, plus whatever android-view already provides. Everything they would have done in Kotlin — insets, intent routing, the SSE follow loop, the notification builder — is Rust reached through those stubs. Writing the stubs in Java rather than Kotlin drops kotlinc from the toolchain; javac comes with the JDK Gradle already needs. Generating the dex from Rust is not worth it: there is no mature Rust dex writer, and the stubs never change.

Can the APK be built without Gradle?

Yes. An APK is a zip containing a binary-XML AndroidManifest.xml, resources.arsc, classes.dex, lib/<abi>/*.so and assets, aligned and signed with the v2 scheme. The tools are aapt2 (manifest and resources), d8 (Java bytecode to dex), zipalign and apksigner, all in the SDK's build-tools, none of them Gradle. Three ways to drive them:

  • A cargo xtask (or build.rs-adjacent script) that runs cargo ndk for each ABI, javac + d8 for the stubs, aapt2 link, zipalign, apksigner. About 150 lines, every step visible, no AGP, no Gradle daemon holding 2.8 GB between builds. The pinned-CA constant becomes a build.rs reading the same certs/ca.pem path.
  • cargo-apk2: the maintained successor to cargo-apk, and unlike it compiles java_sources / kotlin_sources into the dex and declares multiple activities and services with intent filters from [package.metadata.android], with per-profile keystores and optional aapt2. Exactly the shape needed; the question is whether a third-party tool with one maintainer beats 150 lines we own.
  • cargo-apk / xbuild: unmaintained and NativeActivity-only. No.

What Gradle would take with it: Android Lint (which found two real bugs here, but in Kotlin that would no longer exist — with forty lines of Java stubs there is little left for it to find), manifest merging, R8, and the generated-source plumbing. What it gives back: one toolchain, cargo end to end, and Dev Updater keeps calling build-apk.sh exactly as now. Recommendation: the xtask, with cargo-apk2 read for the details it already got right (v2 signing, uses-feature, ABI splits).

The behaviours that are hard to get back

Reading the Compose code for what a replacement must be able to express, rather than what it happens to look like:

  1. The transcript is one selectable body of text. One SelectionContainer around the whole lazy list, so a selection runs from a reply into the tool output beneath it. The framework needs selectable read-only rich text across many rows, with the platform's selection handles and clipboard on the phone.
  2. Rich inline text: markdown with links (one tap detector per text, not a node per link), inline code chips drawn behind the text, tables with wrapping cells and a sideways scroll, syntax-highlighted fences, ANSI colour in tool output, Nerd Font icon glyphs. Needs a text layout engine with spans, not just styled labels.
  3. A bottom-anchored virtualised list of variable-height rows, paged in both directions (800-event pages, HISTORY_SCREENS measured in viewports), with a saved scroll anchor per session, "hold the edge nearest the tap" when a row expands (holdTopEdge, done in the layout pass so the wrong frame is never drawn), and rows keyed so that a run of tool calls stays one row while it grows.
  4. The soft keyboard: the composer resizes with the IME, the guard against a stuck inset animation, drafts per session, autocorrect and suggestions from the phone's own keyboard. This is where most Rust frameworks fail on Android today; see below.
  5. Platform integration through the app model: foreground service, notifications, share sheet, deep link, Keystore, camera, back gesture, edge-to-edge insets, local-network permission.
  6. Accessibility names on icon buttons, which the bench scripts depend on (ui-trace taps by label). A framework with no accessibility tree also breaks the measuring rig.
  7. Measurable frames: the debug render report, and a way to attribute a frame's cost to a widget on the real phone.

The two constraints that decide it

1. Android text input. Every framework built on winit inherits winit's Android backend, and that backend cannot drive the soft keyboard properly: the IME tracking issues (#1823, #2766) are open, ReceivedCharacter is unimplemented on Android (#2305), and the android-activity groundwork for editor actions only merged in February 2026 (PR #214) with the winit half still to come. Composition, autocorrect and suggestions need an InputConnection implemented on the Java side, which winit's NativeActivity/GameActivity model does not offer. The frameworks that type on Android today each wrote their own Java glue (Slint, Makepad), and the one designed to do it the way Android intends is android-view: a Rust implementation of an Android View, with text input through InputConnection, accessibility, touch, callbacks on the UI thread, usable either as a whole app or embedded beside ordinary Android components. It is marked WIP. Both Linebender (its Masonry demo lives in that repo) and Robius/Makepad (Robrix's release notes say Android lacks a "full" keyboard and they are integrating android-view for it) are converging on it. That makes android-view the phone-side foundation whichever widget set sits on top, and the first thing to build and measure here.

2. Rich, selectable text and a virtualised list. Frameworks group by their text stack:

  • Parley + Fontique + Vello (Linebender): rich spans, selection and editing utilities, IME support driven through ui-events, AccessKit text properties (Linebender 2026 Q1, parley). Used by Masonry/Xilem, and by Blitz. Vello proper needs compute shaders; vello_hybrid (CPU path processing, GPU compositing) is "roughly beta" and runs on GLES too, and Vello CPU exists as a no-GPU fallback.
  • cosmic-text (iced, egui optionally): good layout, but the widgets on top decide selection. iced's markdown widget is not selectable (discourse).
  • Slint's own: TextInput with read-only is the selectable-text trick; there is no inline rich text at all (issue #1325, markdown request #6684 both open). A markdown transcript with links and code chips cannot be drawn.
  • Makepad's own: GPU/SDF text, a Markdown widget and a virtualised PortalList in makepad-widgets.

Options

A. Keep Compose, move the logic into a Rust core (uniffi)

A client-core crate (events shared with the server, API client, SSE, transcript fold, cache, markdown model, highlighter, ANSI) exposed to Kotlin through uniffi. Compose keeps drawing. Desktop would be a second UI (iced or Compose Desktop) over the same core.

  • For: the logic and the wire types stop drifting from the server today, with tests in one language. Incremental and always shippable.
  • Against: it is not what was asked for. The 13,000 lines of UI stay Kotlin, the desktop app shares no UI code, and the :link Kotlin module stays. uniffi's Kotlin Multiplatform bindings are a community fork; the Android-only bindings are Mozilla's and solid.
  • Verdict: not the destination, but step one of every other option is building this crate, so it costs nothing to keep it as the fallback.

B. Slint

Rust on Android is officially supported (minSdk 26, android-activity backend, own Java IME glue, safe areas and keyboard insets since 1.15, Skia renderer needs clang). Royalty-free licence requires disclosing Slint use; GPLv3 otherwise. UI is a separate .slint DSL, not Rust.

  • Against: no rich inline text (see above), so the transcript cannot be drawn as it is today; the UI language is not Rust, which forfeits the "compiler catches it" motivation for the half of the code that is UI.
  • Verdict: rejected on rich text alone.

C. iced

Elm-style, Rust-only widgets, desktop-first, winit + wgpu. Has a markdown widget and rich_text with links. The maintainer states mobile is a non-goal (iced); a community Android example exists and its author could not get the soft keyboard working, patched widgets for touch, and notes no accessibility (HN thread). Markdown is not selectable; scrollable is not virtualised.

  • Verdict: a fine desktop toolkit and the one Iris named, but every phone-side gap (IME, touch, accessibility, selection, virtualisation) would be ours to build and maintain against a project that does not want them. Not the shared framework.

D. egui

Immediate mode, winit-based on Android, AccessKit integration, selectable labels across a Ui. Repaints only on input by default, so battery is not the immediate-mode worry. Android IME is blocked on winit (discussion); the workaround is an in-app virtual keyboard, which is exactly the non-native keyboard to avoid. Variable-height virtualised lists are manual (show_rows assumes uniform heights). Looks like egui, not Material.

  • Verdict: workable on desktop, wrong on the phone for the same reason as iced, plus a look that would need a full custom style.

E. Makepad

GPU-rendered, hybrid retained/immediate, live_design! DSL with hot reload, MIT, 1.0 in 2025 (makepad). Ships Android apps today with its own Java glue; Robrix (a Matrix chat client, the closest analogue to this app) is its reference application on Android, iOS and desktop. Has Markdown, PortalList (virtualised), TextInput. Robrix reports the Android keyboard is not "full" and is moving to android-view for it; the README says non-standard targets "may require minor fixes".

  • For: the only option that already ships a chat-shaped app on Android and desktop from one codebase, with the widgets this app needs.
  • Against: the DSL is its own language with its own shader-based styling, so a large part of the UI would not be checked by rustc; the rendering model (SDF everything) is a different world from Compose's, and selection across a Markdown widget is unverified.
  • Verdict: rejected 2026-09-04 — Iris does not want a DSL. Kept here so its Android keyboard status stays a data point about android-view, not as an option.

F. Masonry / Xilem on android-view (Linebender)

Retained widget tree (Masonry) with a reactive view layer (Xilem) that reads like Compose; Rust all the way down; Vello, Parley, Fontique, AccessKit, ui-events. Widgets include Prose (selectable read-only rich text), TextArea, VirtualScroll, and this year Svg, Split, CollapsePanel, a new layout system, and IME through ui-events independent of winit. masonry_android_view exists in the android-view repo and is "not yet generally usable"; Xilem calls itself experimental. Desktop runs on winit. Vello needs a compute-capable GPU or falls back to vello_hybrid/CPU.

  • For: the only stack where every hard behaviour above maps onto a component designed for it: selection and rich text (Parley/Prose), virtualised variable heights (VirtualScroll), native IME (android-view's InputConnection), accessibility (AccessKit, now with an Android crate), one Rust widget language on both platforms. The team is the one writing the Android integration everyone else is adopting.
  • Against: pre-1.0 with API churn each release; a small team; no Material widget set, so every control's look is ours; some of the pieces (masonry_android_view, vello_hybrid) are explicitly unfinished. Being early means fixing things upstream ourselves, which Iris said is acceptable.
  • Verdict: the option to try first, because it is the only one whose gaps are "not finished yet" rather than "not designed for this".

G. iris — the in-house library, and what "from scratch" means here

cat16/iris, read 2026-09-04 from the one public commit (2026-01-31, "portfolio copy"; ~8,700 lines in core, macro and the crate itself). Retained-mode widgets stored outside the render tree, wgpu 28 directly, winit 0.30, cosmic-text 0.16, a relative-anchor-plus-offset layout with rest() and rel() lengths, a postfix builder API (rect(..).radius(30).on(CursorSense::click(), ..) .sized(..).align(..)), events handled where the widget is declared, and a single-threaded context passed explicitly — all of which reads like this codebase's own rules. There is text editing (widget/text/edit.rs), images, masks, spans and stacks; the TODO names text resizing as per-frame slow and scaling as unsolved. It requires nightly (fourteen #![feature] gates, among them const_trait_impl, unboxed_closures, portable_simd, associated_type_defaults). Desktop only; no Android surface, no IME, no accessibility tree, no virtualised list, no rich-text selection.

iris is not a candidate to be tested as it stands. It is the in-house library (Iris, 2026-09-04): "essentially a good start to a rewrite from scratch", to be maintained and extended by the sessions working here. So the list above of what it lacks is a work list, not a score. When the app needs something iris does not have, the answer is to build it into iris. The layer iris has is the widget and layout layer; the layers it needs are the same ones Masonry gets from android-view, Parley and AccessKit, and there is no reason iris cannot sit on those same foundations rather than reinvent them — the surface, the keyboard bridge and the accessibility tree are platform plumbing, not a framework's identity. Whether the text stack stays cosmic-text or moves to Parley is the first real design decision in that work (see I1 below).

Two things to carry into that work honestly. Nightly is the opposite of "holds up long term": a build that breaks on a toolchain update, on the machine Dev Updater builds on, unattended. Pin a dated nightly in rust-toolchain.toml immediately, and keep a list of which #![feature] gates are load-bearing so they can be retired as they stabilise or are designed around. And a one-person framework carries every gap itself, which is what Iris said she is willing to do.

"From scratch" therefore means iris, not a fourth thing. Masonry stays in the plan as the yardstick and the fallback: building its demo and its version of the transcript screen first says what a finished stack costs on this hardware, proves android-view before iris depends on it, and gives a comparison that is measured rather than remembered.

Not considered further: GPUI (Zed) mobile is a community fork that depends on unpublished crates; Dioxus/Blitz is excluded by Iris (its native renderer is Parley/Vello under HTML semantics, and the earlier tdep-survey/app-dioxus spike parked it on a vello_hybrid stroke bug and shipped the WebView); Compose Multiplatform Desktop would give a desktop app for nothing but in Kotlin, which is the opposite direction.

Weight and debug builds

Iris remembers the Linebender stack being slow in debug. What is behind that is the dependency graph — Vello, wgpu, Parley, Fontique, Skrifa — running unoptimised on the CPU side (path encoding, shaping), not the widget layer. Xilem's own advice is only split-debuginfo = "unpacked" to keep target/ small; the fix everyone with this shape of dependency tree uses is to optimise dependencies while leaving the app crate at opt-level = 0:

[profile.dev.package."*"]
opt-level = 2

E1 measures this rather than remembering it: cold and incremental build time, APK size, resident memory at rest and while streaming, and the frame cost of one 800-event page — for the Masonry demo as shipped, then with the profile above. Vello proper needs compute shaders and carries a large shader set; vello_hybrid is lighter and Masonry can now render through either (or Vello CPU) via its imaging abstraction, so "keep it light" has a knob inside the same stack.

Recommendation

  1. Build client-core now, whatever the framework. A Rust crate holding the event model (shared with server/ as one crate, ending the Events.kt mirror), the API and SSE clients, the transcript fold, the cache, the markdown block model, the highlighter and the ANSI parser, with the existing JVM tests ported. It is the part of the app that is already tested, already logic, and already duplicated on the server.
  2. One foundation, two widget layers. The platform plumbing is shared whichever way the decision goes: android-view for the Android surface, keyboard and accessibility bridge; wgpu for the GPU; AccessKit for names; winit on the desktop. On top of it, Masonry as the yardstick (E1, E2) and iris as the thing being built (I0I5), both aimed at the same transcript screen with the same pass conditions.
  3. Decide when the transcript screen exists in both, from the measurements, and record the decision here with the numbers. If iris carries the screen within the Compose baseline, it is the app's framework and Masonry was the calibration. If it does not, the measurement says which parts of Masonry to adopt underneath it.
  4. Then the shell (E3), the desktop window (E4) and the packaging (E5), which do not depend on the choice.

Experiments, in order

Each has a pass condition that is a measurement in this clone. The rig matters: this emulator runs -gpu host with host Vulkan switched off (GPU_HOST_FEATURES in emulator-tools, a gfxstream/Venus gap), so inside the guest a wgpu app gets GLES, not Vulkan; the earlier Dioxus spike also needed WGPU_GLES_MINOR_VERSION=1 for compute shaders and found wgpu's Android backend wants API 26 (a libc symbol). The real phone has Vulkan. Per the standing rule, a rig limit is something to fix before it is accepted.

  • E0 — toolchain (done 2026-09-04). Installed under the user-owned SDK: NDK r29 (29.0.14206865), 2.4 GB at ~/Android/Sdk/ndk/29.0.14206865, the newest stable — r30 is still at rc.3. cargo-ndk 4.1.2. Verified by cross-compiling a scratch cdylib to both ABIs: file reports "for Android 26, built by NDK r29 (14206865)" for aarch64-linux-android and x86_64-linux-android. Two things to know at the call site. cargo-ndk 4's API-level flag is -P, not -p-p is now passed through to cargo as --package, so the old cargo ndk -t arm64-v8a -p 26 panics with unknown package: 26 and dumps the whole environment to stdout as a bug report, which is worth not doing in a log somebody might paste. And the Android targets were installed for stable only; the pinned nightly needs its own, which iris/rust-toolchain.toml now declares.
  • E1 — android-view's Masonry demo on this emulator. Pass: it builds with cargo-ndk + Gradle, renders on the GPU, and the phone's own keyboard types into its editor with autocorrect and suggestions. Note which wgpu backend it took and any environment flags needed.
  • E2 — a transcript in Masonry. One screen: open a sandbox session, page 800 events into VirtualScroll bottom-anchored, draw markdown from pulldown-cmark into Parley rich text with links and code chips, select across two rows with the platform handles, expand a tool row holding its top edge. Pass: the render numbers land within the Compose baseline in transcript-bench.sh on the GPU emulator (same gestures, same session), and every one of the seven behaviours above is either shown or has a written reason it cannot be.
  • E3 — the shell. Kotlin MainActivity + NotificationService + Keystore + share intent calling into Rust over JNI, with the SSE follow loop in Rust. Pass: a notification arrives with the app closed, and a share lands in a session.
  • E4 — the same screen on the desktop in a winit window, from the same crate, with only the layout differing.
  • E5 — the packaging xtask: cargo ndkjavac/d8aapt2zipalignapksigner, signed with the existing release key, installed through Dev Updater. Pass: the APK installs over the Gradle-built one and the notification service starts.

The iris track

These build iris up to carry the app. Each is a feature added to iris with a pass condition, in dependency order. Work in iris/ in this repository on the rustify branch, and record in this file what each step measured.

  • I0a — where iris lives (decided 2026-09-04). For now it is vendored at iris/ in this repository, history not carried, and consumed by path. Iris's decision: keep it close while it is being reshaped for this app, and give it back its own repository — iris/iris on the gitea remote, which already holds the full 244-commit history, on a branch of its own — once it has proved itself. The vendored tree is that repository's main at 7b54aaf ("readme", 2026-01-29), byte-identical to the public GitHub copy, so a later reconciliation has a known base. A crate that uses it says iris = { path = "../iris" }.

  • I0b — make it build here (done 2026-09-04). iris now builds, clippy-clean and rustfmt-clean at the defaults, on a pinned dated nightly, and the tabs example draws on this VM's GPU.

    **The pin** is `nightly-2026-09-03` (rustc 1.100.0-nightly,
    `2e2b193f8`), declared in `iris/rust-toolchain.toml` along with the
    `clippy`/`rustfmt` components and the two Android targets, so a
    fresh clone provisions itself. It is dated rather than `nightly`
    because the whole failure below was a rolling channel moving under
    an unattended build. Installed with `--profile minimal`: 912 MB.
    
    **The 36 errors were one syntax change, and the earlier diagnosis in
    this file was wrong.** It is not that a trait must now be declared
    `const trait` — the vendored tree already declares them that way,
    which is how it was written in January. What changed is the *impl*
    keyword order: `impl const Trait for T` is now
    `const impl Trait for T`, and generics go on the `impl`
    (`const impl<T: [const] Foo> Bar for T`). Bounds are unaffected;
    `T: const Foo`, `T: [const] Foo` and `impl const Foo` in argument
    position all still compile. Everything else — the unresolved
    `UiVec2`/`Vec2`/`impl_op` imports, and a `Color<u8>` that resolved
    to `wgpu_types::Color` — cascaded from the seven files that failed
    to parse. The rewrite was mechanical across 20 sites and took the
    workspace from 36 errors to 0.
    
    **`#![feature]` gates, 12 after this step** (two were declared and
    unused, and were removed: `map_try_insert`, `const_cmp`).
    Load-bearing and worth watching: `const_trait_impl`, `const_ops`,
    `const_convert`, `const_destruct` are the const-traits family and
    the one that has already broken once — they move together, so
    advancing the pin means re-reading this section. `unboxed_closures`
    + `fn_traits` (postfix builder API) and `unsize` +
    `coerce_unsized` (widget handles) are pairs. The rest are
    individually small: `macro_metavar_expr_concat`, `portable_simd`,
    `associated_type_defaults`, `option_into_flat_iter`, and `gen_blocks`
    in the top crate.
    
    **Running it headless.** `iris/run-headless.sh EXAMPLE [--shot PNG]`
    with `iris/headless.conf`, the same trick `emu` uses: a headless
    sway, and `grim` for the picture. It deliberately starts its *own*
    compositor rather than joining `emu`'s — sway tiles, so adding a
    window to the one an emulator sits in resizes that emulator.
    Unlike `emu`'s it disables Xwayland, since winit speaks Wayland.
    **This VM has a real GPU for this**: Vulkan 1.4 through Venus onto
    the host's RX 7900 XT, and GL 4.6 through virgl — so desktop wgpu
    work here is not software-rasterised, unlike inside the emulator.
    
    **iris has no tests at all** (`cargo test --workspace`: 0 passed
    across 6 targets). Nothing to keep passing, and nothing to catch a
    regression — worth knowing before I1 changes the text stack.
    
    **`iris-core` no longer depends on winit, and now cross-compiles to
    Android.** It wanted exactly one thing from it — `PhysicalSize<u32>`
    in `UiRenderNode::resize`'s signature, for two numbers it immediately
    turned into floats — and that pulled a whole windowing backend into
    the layer below it, the wrong direction. `resize` takes
    `impl Into<Vec2>` now, like `UiRenderState::resize` beside it already
    did. The consequence is the point: with winit in the graph an Android
    build of the core failed in `android-activity` (which needs a backend
    feature nothing here selects), and without it
    `cargo ndk -t arm64-v8a -P 26 build -p iris-core` finishes in 30s and
    produces an rlib, wgpu's Android backend included. So **iris's
    widget, layout and render core already builds for the phone**, and
    what I2 has to supply is the surface, the input and the IME — not a
    port of the library.
    
    **Build weight, cold, on this VM's 8 cores** (`rm -rf target`, then
    `cargo build --example tabs`), since "the Linebender stack is slow in
    debug" was the worry behind this question: plain debug **43s** and a
    2.1 GB `target/`; with the `[profile.dev.package."*"] opt-level = 2`
    knob, **1m46s** and 1.5 GB. So iris's own wgpu + winit + cosmic-text
    graph is not the slow thing — which makes it a calibration for E1
    rather than an answer about Masonry, whose graph adds Vello, Parley,
    Fontique and Skrifa. Runtime cost of the knob was not measured here.
    
    **Open defect found while doing this: iris sometimes never adopts
    the window's real size.** Measured on the headless rig, ~3 starts in
    15: the `tabs` example settles showing its 800×600 startup layout in
    the top-left of a 1920×1200 surface, black around it, and stays that
    way indefinitely — it is not a screenshot taken too early, since the
    picture is byte-identical for the next four seconds. What is *not*
    the cause, each checked: the winit event order is identical in good
    and bad runs (`Resized(800×600)`, two redraws, `Resized(1920×1200)`,
    one redraw), the swapchain reports `1920×1200` and
    `suboptimal=false` on that last draw, and `output_size` is
    `(1920, 1200)` going into it. It is timing-sensitive in the way that
    makes it expensive: adding a single `eprintln!` anywhere in the draw
    or event path hides it completely (0 in 16), which is why the
    instrumentation above could not catch it in the act. A pointer move
    does not repair it, because iris only redraws when something
    changed; an output mode change does, because that is another resize.
    Left open rather than guessed at. It matters most for **I2**, where
    every rotation and every keyboard open is a resize, so a stale frame
    would be the normal case rather than a rare one; a Wayland-level
    trace of the xdg-surface configure/ack/commit sequence is the next
    step, not more `eprintln`.
    
    One thing was fixed on the way, and it is not that bug: `update`
    redrew everything when `resized` was set, but `needs_redraw` — which
    is what decides whether to *ask* for a frame — did not know about
    `resized` at all. The two now share one `needs_redraw_all`, since a
    condition in one and not the other is a frame nobody requests. It is
    latent on Wayland only because winit asks for a redraw after a resize
    by itself; on Android, where the surface work of I2 will not have
    winit underneath it, nothing else here would have asked.
    
  • I1 — the text stack decision. iris uses cosmic-text; the transcript needs rich inline spans (links, code chips, colour), selection across many widgets with the platform's handles on the phone, and an editor the IME can drive (composition regions, not just committed characters). Compare cosmic-text and Parley against exactly those three, in iris, on a real transcript's text. Parley is the expectation because android-view's IME bridge and AccessKit's text properties are written against it; measure rather than assume. Pass: a written decision here with what each was tried on, and the TODO's "text resizing per frame is really slow" measured and either fixed or explained.

  • I2 — iris on android-view. An android-view surface as a second backend beside winit: wgpu on the view's surface (GLES here, see the Vulkan section; Vulkan on the phone), touch as pointer events, window insets and the keyboard inset as layout inputs, the back gesture as an event, the IME bridge feeding the editor from I1. Pass: the tabs example and a text field run on the emulator, and the phone's own keyboard types into the field with autocorrect and suggestions — the same bar as E1.

  • I3 — a virtualised, bottom-anchored list. Variable-height rows, keyed, composed only while visible, paged in both directions with a "more" sentinel at each end, a scroll anchor that survives rows being inserted above, and "hold the edge nearest the tap" done in the layout pass. Pass: 800 rows of real transcript text from the sandbox scroll without a frame over the Compose baseline in transcript-bench.sh, measured on the GPU emulator.

  • I4 — accessibility names via AccessKit. Every control carries a name; ui-trace can find and tap it by label. Pass: bench-lib.sh's tap-by-name works against the iris screen unchanged.

  • I5 — the transcript screen in iris. E2's pass conditions, all seven behaviours, against the sandbox with --delay. This is the point the decision in the recommendation is made at.

For the next agent

What to do when you pick this up, in order, so nothing here has to be re-derived:

  1. Read this file, then AGENTS.md and PLAN.md. The rules there (measure, do not read; fix the rig before accepting its limits; the emulator is this checkout's own) all apply.
  2. Work on the rustify branch of this clone (ai-app-2), not on main and not in ai-app. Nothing on this branch is production until Iris says so. Commit and push as you go.
  3. Take the next unchecked box above, in order: E0 first, then E1, then the iris track from I0. E-steps and I-steps can proceed in parallel in separate sessions once E1 has proved android-view on this emulator.
  4. Every step ends with its measurement written into this file beside the box, and the box ticked or the reason it could not be written in its place. A step that is blocked says by what, not "later".
  5. Run the existing rigs rather than inventing new ones: ui-sandbox.sh for a server with fixtures, transcript-bench.sh for the scroll baseline, ui-trace for anything positional, emu up for the emulator. The Vulkan section below says how to get a Vulkan path in the emulator when a wgpu backend needs one.
  6. Decisions belong here with a date and what was rejected, the way PLAN.md does it. Do not put design into commit messages alone.

Vulkan in the emulator (measured 2026-09-04)

A wgpu app in this emulator was going to get GLES only, because host Vulkan is switched off in emulator-tools. Retried today on Mesa 26.1.7: Venus still fails the same way — gfxstream picks externalMemoryMode: OpaqueFd, probes VK_FORMAT_R8G8B8A8_UNORM for an exportable colour buffer, and Venus says the format is unsupported (Failed to find memory type for ColorBuffers, fatal before adb sees the device). Venus does advertise VK_KHR_external_memory_fd and VK_EXT_external_memory_dma_buf, so the gap is specifically opaque-fd image export. gfxstream has a string-valued VulkanExternalMemoryMode setting ("overrides what would otherwise be determined automatically"), but -feature Name=Value is rejected as a bad feature name, and the only mode words compiled into this emulator's libgfxstream_backend.so (37.1.11) are OpaqueFd, Metal and none — there is no dma-buf mode in this build to switch to. So Venus is blocked by the emulator, not by Mesa; retry when the emulator package updates, since upstream gfxstream does have dma-buf external memory.

What does work: pointing the emulator's Vulkan loader at the software ICDs the emulator ships itself, with the feature enabled:

VK_DRIVER_FILES=$HOME/Android/Sdk/emulator/lib64/vulkan/vk_swiftshader_icd.json \
GPU_HOST_FEATURES="-feature Vulkan" emu up

The guest then reports Vulkan 1.3 (cmd gpu vkjson, SwiftShader Subzero) while GLES still runs on the real GPU through virgl — so a Vello/wgpu app can take its real Vulkan path here, with compute shaders, CPU-rasterised. That is enough to test correctness of the Vulkan path in the emulator; GPU performance of it is a phone measurement either way, exactly as MACHINE.md already says about frame times. lavapipe (lvp_icd.json, the other ICD the emulator ships) selected llvmpipe and booted, then the emulator died right after loading the default_boot snapshot with nothing in the log; a snapshot saved under a different Vulkan device is the suspect, and -no-snapshot-load is the untested next step. SwiftShader is the one that works today. Making this an emu option belongs in emulator-tools and is a shared tooling change, so it goes through the other sessions first.

Things a Rust app changes elsewhere

  • wg-app-link's :link (pinned TLS, enrollment store, QR activity) is Kotlin shared with Dev Updater. The certificate code already exists on the Rust side of the submodule; the pinned-CA build step (generatePinnedCert) becomes a build.rs reading the same path. The QR scanner stays a Kotlin activity, since the camera is a platform feature.
  • Tooling becomes cargo for everything but packaging: cargo test, clippy, fmt cover the whole client, which is the motivation. Gradle remains for the APK, signing (~/.config/ai-app/release.jks) and Dev Updater's build modes; build-apk.sh would call cargo ndk first.
  • The bench scripts (ui-trace by accessibility label) keep working only if the framework exposes names through AccessKit on Android; that is part of E2's pass condition, not a nicety.
  • Icons stay Nerd Font glyphs from the committed subset; Parley/Fontique loads a font file directly, so build-icon-font.sh is unchanged.

Sources