iris/ai-app
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Prune commentary and stale Rust port notes

Browse Source
This commit is contained in:
iris committed 2026-09-10 00:44:13 -04:00
1 parent 5428cd75c9
commit 25370731d0
193 files changed
+693 -16219
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/RUST.md
+47 -334
View File
@@ -1,50 +1,18 @@
# Moving the app to Rust
Working document for the port Iris asked for on 2026-09-04: the phone app
in pure Rust, one UI framework shared with a desktop app, at full feature
parity and giving up nothing native -- performance especially. Her
constraints: no Dioxus and nothing that draws through a WebView; **no UI
DSL** (which ruled out Makepad and Slint); the result stays lightweight;
platform-specific pieces are fine to maintain; reimplementing a framework
piece from scratch where it does not fit is fine; effort and elapsed time
do not matter, long-term robustness does.
**The framework question is closed.** Iris chose her own library,
[iris](https://github.com/cat16/iris), over Masonry on 2026-09-05.
The bake-off that got there, and the twelve experiments
that proved it on a device, are summarised in "What the experiments
settled" below rather than kept at length. What is left in this file is
the plan for the rest of the app and the findings that outlive the tasks
that produced them.
Decisions get a date and a reason here, the way `PLAN.md` does.
Plan for a native Rust phone app with full feature parity and a shared desktop
UI. It uses [iris](https://github.com/cat16/iris); platform-specific entry
points are acceptable, but shared screens, widgets, and styling are not
duplicated. The result must stay lightweight and preserve native behavior and
performance.
## Keep this file current as you work
**This file is the handoff, and it is meant to let a session be cleared.**
Write each result into it *as you get it*, not at the end: the box ticked
or the reason it could not be, the measurement with its number, the
decision with its date and what it rejected, and anything that cost time to
find out. Then a session that has filled its context can be cleared and the
next one can pick up from this file alone, which is much cheaper than
carrying a long conversation or re-deriving what was already measured.
Keep open work, current design, measured constraints, and dead ends that would
otherwise be repeated. Delete completed plans and migration narratives. Name
the command and measured value when evidence matters.
Two things that follow. Write for somebody who was not here -- name the
command, the file and the number rather than "the fix" or "the earlier
run". And write the failures and the dead ends too: "Venus is blocked by
the emulator, not by Mesa" and "the present mode was not the cause" are
worth as much as the successes, because they are what stops the next
session spending an afternoon on them again.
**And delete a plan once it has been carried out** (Iris, 2026-09-08:
*"remove everything that's already done and decided... many with checkboxes
already ticked off that just fill up context"*). A ticked box has done its
job; a finished experiment is worth one line saying what it settled, not
the log of settling it. Currency means this file says where things *are*,
not how they got here. What survives a prune is what cannot be cheaply
re-derived: measurements, dead ends, invariants and their reasons.
## Where things stand (2026-09-09)
## Current status
- **The framework is decided and built on.** iris draws the transcript
screen on the desktop, on this checkout's emulator and on Iris's phone.
@@ -52,14 +20,12 @@ re-derived: measurements, dead ends, invariants and their reasons.
phone and the reports are under `docs/bench/`.
- **P1 (session screen parity) is the current work**, and is where the
next session should start. Its box below has the state.
- **The repository was reorganised on 2026-09-08**: the port is one crate,
`app-rust/`, and `iris/` is the UI framework alone. See "One app crate"
at the end -- it is the layout everything else here assumes.
- The port is one crate under `app-rust/`; `iris/` is only the UI framework.
- **Open across the rest of the docs**: `docs/IRIS_TODO.md` is iris's own
list (colour-space correctness is the live one), `docs/TODO.md` is the
Compose app's.
## Desktop and phone share the code (Iris, 2026-09-07)
## Desktop and phone share the code
Iris plans to develop a desktop app as well, and asked that most code be
sharable between desktop and phone. The tree already has that
@@ -332,76 +298,9 @@ rather than what it happens to look like:
7. **Measurable frames**: the debug render report, and a way to attribute
a frame's cost to a widget on the real phone.
## What the experiments settled
## Measurements and constraints
Twelve boxes, all closed between 2026-09-04 and 2026-09-05, and all
deleted on 2026-09-08 now that their conclusions live in the code. One
line each for what a later session must not re-derive; where a decision
needs its reasoning, the reasoning is at the thing itself.
**The framework track (E0-E5), against Masonry:**
- **E0 -- toolchain.** NDK r29 (`29.0.14206865`) under `~/Android/Sdk`,
cargo-ndk 4.x. Its API-level flag is `-P`; `-p` now means `--package`.
- **E1 -- android-view's Masonry demo ran here**, on the GPU, with an
accessibility tree and the phone's real keyboard -- but no autocorrect
and no suggestions. The `android-view` rev this was measured against is
pinned in `app-rust/Cargo.toml` with that history at the pin;
`accesskit_android`'s detach-abort is mitigated in
`iris/src/android/view.rs`'s `raise_if_enabled`, and advancing the
version is not the fix.
- **E2 -- a transcript in Masonry** found the framework-wide gap that
blocked the comparison. It lived in `~/src/android-view/e2-transcript`
and was never committed here.
- **E3/E5 -- the Kotlin shell and the packaging xtask.** Both hold:
`app/shellApp` plus the JNI bridge (now `app-rust`'s `shell` feature)
posts a real notification and receives a real share, and `cargo xtask
apk` packages an installable APK with `javac`/`d8`/`aapt2`/`zipalign`/
`apksigner` and one disclosed Gradle call, documented at
`scripts/xtask/src/apk.rs`'s module doc.
- **E4 -- the same screen on the desktop**, which is now
`app-rust`'s `src/desktop` and the `ai-app-desktop` binary.
**The iris track (I0-I5):**
- **I0a -- iris is vendored at `iris/`**, history not carried, consumed by
path, from `iris/iris` on gitea at `7b54aaf`. It goes back to its own
repository once it has proved itself.
- **I0b -- the nightly pin is dated, not rolling** (`rust-toolchain.toml`,
one copy in `iris/` and one in `app-rust/`, because a pin applies per
directory). Dated because a rolling channel moved `impl const Trait` to
`const impl Trait` underneath the vendored tree and broke it unattended.
- **I1 -- parley, plus a glyph atlas.** Both Iris's call. Parley addresses
text by byte offset into one string, which is why the editing model
looks the way it does.
- **I2 -- iris runs on android-view**: the backend, the Gradle shell,
insets, the back gesture and the full `InputConnection` bridge, with
real Gboard suggestions.
- **I3 -- the virtualised list.** Since renamed `LazySpan`, and scrolling
has moved out of it into `ScrollController` -- `docs/SCROLL.md` is the
current design, not this box.
- **I4 -- accessibility names through AccessKit**, one flat tree with a
synthetic `Role::Window` root and every *named* widget a direct child.
Flat deliberately: nothing upstream of a named leaf needs a node. This
is what lets `ui-trace` tap by label.
- **I5 -- the transcript screen in iris**, with `FrameReport` for
frame timing. Its descendants are `app-rust/src/ui` and every
measurement rig in AGENTS.md.
**Two findings from that period that are still load-bearing, kept where
they belong rather than here:** iris's binding array does not survive real
Android hardware (the measurement and the fix are `docs/TEXTURES.md`'s
"Implemented, 2026-09-04"), and the emulator has no hardware Vulkan while
its GLES *is* the host's real GPU through virgl (moved to the
`this-machine-android` skill on 2026-09-08, with the `gpu-probe` output
that established it).
## Findings that outlive the task that produced them
Kept because the number or the constraint is what stops it being
re-derived; the tasks themselves are done and deleted.
### The Android release profile, and where the APK's size went (2026-09-07)
### The Android release profile and APK size
Iris asked why the iris bench APK was double the Compose one (20.6 MB vs
10.1 MB). It was almost all `libmain.so`, built with `panic = "abort"` and
@@ -423,7 +322,7 @@ vectorisation on a renderer. Everything else is
own rather than `release`, so the desktop build is not also optimised for
size.
### The fling stutter, and what a frame report could not say (2026-09-09)
### The fling stutter and what a frame report cannot say
Iris, from her phone: *"I'm noticing some stuttering when flinging in
particular. Harder to notice with my finger directly moving the scroll."*
@@ -515,7 +414,7 @@ The signature of the fixed loop, from that run: `build p50 0.4ms,
acquire p50 5.7ms, submit p50 1.7ms` -- four tenths of a millisecond of
work and the rest of the refresh period spent waiting its turn.
### Streaming is where the frame time is now (2026-09-09)
### Streaming frame time
Measured after the fling was fixed, and it is not where it looks.
`frame_profile.rs`'s stream run: folding an arriving event is 0.35ms and
@@ -530,7 +429,7 @@ every delta.
exact shape of the Compose lesson in AGENTS.md's "Things that have
bitten" -- and measuring it is what ruled it out.
### Incremental text: parley cannot, and it turns out not to matter (2026-09-09)
### Incremental text shaping
Iris asked to investigate incremental text rendering and hoped parley
supported it. **It does not, by design.** The crate's own docs: a
@@ -635,7 +534,7 @@ the reply into blocks was still right -- it is what makes the fixture
representative, and it halved the CPU half -- but it was never going to
move this, and it slightly increases the primitive count.
### The arenas upload deltas, and stopped being 11x too big (2026-09-09)
### Arena delta uploads
Done, and measured by `scripts/rigs/ui-profile`'s `arena_churn` -- see
AGENTS.md's entry for the rig and the numbers. The arithmetic above was
@@ -697,7 +596,7 @@ instance bytes per frame are **1,488**, from 176,496. This is framework
layout/rendering behaviour and the transcript screen contains no special
case for it.
### The Android release profile is `opt-level = 3`, not `"s"` (2026-09-09)
### The Android release profile uses `opt-level = 3`
The table above was measured in bytes only. `"s"` costs the loop
vectorisation and inlining a renderer runs on: over the same warm fling
@@ -708,7 +607,7 @@ refused for `"z"`, one level further up. Iris raised it herself
(*"I'd make sure it's in release mode"*); the build always was, and this
was the part of "release" that was not about speed.
### Platform fonts, not bundled ones (2026-09-07)
### Platform fonts
Iris: *"remove the font for now; just match what compose does."* The
Compose app takes body text from `FontFamily.Default` and code from
@@ -733,26 +632,9 @@ desktop cannot answer it. Before the next phone build, look at a bold run
and at `CLOSED_MARK`/`OPEN_MARK`/`UP_MARK` (U+25B8/BE/B4) on Iris's own
device; the emulator's font set is not evidence for hers.
### Hit-testing does not consult the mask chain (review R2, 2026-09-07)
## The port, in order
Masks are applied in the fragment shader
(`iris/core/src/render/shader.wgsl`); the CPU hit path
(`UiRenderState::resolved_region`) does not look at `masks` at all. So a
straddling row's clipped-away top is invisible and still tappable -- a tap
on "Run benchmark" can land on an invisible link in the row behind it.
Left deliberately: `docs/LAYOUT.md`'s mask redesign ("masks reference a
drawn primitive instead of copying a shape") is where hit-testing gets the
shape, and intersecting a chain in `resolved_region` now would be a second
mechanism to unpick.
## The port, in order (decided 2026-09-05)
The ordered plan for the rest of the app, decided here per Iris's standing
"decide technical questions yourself" instruction -- no serious
user-facing tradeoff is in play in the ordering itself.
**Where the screens live** was settled by the 2026-09-08 reorganisation
("One app crate", below): every screen is a module under
Every screen is a module under
`app-rust/src/ui`, which holds a `Screen` enum and a back stack -- the
direct equivalent of `AppRoot.kt`'s `when` and `MainScreen.kt`'s tab
`enum` -- with each Compose screen becoming one `iris::widget` subtree.
@@ -776,55 +658,15 @@ AVD, `ui-trace` by accessibility name, GrapheneOS phone quirks, the
running any pass condition below that touches an emulator or a real
device.
- [x] **P0 -- the phone benchmark gate. Passed.** Asked for 2026-09-05,
delivered and run on Iris's own phone; the reports are under
`docs/bench/`. Both halves are still in the tree and are how a
frame-time comparison is taken: the Compose `bench` build type
(`app/`, `BenchFixture.kt`/`BenchRun.kt`) and the Rust `bench`
feature (`app-rust`, `src/android/bench_client.rs`), opening the
same checked-in synthetic transcript
(`app/bench-fixture/assets/transcript.jsonl`, never a real one) with
no server, driving the same scroll loop and streaming phase, and
printing the same report fields. AGENTS.md's "The rigs" is the
current description; `app-rust/build-apk.sh` and `run-bench.sh` are
how it is run. The build source is this repository, but the Dev Updater
publication is the separate `~/repos/ai-app-bench` repository, whose
`iris-bench` component serves a committed APK with no build step. After
an arm64 release build, replace its
`iris/build/outputs/apk/release/iris-bench-arm64.apk` and push that
repository; adding the component here or pushing only `ai-app-2` is the
wrong delivery path.
**Bench-only cleanup still open**: the diagnostics report pane draws over
transcript rows. `REPORT_MAX_HEIGHT_DP` constrains its claimed height, but the
pane is neither masked nor scrollable despite its construction comment saying
it is both. This is an `app-rust` defect, not an iris framework item.
**Bench-only cleanup still open**: the diagnostics report pane draws
over transcript rows. Reproduced on the emulator on 2026-09-09 by
opening the named `Diagnostics` control. `REPORT_MAX_HEIGHT_DP`
constrains its claimed height, but the pane is neither masked nor
scrollable despite its construction comment saying it is both. This is
an `app-rust` bench-screen defect, not an iris framework item.
- [ ] **P1 — session screen parity.** **Started 2026-09-06, on Iris's
word**: "just continue with the plan for now; try to move towards
feature parity for the transcript screen so that the test can be
more fair." So P0's "must pass before P1 starts" is lifted — the
phone bench continues alongside, and parity is what makes its
comparison fair. **Sub-order, by what the bench fixture exercises
and Compose already draws** (tick and date each in place):
- [x] **P1a — markdown block rendering parity.** Done 2026-09-06.
Each top-level block is drawn in one of three frames
(`ui::markdown::BlockFrame`) — plain, verbatim, quote — with
fences and tables verbatim, headings scaled, and inline
styling per span. `app-rust/src/ui/markdown.rs` is the code
and its module doc the design.
- [x] **P1b — tool-call cards and grouping.** Done 2026-09-06.
`ToolRows.kt`/`ToolInput.kt` ported to
`app-rust/src/ui/tool.rs`: a run of calls is one collapsible
group, each card carries its state and summary, and the five
`ToolState` values each have their own appearance.
`tool.rs`'s module doc has what was chosen.
- [ ] **P1 — session screen parity.** Continue in this order:
- [ ] **Before the next parity slice — make iris's colour pipeline
correct.** Raised by Iris on 2026-09-09 as something to settle
sooner rather than later. Both backends currently prefer an
sRGB surface while the shader returns palette/image bytes as
correct.** Both backends currently prefer an sRGB surface while
the shader returns palette/image bytes as
linear values; `IRIS_TODO.md` has the measured mismatch and
pass condition. Do this before judging or centralising the
app's styling. It is correctness, not cosmetic polish.
@@ -1038,169 +880,40 @@ device.
once more against a real `ai-server` (not the sandbox) on a real
phone, side by side with the Compose build until it holds.
## For the next session
## One app crate
What to do when you pick this up, in order, so nothing here has to be
re-derived. **The work is done inline, not handed to subagents** — Iris
said so on 2026-09-08 ("I'm no longer using subagents for this. Please do
the work yourself"), so read the code, make the change, run the tests and
push, in the session that picked the task up.
1. Read this file, then `AGENTS.md` and `PLAN.md`. The rules there
(measure, do not read; fix the rig before accepting its limits; the
emulator is this checkout's own) all apply.
2. Work on the **`rustify`** branch of this clone (`ai-app-2`), not on
`main` and not in `ai-app`. Nothing on this branch is production until
Iris says so. Commit and push as you go.
3. The E- and I-steps (the framework decision) are done — iris won,
decided 2026-09-05. **Fix iris's colour-space pipeline first**, as Iris
requested on 2026-09-09; then take P1c, history paging and
jump-to-latest. The client-side paging pieces it needs are already
ported.
4. Every step ends with its measurement written into this file beside the
box, and the box ticked or the reason it could not be written in its
place. A step that is blocked says by what, not "later". Write it as you
go rather than at the end — see "Keep this file current as you work".
5. Run the existing rigs rather than inventing new ones: `ui-sandbox.sh`
for a server with fixtures, `transcript-bench.sh` for the scroll
baseline, `ui-trace` for anything positional, `emu up` for the
emulator, `iris/run-headless.sh EXAMPLE --shot PNG` for an iris
example on this displayless machine, and `scripts/rigs/gpu-probe` to ask a
device (this VM, the emulator, or a real phone over `adb push`) what
`wgpu` features and limits it actually has before building anything on
the assumption it does. The Vulkan section below says how to get a
Vulkan path in the emulator when a `wgpu` backend needs one.
6. **Bound anything heavy at the moment you start it.** An emulator or a
long build gets a deadline — `timeout`, or a watchdog scoped to the pid
you just started — rather than a plan to stop it later. Scope it to
that pid: a watchdog written as `sleep N; emu down` fired into a later
experiment here and made a working Vulkan build look like a crash. And
stop the emulator when the work needing it is done rather than between
tasks.
7. Decisions belong here with a date and what was rejected, the way
`PLAN.md` does it. Do not put design into commit messages alone.
## Things a Rust app changes elsewhere
- **`wg-app-link`'s `:link`** (pinned TLS, enrollment store, QR activity)
is Kotlin shared with Dev Updater. The certificate code already exists on
the Rust side of the submodule; the pinned-CA build step
(`generatePinnedCert`) becomes a `build.rs` reading the same path. The QR
scanner stays a Kotlin activity, since the camera is a platform feature.
- **Tooling** becomes `cargo` for everything but packaging: `cargo test`,
`clippy`, `fmt` cover the whole client, which is the motivation. Gradle
remains for the APK, signing (`~/.config/ai-app/release.jks`) and Dev
Updater's build modes; `build-apk.sh` would call `cargo ndk` first.
- **The bench scripts** (`ui-trace` by accessibility label) keep working
only if the framework exposes names through AccessKit on Android; that is
part of E2's pass condition, not a nicety.
- **Icons** stay Nerd Font glyphs from the committed subset; Parley/Fontique
loads a font file directly, so `build-icon-font.sh` is unchanged.
## One app crate, 2026-09-08 (the repository reorganised)
Iris, reading the tree: *"the organization of the rust rewrite is a mess
right now… there shouldn't be anything related to the app inside of iris.
Iris is supposed to be the UI framework alone."* Then, on the crate count:
*"I'm confused why the app only code needs more than one crate though."*
### What it was
Nine cargo workspaces, each with its own `Cargo.lock` and `target/`, and
the port's project code in five places — `iris/transcript-ui`,
`iris/transcript-fixture`, `iris/desktop-app`, `iris/android-app` (all
*inside* the framework), plus `client-core` and `android-shell` at the
root. Two root markdown files sat outside
`docs/`.
### What it is
**One crate, `ai-app`, in `app-rust/`.** Modules, not crates:
| was | is |
|----------------------------------------|-----------------------------------|
| `client-core` | `src/client` |
| `iris/transcript-ui` | `src/ui` |
| `iris/transcript-fixture` | `src/ui/fixture.rs` + `tests/`, `touch/` |
| `iris/desktop-app` | `src/desktop` + `src/bin_desktop.rs` |
| `iris/android-app` | `src/android` + `android-project/` |
| `android-shell` | `src/shell` |
The Rust client is one `ai-app` crate in `app-rust/`: platform-free code
is under `src/client` and `src/ui`, while `src/desktop`, `src/android`, and
`src/shell` contain the platform entry points. The fixture is behind its
own feature so its 1.9 MB `include_str!` does not enter ordinary phone
builds.
`iris/` now holds `core`, `macro`, the `iris` crate, `tabs-ui` and
`rig-input` — framework only, with no mention of a session, a transcript,
a setup or a server anywhere in it.
### Why one crate really is enough
`src/client` must not depend on iris. Features select the crate's face:
`screens` for UI builds, `shell` for the Compose shell bridge, and `bench`
for the fixture. The Android faces both produce `libai_app.so`.
Each split had a stated reason at the time; on inspection only two
survived, and one of those is not in `app-rust` at all.
`event-model` remains separate because both the server and client depend
on that wire contract. Iris remains a separate UI-framework workspace and
must contain no product concepts.
- **`client-core` separate from the UI** was "pure logic with no framework
dependency". That property is worth keeping and does not need a crate:
`iris` is behind the `screens` feature and `src/client/` may not reach
it. An invariant on a module instead of on a manifest, stated in
docs/CLIENT_CORE.md.
- **`transcript-fixture` separate from `transcript-ui`** was so the
headless harness and a desktop window opened the same bytes. Both are
now the same crate, so it is `src/ui/fixture.rs` behind a `fixture`
feature (1.9 MB of `include_str!` must not reach a phone build) with the
six harness suites in `tests/`.
- **Two Android `.so` names**, `libmain.so` for the iris app and
`libandroid_shell.so` for the Kotlin shell's JNI bridge, looked like the
one hard constraint: a package produces exactly one library artifact.
It dissolves because **P2 already plans to merge those two Android apps
into one**. So both faces come out of one package as `libai_app.so`,
picked apart by features (`--no-default-features --features shell` keeps
wgpu, parley and iris out of the Compose app's APK), which is the
direction of travel rather than a workaround. `xtask apk` and
`app/shellApp`'s `System.loadLibrary` were updated to match.
- **A desktop binary and an Android cdylib in one package** is not a
problem: `iris` itself already target-gates winit against android-view
in one manifest, and the same table does it here. `build-apk.sh` passes
`--lib` so `cargo ndk` never tries to build the desktop binary.
- **`event-model` stays a crate**, and is the one split that was never
optional: `server/` depends on it too, so a crate is what makes the
backend and the app agree by construction. Iris chose to leave it at the
repo root rather than inside `app-rust/`, since it is the contract
between the two rather than app code.
### Build constraints
So: three workspaces where there were nine — `event-model`, `server`,
`app-rust` — plus `iris` and `xtask`.
### Things that moved with it, worth knowing
- **The toolchain pin is per directory.** `app-rust/rust-toolchain.toml` is
a copy of `iris/`'s, because `client-core` used to build on stable and
now shares iris's dated nightly. Two consequences appeared immediately:
two `needless_range_loop` warnings in the markdown highlighter (fixed),
and four `AtomicBool::fetch_update` deprecations from inside `jni`
0.22's `native_method!` macro. The last are not ours to migrate — the
fix is a `jni` release — so `src/lib.rs` carries an `#[allow(deprecated)]`
scoped to `mod shell` with that reason written at it.
- **The rolling nightly setting is per directory.** The toolchain files in
`app-rust`, `iris`, and `scripts/rigs/ui-profile` must stay synchronized.
- **The Android release profile is `android-release`, not `release`.** The
aggressive settings `iris/android-app` had (`panic = "abort"`,
`opt-level = "s"`, fat LTO) would otherwise apply to the desktop build
too, which is a testing surface. `build-apk.sh` passes
`--profile android-release` / `--profile android-dev`.
- **`iris/run-headless.sh` grew `--dir DIR`**, defaulting to `iris/`. The
rig belongs to the framework; the examples it usually runs no longer do.
`replay-touch` is still built from `iris/`.
- **The log target changed** from `client_core` to `ai_app`
(`src/client/log_ring.rs`'s `is_own_target`).
- **`iris/run-headless.sh --dir DIR`** selects the workspace containing the
example; it defaults to `iris/`.
- **Not renamed, deliberately:** the Android application id and Java
package are still `dev.iris.android.demo` and the label is still "iris
android-view demo", both now misleading. Changing them changes the app's
identity on Iris's phone (a side-by-side install rather than an upgrade)
and the `DevLogProvider` authority Dev Updater reads, so it is hers to
decide rather than a tidy-up to make quietly.
### Verified
`./scripts/run-tests.sh` (event-model, server, app-rust) and `cd iris && cargo
test` green; `cargo clippy --all-targets` and `cargo fmt` clean in every
workspace. `cargo ndk -t x86_64` links `libai_app.so`; `./build-apk.sh
debug --abi x86_64` produces an installable APK; installed and launched on
this checkout's emulator, drawing through `Gl … virgl` as expected. The
phone-sized headless screenshot (`run-headless.sh phone --phone --dir
../app-rust --shot …`) renders the transcript unchanged.
and the `DevLogProvider` authority Dev Updater reads, so changing them
requires an explicit migration decision.