iris/ai-app
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

E3: the Kotlin/Java shell over a JNI bridge into Rust (RUST.md)

Browse Source 
Two Java classes (MainActivity, NotificationService) hand their lifecycle
to a new android-shell crate built on client-core; client-core gains
notifications.rs (the /notifications SSE parse and attention_line, ported
from Notifications.kt). Packaged as a new app/shellApp Gradle module
rather than a rewrite of app/androidApp in place, so that module's working
Compose UI is untouched.

Both pass conditions held on the emulator: a notification arrived in
Android's drawer with the app closed, and a shared text share landed as a
real message in a sandbox session's transcript. Found and fixed three
real bugs along the way (a silently-wrong JNI signature from a generic
JObject parameter, a class-by-name lookup failing on this crate's own
background thread for lack of an app ClassLoader, and onStartCommand
opening two /notifications connections per enrollment -- the last a
latent bug in Notifications.kt itself). Full account, exact commands and
what was deliberately cut are in RUST.md's E3 box.

Co-Authored-By: Claude Sonnet <noreply@anthropic.com>
This commit is contained in:
irisandClaude Sonnet committed 2026-09-05 06:47:32 -04:00
1 parent 8adda94a7a
commit c9b273ff16
16 files changed
+2996 -6
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

RUST.md
+270 -6
View File
@@ -39,8 +39,28 @@ session spending an afternoon on them again.
- **Done**: E0 (toolchain), E1 (Masonry on android-view, which found the
keyboard gap — now explained, see below), E2 (a transcript in Masonry,
which found that Masonry has no touch-scroll on Android at all — see
below), I0a, I0b (iris builds on a pinned nightly and runs), I1 (parley +
glyph atlas).
below), **E3 (the Kotlin/Java shell over a JNI bridge into Rust, both
pass conditions proved on the emulator — see its own box)**, I0a, I0b
(iris builds on a pinned nightly and runs), I1 (parley + glyph atlas),
I2 (iris on android-view).
- **E3 done, 2026-09-05, and unlike E1/E2 it is committed to this repo**
(`android-shell/` — a JNI-bridge crate on `client-core` — plus a new
Gradle module `app/shellApp/`, left deliberately separate from
`app/androidApp` so its ~13,000 lines of working Compose UI are
untouched). Both pass conditions held: a notification arrived in
Android's drawer while the app was closed, and a shared text share
landed as a real `userMessage` in a sandbox session's transcript. Found
and fixed three real bugs along the way — a generic `JObject` native
parameter silently exporting the wrong JNI signature
(`UnsatisfiedLinkError`), a class-by-name lookup failing from this
crate's own background thread because a Rust-attached thread has no app
`ClassLoader` (`Error::NoClassDefFound`, invisible without a logger
installed), and `onStartCommand` opening two `/notifications`
connections per enrollment — the last one a latent bug in
`Notifications.kt` itself, found here rather than there. See E3's own
box for the full account, the exact commands, and what was deliberately
cut (attachment uploads, a session picker, the on-screen/banner
suppression — all pending E4's screen).
- **E2 done, 2026-09-05, and its headline finding changes what "decide
from the measurements" (recommendation item 3) can mean right now.**
Built a real transcript screen (`~/src/android-view/e2-transcript`,
@@ -969,10 +989,236 @@ accepted.
upstream commit does not yet have for this exact case. That is a
point in iris's favour that a frame-time number would not have
shown any more clearly.
- [ ] **E3 — the shell.** Kotlin `MainActivity` + `NotificationService`
+ Keystore + share intent calling into Rust over JNI, with the SSE
follow loop in Rust. Pass: a notification arrives with the app
closed, and a share lands in a session.
- [x] **E3 — the shell (2026-09-05).** Both pass-condition proofs held on
the emulator: a notification arrived while the app was closed, and a
shared text share landed as a real message in a session's transcript.
Committed to this repo (unlike E1/E2's external, uncommitted trees),
since this is lightweight glue rather than a multi-gigabyte native
build.
*Where it lives.* `android-shell/` (new crate, `client-core` as its
only real dependency) is the JNI bridge; `app/shellApp/` is a **new
Gradle module**, not a rewrite of `app/androidApp` in place --
that module is ~13,000 lines of working Compose UI this experiment
does not touch or risk, and the two install side by side on one
development device. `app/shellApp`'s manifest, channel names,
notification wording and share intent-filter are copied from
`androidApp`'s (`Notifications.kt`, `Share.kt`, the manifest) per
AGENTS.md's "reuse rather than re-derive" -- see each file's own doc
comment for exactly what was carried over. Two deliberate
differences, both practical rather than behavioural: application id
`com.example.aiapp.shell` and deep-link scheme `aiappshell` (not
`aiapp`), so this experiment's install cannot collide with the real
app's enrollment or Keystore alias on the same phone -- see
`android-shell/src/settings.rs`'s `SCHEME` doc.
*The Java floor, and one line more than planned.* Two classes,
matching "How much Java is unavoidable" almost exactly:
`MainActivity.java` (`onCreate`/`onNewIntent` forward to
`nativeHandleIntent`) and `NotificationService.java`
(`onStartCommand`/`onDestroy`/a `sync()` companion, three natives).
Both ~30 lines including the license-free boilerplate Java itself
demands (imports, `System.loadLibrary`). **One addition the analysis
did not anticipate**: `MainActivity.toast(Context, String)`, a
plain (non-native) static method Rust *calls* rather than
implements, because posting a `Toast` from `share.rs`'s background
thread needs a hop back to the main looper
(`new Handler(Looper.getMainLooper()).post(...)`), and JNI can call
an existing Java method on any thread but cannot construct a Java
`Runnable` to hand to `Handler.post`/`runOnUiThread` without a
reflection proxy uglier than three lines of Java. Recorded here
because "the floor is two classes of ten lines" undersold this by
exactly one small, call-only method -- the pattern (Rust calls
Java, never Rust implements a Java interface) is worth keeping the
next time this floor is estimated.
*What client-core gained.* `notifications.rs`: `SessionNotification`,
`NotificationKind` (mirroring `server/src/session/mod.rs`'s wire
shape field-for-field) and `follow_notifications`, the SSE parse
over `/notifications` built on the same `sse::SseReader` and
`Transport` trait `event_stream.rs` already uses. `attention_line`
is ported verbatim from `Notifications.kt`. 3 new tests (88 total in
the crate); `android-shell` itself has none, since every function in
it needs a live `Env` and there is no pure logic left to test in
isolation once client-core owns the parsing -- matches E2's
precedent ("a throwaway screen, not a library").
*Scope cuts, each recorded at its own point in the code rather than
only here:*
- **Text-only share.** `Intent.EXTRA_TEXT` becomes a session message;
a shared file/photo URI is not uploaded, because `client-core`'s
`ApiClient` has no `/sessions/{id}/attachments` route yet either
(`CLIENT_CORE.md`'s own "not covered" list) -- porting
`Attachments.kt`'s `ContentResolver` reads and bitmap downscaling
is real work belonging to whichever caller needs it next, not a
detour inside this box.
- **No session picker.** With no screen drawn yet (E4's job), a
share attaches to whichever session has the latest
`last_activity` -- documented as a placeholder in `share.rs`,
not a designed behaviour.
- **No banner/on-screen suppression.** `notify::show` skips
`Notifications.kt`'s "nothing if this session is on screen" /
"hand to the app as a banner" branches entirely: both read
process-wide state that only means something once a screen
exists to register against it, so every notification here takes
the platform-drawer branch -- which is also exactly what the pass
condition asks for. Revisit once E4 draws something.
- **Keystore is not reimplemented in Rust.** `settings.rs` calls
`wg-app-link`'s existing `ServerStore`/`ServerSettings` Kotlin
classes over JNI rather than re-deriving the AES-GCM sealing:
that code is shared with Dev Updater, already tested, and tied to
a Keystore alias an existing enrolled phone depends on. This does
mean `kotlinc` stays in the toolchain regardless of what E5 does
with `javac`/`d8` for this module's own two classes -- a
correction to "Can the APK be built without Gradle?"'s assumption
that dropping Kotlin drops `kotlinc` outright; it drops it for
*this app's own code*, not for a shared submodule pulled in as a
dependency.
*`jni` 0.22, not the older API most examples assume.* This is a
real API split (`Env` for real work, `EnvUnowned` as the FFI-safe
type a native fn receives, joined by `EnvUnowned::with_env`), and
the `native_method!` macro (used for all four natives here, via
`const _: NativeMethod = native_method! { ... }`) generates both the
mangled `Java_...` export and the panic/error-handling wrapper from
one Rust function signature -- chosen over hand-written
`#[unsafe(no_mangle)] extern "system" fn Java_com_..._method` because
a hand-typed export name and a hand-typed JNI signature string
routinely drift from the Java they claim to match, silently (see
the next two findings, both of which were exactly that drift).
`error_policy = LogErrorAndDefault` reports a failure to logcat
rather than throwing it back into Java as an exception that would
crash the app over something recoverable -- matching
`Notifications.kt`'s own "log, don't crash" posture, but it is a
no-op without a logger backend (`android_logger`, Android-only
dependency, `lib.rs`'s `ensure_logger`) installed; the class of bug
this exists to report was found once with no logger and read as
nothing having gone wrong at all.
**Three real findings, each cost a failed run before being
diagnosed, each written where the fix lives so a reader who touches
that file again does not lose an afternoon to it:**
1. **A generic `JObject` parameter type silently exports the wrong
JNI signature.** `native_method!`'s shorthand
`fn native_sync(context: JObject) -> ()` encodes the export as
`(Ljava/lang/Object;)V`, because it has no way to know the
intended Java type is `android.content.Context` from a bare
`JObject`. The real Java method is declared
`(Landroid/content/Context;)V`; the two mangled names never
resolve to each other, and the failure is
`UnsatisfiedLinkError: No implementation found`, thrown the
moment Java calls it -- not a build error on either side. Fixed
by spelling each parameter as its actual Java type in the macro
invocation (`context: android.content.Context`, `activity:
android.app.Activity`, ...), which the macro accepts directly
per its "Java Object Types" syntax, while the Rust implementation
function keeps the parameter as plain `JObject` (the "Built-in
Types" fallback for a Java class with no dedicated Rust
wrapper). `lib.rs`'s comment beside the first `native_method!`
call is the citation.
2. **A class looked up by name from this crate's own background
thread fails, and only for app classes.** `android-shell`'s
follow-loop and share threads are Rust-spawned and attached via
`JavaVM::attach_current_thread`, which the platform never handed
an app `ClassLoader` -- so `FindClass`'s default fallback (used
internally by `find_class`/`new_object`/`call_static_method`/
`get_static_field`, anything that resolves a class *by name*
rather than from an object it already holds) only reaches the
bootstrap loader's framework classes. `androidx.core.app.
NotificationManagerCompat`, packaged inside this app's own APK,
is invisible from there: `Error::NoClassDefFound`, logged by
`notify::show`'s `LogErrorAndDefault` as "failed to resolve Java
class ... (class not found or linkage error)" -- which on a real
device is indistinguishable from "the notification silently
never arrives," since the *ongoing* foreground notification
(built on the main thread, before this thread exists) posts
fine regardless, so nothing else looks wrong. Fixed in
`jcall.rs`: `remember_class_loader` caches the app's own
`ClassLoader` (`context.getClass().getClassLoader()`) the first
time any entry point with a `Context` runs, and every
class-by-name lookup goes through `LoaderContext::Loader`
explicitly rather than the thread-dependent default -- correct
on the main thread and this crate's background threads alike.
`jcall.rs`'s module doc has the full account.
3. **`onStartCommand` spawning a thread unconditionally opens a
second connection, and `Notifications.kt` has the same bug.**
Enrolling calls `sync()` twice in one launch (once
unconditionally in `MainActivity.onCreate`, again inside
`handle_enrollment` after saving the token), each of which starts
the service, and Android runs `onStartCommand` once per start
request -- so the follow-loop thread was spawned twice, caught on
`adb logcat` as two `jni::vm::java_vm: Attached thread
ai-app-notifications` lines for one enrollment. Kotlin's
`onStartCommand` has the identical shape (`thread(isDaemon =
true) { follow(settings) }`, no guard), so this is a latent bug
in the reference implementation this port found by testing
rather than something E3 introduced -- worth carrying the same
guard back to `Notifications.kt` separately, not done here.
Fixed in `notify.rs` with a `RUNNING` `AtomicBool`, `swap`ped
true before spawning and reset in `on_destroy`; see its doc
comment for the accepted race this shares with the pre-existing
`STOPPING` gap below.
**Known gap, not fixed, written where it will be found.**
`notify.rs`'s `STOPPING` flag (checked between reconnects) cannot
interrupt a `ureq` read already blocked inside one connection --
unlike `HttpURLConnection.disconnect()`, `client_core::Transport`
exposes no cancellation handle. `/notifications` is idle between
events (a keep-alive), so in practice a stop is a bounded wait
rather than a hang; closing this for real means adding a
cancellation point to the `Transport` trait itself, a decision
affecting every caller, not an `android-shell`-only fix.
*Verification, exact commands.* `cargo fmt -- --check`,
`cargo clippy --all-targets` (zero warnings) and `cargo build`
clean for both `client-core` and `android-shell` on the host
target; `cargo ndk -t x86_64 -P 26 clippy --all-targets` clean for
`android-shell` on the Android target too (the `android_logger`
dependency is Android-only, so this is the only way to compile-check
it). `./run-tests.sh` from the repo root: 127 `server` tests, 88
`client-core` tests (85 + the 3 new to `notifications.rs`), all
passing -- the port added no regression to what already worked.
`./gradlew :shellApp:lintDebug`: `No issues found` (the report at
`app/shellApp/build/reports/lint-results-debug.txt`).
*The two pass-condition proofs*, both on this checkout's own AVD
(`ai-app-2`, GPU host per the default, torn down with `emu down`
when this session finished) against `app/ui-sandbox.sh`:
- **Notification with the app closed.** Enrolled via
`adb shell "am start -a android.intent.action.VIEW -d
'aiappshell://enroll?host=10.0.2.2&port=<sandbox port>&token=<token>'"`
(per the sandbox's own banner, substituting the scheme), granted
`POST_NOTIFICATIONS`, pressed home, then
`./ui-sandbox.sh spawn e3notif2` and
`./ui-sandbox.sh send <sid> "/question Should I proceed with the deploy?"`.
`adb shell dumpsys notification --noredact` shows a
`channel=sessions` record, `android.title=e3notif2`,
`android.text=Waiting for you` (matching `attention_line` and the
session's own title, exactly what `Notifications.kt` would have
shown) -- posted while the app held no visible activity. Tapping
it (`ui-trace record --do "tap 'e3notif2'"`, found in the
expanded shade after `adb shell cmd statusbar
expand-notifications`) launched
`com.example.aiapp.shell/.MainActivity` with
`dat=aiappshell://session/...`, confirmed in `adb logcat`'s
`ActivityTaskManager: START` line -- the `PendingIntent` names
the right session.
- **A share lands in a session.** With the app enrolled and a
session already active,
`adb shell "am start -a android.intent.action.SEND -t text/plain
--es android.intent.extra.TEXT 'Please check the deploy logs for
errors.' -n com.example.aiapp.shell/.MainActivity"` (the classic
`adb shell` quoting trap from `this-machine-android` applies here
too: the whole `am start` invocation has to be one single-quoted
string handed to the *remote* shell, or the extra's spaces get
re-split away). `./ui-sandbox.sh api
'/sessions/<sid>/transcript?limit=20'` shows
`{"type":"userMessage","text":"Please check the deploy logs for
errors."}` followed by the echo driver's reply -- the share
reached the most-recently-active session as a real message, not
a mock.
- [ ] **E4 — the same screen on the desktop** in a winit window, from the
same crate, with only the layout differing.
- [ ] **E5 — the packaging xtask**: `cargo ndk` → `javac`/`d8` → `aapt2`
@@ -980,6 +1226,24 @@ accepted.
installed through Dev Updater. Pass: the APK installs over the
Gradle-built one and the notification service starts.
**Not blocked by E3's layout**, and one thing E5 will need to
account for. `app/shellApp/`'s two Java classes and generated
`PinnedCa.java` are plain `javac` input with no Kotlin of their own
(`android-shell/src/settings.rs`'s doc explains why `kotlinc` still
appears in the graph regardless: `:link`, the shared `wg-app-link`
submodule, is Kotlin and is a real dependency of this module, not
something E3 introduced). `android-shell` itself builds with plain
`cargo ndk build --release -o .../jniLibs/`, exactly the shape "Can
the APK be built without Gradle?" assumed. What E5 will actually
need to solve that E3 did not: producing `:link`'s classes (or an
equivalent Keystore-sealed-token implementation) without inflating
the toolchain E5 is trying to shrink -- either accept `kotlinc` for
that one shared submodule, prebuild it once into a jar/aar E5
consumes as a binary input, or reimplement the Keystore sealing in
Rust after all (rejected in E3 for reuse and phone-compatibility
reasons, but the calculus is different once Kotlin is otherwise
gone).
### The iris track
These build iris up to carry the app. Each is a feature added to iris