iris/ai-app
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

iris: android-app's Gradle shell, and the emulator run for I2

Browse Source 
The Gradle side of RUST.md's I2: MainActivity, IrisView (extending
android-view's RustView with the two native methods it has no hook
for -- window insets, and unregistering this view's entry in
iris::android::insets's side table), and RustView.java/
RustInputConnection.java vendored from android-view (no published AAR
to depend on) with one deliberate diff noted in a comment: mViewPeer
is protected rather than package-private, so a subclass in a different
package can reach it.

Measured on the emulator (x86_64, API 26, SwiftShader Vulkan):
dumpsys input_method shows the served InputConnection is ours, and
Gboard's suggestion strip reads real buffer content back through
text_before_cursor ("hi | Hi | HI" after typing "hi") -- the same bar
E1 set, met. Not met: nothing draws. The clear colour reaches the
screen (confirmed by swapping it to magenta) and the layout engine
reports the correct widget count and pixel regions (log::debug! calls
left in view.rs's render() for exactly this), but no primitive shows
up, on both Vulkan/SwiftShader and GLES/virgl. Root cause not found;
one unconfirmed lead (a GLES-only D2/D2Array warning that could point
at the glyph atlas) is written up in RUST.md's I2 rather than chased
into core/src/render/, which is mid-flight in a separate benchmark
branch this session.

I2 is therefore built and wired but not tickable -- RUST.md has the
full writeup, what was ruled out, and where to pick this up.

Co-Authored-By: Claude Sonnet <noreply@anthropic.com>
This commit is contained in:
irisandClaude Sonnet committed 2026-09-05 05:08:25 -04:00
1 parent f79bd7ca71
commit 6317685d1a
15 files changed
+838 -10
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

RUST.md
+161 -10
View File
@@ -82,8 +82,22 @@ session spending an afternoon on them again.
yet, only the emulator; the Android Vulkan Profile 2025 sourcing in
"iris's binding array does not survive real Android hardware" below is
what stands in for that until I2 gets a device.
- **Next**: **I2** — iris on android-view. **E2** (a transcript in
Masonry) can go in parallel in another session.
- **I2 — iris on android-view: built 2026-09-05, not tickable.** The
android-view backend (`iris/src/android/`), the `iris-android-app`
cdylib and Gradle shell, insets, the back gesture, and the full
`InputConnection` bridge are all in and measured working — Gboard's
suggestion strip reads real buffer content through it, the same bar E1
set. What is not working: **nothing draws**. The screen shows only the
clear colour on both Vulkan/SwiftShader and GLES/virgl, though the
layout engine reports the correct widget count and pixel regions —
see I2's own entry below for what was ruled out and the one open lead
(a GLES-only `D2`/`D2Array` warning from the glyph atlas, unconfirmed
as the cause, and deliberately not chased into `core/src/render/`
while that tree is mid-flight in a separate benchmark branch this
session). **Next**: root-cause the blank render — start with a single
hardcoded rect and no text, to separate "nothing renders" from "the
atlas path specifically is broken" — then revisit I2's tick. **E2** (a
transcript in Masonry) can go in parallel in another session.
- **`client-core` built (2026-09-04)**, item 1 of the recommendation:
`event-model/` (the event types, now shared with `server/`) and
`client-core/` (REST and SSE clients, transcript fold, cache, highlighter,
@@ -991,14 +1005,151 @@ since I2's pass condition is the phone, not just the emulator, and this
is exactly the kind of thing that passes on a desktop GPU and fails
silently on real hardware.
- [ ] **I2 — iris on android-view.** An `android-view` surface as a second
backend beside winit: `wgpu` on the view's surface (GLES here, see
the Vulkan section; Vulkan on the phone), touch as pointer events,
window insets and the keyboard inset as layout inputs, the back
gesture as an event, the IME bridge feeding the editor from I1.
Pass: the `tabs` example and a text field run on the emulator, and
the phone's own keyboard types into the field with autocorrect and
suggestions — the same bar as E1.
- [ ] **I2 — iris on android-view. Built and wired 2026-09-05; the IME half
passes, rendering does not yet.** Not tickable: the pass condition
names the tabs example running, and today it runs invisibly.
**Layout.** `iris/src/android/` mirrors `default/`'s module split
(`view.rs` is `app.rs`+`state.rs` combined, since android-view has one
harness type where winit splits `ApplicationHandler` from per-window
state; `render.rs`, `input.rs`, `attr.rs` correspond directly;
`ime.rs` and `insets.rs` have no winit counterpart). What used to live
only in `default/` and had no winit dependency — `WidgetState`,
`CursorState`/the sense machinery, `Tasks`, `Selector`/`Selectable`'s
focus handling — moved to crate-root modules (`state.rs`, `sense.rs`,
`task.rs`, `attr.rs`) so both backends use one copy; `Tasks`' redraw
nudge is now behind a `RequestRedraw` trait (`Window` for winit, a
`JavaVM`+`GlobalRef` attach-and-call for android-view) rather than a
concrete `winit::window::Window`. `winit`/`arboard` and
`android-view`/`send_wrapper` are now `[target.'cfg(...)']`
dependencies, and `default`/`android` are target-gated modules,
because winit's own Android support needs `android-activity` with a
backend feature selected — exactly what `iris-core` was kept free of.
Confirmed by trying it before the split (`cargo ndk -t x86_64 -P 26
build -p iris` failed inside `android-activity` itself) and after
(clean). `iris/tabs-ui` is the tabs example's widget tree factored out
of `examples/tabs/main.rs` into a crate generic over `Rsc: HasEvents`
+ `Rsc::State: FocusHost`, so the winit example and
`iris/android-app` (the new cdylib, excluded from the `iris` workspace
because android-view needs the NDK sysroot to link — see that
`Cargo.toml`'s comment) call the same `build()`.
android-view pinned to `bec6c62a96cef8239b0fd7fedeef9b184d02e3a1`, the
commit E1 measured against. `RustView.java`/`RustInputConnection.java`
are vendored (no published AAR to depend on) into
`iris/android-app/app/src/.../org/linebender/android/rustview/`, with
one deliberate diff from upstream noted in a comment: `mViewPeer` is
`protected` rather than package-private, so `IrisView` (a different
package) can pass it to the window-insets native call android-view
has no hook for.
**Insets and the back gesture**, both without touching android-view.
The back gesture takes no new plumbing at all: with no
`OnBackPressedCallback` registered, Android still delivers it as an
ordinary `KEYCODE_BACK` `KeyEvent` through the existing key path (the
legacy behaviour every view-based app gets by default), handled in
`view.rs`'s `on_key_down`. Insets have no such stand-in, so
`android/insets.rs` registers one more native method
(`applyWindowInsetsNative`) directly on `IrisView`, writing into an
`Rc<RefCell<Shared>>` a second copy of which lives in
`AndroidUiState` — the peer id android-view hands back from
`register_view_peer` is opaque outside that crate, so this is a
side table keyed on the same id rather than a way to reach the peer
itself. `MainActivity` wires `setOnApplyWindowInsetsListener`,
including the API 30+ `ime()` inset specifically (falls back to 0
below that). Not yet consumed by any widget's layout — `insets()` is
exposed on `AndroidUiState` but nothing reads it yet, since the tabs
example has no chrome that needs to avoid the keyboard.
**The IME bridge is implemented and its pass condition holds.**
`android/ime.rs` implements the full `InputConnection` trait
(`text_before_cursor`/`after_cursor`/`selected_text`,
`cursor_caps_mode`, `delete_surrounding_text[_in_code_points]`,
`set_composing_text`/`_region`, `finish_composing_text`,
`set_selection`, `begin`/`end_batch_edit`, `send_key_event`,
`request_cursor_updates`) directly against `TextEdit` — the same
preedit-replace bookkeeping `default`'s `Ime::Preedit` handling uses
(`compose_len`, in chars), with new byte<->UTF-16 conversion helpers
since parley (since I1) is byte-indexed and Java strings are not.
Two approximations, both commented in place rather than silently
dropped: `set_composing_region` declines (no separate composing range
exists to move) and `set_selection`/`delete_surrounding_text_in_code_points`
collapse to an approximation rather than a real span/code-point
count. `TextEdit` gained `text()`/`selection_range()`/`caret()`
getters and `TextEditCtx::delete_byte_range`/`set_cursor_byte`, all
unconditional (no winit dependency added); `apply_event`/
`TextInputResult`, which do take a `winit::event::KeyEvent`, are now
`#[cfg(not(target_os = "android"))]` instead of being ported, since
android's own `input.rs` calls `TextEdit`'s primitives
(`backspace`/`delete`/`motion`/`insert`) directly from
`ndk::event::Keycode` and never needed a winit `KeyEvent` shape.
**Measured on the emulator, 2026-09-05, x86_64 API 26,
`-feature Vulkan` + SwiftShader per the Vulkan section below.**
`adb shell dumpsys input_method` after tapping the composer field:
`mInputShown=true`, `mServedInputConnection` is
`org.linebender.android.rustview.RustInputConnection` attached to
`IrisView`. `adb shell input text "hi"` followed by a screenshot
shows **Gboard's suggestion strip populated with "hi | Hi | HI"** —
capitalization variants read back out of the real buffer through
`text_before_cursor`, the same kind of evidence E1 recorded (there:
"dolor | Dolores | door"). That is the bar this box asks for, met.
**What is not met: nothing is visible.** The screen shows only the
clear colour (confirmed black, then swapped to magenta and
reconfirmed via screenshot — the presentation pipeline itself works)
with zero widgets drawn on top, on **both** backends tried:
`Backends::PRIMARY` (Vulkan via SwiftShader, `AdapterInfo` logged as
`SwiftShader Device (Subzero)`) and `Backends::GL` (GLES via virgl on
the real host GPU, logged as `Android Emulator OpenGL ES Translator
(virgl (AMD Radeon RX 7900 XT ...`). Ruled out: the layout engine
itself -- `log::debug!` calls left in `android/view.rs`'s `render()`
show `active=39` widgets after `UiRenderState::update` and
`window_region` reporting the root at the *correct* full-surface
pixel rect (`(0, 0)..(1080, 2298)`, later `(1080, 1478)` once the
keyboard's `adjustResize` shrank the window) -- so this is not a
zero-size-widgets bug. One suspicious but unconfirmed lead: the GLES
run logged `wgpu_hal::gles: wgpu-hal heuristics assumed that the view
dimension will be equal to D2 rather than D2Array` right before the
first `render()` call, which is exactly the shape of a bug in the
glyph atlas's `texture_2d_array` (`core/src/render/texture.rs`,
recently reworked per TEXTURES.md) if its array texture is ever
created with a single layer -- wgpu-hal's GL backend is documented to
guess the GL texture target from layer count at *texture* creation
time, which can disagree with a view later requesting `D2Array`
explicitly. Not chased further: this warning is GL-specific and the
*same* blank result occurs under Vulkan too, where view dimension is
always explicit and this class of ambiguity should not exist, so it
may be a red herring rather than the cause. **`iris/core/src/render/`
is mid-flight in a separate benchmark branch as of this session**, so
deliberately not touched here beyond reading it — the next session
should re-check this finding against whatever lands from that branch
before spending more time on it, and reach first for the simplest
possible reproduction (a single hardcoded coloured rect, no text, no
atlas) to separate "nothing renders" from "the atlas path specifically
is broken."
**Not built yet**: anything consuming `insets()`, a real phone
measurement (only the emulator so far — matches every other Android
finding in this file), and AccessKit (I4's job, so `ui-trace`
couldn't be used here; a raw `adb shell input tap`/`input text` stood
in for driving the UI, which is why this section says "the same bar
as E1" rather than citing a `ui-trace` transcript).
**Verification.** Host: `cargo fmt --all -- --check`,
`cargo build --workspace --all-targets`, `cargo clippy --all-targets`,
`cargo test --workspace` (19 tests) all clean in `iris/`; `iris/run-headless.sh
tabs --shot` still renders pixel-identically (27266 bytes, byte-for-byte
unchanged). Android cross-compile: `cargo ndk -t x86_64 -P 26 build`
and `... clippy` clean for both `iris` (with the android module) and
`iris/android-app`. Emulator: `emu up` with
`VK_DRIVER_FILES=.../vk_swiftshader_icd.json` and
`GPU_HOST_FEATURES="-feature Vulkan -no-snapshot-load -no-snapshot-save"`
per the Vulkan section; `cd android-app && cargo ndk -t x86_64 -P 26
-o app/src/main/jniLibs/ build --release && gradle :app:assembleDebug`
(release native lib per E1's segfault finding, debug Gradle variant --
the jniLibs contents are what matters, not the Gradle build type);
`adb install -r app/build/outputs/apk/debug/app-debug.apk`.
- [ ] **I3 — a virtualised, bottom-anchored list.** Variable-height rows,
keyed, composed only while visible, paged in both directions with a
"more" sentinel at each end, a scroll anchor that survives rows