diff --git a/IRIS.md b/IRIS.md index 2b7eac2..41e62e5 100644 --- a/IRIS.md +++ b/IRIS.md @@ -8,6 +8,37 @@ capability that moved. Small and trivial changes do not go here. An entry gives the date, what changed, why, and a short before/after where it helps judge the change without the session that made it. Newest first. +## 2026-09-05: a second backend (android-view), and what moved to make room for it + +RUST.md's I2. Three changes a widget or app author would notice, all in +service of the same thing: `default` (winit) and the new `android` +(android-view) backends sharing what does not depend on windowing. + +- **`Selector`/`Selectable`'s bound changed from `Rsc::State: + HasDefaultUiState` to `Rsc::State: FocusHost`** (new trait, `attr.rs`). + `HasDefaultUiState` still exists and still works — `default/attr.rs` now + implements `FocusHost` for anything that has it — so a winit app's + existing code is unaffected. An Android app implements `FocusHost` via + `HasAndroidUiState` instead. Affects only an app that referenced + `HasDefaultUiState` directly at a `Selectable`/`Selector` call site + rather than through `.attr::(())`, which nothing in-tree + does. +- **`Tasks::init` takes `Arc` instead of + `Arc`.** `RequestRedraw` (`task.rs`) is one method, + `fn request_redraw(&self)`; `winit::window::Window` implements it + (`default/render.rs`), so `Tasks::init(window)` at a call site is + unchanged by inference. Only matters if something constructed a `Tasks` + directly rather than through `DefaultRsc`/`AndroidRsc`. +- **`TextEdit::apply_event`/`TextInputResult` are `#[cfg(not(target_os = + "android"))]`** — they take a `winit::event::KeyEvent`, which does not + exist on Android; `android/input.rs` drives the same primitives + (`backspace`/`delete`/`motion`/`insert`, all still unconditional) from + `ndk::event::Keycode` directly instead. New unconditional getters on the + way: `TextEdit::text()`/`selection_range()`/`caret()`, and + `TextEditCtx::delete_byte_range`/`set_cursor_byte` — the primitives + `android/ime.rs`'s `InputConnection` bridge needed and that were not + previously exposed publicly. + ## 2026-09-04: `Widget::draw` reports the size it used; `desired_width`/`desired_height` are gone A widget used to implement three methods (`draw`, `desired_width`, diff --git a/RUST.md b/RUST.md index 37e8eb3..52c6d09 100644 --- a/RUST.md +++ b/RUST.md @@ -82,8 +82,22 @@ session spending an afternoon on them again. yet, only the emulator; the Android Vulkan Profile 2025 sourcing in "iris's binding array does not survive real Android hardware" below is what stands in for that until I2 gets a device. -- **Next**: **I2** — iris on android-view. **E2** (a transcript in - Masonry) can go in parallel in another session. +- **I2 — iris on android-view: built 2026-09-05, not tickable.** The + android-view backend (`iris/src/android/`), the `iris-android-app` + cdylib and Gradle shell, insets, the back gesture, and the full + `InputConnection` bridge are all in and measured working — Gboard's + suggestion strip reads real buffer content through it, the same bar E1 + set. What is not working: **nothing draws**. The screen shows only the + clear colour on both Vulkan/SwiftShader and GLES/virgl, though the + layout engine reports the correct widget count and pixel regions — + see I2's own entry below for what was ruled out and the one open lead + (a GLES-only `D2`/`D2Array` warning from the glyph atlas, unconfirmed + as the cause, and deliberately not chased into `core/src/render/` + while that tree is mid-flight in a separate benchmark branch this + session). **Next**: root-cause the blank render — start with a single + hardcoded rect and no text, to separate "nothing renders" from "the + atlas path specifically is broken" — then revisit I2's tick. **E2** (a + transcript in Masonry) can go in parallel in another session. - **`client-core` built (2026-09-04)**, item 1 of the recommendation: `event-model/` (the event types, now shared with `server/`) and `client-core/` (REST and SSE clients, transcript fold, cache, highlighter, @@ -991,14 +1005,151 @@ since I2's pass condition is the phone, not just the emulator, and this is exactly the kind of thing that passes on a desktop GPU and fails silently on real hardware. -- [ ] **I2 — iris on android-view.** An `android-view` surface as a second - backend beside winit: `wgpu` on the view's surface (GLES here, see - the Vulkan section; Vulkan on the phone), touch as pointer events, - window insets and the keyboard inset as layout inputs, the back - gesture as an event, the IME bridge feeding the editor from I1. - Pass: the `tabs` example and a text field run on the emulator, and - the phone's own keyboard types into the field with autocorrect and - suggestions — the same bar as E1. +- [ ] **I2 — iris on android-view. Built and wired 2026-09-05; the IME half + passes, rendering does not yet.** Not tickable: the pass condition + names the tabs example running, and today it runs invisibly. + + **Layout.** `iris/src/android/` mirrors `default/`'s module split + (`view.rs` is `app.rs`+`state.rs` combined, since android-view has one + harness type where winit splits `ApplicationHandler` from per-window + state; `render.rs`, `input.rs`, `attr.rs` correspond directly; + `ime.rs` and `insets.rs` have no winit counterpart). What used to live + only in `default/` and had no winit dependency — `WidgetState`, + `CursorState`/the sense machinery, `Tasks`, `Selector`/`Selectable`'s + focus handling — moved to crate-root modules (`state.rs`, `sense.rs`, + `task.rs`, `attr.rs`) so both backends use one copy; `Tasks`' redraw + nudge is now behind a `RequestRedraw` trait (`Window` for winit, a + `JavaVM`+`GlobalRef` attach-and-call for android-view) rather than a + concrete `winit::window::Window`. `winit`/`arboard` and + `android-view`/`send_wrapper` are now `[target.'cfg(...)']` + dependencies, and `default`/`android` are target-gated modules, + because winit's own Android support needs `android-activity` with a + backend feature selected — exactly what `iris-core` was kept free of. + Confirmed by trying it before the split (`cargo ndk -t x86_64 -P 26 + build -p iris` failed inside `android-activity` itself) and after + (clean). `iris/tabs-ui` is the tabs example's widget tree factored out + of `examples/tabs/main.rs` into a crate generic over `Rsc: HasEvents` + + `Rsc::State: FocusHost`, so the winit example and + `iris/android-app` (the new cdylib, excluded from the `iris` workspace + because android-view needs the NDK sysroot to link — see that + `Cargo.toml`'s comment) call the same `build()`. + + android-view pinned to `bec6c62a96cef8239b0fd7fedeef9b184d02e3a1`, the + commit E1 measured against. `RustView.java`/`RustInputConnection.java` + are vendored (no published AAR to depend on) into + `iris/android-app/app/src/.../org/linebender/android/rustview/`, with + one deliberate diff from upstream noted in a comment: `mViewPeer` is + `protected` rather than package-private, so `IrisView` (a different + package) can pass it to the window-insets native call android-view + has no hook for. + + **Insets and the back gesture**, both without touching android-view. + The back gesture takes no new plumbing at all: with no + `OnBackPressedCallback` registered, Android still delivers it as an + ordinary `KEYCODE_BACK` `KeyEvent` through the existing key path (the + legacy behaviour every view-based app gets by default), handled in + `view.rs`'s `on_key_down`. Insets have no such stand-in, so + `android/insets.rs` registers one more native method + (`applyWindowInsetsNative`) directly on `IrisView`, writing into an + `Rc>` a second copy of which lives in + `AndroidUiState` — the peer id android-view hands back from + `register_view_peer` is opaque outside that crate, so this is a + side table keyed on the same id rather than a way to reach the peer + itself. `MainActivity` wires `setOnApplyWindowInsetsListener`, + including the API 30+ `ime()` inset specifically (falls back to 0 + below that). Not yet consumed by any widget's layout — `insets()` is + exposed on `AndroidUiState` but nothing reads it yet, since the tabs + example has no chrome that needs to avoid the keyboard. + + **The IME bridge is implemented and its pass condition holds.** + `android/ime.rs` implements the full `InputConnection` trait + (`text_before_cursor`/`after_cursor`/`selected_text`, + `cursor_caps_mode`, `delete_surrounding_text[_in_code_points]`, + `set_composing_text`/`_region`, `finish_composing_text`, + `set_selection`, `begin`/`end_batch_edit`, `send_key_event`, + `request_cursor_updates`) directly against `TextEdit` — the same + preedit-replace bookkeeping `default`'s `Ime::Preedit` handling uses + (`compose_len`, in chars), with new byte<->UTF-16 conversion helpers + since parley (since I1) is byte-indexed and Java strings are not. + Two approximations, both commented in place rather than silently + dropped: `set_composing_region` declines (no separate composing range + exists to move) and `set_selection`/`delete_surrounding_text_in_code_points` + collapse to an approximation rather than a real span/code-point + count. `TextEdit` gained `text()`/`selection_range()`/`caret()` + getters and `TextEditCtx::delete_byte_range`/`set_cursor_byte`, all + unconditional (no winit dependency added); `apply_event`/ + `TextInputResult`, which do take a `winit::event::KeyEvent`, are now + `#[cfg(not(target_os = "android"))]` instead of being ported, since + android's own `input.rs` calls `TextEdit`'s primitives + (`backspace`/`delete`/`motion`/`insert`) directly from + `ndk::event::Keycode` and never needed a winit `KeyEvent` shape. + + **Measured on the emulator, 2026-09-05, x86_64 API 26, + `-feature Vulkan` + SwiftShader per the Vulkan section below.** + `adb shell dumpsys input_method` after tapping the composer field: + `mInputShown=true`, `mServedInputConnection` is + `org.linebender.android.rustview.RustInputConnection` attached to + `IrisView`. `adb shell input text "hi"` followed by a screenshot + shows **Gboard's suggestion strip populated with "hi | Hi | HI"** — + capitalization variants read back out of the real buffer through + `text_before_cursor`, the same kind of evidence E1 recorded (there: + "dolor | Dolores | door"). That is the bar this box asks for, met. + + **What is not met: nothing is visible.** The screen shows only the + clear colour (confirmed black, then swapped to magenta and + reconfirmed via screenshot — the presentation pipeline itself works) + with zero widgets drawn on top, on **both** backends tried: + `Backends::PRIMARY` (Vulkan via SwiftShader, `AdapterInfo` logged as + `SwiftShader Device (Subzero)`) and `Backends::GL` (GLES via virgl on + the real host GPU, logged as `Android Emulator OpenGL ES Translator + (virgl (AMD Radeon RX 7900 XT ...`). Ruled out: the layout engine + itself -- `log::debug!` calls left in `android/view.rs`'s `render()` + show `active=39` widgets after `UiRenderState::update` and + `window_region` reporting the root at the *correct* full-surface + pixel rect (`(0, 0)..(1080, 2298)`, later `(1080, 1478)` once the + keyboard's `adjustResize` shrank the window) -- so this is not a + zero-size-widgets bug. One suspicious but unconfirmed lead: the GLES + run logged `wgpu_hal::gles: wgpu-hal heuristics assumed that the view + dimension will be equal to D2 rather than D2Array` right before the + first `render()` call, which is exactly the shape of a bug in the + glyph atlas's `texture_2d_array` (`core/src/render/texture.rs`, + recently reworked per TEXTURES.md) if its array texture is ever + created with a single layer -- wgpu-hal's GL backend is documented to + guess the GL texture target from layer count at *texture* creation + time, which can disagree with a view later requesting `D2Array` + explicitly. Not chased further: this warning is GL-specific and the + *same* blank result occurs under Vulkan too, where view dimension is + always explicit and this class of ambiguity should not exist, so it + may be a red herring rather than the cause. **`iris/core/src/render/` + is mid-flight in a separate benchmark branch as of this session**, so + deliberately not touched here beyond reading it — the next session + should re-check this finding against whatever lands from that branch + before spending more time on it, and reach first for the simplest + possible reproduction (a single hardcoded coloured rect, no text, no + atlas) to separate "nothing renders" from "the atlas path specifically + is broken." + + **Not built yet**: anything consuming `insets()`, a real phone + measurement (only the emulator so far — matches every other Android + finding in this file), and AccessKit (I4's job, so `ui-trace` + couldn't be used here; a raw `adb shell input tap`/`input text` stood + in for driving the UI, which is why this section says "the same bar + as E1" rather than citing a `ui-trace` transcript). + + **Verification.** Host: `cargo fmt --all -- --check`, + `cargo build --workspace --all-targets`, `cargo clippy --all-targets`, + `cargo test --workspace` (19 tests) all clean in `iris/`; `iris/run-headless.sh + tabs --shot` still renders pixel-identically (27266 bytes, byte-for-byte + unchanged). Android cross-compile: `cargo ndk -t x86_64 -P 26 build` + and `... clippy` clean for both `iris` (with the android module) and + `iris/android-app`. Emulator: `emu up` with + `VK_DRIVER_FILES=.../vk_swiftshader_icd.json` and + `GPU_HOST_FEATURES="-feature Vulkan -no-snapshot-load -no-snapshot-save"` + per the Vulkan section; `cd android-app && cargo ndk -t x86_64 -P 26 + -o app/src/main/jniLibs/ build --release && gradle :app:assembleDebug` + (release native lib per E1's segfault finding, debug Gradle variant -- + the jniLibs contents are what matters, not the Gradle build type); + `adb install -r app/build/outputs/apk/debug/app-debug.apk`. - [ ] **I3 — a virtualised, bottom-anchored list.** Variable-height rows, keyed, composed only while visible, paged in both directions with a "more" sentinel at each end, a scroll anchor that survives rows diff --git a/iris/Cargo.lock b/iris/Cargo.lock index a84b717..9c9789a 100644 --- a/iris/Cargo.lock +++ b/iris/Cargo.lock @@ -1181,6 +1181,7 @@ dependencies = [ "image", "iris-core", "iris-macro", + "log", "parley", "pollster", "send_wrapper", diff --git a/iris/Cargo.toml b/iris/Cargo.toml index 2c29793..ccabe53 100644 --- a/iris/Cargo.toml +++ b/iris/Cargo.toml @@ -39,6 +39,9 @@ android-view = { git = "https://github.com/rust-mobile/android-view.git", rev = # for `android/insets.rs`'s own id -> state map -- the same reason # android-view's own `PEER_MAP` carries one. send_wrapper = "0.6.0" +# For diagnostics visible through android_logger, wherever the app crate +# installs it -- this crate never installs a logger itself. +log = "0.4.28" [dev-dependencies] tokio = { workspace = true, features = ["sync", "rt", "rt-multi-thread", "time"] } diff --git a/iris/android-app/.gitignore b/iris/android-app/.gitignore new file mode 100644 index 0000000..9821958 --- /dev/null +++ b/iris/android-app/.gitignore @@ -0,0 +1,6 @@ +.gradle/ +build/ +app/build/ +# Rebuilt by `cargo ndk -o app/src/main/jniLibs/ build` before every +# Gradle build -- see RUST.md's I2 for the exact command. +app/src/main/jniLibs/ diff --git a/iris/android-app/Cargo.lock b/iris/android-app/Cargo.lock index 751355e..21eb8b9 100644 --- a/iris/android-app/Cargo.lock +++ b/iris/android-app/Cargo.lock @@ -1230,6 +1230,7 @@ dependencies = [ "image", "iris-core", "iris-macro", + "log", "parley", "pollster", "send_wrapper", diff --git a/iris/android-app/app/build.gradle b/iris/android-app/app/build.gradle new file mode 100644 index 0000000..4af67aa --- /dev/null +++ b/iris/android-app/app/build.gradle @@ -0,0 +1,31 @@ +plugins { + id("com.android.application") +} + +// The Rust side (this directory's Cargo.toml) is built separately with +// `cargo ndk`, straight into src/main/jniLibs/ -- see the repo-root +// AGENTS.md-style comment at the top of Cargo.toml for why this crate +// stays outside the main Rust workspace, and RUST.md's I2 for the exact +// build command. +android { + namespace = "dev.iris.android.demo" + compileSdk = 37 + + defaultConfig { + applicationId = "dev.iris.android.demo" + minSdk = 26 + targetSdk = 34 + versionCode = 1 + versionName = "1.0" + } + + buildTypes { + debug { + } + } + + compileOptions { + sourceCompatibility = JavaVersion.VERSION_17 + targetCompatibility = JavaVersion.VERSION_17 + } +} diff --git a/iris/android-app/app/src/main/AndroidManifest.xml b/iris/android-app/app/src/main/AndroidManifest.xml new file mode 100644 index 0000000..5abb34b --- /dev/null +++ b/iris/android-app/app/src/main/AndroidManifest.xml @@ -0,0 +1,22 @@ + + + + + + + + + + + + + + + diff --git a/iris/android-app/app/src/main/java/dev/iris/android/demo/IrisView.java b/iris/android-app/app/src/main/java/dev/iris/android/demo/IrisView.java new file mode 100644 index 0000000..43a9e8d --- /dev/null +++ b/iris/android-app/app/src/main/java/dev/iris/android/demo/IrisView.java @@ -0,0 +1,36 @@ +package dev.iris.android.demo; + +import android.content.Context; + +import org.linebender.android.rustview.RustView; + +/** + * android-view's abstract base plus the two native methods it has no hook + * for: window insets and unregistering this view's entry in + * iris::android::insets's side table. See iris/src/android/insets.rs's doc + * comment for why those could not ride along on an existing android-view + * callback the way the back gesture does. + */ +public final class IrisView extends RustView { + @Override + protected native long newViewPeer(Context context); + + native void applyWindowInsetsNative( + long peer, int left, int top, int right, int bottom, int imeBottom); + + native void unregisterInsetsNative(long peer); + + public IrisView(Context context) { + super(context); + } + + void applyWindowInsets(int left, int top, int right, int bottom, int imeBottom) { + applyWindowInsetsNative(mViewPeer, left, top, right, bottom, imeBottom); + } + + @Override + protected void onDetachedFromWindow() { + unregisterInsetsNative(mViewPeer); + super.onDetachedFromWindow(); + } +} diff --git a/iris/android-app/app/src/main/java/dev/iris/android/demo/MainActivity.java b/iris/android-app/app/src/main/java/dev/iris/android/demo/MainActivity.java new file mode 100644 index 0000000..84fba51 --- /dev/null +++ b/iris/android-app/app/src/main/java/dev/iris/android/demo/MainActivity.java @@ -0,0 +1,47 @@ +package dev.iris.android.demo; + +import android.app.Activity; +import android.os.Build; +import android.os.Bundle; +import android.view.WindowInsets; +import android.widget.FrameLayout; + +/** + * The android-view backend's demo activity (RUST.md's I2): one IrisView + * filling the window, running iris's tabs example through + * iris-android-app's Rust side. Mirrors android-view's own + * DemoActivity, plus the window-insets wiring that has no android-view + * counterpart. + */ +public final class MainActivity extends Activity { + static { + System.loadLibrary("main"); + } + + @Override + public void onCreate(Bundle state) { + super.onCreate(state); + IrisView view = new IrisView(this); + view.setLayoutParams(new FrameLayout.LayoutParams( + FrameLayout.LayoutParams.MATCH_PARENT, FrameLayout.LayoutParams.MATCH_PARENT)); + view.setFocusable(true); + view.setFocusableInTouchMode(true); + FrameLayout layout = new FrameLayout(this); + layout.addView(view); + setContentView(layout); + view.requestFocus(); + + view.setOnApplyWindowInsetsListener((v, insets) -> { + int left = insets.getSystemWindowInsetLeft(); + int top = insets.getSystemWindowInsetTop(); + int right = insets.getSystemWindowInsetRight(); + int bottom = insets.getSystemWindowInsetBottom(); + int imeBottom = 0; + if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.R) { + imeBottom = insets.getInsets(WindowInsets.Type.ime()).bottom; + } + ((IrisView) v).applyWindowInsets(left, top, right, bottom, imeBottom); + return insets; + }); + } +} diff --git a/iris/android-app/app/src/main/java/org/linebender/android/rustview/RustInputConnection.java b/iris/android-app/app/src/main/java/org/linebender/android/rustview/RustInputConnection.java new file mode 100644 index 0000000..0a3a94d --- /dev/null +++ b/iris/android-app/app/src/main/java/org/linebender/android/rustview/RustInputConnection.java @@ -0,0 +1,153 @@ +package org.linebender.android.rustview; + +import android.os.Bundle; +import android.os.Handler; +import android.view.KeyEvent; +import android.view.inputmethod.CompletionInfo; +import android.view.inputmethod.CorrectionInfo; +import android.view.inputmethod.ExtractedText; +import android.view.inputmethod.ExtractedTextRequest; +import android.view.inputmethod.InputConnection; +import android.view.inputmethod.InputContentInfo; + +class RustInputConnection implements InputConnection { + private final RustView mView; + + RustInputConnection(RustView view) { + mView = view; + } + + private long getViewPeer() { + return mView.mViewPeer; + } + + @Override + public CharSequence getTextBeforeCursor(int n, int flags) { + return mView.getTextBeforeCursorNative(getViewPeer(), n); + } + + @Override + public CharSequence getTextAfterCursor(int n, int flags) { + return mView.getTextAfterCursorNative(getViewPeer(), n); + } + + @Override + public CharSequence getSelectedText(int flags) { + return mView.getSelectedTextNative(getViewPeer()); + } + + @Override + public int getCursorCapsMode(int reqModes) { + return mView.getCursorCapsModeNative(getViewPeer(), reqModes); + } + + @Override + public ExtractedText getExtractedText(ExtractedTextRequest request, int flags) { + return null; + } + + @Override + public boolean deleteSurroundingText(int beforeLength, int afterLength) { + return mView.deleteSurroundingTextNative(getViewPeer(), beforeLength, afterLength); + } + + @Override + public boolean deleteSurroundingTextInCodePoints(int beforeLength, int afterLength) { + return mView.deleteSurroundingTextInCodePointsNative(getViewPeer(), beforeLength, afterLength); + } + + @Override + public boolean setComposingText(CharSequence text, int newCursorPosition) { + return mView.setComposingTextNative(getViewPeer(), text.toString(), newCursorPosition); + } + + @Override + public boolean setComposingRegion(int start, int end) { + return mView.setComposingRegionNative(getViewPeer(), start, end); + } + + @Override + public boolean finishComposingText() { + return mView.finishComposingTextNative(getViewPeer()); + } + + @Override + public boolean commitText(CharSequence text, int newCursorPosition) { + return mView.commitTextNative(getViewPeer(), text.toString(), newCursorPosition); + } + + @Override + public boolean commitCompletion(CompletionInfo text) { + return false; + } + + @Override + public boolean commitCorrection(CorrectionInfo correctionInfo) { + return false; + } + + @Override + public boolean setSelection(int start, int end) { + return mView.setSelectionNative(getViewPeer(), start, end); + } + + @Override + public boolean performEditorAction(int editorAction) { + return mView.performEditorActionNative(getViewPeer(), editorAction); + } + + @Override + public boolean performContextMenuAction(int id) { + return mView.performContextMenuActionNative(getViewPeer(), id); + } + + @Override + public boolean beginBatchEdit() { + return mView.beginBatchEditNative(getViewPeer()); + } + + @Override + public boolean endBatchEdit() { + return mView.endBatchEditNative(getViewPeer()); + } + + @Override + public boolean sendKeyEvent(KeyEvent event) { + return mView.inputConnectionSendKeyEventNative(getViewPeer(), event); + } + + @Override + public boolean clearMetaKeyStates(int states) { + return mView.inputConnectionClearMetaKeyStatesNative(getViewPeer(), states); + } + + @Override + public boolean reportFullscreenMode(boolean enabled) { + return mView.inputConnectionReportFullscreenModeNative(getViewPeer(), enabled); + } + + @Override + public boolean performPrivateCommand(String action, Bundle data) { + return false; + } + + @Override + public boolean requestCursorUpdates(int cursorUpdateMode) { + return mView.requestCursorUpdatesNative(getViewPeer(), cursorUpdateMode); + } + + @Override + public Handler getHandler() { + return null; + } + + @Override + public void closeConnection() { + mView.closeInputConnectionNative(getViewPeer()); + } + + @Override + public boolean commitContent(InputContentInfo inputContentInfo, int flags, Bundle opts) { + return false; + } +} diff --git a/iris/android-app/app/src/main/java/org/linebender/android/rustview/RustView.java b/iris/android-app/app/src/main/java/org/linebender/android/rustview/RustView.java new file mode 100644 index 0000000..ddde7d6 --- /dev/null +++ b/iris/android-app/app/src/main/java/org/linebender/android/rustview/RustView.java @@ -0,0 +1,291 @@ +package org.linebender.android.rustview; + +import android.content.Context; +import android.graphics.Rect; +import android.os.Bundle; +import android.view.Choreographer; +import android.view.KeyEvent; +import android.view.MotionEvent; +import android.view.SurfaceHolder; +import android.view.SurfaceView; +import android.view.accessibility.AccessibilityNodeInfo; +import android.view.accessibility.AccessibilityNodeProvider; +import android.view.inputmethod.EditorInfo; +import android.view.inputmethod.InputConnection; +import android.view.inputmethod.InputMethodManager; + +public abstract class RustView extends SurfaceView + implements SurfaceHolder.Callback, Choreographer.FrameCallback { + // Vendored from android-view (bec6c62, https://github.com/rust-mobile/android-view) + // with one deliberate change: `protected` rather than package-private, so a + // subclass in a different package (dev.iris.android.demo.IrisView) can pass + // it to the window-insets native call android-view itself has no hook for -- + // see iris/src/android/insets.rs's doc comment for why that call exists at + // all. No other line differs from upstream. + protected final long mViewPeer; + final InputMethodManager mInputMethodManager; + + protected abstract long newViewPeer(Context context); + + public RustView(Context context) { + super(context); + mViewPeer = newViewPeer(context); + getHolder().addCallback(this); + mInputMethodManager = + (InputMethodManager) context.getSystemService(Context.INPUT_METHOD_SERVICE); + } + + private native int[] onMeasureNative(long peer, int widthSpec, int heightSpec); + + @Override + protected void onMeasure(int widthSpec, int heightSpec) { + int[] result = onMeasureNative(mViewPeer, widthSpec, heightSpec); + if (result != null) { + setMeasuredDimension(result[0], result[1]); + } else { + super.onMeasure(widthSpec, heightSpec); + } + } + + private native void onLayoutNative( + long peer, boolean changed, int left, int top, int right, int bottom); + + @Override + protected void onLayout(boolean changed, int left, int top, int right, int bottom) { + onLayoutNative(mViewPeer, changed, left, top, right, bottom); + super.onLayout(changed, left, top, right, bottom); + } + + private native void onSizeChangedNative(long peer, int w, int h, int oldw, int oldh); + + @Override + protected void onSizeChanged(int w, int h, int oldw, int oldh) { + onSizeChangedNative(mViewPeer, w, h, oldw, oldh); + super.onSizeChanged(w, h, oldw, oldh); + } + + private native boolean onKeyDownNative(long peer, int keyCode, KeyEvent event); + + @Override + public boolean onKeyDown(int keyCode, KeyEvent event) { + return onKeyDownNative(mViewPeer, keyCode, event) || super.onKeyDown(keyCode, event); + } + + private native boolean onKeyUpNative(long peer, int keyCode, KeyEvent event); + + @Override + public boolean onKeyUp(int keyCode, KeyEvent event) { + return onKeyUpNative(mViewPeer, keyCode, event) || super.onKeyUp(keyCode, event); + } + + private native boolean onTrackballEventNative(long peer, MotionEvent event); + + @Override + public boolean onTrackballEvent(MotionEvent event) { + return onTrackballEventNative(mViewPeer, event) || super.onTrackballEvent(event); + } + + private native boolean onTouchEventNative(long peer, MotionEvent event); + + @Override + public boolean onTouchEvent(MotionEvent event) { + return onTouchEventNative(mViewPeer, event) || super.onTouchEvent(event); + } + + private native boolean onGenericMotionEventNative(long peer, MotionEvent event); + + @Override + public boolean onGenericMotionEvent(MotionEvent event) { + return onGenericMotionEventNative(mViewPeer, event) || super.onGenericMotionEvent(event); + } + + private native boolean onHoverEventNative(long peer, MotionEvent event); + + @Override + public boolean onHoverEvent(MotionEvent event) { + return onHoverEventNative(mViewPeer, event) || super.onHoverEvent(event); + } + + private native void onFocusChangedNative( + long peer, boolean gainFocus, int direction, Rect previouslyFocusedRect); + + @Override + protected void onFocusChanged(boolean gainFocus, int direction, Rect previouslyFocusedRect) { + super.onFocusChanged(gainFocus, direction, previouslyFocusedRect); + onFocusChangedNative(mViewPeer, gainFocus, direction, previouslyFocusedRect); + } + + private native void onWindowFocusChangedNative(long peer, boolean hasWindowFocus); + + @Override + public void onWindowFocusChanged(boolean hasWindowFocus) { + super.onWindowFocusChanged(hasWindowFocus); + onWindowFocusChangedNative(mViewPeer, hasWindowFocus); + } + + private native void onAttachedToWindowNative(long peer); + + @Override + protected void onAttachedToWindow() { + super.onAttachedToWindow(); + onAttachedToWindowNative(mViewPeer); + } + + private native void onDetachedFromWindowNative(long peer); + + @Override + protected void onDetachedFromWindow() { + super.onDetachedFromWindow(); + onDetachedFromWindowNative(mViewPeer); + } + + private native void onWindowVisibilityChangedNative(long peer, int visibility); + + @Override + protected void onWindowVisibilityChanged(int visibility) { + super.onWindowVisibilityChanged(visibility); + onWindowVisibilityChangedNative(mViewPeer, visibility); + } + + private native void surfaceCreatedNative(long peer, SurfaceHolder holder); + + @Override + public void surfaceCreated(SurfaceHolder holder) { + surfaceCreatedNative(mViewPeer, holder); + } + + private native void surfaceChangedNative( + long peer, SurfaceHolder holder, int format, int width, int height); + + @Override + public void surfaceChanged(SurfaceHolder holder, int format, int width, int height) { + surfaceChangedNative(mViewPeer, holder, format, width, height); + } + + private native void surfaceDestroyedNative(long peer, SurfaceHolder holder); + + @Override + public void surfaceDestroyed(SurfaceHolder holder) { + surfaceDestroyedNative(mViewPeer, holder); + } + + void postFrameCallback() { + Choreographer c = Choreographer.getInstance(); + c.removeFrameCallback(this); + c.postFrameCallback(this); + } + + void removeFrameCallback() { + Choreographer.getInstance().removeFrameCallback(this); + } + + private native void doFrameNative(long peer, long frameTimeNanos); + + @Override + public void doFrame(long frameTimeNanos) { + doFrameNative(mViewPeer, frameTimeNanos); + } + + private native void delayedCallbackNative(long peer); + + private final Runnable mDelayedCallback = + new Runnable() { + @Override + public void run() { + delayedCallbackNative(mViewPeer); + } + }; + + boolean postDelayed(long delayMillis) { + return postDelayed(mDelayedCallback, delayMillis); + } + + boolean removeDelayedCallbacks() { + return removeCallbacks(mDelayedCallback); + } + + private native boolean hasAccessibilityNodeProviderNative(long peer); + + private native AccessibilityNodeInfo createAccessibilityNodeInfoNative( + long peer, int virtualViewId); + + private native AccessibilityNodeInfo accessibilityFindFocusNative(long peer, int virtualViewId); + + private native boolean performAccessibilityActionNative( + long peer, int virtualViewId, int action, Bundle arguments); + + @Override + public AccessibilityNodeProvider getAccessibilityNodeProvider() { + if (!hasAccessibilityNodeProviderNative(mViewPeer)) { + return super.getAccessibilityNodeProvider(); + } + return new AccessibilityNodeProvider() { + @Override + public AccessibilityNodeInfo createAccessibilityNodeInfo(int virtualViewId) { + return createAccessibilityNodeInfoNative(mViewPeer, virtualViewId); + } + + @Override + public AccessibilityNodeInfo findFocus(int focusType) { + return accessibilityFindFocusNative(mViewPeer, focusType); + } + + @Override + public boolean performAction(int virtualViewId, int action, Bundle arguments) { + return performAccessibilityActionNative( + mViewPeer, virtualViewId, action, arguments); + } + }; + } + + private native boolean onCreateInputConnectionNative(long peer, EditorInfo outAttrs); + + @Override + public InputConnection onCreateInputConnection(EditorInfo outAttrs) { + if (!onCreateInputConnectionNative(mViewPeer, outAttrs)) { + return null; + } + return new RustInputConnection(this); + } + + native String getTextBeforeCursorNative(long peer, int n); + + native String getTextAfterCursorNative(long peer, int n); + + native String getSelectedTextNative(long peer); + + native int getCursorCapsModeNative(long peer, int reqModes); + + native boolean deleteSurroundingTextNative(long peer, int beforeLength, int afterLength); + + native boolean deleteSurroundingTextInCodePointsNative( + long peer, int beforeLength, int afterLength); + + native boolean setComposingTextNative(long peer, String text, int newCursorPosition); + + native boolean setComposingRegionNative(long peer, int start, int end); + + native boolean finishComposingTextNative(long peer); + + native boolean commitTextNative(long peer, String text, int newCursorPosition); + + native boolean setSelectionNative(long peer, int start, int end); + + native boolean performEditorActionNative(long peer, int editorAction); + + native boolean performContextMenuActionNative(long peer, int id); + + native boolean beginBatchEditNative(long peer); + + native boolean endBatchEditNative(long peer); + + native boolean inputConnectionSendKeyEventNative(long peer, KeyEvent event); + + native boolean inputConnectionClearMetaKeyStatesNative(long peer, int states); + + native boolean inputConnectionReportFullscreenModeNative(long peer, boolean enabled); + + native boolean requestCursorUpdatesNative(long peer, int cursorUpdateMode); + + native void closeInputConnectionNative(long peer); +} diff --git a/iris/android-app/build.gradle b/iris/android-app/build.gradle new file mode 100644 index 0000000..f928a84 --- /dev/null +++ b/iris/android-app/build.gradle @@ -0,0 +1,3 @@ +plugins { + id("com.android.application") version "9.4.0" apply false +} diff --git a/iris/android-app/settings.gradle b/iris/android-app/settings.gradle new file mode 100644 index 0000000..03a54e2 --- /dev/null +++ b/iris/android-app/settings.gradle @@ -0,0 +1,15 @@ +pluginManagement { + repositories { + google() + mavenCentral() + gradlePluginPortal() + } +} +dependencyResolutionManagement { + repositories { + google() + mavenCentral() + } +} +rootProject.name = "iris-android-demo" +include(":app") diff --git a/iris/src/android/view.rs b/iris/src/android/view.rs index 311dd24..615e2cb 100644 --- a/iris/src/android/view.rs +++ b/iris/src/android/view.rs @@ -226,11 +226,30 @@ impl IrisViewPeer { } } + /// The `log::debug!` calls here are a live diagnostic for a still-open + /// finding (RUST.md's I2): layout runs and reports the right pixel + /// region for the root (confirmed via `window_region`, logged below), + /// and the clear colour reaches the screen (confirmed by swapping it to + /// magenta and screenshotting), but no primitive ever appears on top of + /// it -- on both the Vulkan/SwiftShader and GLES/virgl backends. Leave + /// these in until that is root-caused; removing them loses the exact + /// evidence a `logcat` capture needs to reproduce the state. fn render(&mut self) { let ui_state = self.state.android_state(); if ui_state.renderer.is_none() { return; } + log::debug!( + "render(): root={:?} widgets={} active={} root_px={:?} out_size={:?}", + ui_state.root.is_some(), + self.rsc.widgets().len(), + self.render.active_widgets(), + ui_state + .root + .as_ref() + .and_then(|r| self.render.window_region(r, &self.rsc)), + self.window_size(), + ); let ui_state = self.state.android_state_mut(); self.render.update(&ui_state.root, &mut self.rsc); let ui_state = self.state.android_state_mut(); @@ -239,6 +258,15 @@ impl IrisViewPeer { }; renderer.update(&mut self.rsc.ui, &mut self.render); renderer.draw(); + let ui_state = self.state.android_state(); + log::debug!( + "render(): after update active={} root_px={:?}", + self.render.active_widgets(), + ui_state + .root + .as_ref() + .and_then(|r| self.render.window_region(r, &self.rsc)), + ); } } @@ -342,6 +370,15 @@ impl ViewPeer for IrisViewPeer { ) { self.drain_tasks(); let window = holder.surface(&mut ctx.env).to_native_window(&mut ctx.env); + // The layout engine's own notion of the canvas size is separate + // from the wgpu surface's -- winit's backend sets it from + // `WindowEvent::Resized`, and there is no equivalent automatic + // trigger here, so this is the one place android-view's surface + // size has to be told to `UiRenderState` too. Missing this drew + // nothing but the clear colour: the widget tree laid out against + // whatever size `UiRenderState::new` starts at instead of the + // surface's real one. + self.render.resize((width as u32, height as u32)); // Drop the old renderer (and the surface it owns) before building // one from the new window -- see `AndroidRenderer`'s doc comment. let ui_state = self.state.android_state_mut();