diff --git a/docs/IRIS.md b/docs/IRIS.md index 230d9c0..6490673 100644 --- a/docs/IRIS.md +++ b/docs/IRIS.md @@ -626,3 +626,32 @@ box has the full investigation and the phone verification still to do. built and checked on this checkout's emulator only. RUST.md's P0 box says what she should check for: crisp text at two densities, the keyboard no longer wiping, and the header's background. + +## 2026-09-06: composing text, focus-on-tap, and atlas invalidation on a new renderer + +Three small but public API changes, from the same phone-report pass as the +entry above (RUST.md's P0 box has the full account, including a real bug +still not root-caused). + +- **`FocusHost` gained `is_focused(&self, id) -> bool`** (both platform + impls). `attr.rs`'s `Selector`/`Selectable` used to grant focus (and so + request the IME) on the very first frame of *any* press, before it was + known whether the gesture was a tap or a drag — a swipe over a text + field wrongly summoned the keyboard. They now wait for a completed tap + (press and release with no frame crossing `sense::DRAG_SLOP`) unless the + field is already focused, in which case dragging inside it to select + text is unchanged. `TextEdit` gained one new `pub(crate)` field + (`press_origin`) to track this; no public surface change there. +- **`android::ime`'s `InputConnection` now calls `InputMethodManager:: + updateSelection` after every edit** (`IrisViewPeer::update_ime_selection`, + called from `after_input`). Gboard was holding keystrokes back because + nothing ever told it where the app's own selection/composing region had + moved to — this is what android-view's own demo does in its `render()` + and this bridge never did. +- **`GlyphAtlas::clear()` and `Textures::reset()`** (`iris_core`). Called + together, once, from `android::view`'s `surface_changed` exactly when a + *genuinely new* `AndroidRenderer` is built (backgrounding and returning, + not a keyboard-triggered resize, which already reuses the renderer) — + both CPU-side caches otherwise kept pointing at the old, now-destroyed + device's textures, which is why text used to vanish again after leaving + and returning to the app. diff --git a/docs/IRIS_TODO.md b/docs/IRIS_TODO.md index d6ec842..082dcba 100644 --- a/docs/IRIS_TODO.md +++ b/docs/IRIS_TODO.md @@ -149,6 +149,39 @@ agent takes them without colliding with that pass's `bench_client.rs`/ confirming this was the whole story on real touch input rather than only the arbiter's own unit tests -- worth a follow-up pass before calling it fully closed. +- [x] **Composing text held back until a space, caret not moving, fixed + 2026-09-06.** `InputMethodManager.updateSelection` was never called -- + see IRIS.md's 2026-09-06 entry and RUST.md's P0 box, item 1, for the + full account and the emulator evidence. +- [x] **Swipe over the composer summons the keyboard, fixed 2026-09-06.** + `Selector`/`Selectable` now wait for a completed tap -- see IRIS.md's + 2026-09-06 entry and RUST.md's P0 box, item 5. Verified via `dumpsys + input_method`'s `mInputShown` on the emulator, not yet on the phone. +- [x] **Text disappears again after leaving and returning to the app, + fixed 2026-09-06.** `GlyphAtlas::clear`/`Textures::reset` on a + genuinely new renderer -- see IRIS.md's 2026-09-06 entry and RUST.md's + P0 box, item 4. Verified on the emulator (home, reopen, screenshot); + not yet on the phone. +- [ ] **Composed/typed text never becomes visible at all -- found + 2026-09-06, not fixed.** The composer bar stays empty even once the + buffer genuinely holds the typed text (confirmed indirectly: Gboard's + own suggestion strip reacts correctly to each keystroke). A new unit + test proves the widget tree's own layout math resolves the field's + region correctly across a keyboard resize, so the bug is downstream of + that -- most likely `UiRenderState::redraw`'s single-widget redraw path, + or specific to this emulator's forced `force-gles` backend (untested on + Vulkan or the real phone). RUST.md's P0 box, item 2, has the full + writeup, what was ruled out, and where to look next. **Also unverified + because of this**: item 3's composer rebuild (one `Stack`-based widget, + a capped/scrollable height, bottom padding tied to the IME/nav-bar + inset) -- structurally in place and unit-tested, but its own visual + correctness cannot be screenshotted until text actually renders. +- [ ] **The composer has no touch-drag scroll for overflowing text.** The + 2026-09-06 rebuild caps the field at ~6 lines and wraps it in + `.scrollable()` for a wheel/trackpad scroll, but a real finger drag over + text that has overflowed the cap does not scroll it -- `Scroll`'s touch + handling is a follow-up, the same shape `List`'s own touch-drag pan + needed before I3/I5. ## Build diff --git a/docs/RUST.md b/docs/RUST.md index 52f7e85..6d44dff 100644 --- a/docs/RUST.md +++ b/docs/RUST.md @@ -4581,6 +4581,168 @@ device. the Compose half above is already done and is the reference shape for whoever picks this up. No redelivery this pass. + **Composing text, the tap-vs-swipe focus rule, and app-switch text + loss, 2026-09-06.** Iris's report on this same dc01f88 build: typing + doesn't enter text or move the caret until a space is hit; typed + text doesn't visibly appear and there is empty black space below the + composer bar; text disappears again after leaving and returning to + the app; and (a follow-up message the same day) swiping over the + composer bar wrongly summons the keyboard. + + 1. **The caret/composing bug's cause**: `android/ime.rs`'s + `InputConnection` never called `InputMethodManager.updateSelection` + after an edit -- confirmed by reading android-view's own demo + (`~/src/android-view/demo/src/lib.rs`'s `render()`), which calls it + every time its editor's generation changes. Without it, Gboard has + no confirmation the app is keeping up and holds keystrokes back + rather than trusting a screen it believes is stale -- exactly + "doesn't enter it until I hit space." **Fix**: `IrisViewPeer:: + update_ime_selection` (new, `ime.rs`) reports the real selection + and (an approximation, `compose_len` chars back from the caret) + the composing region, called from `after_input`'s existing tail so + every touch/key/IME callback already runs it. The buffer-level + half (`replace`/`insert_str` correctly advancing the caret) was + already correct and is now covered by four new unit tests in + `iris/src/widget/text/edit.rs` (composing, `commitText`, + `deleteSurroundingText`, `setSelection`). **Verified**: on the + emulator (`force-gles`, no Vulkan adapter on this AVD), tapping a + real Gboard key now shows a real, single-character-appropriate + suggestion strip ("H | How | Hey") rather than stale state, and a + `render()` log line fires for every keystroke -- both confirm the + `InputConnection` calls are landing and are being processed, which + a hand-typed `adb shell input text` did *not* reliably exercise on + this AVD (no `render()` at all followed one such call -- most + likely a modern `input text` no longer round-trips through + `commitText` the way older docs assume; Gboard-key taps are the + real path and the one this fix was verified against). + + 2. **A second, deeper bug found while verifying (1), not root-caused + this pass**: composed text never becomes visible on screen at + all -- the grey composer bar stays empty, with no glyph anywhere + in the frame, confirmed on repeated Gboard-key taps and across a + keyboard-resize. **Ruled out**: the widget tree's own layout math. + A new unit test, `layout_tests:: + composing_text_after_a_keyboard_resize_lands_in_the_bars_own_region`, + builds the composer's exact tree shape (`Stack{rect, Span{Pad{ + TextEdit}}}` inside an outer `Span::DOWN`) with no GPU or window, + resizes it the way a real keyboard-triggered `surface_changed` + does, edits the field both before and after, and asserts the + field's `window_region` stays a small box near the bottom of + whichever window size is current -- it passes, both before and + after this pass's composer rebuild (item 3 below), so the CPU-side + region a redraw lands at is provably correct. The bug is + therefore downstream of that -- most likely something specific to + the GPU-side redraw a content-only edit takes (`UiRenderState:: + redraw`, which redraws a single dirtied widget directly at its + stored region rather than re-running its ancestors' layout) or to + this AVD's forced `force-gles` backend (the only one available + here; Iris's phone deliveries have used real Vulkan) -- neither + isolated this pass. **Not attributable to this pass's changes**: + reproduced identically before touching `composer.rs` (the very + first build tested, before the composer rebuild below, already + had it) and the render-engine files this pass did not touch + (`core/src/render/mod.rs`, `core/src/ui/render_state.rs`) are the + likely next place to look -- specifically `UiRenderState::redraw`'s + reuse of a widget's own last-drawn region versus a full tree walk. + **Needs**: either a Vulkan-capable emulator boot or the real phone + to rule `force-gles` in or out, and a GPU-side primitive dump + (the existing `frame diagnostics` log line, extended to name which + primitives a frame actually wrote) to see whether the glyph quads + are emitted at all or emitted somewhere off-screen. + + 3. **The composer bar rebuilt as one widget**, per this box's own + ask: `transcript_ui::composer::build_composer` (unchanged + `Stack{background, Span{Pad{TextEdit}}}` idiom, the same one the + header row's `HEADER_SURFACE` already uses) now also caps the + field at roughly six lines (`MaxSize` + `.scrollable()` for a + wheel/trackpad overflow scroll -- a real touch-drag scroll on + overflowing composer text is not wired and is a follow-up) and + wraps the whole bar in one `Pad` whose `bottom` a new + `Composer::set_bottom_inset(rsc, inset)` rewrites in place + whenever the platform's insets change, called from + `bench_client.rs`'s existing `on_insets_changed` with + `insets.bottom.max(insets.ime_bottom)` -- the IME's own inset + while it is open, the navigation bar's otherwise. Rewritten in + place rather than rebuilt through a `WidgetPtr` swap (`top_bar`'s + own pattern) because the field is strongly owned inside this tree + and cannot be re-added to a new wrapper without panicking + ("was already added") -- rebuilding would also drop focus, + selection and in-progress text on every keyboard toggle. + **Verified**: `ui-trace` box readouts before/after a keyboard + open on the emulator (the field's row correctly reports a + 547px move matching the real IME-triggered resize); the + known-separate "top row renders twice after a keyboard resize" + bug this box already recorded is unrelated and still open. **Not + fixed by this alone**: item 2 above -- the text still does not + render, so the "empty space at the bottom" symptom's other half + (nothing filling the space the bar itself now correctly reserves) + needs item 2's fix first before a real before/after screenshot is + worth taking. + + 4. **App-switch text loss, fixed and verified.** `surface_destroyed` + (backgrounding) drops the whole `AndroidRenderer` -- device, + atlas, buffers -- and a subsequent `surface_changed` with no live + renderer builds a genuinely new one (`AndroidRenderer::new`, + distinct from the keyboard-resize path this box already fixed by + *reusing* the renderer). But `iris_core::TextData::atlas` (the + CPU-side glyph cache) and `UiData::textures` (the CPU-side texture + bookkeeping the atlas is built on) live on `AndroidRsc`, which + outlives any one `AndroidRenderer` -- so both kept pointing at the + *old*, now-destroyed device's textures across the switch, the + exact "rectangles stay, glyphs disappear" shape, just triggered by + backgrounding instead of the keyboard. **Fix**: new + `GlyphAtlas::clear()` and `Textures::reset()` (`iris/core/src/ + render/atlas.rs`, `iris/core/src/primitive/texture.rs`), called + together from `surface_changed`'s "genuinely new renderer" branch + only -- the same `already_live` check that already decides + reuse-vs-new, so this is one mechanism gated on the one condition + that needs it, not a second ad hoc check. **Verified on the + emulator**: backgrounded via `KEYCODE_HOME`, reopened via + `am start`, screenshotted -- every pre-existing glyph (headings, + body text, the whole diagnostics report) is intact, `frame_count` + resets to 1 confirming a genuinely new renderer was built, no + crash. + + 5. **Swipe-vs-tap focus, fixed and verified** (Iris's follow-up the + same day: "if I swipe over the input bar it brings up the + keyboard... scrolling should be pinned"). `attr.rs`'s `Selector`/ + `Selectable` registered `CursorSense::click_or_drag()`, which + calls `select()` -- and so grants focus and requests the IME -- + on the *first* frame of any press, before it is known whether the + gesture will end up a tap or a drag. Rewritten around a shared + `on_press` dispatcher over `PressStart`/`Pressing`/`PressEnd`: a + field that is **already** focused behaves exactly as before + (every frame updates the selection, so dragging inside a focused + field to select text still works); a field that is **not** + focused records where the press began (`TextEdit::press_origin`, + new field) and only grants focus on `PressEnd` if no intervening + frame crossed `sense::DRAG_SLOP` -- a drag recognised early simply + clears the pending tap and does nothing further, so it is never + consumed and whatever is behind the field still sees every frame + of it. New `FocusHost::is_focused` (both platform impls) is what + lets `on_press` tell the two cases apart. **Verified on the + emulator**: `dumpsys input_method`'s `mInputShown` reads `false` + after a `swipe` gesture starting on the composer bar (`ui-trace` + confirms the field's own box never moved, i.e. no keyboard-driven + resize happened), and reads `true` after an ordinary `tap` on the + same field. **Coordination note**: a concurrent pass is moving + drag arbitration into `sense.rs` behind a new `Drop` event: this + fix touches only `attr.rs` (new `press_track`/`on_press`) and + `iris/src/widget/text/edit.rs` (the new `press_origin` field), not + `sense.rs` itself, so it should merge cleanly, but the next agent + through here should check whether `Selector`/`Selectable`'s + `Pressing`-frame delivery still arrives the way this code assumes + once that lands. + + **Checks this pass**: `cargo fmt --all` clean, `cargo clippy + --workspace --all-targets` and `cargo ndk -t x86_64 -P 26 clippy + --features "transcript-screen bench force-gles"` both zero warnings + beyond the pre-existing `tabs-ui` unused-dependency notice, `cargo + test --workspace` all passing (new tests: four in `edit.rs`, one in + `layout_tests.rs`). **Not done**: item 2's root cause; a real + before/after screenshot pair for item 3 (blocked on item 2); anything + on Vulkan or the real phone. + - [ ] **P1 — session screen parity.** History paging backward (with the page-boundary healing `client-core` does not have yet, below), `TranscriptSource`-backed cache/server stitching, jump-to-latest, diff --git a/iris/android-app/src/bench_client.rs b/iris/android-app/src/bench_client.rs index b35da88..40a691b 100644 --- a/iris/android-app/src/bench_client.rs +++ b/iris/android-app/src/bench_client.rs @@ -272,6 +272,18 @@ impl AndroidAppState for BenchClient { let controls = bench_controls(rsc, insets.top); (self.top_bar)(rsc).set(controls); + // The composer bar sits directly on whichever of the IME or the + // navigation bar is currently the bottom of usable space -- see + // `transcript_ui::composer::Composer::set_bottom_inset`'s doc. + // `ime_bottom` already exceeds the plain nav-bar inset whenever the + // keyboard covers it, so the larger of the two is always the right + // answer without needing to know which is currently showing. + if let Some(screen) = &self.screen { + screen + .composer + .set_bottom_inset(rsc, insets.bottom.max(insets.ime_bottom)); + } + let ime_visible = insets.ime_bottom > 0.0; if ime_visible && !self.keyboard_was_visible { self.keyboard_was_visible = true; diff --git a/iris/core/src/primitive/texture.rs b/iris/core/src/primitive/texture.rs index 8486c0b..ab3ea10 100644 --- a/iris/core/src/primitive/texture.rs +++ b/iris/core/src/primitive/texture.rs @@ -141,6 +141,27 @@ impl Textures { self.updates.push(Update::Patch(handle.slot, rect)); } + /// Forget every image, page and pending update -- what a genuinely new + /// GPU device needs alongside [`crate::render::atlas::GlyphAtlas:: + /// clear`], which this module's own doc references: every slot number + /// and every queued [`Update`] here describes the *old* device's + /// textures (an `Update::Push`/`Update::Patch` already drained into a + /// renderer that no longer exists is gone for good, and a fresh + /// `UiRenderNode`'s own texture manager starts with none of them + /// applied), so nothing is lost by starting this bookkeeping over too. + /// Any `TextureHandle` a caller still holds across the reset (none in + /// the transcript screen this reset is wired up for today -- confirmed + /// by grep, the only standalone (non-atlas) image anywhere in this + /// workspace is `iris/widget/image.rs`'s `Image`, used by the separate + /// `tabs-ui` example) is left pointing at a slot this instance no + /// longer recognises and needs reinserting via `add`/`add_page` again + /// -- the same pre-existing gap a renderer restart already left for + /// such a handle before this method existed, just named rather than + /// silent now. + pub fn reset(&mut self) { + *self = Self::new(); + } + pub fn free(&mut self) { for (kind, idx) in self.recv.try_iter() { self.images[idx as usize] = None; diff --git a/iris/core/src/render/atlas.rs b/iris/core/src/render/atlas.rs index 027b407..4578639 100644 --- a/iris/core/src/render/atlas.rs +++ b/iris/core/src/render/atlas.rs @@ -173,6 +173,25 @@ impl GlyphAtlas { pub fn glyph_count(&self) -> usize { self.entries.len() } + + /// Forget every page and every rasterised entry -- what a genuinely new + /// GPU device needs (`android::view::IrisViewPeer::surface_changed`'s + /// "not already live" branch, e.g. after backgrounding): the pages this + /// atlas remembers are `TextureHandle`s into the *old* device's + /// textures, which no longer exist, and every `GlyphEntry`'s `uv_min`/ + /// `uv_max`/`layer` point into them. Without this, a glyph already + /// cached here is treated as "already placed" and never re-inserted + /// into the fresh (empty) atlas the new renderer actually has -- + /// exactly the "rectangles stay, glyphs disappear" bug the resize path + /// (`AndroidRenderer::resize`) was built to avoid for the reuse case; + /// this is its counterpart for the case where the renderer really is + /// new. Dropping `pages` also drops its `TextureHandle`s, which send a + /// free message back through their `Textures`; see `Textures::reset`'s + /// doc for why that is harmless here. + pub fn clear(&mut self) { + self.pages.clear(); + self.entries.clear(); + } } fn fits(page: &Page, need_w: u32, need_h: u32) -> bool { diff --git a/iris/src/android/attr.rs b/iris/src/android/attr.rs index dbb163f..e980dfa 100644 --- a/iris/src/android/attr.rs +++ b/iris/src/android/attr.rs @@ -12,6 +12,10 @@ impl FocusHost for T { self.android_state_mut().focus = id; } + fn is_focused(&self, id: WeakWidget) -> bool { + self.android_state().focus == Some(id) + } + fn focus_gained(&mut self, region: Option) { // Showing the keyboard is a JNI call (`InputMethodManager.showSoftInput`), // and this runs deep inside the platform-agnostic sensor dispatch diff --git a/iris/src/android/ime.rs b/iris/src/android/ime.rs index 07ac959..7f9a3b6 100644 --- a/iris/src/android/ime.rs +++ b/iris/src/android/ime.rs @@ -49,6 +49,52 @@ impl IrisViewPeer { fn focus(&self) -> Option> { self.state.android_state().focus } + + /// Tell Gboard where the caret/selection and the composing region + /// actually are, via `InputMethodManager.updateSelection` -- every one + /// of android-view's own demo's `set_composing_text_internal`/`render` + /// calls this, and this bridge never did, which is what left Gboard's + /// own model of the field diverging from `TextEdit`'s real one after + /// the very first edit (RUST.md's P0 box, "doesn't enter it until I + /// hit space, and also doesn't move cursor forward" -- Gboard holds + /// its composing keystrokes back until it believes the app has caught + /// up, and without this call it never does). Called from + /// [`IrisViewPeer::after_input`], the one tail every touch/key/IME + /// callback already runs through, rather than duplicated at each of + /// this file's mutating methods. + /// + /// `candidates_start`/`candidates_end` report the composing region; + /// `-1, -1` when nothing is composing, matching `EditorInfo`'s own + /// convention. `compose_len` is tracked in `char`s (this module's doc + /// comment), so this reports it as that many UTF-16 units back from the + /// caret -- exact for the common BMP case, the same approximation + /// `set_composing_text` already makes. + pub(super) fn update_ime_selection(&mut self, ctx: &mut CallbackCtx) { + let Some(focus) = self.focus() else { return }; + let text = &self.rsc[focus]; + let Some(sel) = text.selection_range() else { + return; + }; + let content = text.text(); + let sel_start = byte_to_utf16(content, sel.start) as i32; + let sel_end = byte_to_utf16(content, sel.end) as i32; + let compose_len = self.state.android_state().compose_len; + let (comp_start, comp_end) = if compose_len > 0 { + let caret = byte_to_utf16(content, text.caret().unwrap_or(sel.end)) as i32; + (caret - compose_len as i32, caret) + } else { + (-1, -1) + }; + let imm = ctx.view.input_method_manager(&mut ctx.env); + imm.update_selection( + &mut ctx.env, + &ctx.view, + sel_start, + sel_end, + comp_start, + comp_end, + ); + } } impl InputConnection for IrisViewPeer { diff --git a/iris/src/android/view.rs b/iris/src/android/view.rs index 403ccbb..5a06e97 100644 --- a/iris/src/android/view.rs +++ b/iris/src/android/view.rs @@ -325,6 +325,13 @@ impl IrisViewPeer { show_soft_input(&mut ctx.env, &ctx.view); } + // RUST.md's P0 box, "doesn't enter it until I hit space, and also + // doesn't move cursor forward": Gboard needs `updateSelection` + // after every edit to keep its own model of the field in sync, or + // it holds keystrokes back rather than trusting a screen it + // believes is stale. See `update_ime_selection`'s own doc. + self.update_ime_selection(ctx); + let ui_state = self.state.android_state_mut(); ui_state.cursor.end_frame(); if self.render.needs_redraw(&ui_state.root, self.rsc.widgets()) { @@ -650,6 +657,22 @@ impl ViewPeer for IrisViewPeer { let content_scale = self.state.android_state().content_scale; match AndroidRenderer::new(window, width as u32, height as u32, content_scale) { Ok(renderer) => { + // A genuinely new renderer means a genuinely new GPU device + // and a fresh, empty glyph atlas -- the CPU-side glyph + // cache (`TextData::atlas`) and the texture bookkeeping it + // is built on (`UiData::textures`) both outlive `renderer` + // itself (they live on `self.rsc`, not on `AndroidRenderer`), + // so without this they would keep pointing at the *old* + // device's now-gone textures -- the app-switch counterpart + // to the keyboard-resize glyph wipe this same function's + // `already_live` branch above already fixed by reusing the + // renderer instead of rebuilding it. One mechanism either + // way: this call only runs on the branch that actually + // builds a new renderer, exactly where invalidation is + // needed, never on the reuse branch, where it would throw + // away perfectly valid GPU state for nothing. + self.rsc.ui.text.atlas.clear(); + self.rsc.ui.textures.reset(); self.state.android_state_mut().renderer = Some(renderer); self.render(ctx); } diff --git a/iris/src/attr.rs b/iris/src/attr.rs index 29acb9b..6f100ba 100644 --- a/iris/src/attr.rs +++ b/iris/src/attr.rs @@ -22,6 +22,13 @@ pub trait FocusHost { /// it was hit in (`None` when the widget could not be located, which /// happens for one it was just deselected from). fn focus_gained(&mut self, region: Option); + /// Whether `id` is the current focus target -- what [`select`] uses to + /// tell a fresh press (which must wait to see whether it becomes a tap + /// or a drag before focusing/showing the IME, Iris 2026-09-06: "if I + /// swipe over the input bar it brings up the keyboard") from a drag + /// continuing inside a field that was already focused (an ordinary + /// drag-to-select, unaffected). + fn is_focused(&self, id: WeakWidget) -> bool; } /// Helper shared by every `FocusHost` impl, so the double-click window is @@ -33,6 +40,17 @@ pub fn recent_click(last_click: &mut Instant) -> bool { recent } +/// `PressStart`/`Pressing`/`PressEnd`, all for the left button -- what +/// [`Selector`]/[`Selectable`] register instead of [`CursorSense:: +/// click_or_drag`], so their shared handler (`on_press`, below) sees every +/// frame of a gesture and can tell a completed tap from a drag itself, +/// rather than reacting to `PressStart` alone the way `click_or_drag`'s +/// consumer used to (Iris, 2026-09-06: "if I swipe over the input bar it +/// brings up the keyboard"). +fn press_track() -> CursorSenses { + CursorSense::click() | CursorSense::Pressing(CursorButton::Left) | CursorSense::unclick() +} + pub struct Selector; impl WidgetAttr for Selector @@ -42,7 +60,7 @@ where type Input = WeakWidget; fn run(rsc: &mut Rsc, container: WeakWidget, id: Self::Input) { - rsc.register_event(container, CursorSense::click_or_drag(), move |ctx, rsc| { + rsc.register_event(container, press_track(), move |ctx, rsc| { let region = ctx.data.render.window_region(&id, &*rsc).unwrap(); let id_pos = region.top_left; let container_pos = ctx @@ -53,14 +71,14 @@ where .top_left; let pos = ctx.data.pos + container_pos - id_pos; let size = region.size(); - select( + on_press( rsc, ctx.data.render, ctx.state, id, pos, size, - ctx.data.sense.is_dragging(), + ctx.data.sense, ); }); } @@ -75,31 +93,71 @@ where type Input = (); fn run(rsc: &mut Rsc, id: WeakWidget, _: Self::Input) { - rsc.register_event(id, CursorSense::click_or_drag(), move |ctx, rsc| { - select( + rsc.register_event(id, press_track(), move |ctx, rsc| { + on_press( rsc, ctx.data.render, ctx.state, id, ctx.data.pos, ctx.data.size, - ctx.data.sense.is_dragging(), + ctx.data.sense, ); }); } } -fn select( +/// One press-track frame (`PressStart`, `Pressing` or `PressEnd`) over a +/// selectable field. A field that is *already* focused behaves exactly as +/// `click_or_drag` always did -- every frame updates the selection, which +/// is what lets a finger already inside a focused field drag out a +/// selection. A field that is **not** focused withholds `select`'s +/// focus-granting side effects (and so the platform-specific `focus_gained` +/// that shows the keyboard) until the press resolves as a tap: `PressEnd` +/// with no frame in between having moved past [`DRAG_SLOP`] from where the +/// press began. A drag recognised before release simply cancels the +/// pending tap and does nothing further here -- it is not consumed, so +/// whatever is behind the field (a list to pan) still sees every frame of +/// it, the same as a drag that never touched a selectable field at all. +fn on_press( rsc: &mut impl UiRsc, render: &UiRenderState, state: &mut impl FocusHost, id: WeakWidget, pos: Vec2, size: Vec2, - dragging: bool, + sense: CursorSense, ) { - let recent = state.recent_click(); - id.edit(rsc).select(pos, size, dragging, recent); - state.set_focus(Some(id)); - state.focus_gained(render.window_region(&id, &*rsc)); + if state.is_focused(id) { + let recent = matches!(sense, CursorSense::PressStart(_)) && state.recent_click(); + id.edit(rsc).select(pos, size, sense.is_dragging(), recent); + return; + } + + match sense { + CursorSense::PressStart(_) => { + id.edit(rsc).text.press_origin = Some(pos); + } + CursorSense::Pressing(_) => { + let ctx = id.edit(rsc); + if let Some(origin) = ctx.text.press_origin + && ((pos.x - origin.x).abs() > DRAG_SLOP || (pos.y - origin.y).abs() > DRAG_SLOP) + { + // Past the slop before release: this is a drag, not a tap + // -- give up the pending focus rather than granting it once + // the finger lifts wherever it happens to be by then. + ctx.text.press_origin = None; + } + } + CursorSense::PressEnd(_) => { + let was_tap = id.edit(rsc).text.press_origin.take().is_some(); + if was_tap { + let recent = state.recent_click(); + id.edit(rsc).select(pos, size, false, recent); + state.set_focus(Some(id)); + state.focus_gained(render.window_region(&id, &*rsc)); + } + } + _ => {} + } } diff --git a/iris/src/default/attr.rs b/iris/src/default/attr.rs index 58edb12..d089d45 100644 --- a/iris/src/default/attr.rs +++ b/iris/src/default/attr.rs @@ -10,6 +10,10 @@ impl FocusHost for T { self.default_state_mut().focus = id; } + fn is_focused(&self, id: WeakWidget) -> bool { + self.default_state().focus == Some(id) + } + fn focus_gained(&mut self, region: Option) { let state = self.default_state_mut(); let Some(region) = region else { return }; diff --git a/iris/src/layout_tests.rs b/iris/src/layout_tests.rs index 3380880..1d5c467 100644 --- a/iris/src/layout_tests.rs +++ b/iris/src/layout_tests.rs @@ -183,3 +183,80 @@ fn a_mask_stays_put_while_its_scrolled_content_moves() { assert_eq!(mask_delta_before, [0.0, 0.0]); assert_eq!(mask_delta_after, [0.0, 0.0]); } + +/// Reproduces `transcript_ui::composer::build_composer`'s exact tree shape +/// (a `Rect` background stacked behind a `Span::RIGHT`-wrapped, padded, +/// `rest`-width `TextEdit`, itself the second child of an outer +/// `Span::DOWN` beside a `rest(1)`-height sibling) without the event/ +/// resource plumbing `composer.rs`'s builders need, to isolate whether the +/// bug Iris reported on 2026-09-06 ("text seems to not appear in box") +/// is this crate's layout engine or something specific to the real +/// composer/screen. `TextEditable::edit` only needs `UiRsc`, so a plain +/// insert exercises the exact redraw path a keystroke does. +fn composer_like_tree(rsc: &mut TestRsc) -> (WeakWidget, StrongWidget) { + let field = wtext("") + .editable(EditMode::MultiLine) + .text_align(Align::LEFT) + .wrap(true) + .size(18) + .color(UiColor::WHITE) + .add(rsc); + let bar = (field.pad(dp(12)).width(rest(1)),) + .span(Dir::RIGHT) + .background(rect(UiColor::new(40, 40, 46, 255))) + .add(rsc); + let list_stand_in = rect(UiColor::BLACK).height(rest(1)).add(rsc); + let tree = (list_stand_in, bar).span(Dir::DOWN).add_strong(rsc).any(); + (field, tree) +} + +/// The reproduction itself. A window this tall stands in for the keyboard +/// closed; the second, shorter `resize` stands in for `adjustResize` +/// shrinking the surface when the IME opens -- exactly the sequence +/// `IrisViewPeer::surface_changed` drives on a real keyboard open. Typing +/// happens both before and after, since Iris's report was specifically +/// that text typed *after* the keyboard was already up did not appear. +#[test] +fn composing_text_after_a_keyboard_resize_lands_in_the_bars_own_region() { + let mut rsc = TestRsc { + ui: UiData::default(), + }; + let (field, root) = composer_like_tree(&mut rsc); + let mut render = UiRenderState::new(); + + render.resize((1080.0, 2298.0)); + render.update(&root, &mut rsc); + field.edit(&mut rsc).insert("a"); + render.update(&root, &mut rsc); + + let before_px = render.window_region(&field, &rsc).unwrap(); + // The field is one line plus 12dp of padding on a 2298-tall window -- + // nowhere near the whole window's height, and anchored at the bottom. + assert!( + before_px.bot_right.y - before_px.top_left.y < 200.0, + "before a resize: {before_px:?}" + ); + assert!( + before_px.top_left.y > 1800.0, + "expected the bar near the bottom before a resize: {before_px:?}" + ); + + // The keyboard opens: a real `surface_changed`/`resize` to a shorter + // window, then a further keystroke -- the redraw that must land in the + // bar's new (also short) region, not whatever region a provisional + // measurement pass used along the way. + render.resize((1080.0, 1478.0)); + render.update(&root, &mut rsc); + field.edit(&mut rsc).insert("b"); + render.update(&root, &mut rsc); + + let after_px = render.window_region(&field, &rsc).unwrap(); + assert!( + after_px.bot_right.y - after_px.top_left.y < 200.0, + "after a resize + keystroke: {after_px:?}" + ); + assert!( + after_px.top_left.y > 1200.0, + "expected the bar near the bottom of the shorter window: {after_px:?}" + ); +} diff --git a/iris/src/widget/text/edit.rs b/iris/src/widget/text/edit.rs index 3feb162..effec38 100644 --- a/iris/src/widget/text/edit.rs +++ b/iris/src/widget/text/edit.rs @@ -32,6 +32,15 @@ pub struct TextEdit { #[cfg_attr(target_os = "android", allow(dead_code))] history: Vec<(String, Option)>, double_hit: Option, + /// Where an in-flight press over this field began, while it is still + /// undecided whether the gesture is a tap (focus/show the IME) or a + /// drag (attr.rs's `Selector`/`Selectable`, Iris 2026-09-06: a swipe + /// over the composer must not summon the keyboard). `None` both before + /// any press and once the gesture has been decided either way -- + /// `attr.rs` is the only reader/writer, kept `pub(crate)` rather than + /// behind an accessor since it is pure bookkeeping with no invariant + /// beyond "some press is undecided," same shape as `double_hit` above. + pub(crate) press_origin: Option, pub mode: EditMode, } @@ -48,6 +57,7 @@ impl TextEdit { selection: None, history: Default::default(), double_hit: None, + press_origin: None, mode, } } @@ -698,6 +708,67 @@ mod tests { assert_eq!(content(&t), "に"); } + /// `android/ime.rs`'s `set_composing_text` calls `replace` and expects + /// the caret to land right after the inserted text, growing with it on + /// every re-send -- the buffer-level half of RUST.md's P0 box ("doesn't + /// enter it until I hit space, and also doesn't move cursor forward"). + #[test] + fn composing_advances_the_caret_with_the_growing_text() { + let (mut t, mut d) = edit("", EditMode::SingleLine); + ctx(&mut t, &mut d).set_caret(0); + ctx(&mut t, &mut d).replace(0, "h"); + assert_eq!(t.caret(), Some(1)); + ctx(&mut t, &mut d).replace(1, "hi"); + assert_eq!(content(&t), "hi"); + assert_eq!(t.caret(), Some(2)); + ctx(&mut t, &mut d).replace(2, "hit"); + assert_eq!(content(&t), "hit"); + assert_eq!(t.caret(), Some(3)); + } + + /// The IME's `commitText` (`android_view::InputConnection::commit_text`'s + /// default body): finish a composition in place, same as a real word + /// boundary (a space) landing after Gboard's composing span. + #[test] + fn committing_composed_text_leaves_it_in_place_with_the_caret_after_it() { + let (mut t, mut d) = edit("say ", EditMode::SingleLine); + ctx(&mut t, &mut d).set_caret(4); + ctx(&mut t, &mut d).replace(0, "hi"); + assert_eq!(content(&t), "say hi"); + // `finish_composing_text`/`commit_text` do not themselves touch the + // buffer -- only the IME's own `compose_len` bookkeeping resets, in + // `android/ime.rs`. Confirms the buffer already holds committed + // text as plain, uncomposed content: a further `replace(0, " ")` + // (the space that ends the word) appends rather than overwriting. + ctx(&mut t, &mut d).replace(0, " "); + assert_eq!(content(&t), "say hi "); + assert_eq!(t.caret(), Some(7)); + } + + /// `TextEditCtx::delete_byte_range` is `deleteSurroundingText`'s entry + /// point once `android/ime.rs` has converted UTF-16 code units to + /// bytes -- exercised directly here in bytes, since the UTF-16 math + /// itself is `android/ime.rs`'s own `byte_to_utf16`/`utf16_to_byte`, + /// outside this widget-only test module. + #[test] + fn delete_byte_range_removes_exactly_that_range() { + let (mut t, mut d) = edit("hello world", EditMode::SingleLine); + ctx(&mut t, &mut d).delete_byte_range(5, 11); + assert_eq!(content(&t), "hello"); + assert_eq!(t.caret(), Some(5)); + } + + /// `set_cursor_byte` is `setSelection`'s entry point -- collapses to a + /// caret at the given byte offset regardless of any span that was there. + #[test] + fn set_cursor_byte_collapses_to_a_caret_there() { + let (mut t, mut d) = edit("hello world", EditMode::SingleLine); + ctx(&mut t, &mut d).select_all(); + ctx(&mut t, &mut d).set_cursor_byte(5); + assert_eq!(t.selected_text(), None); + assert_eq!(t.caret(), Some(5)); + } + #[test] fn motion_moves_the_caret_and_shift_extends_a_span() { let (mut t, mut d) = edit("abc", EditMode::SingleLine); diff --git a/iris/transcript-ui/src/composer.rs b/iris/transcript-ui/src/composer.rs index 741c1b1..a91d314 100644 --- a/iris/transcript-ui/src/composer.rs +++ b/iris/transcript-ui/src/composer.rs @@ -8,14 +8,58 @@ //! measures whatever vertical space is left each frame -- nothing here //! computes a height by hand, and growing this field is exactly the //! O(1)-move-chain case LAYOUT.md and I3's benchmark already measured. +//! +//! **Rebuilt 2026-09-06** (Iris's phone report on the dc01f88 build: the +//! grey bar drawn as a short, fixed strip with the typed text ~150px below +//! it on black, and empty black between the bar and the keyboard). One +//! widget now, top to bottom: an opaque background sized to its content +//! (`.background`, the same `Stack` idiom the header row's `HEADER_SURFACE` +//! already uses), the field inside `dp` padding and capped at +//! [`MAX_LINES`] before it scrolls instead of growing forever, and an +//! outer [`Pad`] whose `bottom` [`TranscriptScreen::set_bottom_inset`] +//! rewrites in place whenever the keyboard opens/closes -- never rebuilt, +//! since `field` is strongly owned inside this tree and this crate's +//! widgets cannot be re-parented once added (this module's own comment +//! below on why `build_composer` hands back a **weak** id). use iris::prelude::*; +/// Caps the field's growth at roughly six lines of its own 18px text +/// before it scrolls instead of consuming the whole screen -- an +/// approximation (line-height and padding folded into one round `dp` +/// number) rather than a value derived from the font's real metrics, +/// which nothing in this crate exposes to a caller today. +const MAX_LINES: f32 = 6.0; +const APPROX_LINE_HEIGHT_DP: f32 = 24.0; +const FIELD_PAD_DP: f32 = 12.0; + /// `field` is exposed so the caller can read its content on submit /// (`field.edit(rsc).text()`) and clear it afterward /// (`field.edit(rsc).set("")`). pub struct Composer { pub field: WeakWidget, + /// The bar's own outer padding -- only `bottom` is ever changed, by + /// [`Self::set_bottom_inset`]. A `Pad` around the whole bar rather than + /// a rebuilt tree, because `field` lives inside it and cannot be + /// re-added to a new wrapper once it is strongly owned here. + outer_pad: WeakWidget, +} + +impl Composer { + /// Called by the platform shell (Android's `on_insets_changed`, e.g.) + /// whenever the space below the bar changes: the IME's own inset while + /// it is open, the navigation-bar inset otherwise. Takes a plain + /// `f32` in the caller's own physical-pixel units rather than an + /// Android-specific insets type, so this crate stays usable from the + /// winit backend too, which has no navigation bar to report. + /// Rewrites the existing `Pad` in place (marking it dirty through the + /// ordinary `Widgets::get_mut` path) instead of swapping in a new one, + /// so the field's focus, selection and in-progress text are untouched. + pub fn set_bottom_inset(&self, rsc: &mut impl UiRsc, inset: f32) { + if let Some(pad) = rsc.ui_mut().widgets.get_mut(&self.outer_pad) { + pad.padding.bottom = Len::abs(inset); + } + } } /// Returns the composer plus its own bar as a **weak** id -- the caller @@ -39,10 +83,20 @@ where .label("Message") .add(rsc); - let bar: WeakWidget = (field.pad(dp(12)).width(rest(1)),) - .span(Dir::RIGHT) + // One widget: an opaque bar sized to its own content (`.background`'s + // `Stack{child: 1}`, the header row's own idiom) wrapping the padded, + // height-capped field -- not a background rect and a field drawn as + // two independent siblings, which is what let the two disagree on + // where the bar actually was. + let content = field + .pad(dp(FIELD_PAD_DP)) + .max_height(dp(APPROX_LINE_HEIGHT_DP * MAX_LINES + FIELD_PAD_DP * 2.0)) + .scrollable() + .width(rest(1)) .background(rect(UiColor::new(40, 40, 46, 255))) .add(rsc); - (Composer { field }, bar) + let outer_pad: WeakWidget = content.pad(Padding::ZERO).add(rsc); + + (Composer { field, outer_pad }, outer_pad) }