iris/android: composing text sync, tap-vs-swipe focus, composer rebuild, atlas reset on app-switch
Four fixes from Iris's phone report on the dc01f88 build, plus her same-day
follow-up on swipe-vs-tap:
- android/ime.rs: InputConnection now calls InputMethodManager.updateSelection
after every edit (new update_ime_selection, called from after_input) -- Gboard
was holding keystrokes back with nothing telling it the app's selection/
composing region had moved, which read as "doesn't enter it until I hit
space, doesn't move the caret". New unit tests in widget/text/edit.rs cover
the buffer-level composing/commit/delete/selection operations directly.
- attr.rs: Selector/Selectable rewritten around a shared on_press dispatcher
over PressStart/Pressing/PressEnd instead of click_or_drag(), so a field
that isn't already focused only grants focus (and requests the IME) on a
completed tap -- press and release with no frame past DRAG_SLOP. A drag
is never consumed, so whatever is behind the field still sees it. New
FocusHost::is_focused (both platform impls) and TextEdit::press_origin
back this. Verified on the emulator: dumpsys input_method's mInputShown
stays false after a swipe over the composer, true after a tap.
- iris_core: GlyphAtlas::clear()/Textures::reset(), called together from
android/view.rs's surface_changed exactly when a genuinely new renderer is
built (app-switch, not the keyboard-resize path that already reuses the
renderer) -- both CPU-side caches otherwise kept pointing at the old,
destroyed device's textures. Verified on the emulator: home, reopen, every
glyph still on screen.
- transcript-ui/composer.rs: rebuilt as one widget (unchanged Stack{rect,
span} idiom, capped at ~6 lines via MaxSize + .scrollable(), wrapped in one
Pad whose bottom Composer::set_bottom_inset rewrites in place so the bar
sits on the IME or nav-bar inset with no rebuild -- rebuilding would drop
focus/selection/in-progress text). Wired from bench_client.rs's existing
on_insets_changed.
A second, deeper bug found while verifying the composing fix is NOT fixed
this pass: composed text never becomes visible at all. A new layout_tests.rs
test proves the widget tree's own region math is correct across a keyboard
resize, ruling that out; RUST.md's P0 box has the full writeup and what to
check next (UiRenderState::redraw's single-widget path, or something
force-gles-specific -- this AVD has no Vulkan adapter to rule that out with).
cargo fmt/clippy/test --workspace and cargo ndk clippy all clean.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
1 parent
dc01f88d75
commit
20b12255e1
14 files changed
+628
-15
No files matched your search
@@ -626,3 +626,32 @@ box has the full investigation and the phone verification still to do.
|
||||
built and checked on this checkout's emulator only. RUST.md's P0 box
|
||||
says what she should check for: crisp text at two densities, the
|
||||
keyboard no longer wiping, and the header's background.
|
||||
|
||||
## 2026-09-06: composing text, focus-on-tap, and atlas invalidation on a new renderer
|
||||
|
||||
Three small but public API changes, from the same phone-report pass as the
|
||||
entry above (RUST.md's P0 box has the full account, including a real bug
|
||||
still not root-caused).
|
||||
|
||||
- **`FocusHost` gained `is_focused(&self, id) -> bool`** (both platform
|
||||
impls). `attr.rs`'s `Selector`/`Selectable` used to grant focus (and so
|
||||
request the IME) on the very first frame of *any* press, before it was
|
||||
known whether the gesture was a tap or a drag — a swipe over a text
|
||||
field wrongly summoned the keyboard. They now wait for a completed tap
|
||||
(press and release with no frame crossing `sense::DRAG_SLOP`) unless the
|
||||
field is already focused, in which case dragging inside it to select
|
||||
text is unchanged. `TextEdit` gained one new `pub(crate)` field
|
||||
(`press_origin`) to track this; no public surface change there.
|
||||
- **`android::ime`'s `InputConnection` now calls `InputMethodManager::
|
||||
updateSelection` after every edit** (`IrisViewPeer::update_ime_selection`,
|
||||
called from `after_input`). Gboard was holding keystrokes back because
|
||||
nothing ever told it where the app's own selection/composing region had
|
||||
moved to — this is what android-view's own demo does in its `render()`
|
||||
and this bridge never did.
|
||||
- **`GlyphAtlas::clear()` and `Textures::reset()`** (`iris_core`). Called
|
||||
together, once, from `android::view`'s `surface_changed` exactly when a
|
||||
*genuinely new* `AndroidRenderer` is built (backgrounding and returning,
|
||||
not a keyboard-triggered resize, which already reuses the renderer) —
|
||||
both CPU-side caches otherwise kept pointing at the old, now-destroyed
|
||||
device's textures, which is why text used to vanish again after leaving
|
||||
and returning to the app.
|
||||
@@ -149,6 +149,39 @@ agent takes them without colliding with that pass's `bench_client.rs`/
|
||||
confirming this was the whole story on real touch input rather than
|
||||
only the arbiter's own unit tests -- worth a follow-up pass before
|
||||
calling it fully closed.
|
||||
- [x] **Composing text held back until a space, caret not moving, fixed
|
||||
2026-09-06.** `InputMethodManager.updateSelection` was never called --
|
||||
see IRIS.md's 2026-09-06 entry and RUST.md's P0 box, item 1, for the
|
||||
full account and the emulator evidence.
|
||||
- [x] **Swipe over the composer summons the keyboard, fixed 2026-09-06.**
|
||||
`Selector`/`Selectable` now wait for a completed tap -- see IRIS.md's
|
||||
2026-09-06 entry and RUST.md's P0 box, item 5. Verified via `dumpsys
|
||||
input_method`'s `mInputShown` on the emulator, not yet on the phone.
|
||||
- [x] **Text disappears again after leaving and returning to the app,
|
||||
fixed 2026-09-06.** `GlyphAtlas::clear`/`Textures::reset` on a
|
||||
genuinely new renderer -- see IRIS.md's 2026-09-06 entry and RUST.md's
|
||||
P0 box, item 4. Verified on the emulator (home, reopen, screenshot);
|
||||
not yet on the phone.
|
||||
- [ ] **Composed/typed text never becomes visible at all -- found
|
||||
2026-09-06, not fixed.** The composer bar stays empty even once the
|
||||
buffer genuinely holds the typed text (confirmed indirectly: Gboard's
|
||||
own suggestion strip reacts correctly to each keystroke). A new unit
|
||||
test proves the widget tree's own layout math resolves the field's
|
||||
region correctly across a keyboard resize, so the bug is downstream of
|
||||
that -- most likely `UiRenderState::redraw`'s single-widget redraw path,
|
||||
or specific to this emulator's forced `force-gles` backend (untested on
|
||||
Vulkan or the real phone). RUST.md's P0 box, item 2, has the full
|
||||
writeup, what was ruled out, and where to look next. **Also unverified
|
||||
because of this**: item 3's composer rebuild (one `Stack`-based widget,
|
||||
a capped/scrollable height, bottom padding tied to the IME/nav-bar
|
||||
inset) -- structurally in place and unit-tested, but its own visual
|
||||
correctness cannot be screenshotted until text actually renders.
|
||||
- [ ] **The composer has no touch-drag scroll for overflowing text.** The
|
||||
2026-09-06 rebuild caps the field at ~6 lines and wraps it in
|
||||
`.scrollable()` for a wheel/trackpad scroll, but a real finger drag over
|
||||
text that has overflowed the cap does not scroll it -- `Scroll`'s touch
|
||||
handling is a follow-up, the same shape `List`'s own touch-drag pan
|
||||
needed before I3/I5.
|
||||
|
||||
## Build
|
||||
|
||||
|
||||
+162
@@ -4581,6 +4581,168 @@ device.
|
||||
the Compose half above is already done and is the reference shape
|
||||
for whoever picks this up. No redelivery this pass.
|
||||
|
||||
**Composing text, the tap-vs-swipe focus rule, and app-switch text
|
||||
loss, 2026-09-06.** Iris's report on this same dc01f88 build: typing
|
||||
doesn't enter text or move the caret until a space is hit; typed
|
||||
text doesn't visibly appear and there is empty black space below the
|
||||
composer bar; text disappears again after leaving and returning to
|
||||
the app; and (a follow-up message the same day) swiping over the
|
||||
composer bar wrongly summons the keyboard.
|
||||
|
||||
1. **The caret/composing bug's cause**: `android/ime.rs`'s
|
||||
`InputConnection` never called `InputMethodManager.updateSelection`
|
||||
after an edit -- confirmed by reading android-view's own demo
|
||||
(`~/src/android-view/demo/src/lib.rs`'s `render()`), which calls it
|
||||
every time its editor's generation changes. Without it, Gboard has
|
||||
no confirmation the app is keeping up and holds keystrokes back
|
||||
rather than trusting a screen it believes is stale -- exactly
|
||||
"doesn't enter it until I hit space." **Fix**: `IrisViewPeer::
|
||||
update_ime_selection` (new, `ime.rs`) reports the real selection
|
||||
and (an approximation, `compose_len` chars back from the caret)
|
||||
the composing region, called from `after_input`'s existing tail so
|
||||
every touch/key/IME callback already runs it. The buffer-level
|
||||
half (`replace`/`insert_str` correctly advancing the caret) was
|
||||
already correct and is now covered by four new unit tests in
|
||||
`iris/src/widget/text/edit.rs` (composing, `commitText`,
|
||||
`deleteSurroundingText`, `setSelection`). **Verified**: on the
|
||||
emulator (`force-gles`, no Vulkan adapter on this AVD), tapping a
|
||||
real Gboard key now shows a real, single-character-appropriate
|
||||
suggestion strip ("H | How | Hey") rather than stale state, and a
|
||||
`render()` log line fires for every keystroke -- both confirm the
|
||||
`InputConnection` calls are landing and are being processed, which
|
||||
a hand-typed `adb shell input text` did *not* reliably exercise on
|
||||
this AVD (no `render()` at all followed one such call -- most
|
||||
likely a modern `input text` no longer round-trips through
|
||||
`commitText` the way older docs assume; Gboard-key taps are the
|
||||
real path and the one this fix was verified against).
|
||||
|
||||
2. **A second, deeper bug found while verifying (1), not root-caused
|
||||
this pass**: composed text never becomes visible on screen at
|
||||
all -- the grey composer bar stays empty, with no glyph anywhere
|
||||
in the frame, confirmed on repeated Gboard-key taps and across a
|
||||
keyboard-resize. **Ruled out**: the widget tree's own layout math.
|
||||
A new unit test, `layout_tests::
|
||||
composing_text_after_a_keyboard_resize_lands_in_the_bars_own_region`,
|
||||
builds the composer's exact tree shape (`Stack{rect, Span{Pad{
|
||||
TextEdit}}}` inside an outer `Span::DOWN`) with no GPU or window,
|
||||
resizes it the way a real keyboard-triggered `surface_changed`
|
||||
does, edits the field both before and after, and asserts the
|
||||
field's `window_region` stays a small box near the bottom of
|
||||
whichever window size is current -- it passes, both before and
|
||||
after this pass's composer rebuild (item 3 below), so the CPU-side
|
||||
region a redraw lands at is provably correct. The bug is
|
||||
therefore downstream of that -- most likely something specific to
|
||||
the GPU-side redraw a content-only edit takes (`UiRenderState::
|
||||
redraw`, which redraws a single dirtied widget directly at its
|
||||
stored region rather than re-running its ancestors' layout) or to
|
||||
this AVD's forced `force-gles` backend (the only one available
|
||||
here; Iris's phone deliveries have used real Vulkan) -- neither
|
||||
isolated this pass. **Not attributable to this pass's changes**:
|
||||
reproduced identically before touching `composer.rs` (the very
|
||||
first build tested, before the composer rebuild below, already
|
||||
had it) and the render-engine files this pass did not touch
|
||||
(`core/src/render/mod.rs`, `core/src/ui/render_state.rs`) are the
|
||||
likely next place to look -- specifically `UiRenderState::redraw`'s
|
||||
reuse of a widget's own last-drawn region versus a full tree walk.
|
||||
**Needs**: either a Vulkan-capable emulator boot or the real phone
|
||||
to rule `force-gles` in or out, and a GPU-side primitive dump
|
||||
(the existing `frame diagnostics` log line, extended to name which
|
||||
primitives a frame actually wrote) to see whether the glyph quads
|
||||
are emitted at all or emitted somewhere off-screen.
|
||||
|
||||
3. **The composer bar rebuilt as one widget**, per this box's own
|
||||
ask: `transcript_ui::composer::build_composer` (unchanged
|
||||
`Stack{background, Span{Pad{TextEdit}}}` idiom, the same one the
|
||||
header row's `HEADER_SURFACE` already uses) now also caps the
|
||||
field at roughly six lines (`MaxSize` + `.scrollable()` for a
|
||||
wheel/trackpad overflow scroll -- a real touch-drag scroll on
|
||||
overflowing composer text is not wired and is a follow-up) and
|
||||
wraps the whole bar in one `Pad` whose `bottom` a new
|
||||
`Composer::set_bottom_inset(rsc, inset)` rewrites in place
|
||||
whenever the platform's insets change, called from
|
||||
`bench_client.rs`'s existing `on_insets_changed` with
|
||||
`insets.bottom.max(insets.ime_bottom)` -- the IME's own inset
|
||||
while it is open, the navigation bar's otherwise. Rewritten in
|
||||
place rather than rebuilt through a `WidgetPtr` swap (`top_bar`'s
|
||||
own pattern) because the field is strongly owned inside this tree
|
||||
and cannot be re-added to a new wrapper without panicking
|
||||
("was already added") -- rebuilding would also drop focus,
|
||||
selection and in-progress text on every keyboard toggle.
|
||||
**Verified**: `ui-trace` box readouts before/after a keyboard
|
||||
open on the emulator (the field's row correctly reports a
|
||||
547px move matching the real IME-triggered resize); the
|
||||
known-separate "top row renders twice after a keyboard resize"
|
||||
bug this box already recorded is unrelated and still open. **Not
|
||||
fixed by this alone**: item 2 above -- the text still does not
|
||||
render, so the "empty space at the bottom" symptom's other half
|
||||
(nothing filling the space the bar itself now correctly reserves)
|
||||
needs item 2's fix first before a real before/after screenshot is
|
||||
worth taking.
|
||||
|
||||
4. **App-switch text loss, fixed and verified.** `surface_destroyed`
|
||||
(backgrounding) drops the whole `AndroidRenderer` -- device,
|
||||
atlas, buffers -- and a subsequent `surface_changed` with no live
|
||||
renderer builds a genuinely new one (`AndroidRenderer::new`,
|
||||
distinct from the keyboard-resize path this box already fixed by
|
||||
*reusing* the renderer). But `iris_core::TextData::atlas` (the
|
||||
CPU-side glyph cache) and `UiData::textures` (the CPU-side texture
|
||||
bookkeeping the atlas is built on) live on `AndroidRsc`, which
|
||||
outlives any one `AndroidRenderer` -- so both kept pointing at the
|
||||
*old*, now-destroyed device's textures across the switch, the
|
||||
exact "rectangles stay, glyphs disappear" shape, just triggered by
|
||||
backgrounding instead of the keyboard. **Fix**: new
|
||||
`GlyphAtlas::clear()` and `Textures::reset()` (`iris/core/src/
|
||||
render/atlas.rs`, `iris/core/src/primitive/texture.rs`), called
|
||||
together from `surface_changed`'s "genuinely new renderer" branch
|
||||
only -- the same `already_live` check that already decides
|
||||
reuse-vs-new, so this is one mechanism gated on the one condition
|
||||
that needs it, not a second ad hoc check. **Verified on the
|
||||
emulator**: backgrounded via `KEYCODE_HOME`, reopened via
|
||||
`am start`, screenshotted -- every pre-existing glyph (headings,
|
||||
body text, the whole diagnostics report) is intact, `frame_count`
|
||||
resets to 1 confirming a genuinely new renderer was built, no
|
||||
crash.
|
||||
|
||||
5. **Swipe-vs-tap focus, fixed and verified** (Iris's follow-up the
|
||||
same day: "if I swipe over the input bar it brings up the
|
||||
keyboard... scrolling should be pinned"). `attr.rs`'s `Selector`/
|
||||
`Selectable` registered `CursorSense::click_or_drag()`, which
|
||||
calls `select()` -- and so grants focus and requests the IME --
|
||||
on the *first* frame of any press, before it is known whether the
|
||||
gesture will end up a tap or a drag. Rewritten around a shared
|
||||
`on_press` dispatcher over `PressStart`/`Pressing`/`PressEnd`: a
|
||||
field that is **already** focused behaves exactly as before
|
||||
(every frame updates the selection, so dragging inside a focused
|
||||
field to select text still works); a field that is **not**
|
||||
focused records where the press began (`TextEdit::press_origin`,
|
||||
new field) and only grants focus on `PressEnd` if no intervening
|
||||
frame crossed `sense::DRAG_SLOP` -- a drag recognised early simply
|
||||
clears the pending tap and does nothing further, so it is never
|
||||
consumed and whatever is behind the field still sees every frame
|
||||
of it. New `FocusHost::is_focused` (both platform impls) is what
|
||||
lets `on_press` tell the two cases apart. **Verified on the
|
||||
emulator**: `dumpsys input_method`'s `mInputShown` reads `false`
|
||||
after a `swipe` gesture starting on the composer bar (`ui-trace`
|
||||
confirms the field's own box never moved, i.e. no keyboard-driven
|
||||
resize happened), and reads `true` after an ordinary `tap` on the
|
||||
same field. **Coordination note**: a concurrent pass is moving
|
||||
drag arbitration into `sense.rs` behind a new `Drop` event: this
|
||||
fix touches only `attr.rs` (new `press_track`/`on_press`) and
|
||||
`iris/src/widget/text/edit.rs` (the new `press_origin` field), not
|
||||
`sense.rs` itself, so it should merge cleanly, but the next agent
|
||||
through here should check whether `Selector`/`Selectable`'s
|
||||
`Pressing`-frame delivery still arrives the way this code assumes
|
||||
once that lands.
|
||||
|
||||
**Checks this pass**: `cargo fmt --all` clean, `cargo clippy
|
||||
--workspace --all-targets` and `cargo ndk -t x86_64 -P 26 clippy
|
||||
--features "transcript-screen bench force-gles"` both zero warnings
|
||||
beyond the pre-existing `tabs-ui` unused-dependency notice, `cargo
|
||||
test --workspace` all passing (new tests: four in `edit.rs`, one in
|
||||
`layout_tests.rs`). **Not done**: item 2's root cause; a real
|
||||
before/after screenshot pair for item 3 (blocked on item 2); anything
|
||||
on Vulkan or the real phone.
|
||||
|
||||
- [ ] **P1 — session screen parity.** History paging backward (with the
|
||||
page-boundary healing `client-core` does not have yet, below),
|
||||
`TranscriptSource`-backed cache/server stitching, jump-to-latest,
|
||||
|
||||
Reference in new issue
Block a user