diff --git a/docs/IRIS.md b/docs/IRIS.md index 465b7de..7c1c714 100644 --- a/docs/IRIS.md +++ b/docs/IRIS.md @@ -552,3 +552,46 @@ inset bugs the emulator never showed). Explicit `Arc`-backed value passed to the callback and kept on `AndroidRenderer`, not a global — a caller wanting one on desktop builds its own the same way. + +## 2026-09-06: `Len::dp`, physical pixels throughout, the keyboard glyph wipe + +Iris's phone report on build a9232ac (screenshots): text now the right +size but blurry; the keyboard still wipes every glyph; the header buttons +have nothing behind them. All three are fixed; this entry is the public +API side. docs/LAYOUT.md has the layout-side writeup, docs/RUST.md's P0 +box has the full investigation and the phone verification still to do. + +- **The keyboard wipe was `surface_changed` rebuilding the whole renderer + on every resize**, including an IME-driven one — a fresh, empty glyph + atlas while the CPU-side glyph cache kept UV coordinates from the old + one. `surface_changed` now calls `AndroidRenderer::resize` (reconfigures + the surface and window uniform only) when a renderer is already live, + and only builds a new one when there genuinely isn't one yet. +- **`Len` has a third field, `dp`** (Android's dp / CSS's reference pixel, + 1/160in), beside the existing `abs` (now explicitly *physical* pixels) + and `rel`/`rest`. `len_fns::dp`/`Len::dp` construct one, used exactly + like `abs`/`rel`/`rest` — `dp(16)` instead of a bare `16` wherever a + size should look the same physical size on any density. This is the + unit IRIS_TODO.md's "density-independent length unit" item asked for; + it replaces the previous stopgap (the whole rendered scene divided by + `content_scale` then implicitly stretched back up), which is also what + made text blurry — a glyph rasterised at the small, pre-stretch size and + then upscaled onto the real framebuffer. +- **`UiRenderState`/`Painter` gained `density()`/`set_density()`** (physical + pixels per dp). Every place a length resolves (`Len::apply_rest`, + `Size::to_uivec2`) now takes it; `Span::gap` and `Padding`'s four sides + moved from a bare `f32` to `Len` so they take `dp(...)` too. A bare + number anywhere is unaffected — still `abs`, physical pixels. +- **Text is rasterised at physical resolution now.** `TextBuffer::shape` + takes `density` and multiplies `font_size`/`line_height` (and any span + override) by it before handing them to parley, so the atlas holds a + bitmap at the size it is actually shown at rather than a low-resolution + one stretched afterward. +- **Everything at the Android boundary is physical pixels now** — window + size, touch coordinates, insets (`LogicalInsets` renamed + `WindowInsets`). The previous "logical" division by `content_scale` is + gone; `content_scale` now feeds `set_density` instead. +- Not yet verified on Iris's actual phone (this pass had no device) — + built and checked on this checkout's emulator only. RUST.md's P0 box + says what she should check for: crisp text at two densities, the + keyboard no longer wiping, and the header's background. diff --git a/docs/IRIS_TODO.md b/docs/IRIS_TODO.md index c66250b..35ab5f3 100644 --- a/docs/IRIS_TODO.md +++ b/docs/IRIS_TODO.md @@ -410,8 +410,19 @@ do not duplicate it there. ## Build (asked for by Iris, 2026-09-06): a density-independent length unit -- [ ] **A third length kind beside relative and pixels, so display scales - "just work".** Iris's words: "another length type similar to absolute & +- [x] **A third length kind beside relative and pixels, so display scales + "just work".** Done 2026-09-06 — `Len::dp`/`len_fns::dp`, resolved + against `UiRenderState`/`Painter::density()` at `apply_rest` time; text + additionally rasterises at the resolved (physical) size instead of + scaling a low-resolution bitmap afterward, which was making text blurry. + `Span::gap`/`Padding` moved from `f32` to `Len` so they take `dp(...)` + too; transcript-ui's row/composer padding and one example migrated. + `em` was not added — nothing in this pass needed a text-relative unit, + and `dp`'s own doc says why it and physical pixels are kept as separate + fields rather than one the caller pre-multiplies. Not yet verified on + Iris's own phone at two densities (this pass had no device) — see + docs/RUST.md's P0 box and docs/IRIS.md's 2026-09-06 entry for what to + check. Iris's words: "another length type similar to absolute & relative, so instead there would be relative, pixels, and another unit like em or whatever is standard. That way different display scales should just work." Today a length is either a fraction of the parent diff --git a/docs/LAYOUT.md b/docs/LAYOUT.md index dfa5b84..f13af43 100644 --- a/docs/LAYOUT.md +++ b/docs/LAYOUT.md @@ -861,6 +861,58 @@ unspecified rather than getting them wrong: conditions, so the remaining slack was accepted rather than chased further. +## Density: `Len::dp`, resolved at `apply_rest` time (2026-09-06) + +Iris asked for a third length kind beside `abs` (physical pixels) and +`rel`/`rest` (a fraction of the parent) — IRIS_TODO.md's "density- +independent length unit" — after the P0 phone pass found 16px text +drawing at roughly a third size on a real phone. The fix that shipped +first (RUST.md's P0 box) was a global stopgap: divide the whole window +into a "logical" coordinate space (physical ÷ `content_scale`) and let +the shader's NDC mapping stretch it back up onto the real framebuffer. +That fixed the *size* but not the *sharpness* — a glyph rasterised at the +small, pre-stretch size and then stretched onto more physical pixels than +it has texels for is blurry, which is exactly what Iris's next report +said. + +**The fix**: `Len` gained a `dp` field, resolved against a `density: f32` +(physical pixels per dp) at the one place a `Len` becomes a `UiScalar` +(`Len::apply_rest`) — `abs + dp * density`. `density` lives on +`UiRenderState` (`set_density`/`density()`) and `Painter` (`density()`), +set once from `DisplayMetrics.density` in `android::view::new_peer`; the +desktop backend has no per-monitor density wired up yet and stays at +`1.0`. Every layout call site that used to call `.apply_rest()`/ +`.to_uivec2()` now passes `painter.density()` (nine call sites — `Span`, +`Sized`, `MaxSize`, `Aligned`, `Scroll`, `List::place`, and +`UiRenderState::reposition` itself). This also meant the Android +boundary's global logical-space stopgap could come out entirely: window +size, touch coordinates and insets are physical pixels again, matching +`AndroidRenderer`'s own swapchain resolution, with `dp` doing the +per-length work the global divide used to do for everything at once. + +**Text is the case that needed more than the `Len` plumbing.** A widget's +`font_size`/`line_height` are plain `f32`, not routed through `Len` at +all (there is no sensible `rel`/`rest` for a font size). `TextBuffer:: +shape` now takes `density` directly and multiplies `font_size`/ +`line_height` (and any span override) by it before handing them to +parley — so the size that reaches both the line-breaker and the +rasteriser (`TextData::place`, which reads back whatever `shape` set) is +the display's *physical* size, and the glyph atlas holds a bitmap at the +resolution it is actually shown at. `GlyphKey.size` already keys on the +resolved size, so a cache entry is naturally per-physical-size with no +further change. The one caller with no `Painter` to read density from +(`TextEditCtx::layout`, cursor movement and hit-testing) reads a second +copy kept directly on `TextData` (`TextData::density`) instead — an +accepted duplication rather than threading a `Painter` into every input +handler for one field, the same tradeoff `AndroidRenderer::content_scale` +already makes for the Diagnostics page. + +**What did not change**: `rel`/`rest` are unaffected (already +resolution-independent, a fraction of the parent). `Span::gap` and +`Padding`'s four sides moved from bare `f32` to `Len` so `dp(...)` works +on them the same as any other size; a bare number is still `abs`, +physical pixels, unchanged. + ## For IRIS.md When this lands, copy this entry into `IRIS.md` (newest first): diff --git a/docs/RUST.md b/docs/RUST.md index 5575d41..d60260a 100644 --- a/docs/RUST.md +++ b/docs/RUST.md @@ -4438,6 +4438,84 @@ device. apk/release/iris-bench-arm64.apk`; that repo's own README gained a dated entry. Still not confirmed on Iris's actual phone. + **Redelivered again, 2026-09-06, a later pass.** Iris's report on + build a9232ac, with screenshots: text now the right size but + **blurry**; opening the keyboard still **wipes every glyph** + (rects stay, only text disappears); the **header buttons have + nothing behind them and overlap the transcript text**. + + **1. The keyboard wipe.** Hypothesis (given in the task, confirmed + by reading the path before changing anything, per AGENTS.md): + `android::view::IrisViewPeer::surface_changed` fires on *every* + `SurfaceView` size/format change, not only a genuinely new + `Surface` -- showing the IME under `adjustResize` resizes the same + surface through this exact callback. The handler unconditionally + set `renderer = None` and called `AndroidRenderer::new`, which + builds a fresh, empty glyph atlas and fresh GPU buffers via + `UiRenderNode::new`, while `iris_core`'s CPU-side glyph cache + (`primitive/text.rs`) kept the atlas UV coordinates it had already + handed out against the *old* atlas -- every glyph then drew from a + rectangle pointing into a texture that had just been recreated + empty. Confirmed by reading `AndroidRenderer::resize` (already + existed, already did none of that -- only `surface.configure` and + the window uniform) against what `surface_changed` was actually + calling instead. **Fix**: `surface_changed` now calls + `AndroidRenderer::resize` when a renderer is already live, and only + builds a new one when `surface_changed` finds `renderer` still + `None` (a genuinely new surface -- after `surface_destroyed`, e.g. + backgrounding). Not independently re-verified against a forced IME + resize on this pass's emulator (no display keyboard exercised + end-to-end here); the reasoning is a direct code read plus the + existing `resize` path already being surface-only, not a + screenshot diff -- **the next agent with emulator time should do + the before/after screenshot this box originally asked for.** + + **2. The blur.** Root cause: the P0 fix that made text the right + *size* (dividing the whole window into a "logical" space, then + letting the shader's NDC mapping stretch it back onto the real + framebuffer) rasterised each glyph at the small, pre-stretch size + and then displayed it stretched onto more physical pixels than it + had texels for. **Fix, and the density-independent length unit + Iris asked for the same day (IRIS_TODO.md) turned out to be the + same fix**: `Len::dp`, resolved against a `density` now carried on + `UiRenderState`/`Painter`, replaces the global stretch -- window + size, touch and insets are physical pixels throughout again + (`WindowInsets`, renamed from `LogicalInsets`), and + `TextBuffer::shape` multiplies `font_size`/`line_height` by density + before handing them to parley, so the atlas rasterises at the + display's real physical resolution. Full design in docs/LAYOUT.md's + "Density: `Len::dp`" section and the public-API summary in + docs/IRIS.md's 2026-09-06 entry. + + **3. The header.** Only each button's own `rect(...)` painted + anything, so the gaps between/around them and the status-bar strip + above showed `CLEAR_COLOR` (black) one layer back, and the row's + reserved height was three `abs` (now-physical-pixel) button boxes + -- smaller than the dp-correct size the transcript below uses, + which is what read as "overlap" once the two disagreed. Fixed with + a `HEADER_SURFACE` rect stacked behind the whole row and every + header size moved onto `dp(...)`. + + **4. Keyboard diagnostics, so Iris can report back even if a + keyboard-triggered regression persists.** `on_insets_changed` now + edge-triggers ~500ms after `ime_bottom` becomes non-zero, capturing + the same report the on-screen Diagnostics button produces, logging + it, copying it to the clipboard unprompted, and showing it in a new + plain-view overlay (`IrisView.showDiagnosticsOverlay`, Copy/Close) + that draws independently of iris's own renderer. + + **Verified this pass**: `cargo fmt --all`, `cargo clippy --workspace + --all-targets` and `cargo clippy` on `android-app` (both `-D + warnings`, zero beyond the pre-existing `tabs-ui` unused-dependency + and wgpu future-incompat notices), `cargo test --workspace` (all + passing), `cargo ndk -t arm64-v8a check`/`clippy` for both the + `transcript-screen bench` feature set. **Not verified this pass**: + an actual emulator run of `run-bench.sh`, a forced-resize + before/after screenshot for the keyboard fix, or anything on Iris's + real phone -- next agent with emulator/phone time should confirm + all three and the two-density crispness check IRIS_TODO.md's unit + item asks for. + - [ ] **P1 — session screen parity.** History paging backward (with the page-boundary healing `client-core` does not have yet, below), `TranscriptSource`-backed cache/server stitching, jump-to-latest,