diff --git a/AGENTS.md b/AGENTS.md index 540b74e..56bb52c 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -405,12 +405,19 @@ Each exists because something was invisible without it. transcript -- every later pass over the same rows is p99 0.26ms. A **streamed event** is, and it is not where it looks: folding the event is 0.35ms and applying the diff to the widget tree is 0.41ms, while the - *frame* is 3.86ms here and 9.5ms on Iris's phone. 401 streamed events - move the item count from 652 to 654, so almost every one is a delta - into the same row -- the cost is re-laying out and re-shaping one - growing message on every delta, not the fold. (The fold was the + *frame* is 3.86ms here and 9.5ms on Iris's phone. (The fold was the hypothesis, from `foldEvent`'s Compose lesson under "Things that have - bitten"; measuring it is what ruled it out.) + bitten"; measuring it is what ruled it out.) That frame is one + `TextBuffer::shape` of the block a delta landed in, and **the fixture's + is 14,888 characters in a single block** -- against a largest-ever + 1,580 across 7,706 blocks of real replies. So the stream phase's number + is a property of the fixture, not of streaming; docs/RUST.md's + "Incremental text" has the measurements and why parley cannot help. + + The last two runs (`where_a_streamed_deltas_cost_is`, + `what_the_fixture_streams`) exist to keep that answerable: what a delta + costs to re-split and re-compare, and what the fixture actually + streams. - **The emulator is a GLES rig, deliberately** (Iris, 2026-09-08; docs/RUST.md). Its guest has no hardware Vulkan -- only SwiftShader in software -- while its GLES *is* the host's real GPU through virgl at diff --git a/app-rust/tests/frame_profile.rs b/app-rust/tests/frame_profile.rs index 3ac485f..3051096 100644 --- a/app-rust/tests/frame_profile.rs +++ b/app-rust/tests/frame_profile.rs @@ -188,3 +188,162 @@ fn what_a_streamed_event_costs() { summarise("apply", &apply); summarise("frame", &frame); } + +/// What re-shaping a *growing* message costs, isolated from everything +/// else a frame does -- the measurement that decides whether an +/// incremental-text design would pay for itself (Iris, 2026-09-09: +/// "we should definitely investigate incremental text rendering"). +/// +/// Grows one text buffer a delta at a time, the way a streamed reply +/// grows one row, and reports what `TextBuffer::shape` costs at each +/// length. Linear per-delta cost means the total over a reply is +/// quadratic in its length, which is the thing an incremental shaper +/// would remove. +#[test] +#[ignore] +fn what_reshaping_a_growing_message_costs() { + use iris::prelude::*; + + let mut h = Harness::new(phone_size(), PHONE_SCALE); + // A reply-sized paragraph built a delta at a time. The deltas are + // words rather than characters because that is what a model streams. + const DELTA: &str = "the quick brown fox jumps over the lazy dog "; + let attrs = TextAttrs::default(); + let width = Some(phone_size().x); + + let mut buffer = iris::prelude::TextBuffer::new_empty(); + let mut text = String::new(); + let mut per_delta = Vec::new(); + let mut total = Duration::ZERO; + for n in 1..=400 { + text.push_str(DELTA); + buffer.set_text(text.clone()); + let at = Instant::now(); + buffer.shape(&mut h.rsc.ui.text, &attrs, width, PHONE_SCALE); + let took = at.elapsed(); + per_delta.push(took); + total += took; + if n % 100 == 0 { + println!( + " after {n:>3} deltas ({:>6} chars): this reshape {:?}", + text.len(), + took + ); + } + } + println!( + " 400 deltas: {:?} of shaping in total, {:?} per delta at the end", + total, + per_delta.last().unwrap() + ); + summarise("reshape", &per_delta); + + // The same measurement at the sizes real replies actually reach. + // Measured 2026-09-09 over 7,706 top-level blocks from 3,675 real + // assistant messages on this machine: p50 147 chars, p90 449, p99 + // 836, largest 1,580, and *nothing* above 4,000. The bench fixture's + // streamed message is one 14,888-character block, which is 9x the + // largest real one -- so the sizes below are what a live reshape + // actually costs and the run above is what the benchmark measures. + println!(" at the sizes real replies reach:"); + for chars in [147usize, 449, 836, 1580] { + let mut sample = String::new(); + while sample.len() < chars { + sample.push_str(DELTA); + } + sample.truncate(chars); + let mut buffer = iris::prelude::TextBuffer::new(sample); + let at = Instant::now(); + buffer.shape(&mut h.rsc.ui.text, &attrs, width, PHONE_SCALE); + println!(" {chars:>5} chars: {:?}", at.elapsed()); + } +} + +/// Where a streamed delta's cost actually is, given that `RowBlocks:: +/// apply_delta` already re-shapes only the block the delta landed in. +/// Three candidates, all of which scale with the *whole* message rather +/// than the delta: re-parsing the markdown to find the blocks, comparing +/// them against the ones already drawn, and re-shaping the last block. +#[test] +#[ignore] +fn where_a_streamed_deltas_cost_is() { + use ai_app::client::markdown_blocks::{common_prefix, split_blocks}; + + // A reply with real block structure -- paragraphs separated by blank + // lines, the way a model writes -- so the last block is one paragraph + // rather than the whole message. + const SENTENCE: &str = "The quick brown fox jumps over the lazy dog. "; + let mut src = String::new(); + let mut blocks = Vec::new(); + + let mut split = Vec::new(); + let mut compare = Vec::new(); + for n in 1..=400 { + src.push_str(SENTENCE); + // A paragraph break every eight deltas, so the trailing block + // stays a normal size and only the message grows. + if n % 8 == 0 { + src.push_str("\n\n"); + } + + let at = Instant::now(); + let new_blocks = split_blocks(&src); + split.push(at.elapsed()); + + let at = Instant::now(); + let _ = common_prefix(&blocks, &new_blocks); + compare.push(at.elapsed()); + + blocks = new_blocks; + if n % 100 == 0 { + println!( + " after {n:>3} deltas ({:>6} chars, {} blocks): split {:?} compare {:?}", + src.len(), + blocks.len(), + split[n - 1], + compare[n - 1], + ); + } + } + summarise("split_blocks", &split); + summarise("common_prefix", &compare); + let total: Duration = split.iter().chain(compare.iter()).sum(); + println!(" 400 deltas: {total:?} in block-splitting and comparison alone"); +} + +/// What the bench fixture's streamed tail actually is, since the cost of +/// a delta depends entirely on how big the block it lands in gets. +#[test] +#[ignore] +fn what_the_fixture_streams() { + use ai_app::client::markdown_blocks::split_blocks; + + let mut h = Harness::new(phone_size(), PHONE_SCALE); + let opened = ai_app::ui::fixture::open(&mut h.rsc, &mut h.state).expect("the fixture folds"); + let mut items = opened.items; + let before = items.len(); + for event in &opened.stream_tail { + items = ai_app::client::transcript_fold::fold_event(&items, event); + } + println!("{} items -> {}", before, items.len()); + // The last few items are where the stream landed. Only the message + // variants matter -- those are what a delta appends to. + use ai_app::client::transcript_fold::TranscriptItem; + for item in items.iter().rev().take(4) { + let (kind, text) = match item { + TranscriptItem::AssistantMsg { text, .. } => ("AssistantMsg", text.clone()), + TranscriptItem::UserMsg { text, .. } => ("UserMsg", text.clone()), + TranscriptItem::ToolRun { input, output, .. } => { + ("ToolRun", format!("{input}{output}")) + } + _ => ("other", String::new()), + }; + let blocks = split_blocks(&text); + println!( + " tail {kind}: {} chars in {} blocks; longest block {} chars", + text.len(), + blocks.len(), + blocks.iter().map(|b| b.source.len()).max().unwrap_or(0), + ); + } +} diff --git a/docs/RUST.md b/docs/RUST.md index c39efe9..b4e2099 100644 --- a/docs/RUST.md +++ b/docs/RUST.md @@ -528,10 +528,54 @@ every delta. `fold_event`'s `items.to_vec()` per event was the hypothesis -- it is the exact shape of the Compose lesson in AGENTS.md's "Things that have -bitten" -- and measuring it is what ruled it out. **Not yet designed**: -making a row's text append incrementally rather than reshape touches how -`TranscriptRow` holds its shaped text, which is load-bearing enough to -raise before building. +bitten" -- and measuring it is what ruled it out. + +### Incremental text: parley cannot, and it turns out not to matter (2026-09-09) + +Iris asked to investigate incremental text rendering and hoped parley +supported it. **It does not, by design.** The crate's own docs: a +`Layout` "supports re-linebreaking and re-aligning many times... but if +the text content or the styles applied to that content change then a new +`Layout` must be created". Its `LruCache` caches harfrust's per-font +shaper data, instance and plan -- not shaped runs -- and its own +`PlainEditor::update_layout` rebuilds the whole layout from the whole +buffer on **every keystroke**. So there is nothing to adopt, and adding +it would be upstream work in parley. + +**And the app already does the thing incremental layout would buy.** +`RowBlocks::apply_delta` keeps one `TextEdit` per top-level markdown +block and re-shapes only the block a delta landed in; re-splitting the +markdown to find that block is 18µs at 18,000 characters and comparing +the blocks is 470ns. Neither is the cost. + +**The 9.5ms is a bench-fixture artifact.** Measured with +`frame_profile.rs`: + +- Re-shaping a block is linear in its length -- ~0.23ms per 1,000 + characters on this desktop, so a message grown to 17,600 characters + costs 4.1ms on its *last* delta and 842ms of shaping over the whole + reply. +- The fixture's streamed message is **14,888 characters in one block** -- + a synthetic run-on paragraph with no blank line in it, so every delta + reshapes all of it. That is the whole of the frame: 3.5ms of the + measured 3.86ms. +- Real replies are not like that. Over **7,706 top-level blocks from + 3,675 real assistant messages** on this machine (block lengths only; + no content left the machine): p50 147 characters, p90 449, p99 836, + largest 1,580, and **nothing above 4,000**. Code fences are smaller + still -- 170 of them, p50 126, largest 589. +- At those sizes a live reshape is 48µs (p50), 208µs (p99) and 372µs + (the largest block ever seen), or roughly 0.12-0.93ms on the phone. + Comfortably inside a 120Hz budget, with no incremental anything. + +**So: do not build incremental text.** What is worth doing instead is +giving the fixture's streamed message the paragraph structure a real +reply has, so the stream phase measures something that happens. Both +apps read the same fixture, so the Compose/iris comparison stays sound, +but numbers from before and after the change are not comparable to each +other. Whether to keep a pathological block as a *labelled* stress case +alongside it is Iris's call -- the danger of the current one is only that +its number reads as "streaming costs 9.5ms" when nothing does. ### The Android release profile is `opt-level = 3`, not `"s"` (2026-09-09)