The bench fixture streams a reply shaped like a real one, and keeps the run-on as stress

Iris, on the two findings from the incremental-text investigation:
"let's switch to new lines for the test, and also let's keep the single
line around for stress + could be something to try to optimize later."

The streamed tail now takes a blank line every 4-12 deltas, so it is 53
markdown blocks with a longest of 502 characters instead of one block of
14,888 -- against a measured p50 of 147 and a largest-ever 1,580 over
7,706 blocks of real assistant messages. Layer 1's streaming frame went
from p50 3.86ms / p90 8.65ms / worst 10.95ms to p50 2.20 / p90 5.90 /
worst 8.78.

The run-on message is kept as the first two backlog events, 14,824
characters in one block, just under text_cap's 16 KiB so it draws in
full. The *streaming* pathology stays in frame_profile.rs rather than the
fixture: it needs a growing block, and iterating on it there costs a
second instead of a two-minute phone run.

Adding it is purely additive -- the random state is saved and restored
around those two events, so every other backlog event is byte-identical.
That is not tidiness: the first attempt shifted the backlog and broke
`a_long_press_and_drag_selects_text`, which replays a real recording at
(300, 1000) and needs the content it was recorded against to still be
there. BACKLOG_COUNT is 3202 now, in generate.py, fixture.rs and
BenchFixture.kt, which split the file by line index.

And the answer to Iris's question, which the code already had: the newest
message does *not* cap. `build_row`'s `cap` is false for the live tail
because a row that grew while capped would appear to stop growing, and a
reply growing past the cap is never caught either since it grows through
apply_delta. So a streamed block's shaping cost has no ceiling -- ~29ms
per delta at 50k characters, ~58ms at 100k.

Recorded but not chased: the emulator's `stream: build p50` did not move
(10.4 -> 10.5ms) while layer 1's frame nearly halved, so most of a
streaming frame on a GPU path is the whole-arena primitive re-upload
layer 1 never performs -- 11,568 primitives rewritten per delta, with the
fling phase as the control at 0.4ms for the same primitives moved
through move_offsets.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

app-rust/src/ui/fixture.rs

@@ -25,7 +25,7 @@ use iris::prelude::*;
 /// `BenchFixture.kt`'s identical constant by hand -- both read the same
 /// checked-in file, so a mismatch would only mean the two apps' bench
 /// builds open a different split of it, not a wrong-vs-right answer.
-pub const BACKLOG_COUNT: usize = 3200;
+pub const BACKLOG_COUNT: usize = 3202;
 
 const FIXTURE_JSONL: &str = include_str!("../../../app/bench-fixture/assets/transcript.jsonl");
 

app-rust/tests/frame_profile.rs

@@ -154,13 +154,27 @@ fn what_a_streamed_event_costs() {
     let mut fold = Vec::new();
     let mut apply = Vec::new();
     let mut frame = Vec::new();
+    // Split by whether the delta started a new markdown block, since that
+    // is the delta that builds a widget rather than re-shaping one.
+    let mut frame_same_block = Vec::new();
+    let mut frame_new_block = Vec::new();
     let mut t = PHONE_FRAME_MS;
+    let block_count = |items: &[ai_app::client::transcript_fold::TranscriptItem]| {
+        use ai_app::client::markdown_blocks::split_blocks;
+        use ai_app::client::transcript_fold::TranscriptItem;
+        match items.last() {
+            Some(TranscriptItem::AssistantMsg { text, .. }) => split_blocks(text).len(),
+            _ => 0,
+        }
+    };
     for (n, event) in opened.stream_tail.iter().enumerate() {
-        let at = Instant::now();
+        let blocks_before = block_count(&items);
         let old = items.clone();
+        let at = Instant::now();
         let folded = ai_app::client::transcript_fold::fold_event(&items, event);
         fold.push(at.elapsed());
         items = folded;
+        let added_block = block_count(&items) > blocks_before;
 
         let at = Instant::now();
         opened.screen.apply(&mut h.rsc, &old, &items);
@@ -169,7 +183,13 @@ fn what_a_streamed_event_costs() {
         t += PHONE_FRAME_MS;
         let at = Instant::now();
         h.frame(t);
-        frame.push(at.elapsed());
+        let took = at.elapsed();
+        frame.push(took);
+        if added_block {
+            frame_new_block.push(took);
+        } else {
+            frame_same_block.push(took);
+        }
 
         // Where the cost sits as the transcript grows -- one line early,
         // one late, is enough to see a per-event cost from a quadratic.
@@ -187,6 +207,18 @@ fn what_a_streamed_event_costs() {
     summarise("fold", &fold);
     summarise("apply", &apply);
     summarise("frame", &frame);
+    summarise("frame/same-block", &frame_same_block);
+    summarise("frame/new-block", &frame_new_block);
+    // What the GPU side has to carry, which layer 1 builds but never
+    // uploads and so cannot time: every primitive is re-uploaded whenever
+    // the arena changes, and the buffer is recreated when its length does
+    // (`ArrBuf::update`). Splitting the streamed reply into blocks trades
+    // shaping cost for more widgets, so this is the number that says
+    // whether that trade is free on a real GPU path.
+    println!(
+        "  primitives on screen at the end: {}",
+        h.render.active_primitive_count()
+    );
 }
 
 /// What re-shaping a *growing* message costs, isolated from everything
@@ -317,18 +349,41 @@ fn where_a_streamed_deltas_cost_is() {
 #[ignore]
 fn what_the_fixture_streams() {
     use ai_app::client::markdown_blocks::split_blocks;
+    use ai_app::client::transcript_fold::TranscriptItem;
 
     let mut h = Harness::new(phone_size(), PHONE_SCALE);
     let opened = ai_app::ui::fixture::open(&mut h.rsc, &mut h.state).expect("the fixture folds");
+    let backlog = opened.items.clone();
     let mut items = opened.items;
     let before = items.len();
     for event in &opened.stream_tail {
         items = ai_app::client::transcript_fold::fold_event(&items, event);
     }
     println!("{} items -> {}", before, items.len());
+    // The stress message the generator plants in the backlog: one block,
+    // no blank line, just under `text_cap`'s MESSAGE_BYTES.
+    let biggest = backlog
+        .iter()
+        .filter_map(|item| match item {
+            TranscriptItem::AssistantMsg { text, .. } => Some(text),
+            _ => None,
+        })
+        .map(|text| {
+            let blocks = split_blocks(text);
+            (
+                text.len(),
+                blocks.len(),
+                blocks.iter().map(|b| b.source.len()).max().unwrap_or(0),
+            )
+        })
+        .max_by_key(|(_, _, longest)| *longest);
+    if let Some((chars, blocks, longest)) = biggest {
+        println!(
+            "  backlog's largest single block: {longest} chars (in a {chars}-char message of {blocks} blocks)"
+        );
+    }
     // The last few items are where the stream landed. Only the message
     // variants matter -- those are what a delta appends to.
-    use ai_app::client::transcript_fold::TranscriptItem;
     for item in items.iter().rev().take(4) {
         let (kind, text) = match item {
             TranscriptItem::AssistantMsg { text, .. } => ("AssistantMsg", text.clone()),

app/androidApp/src/main/kotlin/com/example/aiapp/BenchFixture.kt

@@ -4,8 +4,8 @@ import android.content.Context
 import java.util.concurrent.CopyOnWriteArrayList
 
 /**
- * P0's benchmark gate (see docs/RUST.md and the 2026-09-05 decision): an in-process
- * fake of the backend, so the `bench` build type can drive a real session screen -- the real
+ * P0's benchmark gate (see docs/RUST.md and the 2026-09-05 decision): an in-process fake of the
+ * backend, so the `bench` build type can drive a real session screen -- the real
  * [TranscriptSource], the real fold, the real paging -- with no server and no network permission.
  *
  * Only ever installed when [BuildConfig.FIXTURE_MODE] is true (see [MainActivity]); everything else
@@ -24,7 +24,7 @@ object BenchFixture {
     const val FIXTURE_PORT = 1
 
     /** How many of the fixture's events are the opening backlog; see bench-fixture/README.md. */
-    private const val BACKLOG_COUNT = 3200
+    private const val BACKLOG_COUNT = 3202
 
     val settings = ServerSettings(FIXTURE_HOST, FIXTURE_PORT, "bench")
 

app/bench-fixture/README.md

@@ -12,17 +12,37 @@ script and README, because the Compose `bench` build type points its own asset s
 at `assets/` (`app/androidApp/build.gradle.kts`'s `sourceSets { getByName("bench") }`), and a Python
 script and a markdown file have no business inside an APK:
 
-- `transcript.jsonl` -- 3,601 events. The first 3,200 (`BACKLOG_COUNT`) are the scrolled-back
+- `transcript.jsonl` -- 3,603 events. The first 3,202 (`BACKLOG_COUNT`) are the scrolled-back
   history the benchmark opens with: user turns, tool calls with kilobyte-scale input/output,
   assistant replies built from headings, bold/italic/inline code, a link, fenced code blocks that
   rotate through rust/kotlin/python/sh/json/toml, a markdown table, two embedded images, and
   periodic `usageDelta`/`compacted` events. The remaining 400 (`STREAM_COUNT`) are not part of the
   opening window -- both bench harnesses replay them at a fixed rate (20/s) through the same live
   fold path a real SSE reply arrives on, which is P0's "streaming phase."
+
+  **The streamed reply has a blank line every few deltas** (2026-09-09), so the markdown block a
+  delta lands in stays the size a real reply's blocks are -- 53 blocks, longest 502 characters,
+  against a measured p50 of 147 and a largest-ever 1,580 over 7,706 blocks of real assistant
+  messages. It used to be one run-on 14,888-character block, and since a row re-shapes the block a
+  delta lands in, every delta re-shaped all of it: quadratic in the reply's length, and 9.5ms of
+  frame time on a phone spent on a shape that does not occur. docs/RUST.md's "Incremental text"
+  has the measurements.
+
+  **The run-on message is kept**, as the first two events of the backlog: 14,824 characters in a
+  single block, just under `text_cap`'s 16 KiB `MESSAGE_BYTES` so it draws in full rather than
+  behind a "Show all". It is deliberately *not* streamed -- the repeated-reshape pathology needs a
+  growing block, and that lives in `app-rust/tests/frame_profile.rs` where it can be iterated on
+  in a second rather than in a two-minute phone run. It is emitted with the random state saved and
+  restored around it, so adding it left every other backlog event byte-identical; that is what
+  keeps `phone_screen.rs`'s recorded gestures landing on the content they were recorded against.
 - `bench1.png`, `bench2.png` -- tiny (8x8) flat-colour PNGs, base64-free on disk but served the
   same way a real attachment is (`GET /sessions/{id}/files/{name}`), referenced by the two
   `"type":"image"` events in the transcript.
 
+`BACKLOG_COUNT` lives in three places and moves in all of them or none: here, `app-rust/src/ui/
+fixture.rs` and `BenchFixture.kt`. The split is by line index, so a stale copy makes that app open
+a different half of the file.
+
 Regenerate after changing the shape (a new event type, a different backlog/stream split) with
 `./generate.py`, and commit the result -- it is checked in rather than generated at build time so
 both apps' bench builds embed the identical bytes without needing this script at build time.

app/bench-fixture/generate.py

@@ -26,8 +26,40 @@ import zlib
 from pathlib import Path
 
 SEED = 20260905
-BACKLOG_COUNT = 3200
+# The stress message below, two events, added to the 3,200 the backlog used to be. Both apps
+# hardcode this to split the file by line index (`fixture.rs`'s `BACKLOG_COUNT`,
+# `BenchFixture.kt`'s), so it moves in three places or none.
+STRESS_EVENTS = 2
+BACKLOG_COUNT = 3200 + STRESS_EVENTS
 STREAM_COUNT = 400
+
+# How many deltas fold into one markdown block of the streamed reply.
+#
+# **The blank line between them is the point** (2026-09-09). Until this run the streaming tail
+# appended `paragraph(5) + " "` STREAM_COUNT times with no blank line anywhere, so all 400 deltas
+# folded into a *single* 14,888-character block -- and a row re-shapes the block a delta lands in
+# (`RowBlocks::apply_delta`), so every delta re-shaped the whole thing. That is quadratic in the
+# reply's length, and it put 9.5ms of frame time on Iris's phone into a shape that does not occur:
+# measured over 7,706 top-level blocks from 3,675 real assistant messages, block length is p50 147
+# characters, p90 449, p99 836, largest 1,580, nothing above 4,000. `paragraph(5)` is ~35
+# characters, so 4-12 of them per block lands in that range.
+#
+# The run-on version is kept, as STRESS_CHARS below -- Iris, 2026-09-09: "let's switch to new
+# lines for the test, and also let's keep the single line around for stress".
+DELTAS_PER_BLOCK = (4, 12)
+
+# One deliberately pathological message in the backlog: a single markdown block with no blank line
+# in it, the shape the streaming tail used to have. Sized just under `text_cap`'s MESSAGE_BYTES
+# (16 KiB) so it is drawn in full rather than behind a "Show all" -- which makes it a genuine
+# stress for one-shot shaping and puts a row right on the cap boundary, where nothing else is.
+#
+# It is *not* streamed: the repeated-reshape pathology needs a growing block, and that lives in
+# `app-rust/tests/frame_profile.rs`'s `what_reshaping_a_growing_message_costs`, where it can be
+# iterated on in a second rather than in a two-minute phone run. Note that the cap would not save
+# a real one anyway -- `row::build_row`'s `cap` is deliberately `false` for the live tail, because
+# a row that grew while capped would appear to stop growing, so a streamed block's shaping cost
+# has no ceiling.
+STRESS_CHARS = 14_800
 HERE = Path(__file__).resolve().parent / "assets"
 
 random.seed(SEED)
@@ -122,6 +154,26 @@ def main():
     emit("status", state="running")
     emit("settings", model="bench-model", permissionMode="auto")
 
+    # The stress message, emitted first and with the random state put back afterwards, so that
+    # adding it is **purely additive**: every turn the loop below generates is byte-identical to
+    # what it generated before this existed, and only the streaming tail changed. That matters
+    # because the fixture's opening view is what `phone_screen.rs`'s recorded gestures press --
+    # `a_long_press_and_drag_selects_text` drives a real recording at (300, 1000) and fails the
+    # moment different content lands under it. `ts` still shifts by the two draws, which nothing
+    # reads.
+    rng_state = random.getstate()
+    emit(
+        "userMessage",
+        text="And the pathological one: a reply with no paragraph break in it.",
+        id=None,
+        attachments=[],
+    )
+    run_on = ""
+    while len(run_on) < STRESS_CHARS:
+        run_on += paragraph(5) + " "
+    emit("assistantText", delta=run_on.rstrip())
+    random.setstate(rng_state)
+
     image_refs = []
     turn = 0
     while seq <= BACKLOG_COUNT:
@@ -167,11 +219,21 @@ def main():
                 trigger="auto",
             )
 
-    # The streaming-phase tail: one long reply, built entirely from text deltas, the shape a
-    # bench harness replays at a fixed events/sec through the live fold path.
+    # The streaming-phase tail: one reply built entirely from text deltas, the shape a bench
+    # harness replays at a fixed events/sec through the live fold path -- with a blank line every
+    # few deltas, so the block a delta lands in stays the size a real reply's blocks are. See
+    # DELTAS_PER_BLOCK for what the alternative measured.
     emit("userMessage", text="One more, streamed live for the benchmark's timing phase.", id=None, attachments=[])
+    until_break = random.randint(*DELTAS_PER_BLOCK)
     while seq <= BACKLOG_COUNT + STREAM_COUNT:
-        emit("assistantText", delta=paragraph(5) + " ")
+        until_break -= 1
+        if until_break <= 0:
+            # The break rides on the last delta of the block rather than being an event of its
+            # own, so STREAM_COUNT still counts deltas a reader sees text arrive from.
+            emit("assistantText", delta=paragraph(5) + "\n\n")
+            until_break = random.randint(*DELTAS_PER_BLOCK)
+        else:
+            emit("assistantText", delta=paragraph(5) + " ")
     emit("status", state="idle")
 
     (HERE / "transcript.jsonl").write_text("\n".join(lines) + "\n")

docs/RUST.md

@@ -568,14 +568,72 @@ the blocks is 470ns. Neither is the cost.
   (the largest block ever seen), or roughly 0.12-0.93ms on the phone.
   Comfortably inside a 120Hz budget, with no incremental anything.
 
-**So: do not build incremental text.** What is worth doing instead is
-giving the fixture's streamed message the paragraph structure a real
-reply has, so the stream phase measures something that happens. Both
-apps read the same fixture, so the Compose/iris comparison stays sound,
-but numbers from before and after the change are not comparable to each
-other. Whether to keep a pathological block as a *labelled* stress case
-alongside it is Iris's call -- the danger of the current one is only that
-its number reads as "streaming costs 9.5ms" when nothing does.
+**So: incremental text is not worth building** -- and Iris agreed, with
+the fixture changed instead (2026-09-09: *"let's switch to new lines for
+the test, and also let's keep the single line around for stress + could
+be something to try to optimize later"*). What landed:
+
+- The streamed reply gets a blank line every 4-12 deltas, so it is 53
+  blocks with a longest of 502 characters instead of one of 14,888. The
+  streaming frame went from p50 3.86ms / p90 8.65ms / worst 10.95ms to
+  **p50 2.20ms / p90 5.90ms / worst 8.78ms** here.
+- The run-on message is kept as the first two backlog events, sized just
+  under `text_cap`'s 16 KiB so it draws in full. The *streaming*
+  pathology is kept in `frame_profile.rs` instead of the fixture, because
+  it needs a growing block and iterating on it there costs a second
+  rather than a two-minute phone run.
+- Adding it is **purely additive**: the random state is saved and
+  restored around those two events, so every other backlog event is
+  byte-identical. That is not cosmetic -- `phone_screen.rs`'s
+  `a_long_press_and_drag_selects_text` replays a real recording at
+  (300, 1000) and failed the first time round, when the insertion shifted
+  what was under it.
+- `BACKLOG_COUNT` is 3202 now, in `generate.py`, `fixture.rs` and
+  `BenchFixture.kt`. The split is by line index, so a stale copy opens a
+  different half of the file.
+
+**The cap does not save a streamed reply, and this is worth knowing
+before optimising anything here.** Iris asked whether the newest message
+caps: it does not, deliberately -- `row::build_row`'s `cap` is `false`
+for the live tail because a row that grew while capped would appear to
+stop growing, and a reply that grows *past* the cap never gets caught
+either, since it grows through `apply_delta`. So a streamed block's
+shaping cost has no ceiling: at the measured ~0.23ms per 1,000
+characters (about 2.5x that on the phone), a 50,000-character block
+would be ~29ms per delta and a 100,000-character one ~58ms. Real replies
+do not do this, which is why it is not urgent; nothing *stops* one doing
+it, which is why the stress case is kept.
+
+**What the remaining streaming cost is, and is not.** With realistic
+blocks the reshape is no longer the cost: layer 1's frame went to p50
+2.20ms, spread over frames that added a block (p50 3.56ms, 56 of 401)
+and frames that did not (p50 1.94ms). Folding is 0.12ms and applying the
+diff 0.35ms.
+
+**But the emulator's `stream: build p50` did not move -- 10.4ms before
+the fixture change, 10.5ms after** -- while layer 1's CPU frame nearly
+halved. So most of a streaming frame on a real GPU path is something
+layer 1 builds and never uploads, and therefore cannot time. The
+candidate, and the arithmetic behind it:
+
+- The screen holds **11,568 primitives** by the end of the stream phase.
+- `UiRenderNode::update` re-uploads the *entire* instance and primitive
+  arenas whenever `primitives.updated` is set, which a text change sets
+  every delta -- about 370 KB per delta at 32 bytes an instance, before
+  the primitive data itself. `ArrBuf::update` also **recreates the
+  buffer** whenever its length changes, which adding glyphs does on
+  nearly every delta, and a recreated buffer means a fresh bind group
+  too.
+- The fling phase is the control that makes this convincing: it moves
+  the same 11,568 primitives every frame through `move_offsets` -- a
+  small buffer, no arena rewrite -- and its `build p50` is **0.4ms**
+  against streaming's 10.5ms, on the same screen and the same content.
+
+So the next thing to look at for streaming is **uploading only what
+changed** rather than the whole arena, not anything about text. Splitting
+the reply into blocks was still right -- it is what makes the fixture
+representative, and it halved the CPU half -- but it was never going to
+move this, and it slightly increases the primitive count.
 
 ### The Android release profile is `opt-level = 3`, not `"s"` (2026-09-09)