Incremental text: parley cannot, the app already does it, and the 9.5ms is the fixture
Iris asked to look into incremental text rendering, hoping parley supported it. It does not, by design: a `Layout` re-linebreaks and re-aligns freely but "if the text content or the styles applied to that content change then a new `Layout` must be created", its LRU cache holds harfrust's per-font shaper data rather than shaped runs, and its own `PlainEditor` rebuilds the whole layout from the whole buffer on every keystroke. The app already does what incremental layout would buy: `RowBlocks:: apply_delta` keeps one `TextEdit` per markdown block and re-shapes only the one a delta landed in. Re-splitting the markdown to find it is 18µs at 18,000 characters; comparing the blocks is 470ns. What is left is one `TextBuffer::shape` of that block, linear in its length at ~0.23ms per 1,000 characters here -- and the bench fixture's streamed message is 14,888 characters in a *single* block, a run-on paragraph with no blank line in it, so every delta reshapes all of it. That is 3.5ms of the measured 3.86ms frame. Real replies are not that: across 7,706 top-level blocks from 3,675 real assistant messages on this machine (lengths only, no content copied anywhere), p50 147 characters, p90 449, p99 836, largest 1,580, nothing above 4,000; code fences p50 126, largest 589. At those sizes a reshape is 48µs to 372µs here, roughly 0.12-0.93ms on the phone -- inside a 120Hz budget with no incremental anything. So the recommendation is not to build it, and to give the fixture's streamed message the paragraph structure a real reply has instead. Three runs added to `frame_profile.rs` so none of this is re-derived: what reshaping a growing message costs (including at the sizes real replies reach), where a delta's cost is, and what the fixture actually streams. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
1 parent
9bf714fa2e
commit
43a3a345e4
3 files changed
+219
-9
No files matched your search
@@ -188,3 +188,162 @@ fn what_a_streamed_event_costs() {
|
||||
summarise("apply", &apply);
|
||||
summarise("frame", &frame);
|
||||
}
|
||||
|
||||
/// What re-shaping a *growing* message costs, isolated from everything
|
||||
/// else a frame does -- the measurement that decides whether an
|
||||
/// incremental-text design would pay for itself (Iris, 2026-09-09:
|
||||
/// "we should definitely investigate incremental text rendering").
|
||||
///
|
||||
/// Grows one text buffer a delta at a time, the way a streamed reply
|
||||
/// grows one row, and reports what `TextBuffer::shape` costs at each
|
||||
/// length. Linear per-delta cost means the total over a reply is
|
||||
/// quadratic in its length, which is the thing an incremental shaper
|
||||
/// would remove.
|
||||
#[test]
|
||||
#[ignore]
|
||||
fn what_reshaping_a_growing_message_costs() {
|
||||
use iris::prelude::*;
|
||||
|
||||
let mut h = Harness::new(phone_size(), PHONE_SCALE);
|
||||
// A reply-sized paragraph built a delta at a time. The deltas are
|
||||
// words rather than characters because that is what a model streams.
|
||||
const DELTA: &str = "the quick brown fox jumps over the lazy dog ";
|
||||
let attrs = TextAttrs::default();
|
||||
let width = Some(phone_size().x);
|
||||
|
||||
let mut buffer = iris::prelude::TextBuffer::new_empty();
|
||||
let mut text = String::new();
|
||||
let mut per_delta = Vec::new();
|
||||
let mut total = Duration::ZERO;
|
||||
for n in 1..=400 {
|
||||
text.push_str(DELTA);
|
||||
buffer.set_text(text.clone());
|
||||
let at = Instant::now();
|
||||
buffer.shape(&mut h.rsc.ui.text, &attrs, width, PHONE_SCALE);
|
||||
let took = at.elapsed();
|
||||
per_delta.push(took);
|
||||
total += took;
|
||||
if n % 100 == 0 {
|
||||
println!(
|
||||
" after {n:>3} deltas ({:>6} chars): this reshape {:?}",
|
||||
text.len(),
|
||||
took
|
||||
);
|
||||
}
|
||||
}
|
||||
println!(
|
||||
" 400 deltas: {:?} of shaping in total, {:?} per delta at the end",
|
||||
total,
|
||||
per_delta.last().unwrap()
|
||||
);
|
||||
summarise("reshape", &per_delta);
|
||||
|
||||
// The same measurement at the sizes real replies actually reach.
|
||||
// Measured 2026-09-09 over 7,706 top-level blocks from 3,675 real
|
||||
// assistant messages on this machine: p50 147 chars, p90 449, p99
|
||||
// 836, largest 1,580, and *nothing* above 4,000. The bench fixture's
|
||||
// streamed message is one 14,888-character block, which is 9x the
|
||||
// largest real one -- so the sizes below are what a live reshape
|
||||
// actually costs and the run above is what the benchmark measures.
|
||||
println!(" at the sizes real replies reach:");
|
||||
for chars in [147usize, 449, 836, 1580] {
|
||||
let mut sample = String::new();
|
||||
while sample.len() < chars {
|
||||
sample.push_str(DELTA);
|
||||
}
|
||||
sample.truncate(chars);
|
||||
let mut buffer = iris::prelude::TextBuffer::new(sample);
|
||||
let at = Instant::now();
|
||||
buffer.shape(&mut h.rsc.ui.text, &attrs, width, PHONE_SCALE);
|
||||
println!(" {chars:>5} chars: {:?}", at.elapsed());
|
||||
}
|
||||
}
|
||||
|
||||
/// Where a streamed delta's cost actually is, given that `RowBlocks::
|
||||
/// apply_delta` already re-shapes only the block the delta landed in.
|
||||
/// Three candidates, all of which scale with the *whole* message rather
|
||||
/// than the delta: re-parsing the markdown to find the blocks, comparing
|
||||
/// them against the ones already drawn, and re-shaping the last block.
|
||||
#[test]
|
||||
#[ignore]
|
||||
fn where_a_streamed_deltas_cost_is() {
|
||||
use ai_app::client::markdown_blocks::{common_prefix, split_blocks};
|
||||
|
||||
// A reply with real block structure -- paragraphs separated by blank
|
||||
// lines, the way a model writes -- so the last block is one paragraph
|
||||
// rather than the whole message.
|
||||
const SENTENCE: &str = "The quick brown fox jumps over the lazy dog. ";
|
||||
let mut src = String::new();
|
||||
let mut blocks = Vec::new();
|
||||
|
||||
let mut split = Vec::new();
|
||||
let mut compare = Vec::new();
|
||||
for n in 1..=400 {
|
||||
src.push_str(SENTENCE);
|
||||
// A paragraph break every eight deltas, so the trailing block
|
||||
// stays a normal size and only the message grows.
|
||||
if n % 8 == 0 {
|
||||
src.push_str("\n\n");
|
||||
}
|
||||
|
||||
let at = Instant::now();
|
||||
let new_blocks = split_blocks(&src);
|
||||
split.push(at.elapsed());
|
||||
|
||||
let at = Instant::now();
|
||||
let _ = common_prefix(&blocks, &new_blocks);
|
||||
compare.push(at.elapsed());
|
||||
|
||||
blocks = new_blocks;
|
||||
if n % 100 == 0 {
|
||||
println!(
|
||||
" after {n:>3} deltas ({:>6} chars, {} blocks): split {:?} compare {:?}",
|
||||
src.len(),
|
||||
blocks.len(),
|
||||
split[n - 1],
|
||||
compare[n - 1],
|
||||
);
|
||||
}
|
||||
}
|
||||
summarise("split_blocks", &split);
|
||||
summarise("common_prefix", &compare);
|
||||
let total: Duration = split.iter().chain(compare.iter()).sum();
|
||||
println!(" 400 deltas: {total:?} in block-splitting and comparison alone");
|
||||
}
|
||||
|
||||
/// What the bench fixture's streamed tail actually is, since the cost of
|
||||
/// a delta depends entirely on how big the block it lands in gets.
|
||||
#[test]
|
||||
#[ignore]
|
||||
fn what_the_fixture_streams() {
|
||||
use ai_app::client::markdown_blocks::split_blocks;
|
||||
|
||||
let mut h = Harness::new(phone_size(), PHONE_SCALE);
|
||||
let opened = ai_app::ui::fixture::open(&mut h.rsc, &mut h.state).expect("the fixture folds");
|
||||
let mut items = opened.items;
|
||||
let before = items.len();
|
||||
for event in &opened.stream_tail {
|
||||
items = ai_app::client::transcript_fold::fold_event(&items, event);
|
||||
}
|
||||
println!("{} items -> {}", before, items.len());
|
||||
// The last few items are where the stream landed. Only the message
|
||||
// variants matter -- those are what a delta appends to.
|
||||
use ai_app::client::transcript_fold::TranscriptItem;
|
||||
for item in items.iter().rev().take(4) {
|
||||
let (kind, text) = match item {
|
||||
TranscriptItem::AssistantMsg { text, .. } => ("AssistantMsg", text.clone()),
|
||||
TranscriptItem::UserMsg { text, .. } => ("UserMsg", text.clone()),
|
||||
TranscriptItem::ToolRun { input, output, .. } => {
|
||||
("ToolRun", format!("{input}{output}"))
|
||||
}
|
||||
_ => ("other", String::new()),
|
||||
};
|
||||
let blocks = split_blocks(&text);
|
||||
println!(
|
||||
" tail {kind}: {} chars in {} blocks; longest block {} chars",
|
||||
text.len(),
|
||||
blocks.len(),
|
||||
blocks.iter().map(|b| b.source.len()).max().unwrap_or(0),
|
||||
);
|
||||
}
|
||||
}
|
||||
Reference in new issue
Block a user