irisandClaude Fable 5.1
0be6a571c4
Deterministic (seeded), in the app's own event model rather than a real transcript: 3,601 events split into a 3,200-event opening backlog and a 400-event tail both bench harnesses replay as the streaming phase, with headings, inline markdown, fenced code in six languages, a table, tool calls with kilobyte-scale input/output, and two embedded PNGs. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2.0 KiB
The P0 benchmark fixture
transcript.jsonl is a synthetic transcript in the app's own event model (the JSON lines
GET /sessions/{id}/transcript returns; see Events.kt's parseSeqEvent and
server/src/session/driver.rs) -- never a real one. It is what both the Compose bench build
and iris's bench build open with no server, so the two apps draw exactly the same content and a
frame-time comparison is measuring the renderer rather than the data.
Generated by ./generate.py (Python stdlib only, seeded -- SEED = 20260905 -- so re-running it
reproduces the same file byte for byte). It writes into assets/ -- a separate directory from this
script and README, because the Compose bench build type points its own asset source set straight
at assets/ (app/androidApp/build.gradle.kts's sourceSets { getByName("bench") }), and a Python
script and a markdown file have no business inside an APK:
transcript.jsonl-- 3,601 events. The first 3,200 (BACKLOG_COUNT) are the scrolled-back history the benchmark opens with: user turns, tool calls with kilobyte-scale input/output, assistant replies built from headings, bold/italic/inline code, a link, fenced code blocks that rotate through rust/kotlin/python/sh/json/toml, a markdown table, two embedded images, and periodicusageDelta/compactedevents. The remaining 400 (STREAM_COUNT) are not part of the opening window -- both bench harnesses replay them at a fixed rate (20/s) through the same live fold path a real SSE reply arrives on, which is P0's "streaming phase."bench1.png,bench2.png-- tiny (8x8) flat-colour PNGs, base64-free on disk but served the same way a real attachment is (GET /sessions/{id}/files/{name}), referenced by the two"type":"image"events in the transcript.
Regenerate after changing the shape (a new event type, a different backlog/stream split) with
./generate.py, and commit the result -- it is checked in rather than generated at build time so
both apps' bench builds embed the identical bytes without needing this script at build time.