2 Commits
Author SHA1 Message Date
irisandClaude Fable 5.1 5b2ca039f1 docs/RUST.md: bench v2 spec and the emulator smoke run
Iris's ask (2026-09-06): the fling should travel much faster for
stress-testing, plus typing and keyboard phases. Written once into the
P0 box so the iris agent implements the identical four-phase spec --
constants, ordering and report shape -- rather than a second one that
looks the same but isn't.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-05 23:41:03 -04:00
irisandClaude Fable 5.1 a8d24553d5 app: bench v2 -- a real fling, typing and keyboard phases
Iris's ask after using the Compose bench build on her phone: the old
scroll phase used animateScrollBy, which can only ever cover the fixed
distance/time it's given, so it never flings the way a real fast swipe
does. BenchRun.run now has four phases: fling (8 flings out + 8 back
through the list's own FlingBehavior at 12,000px/s), stream (unchanged),
type (600 fixed characters into the real composer TextFieldValue, then
deleted, to exercise wrapping and the transcript being pushed upward),
and keyboard (five show/hide cycles via WindowInsetsControllerCompat,
each confirmed by isImeVisible rather than assumed).

FrameStats.markPhase/phaseLines slice the same FrameMetrics recording
by phase rather than running a second recorder; debugReport gains a
phaseFrames section ahead of the existing whole-run frames/accounting/
work sections, which are otherwise unchanged.

Also fixes a pre-existing, unrelated break in MainActivity.kt's
benchSessionSummary() -- missing several SessionSummary constructor
arguments from an earlier change -- since it blocked compileBenchKotlin
outright.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-05 23:40:59 -04:00
6 changed files with 499 additions and 39 deletions

No files matched your search

@@ -3,9 +3,13 @@ package com.example.aiapp
import android.content.Context import android.content.Context
import android.os.BatteryManager import android.os.BatteryManager
import android.os.Process import android.os.Process
import androidx.compose.animation.core.tween import android.view.View
import androidx.compose.foundation.gestures.animateScrollBy import androidx.compose.foundation.gestures.FlingBehavior
import androidx.compose.foundation.lazy.LazyListState import androidx.compose.foundation.lazy.LazyListState
import androidx.compose.ui.focus.FocusRequester
import androidx.core.view.ViewCompat
import androidx.core.view.WindowInsetsCompat
import androidx.core.view.WindowInsetsControllerCompat
import java.io.File import java.io.File
import kotlinx.coroutines.CoroutineScope import kotlinx.coroutines.CoroutineScope
import kotlinx.coroutines.delay import kotlinx.coroutines.delay
@@ -19,27 +23,84 @@ import kotlinx.coroutines.launch
* here against [LazyListState] and [BenchFixture] directly. Only reachable from the `bench` build * here against [LazyListState] and [BenchFixture] directly. Only reachable from the `bench` build
* (see [SessionSettingsDialog]'s `onRunBenchmark`), but compiled into every build for the reason * (see [SessionSettingsDialog]'s `onRunBenchmark`), but compiled into every build for the reason
* [BenchFixture]'s doc comment gives. * [BenchFixture]'s doc comment gives.
*
* **v2 (2026-09-06)**, asked for by Iris because the v1 fling was too gentle to stress-test the
* scroll path and said nothing about typing or the keyboard. Four phases now, each a slice of the
* same [FrameStats] recording ([FrameStats.markPhase]/[FrameStats.phaseLines] -- one recorder, not
* two): **fling** (real `FlingBehavior`, not `animateScrollBy`), **stream** (unchanged from v1),
* **type** (600 fixed characters into the real composer `TextFieldValue`, then deleted), and
* **keyboard** (five show/hide cycles). The exact constants below are also written into
* `docs/RUST.md`'s P0 box, "Benchmark v2 (2026-09-06)", so the iris half implements the identical
* spec -- changing a number here without updating that box makes the two apps measure different
* things while looking like the same benchmark.
*/ */
object BenchRun { object BenchRun {
/** transcript-bench.sh's default: 6 cycles of 4 swipes each, 900px over 200ms, 500ms apart. */ /** transcript-bench.sh's default: 6 cycles of 4 swipes each, kept as the pre-v2 comparison. */
private const val CYCLES = 6 private const val CYCLES = 6
private const val SWIPE_PX = 900f private const val SWIPE_PX = 900f
private const val SWIPE_MS = 200 private const val SWIPE_MS = 200
private const val SWIPE_PAUSE_MS = 500L private const val SWIPE_PAUSE_MS = 500L
/**
* Fling phase (v2): a real fling through the list's own [FlingBehavior], not `animateScrollBy`
* -- Iris's ask was that it "travel way faster" than the old tween-based swipe, and a tween can
* never exceed the distance it is told to cover in the time it is given, while a real fling
* decays from an initial velocity the way a finger flick does. 12,000 px/s is roughly a hard,
* fast flick on a ~420dp/in device (about 30 dp/ms-equivalent initial speed); chosen well above
* the ~4,500 px/s a moderate `animateScrollBy` swipe implies, so this phase exercises the fast
* end of what the platform's fling decay produces rather than the gentle one v1 measured.
*/
private const val FLING_VELOCITY_PX_S = 12_000f
private const val FLING_COUNT = 8
private const val FLING_SETTLE_CAP_MS = 3_000L
private const val FLING_PAUSE_MS = 300L
/** stream-bench.sh's shape: a real reply arrives as many small deltas, not one big write. */ /** stream-bench.sh's shape: a real reply arrives as many small deltas, not one big write. */
private const val STREAM_EVENTS_PER_SEC = 20 private const val STREAM_EVENTS_PER_SEC = 20
private const val STREAM_SECONDS = 20 private const val STREAM_SECONDS = 20
/** /**
* Scrolls, then streams, then returns the extra report lines P0 asked for (CPU time, peak RSS, * Type phase (v2): sentences built from long, multisyllabic words so the composer actually
* battery current) -- [FrameStats] and [DebugStats] are reset first, exactly as * wraps across lines rather than fitting one, and long enough (600 chars) that the composer's
* `copyRenderReport` resets them, so the two accountings cover the same stretch of work. * own height grows over several frames, pushing the transcript above it upward the same way a
* real long message does. Exactly this string is also in `docs/RUST.md`'s P0 box so the iris
* half types the identical content.
*/
const val TYPE_TEXT =
"Benchmarking this transcript screen requires unusually long, multisyllabic words so " +
"wrapping and reflow are properly exercised: internationalization, " +
"counterproductiveness, disproportionately, incomprehensibility, " +
"deinstitutionalization, uncharacteristically, overenthusiastically, " +
"misunderstanding, straightforwardness, telecommunications, and interdisciplinary " +
"collaboration all push a narrow composer field to wrap across several lines while " +
"the transcript above is pushed upward by the growing keyboard-adjacent box, which " +
"is exactly what a real reader typing a long message sees happening now!!!"
private const val TYPE_CHAR_DELAY_MS = 50L
/**
* Keyboard phase (v2): five show/hide cycles, a second apart, is enough to see whether the
* transition is ever actually observed rather than being a one-off fluke either way.
*/
private const val KEYBOARD_CYCLES = 5
private const val KEYBOARD_SHOW_WAIT_MS = 1_000L
private const val KEYBOARD_HIDE_WAIT_MS = 1_000L
/**
* Scrolls, flings, streams, types and toggles the keyboard, then returns the extra report lines
* P0 asked for (per-phase travel/typing/keyboard counts, plus CPU time, peak RSS, battery
* current) -- [FrameStats] and [DebugStats] are reset first, exactly as `copyRenderReport`
* resets them, so the two accountings cover the same stretch of work.
*/ */
suspend fun run( suspend fun run(
context: Context, context: Context,
scope: CoroutineScope, scope: CoroutineScope,
listState: LazyListState, listState: LazyListState,
flingBehavior: FlingBehavior,
composerFocus: FocusRequester,
setComposerText: (String) -> Unit,
view: View,
): List<String> { ): List<String> {
FrameStats.reset() FrameStats.reset()
DebugStats.reset() DebugStats.reset()
@@ -55,34 +116,10 @@ object BenchRun {
} }
} }
// The swipe loop: transcript-bench.sh's four swipes per cycle are two drags toward newer val travel = runFlingPhase(listState, flingBehavior)
// content and two back, so a cycle returns to where it started and the whole loop measures val sent = runStreamPhase()
// steady-state scrolling rather than travelling somewhere new each time. runTypePhase(listState, composerFocus, setComposerText, view)
repeat(CYCLES) { val keyboard = runKeyboardPhase(context, view)
repeat(2) {
listState.animateScrollBy(SWIPE_PX, tween(SWIPE_MS))
delay(SWIPE_PAUSE_MS)
}
repeat(2) {
listState.animateScrollBy(-SWIPE_PX, tween(SWIPE_MS))
delay(SWIPE_PAUSE_MS)
}
}
// Pinned to the newest end before streaming starts, the way stream-bench.sh's "Jump to
// latest" tap is -- a reply streamed into a list parked further back arrives off-screen and
// the report would show nothing happened.
listState.scrollToItem(0)
var sent = 0
val total = STREAM_EVENTS_PER_SEC * STREAM_SECONDS
while (sent < total && BenchFixture.remainingStreamEvents() > 0) {
BenchFixture.pushNextLiveEvent()
sent++
delay(1000L / STREAM_EVENTS_PER_SEC)
}
// Lets the last few deltas land and draw before the report is read.
delay(300)
samplerJob.cancel() samplerJob.cancel()
val cpuMs = Process.getElapsedCpuTime() - cpuStartMs val cpuMs = Process.getElapsedCpuTime() - cpuStartMs
@@ -90,13 +127,158 @@ object BenchRun {
val batteryLine = battery.finish() val batteryLine = battery.finish()
return listOf( return listOf(
" scroll: $CYCLES cycles (${CYCLES * 4} swipes), streamed $sent/$total fixture events", " fling: $FLING_COUNT flings out + $FLING_COUNT back at" +
" ${FLING_VELOCITY_PX_S.toInt()}px/s, travel $travel",
" scroll: $CYCLES cycles (${CYCLES * 4} swipes, legacy tween), " +
"streamed $sent/${STREAM_EVENTS_PER_SEC * STREAM_SECONDS} fixture events",
" type: ${TYPE_TEXT.length} characters inserted then deleted, one per" +
" ${TYPE_CHAR_DELAY_MS}ms",
keyboard,
" process CPU time over this run: ${cpuMs}ms", " process CPU time over this run: ${cpuMs}ms",
rssLine, rssLine,
batteryLine, batteryLine,
) )
} }
/**
* Phase 1: starting pinned at the newest end, [FLING_COUNT] flings away from it (toward older
* messages) through the list's real fling path, then [FLING_COUNT] back. Positive velocity here
* matches this list's existing scroll-offset convention (`TranscriptList`'s `reverseLayout`
* pins index 0 -- the newest item -- at the bottom; a positive scroll offset moves the viewport
* toward higher indices, i.e. away from the newest end and toward older content), the same sign
* the pre-v2 swipe loop below already used for its first two swipes.
*/
private suspend fun runFlingPhase(
listState: LazyListState,
flingBehavior: FlingBehavior,
): String {
FrameStats.markPhase("fling")
listState.scrollToItem(0)
val start = position(listState)
repeat(FLING_COUNT) {
listState.scroll { with(flingBehavior) { performFling(FLING_VELOCITY_PX_S) } }
waitForSettle(listState)
delay(FLING_PAUSE_MS)
}
val outward = position(listState)
repeat(FLING_COUNT) {
listState.scroll { with(flingBehavior) { performFling(-FLING_VELOCITY_PX_S) } }
waitForSettle(listState)
delay(FLING_PAUSE_MS)
}
val back = position(listState)
return "start=$start outward=$outward end=$back"
}
private fun position(listState: LazyListState) =
"idx=${listState.firstVisibleItemIndex}/off=${listState.firstVisibleItemScrollOffset}px"
/** Belt-and-suspenders on top of `performFling` already suspending until its own decay ends. */
private suspend fun waitForSettle(listState: LazyListState) {
val startedAt = System.currentTimeMillis()
while (
listState.isScrollInProgress &&
System.currentTimeMillis() - startedAt < FLING_SETTLE_CAP_MS
) {
delay(16)
}
}
/**
* Phase 2 (unchanged from v1): pinned to the newest end before streaming starts, the way
* stream-bench.sh's "Jump to latest" tap is -- a reply streamed into a list parked further back
* arrives off-screen and the report would show nothing happened.
*/
private suspend fun runStreamPhase(): Int {
FrameStats.markPhase("stream")
var sent = 0
val total = STREAM_EVENTS_PER_SEC * STREAM_SECONDS
while (sent < total && BenchFixture.remainingStreamEvents() > 0) {
BenchFixture.pushNextLiveEvent()
sent++
delay(1000L / STREAM_EVENTS_PER_SEC)
}
// Lets the last few deltas land and draw before the next phase starts.
delay(300)
return sent
}
/**
* Phase 3: focuses the real composer, shows the keyboard if the platform allows it, then types
* [TYPE_TEXT] one character at a time through the same `TextFieldValue` state a real keystroke
* updates, and deletes it the same way -- this is what exercises wrapping and the transcript
* being pushed upward, not a single big write.
*/
private suspend fun runTypePhase(
listState: LazyListState,
composerFocus: FocusRequester,
setComposerText: (String) -> Unit,
view: View,
) {
FrameStats.markPhase("type")
listState.scrollToItem(0)
composerFocus.requestFocus()
showIme(view.context, view)
// Lets focus and the keyboard's opening animation land before typing starts, so the frames
// this phase records are the wrap/reflow it is measuring, not the keyboard opening.
delay(300)
var typed = ""
for (ch in TYPE_TEXT) {
typed += ch
setComposerText(typed)
delay(TYPE_CHAR_DELAY_MS)
}
delay(200)
while (typed.isNotEmpty()) {
typed = typed.dropLast(1)
setComposerText(typed)
delay(TYPE_CHAR_DELAY_MS)
}
}
/**
* Phase 4: [KEYBOARD_CYCLES] show/hide cycles through the same [WindowInsetsControllerCompat]
* path a real IME toggle goes through, reporting how many of each were actually confirmed by
* [android.view.WindowInsets.isVisible] rather than assumed from having asked -- UI_RULES:
* never present an inferred value as a measured one. If the platform never shows it even once,
* this says so in words rather than reporting a phase with no keyboard in it.
*/
private suspend fun runKeyboardPhase(context: Context, view: View): String {
FrameStats.markPhase("keyboard")
var shown = 0
var hidden = 0
repeat(KEYBOARD_CYCLES) {
showIme(context, view)
delay(KEYBOARD_SHOW_WAIT_MS)
if (imeVisible(view)) shown++
hideIme(context, view)
delay(KEYBOARD_HIDE_WAIT_MS)
if (!imeVisible(view)) hidden++
}
return if (shown == 0) {
" keyboard: could not be shown ($KEYBOARD_CYCLES attempts, 0 confirmed visible)"
} else {
" keyboard: shown $shown/$KEYBOARD_CYCLES, hidden $hidden/$KEYBOARD_CYCLES" +
" (confirmed via isImeVisible)"
}
}
private fun controller(context: Context, view: View): WindowInsetsControllerCompat? {
val window = context.activity()?.window ?: return null
return WindowInsetsControllerCompat(window, view)
}
private fun showIme(context: Context, view: View) {
controller(context, view)?.show(WindowInsetsCompat.Type.ime())
}
private fun hideIme(context: Context, view: View) {
controller(context, view)?.hide(WindowInsetsCompat.Type.ime())
}
private fun imeVisible(view: View): Boolean =
ViewCompat.getRootWindowInsets(view)?.isVisible(WindowInsetsCompat.Type.ime()) ?: false
/** VmHWM from /proc/self/status: the process's high-water mark, in kB, since it started. */ /** VmHWM from /proc/self/status: the process's high-water mark, in kB, since it started. */
private fun peakRssLine(): String { private fun peakRssLine(): String {
val kb = val kb =
@@ -134,6 +134,13 @@ fun debugReport(
* render-report button reads exactly as it did before this existed. * render-report button reads exactly as it did before this existed.
*/ */
extra: List<String> = emptyList(), extra: List<String> = emptyList(),
/**
* Bench v2's per-phase frame accounting ([FrameStats.phaseLines]) --
* fling/stream/type/keyboard, each a slice of the same frames the whole-run sections below
* still cover in full. Empty on every path but the scripted bench run, same reasoning as
* [extra].
*/
phaseFrames: List<String> = emptyList(),
): String = buildString { ): String = buildString {
appendLine("ai-app render report") appendLine("ai-app render report")
appendLine(device) appendLine(device)
@@ -148,6 +155,11 @@ fun debugReport(
appendLine("transcript:") appendLine("transcript:")
transcript.forEach { appendLine(it) } transcript.forEach { appendLine(it) }
appendLine() appendLine()
if (phaseFrames.isNotEmpty()) {
appendLine("per phase:")
phaseFrames.forEach { appendLine(it) }
appendLine()
}
appendLine("frames:") appendLine("frames:")
frames.forEach { appendLine(it) } frames.forEach { appendLine(it) }
appendLine() appendLine()
@@ -42,6 +42,21 @@ object FrameStats {
private val gpu = ArrayList<Long>() private val gpu = ArrayList<Long>()
private var since = System.currentTimeMillis() private var since = System.currentTimeMillis()
/**
* Where a named phase of a scripted run (bench v2's fling/stream/type/keyboard) started, as an
* index into [total] and a wall-clock time -- not a second recorder, just a mark on this one,
* so a phase's frames are the same [FrameMetrics] the whole-run report already has, sliced.
*/
private data class PhaseMark(val name: String, val startIndex: Int, val startMs: Long)
private val phaseMarks = ArrayList<PhaseMark>()
/** Call at the start of each named phase of a scripted run; see [BenchRun]. */
@Synchronized
fun markPhase(name: String) {
phaseMarks += PhaseMark(name, total.size, System.currentTimeMillis())
}
@Synchronized @Synchronized
fun add(metrics: FrameMetrics) { fun add(metrics: FrameMetrics) {
// The first frame after a window opens includes inflating it and is nobody's scroll. // The first frame after a window opens includes inflating it and is nobody's scroll.
@@ -69,6 +84,7 @@ object FrameStats {
listOf(total, waited, input, animation, layout, draw, sync, issue, swap, gpu).forEach { listOf(total, waited, input, animation, layout, draw, sync, issue, swap, gpu).forEach {
it.clear() it.clear()
} }
phaseMarks.clear()
since = System.currentTimeMillis() since = System.currentTimeMillis()
} }
@@ -95,6 +111,38 @@ object FrameStats {
) + if (gpu.isEmpty()) emptyList() else listOf(phase("gpu ", gpu)) ) + if (gpu.isEmpty()) emptyList() else listOf(phase("gpu ", gpu))
} }
/**
* One block per [markPhase] call: how many frames landed between that mark and the next (or the
* end of the run, for the last one), how many were late, the total/p50/p90/p99, the worst
* single frame, and how long the phase actually ran. Marks with no frames between them (a phase
* that finished before a frame was drawn) still get a line rather than being silently dropped
* -- UI_RULES' "say what you don't know" applies to a phase as much as to a single number.
*/
@Synchronized
fun phaseLines(refreshHz: Float): List<String> {
if (phaseMarks.isEmpty()) return emptyList()
val budget = if (refreshHz > 0) 1000.0 / refreshHz else 16.7
val lines = ArrayList<String>()
phaseMarks.forEachIndexed { i, mark ->
val endIndex = if (i + 1 < phaseMarks.size) phaseMarks[i + 1].startIndex else total.size
val endMs =
if (i + 1 < phaseMarks.size) phaseMarks[i + 1].startMs
else System.currentTimeMillis()
val samples = total.subList(mark.startIndex, endIndex)
val seconds = (endMs - mark.startMs) / 1000.0
lines += " ${mark.name}: ${samples.size} frames over ${"%.1f".format(seconds)}s"
if (samples.isEmpty()) {
lines += " no frames recorded in this phase"
} else {
val late = samples.count { it / 1_000_000.0 > budget }
lines += " late: $late (${percent(late, samples.size)})"
lines += " " + phase("total ", samples)
lines += " worst ${"%.1fms".format(samples.max() / 1_000_000.0)}"
}
}
return lines
}
/** How long the frames recorded here spent in their draw phase, and how many there were. */ /** How long the frames recorded here spent in their draw phase, and how many there were. */
@Synchronized fun drawPhase(): Pair<Long, Int> = draw.sum() to draw.size @Synchronized fun drawPhase(): Pair<Long, Int> = draw.sum() to draw.size
@@ -187,13 +187,20 @@ class MainActivity : ComponentActivity() {
model = null, model = null,
keepsOwnTranscript = false, keepsOwnTranscript = false,
permissionMode = null, permissionMode = null,
effort = null,
takesEffort = false,
imported = false, imported = false,
notify = false, notify = false,
autoResume = false,
autoResumeMessage = "",
resumeAt = null,
cwd = null, cwd = null,
contextTokens = null, contextTokens = null,
maxImageEdge = null, maxImageEdge = null,
usageProvider = null,
status = "idle", status = "idle",
lastActivity = 0.0, lastActivity = 0.0,
subagents = 0,
) )
// launchMode="singleTop": an enrollment scan, or a notification tapped while the app is open, // launchMode="singleTop": an enrollment scan, or a notification tapped while the app is open,
@@ -12,6 +12,7 @@ import androidx.activity.result.PickVisualMediaRequest
import androidx.activity.result.contract.ActivityResultContracts import androidx.activity.result.contract.ActivityResultContracts
import androidx.compose.foundation.background import androidx.compose.foundation.background
import androidx.compose.foundation.clickable import androidx.compose.foundation.clickable
import androidx.compose.foundation.gestures.ScrollableDefaults
import androidx.compose.foundation.gestures.awaitEachGesture import androidx.compose.foundation.gestures.awaitEachGesture
import androidx.compose.foundation.gestures.awaitFirstDown import androidx.compose.foundation.gestures.awaitFirstDown
import androidx.compose.foundation.layout.Box import androidx.compose.foundation.layout.Box
@@ -64,12 +65,15 @@ import androidx.compose.runtime.snapshots.Snapshot
import androidx.compose.ui.Alignment import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier import androidx.compose.ui.Modifier
import androidx.compose.ui.draw.drawWithContent import androidx.compose.ui.draw.drawWithContent
import androidx.compose.ui.focus.FocusRequester
import androidx.compose.ui.focus.focusRequester
import androidx.compose.ui.graphics.graphicsLayer import androidx.compose.ui.graphics.graphicsLayer
import androidx.compose.ui.input.pointer.PointerEventPass import androidx.compose.ui.input.pointer.PointerEventPass
import androidx.compose.ui.input.pointer.pointerInput import androidx.compose.ui.input.pointer.pointerInput
import androidx.compose.ui.layout.onSizeChanged import androidx.compose.ui.layout.onSizeChanged
import androidx.compose.ui.platform.LocalContext import androidx.compose.ui.platform.LocalContext
import androidx.compose.ui.platform.LocalDensity import androidx.compose.ui.platform.LocalDensity
import androidx.compose.ui.platform.LocalView
import androidx.compose.ui.semantics.contentDescription import androidx.compose.ui.semantics.contentDescription
import androidx.compose.ui.semantics.semantics import androidx.compose.ui.semantics.semantics
import androidx.compose.ui.text.TextRange import androidx.compose.ui.text.TextRange
@@ -354,6 +358,17 @@ fun SessionScreen(
// `rememberSaveable`, and this screen restores by its own anchor instead -- two restores would // `rememberSaveable`, and this screen restores by its own anchor instead -- two restores would
// fight over the first frame. // fight over the first frame.
val listState = remember(address) { LazyListState() } val listState = remember(address) { LazyListState() }
// The list's own fling path -- what a real flick decays through -- captured here so BenchRun's
// fling phase can drive `LazyListState.scroll` through exactly the `FlingBehavior` this
// screen's
// `TranscriptList` already uses by not overriding it (its `LazyColumn` takes no `flingBehavior`
// argument, so this is the same default it gets).
val flingBehavior = ScrollableDefaults.flingBehavior()
// Where BenchRun's type phase focuses before it types, and the view it toggles the keyboard on
// -- both bench-only, but cheap enough (a remembered object, a CompositionLocal read) to hold
// unconditionally rather than behind a second code path only the bench build compiles.
val composerFocus = remember { FocusRequester() }
val view = LocalView.current
// Whether the newest message is on screen right now. The list is reversed, so the newest end is // Whether the newest message is on screen right now. The list is reversed, so the newest end is
// the scrolling start: nothing behind you is exactly being at the bottom. Asked of the scroll // the scrolling start: nothing behind you is exactly being at the bottom. Asked of the scroll
// state rather than of item indices, because a zero-height first item makes an index ambiguous. // state rather than of item indices, because a zero-height first item makes an index ambiguous.
@@ -1256,6 +1271,10 @@ fun SessionScreen(
FrameStats.drawPhase().let { (nanos, count) -> drawAccounting(nanos, count) }, FrameStats.drawPhase().let { (nanos, count) -> drawAccounting(nanos, count) },
crash = lastCrash(context), crash = lastCrash(context),
extra = extra, extra = extra,
// Empty outside a BenchRun.run pass -- copyRenderReport's own reset below clears
// the
// marks along with everything else, so an ordinary copy never has any to show.
phaseFrames = FrameStats.phaseLines(context.refreshHz()),
) )
context.copyToClipboard("ai-app render report", report) context.copyToClipboard("ai-app render report", report)
// Also to the log, so a session driving the app over adb can read the same report the // Also to the log, so a session driving the app over adb can read the same report the
@@ -1270,15 +1289,24 @@ fun SessionScreen(
Toast.makeText(context, "Copied render report", Toast.LENGTH_SHORT).show() Toast.makeText(context, "Copied render report", Toast.LENGTH_SHORT).show()
} }
val copyRenderReport = { buildAndCopyReport() } val copyRenderReport = { buildAndCopyReport() }
// Bench build only: P0's scripted scroll-and-stream benchmark (BenchRun.kt), against the // Bench build only: P0's scripted fling/stream/type/keyboard benchmark (BenchRun.kt), against
// fixture session opened below instead of a real server. Null everywhere else -- see // the fixture session opened below instead of a real server. Null everywhere else -- see
// [SessionSettingsDialog]'s onRunBenchmark. // [SessionSettingsDialog]'s onRunBenchmark.
val runBenchmark: (() -> Unit)? = val runBenchmark: (() -> Unit)? =
if (BuildConfig.FIXTURE_MODE) { if (BuildConfig.FIXTURE_MODE) {
{ {
settingsOpen = false settingsOpen = false
scope.launch { scope.launch {
val extra = BenchRun.run(context, scope, listState) val extra =
BenchRun.run(
context = context,
scope = scope,
listState = listState,
flingBehavior = flingBehavior,
composerFocus = composerFocus,
setComposerText = { text -> input = atEnd(text) },
view = view,
)
buildAndCopyReport(extra) buildAndCopyReport(extra)
} }
} }
@@ -1777,7 +1805,10 @@ fun SessionScreen(
input = it input = it
saveDraft(context, summary.id, it.text) saveDraft(context, summary.id, it.text)
}, },
modifier = Modifier.fillMaxWidth(), // BenchRun's type phase requests focus on this exact field
// (`composerFocus`)
// so it types through the real composer rather than a stand-in.
modifier = Modifier.fillMaxWidth().focusRequester(composerFocus),
// No longer "(+image)": the images are on screen above this, and a // No longer "(+image)": the images are on screen above this, and a
// placeholder saying so said it in words beside the thing itself. // placeholder saying so said it in words beside the thing itself.
placeholder = { Text("Message") }, placeholder = { Text("Message") },
+180
View File
@@ -4135,6 +4135,186 @@ device.
back with a diagnostic that will say the real story on the next back with a diagnostic that will say the real story on the next
run either way. run either way.
**Benchmark v2 (2026-09-06), asked for by Iris after using the
Compose build on her phone**: "it doesn't fling like I typically do
when scrolling up to find old messages. It should travel way faster
which is better for stress testing. You may also want to add typing
in the textbox as well and seeing how performant wrapping & pushing
the transcript up are, and also keyboard performance if possible."
**This is the one spec** -- written once here so both apps' "Run
benchmark" implement the identical four phases; a change to a
constant below has to be made in both `app/`'s `BenchRun.kt` and
`iris/`'s bench client, together, or the two reports stop measuring
the same thing while still looking like they do.
1. **fling.** Starting pinned at the newest end
(`listState.scrollToItem(0)` / iris's equivalent), 8 flings away
from it (toward older messages) through the list's own real fling
path -- Compose: `LazyListState.scroll { with(flingBehavior) {
performFling(velocity) } }` using the screen's actual
`FlingBehavior` (`ScrollableDefaults.flingBehavior()`, since
`TranscriptList`'s `LazyColumn` never overrides it -- **not**
`animateScrollBy`, which can only ever cover the fixed distance
and time it is given and was Iris's complaint) -- each fling's
`initialVelocity = 12,000 px/s`. That number is well above a
moderate tween-swipe's implied speed (v1's `SWIPE_PX`/`SWIPE_MS`
is roughly 4,500 px/s) and is meant to be a hard, fast flick for
stress-testing, per Iris's ask. After each fling, wait for
`isScrollInProgress` to clear (cap 3s; `performFling` already
suspends until its own decay ends, this is belt-and-suspenders)
plus 300ms between flings. Then 8 more flings back toward the
newest end (`-12,000 px/s`). Record the list's first visible
index/offset at the start, after the 8 outward flings, and at the
end, so the two apps' *travel* can be compared directly rather
than just their frame times.
2. **stream. Unchanged from v1**: 400 tail events at 20/s (20
seconds), pinned to the newest end before it starts (the same
"Jump to latest" pin `stream-bench.sh` does).
3. **type.** Pin to the newest end, focus the composer, show the IME
if the platform allows it, then insert this **exact 600-character
string** one character per 50ms through the composer's real
`TextFieldValue` state (Compose: the same `input` state
`onValueChange` writes; iris: whatever holds the composer's text
today), then delete it the same way, one character per 50ms.
Chosen for long, multisyllabic words specifically so the composer
wraps across lines and the transcript above it is pushed upward
by a growing box, which is what Iris asked to see measured:
Benchmarking this transcript screen requires unusually long, multisyllabic words so wrapping and reflow are properly exercised: internationalization, counterproductiveness, disproportionately, incomprehensibility, deinstitutionalization, uncharacteristically, overenthusiastically, misunderstanding, straightforwardness, telecommunications, and interdisciplinary collaboration all push a narrow composer field to wrap across several lines while the transcript above is pushed upward by the growing keyboard-adjacent box, which is exactly what a real reader typing a long message sees happening now!!!
Report whether the IME was actually open during typing (it should
be, from this phase's own show-IME step -- see phase 4 for what
to say if the platform refuses to show it at all).
4. **keyboard.** Show the IME (`WindowInsetsControllerCompat.show
(WindowInsetsCompat.Type.ime())` against the window/view; iris's
equivalent through its own shell), wait 1s, hide it, wait 1s;
five cycles. Confirm each show/hide with the platform's own
answer (Compose: `ViewCompat.getRootWindowInsets(view)
?.isVisible(WindowInsetsCompat.Type.ime())`, i.e. the same
`WindowInsets.isImeVisible` fact `SessionScreen`'s composer-inset
bug fix already relies on) rather than assuming the request
worked -- if it is never confirmed shown even once, the report's
`keyboard:` line has to say "**keyboard: could not be shown**"
in words (UI_RULES: never present an inferred value as a
measured one, and design the unknown/failed state before the
answer's).
**Frame accounting**: one recorder, not two. Mark each phase's start
in the existing per-frame recorder (Compose: `FrameStats.markPhase
(name)`, a list of `(name, frameIndexAtStart, wallClockAtStart)`
alongside the existing `total`/`waited`/... arrays) and slice the
same `FrameMetrics` samples by phase afterward
(`FrameStats.phaseLines`) rather than running a second listener.
**Report shape**: a `per phase:` block appears once any phase marks
exist (empty/absent on an ordinary "Copy" press, which never marks a
phase), one entry per phase: frame count, the phase's wall-clock
duration, late count/percent (against the same refresh-rate budget
the whole-run section uses), p50/p90/p99, and the worst single
frame. Then **every existing whole-run section stays, unchanged in
shape** -- `frames:`, `where the draw phase went:`, `work since this
was last copied:` -- because that is what the emulator-baseline and
phone-baseline numbers already on record in this file were read
against, and a report that dropped or renamed one of those lines
would silently stop being comparable to them. Finally `bench:` gains
new lines beside the existing CPU/RSS/battery ones: the fling
phase's total travel (start/outward/end index+offset), the typed
character count, and the keyboard phase's shown/hidden-confirmed
counts (or the "could not be shown" line).
**Compose half: done, 2026-09-06.** `FrameStats.markPhase`/
`phaseLines` (`app/androidApp/src/main/kotlin/com/example/aiapp/
FrameStats.kt`), `debugReport`'s new `phaseFrames` parameter
(`DebugStats.kt`), and `BenchRun.kt`'s four-phase `run` (fling via
`ScrollableDefaults.flingBehavior()` captured in `SessionScreen` and
passed down since it needs a `@Composable` call site; type via a new
`composerFocus: FocusRequester` attached to the composer's
`OutlinedTextField` plus a `setComposerText` callback that writes
`input` the same way a keystroke does; keyboard via
`WindowInsetsControllerCompat` against `LocalView.current`) are all
in. `BenchRun.TYPE_TEXT` is the exact 600-character constant quoted
above (verified `.length == 600`). A pre-existing, unrelated break
in `MainActivity.kt`'s `benchSessionSummary()` (missing several
`SessionSummary` constructor arguments added by a change this pass
did not otherwise touch -- confirmed pre-existing by reproducing the
same compile failure after stashing this pass's own diff) was fixed
alongside this, since it blocked `compileBenchKotlin` outright and
is in this session's own `app/` scope.
Checks all clean: `ktfmtFormat`, `compileDebugKotlin`,
`compileBenchKotlin`, `lintDebug`, `lintBench` (both "No issues
found"), `testDebugUnitTest`. `grep -n "tap [0-9]" app/*.sh` still
has its one pre-existing, unrelated hit.
**Compose bench v2, emulator smoke run, 2026-09-05** (this
checkout's AVD, cold `emu up`, `ui-trace` tap-by-label throughout --
the dialog needed a swipe to reach "Run benchmark" below the fold,
report read back over `adb logcat`):
ai-app render report
device: sdk_gphone64_x86_64 (Google), Android 16
build: release
transcript:
124 events, 26 rows, 58 units loaded
viewport 1714px, 2 units visible
on screen: the list's own 0px, AssistantMsg 18732px
0 tool calls and 0 groups open
per phase:
fling: 1620 frames over 32.3s
late: 1537 (94.9%)
total p50 20.5ms p90 29.2ms p99 45.9ms
worst 61.8ms
stream: 1079 frames over 20.6s
late: 1037 (96.1%)
total p50 21.0ms p90 33.5ms p99 39.3ms
worst 51.2ms
type: 3568 frames over 61.4s
late: 3536 (99.1%)
total p50 23.8ms p90 32.1ms p99 38.5ms
worst 50.3ms
keyboard: 215 frames over 10.0s
late: 212 (98.6%)
total p50 21.3ms p90 37.6ms p99 48.4ms
worst 50.2ms
frames:
6482 frames over 124.3s at 60Hz (16.7ms budget)
late: 6322 (97.5%)
total p50 21.7ms p90 33.1ms p99 45.3ms
gpu p50 17.4ms p90 27.0ms p99 30.6ms
where the draw phase went:
draw phase 1.27ms per frame, of which:
the transcript: 0.16ms (measure 0.09, place 0.07, record 0.00)
everything else: 1.10ms (87%)
bench:
fling: 8 flings out + 8 back at 12000px/s, travel start=idx=0/off=0px outward=idx=218/off=73px end=idx=0/off=0px
scroll: 6 cycles (24 swipes, legacy tween), streamed 400/400 fixture events
type: 600 characters inserted then deleted, one per 50ms
keyboard: shown 5/5, hidden 5/5 (confirmed via isImeVisible)
process CPU time over this run: 61192ms
peak RSS: 195716kB
battery current: mean 900000µA over 125 samples (min 900000, max 900000)
Read this the same way the v1 emulator smoke run above is read: it
proves the harness runs end to end and produces every field this
spec asked for, not a phone number -- software rasterisation, and
the fixed 900mA battery reading is the emulator's mocked charger
again. Two things worth carrying forward: the **fling phase reached
index 218** in 8 flings (against v1's `animateScrollBy` loop, which
never moved past a handful of indices in the same 8-swipe count),
which is the direct evidence the new fling travels "way faster" as
asked; and **the emulator's software keyboard toggled and was
confirmed by `isImeVisible` all 10 times**, so phase 4 is not a
guaranteed "could not be shown" on every platform, only where the
IME genuinely refuses. `frames:`'s 6,482-frame, 124.3s total matches
the sum of the four phase durations (32.3+20.6+61.4+10.0 ≈ 124.3s),
confirming the phase marks partition the whole run rather than
overlapping or dropping frames between them.
- [ ] **P1 — session screen parity.** History paging backward (with the - [ ] **P1 — session screen parity.** History paging backward (with the
page-boundary healing `client-core` does not have yet, below), page-boundary healing `client-core` does not have yet, below),
`TranscriptSource`-backed cache/server stitching, jump-to-latest, `TranscriptSource`-backed cache/server stitching, jump-to-latest,