app: bench v2 -- a real fling, typing and keyboard phases

Iris's ask after using the Compose bench build on her phone: the old
scroll phase used animateScrollBy, which can only ever cover the fixed
distance/time it's given, so it never flings the way a real fast swipe
does. BenchRun.run now has four phases: fling (8 flings out + 8 back
through the list's own FlingBehavior at 12,000px/s), stream (unchanged),
type (600 fixed characters into the real composer TextFieldValue, then
deleted, to exercise wrapping and the transcript being pushed upward),
and keyboard (five show/hide cycles via WindowInsetsControllerCompat,
each confirmed by isImeVisible rather than assumed).

FrameStats.markPhase/phaseLines slice the same FrameMetrics recording
by phase rather than running a second recorder; debugReport gains a
phaseFrames section ahead of the existing whole-run frames/accounting/
work sections, which are otherwise unchanged.

Also fixes a pre-existing, unrelated break in MainActivity.kt's
benchSessionSummary() -- missing several SessionSummary constructor
arguments from an earlier change -- since it blocked compileBenchKotlin
outright.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
irisandClaude Fable 5.1 committed 2026-09-05 23:40:59 -04:00
1 parent 3b80a88f3b
commit a8d24553d5
5 files changed
+319 -39

No files matched your search

@@ -3,9 +3,13 @@ package com.example.aiapp
import android.content.Context
import android.os.BatteryManager
import android.os.Process
import androidx.compose.animation.core.tween
import androidx.compose.foundation.gestures.animateScrollBy
import android.view.View
import androidx.compose.foundation.gestures.FlingBehavior
import androidx.compose.foundation.lazy.LazyListState
import androidx.compose.ui.focus.FocusRequester
import androidx.core.view.ViewCompat
import androidx.core.view.WindowInsetsCompat
import androidx.core.view.WindowInsetsControllerCompat
import java.io.File
import kotlinx.coroutines.CoroutineScope
import kotlinx.coroutines.delay
@@ -19,27 +23,84 @@ import kotlinx.coroutines.launch
* here against [LazyListState] and [BenchFixture] directly. Only reachable from the `bench` build
* (see [SessionSettingsDialog]'s `onRunBenchmark`), but compiled into every build for the reason
* [BenchFixture]'s doc comment gives.
*
* **v2 (2026-09-06)**, asked for by Iris because the v1 fling was too gentle to stress-test the
* scroll path and said nothing about typing or the keyboard. Four phases now, each a slice of the
* same [FrameStats] recording ([FrameStats.markPhase]/[FrameStats.phaseLines] -- one recorder, not
* two): **fling** (real `FlingBehavior`, not `animateScrollBy`), **stream** (unchanged from v1),
* **type** (600 fixed characters into the real composer `TextFieldValue`, then deleted), and
* **keyboard** (five show/hide cycles). The exact constants below are also written into
* `docs/RUST.md`'s P0 box, "Benchmark v2 (2026-09-06)", so the iris half implements the identical
* spec -- changing a number here without updating that box makes the two apps measure different
* things while looking like the same benchmark.
*/
object BenchRun {
/** transcript-bench.sh's default: 6 cycles of 4 swipes each, 900px over 200ms, 500ms apart. */
/** transcript-bench.sh's default: 6 cycles of 4 swipes each, kept as the pre-v2 comparison. */
private const val CYCLES = 6
private const val SWIPE_PX = 900f
private const val SWIPE_MS = 200
private const val SWIPE_PAUSE_MS = 500L
/**
* Fling phase (v2): a real fling through the list's own [FlingBehavior], not `animateScrollBy`
* -- Iris's ask was that it "travel way faster" than the old tween-based swipe, and a tween can
* never exceed the distance it is told to cover in the time it is given, while a real fling
* decays from an initial velocity the way a finger flick does. 12,000 px/s is roughly a hard,
* fast flick on a ~420dp/in device (about 30 dp/ms-equivalent initial speed); chosen well above
* the ~4,500 px/s a moderate `animateScrollBy` swipe implies, so this phase exercises the fast
* end of what the platform's fling decay produces rather than the gentle one v1 measured.
*/
private const val FLING_VELOCITY_PX_S = 12_000f
private const val FLING_COUNT = 8
private const val FLING_SETTLE_CAP_MS = 3_000L
private const val FLING_PAUSE_MS = 300L
/** stream-bench.sh's shape: a real reply arrives as many small deltas, not one big write. */
private const val STREAM_EVENTS_PER_SEC = 20
private const val STREAM_SECONDS = 20
/**
* Scrolls, then streams, then returns the extra report lines P0 asked for (CPU time, peak RSS,
* battery current) -- [FrameStats] and [DebugStats] are reset first, exactly as
* `copyRenderReport` resets them, so the two accountings cover the same stretch of work.
* Type phase (v2): sentences built from long, multisyllabic words so the composer actually
* wraps across lines rather than fitting one, and long enough (600 chars) that the composer's
* own height grows over several frames, pushing the transcript above it upward the same way a
* real long message does. Exactly this string is also in `docs/RUST.md`'s P0 box so the iris
* half types the identical content.
*/
const val TYPE_TEXT =
"Benchmarking this transcript screen requires unusually long, multisyllabic words so " +
"wrapping and reflow are properly exercised: internationalization, " +
"counterproductiveness, disproportionately, incomprehensibility, " +
"deinstitutionalization, uncharacteristically, overenthusiastically, " +
"misunderstanding, straightforwardness, telecommunications, and interdisciplinary " +
"collaboration all push a narrow composer field to wrap across several lines while " +
"the transcript above is pushed upward by the growing keyboard-adjacent box, which " +
"is exactly what a real reader typing a long message sees happening now!!!"
private const val TYPE_CHAR_DELAY_MS = 50L
/**
* Keyboard phase (v2): five show/hide cycles, a second apart, is enough to see whether the
* transition is ever actually observed rather than being a one-off fluke either way.
*/
private const val KEYBOARD_CYCLES = 5
private const val KEYBOARD_SHOW_WAIT_MS = 1_000L
private const val KEYBOARD_HIDE_WAIT_MS = 1_000L
/**
* Scrolls, flings, streams, types and toggles the keyboard, then returns the extra report lines
* P0 asked for (per-phase travel/typing/keyboard counts, plus CPU time, peak RSS, battery
* current) -- [FrameStats] and [DebugStats] are reset first, exactly as `copyRenderReport`
* resets them, so the two accountings cover the same stretch of work.
*/
suspend fun run(
context: Context,
scope: CoroutineScope,
listState: LazyListState,
flingBehavior: FlingBehavior,
composerFocus: FocusRequester,
setComposerText: (String) -> Unit,
view: View,
): List<String> {
FrameStats.reset()
DebugStats.reset()
@@ -55,34 +116,10 @@ object BenchRun {
}
}
// The swipe loop: transcript-bench.sh's four swipes per cycle are two drags toward newer
// content and two back, so a cycle returns to where it started and the whole loop measures
// steady-state scrolling rather than travelling somewhere new each time.
repeat(CYCLES) {
repeat(2) {
listState.animateScrollBy(SWIPE_PX, tween(SWIPE_MS))
delay(SWIPE_PAUSE_MS)
}
repeat(2) {
listState.animateScrollBy(-SWIPE_PX, tween(SWIPE_MS))
delay(SWIPE_PAUSE_MS)
}
}
// Pinned to the newest end before streaming starts, the way stream-bench.sh's "Jump to
// latest" tap is -- a reply streamed into a list parked further back arrives off-screen and
// the report would show nothing happened.
listState.scrollToItem(0)
var sent = 0
val total = STREAM_EVENTS_PER_SEC * STREAM_SECONDS
while (sent < total && BenchFixture.remainingStreamEvents() > 0) {
BenchFixture.pushNextLiveEvent()
sent++
delay(1000L / STREAM_EVENTS_PER_SEC)
}
// Lets the last few deltas land and draw before the report is read.
delay(300)
val travel = runFlingPhase(listState, flingBehavior)
val sent = runStreamPhase()
runTypePhase(listState, composerFocus, setComposerText, view)
val keyboard = runKeyboardPhase(context, view)
samplerJob.cancel()
val cpuMs = Process.getElapsedCpuTime() - cpuStartMs
@@ -90,13 +127,158 @@ object BenchRun {
val batteryLine = battery.finish()
return listOf(
" scroll: $CYCLES cycles (${CYCLES * 4} swipes), streamed $sent/$total fixture events",
" fling: $FLING_COUNT flings out + $FLING_COUNT back at" +
" ${FLING_VELOCITY_PX_S.toInt()}px/s, travel $travel",
" scroll: $CYCLES cycles (${CYCLES * 4} swipes, legacy tween), " +
"streamed $sent/${STREAM_EVENTS_PER_SEC * STREAM_SECONDS} fixture events",
" type: ${TYPE_TEXT.length} characters inserted then deleted, one per" +
" ${TYPE_CHAR_DELAY_MS}ms",
keyboard,
" process CPU time over this run: ${cpuMs}ms",
rssLine,
batteryLine,
)
}
/**
* Phase 1: starting pinned at the newest end, [FLING_COUNT] flings away from it (toward older
* messages) through the list's real fling path, then [FLING_COUNT] back. Positive velocity here
* matches this list's existing scroll-offset convention (`TranscriptList`'s `reverseLayout`
* pins index 0 -- the newest item -- at the bottom; a positive scroll offset moves the viewport
* toward higher indices, i.e. away from the newest end and toward older content), the same sign
* the pre-v2 swipe loop below already used for its first two swipes.
*/
private suspend fun runFlingPhase(
listState: LazyListState,
flingBehavior: FlingBehavior,
): String {
FrameStats.markPhase("fling")
listState.scrollToItem(0)
val start = position(listState)
repeat(FLING_COUNT) {
listState.scroll { with(flingBehavior) { performFling(FLING_VELOCITY_PX_S) } }
waitForSettle(listState)
delay(FLING_PAUSE_MS)
}
val outward = position(listState)
repeat(FLING_COUNT) {
listState.scroll { with(flingBehavior) { performFling(-FLING_VELOCITY_PX_S) } }
waitForSettle(listState)
delay(FLING_PAUSE_MS)
}
val back = position(listState)
return "start=$start outward=$outward end=$back"
}
private fun position(listState: LazyListState) =
"idx=${listState.firstVisibleItemIndex}/off=${listState.firstVisibleItemScrollOffset}px"
/** Belt-and-suspenders on top of `performFling` already suspending until its own decay ends. */
private suspend fun waitForSettle(listState: LazyListState) {
val startedAt = System.currentTimeMillis()
while (
listState.isScrollInProgress &&
System.currentTimeMillis() - startedAt < FLING_SETTLE_CAP_MS
) {
delay(16)
}
}
/**
* Phase 2 (unchanged from v1): pinned to the newest end before streaming starts, the way
* stream-bench.sh's "Jump to latest" tap is -- a reply streamed into a list parked further back
* arrives off-screen and the report would show nothing happened.
*/
private suspend fun runStreamPhase(): Int {
FrameStats.markPhase("stream")
var sent = 0
val total = STREAM_EVENTS_PER_SEC * STREAM_SECONDS
while (sent < total && BenchFixture.remainingStreamEvents() > 0) {
BenchFixture.pushNextLiveEvent()
sent++
delay(1000L / STREAM_EVENTS_PER_SEC)
}
// Lets the last few deltas land and draw before the next phase starts.
delay(300)
return sent
}
/**
* Phase 3: focuses the real composer, shows the keyboard if the platform allows it, then types
* [TYPE_TEXT] one character at a time through the same `TextFieldValue` state a real keystroke
* updates, and deletes it the same way -- this is what exercises wrapping and the transcript
* being pushed upward, not a single big write.
*/
private suspend fun runTypePhase(
listState: LazyListState,
composerFocus: FocusRequester,
setComposerText: (String) -> Unit,
view: View,
) {
FrameStats.markPhase("type")
listState.scrollToItem(0)
composerFocus.requestFocus()
showIme(view.context, view)
// Lets focus and the keyboard's opening animation land before typing starts, so the frames
// this phase records are the wrap/reflow it is measuring, not the keyboard opening.
delay(300)
var typed = ""
for (ch in TYPE_TEXT) {
typed += ch
setComposerText(typed)
delay(TYPE_CHAR_DELAY_MS)
}
delay(200)
while (typed.isNotEmpty()) {
typed = typed.dropLast(1)
setComposerText(typed)
delay(TYPE_CHAR_DELAY_MS)
}
}
/**
* Phase 4: [KEYBOARD_CYCLES] show/hide cycles through the same [WindowInsetsControllerCompat]
* path a real IME toggle goes through, reporting how many of each were actually confirmed by
* [android.view.WindowInsets.isVisible] rather than assumed from having asked -- UI_RULES:
* never present an inferred value as a measured one. If the platform never shows it even once,
* this says so in words rather than reporting a phase with no keyboard in it.
*/
private suspend fun runKeyboardPhase(context: Context, view: View): String {
FrameStats.markPhase("keyboard")
var shown = 0
var hidden = 0
repeat(KEYBOARD_CYCLES) {
showIme(context, view)
delay(KEYBOARD_SHOW_WAIT_MS)
if (imeVisible(view)) shown++
hideIme(context, view)
delay(KEYBOARD_HIDE_WAIT_MS)
if (!imeVisible(view)) hidden++
}
return if (shown == 0) {
" keyboard: could not be shown ($KEYBOARD_CYCLES attempts, 0 confirmed visible)"
} else {
" keyboard: shown $shown/$KEYBOARD_CYCLES, hidden $hidden/$KEYBOARD_CYCLES" +
" (confirmed via isImeVisible)"
}
}
private fun controller(context: Context, view: View): WindowInsetsControllerCompat? {
val window = context.activity()?.window ?: return null
return WindowInsetsControllerCompat(window, view)
}
private fun showIme(context: Context, view: View) {
controller(context, view)?.show(WindowInsetsCompat.Type.ime())
}
private fun hideIme(context: Context, view: View) {
controller(context, view)?.hide(WindowInsetsCompat.Type.ime())
}
private fun imeVisible(view: View): Boolean =
ViewCompat.getRootWindowInsets(view)?.isVisible(WindowInsetsCompat.Type.ime()) ?: false
/** VmHWM from /proc/self/status: the process's high-water mark, in kB, since it started. */
private fun peakRssLine(): String {
val kb =
@@ -134,6 +134,13 @@ fun debugReport(
* render-report button reads exactly as it did before this existed.
*/
extra: List<String> = emptyList(),
/**
* Bench v2's per-phase frame accounting ([FrameStats.phaseLines]) --
* fling/stream/type/keyboard, each a slice of the same frames the whole-run sections below
* still cover in full. Empty on every path but the scripted bench run, same reasoning as
* [extra].
*/
phaseFrames: List<String> = emptyList(),
): String = buildString {
appendLine("ai-app render report")
appendLine(device)
@@ -148,6 +155,11 @@ fun debugReport(
appendLine("transcript:")
transcript.forEach { appendLine(it) }
appendLine()
if (phaseFrames.isNotEmpty()) {
appendLine("per phase:")
phaseFrames.forEach { appendLine(it) }
appendLine()
}
appendLine("frames:")
frames.forEach { appendLine(it) }
appendLine()
@@ -42,6 +42,21 @@ object FrameStats {
private val gpu = ArrayList<Long>()
private var since = System.currentTimeMillis()
/**
* Where a named phase of a scripted run (bench v2's fling/stream/type/keyboard) started, as an
* index into [total] and a wall-clock time -- not a second recorder, just a mark on this one,
* so a phase's frames are the same [FrameMetrics] the whole-run report already has, sliced.
*/
private data class PhaseMark(val name: String, val startIndex: Int, val startMs: Long)
private val phaseMarks = ArrayList<PhaseMark>()
/** Call at the start of each named phase of a scripted run; see [BenchRun]. */
@Synchronized
fun markPhase(name: String) {
phaseMarks += PhaseMark(name, total.size, System.currentTimeMillis())
}
@Synchronized
fun add(metrics: FrameMetrics) {
// The first frame after a window opens includes inflating it and is nobody's scroll.
@@ -69,6 +84,7 @@ object FrameStats {
listOf(total, waited, input, animation, layout, draw, sync, issue, swap, gpu).forEach {
it.clear()
}
phaseMarks.clear()
since = System.currentTimeMillis()
}
@@ -95,6 +111,38 @@ object FrameStats {
) + if (gpu.isEmpty()) emptyList() else listOf(phase("gpu ", gpu))
}
/**
* One block per [markPhase] call: how many frames landed between that mark and the next (or the
* end of the run, for the last one), how many were late, the total/p50/p90/p99, the worst
* single frame, and how long the phase actually ran. Marks with no frames between them (a phase
* that finished before a frame was drawn) still get a line rather than being silently dropped
* -- UI_RULES' "say what you don't know" applies to a phase as much as to a single number.
*/
@Synchronized
fun phaseLines(refreshHz: Float): List<String> {
if (phaseMarks.isEmpty()) return emptyList()
val budget = if (refreshHz > 0) 1000.0 / refreshHz else 16.7
val lines = ArrayList<String>()
phaseMarks.forEachIndexed { i, mark ->
val endIndex = if (i + 1 < phaseMarks.size) phaseMarks[i + 1].startIndex else total.size
val endMs =
if (i + 1 < phaseMarks.size) phaseMarks[i + 1].startMs
else System.currentTimeMillis()
val samples = total.subList(mark.startIndex, endIndex)
val seconds = (endMs - mark.startMs) / 1000.0
lines += " ${mark.name}: ${samples.size} frames over ${"%.1f".format(seconds)}s"
if (samples.isEmpty()) {
lines += " no frames recorded in this phase"
} else {
val late = samples.count { it / 1_000_000.0 > budget }
lines += " late: $late (${percent(late, samples.size)})"
lines += " " + phase("total ", samples)
lines += " worst ${"%.1fms".format(samples.max() / 1_000_000.0)}"
}
}
return lines
}
/** How long the frames recorded here spent in their draw phase, and how many there were. */
@Synchronized fun drawPhase(): Pair<Long, Int> = draw.sum() to draw.size
@@ -187,13 +187,20 @@ class MainActivity : ComponentActivity() {
model = null,
keepsOwnTranscript = false,
permissionMode = null,
effort = null,
takesEffort = false,
imported = false,
notify = false,
autoResume = false,
autoResumeMessage = "",
resumeAt = null,
cwd = null,
contextTokens = null,
maxImageEdge = null,
usageProvider = null,
status = "idle",
lastActivity = 0.0,
subagents = 0,
)
// launchMode="singleTop": an enrollment scan, or a notification tapped while the app is open,
@@ -12,6 +12,7 @@ import androidx.activity.result.PickVisualMediaRequest
import androidx.activity.result.contract.ActivityResultContracts
import androidx.compose.foundation.background
import androidx.compose.foundation.clickable
import androidx.compose.foundation.gestures.ScrollableDefaults
import androidx.compose.foundation.gestures.awaitEachGesture
import androidx.compose.foundation.gestures.awaitFirstDown
import androidx.compose.foundation.layout.Box
@@ -64,12 +65,15 @@ import androidx.compose.runtime.snapshots.Snapshot
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.draw.drawWithContent
import androidx.compose.ui.focus.FocusRequester
import androidx.compose.ui.focus.focusRequester
import androidx.compose.ui.graphics.graphicsLayer
import androidx.compose.ui.input.pointer.PointerEventPass
import androidx.compose.ui.input.pointer.pointerInput
import androidx.compose.ui.layout.onSizeChanged
import androidx.compose.ui.platform.LocalContext
import androidx.compose.ui.platform.LocalDensity
import androidx.compose.ui.platform.LocalView
import androidx.compose.ui.semantics.contentDescription
import androidx.compose.ui.semantics.semantics
import androidx.compose.ui.text.TextRange
@@ -354,6 +358,17 @@ fun SessionScreen(
// `rememberSaveable`, and this screen restores by its own anchor instead -- two restores would
// fight over the first frame.
val listState = remember(address) { LazyListState() }
// The list's own fling path -- what a real flick decays through -- captured here so BenchRun's
// fling phase can drive `LazyListState.scroll` through exactly the `FlingBehavior` this
// screen's
// `TranscriptList` already uses by not overriding it (its `LazyColumn` takes no `flingBehavior`
// argument, so this is the same default it gets).
val flingBehavior = ScrollableDefaults.flingBehavior()
// Where BenchRun's type phase focuses before it types, and the view it toggles the keyboard on
// -- both bench-only, but cheap enough (a remembered object, a CompositionLocal read) to hold
// unconditionally rather than behind a second code path only the bench build compiles.
val composerFocus = remember { FocusRequester() }
val view = LocalView.current
// Whether the newest message is on screen right now. The list is reversed, so the newest end is
// the scrolling start: nothing behind you is exactly being at the bottom. Asked of the scroll
// state rather than of item indices, because a zero-height first item makes an index ambiguous.
@@ -1256,6 +1271,10 @@ fun SessionScreen(
FrameStats.drawPhase().let { (nanos, count) -> drawAccounting(nanos, count) },
crash = lastCrash(context),
extra = extra,
// Empty outside a BenchRun.run pass -- copyRenderReport's own reset below clears
// the
// marks along with everything else, so an ordinary copy never has any to show.
phaseFrames = FrameStats.phaseLines(context.refreshHz()),
)
context.copyToClipboard("ai-app render report", report)
// Also to the log, so a session driving the app over adb can read the same report the
@@ -1270,15 +1289,24 @@ fun SessionScreen(
Toast.makeText(context, "Copied render report", Toast.LENGTH_SHORT).show()
}
val copyRenderReport = { buildAndCopyReport() }
// Bench build only: P0's scripted scroll-and-stream benchmark (BenchRun.kt), against the
// fixture session opened below instead of a real server. Null everywhere else -- see
// Bench build only: P0's scripted fling/stream/type/keyboard benchmark (BenchRun.kt), against
// the fixture session opened below instead of a real server. Null everywhere else -- see
// [SessionSettingsDialog]'s onRunBenchmark.
val runBenchmark: (() -> Unit)? =
if (BuildConfig.FIXTURE_MODE) {
{
settingsOpen = false
scope.launch {
val extra = BenchRun.run(context, scope, listState)
val extra =
BenchRun.run(
context = context,
scope = scope,
listState = listState,
flingBehavior = flingBehavior,
composerFocus = composerFocus,
setComposerText = { text -> input = atEnd(text) },
view = view,
)
buildAndCopyReport(extra)
}
}
@@ -1777,7 +1805,10 @@ fun SessionScreen(
input = it
saveDraft(context, summary.id, it.text)
},
modifier = Modifier.fillMaxWidth(),
// BenchRun's type phase requests focus on this exact field
// (`composerFocus`)
// so it types through the real composer rather than a stand-in.
modifier = Modifier.fillMaxWidth().focusRequester(composerFocus),
// No longer "(+image)": the images are on screen above this, and a
// placeholder saying so said it in words beside the thing itself.
placeholder = { Text("Message") },