irisandClaude Fable 5.1
a8d24553d5
Iris's ask after using the Compose bench build on her phone: the old scroll phase used animateScrollBy, which can only ever cover the fixed distance/time it's given, so it never flings the way a real fast swipe does. BenchRun.run now has four phases: fling (8 flings out + 8 back through the list's own FlingBehavior at 12,000px/s), stream (unchanged), type (600 fixed characters into the real composer TextFieldValue, then deleted, to exercise wrapping and the transcript being pushed upward), and keyboard (five show/hide cycles via WindowInsetsControllerCompat, each confirmed by isImeVisible rather than assumed). FrameStats.markPhase/phaseLines slice the same FrameMetrics recording by phase rather than running a second recorder; debugReport gains a phaseFrames section ahead of the existing whole-run frames/accounting/ work sections, which are otherwise unchanged. Also fixes a pre-existing, unrelated break in MainActivity.kt's benchSessionSummary() -- missing several SessionSummary constructor arguments from an earlier change -- since it blocked compileBenchKotlin outright. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
326 lines
14 KiB
Kotlin
326 lines
14 KiB
Kotlin
package com.example.aiapp
|
|
|
|
import android.content.Context
|
|
import android.os.BatteryManager
|
|
import android.os.Process
|
|
import android.view.View
|
|
import androidx.compose.foundation.gestures.FlingBehavior
|
|
import androidx.compose.foundation.lazy.LazyListState
|
|
import androidx.compose.ui.focus.FocusRequester
|
|
import androidx.core.view.ViewCompat
|
|
import androidx.core.view.WindowInsetsCompat
|
|
import androidx.core.view.WindowInsetsControllerCompat
|
|
import java.io.File
|
|
import kotlinx.coroutines.CoroutineScope
|
|
import kotlinx.coroutines.delay
|
|
import kotlinx.coroutines.isActive
|
|
import kotlinx.coroutines.launch
|
|
|
|
/**
|
|
* P0's scripted benchmark, run in-process instead of by a shell script: the phone has no usable
|
|
* system tracing (this-machine-android's skill) and no agent can drive it, so the same scroll loop
|
|
* and streaming phase `transcript-bench.sh`/`stream-bench.sh` drive over `ui-trace` are reproduced
|
|
* here against [LazyListState] and [BenchFixture] directly. Only reachable from the `bench` build
|
|
* (see [SessionSettingsDialog]'s `onRunBenchmark`), but compiled into every build for the reason
|
|
* [BenchFixture]'s doc comment gives.
|
|
*
|
|
* **v2 (2026-09-06)**, asked for by Iris because the v1 fling was too gentle to stress-test the
|
|
* scroll path and said nothing about typing or the keyboard. Four phases now, each a slice of the
|
|
* same [FrameStats] recording ([FrameStats.markPhase]/[FrameStats.phaseLines] -- one recorder, not
|
|
* two): **fling** (real `FlingBehavior`, not `animateScrollBy`), **stream** (unchanged from v1),
|
|
* **type** (600 fixed characters into the real composer `TextFieldValue`, then deleted), and
|
|
* **keyboard** (five show/hide cycles). The exact constants below are also written into
|
|
* `docs/RUST.md`'s P0 box, "Benchmark v2 (2026-09-06)", so the iris half implements the identical
|
|
* spec -- changing a number here without updating that box makes the two apps measure different
|
|
* things while looking like the same benchmark.
|
|
*/
|
|
object BenchRun {
|
|
/** transcript-bench.sh's default: 6 cycles of 4 swipes each, kept as the pre-v2 comparison. */
|
|
private const val CYCLES = 6
|
|
private const val SWIPE_PX = 900f
|
|
private const val SWIPE_MS = 200
|
|
private const val SWIPE_PAUSE_MS = 500L
|
|
|
|
/**
|
|
* Fling phase (v2): a real fling through the list's own [FlingBehavior], not `animateScrollBy`
|
|
* -- Iris's ask was that it "travel way faster" than the old tween-based swipe, and a tween can
|
|
* never exceed the distance it is told to cover in the time it is given, while a real fling
|
|
* decays from an initial velocity the way a finger flick does. 12,000 px/s is roughly a hard,
|
|
* fast flick on a ~420dp/in device (about 30 dp/ms-equivalent initial speed); chosen well above
|
|
* the ~4,500 px/s a moderate `animateScrollBy` swipe implies, so this phase exercises the fast
|
|
* end of what the platform's fling decay produces rather than the gentle one v1 measured.
|
|
*/
|
|
private const val FLING_VELOCITY_PX_S = 12_000f
|
|
|
|
private const val FLING_COUNT = 8
|
|
private const val FLING_SETTLE_CAP_MS = 3_000L
|
|
private const val FLING_PAUSE_MS = 300L
|
|
|
|
/** stream-bench.sh's shape: a real reply arrives as many small deltas, not one big write. */
|
|
private const val STREAM_EVENTS_PER_SEC = 20
|
|
private const val STREAM_SECONDS = 20
|
|
|
|
/**
|
|
* Type phase (v2): sentences built from long, multisyllabic words so the composer actually
|
|
* wraps across lines rather than fitting one, and long enough (600 chars) that the composer's
|
|
* own height grows over several frames, pushing the transcript above it upward the same way a
|
|
* real long message does. Exactly this string is also in `docs/RUST.md`'s P0 box so the iris
|
|
* half types the identical content.
|
|
*/
|
|
const val TYPE_TEXT =
|
|
"Benchmarking this transcript screen requires unusually long, multisyllabic words so " +
|
|
"wrapping and reflow are properly exercised: internationalization, " +
|
|
"counterproductiveness, disproportionately, incomprehensibility, " +
|
|
"deinstitutionalization, uncharacteristically, overenthusiastically, " +
|
|
"misunderstanding, straightforwardness, telecommunications, and interdisciplinary " +
|
|
"collaboration all push a narrow composer field to wrap across several lines while " +
|
|
"the transcript above is pushed upward by the growing keyboard-adjacent box, which " +
|
|
"is exactly what a real reader typing a long message sees happening now!!!"
|
|
|
|
private const val TYPE_CHAR_DELAY_MS = 50L
|
|
|
|
/**
|
|
* Keyboard phase (v2): five show/hide cycles, a second apart, is enough to see whether the
|
|
* transition is ever actually observed rather than being a one-off fluke either way.
|
|
*/
|
|
private const val KEYBOARD_CYCLES = 5
|
|
private const val KEYBOARD_SHOW_WAIT_MS = 1_000L
|
|
private const val KEYBOARD_HIDE_WAIT_MS = 1_000L
|
|
|
|
/**
|
|
* Scrolls, flings, streams, types and toggles the keyboard, then returns the extra report lines
|
|
* P0 asked for (per-phase travel/typing/keyboard counts, plus CPU time, peak RSS, battery
|
|
* current) -- [FrameStats] and [DebugStats] are reset first, exactly as `copyRenderReport`
|
|
* resets them, so the two accountings cover the same stretch of work.
|
|
*/
|
|
suspend fun run(
|
|
context: Context,
|
|
scope: CoroutineScope,
|
|
listState: LazyListState,
|
|
flingBehavior: FlingBehavior,
|
|
composerFocus: FocusRequester,
|
|
setComposerText: (String) -> Unit,
|
|
view: View,
|
|
): List<String> {
|
|
FrameStats.reset()
|
|
DebugStats.reset()
|
|
val cpuStartMs = Process.getElapsedCpuTime()
|
|
|
|
val battery = BatterySampler(context)
|
|
// Launched in the caller's scope rather than a fresh coroutineScope{} here, which would
|
|
// suspend this function until the sampler job ended -- and it only ends when told to.
|
|
val samplerJob = scope.launch {
|
|
while (isActive) {
|
|
battery.sample()
|
|
delay(1000)
|
|
}
|
|
}
|
|
|
|
val travel = runFlingPhase(listState, flingBehavior)
|
|
val sent = runStreamPhase()
|
|
runTypePhase(listState, composerFocus, setComposerText, view)
|
|
val keyboard = runKeyboardPhase(context, view)
|
|
|
|
samplerJob.cancel()
|
|
val cpuMs = Process.getElapsedCpuTime() - cpuStartMs
|
|
val rssLine = peakRssLine()
|
|
val batteryLine = battery.finish()
|
|
|
|
return listOf(
|
|
" fling: $FLING_COUNT flings out + $FLING_COUNT back at" +
|
|
" ${FLING_VELOCITY_PX_S.toInt()}px/s, travel $travel",
|
|
" scroll: $CYCLES cycles (${CYCLES * 4} swipes, legacy tween), " +
|
|
"streamed $sent/${STREAM_EVENTS_PER_SEC * STREAM_SECONDS} fixture events",
|
|
" type: ${TYPE_TEXT.length} characters inserted then deleted, one per" +
|
|
" ${TYPE_CHAR_DELAY_MS}ms",
|
|
keyboard,
|
|
" process CPU time over this run: ${cpuMs}ms",
|
|
rssLine,
|
|
batteryLine,
|
|
)
|
|
}
|
|
|
|
/**
|
|
* Phase 1: starting pinned at the newest end, [FLING_COUNT] flings away from it (toward older
|
|
* messages) through the list's real fling path, then [FLING_COUNT] back. Positive velocity here
|
|
* matches this list's existing scroll-offset convention (`TranscriptList`'s `reverseLayout`
|
|
* pins index 0 -- the newest item -- at the bottom; a positive scroll offset moves the viewport
|
|
* toward higher indices, i.e. away from the newest end and toward older content), the same sign
|
|
* the pre-v2 swipe loop below already used for its first two swipes.
|
|
*/
|
|
private suspend fun runFlingPhase(
|
|
listState: LazyListState,
|
|
flingBehavior: FlingBehavior,
|
|
): String {
|
|
FrameStats.markPhase("fling")
|
|
listState.scrollToItem(0)
|
|
val start = position(listState)
|
|
repeat(FLING_COUNT) {
|
|
listState.scroll { with(flingBehavior) { performFling(FLING_VELOCITY_PX_S) } }
|
|
waitForSettle(listState)
|
|
delay(FLING_PAUSE_MS)
|
|
}
|
|
val outward = position(listState)
|
|
repeat(FLING_COUNT) {
|
|
listState.scroll { with(flingBehavior) { performFling(-FLING_VELOCITY_PX_S) } }
|
|
waitForSettle(listState)
|
|
delay(FLING_PAUSE_MS)
|
|
}
|
|
val back = position(listState)
|
|
return "start=$start outward=$outward end=$back"
|
|
}
|
|
|
|
private fun position(listState: LazyListState) =
|
|
"idx=${listState.firstVisibleItemIndex}/off=${listState.firstVisibleItemScrollOffset}px"
|
|
|
|
/** Belt-and-suspenders on top of `performFling` already suspending until its own decay ends. */
|
|
private suspend fun waitForSettle(listState: LazyListState) {
|
|
val startedAt = System.currentTimeMillis()
|
|
while (
|
|
listState.isScrollInProgress &&
|
|
System.currentTimeMillis() - startedAt < FLING_SETTLE_CAP_MS
|
|
) {
|
|
delay(16)
|
|
}
|
|
}
|
|
|
|
/**
|
|
* Phase 2 (unchanged from v1): pinned to the newest end before streaming starts, the way
|
|
* stream-bench.sh's "Jump to latest" tap is -- a reply streamed into a list parked further back
|
|
* arrives off-screen and the report would show nothing happened.
|
|
*/
|
|
private suspend fun runStreamPhase(): Int {
|
|
FrameStats.markPhase("stream")
|
|
var sent = 0
|
|
val total = STREAM_EVENTS_PER_SEC * STREAM_SECONDS
|
|
while (sent < total && BenchFixture.remainingStreamEvents() > 0) {
|
|
BenchFixture.pushNextLiveEvent()
|
|
sent++
|
|
delay(1000L / STREAM_EVENTS_PER_SEC)
|
|
}
|
|
// Lets the last few deltas land and draw before the next phase starts.
|
|
delay(300)
|
|
return sent
|
|
}
|
|
|
|
/**
|
|
* Phase 3: focuses the real composer, shows the keyboard if the platform allows it, then types
|
|
* [TYPE_TEXT] one character at a time through the same `TextFieldValue` state a real keystroke
|
|
* updates, and deletes it the same way -- this is what exercises wrapping and the transcript
|
|
* being pushed upward, not a single big write.
|
|
*/
|
|
private suspend fun runTypePhase(
|
|
listState: LazyListState,
|
|
composerFocus: FocusRequester,
|
|
setComposerText: (String) -> Unit,
|
|
view: View,
|
|
) {
|
|
FrameStats.markPhase("type")
|
|
listState.scrollToItem(0)
|
|
composerFocus.requestFocus()
|
|
showIme(view.context, view)
|
|
// Lets focus and the keyboard's opening animation land before typing starts, so the frames
|
|
// this phase records are the wrap/reflow it is measuring, not the keyboard opening.
|
|
delay(300)
|
|
var typed = ""
|
|
for (ch in TYPE_TEXT) {
|
|
typed += ch
|
|
setComposerText(typed)
|
|
delay(TYPE_CHAR_DELAY_MS)
|
|
}
|
|
delay(200)
|
|
while (typed.isNotEmpty()) {
|
|
typed = typed.dropLast(1)
|
|
setComposerText(typed)
|
|
delay(TYPE_CHAR_DELAY_MS)
|
|
}
|
|
}
|
|
|
|
/**
|
|
* Phase 4: [KEYBOARD_CYCLES] show/hide cycles through the same [WindowInsetsControllerCompat]
|
|
* path a real IME toggle goes through, reporting how many of each were actually confirmed by
|
|
* [android.view.WindowInsets.isVisible] rather than assumed from having asked -- UI_RULES:
|
|
* never present an inferred value as a measured one. If the platform never shows it even once,
|
|
* this says so in words rather than reporting a phase with no keyboard in it.
|
|
*/
|
|
private suspend fun runKeyboardPhase(context: Context, view: View): String {
|
|
FrameStats.markPhase("keyboard")
|
|
var shown = 0
|
|
var hidden = 0
|
|
repeat(KEYBOARD_CYCLES) {
|
|
showIme(context, view)
|
|
delay(KEYBOARD_SHOW_WAIT_MS)
|
|
if (imeVisible(view)) shown++
|
|
hideIme(context, view)
|
|
delay(KEYBOARD_HIDE_WAIT_MS)
|
|
if (!imeVisible(view)) hidden++
|
|
}
|
|
return if (shown == 0) {
|
|
" keyboard: could not be shown ($KEYBOARD_CYCLES attempts, 0 confirmed visible)"
|
|
} else {
|
|
" keyboard: shown $shown/$KEYBOARD_CYCLES, hidden $hidden/$KEYBOARD_CYCLES" +
|
|
" (confirmed via isImeVisible)"
|
|
}
|
|
}
|
|
|
|
private fun controller(context: Context, view: View): WindowInsetsControllerCompat? {
|
|
val window = context.activity()?.window ?: return null
|
|
return WindowInsetsControllerCompat(window, view)
|
|
}
|
|
|
|
private fun showIme(context: Context, view: View) {
|
|
controller(context, view)?.show(WindowInsetsCompat.Type.ime())
|
|
}
|
|
|
|
private fun hideIme(context: Context, view: View) {
|
|
controller(context, view)?.hide(WindowInsetsCompat.Type.ime())
|
|
}
|
|
|
|
private fun imeVisible(view: View): Boolean =
|
|
ViewCompat.getRootWindowInsets(view)?.isVisible(WindowInsetsCompat.Type.ime()) ?: false
|
|
|
|
/** VmHWM from /proc/self/status: the process's high-water mark, in kB, since it started. */
|
|
private fun peakRssLine(): String {
|
|
val kb =
|
|
try {
|
|
File("/proc/self/status")
|
|
.readLines()
|
|
.firstOrNull { it.startsWith("VmHWM:") }
|
|
?.trim()
|
|
?.removePrefix("VmHWM:")
|
|
?.trim()
|
|
?.removeSuffix("kB")
|
|
?.trim()
|
|
?.toLongOrNull()
|
|
} catch (_: Exception) {
|
|
null
|
|
}
|
|
return " peak RSS: " +
|
|
(kb?.let { "${it}kB" } ?: "unavailable (/proc/self/status unreadable)")
|
|
}
|
|
}
|
|
|
|
/**
|
|
* Samples [BatteryManager.BATTERY_PROPERTY_CURRENT_NOW] (microamps) once a second for the length of
|
|
* a run. The property returns `Int.MIN_VALUE` on hardware that does not support it -- most
|
|
* emulators -- and that is reported as "unavailable" rather than folded into an average with the
|
|
* real samples, which would silently understate every number after it. See UI_RULES: never present
|
|
* an inferred value as a measured one.
|
|
*/
|
|
private class BatterySampler(context: Context) {
|
|
private val manager = context.getSystemService(BatteryManager::class.java)
|
|
private val samples = mutableListOf<Int>()
|
|
|
|
fun sample() {
|
|
val value = manager?.getIntProperty(BatteryManager.BATTERY_PROPERTY_CURRENT_NOW)
|
|
if (value != null && value != Int.MIN_VALUE) samples.add(value)
|
|
}
|
|
|
|
fun finish(): String {
|
|
if (samples.isEmpty()) return " battery current: unavailable on this device"
|
|
val meanUa = samples.sum() / samples.size
|
|
return " battery current: mean ${meanUa}µA over ${samples.size} samples" +
|
|
" (min ${samples.min()}, max ${samples.max()})"
|
|
}
|
|
}
|