iris/ai-app
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Keep the transcript ahead of the reader, and off the thread that draws

Browse Source 
Three things, all of them the same complaint: scrolling back through a long
session stalls.

**The fold was on the main thread.** Only `fetchTranscript` was inside
`withContext(Dispatchers.IO)`; the fold loop that turns a page into rows ran
on the caller's dispatcher, which is Main. `foldEvent` returns a new list
per event, so a page is that many copies of a list growing to that length --
about three hundred thousand element copies -- run in the middle of the
scroll that asked for it. Affordable at 80 events per page and not at 800.

**`warm` scanned the whole transcript on the calling thread.** Only
`replies.warm` was off it; the `markdownIn` split that decides *what* to
parse ran before the hop, over every assistant message loaded, on every
page. The scan grew with the conversation while the work it found stayed one
page's worth.

**The cushion was eight rows, which is not a distance.** A row is anything
from one line to a page: on a tool-heavy transcript eight rows is less than
one screen, so the reader reached the end of what was loaded on every swipe
and waited a round trip standing there. It is three screenfuls now, measured
from what is actually on screen. On the emulator against a 24,000-event
transcript that is 3 page fetches for 10 swipes rather than 10.

Also a spinner while a chat loads. Nothing is drawn while the newest page is
in flight or a saved position is being put back, and a blank page is what
this screen otherwise means by "there is nothing here" -- so the state that
does not know needed its own appearance.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
irisandClaude Opus 5 committed 2026-08-30 19:49:59 -04:00
1 parent e76617e210
commit b3070f16ff
1 file changed
+93 -33
Download Patch File Download Diff File Expand all files Collapse all files
app/androidApp/src/main/kotlin/com/example/aiapp/SessionScreen.kt
+93 -33
View File
@@ -15,6 +15,7 @@ import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.height
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.layout.size
import androidx.compose.foundation.layout.width
import androidx.compose.foundation.lazy.LazyColumn
import androidx.compose.foundation.lazy.items
@@ -71,17 +72,31 @@ import kotlinx.coroutines.withContext
private const val RECONNECT_DELAY_MS = 1500L
/**
* How many rows to keep loaded past the oldest one on screen.
* How big the "still loading this conversation" spinner is.
*
* Bigger than the ones inside a tool card, which are 16dp and report on one call among many, and
* smaller than a splash: this one is standing in for the whole screen while there is nothing else
* on it, and it is the only thing to look at.
*/
private val LOADING_SPINNER = 48.dp
/**
* How much history to keep loaded past the oldest row on screen, counted in screenfuls.
*
* Both the point at which history starts loading and how much of it a load has to produce before it
* stops. A cushion rather than a page count because a page is measured in events and this list is
* measured in rows, and the two are not close: a page of eighty events can be one message.
* stops. Measured against what is on screen rather than written down as a number of rows, because a
* row is anything from one line to a page and a fixed count is therefore a distance only by
* accident. Eight rows was the number, and on a tool-heavy transcript eight rows is less than one
* screen: the reader reached the end of what was loaded on *every* swipe and waited a round trip
* standing there. That is what "scrolling is laggy" turned out to be -- not a slow frame, but the
* list running out of transcript, which the emulator showed as a swipe that moved nothing for 689ms
* and then jumped.
*
* Small enough that opening a long session still costs one page, large enough that a fling upwards
* lands on rows that are already there. Fewer, and reading back means waiting for the network at
* every screenful, which is what it did.
* Three, so a fling lands on rows that are already there and the page after them is on its way. The
* cost of being generous is a page fetched that nobody reads; the cost of being mean is a list that
* stops under a finger, and those are not the same size.
*/
private const val HISTORY_LOOKAHEAD = 8
private const val HISTORY_SCREENS = 3
/**
* How many events a backwards page asks for, which is ten times what the opening page takes.
@@ -89,8 +104,8 @@ private const val HISTORY_LOOKAHEAD = 8
* Because an event is not a row, and the ratio is nothing like one to one. Measured on a real
* transcript (2,426 events, 2026-08-30): the whole conversation is *seven* assistant messages, and
* the median run of consecutive text deltas that fold into one of them is four hundred. A page of
* eighty is therefore a fifth of a single row, and reaching [HISTORY_LOOKAHEAD] fresh rows took
* about thirty sequential round trips inside one collect -- a stutter on loopback, and four or five
* eighty is therefore a fifth of a single row, and reaching a screenful of fresh rows took about
* thirty sequential round trips inside one collect -- a stutter on loopback, and four or five
* seconds of a list that will not move over the tunnel, which reads as history having run out.
*
* The opening page stays small: it is the one on the critical path of showing the screen at all,
@@ -565,9 +580,15 @@ private fun updateTool(
* [ParsedReplies].
*/
private suspend fun warm(replies: ParsedReplies, rows: List<TranscriptItem>) {
val texts = rows.filterIsInstance<TranscriptItem.AssistantMsg>().flatMap { markdownIn(it.text) }
if (texts.isEmpty()) return
withContext(Dispatchers.Default) { replies.warm(texts) }
// Including the search for what to parse, which is not the cheap half it looks like:
// [markdownIn] splits every assistant message looking for memory notes, and this is handed
// the *whole* loaded transcript on every page, so the scan grows with the conversation while
// the work it finds stays one page's worth. Off the calling thread it is nobody's frame.
withContext(Dispatchers.Default) {
val texts =
rows.filterIsInstance<TranscriptItem.AssistantMsg>().flatMap { markdownIn(it.text) }
if (texts.isNotEmpty()) replies.warm(texts)
}
}
@Composable
@@ -849,25 +870,44 @@ fun SessionScreen(settings: ServerSettings, summary: SessionSummary, onBack: ()
* value, which does not change under a running one.
*/
suspend fun loadOlderPage(): Boolean {
val older =
// The fetch *and* the fold, both off the thread that draws. Only the fetch used to be,
// and the fold is the expensive half: `foldEvent` returns a new list per event, so a page
// of [HISTORY_PAGE] events is that many copies of a list growing to that length -- around
// three hundred thousand element copies for one page, run on the main thread in the
// middle of the scroll that asked for it. It was affordable at eighty events and is not
// at eight hundred, which is why the page that made scrolling back reach the top made it
// stutter to get there.
//
// `Dispatchers.IO` for both rather than a hop to `Default` between them: the two are one
// errand, and this way the page costs one context switch instead of three. Neither half
// touches anything the composition owns -- `older` and `earlier` are local, and the
// `items` read below happens back on the caller's thread, where the write does too.
val page =
withContext(Dispatchers.IO) {
val older =
fetchTranscript(settings, summary.id, before = oldestSeq, limit = HISTORY_PAGE)
}
if (older.isEmpty()) {
moreHistory = false
return false
}
oldestSeq = older.first().seq
moreHistory = oldestSeq > 1L
// Folded oldest-first into a list of their own, then put in front: `foldEvent` merges
// streaming text into the item before it, so replaying an older page through the live
// list would glue it onto the newest message rather than its own.
if (older.isEmpty()) return@withContext null
// Folded oldest-first into a list of their own, then put in front: `foldEvent`
// merges streaming text into the item before it, so replaying an older page
// through the live list would glue it onto the newest message rather than its own.
var earlier = listOf<TranscriptItem>()
older.forEach { entry ->
if (entry.event !is SessionEvent.UsageDelta) {
earlier = foldEvent(earlier, entry)
}
}
older.first().seq to earlier
}
if (page == null) {
moreHistory = false
return false
}
val (oldest, earlier) = page
oldestSeq = oldest
moreHistory = oldestSeq > 1L
// Joined here rather than above, because it is the one step that reads what is already
// loaded: `items` must be read where it is written, and it is a single pass over the two
// lists against the page's quadratic fold.
val joined = joinPages(earlier, items)
// After the join rather than on the page alone: a boundary that fell through a reply
// leaves `joinPages` holding a message made of both halves, and that text has existed for
@@ -1139,26 +1179,34 @@ fun SessionScreen(settings: ServerSettings, summary: SessionSummary, onBack: ()
LaunchedEffect(listState, rows.size, moreHistory) {
snapshotFlow {
val layout = listState.layoutInfo
Pair(layout.visibleItemsInfo.lastOrNull()?.index ?: 0, layout.totalItemsCount)
// How many rows are on screen is the third thing this needs, and it is what turns a
// cushion measured in rows into one measured in screens; see [HISTORY_SCREENS].
Triple(
layout.visibleItemsInfo.lastOrNull()?.index ?: 0,
layout.totalItemsCount,
layout.visibleItemsInfo.size,
)
}
.collect { (last, total) ->
.collect { (last, total, onScreen) ->
if (!moreHistory || loadingHistory || total == 0) return@collect
if (last < total - HISTORY_LOOKAHEAD) return@collect
val cushion = onScreen.coerceAtLeast(1) * HISTORY_SCREENS
if (last < total - cushion) return@collect
loadingHistory = true
try {
// Pages until there are rows behind them again, not one page and stop.
//
// A page is eighty *events*, and eighty events are routinely one row: a
// reply arrives as hundreds of text deltas that fold into a single message.
// So a page that lands can leave the far end exactly where it was -- and
// since this is triggered by the far end moving, nothing asks for the next
// one. The list then only loads when somebody drags it again, a page at a
// time, which is what "it only loads when you touch the top" was.
// A page is eight hundred *events*, and events are not rows: a reply arrives
// as hundreds of text deltas that fold into a single message, and a run of
// thirty-five tool calls is one row. So a page that lands can leave the far
// end almost exactly where it was -- and since this is triggered by the far
// end moving, nothing asks for the next one. The list then only loads when
// somebody drags it again, a page at a time, which is what "it only loads
// when you touch the top" was, and later what the wait at every swipe was.
// Counted from `items` rather than from `rows`, which is the
// composition's value and does not change under a running coroutine.
val start = groupToolRuns(items).size
var have = start
while (moreHistory && have - start < HISTORY_LOOKAHEAD && loadOlderPage()) {
while (moreHistory && have - start < cushion && loadOlderPage()) {
have = groupToolRuns(items).size
}
} catch (_: ApiException) {
@@ -1545,6 +1593,18 @@ fun SessionScreen(settings: ServerSettings, summary: SessionSummary, onBack: ()
}
}
// Still finding out what this conversation is: the newest page has not arrived, or
// it has and the list is being put back where reading stopped. Both draw no rows at
// all, and a blank page is what this screen otherwise means by "there is nothing
// here" -- so the state that does not know needs its own appearance rather than
// sharing one with the empty answer.
//
// In the middle of the transcript rather than at either end, because it is not
// reporting on the newest message or the oldest; it is standing in for all of them.
if (!ready || restoring) {
CircularProgressIndicator(Modifier.align(Alignment.Center).size(LOADING_SPINNER))
}
// Only while the newest message is off-screen. Reading back
// through a conversation is a place to be, not a state to be
// rescued from, so this waits to be wanted.