From b3070f16ff6aba8e9df3923c6d7c71903a5da28a Mon Sep 17 00:00:00 2001 From: iris <2+iris@noreply.localhost> Date: Sun, 30 Aug 2026 19:49:59 -0400 Subject: [PATCH] Keep the transcript ahead of the reader, and off the thread that draws Three things, all of them the same complaint: scrolling back through a long session stalls. **The fold was on the main thread.** Only `fetchTranscript` was inside `withContext(Dispatchers.IO)`; the fold loop that turns a page into rows ran on the caller's dispatcher, which is Main. `foldEvent` returns a new list per event, so a page is that many copies of a list growing to that length -- about three hundred thousand element copies -- run in the middle of the scroll that asked for it. Affordable at 80 events per page and not at 800. **`warm` scanned the whole transcript on the calling thread.** Only `replies.warm` was off it; the `markdownIn` split that decides *what* to parse ran before the hop, over every assistant message loaded, on every page. The scan grew with the conversation while the work it found stayed one page's worth. **The cushion was eight rows, which is not a distance.** A row is anything from one line to a page: on a tool-heavy transcript eight rows is less than one screen, so the reader reached the end of what was loaded on every swipe and waited a round trip standing there. It is three screenfuls now, measured from what is actually on screen. On the emulator against a 24,000-event transcript that is 3 page fetches for 10 swipes rather than 10. Also a spinner while a chat loads. Nothing is drawn while the newest page is in flight or a saved position is being put back, and a blank page is what this screen otherwise means by "there is nothing here" -- so the state that does not know needed its own appearance. Co-Authored-By: Claude Opus 5 --- .../kotlin/com/example/aiapp/SessionScreen.kt | 130 +++++++++++++----- 1 file changed, 95 insertions(+), 35 deletions(-) diff --git a/app/androidApp/src/main/kotlin/com/example/aiapp/SessionScreen.kt b/app/androidApp/src/main/kotlin/com/example/aiapp/SessionScreen.kt index fed289c..2a8445d 100644 --- a/app/androidApp/src/main/kotlin/com/example/aiapp/SessionScreen.kt +++ b/app/androidApp/src/main/kotlin/com/example/aiapp/SessionScreen.kt @@ -15,6 +15,7 @@ import androidx.compose.foundation.layout.fillMaxSize import androidx.compose.foundation.layout.fillMaxWidth import androidx.compose.foundation.layout.height import androidx.compose.foundation.layout.padding +import androidx.compose.foundation.layout.size import androidx.compose.foundation.layout.width import androidx.compose.foundation.lazy.LazyColumn import androidx.compose.foundation.lazy.items @@ -71,17 +72,31 @@ import kotlinx.coroutines.withContext private const val RECONNECT_DELAY_MS = 1500L /** - * How many rows to keep loaded past the oldest one on screen. + * How big the "still loading this conversation" spinner is. + * + * Bigger than the ones inside a tool card, which are 16dp and report on one call among many, and + * smaller than a splash: this one is standing in for the whole screen while there is nothing else + * on it, and it is the only thing to look at. + */ +private val LOADING_SPINNER = 48.dp + +/** + * How much history to keep loaded past the oldest row on screen, counted in screenfuls. * * Both the point at which history starts loading and how much of it a load has to produce before it - * stops. A cushion rather than a page count because a page is measured in events and this list is - * measured in rows, and the two are not close: a page of eighty events can be one message. + * stops. Measured against what is on screen rather than written down as a number of rows, because a + * row is anything from one line to a page and a fixed count is therefore a distance only by + * accident. Eight rows was the number, and on a tool-heavy transcript eight rows is less than one + * screen: the reader reached the end of what was loaded on *every* swipe and waited a round trip + * standing there. That is what "scrolling is laggy" turned out to be -- not a slow frame, but the + * list running out of transcript, which the emulator showed as a swipe that moved nothing for 689ms + * and then jumped. * - * Small enough that opening a long session still costs one page, large enough that a fling upwards - * lands on rows that are already there. Fewer, and reading back means waiting for the network at - * every screenful, which is what it did. + * Three, so a fling lands on rows that are already there and the page after them is on its way. The + * cost of being generous is a page fetched that nobody reads; the cost of being mean is a list that + * stops under a finger, and those are not the same size. */ -private const val HISTORY_LOOKAHEAD = 8 +private const val HISTORY_SCREENS = 3 /** * How many events a backwards page asks for, which is ten times what the opening page takes. @@ -89,8 +104,8 @@ private const val HISTORY_LOOKAHEAD = 8 * Because an event is not a row, and the ratio is nothing like one to one. Measured on a real * transcript (2,426 events, 2026-08-30): the whole conversation is *seven* assistant messages, and * the median run of consecutive text deltas that fold into one of them is four hundred. A page of - * eighty is therefore a fifth of a single row, and reaching [HISTORY_LOOKAHEAD] fresh rows took - * about thirty sequential round trips inside one collect -- a stutter on loopback, and four or five + * eighty is therefore a fifth of a single row, and reaching a screenful of fresh rows took about + * thirty sequential round trips inside one collect -- a stutter on loopback, and four or five * seconds of a list that will not move over the tunnel, which reads as history having run out. * * The opening page stays small: it is the one on the critical path of showing the screen at all, @@ -565,9 +580,15 @@ private fun updateTool( * [ParsedReplies]. */ private suspend fun warm(replies: ParsedReplies, rows: List) { - val texts = rows.filterIsInstance().flatMap { markdownIn(it.text) } - if (texts.isEmpty()) return - withContext(Dispatchers.Default) { replies.warm(texts) } + // Including the search for what to parse, which is not the cheap half it looks like: + // [markdownIn] splits every assistant message looking for memory notes, and this is handed + // the *whole* loaded transcript on every page, so the scan grows with the conversation while + // the work it finds stays one page's worth. Off the calling thread it is nobody's frame. + withContext(Dispatchers.Default) { + val texts = + rows.filterIsInstance().flatMap { markdownIn(it.text) } + if (texts.isNotEmpty()) replies.warm(texts) + } } @Composable @@ -849,25 +870,44 @@ fun SessionScreen(settings: ServerSettings, summary: SessionSummary, onBack: () * value, which does not change under a running one. */ suspend fun loadOlderPage(): Boolean { - val older = + // The fetch *and* the fold, both off the thread that draws. Only the fetch used to be, + // and the fold is the expensive half: `foldEvent` returns a new list per event, so a page + // of [HISTORY_PAGE] events is that many copies of a list growing to that length -- around + // three hundred thousand element copies for one page, run on the main thread in the + // middle of the scroll that asked for it. It was affordable at eighty events and is not + // at eight hundred, which is why the page that made scrolling back reach the top made it + // stutter to get there. + // + // `Dispatchers.IO` for both rather than a hop to `Default` between them: the two are one + // errand, and this way the page costs one context switch instead of three. Neither half + // touches anything the composition owns -- `older` and `earlier` are local, and the + // `items` read below happens back on the caller's thread, where the write does too. + val page = withContext(Dispatchers.IO) { - fetchTranscript(settings, summary.id, before = oldestSeq, limit = HISTORY_PAGE) + val older = + fetchTranscript(settings, summary.id, before = oldestSeq, limit = HISTORY_PAGE) + if (older.isEmpty()) return@withContext null + // Folded oldest-first into a list of their own, then put in front: `foldEvent` + // merges streaming text into the item before it, so replaying an older page + // through the live list would glue it onto the newest message rather than its own. + var earlier = listOf() + older.forEach { entry -> + if (entry.event !is SessionEvent.UsageDelta) { + earlier = foldEvent(earlier, entry) + } + } + older.first().seq to earlier } - if (older.isEmpty()) { + if (page == null) { moreHistory = false return false } - oldestSeq = older.first().seq + val (oldest, earlier) = page + oldestSeq = oldest moreHistory = oldestSeq > 1L - // Folded oldest-first into a list of their own, then put in front: `foldEvent` merges - // streaming text into the item before it, so replaying an older page through the live - // list would glue it onto the newest message rather than its own. - var earlier = listOf() - older.forEach { entry -> - if (entry.event !is SessionEvent.UsageDelta) { - earlier = foldEvent(earlier, entry) - } - } + // Joined here rather than above, because it is the one step that reads what is already + // loaded: `items` must be read where it is written, and it is a single pass over the two + // lists against the page's quadratic fold. val joined = joinPages(earlier, items) // After the join rather than on the page alone: a boundary that fell through a reply // leaves `joinPages` holding a message made of both halves, and that text has existed for @@ -1139,26 +1179,34 @@ fun SessionScreen(settings: ServerSettings, summary: SessionSummary, onBack: () LaunchedEffect(listState, rows.size, moreHistory) { snapshotFlow { val layout = listState.layoutInfo - Pair(layout.visibleItemsInfo.lastOrNull()?.index ?: 0, layout.totalItemsCount) + // How many rows are on screen is the third thing this needs, and it is what turns a + // cushion measured in rows into one measured in screens; see [HISTORY_SCREENS]. + Triple( + layout.visibleItemsInfo.lastOrNull()?.index ?: 0, + layout.totalItemsCount, + layout.visibleItemsInfo.size, + ) } - .collect { (last, total) -> + .collect { (last, total, onScreen) -> if (!moreHistory || loadingHistory || total == 0) return@collect - if (last < total - HISTORY_LOOKAHEAD) return@collect + val cushion = onScreen.coerceAtLeast(1) * HISTORY_SCREENS + if (last < total - cushion) return@collect loadingHistory = true try { // Pages until there are rows behind them again, not one page and stop. // - // A page is eighty *events*, and eighty events are routinely one row: a - // reply arrives as hundreds of text deltas that fold into a single message. - // So a page that lands can leave the far end exactly where it was -- and - // since this is triggered by the far end moving, nothing asks for the next - // one. The list then only loads when somebody drags it again, a page at a - // time, which is what "it only loads when you touch the top" was. + // A page is eight hundred *events*, and events are not rows: a reply arrives + // as hundreds of text deltas that fold into a single message, and a run of + // thirty-five tool calls is one row. So a page that lands can leave the far + // end almost exactly where it was -- and since this is triggered by the far + // end moving, nothing asks for the next one. The list then only loads when + // somebody drags it again, a page at a time, which is what "it only loads + // when you touch the top" was, and later what the wait at every swipe was. // Counted from `items` rather than from `rows`, which is the // composition's value and does not change under a running coroutine. val start = groupToolRuns(items).size var have = start - while (moreHistory && have - start < HISTORY_LOOKAHEAD && loadOlderPage()) { + while (moreHistory && have - start < cushion && loadOlderPage()) { have = groupToolRuns(items).size } } catch (_: ApiException) { @@ -1545,6 +1593,18 @@ fun SessionScreen(settings: ServerSettings, summary: SessionSummary, onBack: () } } + // Still finding out what this conversation is: the newest page has not arrived, or + // it has and the list is being put back where reading stopped. Both draw no rows at + // all, and a blank page is what this screen otherwise means by "there is nothing + // here" -- so the state that does not know needs its own appearance rather than + // sharing one with the empty answer. + // + // In the middle of the transcript rather than at either end, because it is not + // reporting on the newest message or the oldest; it is standing in for all of them. + if (!ready || restoring) { + CircularProgressIndicator(Modifier.align(Alignment.Center).size(LOADING_SPINNER)) + } + // Only while the newest message is off-screen. Reading back // through a conversation is a place to be, not a state to be // rescued from, so this waits to be wanted.