Draw a reply one block at a time, and skip the blocks off screen

A reply's display list holds every glyph of it and is re-recorded
whenever drawing is invalidated, so one long message costs as much to
draw as a hundred short ones and skipping the rows around it cannot help
while it is the one on screen. That was the whole of the remaining draw
cost: 97% of rows correctly skipped, and the tallest one still drawn was
36,982px -- about twenty-five screens in a single message.

So a message is cut into its top-level blocks and each is drawn, or not,
on its own. The cut comes from the parser's own boundaries rather than
from a line scanner looking for blank lines, which is what makes it
safe: a heading, a table, a fenced block and a list are each one node
whatever is inside them, so a loose list does not become five one-item
lists and a fence is never split down the middle. Checked against a real
reply whose list items are separated by blank lines -- it still draws as
one list with its bullets aligned, which is the case a blank-line split
gets wrong.

It bounds parsing too, which was the other symptom in the same reading:
one message took 1.4 seconds to parse as a single unit, and a block is a
paragraph.

The row and the message each know half of where a block is, so they meet
at an interface declared where it is used: the list supplies the row's
position, the message supplies the block's offset inside it, and drawing
a message does not have to know it is inside a transcript.

What this cannot divide is a single node, and a long fenced code block
is one -- so the report now also carries the tallest *drawn block*,
which is the number that says whether splitting bounded anything. This
emulator's tallest row is one such fence, which is why its own figures
do not move.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

app/androidApp/src/main/kotlin/com/example/aiapp/Markdown.kt

@@ -169,6 +169,17 @@ private fun parsedMarkdown(text: String, replies: ParsedReplies): State {
 class ParsedReplies {
     private val parsed = ConcurrentHashMap<String, State>()
 
+    /**
+     * How each message divides into blocks, cached beside the parses of those blocks.
+     *
+     * Here rather than in a `remember` because the answer is wanted on two threads: by [warm], to
+     * know which strings to make ready, and by the row that draws them. Finding it costs a parse of
+     * the whole message, so doing it twice would undo what splitting is for.
+     */
+    private val blocks = ConcurrentHashMap<String, List<String>>()
+
+    fun blocksOf(text: String): List<String> = blocks.computeIfAbsent(text) { markdownBlocks(it) }
+
     /** The parse of [text] -- the one made ahead, or one made now. */
     fun of(text: String): State =
         parsed[text]?.also { DebugStats.count("markdown ready") }

app/androidApp/src/main/kotlin/com/example/aiapp/MemoryNote.kt

@@ -30,13 +30,13 @@ fun AssistantMessage(text: String, replies: ParsedReplies, modifier: Modifier =
     val parts = remember(text) { partsOf(text) }
     val only = parts.singleOrNull()
     if (only is MessagePart.Prose) {
-        MarkdownText(only.text, replies, modifier)
+        BlockedMarkdown(only.text, replies, modifier)
         return
     }
     Column(modifier.fillMaxWidth(), verticalArrangement = Arrangement.spacedBy(6.dp)) {
         parts.forEach { part ->
             when (part) {
-                is MessagePart.Prose -> MarkdownText(part.text, replies)
+                is MessagePart.Prose -> BlockedMarkdown(part.text, replies)
                 is MessagePart.Remembered -> MemoryNote(part, replies)
             }
         }

app/androidApp/src/main/kotlin/com/example/aiapp/MessageBlocks.kt

@@ -0,0 +1,150 @@
+package com.example.aiapp
+
+import androidx.compose.foundation.layout.Arrangement
+import androidx.compose.foundation.layout.Box
+import androidx.compose.foundation.layout.Column
+import androidx.compose.foundation.layout.fillMaxWidth
+import androidx.compose.runtime.Composable
+import androidx.compose.runtime.compositionLocalOf
+import androidx.compose.runtime.remember
+import androidx.compose.ui.Modifier
+import androidx.compose.ui.draw.drawWithContent
+import androidx.compose.ui.layout.onSizeChanged
+import androidx.compose.ui.platform.LocalDensity
+import androidx.compose.ui.unit.dp
+import com.mikepenz.markdown.model.State
+import com.mikepenz.markdown.model.parseMarkdown
+
+/**
+ * Whether a piece of a row is close enough to the screen to be worth drawing.
+ *
+ * Defined here, where it is needed, and implemented by the list that knows where rows are -- so
+ * drawing a message does not have to know it is inside a transcript, and a message drawn anywhere
+ * else simply draws all of itself.
+ *
+ * The offsets are the row's own: a block's top measured from the top of the message it is part of.
+ */
+interface RowWindow {
+    fun visible(top: Int, height: Int): Boolean
+}
+
+val LocalRowWindow = compositionLocalOf<RowWindow?> { null }
+
+/**
+ * A message's top-level markdown blocks, cut where the parser says the blocks are.
+ *
+ * The point is the draw phase. A reply's display list holds every glyph of it, and it is
+ * re-recorded whenever drawing is invalidated -- so one long message is as expensive to draw as a
+ * hundred short ones, and skipping the rows around it cannot help while it is the one on screen.
+ * Measured on a Pixel 9 Pro XL: 97% of rows correctly skipped, and the tallest row still being
+ * drawn was 36,982px, about twenty-five screens in a single message. Cut into blocks, only the
+ * screen or two actually being read is ever recorded.
+ *
+ * Cut at the parser's own boundaries rather than at blank lines, which is the whole reason this is
+ * safe: a heading, a fenced code block, a table and a list are each one node whatever is inside
+ * them, so a loose list does not become five one-item lists and a fence is never split down the
+ * middle. Guessing at block boundaries with a line scanner gets all three of those wrong.
+ *
+ * It also bounds parsing, which was the other symptom: one message took **1.4 seconds** to parse as
+ * a single unit, and a block is a paragraph.
+ */
+fun markdownBlocks(text: String): List<String> {
+    // A reference definition sits at the foot of a message and is used by links above it. Parsed on
+    // its own each block would lose the definition, and the link would draw as literal brackets --
+    // so a message carrying one is kept whole. Rare enough to be worth giving up the split for.
+    if (REFERENCE_DEFINITION.containsMatchIn(text)) return listOf(text)
+    val parsed = parseMarkdown(text) as? State.Success ?: return listOf(text)
+    val blocks =
+        parsed.node.children
+            .map { text.substring(it.startOffset, it.endOffset) }
+            .filter { it.isNotBlank() }
+    return if (blocks.size <= 1) listOf(text) else blocks
+}
+
+/** `[label]: https://…` at the start of a line -- see [markdownBlocks]. */
+private val REFERENCE_DEFINITION = Regex("""^ {0,3}\[[^\]]+]:\s""", RegexOption.MULTILINE)
+
+/**
+ * A reply drawn a block at a time, with the blocks that are off screen not drawn at all.
+ *
+ * Each block keeps its composition and its layout whichever way it is scrolled -- that is what
+ * stops a message being rebuilt when somebody comes back to it -- and only the drawing is skipped.
+ * The heights come from the blocks themselves as they are measured, so the running total is the
+ * same arrangement the list uses one level up.
+ */
+@Composable
+fun BlockedMarkdown(text: String, replies: ParsedReplies, modifier: Modifier = Modifier) {
+    val blocks = remember(text) { replies.blocksOf(text) }
+    if (blocks.size == 1) {
+        MarkdownText(blocks.first(), replies, modifier)
+        return
+    }
+    val window = LocalRowWindow.current
+    val spacing = with(LocalDensity.current) { BLOCK_SPACING.roundToPx() }
+    val offsets = remember(blocks, spacing) { BlockOffsets(blocks.size, spacing) }
+    Column(modifier.fillMaxWidth(), verticalArrangement = Arrangement.spacedBy(BLOCK_SPACING)) {
+        blocks.forEachIndexed { index, block ->
+            Box(
+                Modifier.fillMaxWidth()
+                    .onSizeChanged { offsets.measured(index, it.height) }
+                    .drawWithContent {
+                        val near =
+                            window == null ||
+                                window.visible(offsets.top(index), offsets.height(index))
+                        if (near) {
+                            DebugStats.count("block drawn")
+                            // The number that says whether splitting bounded anything. A row can
+                            // stay enormous and be fine, so long as no single *drawn* piece of it
+                            // is -- and one node the parser will not divide, a long fenced code
+                            // block above all, stays one piece however tall it is.
+                            DebugStats.atLeast(
+                                "tallest drawn block px",
+                                offsets.height(index).toLong(),
+                            )
+                            drawContent()
+                        } else {
+                            DebugStats.count("block skipped")
+                        }
+                    }
+            ) {
+                MarkdownText(block, replies)
+            }
+        }
+    }
+}
+
+/**
+ * Where each block of one message sits inside it, added up from the heights before it.
+ *
+ * The same shape as the list's own bookkeeping and for the same reason: adding the heights up on
+ * every lookup would be quadratic, and the lookup happens once per block per frame.
+ */
+private class BlockOffsets(count: Int, private val spacing: Int) {
+    private val heights = IntArray(count)
+    private var tops = IntArray(count)
+    private var stale = true
+
+    fun measured(index: Int, height: Int) {
+        if (index in heights.indices && heights[index] != height) {
+            heights[index] = height
+            stale = true
+        }
+    }
+
+    fun height(index: Int) = heights.getOrElse(index) { 0 }
+
+    fun top(index: Int): Int {
+        if (stale) {
+            var y = 0
+            for (i in heights.indices) {
+                tops[i] = y
+                y += heights[i] + spacing
+            }
+            stale = false
+        }
+        return tops.getOrElse(index) { 0 }
+    }
+}
+
+/** The gap between one block of a reply and the next. */
+private val BLOCK_SPACING = 6.dp

app/androidApp/src/main/kotlin/com/example/aiapp/SessionScreen.kt

@@ -578,7 +578,10 @@ private suspend fun warm(replies: ParsedReplies, rows: List<TranscriptItem>) {
     // the work it finds stays one page's worth. Off the calling thread it is nobody's frame.
     withContext(Dispatchers.Default) {
         val texts =
-            rows.filterIsInstance<TranscriptItem.AssistantMsg>().flatMap { markdownIn(it.text) }
+            rows
+                .filterIsInstance<TranscriptItem.AssistantMsg>()
+                .flatMap { markdownIn(it.text) }
+                .flatMap { replies.blocksOf(it) }
         if (texts.isNotEmpty()) replies.warm(texts)
     }
 }

app/androidApp/src/main/kotlin/com/example/aiapp/TranscriptScroll.kt

@@ -12,6 +12,7 @@ import androidx.compose.foundation.layout.padding
 import androidx.compose.foundation.rememberScrollState
 import androidx.compose.foundation.verticalScroll
 import androidx.compose.runtime.Composable
+import androidx.compose.runtime.CompositionLocalProvider
 import androidx.compose.runtime.Stable
 import androidx.compose.runtime.getValue
 import androidx.compose.runtime.key
@@ -136,11 +137,7 @@ class TranscriptScroll(internal val scroll: ScrollState) {
         val height = heights[seq] ?: return true
         refreshTops()
         val top = tops.getOrNull(index) ?: return true
-        val viewportTop = scroll.maxValue - scroll.value
-        val margin = scroll.viewportSize
-        val near =
-            top + height >= viewportTop - margin &&
-                top <= viewportTop + scroll.viewportSize + margin
+        val near = near(top, height)
         // Counted so that "drawing costs too much" can be told apart from "drawing was skipped and
         // still costs too much". They need opposite fixes: the first is a row that should not have
         // been drawn, the second is a single row too tall to record cheaply -- and one enormous
@@ -237,6 +234,36 @@ class TranscriptScroll(internal val scroll: ScrollState) {
     val roomAbove: Int
         get() = scroll.maxValue - scroll.value
 
+    /**
+     * What a block inside the row named by [seq] should ask to find out whether it is on screen.
+     *
+     * The list is the only thing that knows where a row sits, and a message being drawn is the only
+     * thing that knows where its blocks sit inside it, so the two meet at [RowWindow]: the row
+     * supplies the base and the message supplies the offset. Remembered per row by the caller,
+     * because it is captured by every block's draw.
+     */
+    fun windowFor(seq: Long): RowWindow =
+        object : RowWindow {
+            override fun visible(top: Int, height: Int): Boolean {
+                val base = rowTop(seq) ?: return true
+                return near(base + top, height)
+            }
+        }
+
+    private fun rowTop(seq: Long): Int? {
+        val index = rowIndex[seq] ?: return null
+        refreshTops()
+        return tops.getOrNull(index)
+    }
+
+    /** Whether a span of content, in content coordinates, is within a screen of the viewport. */
+    private fun near(top: Int, height: Int): Boolean {
+        val viewportTop = scroll.maxValue - scroll.value
+        val margin = scroll.viewportSize
+        return top + height >= viewportTop - margin &&
+            top <= viewportTop + scroll.viewportSize + margin
+    }
+
     /** The height of the visible area, 0 until the first measurement. */
     val viewport: Int
         get() = scroll.viewportSize
@@ -362,7 +389,10 @@ fun TranscriptColumn(
                         // the draw phase, so moving the list invalidates drawing and nothing else.
                         .drawWithContent { if (state.onScreen(item.startSeq)) drawContent() }
                 ) {
-                    row(item)
+                    // So a block of a long reply can ask the same question the row just answered,
+                    // about its own part of it; see [RowWindow].
+                    val window = remember(state, item.startSeq) { state.windowFor(item.startSeq) }
+                    CompositionLocalProvider(LocalRowWindow provides window) { row(item) }
                 }
             }
         }