Parse ahead on two threads, not on every core

The frame is now failing to *start* rather than taking too long once it
has: 21ms of `waited` at the 90th percentile against a median frame of
8.8ms that is inside budget. What is holding it up arrives with a page
of history -- 1.5 seconds of markdown parsed in a twelve second scroll,
all of it work nobody is waiting for.

Off the composing thread was the right call and it is not the same as
free. The default dispatcher sizes itself to the machine, which is right
for work somebody is waiting on: a page's worth of parses takes every
core, and the thread that draws the frame queues behind one of them. Two
threads, and a yield between messages, leaves the phone somewhere to run
the frame.

Nothing here makes the parsing faster, and it should not: the whole
point of doing it ahead is that its duration does not matter. What
matters is that it stops being in the way.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

app/androidApp/src/main/kotlin/com/example/aiapp/Markdown.kt

@@ -185,8 +185,16 @@ class ParsedReplies {
         parsed[text]?.also { DebugStats.count("markdown ready") }
             ?: DebugStats.timed("markdown parsed while composing") { parseMarkdown(text) }
 
-    /** Parses whatever is not held yet. Call off the composing thread; that is the whole point. */
+    /**
-    fun warm(texts: List<String>) {
+     * Parses whatever is not held yet. Call off the composing thread; that is the whole point.
+     *
+     * Suspending, and yielding between messages, because "off the composing thread" is not the same
+     * as "free". A page of history arrives as hundreds of parses at once -- 1.5 seconds of them in
+     * a twelve second scroll, measured on a Pixel 9 Pro XL -- and on the default dispatcher that is
+     * every core busy, with the frame's own thread waiting for one. That showed up as 21ms of
+     * `waited` at the 90th percentile: the frame could not start, rather than taking too long.
+     */
+    suspend fun warm(texts: List<String>) {
         texts.forEach { text ->
             parsed.computeIfAbsent(text) {
                 DebugStats.timed("markdown warmed") { parseMarkdown(it) }

app/androidApp/src/main/kotlin/com/example/aiapp/SessionScreen.kt

@@ -564,6 +564,18 @@ private fun updateTool(
     if (it is TranscriptItem.ToolRun && it.id == id) change(it) else it
 }
 
+/**
+ * Where markdown is parsed ahead of being drawn: two threads, never all of them.
+ *
+ * The default dispatcher sizes itself to the machine, which is right for work somebody is waiting
+ * on and wrong for work nobody is. A page of history is hundreds of parses arriving at once, and
+ * taking every core for them leaves the thread that draws the frame queueing behind one -- measured
+ * on a Pixel 9 Pro XL as 21ms of `waited` at the 90th percentile, which is the frame failing to
+ * *start* rather than taking too long once it had.
+ */
+@OptIn(kotlinx.coroutines.ExperimentalCoroutinesApi::class)
+private val parsingThreads = Dispatchers.Default.limitedParallelism(2)
+
 /**
  * Parses the replies among [rows], off whatever thread is drawing.
  *
@@ -576,7 +588,7 @@ private suspend fun warm(replies: ParsedReplies, rows: List<TranscriptItem>) {
     // [markdownIn] splits every assistant message looking for memory notes, and this is handed
     // the *whole* loaded transcript on every page, so the scan grows with the conversation while
     // the work it finds stays one page's worth. Off the calling thread it is nobody's frame.
-    withContext(Dispatchers.Default) {
+    withContext(parsingThreads) {
         val texts =
             rows
                 .filterIsInstance<TranscriptItem.AssistantMsg>()