iris/ai-app
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Draw text on the platform directly, own the table's outer box, and parse a live reply a block at a time

Browse Source 
Three costs left in the renderer's composition layer, taken one at a time:

The text leaf. Every paragraph went through the renderer's text composable,
which exists to place inline images and charged each text for the
possibility: a placement callback, a derived map of inline content, a
semantics group and a size animation. A paragraph with no image -- nearly
all of them -- now goes straight to BasicText, with the renderer's own rule
for a style that names no colour. One with an image keeps the old path.

The table. The renderer decided "spread or scroll" with a BoxWithConstraints,
a subcomposition. LinkedTable does it with one layout modifier placed after
the horizontal scroll: fillMaxWidth fixes the minimum to the room, the scroll
passes that minimum through while lifting the maximum, and the modifier sizes
the rows to the larger of the room and the columns' floor. Rows no longer
need a width handed to them or a row index from a composition local.

The live reply. Every delta reparsed the whole message off-thread; for a
long reply that was tens of milliseconds hundreds of times, every core busy
while the frame's thread waited. LiveParse freezes every top-level block that
a later block has started after -- markdown's block rules make that safe --
and reparses only the tail. Each piece is keyed by where it starts in the
message, so a block keeps its composition when it freezes.

Verified on the emulator: the block-kind fixture draws the same with links
opening from a paragraph and a bullet and plain text opening nothing; a
six-column table still scrolls sideways; a 58-word mixed stream of list,
fence, table and quote drew every block as it arrived, 47 tail reparses at
1.7ms mean against whole-message parses before. ktfmt, build and lint clean
but for the AGP 9.4.0 notice.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
irisandClaude Fable 5.1 committed 2026-09-03 00:00:01 -04:00
1 parent 48bb7de304
commit 4bc69e8f9c
2 files changed
+232 -106
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

app/androidApp/src/main/kotlin/com/example/aiapp/Markdown.kt
+204 -101
View File
@@ -1,10 +1,13 @@
package com.example.aiapp
import androidx.compose.foundation.background
import androidx.compose.foundation.horizontalScroll
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.layout.widthIn
import androidx.compose.foundation.rememberScrollState
import androidx.compose.foundation.shape.RoundedCornerShape
import androidx.compose.material3.MaterialTheme
import androidx.compose.runtime.Composable
import androidx.compose.runtime.LaunchedEffect
@@ -16,7 +19,10 @@ import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.draw.drawWithContent
import androidx.compose.ui.graphics.graphicsLayer
import androidx.compose.ui.layout.layout
import androidx.compose.ui.semantics.CollectionInfo
import androidx.compose.ui.semantics.CollectionItemInfo
import androidx.compose.ui.semantics.collectionInfo
import androidx.compose.ui.semantics.collectionItemInfo
import androidx.compose.ui.semantics.heading
import androidx.compose.ui.semantics.semantics
@@ -25,13 +31,12 @@ import androidx.compose.ui.text.TextStyle
import androidx.compose.ui.text.font.FontFamily
import androidx.compose.ui.text.font.FontWeight
import androidx.compose.ui.text.style.TextDecoration
import androidx.compose.ui.unit.Dp
import androidx.compose.ui.unit.TextUnit
import androidx.compose.ui.unit.dp
import com.mikepenz.markdown.compose.LocalMarkdownColors
import com.mikepenz.markdown.compose.LocalMarkdownDimens
import com.mikepenz.markdown.compose.components.markdownComponents
import com.mikepenz.markdown.compose.elements.LocalTableRowIndex
import com.mikepenz.markdown.compose.elements.MarkdownTable
import com.mikepenz.markdown.compose.elements.MarkdownDivider
import com.mikepenz.markdown.compose.elements.listDepth
import com.mikepenz.markdown.m3.Markdown
import com.mikepenz.markdown.m3.elements.MarkdownCheckBox
@@ -44,28 +49,22 @@ import java.util.concurrent.ConcurrentHashMap
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.withContext
import org.intellij.markdown.ast.ASTNode
import org.intellij.markdown.ast.findChildOfType
import org.intellij.markdown.flavours.gfm.GFMElementTypes
import org.intellij.markdown.flavours.gfm.GFMTokenTypes
/**
* An assistant's reply, rendered as the markdown it is written in.
* [text] drawn as its pieces, one under the other; see [Piece].
*
* The parsing is the library's. Markdown is somebody else's specification, and a hand-written
* subset of one disagrees with it at the edges -- which is where the bug reports come from, one
* case at a time. This file's whole job is the mapping onto the app's palette and type scale.
*
* Colours come from the theme rather than from the renderer's defaults, so code, links and rules
* are the same Catppuccin values the rest of the app uses. Nothing here picks a colour of its own.
*/
/**
* [text] drawn as its pieces, one under the other; see [Piece]. [live] is the reply still arriving,
* and it is the only message whose pieces get a layer each. A layer buys one thing here: when
* drawing is invalidated, only the piece that changed is re-recorded instead of the whole reply.
* That is worth a great deal while a reply is streaming, because every delta invalidates the
* message and a finished one can be twenty-five screens tall. It is worth nothing once the message
* stops changing -- measured on a Pixel 9 Pro XL, whole rows were re-recorded 65 times in fifty
* seconds of reading -- and it is not free: each layer is a layout node and a display list held for
* the life of the row, and live node count is what the per-frame cost of the transcript scales
* with.
* [live] is the reply still arriving, and two things are different for it. Its parse is incremental
* -- see [LiveParse] -- so a delta costs a parse of the block it landed in rather than of the whole
* message. And its pieces get a layer each: when drawing is invalidated, only the piece that
* changed is re-recorded instead of the whole reply, which is worth a great deal while every delta
* invalidates the message and a finished one can be twenty-five screens tall. It is worth nothing
* once the message stops changing -- measured on a Pixel 9 Pro XL, whole rows were re-recorded 65
* times in fifty seconds of reading -- and it is not free: each layer is a layout node and a
* display list held for the life of the row, and live node count is what the per-frame cost of the
* transcript scales with.
*/
@Composable
fun MarkdownText(
@@ -74,35 +73,149 @@ fun MarkdownText(
modifier: Modifier = Modifier,
live: Boolean = false,
) {
val parse = parsedMarkdown(text, replies)
val pieces = remember(parse) { pieces(parse) }
MarkdownRoot(parse) {
val segments =
if (live) liveSegments(text)
else remember(text) { listOf(Segment(text, 0, replies.of(text), replies.piecesOf(text))) }
Column(modifier.fillMaxWidth()) {
var previous: Piece? = null
pieces.forEach { piece ->
// Keyed by address rather than by position in this column, so a delta that
// lands in the last paragraph leaves every other piece's composition alone.
key(piece) {
var previousSegment: Segment? = null
segments.forEach { segment ->
MarkdownRoot(segment.parse) {
segment.pieces.forEach { piece ->
val gap =
when {
previousSegment == null -> 0.dp
previousSegment !== segment -> BLOCK_SPACING
else -> gapBefore(previous, piece)
}
// Keyed by where the piece starts in the message rather than by its position
// in this column, so a delta landing in the last block leaves every other
// piece's composition alone -- and a block keeps its key when it freezes.
key(segment.start, piece) {
MarkdownPiece(
parse,
text,
segment.parse,
segment.text,
piece,
Modifier.padding(
top = if (previous == null) 0.dp else gapBefore(previous, piece)
)
Modifier.padding(top = gap)
.then(if (live) Modifier.graphicsLayer() else Modifier)
.drawWithContent {
val started = System.nanoTime()
drawContent()
DebugStats.record("record: one block", System.nanoTime() - started)
DebugStats.record(
"record: one block",
System.nanoTime() - started,
)
},
)
}
previous = piece
previousSegment = segment
}
}
}
}
}
/**
* A stretch of a message with a parse of its own: the whole of a settled message, or one block or
* the unfinished tail of a live one. [start] is where [text] begins in the message.
*/
private class Segment(val text: String, val start: Int, val parse: State, val pieces: List<Piece>)
/**
* The live reply's segments: parsed on the composing thread the first time the row is drawn, and
* incrementally off it for every delta afterwards.
*
* The first parse has to be inline. The renderer's own asynchronous path draws an empty loading
* slot until its result arrives, so a row is measured at nothing before it is measured at its real
* height, and the transcript above it collapses and springs back. Seen with five replies on screen
* at once, every one of them blank, the whole conversation shrunk to fit a single screen; a moment
* later it was all there again. That is the "skipping up and down" this list must never do.
*
* Every parse after the first is off the composing thread, and the row keeps drawing the parse it
* already has until the new one lands, so there is never a frame without a height. What is on
* screen is always a real prefix of the reply rather than a guess at it; it is simply one parse
* behind.
*/
@Composable
private fun liveSegments(text: String): List<Segment> {
val parsed = remember {
mutableStateOf(
DebugStats.timed("markdown parsed while composing") { LiveParse.whole(text) }
)
}
LaunchedEffect(text) {
if (parsed.value.text == text) return@LaunchedEffect
val previous = parsed.value
parsed.value =
withContext(Dispatchers.Default) {
DebugStats.timed("markdown reparsed while streaming") { previous.advanceTo(text) }
}
}
return parsed.value.segments
}
/**
* A reply still arriving, parsed a block at a time.
*
* Reparsing the whole message per delta was fine for a short reply and not for a long one: a
* twenty-five-screen reply parses in tens of milliseconds, hundreds of times, and although that ran
* off the composing thread it was every core busy while the frame's own thread waited for one.
* Markdown's blocks make the cut safe: a top-level block that another block has started *after* is
* finished -- nothing appended later can reach back into it, since a paragraph ends at the blank
* line or the block that interrupts it, a fence at its closing fence, a list at the first line that
* is neither an item nor indented under one. So every block but the last is [frozen] with the parse
* that finished it, and only the tail -- the last block and whatever has arrived since -- is parsed
* again.
*
* What the cut gives up is one thing: a reference definition arriving later than a link that uses
* it, since the frozen block's parse never sees it. The link draws as its brackets until the reply
* settles and is parsed whole by [warm], which is the same moment every other transient of
* streaming is put right.
*/
private class LiveParse(
val text: String,
private val frozen: List<Segment>,
/** How much of [text] the frozen segments cover; the tail starts here. */
private val consumed: Int,
private val tail: Segment,
) {
val segments: List<Segment>
get() = frozen + tail
fun advanceTo(next: String): LiveParse {
// Anything but an append to what was frozen -- a message replaced, a stream reset --
// starts over.
if (!next.regionMatches(0, text, 0, consumed)) return whole(next)
val tailText = next.substring(consumed)
val parse = parseMarkdown(tailText)
val all = pieces(parse)
val blocks = all.map { it.block }.distinct()
if (blocks.size <= 1 || parse !is State.Success) {
return LiveParse(next, frozen, consumed, Segment(tailText, consumed, parse, all))
}
val done =
blocks.dropLast(1).map { block ->
Segment(tailText, consumed, parse, all.filter { it.block == block })
}
val cut = parse.node.children[blocks.last()].startOffset
val rest = tailText.substring(cut)
val restParse = parseMarkdown(rest)
return LiveParse(
next,
frozen + done,
consumed + cut,
Segment(rest, consumed + cut, restParse, pieces(restParse)),
)
}
companion object {
fun whole(text: String): LiveParse {
val parse = parseMarkdown(text)
return LiveParse(text, emptyList(), 0, Segment(text, 0, parse, pieces(parse)))
}
}
}
/** One [piece] of [text], drawn on its own -- a unit of the transcript list. */
@Composable
@@ -254,19 +367,7 @@ private fun MarkdownRoot(parse: State, content: @Composable () -> Unit) {
// so they draw like the top-level ones the transcript cuts into items.
orderedList = { MarkdownList(it.content, it.node, it.listDepth) },
unorderedList = { MarkdownList(it.content, it.node, it.listDepth) },
table = {
MarkdownTable(
it.content,
it.node,
style = it.typography.table,
headerBlock = { content, row, width, style ->
LinkedTableRow(content, row, width, style, header = true)
},
rowBlock = { content, row, width, style ->
LinkedTableRow(content, row, width, style, header = false)
},
)
},
table = { LinkedTable(it.content, it.node, it.typography.table) },
),
modifier = Modifier,
success = { _, _, _ -> content() },
@@ -274,38 +375,76 @@ private fun MarkdownRoot(parse: State, content: @Composable () -> Unit) {
}
/**
* One row of a table -- the header when [header] -- with every cell a [LinkedText].
* A table: its rows, on the renderer's tinted, rounded background, as wide as its columns need.
*
* Each column has a floor ([markdownDimens]'s `tableCellWidth`), so the table is at least
* columns-times-floor wide; narrower than the room it has, it spreads to fill it, and wider, it
* scrolls sideways rather than squeezing. The renderer decided that with a `BoxWithConstraints`,
* which is a subcomposition; here it is one layout modifier, and the trick is where it sits.
* `fillMaxWidth` fixes the minimum width to the room available, the horizontal scroll passes that
* minimum through to its content while lifting the maximum to unbounded, and the modifier after it
* reads the minimum back as the room and sizes the rows to the larger of that and the floor. The
* scroll then has exactly the overflow to scroll, which is none when the table fits.
*/
@Composable
private fun LinkedTable(content: String, node: ASTNode, style: TextStyle) {
val dimens = LocalMarkdownDimens.current
val colors = LocalMarkdownColors.current
val columns =
remember(node) {
node.findChildOfType(GFMElementTypes.HEADER)?.children?.count {
it.type == GFMTokenTypes.CELL
} ?: 0
}
val rows = remember(node) { node.children.count { it.type == GFMElementTypes.ROW } + 1 }
val floor = dimens.tableCellWidth * columns
Column(
Modifier.background(colors.tableBackground, RoundedCornerShape(dimens.tableCornerSize))
.semantics { collectionInfo = CollectionInfo(rowCount = rows, columnCount = columns) }
.fillMaxWidth()
.horizontalScroll(rememberScrollState())
.layout { measurable, constraints ->
val width = maxOf(constraints.minWidth, floor.roundToPx())
val placeable =
measurable.measure(constraints.copy(minWidth = width, maxWidth = width))
layout(width, placeable.height) { placeable.place(0, 0) }
}
) {
var rowIndex = 1
node.children.forEach { child ->
when (child.type) {
GFMElementTypes.HEADER -> LinkedTableRow(content, child, style, rowIndex = 0)
GFMElementTypes.ROW -> LinkedTableRow(content, child, style, rowIndex = rowIndex++)
GFMTokenTypes.TABLE_SEPARATOR -> MarkdownDivider()
}
}
}
}
/**
* One row of a table -- the header when [rowIndex] is zero -- with every cell a [LinkedText].
*
* The renderer's own rows draw each cell at `maxLines = 1` with an ellipsis, which on a phone means
* most of a table is simply not readable: anything past about twenty characters ends in "..." with
* no way to see the rest, and an elided cell looks like a short one, so a table of measurements
* reads as a table of plausible shorter measurements. And they draw a link in a cell as its own
* layout node, the cost [LinkedText] exists to avoid. Both are decided inside the cell, where the
* renderer offers no slot, so the row is ours: the outer table -- its width, sideways scroll,
* corners and row dividers -- is still the renderer's.
* layout node, the cost [LinkedText] exists to avoid.
*
* So: as many lines as the cell needs, cells aligned to the top of the row, because a two-line cell
* beside a one-line one centred the short one against the middle of the tall one and lost the line
* the reader was reading across. What the wrapping does *not* do is make a wide table fit. The
* table gives each column its minimum width and scrolls sideways when they do not fit, which is the
* right answer for too many columns -- wrapping a six-column table into the width of a phone would
* give every cell one word per line.
* the reader was reading across. What the wrapping does *not* do is make a wide table fit;
* [LinkedTable] scrolls it instead, which is the right answer for too many columns -- wrapping a
* six-column table into the width of a phone would give every cell one word per line.
*
* The semantics are the renderer's: each cell is an item of the table's collection, and a header
* cell is a heading.
*/
@Composable
private fun LinkedTableRow(
content: String,
row: ASTNode,
tableWidth: Dp,
style: TextStyle,
header: Boolean,
) {
private fun LinkedTableRow(content: String, row: ASTNode, style: TextStyle, rowIndex: Int) {
val padding = LocalMarkdownDimens.current.tableCellPadding
val rowIndex = if (header) 0 else LocalTableRowIndex.current
val header = rowIndex == 0
val cellStyle = if (header) style.copy(fontWeight = FontWeight.Bold) else style
Row(verticalAlignment = Alignment.Top, modifier = Modifier.widthIn(tableWidth)) {
Row(verticalAlignment = Alignment.Top, modifier = Modifier.fillMaxWidth()) {
row.children
.filter { it.type == GFMTokenTypes.CELL }
.forEachIndexed { column, cell ->
@@ -328,42 +467,6 @@ private fun LinkedTableRow(
}
}
/**
* [text] parsed: on the composing thread the first time this row is drawn, and off it every time
* afterwards.
*
* The first parse has to be inline. The renderer's own asynchronous path draws an empty loading
* slot until its result arrives, so a row is measured at nothing before it is measured at its real
* height, and the transcript above it collapses and springs back. Seen with five replies on screen
* at once, every one of them blank, the whole conversation shrunk to fit a single screen; a moment
* later it was all there again. That is the "skipping up and down" this list must never do, and no
* amount of scroll anchoring can survive a row that lies about its height first.
*
* Every parse *after* the first is a different case, and it is the one that was costing: a reply
* arrives as hundreds of deltas, each one re-parsing the whole message it has grown into. Measured
* against `/stream 200` on the emulator, that was fifty-eight parses and 78ms of main-thread work
* in three seconds, with single parses reaching 7ms -- most of a frame at 60Hz and more than one at
* 120. Those go to a background thread, and the row keeps drawing the parse it already has until
* the new one lands, so there is never a frame without a height. What is on screen is always a
* real prefix of the reply rather than a guess at it; it is simply one parse behind.
*/
@Composable
private fun parsedMarkdown(text: String, replies: ParsedReplies): State {
// The text each parse came from, so the first composition's is not immediately repeated.
val parsed = remember { mutableStateOf(text to replies.of(text)) }
LaunchedEffect(text) {
if (parsed.value.first == text) return@LaunchedEffect
// Not through [replies]: this is a reply still arriving, and every delta would leave
// another copy of a message that is about to be superseded.
parsed.value =
text to
withContext(Dispatchers.Default) {
DebugStats.timed("markdown reparsed while streaming") { parseMarkdown(text) }
}
}
return parsed.value.second
}
/**
* Replies parsed before the row that draws them is composed.
*
app/androidApp/src/main/kotlin/com/example/aiapp/MarkdownLinks.kt
+28 -5
View File
@@ -1,10 +1,12 @@
package com.example.aiapp
import androidx.compose.foundation.gestures.detectTapGestures
import androidx.compose.foundation.text.BasicText
import androidx.compose.runtime.Composable
import androidx.compose.runtime.remember
import androidx.compose.ui.Modifier
import androidx.compose.ui.geometry.Offset
import androidx.compose.ui.graphics.isSpecified
import androidx.compose.ui.input.pointer.pointerInput
import androidx.compose.ui.node.Ref
import androidx.compose.ui.platform.LocalUriHandler
@@ -17,6 +19,7 @@ import androidx.compose.ui.text.TextStyle
import com.mikepenz.markdown.annotator.AnnotatorSettings
import com.mikepenz.markdown.annotator.annotatorSettings
import com.mikepenz.markdown.annotator.buildMarkdownAnnotatedString
import com.mikepenz.markdown.compose.LocalMarkdownColors
import com.mikepenz.markdown.compose.components.MarkdownComponentModel
import com.mikepenz.markdown.compose.elements.MarkdownText
import com.mikepenz.markdown.model.markdownAnnotator
@@ -79,20 +82,40 @@ fun LinkedText(content: String, node: ASTNode, style: TextStyle, modifier: Modif
}
val uriHandler = LocalUriHandler.current
val layout = remember { Ref<TextLayoutResult>() }
MarkdownText(
content = text,
node = node,
modifier =
val tapping =
modifier.pointerInput(text) {
detectTapGestures { position ->
val url = text.linkAt(layout.value, position) ?: return@detectTapGestures
uriHandler.openUri(url)
}
},
}
// The renderer's text composable exists to place inline images, and it charges every text
// for the possibility: a placement callback, a derived map of inline content, a semantics
// group and a size animation, per paragraph. Almost no paragraph has an image, so those go
// straight to the platform text; the few that do keep the renderer's path.
if (remember(node) { node.hasImage() }) {
MarkdownText(
content = text,
node = node,
modifier = tapping,
style = style,
onTextLayout = { result, _ -> layout.value = result },
)
} else {
// The renderer's own rule for a style that names no colour: the theme's text colour.
val color = if (style.color.isSpecified) style.color else LocalMarkdownColors.current.text
BasicText(
text = text,
modifier = tapping,
style = style,
color = { color },
onTextLayout = { layout.value = it },
)
}
}
private fun ASTNode.hasImage(): Boolean =
type == MarkdownElementTypes.IMAGE || children.any { it.hasImage() }
/**
* The address under [position], if a link's glyph is there rather than merely nearest to it.