Files
ai-app/app/androidApp/src/main/kotlin/com/example/aiapp/CodeFence.kt
T
irisandClaude Opus 5 a2b11d516f Colour code with a scanner of our own instead of the library
dev.snipme:highlights 1.1.0 found comments before it knew the language and
paired /* with */ by ordinal, so `//` in any URL commented out the rest of
its line, every Rust `#[derive(...)]` greyed out as a comment, a `#` inside
a Kotlin string swallowed the line, and `x '*/a/*'` in shell produced a span
whose end preceded its start -- the one that crashed a card holding
`-path '*/.git/*'`. None of that could be post-processed away, because
comments won over strings before the language was known.

Highlighter.kt is one left-to-right scanner: at each position it is in a
line comment, a block comment, a string, or ordinary code, and every span is
emitted by advancing an index, so spans cannot overlap, arrive out of order
or run backwards. Languages.kt is a `Rules` row per language -- comment
tokens, block comment and whether it nests, the string forms, what opens an
attribute, and the keyword set -- so a new language is a table entry. The
keyword lists came from the library's SyntaxTokens.kt (Apache-2.0, noted at
the table) so nothing that is coloured today turns plain, and RON, TOML,
fish and JSON are coloured for the first time.

HighlighterTest.kt is a new JVM unit test source set -- 24 cases, the
library's mistakes kept as regressions, plus a sweep asserting no span
escapes the code for any language on unterminated and empty input.
AGENTS.md's app line now runs :androidApp:testDebugUnitTest.

Measured on the ai-app emulator, debug build, a ~200-line Kotlin fence sent
into a sandbox session:

  before  code highlighted: 1, 101.9ms total, 101.9ms mean, 101.9ms worst
  after   code highlighted: 1,  15.0ms total,  15.0ms mean,  15.0ms worst

and a second fence in the same run took 13.9ms, so that is the steady cost
rather than class loading. stream-bench.sh after the change:

  code highlighted: 1, 12.1ms total, 12.1ms mean, 12.1ms worst
  markdown reparsed while streaming: 1329, 2130.7ms total, 1.6ms mean, 8.7ms worst
  record: one block: 131, 11.4ms total, 0.1ms mean, 0.4ms worst
  draw phase 1.21ms per frame, the transcript 0.23ms of it

transcript-bench.sh after: draw phase 1.10ms per frame, the transcript
0.49ms (place 0.48), worst place 4.3ms -- unchanged within run-to-run noise,
as expected, since the scan happens in `warm` and not while drawing.

Looked at on the emulator: a URL inside a Kotlin string, a Rust attribute
with a lifetime and a raw string, a shell line with globs and `$#`, a RON
fence and a TOML fence all colour correctly; a Bash tool card still colours
its command; a plain Python fence -- which this change had no reason to
touch -- looks as it did; an unknown language stays plain; and a fence is
plain while it streams and colours when it freezes.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-03 20:23:59 -04:00

209 lines
8.3 KiB
Kotlin

package com.example.aiapp
import androidx.compose.foundation.background
import androidx.compose.foundation.horizontalScroll
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.rememberScrollState
import androidx.compose.foundation.shape.RoundedCornerShape
import androidx.compose.foundation.text.BasicText
import androidx.compose.runtime.Composable
import androidx.compose.runtime.remember
import androidx.compose.ui.Modifier
import androidx.compose.ui.semantics.isTraversalGroup
import androidx.compose.ui.semantics.semantics
import androidx.compose.ui.text.TextStyle
import androidx.compose.ui.unit.dp
import com.mikepenz.markdown.compose.LocalMarkdownColors
import com.mikepenz.markdown.compose.LocalMarkdownDimens
import com.mikepenz.markdown.compose.LocalMarkdownPadding
import com.mikepenz.markdown.model.State
import org.intellij.markdown.MarkdownElementTypes
import org.intellij.markdown.MarkdownTokenTypes
import org.intellij.markdown.ast.ASTNode
import org.intellij.markdown.ast.findChildOfType
import org.intellij.markdown.ast.getTextInNode
/**
* A fenced code block in a reply: the code highlighted, on the dark surface every verbatim thing
* sits on, scrolling sideways rather than wrapping.
*
* The renderer's own fence drew the same block in plain text. The scanner that colours a tool
* call's command colours a reply's code the same way, through [highlighted] and one palette, so a
* `kotlin` fence and the Kotlin a tool wrote are the same colours. A fence in a language [scan] has
* no rules for is plain rather than wrongly coloured: [fenceLanguage] answers null for those, and
* plain is what the reader would have seen before.
*
* Finding the code is still the library's: which children of the node are the fence markers, the
* language word and the code between them is its knowledge of the parser, and [MarkdownCodeFence]
* hands out the code and the language and leaves the drawing to the block it is given.
*/
@Composable
fun CodeFence(
content: String,
node: ASTNode,
style: TextStyle,
replies: ParsedReplies,
streaming: Boolean = false,
) {
val (code, language) = remember(content, node) { fenceContent(content, node) } ?: return
CodeBlockText(code, language, style, replies, streaming)
}
/** An indented code block, which is a fence with no language word. */
@Composable
fun CodeBlock(
content: String,
node: ASTNode,
style: TextStyle,
replies: ParsedReplies,
streaming: Boolean = false,
) {
val (code, language) = remember(content, node) { fenceContent(content, node) } ?: return
CodeBlockText(code, language, style, replies, streaming)
}
/**
* The code inside a fence or indented block, and the highlighter's language for its info word.
*
* Which children of the node are the fence markers, the language word and the code between them is
* the library's knowledge of the parser, copied from its `MarkdownCodeFence` rather than called:
* that one is a composable, and the whole point of this function is that [warm] can run it on a
* background thread and highlight the same string the drawing will ask for. Two extractions would
* be two keys, and the warmed answer would be silently missed at every fence.
*
* Null for a fence too short to hold anything -- an unterminated one still arriving, which the
* library skips as invalid.
*/
fun fenceContent(content: String, node: ASTNode): Pair<String, Language?>? {
val word =
node.findChildOfType(MarkdownTokenTypes.FENCE_LANG)?.getTextInNode(content)?.toString()
val language = fenceLanguage(word)
if (node.type == MarkdownElementTypes.CODE_BLOCK) {
val start = node.children.firstOrNull()?.startOffset ?: return null
val end = node.children.lastOrNull()?.endOffset ?: return null
return content.substring(start, end).replaceIndent() to language
}
if (node.children.size < 3) return null
val start = node.children[2].startOffset
val fenceCount = if (word != null && node.children.size > 3) 3 else 2
val end = node.children[(node.children.size - 2).coerceAtLeast(fenceCount)].endOffset
return content.substring(start, end).replaceIndent() to language
}
/**
* Plain while [streaming], coloured once the block is finished; see [MarkdownRoot].
*
* The renderer's own block, less what nothing here needs: the same background, corner, padding and
* sideways scroll, without the shadow, the border and the empty pointer handler it also carried.
* The vertical margin is the renderer's too, kept so a reply's fences sit where they always have.
*/
@Composable
private fun CodeBlockText(
code: String,
language: Language?,
style: TextStyle,
replies: ParsedReplies,
streaming: Boolean,
) {
val colors = LocalMarkdownColors.current
val dimens = LocalMarkdownDimens.current
val padding = LocalMarkdownPadding.current
Box(
Modifier.fillMaxWidth()
.padding(vertical = 8.dp)
.background(colors.codeBackground, RoundedCornerShape(dimens.codeBackgroundCornerSize))
.semantics { isTraversalGroup = true }
) {
BasicText(
// No language while the block is still being written, which is what draws it plain;
// see [MarkdownRoot]'s `streaming`.
replies.highlighted(code, language.takeUnless { streaming }),
style = style,
modifier = Modifier.horizontalScroll(rememberScrollState()).padding(padding.codeBlock),
)
}
}
/**
* The highlighter's language for a fence's info word, or null for one it has no rules for.
*
* The aliases are what people actually write after the backticks: the file extension as often as
* the name. A word not here gets no colour rather than the nearest language's, because a fence
* coloured by the wrong language's rules looks highlighted and is wrong in a way the reader cannot
* see.
*/
fun fenceLanguage(name: String?): Language? =
FENCE_LANGUAGES[name?.trim()?.lowercase() ?: return null]
private val FENCE_LANGUAGES: Map<String, Language> =
mapOf(
"kotlin" to Language.KOTLIN,
"kt" to Language.KOTLIN,
"kts" to Language.KOTLIN,
"rust" to Language.RUST,
"rs" to Language.RUST,
"sh" to Language.SHELL,
"bash" to Language.SHELL,
"shell" to Language.SHELL,
"zsh" to Language.SHELL,
"console" to Language.SHELL,
"python" to Language.PYTHON,
"py" to Language.PYTHON,
"javascript" to Language.JAVASCRIPT,
"js" to Language.JAVASCRIPT,
"jsx" to Language.JAVASCRIPT,
"typescript" to Language.TYPESCRIPT,
"ts" to Language.TYPESCRIPT,
"tsx" to Language.TYPESCRIPT,
"java" to Language.JAVA,
"c" to Language.C,
"h" to Language.C,
"cpp" to Language.CPP,
"c++" to Language.CPP,
"cc" to Language.CPP,
"hpp" to Language.CPP,
"csharp" to Language.CSHARP,
"cs" to Language.CSHARP,
"c#" to Language.CSHARP,
"go" to Language.GO,
"golang" to Language.GO,
"swift" to Language.SWIFT,
"dart" to Language.DART,
"ruby" to Language.RUBY,
"rb" to Language.RUBY,
"php" to Language.PHP,
"perl" to Language.PERL,
"pl" to Language.PERL,
"coffeescript" to Language.COFFEESCRIPT,
"coffee" to Language.COFFEESCRIPT,
"ron" to Language.RON,
"toml" to Language.TOML,
"fish" to Language.FISH,
"json" to Language.JSON,
)
/**
* Every fence in [parse], as the code and language [highlight] will be asked for.
*
* Walks the whole tree rather than the top level: a fence inside a list item or a quote is drawn
* the same way and costs the same to lex.
*/
fun fences(parse: State): List<Pair<String, Language?>> {
val success = parse as? State.Success ?: return emptyList()
val out = ArrayList<Pair<String, Language?>>()
fun walk(node: ASTNode) {
if (
node.type == MarkdownElementTypes.CODE_FENCE ||
node.type == MarkdownElementTypes.CODE_BLOCK
) {
fenceContent(success.content, node)?.let { if (it.second != null) out += it }
return
}
node.children.forEach(::walk)
}
walk(success.node)
return out
}