Say which half of the wait a llama turn is in

A turn has two waits in front of the first token and they were one word.
`SessionStatus::Loading` was already the model coming off disk; this adds
`SessionStatus::Reading` for llama-server processing the prompt -- emitted when
the request goes out, cleared by the first thing the model says of any kind, so
it covers every generate in a tool loop rather than only the first.

Prefill is the expensive half on this machine: measured 9.5s for 6,068 tokens
and 22s for 14,068 on the 27B with the GPU to itself. Reported as `running`
that was indistinguishable from a model thinking, which is the thing the reader
is waiting for. The phone draws both with the working spinner and its own
words -- "loading model" and "reading prompt" -- and the session screen's
status row now spins for all three busy states instead of only `running`,
which is also how `loading` stops being a bare word with nothing moving.

Measured while checking the tok/s figure, and recorded in the rigs skill: the
27B holds 55.5 to 50.3 tok/s between 1.5k and 14k of context, so decode decays
gently, while the 0.6B on the CPU falls 30.1 to 11.5 over 6k. A shared GPU is a
different failure -- the model does not load at all.

Verified on the emulator against a real llama session: "loading model" while
the server started, then "reading prompt" with the spinner through prompt
processing, then the thinking card.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
iris-aiandClaude Opus 5 committed 2026-09-19 15:44:04 -04:00
1 parent bb5ac1a242
commit b660905098
8 files changed
+114 -6

No files matched your search

@@ -357,7 +357,7 @@ fun parseSeqEvent(json: String): SeqEvent {
* split mid-stream.
*/
fun sessionWorking(state: String): Boolean =
state == "running" || state == "compacting" || state == "loading"
state == "running" || state == "compacting" || state == "loading" || state == "reading"
/** Whether the latest events still say this session needs an explicit provider login. */
internal fun authenticationPromptAfter(open: Boolean, event: SessionEvent): Boolean =
@@ -2628,7 +2628,15 @@ private fun SessionStatusRow(
modifier = Modifier.weight(1f).padding(horizontal = 8.dp),
)
}
"running" -> {
// The three states with something happening in them and nothing wanted from the
// reader. One branch, because what they share is the spinner -- the machine is busy --
// and what differs is only which part of the wait this is: the model coming off disk,
// the model reading what it was given, and the model answering. Reading used to be
// reported as running, so the minutes a long conversation spends on prompt processing
// were indistinguishable from a model thinking.
"running",
"reading",
"loading" -> {
CircularProgressIndicator(
// Smaller than the line beside it, so the row keeps the text's own height: a
// control taller than a line re-centres it and knocks it out of line with the
@@ -2637,7 +2645,9 @@ private fun SessionStatusRow(
strokeWidth = 2.dp,
)
Text(
"working",
// "working" rather than "running" for the one that is generating: the word is
// there to say the machine is busy, and the other two say what it is busy at.
if (status == "running") "working" else sessionStatusWord(status, subagent),
style = MaterialTheme.typography.labelSmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
modifier = Modifier.padding(start = 8.dp),
@@ -25,7 +25,15 @@ fun sessionStatusWord(status: String, subagent: Boolean = false): String =
// Not "running": a model coming off disk is not a model answering, and the difference is
// minutes. Said in its own word so a first message that waits is explained rather than
// looking like a session that has stopped responding. See `SessionStatus::Loading`.
"loading" -> "loading"
//
// "model" rather than "loading" alone, because there are two waits before an answer and
// the reader is entitled to know which one they are in: this one happens once, and
// "reading prompt" below happens on every turn.
"loading" -> "loading model"
// The model has the prompt and has not started answering. Its own word for the same
// reason: a long conversation spends real time here, and reported as "running" it looked
// like a model thinking. See `SessionStatus::Reading`.
"reading" -> "reading prompt"
// Its own word, because the state it is easily mistaken for means the opposite: "idle"
// invites the reader to type something, and a waiting session is going to carry on without
// them. See `SessionStatus::Waiting`.
@@ -59,7 +67,8 @@ fun sessionStatusColour(status: String): Color =
"compacting" -> commandColor
// The same accent as the other states that are busy on their own account, because that is
// what this is: something is happening and nothing is wanted from the reader.
"loading" -> commandColor
"loading",
"reading" -> commandColor
"waiting" -> waitingColor
else -> MaterialTheme.colorScheme.onSurfaceVariant
}