Say how far a llama session's wait has got

Both of its waits are measured somewhere and neither reached the phone: a
model coming off disk, which the router publishes on its event stream and
nowhere else, and a prompt being read, which the generation stream will
report when asked. A session now answers GET /sessions/{id}/progress with
{of, fraction, stage?}, one thread per router keeping the load's fraction
per model, and the session screen asks twice a second while it is drawing
a wait that has one.

Asked for rather than emitted: a load reports five times a second, and an
event is a transcript line for ever. The sample says which status it
measures, so one that outlived its wait cannot be drawn under another
word. The phone puts the bar in the status row's free width and the
percentage where the context figure sits -- a row of its own would move
the transcript every time a turn started -- and names the stage where a
model loads more than one file, because the fraction starts again for
each. /loading and /reading in an echo session are the rig.
This commit is contained in:
iris-ai committed 2026-09-21 02:44:03 -04:00
1 parent 78f2fe3b79
commit 049780fda6
10 files changed
+549 -16

No files matched your search

@@ -362,6 +362,46 @@ fun fetchBackgroundTasks(
}
}
/**
* How far along the wait a session is in has got. See `GET /sessions/{id}/progress`.
*
* [of] is the status it measures, so a sample that outlived the wait it came from is never drawn
* under a different word: the row shows it only where the two agree.
*/
data class SessionProgress(
val of: String,
/** Between 0 and 1. */
val fraction: Float,
/**
* Which part of the wait this is, in the provider's own word, where it has more than one and
* the fraction starts again for each. Null where there is only one.
*/
val stage: String?,
)
/**
* How far [sessionId]'s current wait has got, or null when it is not in one that anything can
* measure -- which is most sessions most of the time, and every provider but llama.cpp.
*
* Asked for rather than streamed: a model load reports five times a second, and none of that
* belongs in the transcript the event stream carries.
*/
fun fetchProgress(settings: ServerSettings, sessionId: String): SessionProgress? =
requestFromServer(settings, "/sessions/$sessionId/progress") { connection ->
val body = connection.inputStream.bufferedReader().readText()
if (body.trim() == "null") null
else
JSONObject(body).let { row ->
SessionProgress(
of = row.getString("of"),
fraction = row.getDouble("fraction").toFloat(),
stage =
if (row.isNull("stage")) null
else row.optString("stage", "").ifEmpty { null },
)
}
}
fun fetchSubagents(settings: ServerSettings, sessionId: String): List<SubagentSummary> =
requestFromServer(settings, "/sessions/$sessionId/subagents") {
it.jsonObjects { row ->
@@ -255,6 +255,7 @@ fun SessionScreen(
// When the current compaction started. The moment comes off the `compacting` status event
// itself -- the server timestamps every transcript line -- rather than off this device noticing
// one, which is what makes it survive leaving the session and reopening it.
var progress by remember(address) { mutableStateOf<SessionProgress?>(null) }
var compactingSince by remember { mutableStateOf<Double?>(null) }
var compactingFor by remember { mutableStateOf<Long?>(null) }
var streamError by remember { mutableStateOf<String?>(null) }
@@ -957,6 +958,27 @@ fun SessionScreen(
if (fresh.isNotEmpty()) everGrouped = everGrouped + fresh
}
// How far the wait the session is in has got, for the two waits that can say: a model coming
// off disk and a prompt being read. Asked for while one of those is the status and not
// otherwise -- the answer is null for every other state, so polling outside them would be two
// requests a second to be told nothing.
//
// A failed ask leaves it null, which is the same as nothing having said: the row then draws the
// wait without a bar rather than a bar that has stopped moving.
LaunchedEffect(address, status) {
if (isSubagent || status !in measurableWaits) {
progress = null
return@LaunchedEffect
}
while (true) {
progress =
withContext(Dispatchers.IO) {
runCatching { fetchProgress(settings, summary.id) }.getOrNull()
}
delay(PROGRESS_POLL_MS)
}
}
// A compaction reports nothing about its own progress -- measured against the CLI, which says
// it has started and then nothing at all until it is done. So what this counts is the one thing
// anybody here can measure: how long it has been going. A bar filling up would be this screen
@@ -2151,6 +2173,9 @@ fun SessionScreen(
SessionStatusRow(
status = status,
compactingFor = compactingFor,
// Only where the two agree: a sample taken during a wait that has since ended
// would otherwise fill a bar under whatever the session is doing now.
progress = progress?.takeIf { it.of == status },
contextTokens = contextTokens,
contextLimit = contextLimit,
backgroundTasks = backgroundTasks,
@@ -2752,6 +2777,11 @@ private fun SessionStatusRow(
status: String,
/** Seconds since this device saw the compaction start; null if it did not see it. */
compactingFor: Long?,
/**
* How far the wait named by [status] has got, where something measured it. Null is the ordinary
* case -- most waits have nothing behind them that can say.
*/
progress: SessionProgress?,
/** Context the session is holding, or null where nothing has measured it. */
contextTokens: Long?,
/** What that is out of, or null where the provider does not say. */
@@ -2837,7 +2867,21 @@ private fun SessionStatusRow(
modifier = Modifier.padding(start = 8.dp),
)
}
Spacer(Modifier.weight(1f))
// The wait's own measurement, in the row's free width rather than in a line of its
// own: a bar that came and went would move the transcript and the box under the
// reader every time a turn started. The same place a compaction's bar goes, and
// determinate here because unlike a compaction this one is actually being measured
// -- see `SessionProgress`.
if (progress != null) {
LinearProgressIndicator(
progress = { progress.fraction },
color = commandColor,
trackColor = MaterialTheme.colorScheme.surfaceContainerHigh,
modifier = Modifier.weight(1f).padding(horizontal = 8.dp),
)
} else {
Spacer(Modifier.weight(1f))
}
}
// Every remaining state says which one it is, including the quiet one. The row used to
// name only `exited` and leave the rest blank, so a session sitting idle and one whose
@@ -2869,14 +2913,41 @@ private fun SessionStatusRow(
// and the two used to share an appearance: a session just cleared, one whose provider never
// reports usage, and one that has not run a turn all showed nothing at all, which reads as
// a conversation with room to spare.
//
// A wait that is being measured takes this place instead, because what the reader is
// asking while one is on is how much longer -- and for the wait that has one, the context
// figure is about to change anyway: a model that has not finished loading is holding
// nothing, and a prompt half read is a context still being counted.
Text(
contextLabel(contextTokens, contextLimit),
progress?.let { percentLabel(it) } ?: contextLabel(contextTokens, contextLimit),
style = MaterialTheme.typography.labelSmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
}
}
/**
* A wait's fraction as words: how far along, and which part where there is more than one.
*
* The stage is named because without it the number goes back to nothing part way through and reads
* as a bar that has broken -- a model with a projector and a draft head loads three files, each
* counted from zero. The provider's own word is passed through where this build does not know it,
* the same rule `sessionStatusWord` follows: the nearest word we do know would read as a fact
* somebody established.
*/
private fun percentLabel(progress: SessionProgress): String {
val percent = "${(progress.fraction.coerceIn(0f, 1f) * 100).toInt()}%"
val stage =
when (progress.stage) {
null -> return percent
"text_model" -> "weights"
"mmproj_model" -> "projector"
"spec_model" -> "draft head"
else -> progress.stage
}
return "$stage $percent"
}
/**
* A question (or permission request -- same shape) inline in the transcript. Option buttons until
* answered; then the chosen answer, which the `answered` event also resolves on every other
@@ -2907,6 +2978,23 @@ private fun QuestionRow(
*/
private fun atEnd(text: String) = TextFieldValue(text, TextRange(text.length))
/**
* The statuses that have a measurement behind them, and so are worth asking about.
*
* Both are llama.cpp's: a model coming off disk, and a prompt being read. Every other state answers
* null, and a session that is idle would be asked for ever.
*/
private val measurableWaits = setOf("loading", "reading")
/**
* How often the wait's fraction is asked for while one is on.
*
* Fast enough that a bar moves rather than steps, slow enough to be nothing next to what it is
* measuring: a model load is tens of seconds at best. It is two requests a second for as long as
* somebody is watching a wait, and none at all otherwise.
*/
private const val PROGRESS_POLL_MS = 500L
/**
* How long after a menu closes a press on its own button still counts as the press that closed it.
*