Say how far a llama session's wait has got

Both of its waits are measured somewhere and neither reached the phone: a
model coming off disk, which the router publishes on its event stream and
nowhere else, and a prompt being read, which the generation stream will
report when asked. A session now answers GET /sessions/{id}/progress with
{of, fraction, stage?}, one thread per router keeping the load's fraction
per model, and the session screen asks twice a second while it is drawing
a wait that has one.

Asked for rather than emitted: a load reports five times a second, and an
event is a transcript line for ever. The sample says which status it
measures, so one that outlived its wait cannot be drawn under another
word. The phone puts the bar in the status row's free width and the
percentage where the context figure sits -- a row of its own would move
the transcript every time a turn started -- and names the stage where a
model loads more than one file, because the fraction starts again for
each. /loading and /reading in an echo session are the rig.
This commit is contained in:
iris-ai committed 2026-09-21 02:44:03 -04:00
1 parent 78f2fe3b79
commit 049780fda6
10 files changed
+549 -16

No files matched your search

@@ -362,6 +362,46 @@ fun fetchBackgroundTasks(
}
}
/**
* How far along the wait a session is in has got. See `GET /sessions/{id}/progress`.
*
* [of] is the status it measures, so a sample that outlived the wait it came from is never drawn
* under a different word: the row shows it only where the two agree.
*/
data class SessionProgress(
val of: String,
/** Between 0 and 1. */
val fraction: Float,
/**
* Which part of the wait this is, in the provider's own word, where it has more than one and
* the fraction starts again for each. Null where there is only one.
*/
val stage: String?,
)
/**
* How far [sessionId]'s current wait has got, or null when it is not in one that anything can
* measure -- which is most sessions most of the time, and every provider but llama.cpp.
*
* Asked for rather than streamed: a model load reports five times a second, and none of that
* belongs in the transcript the event stream carries.
*/
fun fetchProgress(settings: ServerSettings, sessionId: String): SessionProgress? =
requestFromServer(settings, "/sessions/$sessionId/progress") { connection ->
val body = connection.inputStream.bufferedReader().readText()
if (body.trim() == "null") null
else
JSONObject(body).let { row ->
SessionProgress(
of = row.getString("of"),
fraction = row.getDouble("fraction").toFloat(),
stage =
if (row.isNull("stage")) null
else row.optString("stage", "").ifEmpty { null },
)
}
}
fun fetchSubagents(settings: ServerSettings, sessionId: String): List<SubagentSummary> =
requestFromServer(settings, "/sessions/$sessionId/subagents") {
it.jsonObjects { row ->