Say how far a llama session's wait has got
Both of its waits are measured somewhere and neither reached the phone: a
model coming off disk, which the router publishes on its event stream and
nowhere else, and a prompt being read, which the generation stream will
report when asked. A session now answers GET /sessions/{id}/progress with
{of, fraction, stage?}, one thread per router keeping the load's fraction
per model, and the session screen asks twice a second while it is drawing
a wait that has one.
Asked for rather than emitted: a load reports five times a second, and an
event is a transcript line for ever. The sample says which status it
measures, so one that outlived its wait cannot be drawn under another
word. The phone puts the bar in the status row's free width and the
percentage where the context figure sits -- a row of its own would move
the transcript every time a turn started -- and names the stage where a
model loads more than one file, because the fraction starts again for
each. /loading and /reading in an echo session are the rig.
This commit is contained in:
1 parent
78f2fe3b79
commit
049780fda6
10 files changed
+549
-16
No files matched your search
@@ -121,6 +121,18 @@ Module-by-module intent is in PLAN.md's "Backend layout".
|
||||
"read 9.5s · 50.3 tok/s · 3:00 PM" under a finished reply — nothing else here
|
||||
measures either, so every other driver sends `None`, and the clock is last so
|
||||
that it does not move when a provider reports fewer of them.
|
||||
**A wait that can be measured says how far along it is** (2026-09-21):
|
||||
`GET /sessions/{id}/progress` answers `{of, fraction, stage?}` and `null`
|
||||
for a session that is not in one -- runtime state, asked for twice a second
|
||||
by the session screen while it is drawing a wait, and deliberately never an
|
||||
event, since a load reports five times a second and every event is a
|
||||
transcript line for ever. The two sources are the router's `/models/sse`
|
||||
stream, which is the **only** place a model's load progress appears (`GET
|
||||
/models` says "loading" and no more), and `prompt_progress` chunks that
|
||||
`"return_progress": true` adds to the generation stream. The sample says
|
||||
which status it measures so it cannot be drawn under another one, and
|
||||
`/loading [seconds] [stages]` / `/reading [seconds]` in an echo session are
|
||||
the rig for the phone's half.
|
||||
**A turn's wait has two halves and says which** (2026-09-19):
|
||||
`SessionStatus::Loading` is the model coming off disk and
|
||||
`SessionStatus::Reading` is `llama-server` processing the prompt -- emitted
|
||||
|
||||
Reference in new issue
Block a user