Say how far a llama session's wait has got

Both of its waits are measured somewhere and neither reached the phone: a
model coming off disk, which the router publishes on its event stream and
nowhere else, and a prompt being read, which the generation stream will
report when asked. A session now answers GET /sessions/{id}/progress with
{of, fraction, stage?}, one thread per router keeping the load's fraction
per model, and the session screen asks twice a second while it is drawing
a wait that has one.

Asked for rather than emitted: a load reports five times a second, and an
event is a transcript line for ever. The sample says which status it
measures, so one that outlived its wait cannot be drawn under another
word. The phone puts the bar in the status row's free width and the
percentage where the context figure sits -- a row of its own would move
the transcript every time a turn started -- and names the stage where a
model loads more than one file, because the fraction starts again for
each. /loading and /reading in an echo session are the rig.
This commit is contained in:
iris-ai committed 2026-09-21 02:44:03 -04:00
1 parent 78f2fe3b79
commit 049780fda6
10 files changed
+549 -16

No files matched your search

+25
View File
@@ -477,6 +477,31 @@ deliberate and easy to undo by accident:
only a turn whose model was changed under it. The conversation is read
*before* the message is announced, which is what makes "everything before
this message" true rather than a race against the pump.
- **A wait that can be measured says how far along it is** (2026-09-21,
`GET /sessions/{id}/progress`, `Driver::progress`). Both of this
driver's waits have a real number behind them and neither used to reach
the phone: the router publishes a model's load progress on its
`/models/sse` event stream -- and *only* there, since `GET /models`
reports that a model is loading and not how far -- while
`"return_progress": true` puts `prompt_progress` chunks in the generation
stream. One thread per router follows the event stream and keeps the
fraction per model (`router::watch_loads`), because a load belongs to the
machine: two sessions opening one model watch the same one arrive.
**It is state to be asked for, never an event.** A load reports five
times a second, and the transcript is the one thing every event a session
produces is written into -- a record of where a load got to is of no
interest to anybody once it has got there, and the phone is the only
reader. So the session screen asks twice a second while it is showing a
wait that has one, and not at all otherwise. The sample carries **which
status it measures**, so one taken during a wait that has since ended
cannot be drawn under a different word, and the phone shows it in the
status row's free width with the percentage where the context figure sits
-- a bar in a row of its own would move the transcript and the composer
every time a turn started. The stage is named where a model loads more
than one file (weights, projector, draft head), because the fraction
starts again for each and a bar that only counted up would be lying.
`/loading [seconds] [stages]` and `/reading [seconds]` in an echo session
are the rig for the phone's half.
- **Which tools a session offers is a filter here, not a flag there**
(2026-09-19). The router is always started with `--tools all` and hosts one
set of tools for the machine — one per session is not a thing a shared