Don't draw a wait's bar at nothing
A fraction of zero is the absence of a sample, not a measurement of the work: a bar at nothing for two minutes says the load has not started, which of one 8 GB into a 12 GB file is false. Measured raw off the router's event stream today: llama-server reports a model's load as 0 and then 1 with nothing in between, on both the 0.6B and the 27B, and reports prompt processing once a batch. So the bar now draws for a long prompt -- the wait worth watching -- and a load that says nothing draws none, which is what not knowing should look like.
This commit is contained in:
1 parent
66b3403a71
commit
849c3b599f
4 files changed
+50
-15
No files matched your search
@@ -132,7 +132,12 @@ Module-by-module intent is in PLAN.md's "Backend layout".
|
||||
`"return_progress": true` adds to the generation stream. The sample says
|
||||
which status it measures so it cannot be drawn under another one, and
|
||||
`/loading [seconds] [stages]` / `/reading [seconds]` in an echo session are
|
||||
the rig for the phone's half.
|
||||
the rig for the phone's half. **Zero is never reported**, being the absence
|
||||
of a sample rather than a measurement -- which matters because
|
||||
`llama-server` reports a model's load as 0 and then 1 with nothing between
|
||||
(measured 2026-09-21 on both the 0.6B and the 27B), and reports prompt
|
||||
processing once a batch; so the bar usually draws for a long prompt and not
|
||||
for a load.
|
||||
**A turn's wait has two halves and says which** (2026-09-19):
|
||||
`SessionStatus::Loading` is the model coming off disk and
|
||||
`SessionStatus::Reading` is `llama-server` processing the prompt -- emitted
|
||||
|
||||
Reference in new issue
Block a user