Don't draw a wait's bar at nothing
A fraction of zero is the absence of a sample, not a measurement of the work: a bar at nothing for two minutes says the load has not started, which of one 8 GB into a 12 GB file is false. Measured raw off the router's event stream today: llama-server reports a model's load as 0 and then 1 with nothing in between, on both the 0.6B and the 27B, and reports prompt processing once a batch. So the bar now draws for a long prompt -- the wait worth watching -- and a load that says nothing draws none, which is what not knowing should look like.
This commit is contained in:
1 parent
66b3403a71
commit
849c3b599f
4 files changed
+50
-15
No files matched your search
@@ -502,6 +502,15 @@ deliberate and easy to undo by accident:
|
||||
starts again for each and a bar that only counted up would be lying.
|
||||
`/loading [seconds] [stages]` and `/reading [seconds]` in an echo session
|
||||
are the rig for the phone's half.
|
||||
**A fraction of zero is not reported at all**, because it is the absence of
|
||||
a sample rather than a measurement: a bar sitting at nothing for two
|
||||
minutes says "this has not started", which of a load 8 GB into a 12 GB file
|
||||
is false. That is not theoretical -- measured 2026-09-21, raw off
|
||||
`/models/sse`, `llama-server` reports a load as **0 and then 1** with
|
||||
nothing in between (both the 0.6B and the 27B), and prompt processing
|
||||
reports **once a batch**, so a prompt under `n_batch` is one step as well.
|
||||
So what usually draws a moving bar is a long prompt, which is also the wait
|
||||
worth watching -- tens of seconds at the context sizes measured above.
|
||||
- **Which tools a session offers is a filter here, not a flag there**
|
||||
(2026-09-19). The router is always started with `--tools all` and hosts one
|
||||
set of tools for the machine — one per session is not a thing a shared
|
||||
|
||||
Reference in new issue
Block a user