Don't draw a wait's bar at nothing

A fraction of zero is the absence of a sample, not a measurement of the
work: a bar at nothing for two minutes says the load has not started,
which of one 8 GB into a 12 GB file is false.

Measured raw off the router's event stream today: llama-server reports a
model's load as 0 and then 1 with nothing in between, on both the 0.6B and
the 27B, and reports prompt processing once a batch. So the bar now draws
for a long prompt -- the wait worth watching -- and a load that says
nothing draws none, which is what not knowing should look like.
This commit is contained in:
iris-ai committed 2026-09-21 03:01:02 -04:00
1 parent 66b3403a71
commit 849c3b599f
4 files changed
+50 -15

No files matched your search

+6 -1
View File
@@ -132,7 +132,12 @@ Module-by-module intent is in PLAN.md's "Backend layout".
`"return_progress": true` adds to the generation stream. The sample says `"return_progress": true` adds to the generation stream. The sample says
which status it measures so it cannot be drawn under another one, and which status it measures so it cannot be drawn under another one, and
`/loading [seconds] [stages]` / `/reading [seconds]` in an echo session are `/loading [seconds] [stages]` / `/reading [seconds]` in an echo session are
the rig for the phone's half. the rig for the phone's half. **Zero is never reported**, being the absence
of a sample rather than a measurement -- which matters because
`llama-server` reports a model's load as 0 and then 1 with nothing between
(measured 2026-09-21 on both the 0.6B and the 27B), and reports prompt
processing once a batch; so the bar usually draws for a long prompt and not
for a load.
**A turn's wait has two halves and says which** (2026-09-19): **A turn's wait has two halves and says which** (2026-09-19):
`SessionStatus::Loading` is the model coming off disk and `SessionStatus::Loading` is the model coming off disk and
`SessionStatus::Reading` is `llama-server` processing the prompt -- emitted `SessionStatus::Reading` is `llama-server` processing the prompt -- emitted
+9
View File
@@ -502,6 +502,15 @@ deliberate and easy to undo by accident:
starts again for each and a bar that only counted up would be lying. starts again for each and a bar that only counted up would be lying.
`/loading [seconds] [stages]` and `/reading [seconds]` in an echo session `/loading [seconds] [stages]` and `/reading [seconds]` in an echo session
are the rig for the phone's half. are the rig for the phone's half.
**A fraction of zero is not reported at all**, because it is the absence of
a sample rather than a measurement: a bar sitting at nothing for two
minutes says "this has not started", which of a load 8 GB into a 12 GB file
is false. That is not theoretical -- measured 2026-09-21, raw off
`/models/sse`, `llama-server` reports a load as **0 and then 1** with
nothing in between (both the 0.6B and the 27B), and prompt processing
reports **once a batch**, so a prompt under `n_batch` is one step as well.
So what usually draws a moving bar is a long prompt, which is also the wait
worth watching -- tens of seconds at the context sizes measured above.
- **Which tools a session offers is a filter here, not a flag there** - **Which tools a session offers is a filter here, not a flag there**
(2026-09-19). The router is always started with `--tools all` and hosts one (2026-09-19). The router is always started with `--tools all` and hosts one
set of tools for the machine — one per session is not a thing a shared set of tools for the machine — one per session is not a thing a shared
+28 -10
View File
@@ -1501,6 +1501,15 @@ impl Driver for LlamaDriver {
/// Which of the two waits this session is in decides which measurement /// Which of the two waits this session is in decides which measurement
/// answers, so a fraction from one can never be drawn under the other -- /// answers, so a fraction from one can never be drawn under the other --
/// and a session that is in neither says nothing rather than zero. /// and a session that is in neither says nothing rather than zero.
///
/// **Nothing is reported at a fraction of zero**, because that is the
/// absence of a sample rather than a measurement of the work: a bar
/// sitting at nothing for two minutes says "this has not started", which
/// of a load 8 GB into a 12 GB file is simply false. It matters here
/// rather than in theory -- `llama-server` reports a model's load as 0
/// and then 1 with nothing in between (measured 2026-09-21 against both
/// the 0.6B and the 27B, raw off `/models/sse`), so what usually draws a
/// bar is prompt processing, which reports a batch at a time.
fn progress(&self) -> Option<Progress> { fn progress(&self) -> Option<Progress> {
let coming = match &*self let coming = match &*self
.shared .shared
@@ -1513,20 +1522,19 @@ impl Driver for LlamaDriver {
}; };
if let Some(model) = coming { if let Some(model) = coming {
let coming = self.respawn.router.loading(&model)?; let coming = self.respawn.router.loading(&model)?;
return Some(Progress { return measured(SessionStatus::Loading, coming.fraction, coming.stage);
of: SessionStatus::Loading,
fraction: coming.fraction,
stage: coming.stage,
});
} }
let (processed, total) = (*self.shared.reading.lock().unwrap())?; let (processed, total) = (*self.shared.reading.lock().unwrap())?;
// A prompt of nothing is not a wait, and it is what a division would // A prompt of nothing is not a wait, and it is what a division would
// fail on. // fail on.
(total > 0).then(|| Progress { if total == 0 {
of: SessionStatus::Reading, return None;
fraction: processed as f32 / total as f32, }
stage: None, measured(
}) SessionStatus::Reading,
processed as f32 / total as f32,
None,
)
} }
fn interrupt(&self) { fn interrupt(&self) {
@@ -1767,6 +1775,16 @@ fn conversation(path: &Path, images: Option<&Path>) -> Vec<Message> {
fold.messages fold.messages
} }
/// A sample, or `None` where it says nothing -- see [`Driver::progress`] on
/// why zero is not a measurement.
fn measured(of: SessionStatus, fraction: f32, stage: Option<String>) -> Option<Progress> {
(fraction > 0.0).then_some(Progress {
of,
fraction,
stage,
})
}
/// One message somebody sent, as the model is to read it. /// One message somebody sent, as the model is to read it.
/// ///
/// `images` is [`conversation`]'s: the session directory for a model with /// `images` is [`conversation`]'s: the session directory for a model with
+7 -4
View File
@@ -344,15 +344,18 @@ pub struct TranscriptFile {
pub path: String, pub path: String,
} }
/// Where a session directory keeps its transcript.
fn transcript_in(dir: &Path) -> PathBuf {
dir.join("transcript.jsonl")
}
/// Where `id`'s transcript is, said the way the explorer takes it. /// Where `id`'s transcript is, said the way the explorer takes it.
fn transcript_file(config: &Config, data_dir: &Path, id: &str) -> Option<TranscriptFile> { fn transcript_file(config: &Config, data_dir: &Path, id: &str) -> Option<TranscriptFile> {
let machine = config.machine(crate::config::LOCAL_MACHINE_ID)?; let machine = config.machine(crate::config::LOCAL_MACHINE_ID)?;
Some(TranscriptFile { Some(TranscriptFile {
machine: machine.id.clone(), machine: machine.id.clone(),
machine_name: machine.name.clone(), machine_name: machine.name.clone(),
path: data_dir path: transcript_in(&data_dir.join(id))
.join(id)
.join("transcript.jsonl")
.to_string_lossy() .to_string_lossy()
.into_owned(), .into_owned(),
}) })
@@ -2629,7 +2632,7 @@ fn launch(
) -> Result<Arc<LiveSession>> { ) -> Result<Arc<LiveSession>> {
let dir = env.data_dir.join(&meta.id); let dir = env.data_dir.join(&meta.id);
wg_app_link::private::create_dir(&dir)?; wg_app_link::private::create_dir(&dir)?;
let transcript_path = dir.join("transcript.jsonl"); let transcript_path = transcript_in(&dir);
let mut transcript = Transcript::open(&transcript_path)?; let mut transcript = Transcript::open(&transcript_path)?;
let last_status = transcript.last_status().unwrap_or(SessionStatus::Idle); let last_status = transcript.last_status().unwrap_or(SessionStatus::Idle);
// Before the driver starts, so the token is there when it looks and the // Before the driver starts, so the token is there when it looks and the