Don't draw a wait's bar at nothing

A fraction of zero is the absence of a sample, not a measurement of the
work: a bar at nothing for two minutes says the load has not started,
which of one 8 GB into a 12 GB file is false.

Measured raw off the router's event stream today: llama-server reports a
model's load as 0 and then 1 with nothing in between, on both the 0.6B and
the 27B, and reports prompt processing once a batch. So the bar now draws
for a long prompt -- the wait worth watching -- and a load that says
nothing draws none, which is what not knowing should look like.
This commit is contained in:
iris-ai committed 2026-09-21 03:01:02 -04:00
1 parent 66b3403a71
commit 849c3b599f
4 files changed
+50 -15

No files matched your search

+6 -1
View File
@@ -132,7 +132,12 @@ Module-by-module intent is in PLAN.md's "Backend layout".
`"return_progress": true` adds to the generation stream. The sample says
which status it measures so it cannot be drawn under another one, and
`/loading [seconds] [stages]` / `/reading [seconds]` in an echo session are
the rig for the phone's half.
the rig for the phone's half. **Zero is never reported**, being the absence
of a sample rather than a measurement -- which matters because
`llama-server` reports a model's load as 0 and then 1 with nothing between
(measured 2026-09-21 on both the 0.6B and the 27B), and reports prompt
processing once a batch; so the bar usually draws for a long prompt and not
for a load.
**A turn's wait has two halves and says which** (2026-09-19):
`SessionStatus::Loading` is the model coming off disk and
`SessionStatus::Reading` is `llama-server` processing the prompt -- emitted
+9
View File
@@ -502,6 +502,15 @@ deliberate and easy to undo by accident:
starts again for each and a bar that only counted up would be lying.
`/loading [seconds] [stages]` and `/reading [seconds]` in an echo session
are the rig for the phone's half.
**A fraction of zero is not reported at all**, because it is the absence of
a sample rather than a measurement: a bar sitting at nothing for two
minutes says "this has not started", which of a load 8 GB into a 12 GB file
is false. That is not theoretical -- measured 2026-09-21, raw off
`/models/sse`, `llama-server` reports a load as **0 and then 1** with
nothing in between (both the 0.6B and the 27B), and prompt processing
reports **once a batch**, so a prompt under `n_batch` is one step as well.
So what usually draws a moving bar is a long prompt, which is also the wait
worth watching -- tens of seconds at the context sizes measured above.
- **Which tools a session offers is a filter here, not a flag there**
(2026-09-19). The router is always started with `--tools all` and hosts one
set of tools for the machine — one per session is not a thing a shared
+28 -10
View File
@@ -1501,6 +1501,15 @@ impl Driver for LlamaDriver {
/// Which of the two waits this session is in decides which measurement
/// answers, so a fraction from one can never be drawn under the other --
/// and a session that is in neither says nothing rather than zero.
///
/// **Nothing is reported at a fraction of zero**, because that is the
/// absence of a sample rather than a measurement of the work: a bar
/// sitting at nothing for two minutes says "this has not started", which
/// of a load 8 GB into a 12 GB file is simply false. It matters here
/// rather than in theory -- `llama-server` reports a model's load as 0
/// and then 1 with nothing in between (measured 2026-09-21 against both
/// the 0.6B and the 27B, raw off `/models/sse`), so what usually draws a
/// bar is prompt processing, which reports a batch at a time.
fn progress(&self) -> Option<Progress> {
let coming = match &*self
.shared
@@ -1513,20 +1522,19 @@ impl Driver for LlamaDriver {
};
if let Some(model) = coming {
let coming = self.respawn.router.loading(&model)?;
return Some(Progress {
of: SessionStatus::Loading,
fraction: coming.fraction,
stage: coming.stage,
});
return measured(SessionStatus::Loading, coming.fraction, coming.stage);
}
let (processed, total) = (*self.shared.reading.lock().unwrap())?;
// A prompt of nothing is not a wait, and it is what a division would
// fail on.
(total > 0).then(|| Progress {
of: SessionStatus::Reading,
fraction: processed as f32 / total as f32,
stage: None,
})
if total == 0 {
return None;
}
measured(
SessionStatus::Reading,
processed as f32 / total as f32,
None,
)
}
fn interrupt(&self) {
@@ -1767,6 +1775,16 @@ fn conversation(path: &Path, images: Option<&Path>) -> Vec<Message> {
fold.messages
}
/// A sample, or `None` where it says nothing -- see [`Driver::progress`] on
/// why zero is not a measurement.
fn measured(of: SessionStatus, fraction: f32, stage: Option<String>) -> Option<Progress> {
(fraction > 0.0).then_some(Progress {
of,
fraction,
stage,
})
}
/// One message somebody sent, as the model is to read it.
///
/// `images` is [`conversation`]'s: the session directory for a model with
+7 -4
View File
@@ -344,15 +344,18 @@ pub struct TranscriptFile {
pub path: String,
}
/// Where a session directory keeps its transcript.
fn transcript_in(dir: &Path) -> PathBuf {
dir.join("transcript.jsonl")
}
/// Where `id`'s transcript is, said the way the explorer takes it.
fn transcript_file(config: &Config, data_dir: &Path, id: &str) -> Option<TranscriptFile> {
let machine = config.machine(crate::config::LOCAL_MACHINE_ID)?;
Some(TranscriptFile {
machine: machine.id.clone(),
machine_name: machine.name.clone(),
path: data_dir
.join(id)
.join("transcript.jsonl")
path: transcript_in(&data_dir.join(id))
.to_string_lossy()
.into_owned(),
})
@@ -2629,7 +2632,7 @@ fn launch(
) -> Result<Arc<LiveSession>> {
let dir = env.data_dir.join(&meta.id);
wg_app_link::private::create_dir(&dir)?;
let transcript_path = dir.join("transcript.jsonl");
let transcript_path = transcript_in(&dir);
let mut transcript = Transcript::open(&transcript_path)?;
let last_status = transcript.last_status().unwrap_or(SessionStatus::Idle);
// Before the driver starts, so the token is there when it looks and the