Don't draw a wait's bar at nothing
A fraction of zero is the absence of a sample, not a measurement of the work: a bar at nothing for two minutes says the load has not started, which of one 8 GB into a 12 GB file is false. Measured raw off the router's event stream today: llama-server reports a model's load as 0 and then 1 with nothing in between, on both the 0.6B and the 27B, and reports prompt processing once a batch. So the bar now draws for a long prompt -- the wait worth watching -- and a load that says nothing draws none, which is what not knowing should look like.
This commit is contained in:
1 parent
66b3403a71
commit
849c3b599f
4 files changed
+50
-15
No files matched your search
@@ -132,7 +132,12 @@ Module-by-module intent is in PLAN.md's "Backend layout".
|
||||
`"return_progress": true` adds to the generation stream. The sample says
|
||||
which status it measures so it cannot be drawn under another one, and
|
||||
`/loading [seconds] [stages]` / `/reading [seconds]` in an echo session are
|
||||
the rig for the phone's half.
|
||||
the rig for the phone's half. **Zero is never reported**, being the absence
|
||||
of a sample rather than a measurement -- which matters because
|
||||
`llama-server` reports a model's load as 0 and then 1 with nothing between
|
||||
(measured 2026-09-21 on both the 0.6B and the 27B), and reports prompt
|
||||
processing once a batch; so the bar usually draws for a long prompt and not
|
||||
for a load.
|
||||
**A turn's wait has two halves and says which** (2026-09-19):
|
||||
`SessionStatus::Loading` is the model coming off disk and
|
||||
`SessionStatus::Reading` is `llama-server` processing the prompt -- emitted
|
||||
|
||||
@@ -502,6 +502,15 @@ deliberate and easy to undo by accident:
|
||||
starts again for each and a bar that only counted up would be lying.
|
||||
`/loading [seconds] [stages]` and `/reading [seconds]` in an echo session
|
||||
are the rig for the phone's half.
|
||||
**A fraction of zero is not reported at all**, because it is the absence of
|
||||
a sample rather than a measurement: a bar sitting at nothing for two
|
||||
minutes says "this has not started", which of a load 8 GB into a 12 GB file
|
||||
is false. That is not theoretical -- measured 2026-09-21, raw off
|
||||
`/models/sse`, `llama-server` reports a load as **0 and then 1** with
|
||||
nothing in between (both the 0.6B and the 27B), and prompt processing
|
||||
reports **once a batch**, so a prompt under `n_batch` is one step as well.
|
||||
So what usually draws a moving bar is a long prompt, which is also the wait
|
||||
worth watching -- tens of seconds at the context sizes measured above.
|
||||
- **Which tools a session offers is a filter here, not a flag there**
|
||||
(2026-09-19). The router is always started with `--tools all` and hosts one
|
||||
set of tools for the machine — one per session is not a thing a shared
|
||||
|
||||
@@ -1501,6 +1501,15 @@ impl Driver for LlamaDriver {
|
||||
/// Which of the two waits this session is in decides which measurement
|
||||
/// answers, so a fraction from one can never be drawn under the other --
|
||||
/// and a session that is in neither says nothing rather than zero.
|
||||
///
|
||||
/// **Nothing is reported at a fraction of zero**, because that is the
|
||||
/// absence of a sample rather than a measurement of the work: a bar
|
||||
/// sitting at nothing for two minutes says "this has not started", which
|
||||
/// of a load 8 GB into a 12 GB file is simply false. It matters here
|
||||
/// rather than in theory -- `llama-server` reports a model's load as 0
|
||||
/// and then 1 with nothing in between (measured 2026-09-21 against both
|
||||
/// the 0.6B and the 27B, raw off `/models/sse`), so what usually draws a
|
||||
/// bar is prompt processing, which reports a batch at a time.
|
||||
fn progress(&self) -> Option<Progress> {
|
||||
let coming = match &*self
|
||||
.shared
|
||||
@@ -1513,20 +1522,19 @@ impl Driver for LlamaDriver {
|
||||
};
|
||||
if let Some(model) = coming {
|
||||
let coming = self.respawn.router.loading(&model)?;
|
||||
return Some(Progress {
|
||||
of: SessionStatus::Loading,
|
||||
fraction: coming.fraction,
|
||||
stage: coming.stage,
|
||||
});
|
||||
return measured(SessionStatus::Loading, coming.fraction, coming.stage);
|
||||
}
|
||||
let (processed, total) = (*self.shared.reading.lock().unwrap())?;
|
||||
// A prompt of nothing is not a wait, and it is what a division would
|
||||
// fail on.
|
||||
(total > 0).then(|| Progress {
|
||||
of: SessionStatus::Reading,
|
||||
fraction: processed as f32 / total as f32,
|
||||
stage: None,
|
||||
})
|
||||
if total == 0 {
|
||||
return None;
|
||||
}
|
||||
measured(
|
||||
SessionStatus::Reading,
|
||||
processed as f32 / total as f32,
|
||||
None,
|
||||
)
|
||||
}
|
||||
|
||||
fn interrupt(&self) {
|
||||
@@ -1767,6 +1775,16 @@ fn conversation(path: &Path, images: Option<&Path>) -> Vec<Message> {
|
||||
fold.messages
|
||||
}
|
||||
|
||||
/// A sample, or `None` where it says nothing -- see [`Driver::progress`] on
|
||||
/// why zero is not a measurement.
|
||||
fn measured(of: SessionStatus, fraction: f32, stage: Option<String>) -> Option<Progress> {
|
||||
(fraction > 0.0).then_some(Progress {
|
||||
of,
|
||||
fraction,
|
||||
stage,
|
||||
})
|
||||
}
|
||||
|
||||
/// One message somebody sent, as the model is to read it.
|
||||
///
|
||||
/// `images` is [`conversation`]'s: the session directory for a model with
|
||||
|
||||
@@ -344,15 +344,18 @@ pub struct TranscriptFile {
|
||||
pub path: String,
|
||||
}
|
||||
|
||||
/// Where a session directory keeps its transcript.
|
||||
fn transcript_in(dir: &Path) -> PathBuf {
|
||||
dir.join("transcript.jsonl")
|
||||
}
|
||||
|
||||
/// Where `id`'s transcript is, said the way the explorer takes it.
|
||||
fn transcript_file(config: &Config, data_dir: &Path, id: &str) -> Option<TranscriptFile> {
|
||||
let machine = config.machine(crate::config::LOCAL_MACHINE_ID)?;
|
||||
Some(TranscriptFile {
|
||||
machine: machine.id.clone(),
|
||||
machine_name: machine.name.clone(),
|
||||
path: data_dir
|
||||
.join(id)
|
||||
.join("transcript.jsonl")
|
||||
path: transcript_in(&data_dir.join(id))
|
||||
.to_string_lossy()
|
||||
.into_owned(),
|
||||
})
|
||||
@@ -2629,7 +2632,7 @@ fn launch(
|
||||
) -> Result<Arc<LiveSession>> {
|
||||
let dir = env.data_dir.join(&meta.id);
|
||||
wg_app_link::private::create_dir(&dir)?;
|
||||
let transcript_path = dir.join("transcript.jsonl");
|
||||
let transcript_path = transcript_in(&dir);
|
||||
let mut transcript = Transcript::open(&transcript_path)?;
|
||||
let last_status = transcript.last_status().unwrap_or(SessionStatus::Idle);
|
||||
// Before the driver starts, so the token is there when it looks and the
|
||||
|
||||
Reference in new issue
Block a user