Meter a session by its provider, and let llama.cpp run over ssh

The rate-limit bar answered a question about an account, and picked the
answer by machine. One machine runs echo, the Claude CLI and a local
model side by side, so every echo session on it drew the CLI's five-hour
window: a quota that session cannot spend and could never run down. A
session now names its meter (`usageProvider`, from
`DriverKind::usage_provider`, which `usage::providers_for` reads too so
the two lists cannot disagree), and the phone matches on machine *and*
provider. Nothing meters echo or llama, and nothing at all is drawn --
including while the first fetch is out, since "checking" under a session
that turns out to meter nothing is a row the screen then withdraws.

Echo gets a meter it can be *told* about instead: `/usage 42`,
`/usage 95 20`, `/usage 42 never`, `/usage notloggedin`,
`/usage unreachable`, `/usage failed`, `/usage off`. Those states cost
real quota to arrange, which is why none of them had been looked at.

And llama.cpp runs wherever a setup says, which was the last of phase 5.
`Transport::reserve_port` is the second half of what a transport is --
"run this" plus "reach this port" -- returning the port the server binds
there and the port that reaches it here, and `Launch::reaching` puts the
`-L` tunnel on the connection that already carries the command. Three
things that came out of building it:

- A forwarded launch gets a pty and every other one keeps `-T`. Killing
  the ssh client ends a CLI by closing the stdin it reads; llama-server
  never reads its stdin, so the same kill left it running on the far
  machine with the model loaded -- one orphan per stopped session.
- The model is looked for on the machine that will serve it, at that
  machine's own models directory, so `GET /setups/{id}/models` is what
  the spawn screen offers rather than the backend's own downloads.
- The readiness poll watches the process, not only the port: a model
  that will not load exits in a second and would otherwise have been
  reported as "gave up after 300s". The failure carries the log's tail.

Exercised end to end against this VM over ssh to itself: spawn, load,
answer, outlive a backend restart, be adopted, answer again, and stop --
with both the ssh client and the far llama-server gone afterwards. The
local path, the Claude bar and the spawn screen checked on the emulator.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
irisandClaude Opus 5 committed 2026-09-04 17:45:32 -04:00
1 parent 74110b4d72
commit 127b25e60a
20 files changed
+1212 -143

No files matched your search

+37 -1
View File
@@ -28,6 +28,13 @@
//! - `/error [text]` -- a failure, which is otherwise awkward to cause.
//! - `/peer [text]` -- a message from another agent, which otherwise takes
//! two live sessions and one of them deciding to write.
//! - `/usage [what]` -- puts up an invented rate-limit answer, or takes
//! it away again (`/usage off`). An echo session meters nothing, so it
//! draws no usage bar at all until this is set; what it exists for is
//! the states the bar can be in, which otherwise cost real quota to
//! reach. `/usage 42`, `/usage 95 20`, `/usage 42 never`,
//! `/usage notloggedin`, `/usage unreachable`, `/usage failed`. The
//! vocabulary is `usage::Fixture`'s, which is where the states live.
//! - `/compact` -- a compaction, start to finish. Typed rather than
//! pressed, because the real dialects take it as a typed command too and
//! the phone no longer has a button for it.
@@ -106,6 +113,11 @@ pub struct EchoDriver {
/// way AskUserQuestion does, and the turn resumes when the last of
/// them is answered rather than the first.
pending_questions: Mutex<Vec<PendingQuestion>>,
/// The invented rate-limit answer `/usage` sets, shared with the
/// usage monitor that serves it. An echo session meters nothing, so
/// this is unset until a test asks for something -- see
/// [`crate::usage::Fixture`].
usage: crate::usage::Fixture,
/// A pretend context, so the status row has something that behaves the
/// way a real one does: it grows with each turn, drops to what the
/// compaction says it recovered, and a clear leaves it unmeasured. The
@@ -345,6 +357,29 @@ impl EchoDriver {
return;
}
// Answered here rather than in the turn below, because it is not
// a turn: nothing is generated, and what is being exercised is
// the *other* screens -- the bar under the header, the button
// beside it and the dialog it opens, all of which read the usage
// route rather than this transcript.
if let Some(rest) = text.strip_prefix("/usage") {
if announce {
self.emit(Event::MessageTaken {
id: None,
text: text.clone(),
attachments,
});
}
let said = self.usage.command(rest);
self.emit(Event::AssistantText {
delta: format!("{said}\n"),
});
self.emit(Event::Status {
state: SessionStatus::Idle,
});
return;
}
// The same word the real CLI takes, so a phone drives both the same
// way. `Driver::compact` is what the manager's own route calls;
// this is the typed path onto it.
@@ -651,7 +686,7 @@ impl EchoDriver {
});
}
pub fn new(sink: EventSink, session_dir: PathBuf) -> Self {
pub fn new(sink: EventSink, session_dir: PathBuf, usage: crate::usage::Fixture) -> Self {
let driver = Self {
sink,
pending_questions: Mutex::new(Vec::new()),
@@ -659,6 +694,7 @@ impl EchoDriver {
busy: Arc::new(AtomicBool::new(false)),
queued: Arc::new(Mutex::new(Vec::new())),
session_dir,
usage,
};
driver.emit(Event::Status {
state: SessionStatus::Idle,