Meter a session by its provider, and let llama.cpp run over ssh
The rate-limit bar answered a question about an account, and picked the
answer by machine. One machine runs echo, the Claude CLI and a local
model side by side, so every echo session on it drew the CLI's five-hour
window: a quota that session cannot spend and could never run down. A
session now names its meter (`usageProvider`, from
`DriverKind::usage_provider`, which `usage::providers_for` reads too so
the two lists cannot disagree), and the phone matches on machine *and*
provider. Nothing meters echo or llama, and nothing at all is drawn --
including while the first fetch is out, since "checking" under a session
that turns out to meter nothing is a row the screen then withdraws.
Echo gets a meter it can be *told* about instead: `/usage 42`,
`/usage 95 20`, `/usage 42 never`, `/usage notloggedin`,
`/usage unreachable`, `/usage failed`, `/usage off`. Those states cost
real quota to arrange, which is why none of them had been looked at.
And llama.cpp runs wherever a setup says, which was the last of phase 5.
`Transport::reserve_port` is the second half of what a transport is --
"run this" plus "reach this port" -- returning the port the server binds
there and the port that reaches it here, and `Launch::reaching` puts the
`-L` tunnel on the connection that already carries the command. Three
things that came out of building it:
- A forwarded launch gets a pty and every other one keeps `-T`. Killing
the ssh client ends a CLI by closing the stdin it reads; llama-server
never reads its stdin, so the same kill left it running on the far
machine with the model loaded -- one orphan per stopped session.
- The model is looked for on the machine that will serve it, at that
machine's own models directory, so `GET /setups/{id}/models` is what
the spawn screen offers rather than the backend's own downloads.
- The readiness poll watches the process, not only the port: a model
that will not load exits in a second and would otherwise have been
reported as "gave up after 300s". The failure carries the log's tail.
Exercised end to end against this VM over ssh to itself: spawn, load,
answer, outlive a backend restart, be adopted, answer again, and stop --
with both the ssh client and the far llama-server gone afterwards. The
local path, the Claude bar and the spawn screen checked on the emulator.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
1 parent
74110b4d72
commit
127b25e60a
20 files changed
+1212
-143
No files matched your search
@@ -28,6 +28,13 @@
|
||||
//! - `/error [text]` -- a failure, which is otherwise awkward to cause.
|
||||
//! - `/peer [text]` -- a message from another agent, which otherwise takes
|
||||
//! two live sessions and one of them deciding to write.
|
||||
//! - `/usage [what]` -- puts up an invented rate-limit answer, or takes
|
||||
//! it away again (`/usage off`). An echo session meters nothing, so it
|
||||
//! draws no usage bar at all until this is set; what it exists for is
|
||||
//! the states the bar can be in, which otherwise cost real quota to
|
||||
//! reach. `/usage 42`, `/usage 95 20`, `/usage 42 never`,
|
||||
//! `/usage notloggedin`, `/usage unreachable`, `/usage failed`. The
|
||||
//! vocabulary is `usage::Fixture`'s, which is where the states live.
|
||||
//! - `/compact` -- a compaction, start to finish. Typed rather than
|
||||
//! pressed, because the real dialects take it as a typed command too and
|
||||
//! the phone no longer has a button for it.
|
||||
@@ -106,6 +113,11 @@ pub struct EchoDriver {
|
||||
/// way AskUserQuestion does, and the turn resumes when the last of
|
||||
/// them is answered rather than the first.
|
||||
pending_questions: Mutex<Vec<PendingQuestion>>,
|
||||
/// The invented rate-limit answer `/usage` sets, shared with the
|
||||
/// usage monitor that serves it. An echo session meters nothing, so
|
||||
/// this is unset until a test asks for something -- see
|
||||
/// [`crate::usage::Fixture`].
|
||||
usage: crate::usage::Fixture,
|
||||
/// A pretend context, so the status row has something that behaves the
|
||||
/// way a real one does: it grows with each turn, drops to what the
|
||||
/// compaction says it recovered, and a clear leaves it unmeasured. The
|
||||
@@ -345,6 +357,29 @@ impl EchoDriver {
|
||||
return;
|
||||
}
|
||||
|
||||
// Answered here rather than in the turn below, because it is not
|
||||
// a turn: nothing is generated, and what is being exercised is
|
||||
// the *other* screens -- the bar under the header, the button
|
||||
// beside it and the dialog it opens, all of which read the usage
|
||||
// route rather than this transcript.
|
||||
if let Some(rest) = text.strip_prefix("/usage") {
|
||||
if announce {
|
||||
self.emit(Event::MessageTaken {
|
||||
id: None,
|
||||
text: text.clone(),
|
||||
attachments,
|
||||
});
|
||||
}
|
||||
let said = self.usage.command(rest);
|
||||
self.emit(Event::AssistantText {
|
||||
delta: format!("{said}\n"),
|
||||
});
|
||||
self.emit(Event::Status {
|
||||
state: SessionStatus::Idle,
|
||||
});
|
||||
return;
|
||||
}
|
||||
|
||||
// The same word the real CLI takes, so a phone drives both the same
|
||||
// way. `Driver::compact` is what the manager's own route calls;
|
||||
// this is the typed path onto it.
|
||||
@@ -651,7 +686,7 @@ impl EchoDriver {
|
||||
});
|
||||
}
|
||||
|
||||
pub fn new(sink: EventSink, session_dir: PathBuf) -> Self {
|
||||
pub fn new(sink: EventSink, session_dir: PathBuf, usage: crate::usage::Fixture) -> Self {
|
||||
let driver = Self {
|
||||
sink,
|
||||
pending_questions: Mutex::new(Vec::new()),
|
||||
@@ -659,6 +694,7 @@ impl EchoDriver {
|
||||
busy: Arc::new(AtomicBool::new(false)),
|
||||
queued: Arc::new(Mutex::new(Vec::new())),
|
||||
session_dir,
|
||||
usage,
|
||||
};
|
||||
driver.emit(Event::Status {
|
||||
state: SessionStatus::Idle,
|
||||
|
||||
Reference in new issue
Block a user