Meter a session by its provider, and let llama.cpp run over ssh
The rate-limit bar answered a question about an account, and picked the
answer by machine. One machine runs echo, the Claude CLI and a local
model side by side, so every echo session on it drew the CLI's five-hour
window: a quota that session cannot spend and could never run down. A
session now names its meter (`usageProvider`, from
`DriverKind::usage_provider`, which `usage::providers_for` reads too so
the two lists cannot disagree), and the phone matches on machine *and*
provider. Nothing meters echo or llama, and nothing at all is drawn --
including while the first fetch is out, since "checking" under a session
that turns out to meter nothing is a row the screen then withdraws.
Echo gets a meter it can be *told* about instead: `/usage 42`,
`/usage 95 20`, `/usage 42 never`, `/usage notloggedin`,
`/usage unreachable`, `/usage failed`, `/usage off`. Those states cost
real quota to arrange, which is why none of them had been looked at.
And llama.cpp runs wherever a setup says, which was the last of phase 5.
`Transport::reserve_port` is the second half of what a transport is --
"run this" plus "reach this port" -- returning the port the server binds
there and the port that reaches it here, and `Launch::reaching` puts the
`-L` tunnel on the connection that already carries the command. Three
things that came out of building it:
- A forwarded launch gets a pty and every other one keeps `-T`. Killing
the ssh client ends a CLI by closing the stdin it reads; llama-server
never reads its stdin, so the same kill left it running on the far
machine with the model loaded -- one orphan per stopped session.
- The model is looked for on the machine that will serve it, at that
machine's own models directory, so `GET /setups/{id}/models` is what
the spawn screen offers rather than the backend's own downloads.
- The readiness poll watches the process, not only the port: a model
that will not load exits in a second and would otherwise have been
reported as "gave up after 300s". The failure carries the log's tail.
Exercised end to end against this VM over ssh to itself: spawn, load,
answer, outlive a backend restart, be adopted, answer again, and stop --
with both the ssh client and the far llama-server gone afterwards. The
local path, the Claude bar and the spawn screen checked on the emulator.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
1 parent
74110b4d72
commit
127b25e60a
20 files changed
+1212
-143
No files matched your search
@@ -74,13 +74,22 @@ class UsageFeed(
|
||||
/** Ask the backend again now. The dialog's refresh button; the poll does it on its own. */
|
||||
val refresh: () -> Unit,
|
||||
) {
|
||||
/** What [setup]'s own limits came back as. See [usageFor] for why the states are these. */
|
||||
fun forSetup(setup: String): SessionUsage =
|
||||
when (val state = snapshots) {
|
||||
/**
|
||||
* What meters [session], and what that meter came back as. See [usageFor] for the states.
|
||||
*
|
||||
* A session rather than a machine, because a machine is not what is metered: one machine runs
|
||||
* the Claude CLI and an echo session side by side, and only the first of them spends anything.
|
||||
*/
|
||||
fun forSession(session: SessionSummary): SessionUsage {
|
||||
// Settled without asking anybody: a session nothing meters has nothing to check, and
|
||||
// "checking" is what the fetch's own states would say about it for as long as one is out.
|
||||
val provider = session.usageProvider ?: return SessionUsage.NotMetered
|
||||
return when (val state = snapshots) {
|
||||
is LoadState.Loading -> SessionUsage.Waiting
|
||||
is LoadState.Error -> SessionUsage.Unavailable(state.message)
|
||||
is LoadState.Loaded -> usageFor(state.value, setup)
|
||||
is LoadState.Loaded -> usageFor(state.value, session.setup, provider)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -165,9 +174,16 @@ fun SessionUsageBar(usage: SessionUsage, modifier: Modifier = Modifier) {
|
||||
}
|
||||
}
|
||||
|
||||
// Nothing at all for a machine that meters nothing: a row saying "unknown" there would
|
||||
// Nothing at all for a session that meters nothing: a row saying "unknown" there would
|
||||
// report a problem about a setup somebody chose, on every screen, forever.
|
||||
if (usage is SessionUsage.NotMetered) {
|
||||
//
|
||||
// And nothing while the first fetch is out, which is not the same kind of silence. A
|
||||
// request in flight is not a state to report -- and the session that meters nothing is
|
||||
// exactly the one this cannot yet tell apart, so "5-hour usage: checking" appeared under
|
||||
// an echo session for half a second and was then taken away. A row that has to be
|
||||
// withdrawn is worse than one that arrives late, and this is the only state here whose
|
||||
// wrongness is a matter of timing rather than of fact.
|
||||
if (usage is SessionUsage.NotMetered || usage is SessionUsage.Waiting) {
|
||||
return
|
||||
}
|
||||
|
||||
@@ -178,9 +194,10 @@ fun SessionUsageBar(usage: SessionUsage, modifier: Modifier = Modifier) {
|
||||
// Words, not a colour and not an empty bar: every one of these is a different kind of
|
||||
// answer from "this much is used", and only words carry a difference in kind.
|
||||
when (val state = usage) {
|
||||
SessionUsage.NotMetered -> Unit
|
||||
// Both handled above, before the row exists at all.
|
||||
SessionUsage.NotMetered,
|
||||
SessionUsage.Waiting -> Unit
|
||||
is SessionUsage.Unavailable -> UsageNote("5-hour usage unknown -- ${state.why}")
|
||||
SessionUsage.Waiting -> UsageNote("5-hour usage: checking")
|
||||
is SessionUsage.Known -> {
|
||||
val window = state.windows.firstOrNull { it.kind == "session" }
|
||||
if (window == null) {
|
||||
@@ -242,17 +259,23 @@ private fun fiveHourLabel(window: UsageWindow, now: OffsetDateTime): String {
|
||||
}
|
||||
|
||||
/**
|
||||
* One machine's snapshot, out of every machine's.
|
||||
* One meter's snapshot, out of every machine's: [setup]'s row for [provider].
|
||||
*
|
||||
* Both halves are needed to pick it. A machine can hold more than one meter -- the Claude CLI's
|
||||
* account and, while a test has one set, an echo session's invented one -- and a snapshot is one
|
||||
* service on one machine.
|
||||
*
|
||||
* Every way of having *failed* to get numbers is [SessionUsage.Unavailable] with the reason in it:
|
||||
* a machine nobody logged into, one that could not be reached, a snapshot that came back empty.
|
||||
* None of them may look like zero, and none may look like [SessionUsage.NotMetered], which is the
|
||||
* machine having no quota rather than the question going unanswered.
|
||||
*/
|
||||
fun usageFor(snapshots: List<UsageSnapshot>, setup: String): SessionUsage {
|
||||
// No snapshot at all means the backend never asked, which it only does for a machine with
|
||||
// nothing metered on it. That is a different answer from having asked and failed.
|
||||
val mine = snapshots.firstOrNull { it.setup == setup } ?: return SessionUsage.NotMetered
|
||||
fun usageFor(snapshots: List<UsageSnapshot>, setup: String, provider: String): SessionUsage {
|
||||
// No snapshot at all means the backend never asked, which it only does where there is nothing
|
||||
// to ask about. That is a different answer from having asked and failed.
|
||||
val mine =
|
||||
snapshots.firstOrNull { it.setup == setup && it.provider == provider }
|
||||
?: return SessionUsage.NotMetered
|
||||
if (mine.state != "ok") {
|
||||
return SessionUsage.Unavailable(mine.detail ?: mine.state)
|
||||
}
|
||||
|
||||
Reference in new issue
Block a user