3 Commits
Author SHA1 Message Date
irisandClaude Opus 5 e4f0935f98 Keep the second auth test under a subscriber, so the tripwire is not flaky
`gates_every_route_and_never_logs_the_token` failed about one full-suite
run in ten, on the assertion that a rejection *was* logged. Its sibling
ends with an unauthenticated request of its own, made with no subscriber
on that thread -- and tracing caches a callsite's interest process-wide
the first time it is reached, so whichever test got there first decided
whether the warning would ever be recorded.

That is the rule already written at the top of "Things that have bitten",
applied to one member of a set: the combined gating+logging test exists
because of it, and the enrollment test added later did not get it.
Twenty runs clean since.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-04 17:58:11 -04:00
iris 3c0214ece8 Merge branch 'main' of git.arirex.me:iris/ai-app
# Conflicts:
#	AGENTS.md
#	PLAN.md
#	app/androidApp/src/main/kotlin/com/example/aiapp/SessionUsageBar.kt
#	app/androidApp/src/main/kotlin/com/example/aiapp/SpawnScreen.kt
#	server/src/config.rs
#	server/src/main.rs
#	server/src/routes.rs
#	server/src/session/echo.rs
#	server/src/session/llama.rs
#	server/src/session/transport.rs
#	server/src/ssh.rs
#	server/src/usage.rs
2026-09-04 17:56:50 -04:00
irisandClaude Opus 5 127b25e60a Meter a session by its provider, and let llama.cpp run over ssh
The rate-limit bar answered a question about an account, and picked the
answer by machine. One machine runs echo, the Claude CLI and a local
model side by side, so every echo session on it drew the CLI's five-hour
window: a quota that session cannot spend and could never run down. A
session now names its meter (`usageProvider`, from
`DriverKind::usage_provider`, which `usage::providers_for` reads too so
the two lists cannot disagree), and the phone matches on machine *and*
provider. Nothing meters echo or llama, and nothing at all is drawn --
including while the first fetch is out, since "checking" under a session
that turns out to meter nothing is a row the screen then withdraws.

Echo gets a meter it can be *told* about instead: `/usage 42`,
`/usage 95 20`, `/usage 42 never`, `/usage notloggedin`,
`/usage unreachable`, `/usage failed`, `/usage off`. Those states cost
real quota to arrange, which is why none of them had been looked at.

And llama.cpp runs wherever a setup says, which was the last of phase 5.
`Transport::reserve_port` is the second half of what a transport is --
"run this" plus "reach this port" -- returning the port the server binds
there and the port that reaches it here, and `Launch::reaching` puts the
`-L` tunnel on the connection that already carries the command. Three
things that came out of building it:

- A forwarded launch gets a pty and every other one keeps `-T`. Killing
  the ssh client ends a CLI by closing the stdin it reads; llama-server
  never reads its stdin, so the same kill left it running on the far
  machine with the model loaded -- one orphan per stopped session.
- The model is looked for on the machine that will serve it, at that
  machine's own models directory, so `GET /setups/{id}/models` is what
  the spawn screen offers rather than the backend's own downloads.
- The readiness poll watches the process, not only the port: a model
  that will not load exits in a second and would otherwise have been
  reported as "gave up after 300s". The failure carries the log's tail.

Exercised end to end against this VM over ssh to itself: spawn, load,
answer, outlive a backend restart, be adopted, answer again, and stop --
with both the ssh client and the far llama-server gone afterwards. The
local path, the Claude bar and the spawn screen checked on the emulator.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-04 17:45:32 -04:00
21 changed files with 1162 additions and 134 deletions

No files matched your search

+53 -14
View File
@@ -26,6 +26,19 @@ Module-by-module intent is in PLAN.md's "Backend layout".
- `server/` — the Rust backend (`ai-server`). `routes.rs`'s module doc
comment is the HTTP table and the surface's source of truth.
**A llama.cpp session runs on whatever machine its setup names** (built
2026-09-04, the last of phase 5): `Transport::reserve_port` returns the
port the server binds *there* and the port that reaches it *here*, and
`Launch::reaching` puts the `-L` tunnel on the connection already carrying
the command. Three things fell out of it and are easy to get wrong again —
a forwarded launch gets a pty (`-tt`) and every other one keeps `-T`,
because `llama-server` never reads the stdin whose closing ends a CLI and
the same kill left it loaded on the far machine; the model is looked for on
the machine that will serve it, so the spawn screen offers
`GET /setups/{id}/models` rather than the backend's own downloads; and the
readiness poll watches the process as well as the port, since a model that
will not load exits in a second and was being reported as "gave up after
300s". See PLAN.md's "Transport" and "llama-server management".
- `app/` — the Compose app, package `com.example.aiapp`, label "AI Sessions".
`AppRoot.kt` is the navigation `when`; `MainScreen.kt` the root's four tabs
(sessions, import, models, setups); `Api.kt`/`EventStream.kt` the REST + SSE
@@ -193,6 +206,13 @@ Each exists because something was invisible without it.
conversation somebody may still be in. **A transcript never goes in this
repository**: they hold whatever was said, read and written in that
session, and `~/repos` is shared with the host besides.
- **`/usage` in an echo session puts up an invented meter**, which is how the
rate-limit screens' states are reached without spending quota: `/usage 42`,
`/usage 95 20` (minutes left), `/usage 42 never` (the between-blocks window
with no reset time), `/usage 42 unreadable`, `/usage notloggedin`,
`/usage unreachable`, `/usage failed`, `/usage off`. The vocabulary is
`usage::Fixture`'s, since those are its states. With none set an echo
session meters nothing, which is the ordinary case and draws no bar.
- **A fake CLI exercises the process lifecycle without a token.** Point a
`claude_cli` provider's `command` at a two-line script — `#!/bin/sh` and
`cat > /dev/null` — and it behaves the way the lifecycle code cares about:
@@ -263,24 +283,32 @@ moment you use it — `ANDROID_SERIAL=$(emu serial) ./gradlew …`.
### Testing llama.cpp and ssh here
The prebuilt CPU llama.cpp lives outside the repo at
`~/.local/opt/llama.cpp` (the 15 MB `ubuntu-x64` release asset). It needs its
own directory on `LD_LIBRARY_PATH`, so start the server as
`LD_LIBRARY_PATH=~/.local/opt/llama.cpp ai-server …` and point a provider's
`command` at `~/.local/opt/llama.cpp/llama-server`. A 0.6B Q8_0 answers at
usable speed on this VM's 8 cores. **Do not test with a 2-bit quant**: the
**Both are set up here as of 2026-09-04** and need nothing typed. The
prebuilt CPU llama.cpp lives outside the repo at `~/.local/opt/llama.cpp`
(the 15 MB `ubuntu-x64` release asset) and is symlinked as
`/usr/local/bin/llama-server`, which is what makes **discovery find it over
ssh**: `~/.local/bin` is not on the PATH a non-interactive ssh session gets.
It resolves its own libraries through `$ORIGIN`, so no `LD_LIBRARY_PATH` is
needed. One model is downloaded — `unsloth/Qwen3-0.6B-GGUF/Qwen3-0.6B-Q8_0.gguf`,
639 MB under `~/.local/share/ai-app/models` — and answers at usable speed on
this VM's 8 cores. **Do not test with a 2-bit quant**: the
IQ2_XXS of that model produces fluent nonsense, which reads exactly like a
broken driver — `llama-cli` produces the same from the file directly, which
is how to tell the two apart in a hurry.
There is no second machine, so **ssh this VM to itself**: generate a
throwaway key, append the public half to `~/.ssh/authorized_keys`, and
configure a host of `bob@127.0.0.1` with `identityFile` pointing at it plus
`options: ["StrictHostKeyChecking=no", "UserKnownHostsFile=…"]` so it touches
nothing real. Point a provider's `command` at something harmless like
`/bin/echo` rather than at `claude`: the transport is what is under test, the
process exiting immediately is the signal, and it costs no tokens. **Take the
key back out afterwards.** The remote login shell here is **fish**; the
There is no second machine, so **ssh this VM to itself**. That is set up
too: the key is `~/.config/ai-app/ssh-self` (its public half is in
`~/.ssh/authorized_keys`, labelled removable), and the real config carries a
setup called **"this vm over ssh"** — `bob@127.0.0.1` with that
`identityFile` plus
`options: ["StrictHostKeyChecking=no", "UserKnownHostsFile=/tmp/ai-app-known-hosts"]`
so it touches nothing real — offering `claude-cli` and `llama-cpp`. It is the
whole rig for "does a remote llama session work", since the far machine is
this one and the model file is the same file. For a throwaway setup of your
own, point a provider's `command` at something harmless like `/bin/echo`
rather than at `claude`: the transport is what is under test, the process
exiting immediately is the signal, and it costs no tokens. The remote login
shell here is **fish**; the
remote script and `ssh.rs`'s POSIX quoting happen to mean the same thing in
both, but that is luck rather than design, and a shell that is neither is the
thing to suspect first if a remote spawn ever mangles an argument.
@@ -379,6 +407,17 @@ where it was instead of half-deleted.
swallowed the drag along with the tap, so a list could not be scrolled
while anything in it was busy.
- **A rate-limit bar belongs to a session's provider, not to its machine.**
One machine offers echo, the Claude CLI and a local model at once and only
the CLI spends anything, so a session says which meter reports on it
(`usageProvider`, from `DriverKind::usage_provider`, which
`usage::providers_for` reads too so the two lists cannot disagree) and the
phone matches a snapshot on machine *and* provider. Nothing meters a llama
or echo session, and the phone draws **nothing** for one — not a zero, and
not "unknown". Nothing while the first fetch is out either: "checking"
under a session that turns out to meter nothing is a row the screen then
has to withdraw.
## Things that have bitten
Project-specific only — a lesson that would bite any project on this machine
+56 -6
View File
@@ -169,10 +169,33 @@ deliberate and easy to undo by accident:
when the process restarts. That leaves the Claude driver as the odd one
out rather than this one — the CLI's memory is a cache in front of the same
transcript. Resolve any inconsistency in this direction.
- **A llama session on an ssh host is refused.** The model is reached over
HTTP and forwarding that port is not built, so refusing beats silently
talking to the wrong machine. A transport is "run this" plus "reach this
port", and only the first half exists.
- **A llama session runs on whatever machine its setup names** (2026-09-04,
the last of phase 5). A transport is "run this" plus "reach this port", and
the second half is `Transport::reserve_port` — the port the server binds
*there* and the port that reaches it *here*, the same number locally —
carried by `Launch::reaching` onto the connection that already runs the
command. `llama-server` binds loopback on the far machine, so nothing is
served to its network. The far port is a guess from a range below the
ephemeral one, because no portable way to ask a machine for a free port
avoids racing the bind anyway; a collision is not silent, since the server
fails to bind and the readiness poll reports what its log said.
- **The model file lives on the machine that serves it** (2026-09-04). Each
setup names its own models directory (`SshConfig::models_dir`, default
`~/.local/share/ai-app/models` expanded *there*), and a spawn resolves the
key on that machine — one round trip answering "at /abs/path" or "missing",
so a model that is not there is refused at the spawn rather than becoming a
server that never becomes ready. The spawn screen offers
`GET /setups/{id}/models`, that machine's list, rather than `GET /models`,
which is this backend's downloads. Downloading *to* another machine is
deliberately not built: a multi-gigabyte transfer with no progress
anywhere, and the file gets there however anything else on that machine
did.
- **The readiness poll watches the process, not only the port.** A model that
will not load, a port already taken, a flag an older build does not know:
all exit within a second and none will ever answer `/health`, so waiting
out the 300s timeout turned the server's own account of the problem into
"gave up". The failure carries the tail of `llama-server.log`, which on a
remote session is the only copy anybody reading the phone can see.
### Models (2026-08-28)
@@ -203,6 +226,13 @@ deliberate and easy to undo by accident:
A driver says what to run; something above it turns that into a process.
Otherwise transport knowledge sits inside a translator whose job is a wire
format, and every future driver has to remember to do the same.
- **A forwarded launch gets a pty and every other one does not** (measured
2026-09-04). Killing the ssh client ends a CLI because it closes the stdin
that CLI is reading; `llama-server` never reads its stdin, so the same kill
left it running on the far machine with the model loaded — one orphan per
stopped session. With `-tt` the far side takes SIGHUP when the connection
goes. Its log then arrives through a line discipline, which nothing parses.
`-T` stays everywhere else, where a pty would rewrite the JSONL.
- **`command -v` follows ssh's non-login PATH**, which is narrower than an
interactive shell's, so a binary somewhere unusual is invisible to
discovery. Point `command` at an absolute path.
@@ -499,6 +529,25 @@ rate-limited bucket). Poll at ≥180 s, only while a Claude session exists or
the usage screen is open, and cache the last answer. It is undocumented, so
`usage.rs` treats every field as optional and degrades rather than erroring.
**Per provider, not per machine (2026-09-04).** A machine is not what is
metered; the provider a session runs is. One machine offers echo, the Claude
CLI and a local model side by side, and only the second spends anything — so
pairing a session with a snapshot by machine alone drew the CLI's five-hour
window under every echo session on it, a quota that session cannot spend. A
session now names its meter (`usageProvider`, from
`DriverKind::usage_provider`, which `usage::providers_for` reads too, so the
two lists cannot disagree) and `GET /usage` is matched on machine *and*
provider. `None` is a session that meters nothing, and the phone draws
nothing at all for it — not a zero, and not "unknown".
`DriverKind::Echo` names a meter of its own that exists only when a test has
asked for one: `/usage` in an echo session sets an invented answer
(`usage::Fixture`), and with none set there is no snapshot and no bar. That
is what makes those screens' states reachable — a number near the top, a
window between blocks with no reset time, a machine nobody logged into, one
that could not be reached — without spending real quota to arrange them,
which is why none of them had ever been looked at.
**Per machine, not per backend (2026-08-29).** The credential store that
matters is the one on the machine the session runs on, because that is the
account being billed — and in the layout this aims at, `ai-server` is on the
@@ -800,8 +849,9 @@ Noticed and deliberately not fixed, so they are not re-found from scratch.
Phases 13 (the skeleton pipe, the full Claude driver, the usage screen) done
2026-08-24. Phase 4 (llama.cpp: model browsing, downloads, and `llama-server`
through its OpenAI-compatible endpoint) and phase 5 (ssh) done 2026-08-28.
The file explorer and the transcript cache followed in September. What is
through its OpenAI-compatible endpoint) and phase 5 (ssh) done 2026-08-28,
except for the remote `llama-server` and its port forward, which landed
2026-09-04. The file explorer and the transcript cache followed in September. What is
left is real-phone/WireGuard bring-up, which is operational rather than code.
Each phase ended runnable and verified against the real thing. The backend
@@ -178,6 +178,17 @@ data class SessionSummary(
* server because that is where a provider's kind is known.
*/
val maxImageEdge: Int?,
/**
* Which of `GET /usage`'s snapshots is about this session, and null where nothing meters it.
*
* The rate-limit bar answers a question about an *account*, and what decides which account --
* if any -- is the provider this session runs, not the machine it runs on. Pairing by machine
* alone drew the Claude CLI's five-hour window under every echo session on a machine that also
* has the CLI: a quota that session cannot spend and could never run down. Decided by the
* server for the same reason [maxImageEdge] is -- it is a fact about the provider's kind, and
* this app has only its name.
*/
val usageProvider: String?,
val status: String,
val lastActivity: Double,
)
@@ -198,6 +209,7 @@ private fun parseSession(session: JSONObject) =
contextTokens =
if (session.has("contextTokens")) session.getLong("contextTokens") else null,
maxImageEdge = session.optInt("maxImageEdge", 0).takeIf { it > 0 },
usageProvider = session.optString("usageProvider").ifEmpty { null },
status = session.getString("status"),
lastActivity = session.getDouble("lastActivity"),
)
@@ -375,6 +387,12 @@ data class SshDetails(
* Where files attached from here land on that machine; null for the session's own directory.
*/
val attachmentsDir: String? = null,
/**
* Where that machine keeps its GGUF models; null for the same place the backend keeps its own
* (`~/.local/share/ai-app/models`, read on that machine). A llama.cpp session serves the file
* from the machine it runs on, so this is where its models are looked for and listed.
*/
val modelsDir: String? = null,
)
private fun SshDetails.toJson() =
@@ -382,6 +400,7 @@ private fun SshDetails.toJson() =
if (port != null) put("port", port)
if (!identityFile.isNullOrBlank()) put("identityFile", identityFile)
if (!attachmentsDir.isNullOrBlank()) put("attachmentsDir", attachmentsDir)
if (!modelsDir.isNullOrBlank()) put("modelsDir", modelsDir)
}
/** What a machine turns out to have, without saving anything. */
@@ -1074,6 +1093,26 @@ private fun parseDownload(o: JSONObject) =
error = if (o.has("error")) o.getString("error") else null,
)
/**
* The models on one machine, which is the list a llama.cpp session there can choose from.
*
* Not [fetchModels], which is what the *backend* has downloaded. A session serves its model from
* the machine it runs on, so for a machine reached over ssh those are two different lists -- and
* offering the backend's would name files that are not there, turning a choice that cannot work
* into a session that fails when it tries to load one.
*/
fun fetchSetupModels(settings: ServerSettings, setupId: String): List<LocalModel> =
requestFromServer(settings, "/setups/${setupId.urlEncoded()}/models") { connection ->
JSONArray(connection.inputStream.bufferedReader().readText()).mapObjects { m ->
LocalModel(
key = m.getString("key"),
repo = m.getString("repo"),
file = m.getString("file"),
bytes = m.getLong("bytes"),
)
}
}
fun fetchModels(settings: ServerSettings): Models =
requestFromServer(settings, "/models") { connection ->
val body = JSONObject(connection.inputStream.bufferedReader().readText())
@@ -1149,7 +1149,7 @@ fun SessionScreen(
// One poll for the machines' limits, read by everything on this screen that reports them.
val usageFeed = rememberUsageFeed(settings)
val usage = usageFeed.forSetup(summary.setup)
val usage = usageFeed.forSession(summary)
RecordFrames()
var usageOpen by remember { mutableStateOf(false) }
var settingsOpen by remember { mutableStateOf(false) }
@@ -70,12 +70,21 @@ class UsageFeed(
/** Ask the backend again now. The dialog's refresh button; the poll does it on its own. */
val refresh: () -> Unit,
) {
/** What [setup]'s own limits came back as. See [usageFor] for why the states are these. */
fun forSetup(setup: String): SessionUsage =
when (val state = snapshots) {
/**
* What meters [session], and what that meter came back as. See [usageFor] for the states.
*
* A session rather than a machine, because a machine is not what is metered: one machine runs
* the Claude CLI and an echo session side by side, and only the first of them spends anything.
*/
fun forSession(session: SessionSummary): SessionUsage {
// Settled without asking anybody: a session nothing meters has nothing to check, and
// "checking" is what the fetch's own states would say about it for as long as one is out.
val provider = session.usageProvider ?: return SessionUsage.NotMetered
return when (val state = snapshots) {
is LoadState.Loading -> SessionUsage.Waiting
is LoadState.Error -> SessionUsage.Unavailable(state.message)
is LoadState.Loaded -> usageFor(state.value, setup)
is LoadState.Loaded -> usageFor(state.value, session.setup, provider)
}
}
}
@@ -158,9 +167,15 @@ fun SessionUsageBar(usage: SessionUsage, modifier: Modifier = Modifier) {
}
}
// Nothing at all for a machine that meters nothing: a row saying "unknown" there would report a
// problem about a setup somebody chose, on every screen, forever.
if (usage is SessionUsage.NotMetered) {
// Nothing at all for a session that meters nothing: a row saying "unknown" there would report
// a problem about a setup somebody chose, on every screen, forever.
//
// And nothing while the first fetch is out, which is a different silence. A request in flight
// is not a state to report -- and the session that meters nothing is exactly the one this
// cannot yet tell apart, so "5-hour usage: checking" appeared under an echo session for half a
// second and was then taken away. A row that has to be withdrawn is worse than one that
// arrives late.
if (usage is SessionUsage.NotMetered || usage is SessionUsage.Waiting) {
return
}
@@ -171,9 +186,10 @@ fun SessionUsageBar(usage: SessionUsage, modifier: Modifier = Modifier) {
// Words, not a colour and not an empty bar: every one of these is a different kind of
// answer from "this much is used", and only words carry a difference in kind.
when (val state = usage) {
SessionUsage.NotMetered -> Unit
// Both handled above, before the row exists at all.
SessionUsage.NotMetered,
SessionUsage.Waiting -> Unit
is SessionUsage.Unavailable -> UsageNote("5-hour usage unknown -- ${state.why}")
SessionUsage.Waiting -> UsageNote("5-hour usage: checking")
is SessionUsage.Known -> {
val window = state.windows.firstOrNull { it.kind == "session" }
if (window == null) {
@@ -234,16 +250,22 @@ private fun fiveHourLabel(window: UsageWindow, now: OffsetDateTime): String {
}
/**
* One machine's snapshot, out of every machine's.
* One meter's snapshot, out of every machine's: [setup]'s row for [provider].
*
* Both halves are needed to pick it. A machine can hold more than one meter -- the Claude CLI's
* account and, while a test has one set, an echo session's invented one -- and a snapshot is one
* service on one machine.
*
* Every way of having *failed* to get numbers is [SessionUsage.Unavailable] with the reason in it.
* None of them may look like zero, and none may look like [SessionUsage.NotMetered], which is the
* machine having no quota rather than the question going unanswered.
*/
fun usageFor(snapshots: List<UsageSnapshot>, setup: String): SessionUsage {
// No snapshot at all means the backend never asked, which it only does for a machine with
// nothing metered on it. That is a different answer from having asked and failed.
val mine = snapshots.firstOrNull { it.setup == setup } ?: return SessionUsage.NotMetered
fun usageFor(snapshots: List<UsageSnapshot>, setup: String, provider: String): SessionUsage {
// No snapshot at all means the backend never asked, which it only does where there is nothing
// to ask about. That is a different answer from having asked and failed.
val mine =
snapshots.firstOrNull { it.setup == setup && it.provider == provider }
?: return SessionUsage.NotMetered
if (mine.state != "ok") {
return SessionUsage.Unavailable(mine.detail ?: mine.state)
}
@@ -236,6 +236,7 @@ private fun AddSetupDialog(
var address by remember { mutableStateOf("") }
var identity by remember { mutableStateOf("") }
var attachmentsDir by remember { mutableStateOf("") }
var modelsDir by remember { mutableStateOf("") }
var tested by remember { mutableStateOf<String?>(null) }
var testing by remember { mutableStateOf(false) }
@@ -250,6 +251,7 @@ private fun AddSetupDialog(
port = typedPort,
identityFile = identity.trim().ifEmpty { null },
attachmentsDir = attachmentsDir.trim().ifEmpty { null },
modelsDir = modelsDir.trim().ifEmpty { null },
)
}
@@ -293,6 +295,14 @@ private fun AddSetupDialog(
label = { Text("Folder for attached files (optional)") },
singleLine = true,
)
// Where that machine's GGUFs are, for a llama.cpp session on it. Blank means
// the same place this backend keeps its own downloads, read on that machine.
OutlinedTextField(
value = modelsDir,
onValueChange = { modelsDir = it },
label = { Text("Folder for models (optional)") },
singleLine = true,
)
tested?.let {
Spacer(Modifier.height(8.dp))
Text(it, style = MaterialTheme.typography.bodySmall)
@@ -65,8 +65,10 @@ fun SpawnScreen(
// Only the spawn's own failure. The fetch's lives in `options`: this one leaves a filled-in
// form worth keeping, and that one leaves nothing to fill in.
var spawnError by remember { mutableStateOf<String?>(null) }
// Downloaded models, for a llama provider to choose between. Kept separate from the setups: a
// Claude session needs none, so failing to list them must not stop the screen rendering.
// The models on the *chosen machine*, for a llama provider to choose between. Kept separate
// from the setups: a Claude session needs none, so failing to list them must not stop the
// screen rendering. Refetched when the machine changes, because a model is a file on one
// machine -- see [fetchSetupModels].
var models by remember { mutableStateOf<List<LocalModel>>(emptyList()) }
var modelKey by remember { mutableStateOf<String?>(null) }
var contextSize by remember { mutableStateOf("") }
@@ -83,9 +85,6 @@ fun SpawnScreen(
} catch (e: ApiException) {
LoadState.failed(e)
}
models =
runCatching { withContext(Dispatchers.IO) { fetchModels(settings).local } }
.getOrDefault(emptyList())
}
Column(Modifier.fillMaxSize().verticalScroll(rememberScrollState()).padding(16.dp)) {
@@ -115,6 +114,17 @@ fun SpawnScreen(
is LoadState.Loaded -> state.value
}
val setup = setups.firstOrNull { it.name == setupName }
// Whichever machine is chosen now, asked again when that changes. The old machine's list
// is dropped first rather than left on screen: a file name from another machine looks
// exactly like one from this one.
LaunchedEffect(setup?.id) {
models = emptyList()
modelKey = null
val id = setup?.id ?: return@LaunchedEffect
models =
runCatching { withContext(Dispatchers.IO) { fetchSetupModels(settings, id) } }
.getOrDefault(emptyList())
}
val current = setup?.providers?.firstOrNull { it.name == providerName }
// Only the Claude CLI has models, a working directory and permission modes; keying the
// extra fields on the kind rather than the provider name keeps a second Claude provider
@@ -175,12 +185,13 @@ fun SpawnScreen(
)
if (isLlama) {
// A llama session names one of the models this backend has downloaded, so the choice is
// that list rather than free text -- a name that is not on disk is a session that
// cannot start.
// A llama session names one of the models on the machine it will run on, so the
// choice is that list rather than free text -- a name that is not on that machine's
// disk is a session that cannot start.
if (models.isEmpty()) {
Text(
"No models downloaded yet. Get one from the Models screen first.",
"No models on ${setup?.name ?: "this machine"}. The Models screen downloads " +
"to the backend; another machine needs the file put there itself.",
style = MaterialTheme.typography.bodyMedium,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
+1
View File
@@ -35,6 +35,7 @@ dependencies = [
"sha2",
"tempfile",
"thiserror",
"time",
"tokio",
"tokio-stream",
"tower",
+6
View File
@@ -53,6 +53,12 @@ ureq = { version = "3", features = ["json"] }
# both in the graph rustls refuses to auto-select one.
rustls = "0.23"
libc = "0.2.189"
# One ISO-8601 timestamp: the reset time on the invented rate-limit window
# an echo session's `/usage` puts up. Already in the tree behind the
# certificate machinery, so this is a direct name for what is compiled
# anyway rather than a new crate -- and the alternative was hand-rolling a
# civil-from-days conversion to print one line.
time = { version = "0.3", features = ["formatting"] }
[dev-dependencies]
tempfile = "3"
+14
View File
@@ -207,6 +207,20 @@ mod tests {
/// like any other.
#[tokio::test]
async fn a_spooled_enrollment_is_adopted_on_first_use() {
// Under a subscriber, like every other exercise of this middleware.
// `tracing` caches a callsite's interest process-wide the first time it
// is reached, so the refusal at the end of this test -- reached with no
// subscriber on this thread -- could cache the rejection warning as
// never-enabled and make the tripwire above see an empty log. That
// failed about one full-suite run in ten, in the test that exists to
// notice a credential leak, which is the worst place for a flake.
let _guard = tracing::subscriber::set_default(
tracing_subscriber::fmt()
.with_max_level(tracing::Level::TRACE)
.with_writer(std::io::sink)
.finish(),
);
let dir = tempfile::tempdir().expect("tempdir");
let manager = manager_with_token(dir.path(), "first");
let spooled = generate_token();
+38
View File
@@ -94,6 +94,19 @@ pub struct SshConfig {
/// Extra `-o` settings, each written as `Key=value`.
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub options: Vec<String>,
/// Where this machine keeps the GGUF models it can serve, absent for
/// the same default this backend uses (`~/.local/share/ai-app/models`
/// -- `$XDG_DATA_HOME` is not read on the far side, since it is this
/// machine's environment that would answer). A `~` prefix is the
/// remote home.
///
/// Here rather than on the provider because it is a fact about the
/// machine, and because a machine reached over ssh is where the model
/// has to be: a llama.cpp session serves the file from the machine
/// that runs `llama-server`, and this backend's own downloads are on
/// whichever machine that is only when they are the same one.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub models_dir: Option<PathBuf>,
/// Where a file attached from the phone is put on this machine so the
/// session can read it. Absent means the session's own working directory,
/// or the login home for a session that has none. A `~` prefix is the
@@ -145,6 +158,30 @@ impl DriverKind {
}
}
/// Which paid service meters a session of this kind, and `None` for one
/// that costs nothing.
///
/// What decides which account -- if any -- a rate-limit bar is about is the
/// provider a session runs, not the machine it runs on: an echo session on
/// a machine that also has the Claude CLI was drawn with that CLI's
/// five-hour window, a quota it cannot spend.
///
/// Echo names a meter of its own that exists only when a test has asked for
/// one (`/usage` in `session::echo`), which is how the bar's states are
/// reached without an account. With none set there is no snapshot, and the
/// phone draws nothing.
///
/// The string is a [`crate::usage::UsageProvider::name`], and it is what
/// pairs a session with one of `GET /usage`'s snapshots -- so
/// `usage::providers_for` reads this rather than matching on kinds again.
pub fn usage_provider(self) -> Option<&'static str> {
match self {
Self::ClaudeCli => Some(crate::usage::CLAUDE),
Self::Echo => Some(crate::usage::ECHO),
Self::LlamaCpp => None,
}
}
/// Whether the conversation exists outside this app, so that deleting the
/// session here does not end it.
///
@@ -385,6 +422,7 @@ mod tests {
port: Some(2222),
identity_file: None,
options: Vec::new(),
models_dir: None,
attachments_dir: None,
}),
providers: vec![ProviderConfig {
+3 -1
View File
@@ -264,7 +264,9 @@ async fn main() -> Result<()> {
// No providers listed here any more: which machines can be asked, and about
// what, comes from the setups at the moment the screen is opened -- so a
// machine added from the phone reports its limits without a restart.
let monitor = Arc::new(usage::UsageMonitor::new());
// The fixture is the manager's, because that is where the `/usage` command
// that sets it is typed; the monitor is what serves it.
let monitor = Arc::new(usage::UsageMonitor::new(manager.usage_fixture()));
// The bearer-token middleware wraps the entire router -- routes and fallback
// alike -- here and only here, so a new route can't forget auth.
+79
View File
@@ -29,6 +29,8 @@ use serde::Serialize;
use wg_app_link::private;
use crate::session::transport::{Launch, Transport};
/// Identifies this client to HuggingFace. They ask for one, and a request
/// without it is more likely to be rate-limited.
const USER_AGENT: &str = concat!("ai-server/", env!("CARGO_PKG_VERSION"));
@@ -521,6 +523,83 @@ fn collect(root: &Path, dir: &Path, found: &mut Vec<LocalModel>) {
}
}
/// Where a machine reached over ssh keeps its models, when its setup does
/// not say.
///
/// The same place this backend puts its own downloads, written out rather
/// than derived: `$XDG_DATA_HOME` here describes *this* machine's
/// environment, and the far machine's is the far machine's business. A
/// setup whose models are elsewhere says so (`SshConfig::models_dir`).
const FAR_MODELS_DIR: &str = "~/.local/share/ai-app/models";
/// Which directory holds the models on the machine `transport` reaches.
///
/// One answer, because two things ask: the list a spawn screen offers,
/// and the path a session hands `llama-server`. A machine that listed one
/// directory and served from another would offer models that then failed
/// to load, which reads as the model being broken.
pub fn dir_on(transport: &Transport, local: &Path) -> String {
match transport {
Transport::Here => local.to_string_lossy().into_owned(),
Transport::Ssh { ssh, .. } => ssh
.models_dir
.as_ref()
.map_or(FAR_MODELS_DIR.to_string(), |dir| {
dir.to_string_lossy().into_owned()
}),
}
}
/// Every GGUF on the machine a setup names, which is the machine that
/// would have to serve it.
///
/// The local half of this is [`ModelStore::list`], reading the same shape
/// off this machine's disk; a caller picks by transport, since a setup
/// with no ssh *is* this machine and asking a shell about it would be a
/// slower way to the same answer. What must not happen is offering this
/// backend's downloads for a session on another machine: the file has to
/// be where `llama-server` runs, and a list that says otherwise is a
/// claim about the wrong filesystem.
///
/// `dir` is that machine's models directory, `~` included -- expanded on
/// the far side, which is the only place that knows what it is. A
/// directory that is not there is an empty list rather than a failure: a
/// machine that has never had a model put on it is an ordinary state, and
/// the same one as a machine whose directory exists and is empty.
pub async fn on_machine(transport: &Transport, dir: &str) -> Result<Vec<LocalModel>> {
let script = "p=$1; case $p in \"~\") p=$HOME;; \"~/\"*) p=$HOME/${p#\"~/\"};; esac; \
[ -d \"$p\" ] || exit 0; \
find \"$p\" -type f -name '*.gguf' -printf '%s\\t%P\\0'";
let launch = Launch::new(
"sh",
vec![
"-c".to_string(),
script.to_string(),
"sh".to_string(),
dir.to_string(),
],
None,
);
let out = transport.capture(&launch).await?;
let mut found: Vec<LocalModel> = out
.split('\0')
.filter(|record| !record.is_empty())
// Two fields, and the name last, so a `\t` in a filename survives.
.filter_map(|record| record.split_once('\t'))
.filter_map(|(bytes, key)| {
let (repo, file) = key.rsplit_once('/')?;
Some(LocalModel {
key: key.to_string(),
repo: repo.to_string(),
file: file.to_string(),
bytes: bytes.trim().parse().unwrap_or(0),
})
})
.collect();
found.sort_by(|a, b| a.key.cmp(&b.key));
Ok(found)
}
/// A model repository on HuggingFace, as the browse screen shows it.
#[derive(Debug, Clone, Serialize)]
#[serde(rename_all = "camelCase")]
+35
View File
@@ -7,6 +7,7 @@
//! POST /setups add {name, ssh?} -- providers are discovered
//! POST /setups/probe dry run {ssh?}: what would be found there
//! GET /setups/{id} one machine, for refetching after a change
//! GET /setups/{id}/models GGUFs on that machine, for a llama session
//! GET /setups/{id}/dir?path=P entries of directory P, and P resolved
//! GET /setups/{id}/file?path=P content of file P, or why not
//! PUT /setups/{id}/file {path, content, ifSha256} -> new size/mtime/sha256
@@ -109,6 +110,8 @@ pub fn router(manager: Arc<SessionManager>) -> Router {
"/setups/{id}",
get(read_setup).put(update_setup).delete(delete_setup),
)
// The models on the machine a setup names, for a llama session there.
.route("/setups/{id}/models", get(setup_models))
// The filesystem of the machine a setup names. Under the setup
// rather than under a session because a filesystem is a property of
// a machine; a session only says where to start looking.
@@ -291,6 +294,9 @@ struct SshRequest {
/// Where attached files land on that machine; see `SshConfig`.
#[serde(default)]
attachments_dir: Option<String>,
/// Where that machine keeps its GGUF models; see `SshConfig`.
#[serde(default)]
models_dir: Option<String>,
}
impl SshRequest {
@@ -320,6 +326,14 @@ impl SshRequest {
.map(str::trim)
.filter(|dir| !dir.is_empty())
.map(std::path::PathBuf::from),
// The same rule, and for the same reason: this directory is
// on the other machine, so a `~` in it is that machine's home.
models_dir: self
.models_dir
.as_deref()
.map(str::trim)
.filter(|dir| !dir.is_empty())
.map(std::path::PathBuf::from),
})
}
}
@@ -495,6 +509,27 @@ struct PathQuery {
path: String,
}
/// The models **that machine** has, which is the list a llama.cpp session
/// on it can choose from.
///
/// Not `GET /models`, which is this backend's own downloads: those are on
/// the machine a session runs on only when they are the same machine. A
/// spawn screen offering this backend's list for a remote setup would be
/// naming files that are not there, and the session would fail at the
/// point of loading rather than at the point of choosing.
async fn setup_models(
State(manager): State<Arc<SessionManager>>,
UrlPath(id): UrlPath<String>,
) -> Result<axum::Json<Vec<crate::models::LocalModel>>, ApiError> {
let setup = setup_by_id(&manager, &id)?;
let transport = crate::session::transport::Transport::for_setup(&setup);
let dir = crate::models::dir_on(&transport, manager.models_dir());
crate::models::on_machine(&transport, &dir)
.await
.map(axum::Json)
.map_err(from_machine)
}
/// What is in a directory, and what that directory resolved to.
async fn list_dir(
State(manager): State<Arc<SessionManager>>,
+35 -1
View File
@@ -26,6 +26,13 @@
//! - `/error [text]` -- a failure, which is otherwise awkward to cause.
//! - `/peer [text]`, `/peer-turn` -- a message from another agent, in the
//! in-place and the live shapes.
//! - `/usage [what]` -- an invented rate-limit answer, or `/usage off` to take
//! it away. An echo session meters nothing, so it draws no usage bar until
//! this is set; what it exists for is the states that bar can be in, which
//! otherwise cost real quota to reach. `/usage 42`, `/usage 95 20`,
//! `/usage 42 never`, `/usage notloggedin`, `/usage unreachable`,
//! `/usage failed`. The vocabulary is `usage::Fixture`'s, where the states
//! live.
//! - `/compact` -- a compaction, start to finish.
//! - `/stream N` -- one long answer in N small pieces, 50ms apart: the shape a
//! real model's reply arrives in, and the one where the row a reader is
@@ -85,6 +92,10 @@ pub struct EchoDriver {
/// because `/ask` puts up to four on one tool call, and the turn resumes
/// when the last is answered rather than the first.
pending_questions: Mutex<Vec<PendingQuestion>>,
/// The invented rate-limit answer `/usage` sets, shared with the usage
/// monitor that serves it. An echo session meters nothing, so this is unset
/// until a test asks for something -- see [`crate::usage::Fixture`].
usage: crate::usage::Fixture,
/// A pretend context, so the status row has something that behaves the way
/// a real one does: it grows with each turn, drops to what the compaction
/// says it recovered, and a clear leaves it unmeasured. What is real is
@@ -311,6 +322,28 @@ impl EchoDriver {
return;
}
// Answered here rather than in the turn below, because it is not a
// turn: nothing is generated, and what is being exercised is the
// *other* screens -- the bar under the header, the button beside it and
// the dialog it opens, which read the usage route, not this transcript.
if let Some(rest) = text.strip_prefix("/usage") {
if announce {
self.emit(Event::MessageTaken {
id: None,
text: text.clone(),
attachments,
});
}
let said = self.usage.command(rest);
self.emit(Event::AssistantText {
delta: format!("{said}\n"),
});
self.emit(Event::Status {
state: SessionStatus::Idle,
});
return;
}
// The same word the real CLI takes, so a phone drives both the same way.
// `Driver::compact` is what the manager's route calls; this is the typed
// path onto it.
@@ -606,7 +639,7 @@ impl EchoDriver {
});
}
pub fn new(sink: EventSink, session_dir: PathBuf) -> Self {
pub fn new(sink: EventSink, session_dir: PathBuf, usage: crate::usage::Fixture) -> Self {
let driver = Self {
sink,
pending_questions: Mutex::new(Vec::new()),
@@ -614,6 +647,7 @@ impl EchoDriver {
busy: Arc::new(AtomicBool::new(false)),
queued: Arc::new(Mutex::new(Vec::new())),
session_dir,
usage,
};
driver.emit(Event::Status {
state: SessionStatus::Idle,
+137 -28
View File
@@ -5,9 +5,19 @@
//!
//! **It is spawned but not spoken to over stdio.** The process is started
//! through the same [`Transport`] as any other and then reached over HTTP on a
//! loopback port. A remote llama-server would need its port forwarded as well
//! as its command wrapped, which is not built, so a session on an ssh host is
//! refused rather than silently talking to the wrong machine.
//! loopback port. That is the second half of what a transport is -- "run this"
//! plus "reach this port" -- and it is what lets a session run on another
//! machine: [`Transport::reserve_port`] hands back a port the server binds
//! *there* and one that reaches it *here*, and the ssh connection carrying the
//! command carries the tunnel between them. The far `llama-server` binds
//! loopback only, so a model is never served to that machine's network.
//!
//! **The model file is the far machine's, not this one's.** A remote setup
//! names its own models directory (`SshConfig::models_dir`, defaulting to where
//! this backend keeps its downloads), and the file is looked for *there* -- so
//! a session naming a model that machine does not have says so, instead of
//! starting a server that will never load one. Downloading to another machine
//! is not built; the model gets there however anything else does.
//!
//! **The server is stateless between requests**, so the whole conversation goes
//! with every one. It is rebuilt from the session's transcript rather than kept
@@ -76,16 +86,10 @@ impl LlamaDriver {
session_dir: &Path,
sink: EventSink,
) -> Result<Self> {
if !matches!(transport, Transport::Here) {
bail!(
"llama.cpp sessions can only run on this machine for now: the model is served \
over HTTP, and forwarding that port to another host isn't built yet."
);
}
let model = meta.model.as_deref().context(
"a llama.cpp session needs a model -- one of the downloaded ones, by its key",
)?;
let path = model_path(models_dir, model)?;
let path = model_on(transport, models_dir, model)?;
// Already loaded and still running: keep talking to it. The health poll
// below confirms it is really answering, so adopting a pid whose server
@@ -111,14 +115,21 @@ impl LlamaDriver {
));
}
let port = free_port().context("finding a port for llama-server")?;
// Where it listens on its own machine, and where that is reached
// from here -- the same number when that machine is this one.
let forward = transport
.reserve_port()
.context("finding a port for llama-server")?;
let mut args: Vec<String> = vec![
"-m".into(),
path.to_string_lossy().into_owned(),
path.clone(),
// Loopback there, whichever machine there is: what reaches it
// from outside that machine is the ssh tunnel and nothing
// else.
"--host".into(),
"127.0.0.1".into(),
"--port".into(),
port.to_string(),
forward.there.to_string(),
];
// Settings that belong to the server because they decide how the model
// is loaded; the sampling ones ride on each request instead, so changing
@@ -135,7 +146,7 @@ impl LlamaDriver {
}
let program = provider.command.as_deref().unwrap_or("llama-server");
let launch = Launch::new(program, args, meta.cwd.as_deref());
let launch = Launch::new(program, args, meta.cwd.as_deref()).reaching(forward);
// Its output goes to files, not pipes. Not only so the process can
// outlive this server: nothing ever read those pipes, so a chatty
// llama-server filled the 64 KB buffer and blocked mid-load with no sign
@@ -152,8 +163,12 @@ impl LlamaDriver {
.id()
.context("llama-server exited before it could be recorded")?;
tracing::info!(
"session {} running {program} for {model} on 127.0.0.1:{port} as pid {pid}",
meta.id
"session {} running {program} for {model} {} on 127.0.0.1:{} there, \
reached at 127.0.0.1:{} here, as pid {pid}",
meta.id,
transport.describe(),
forward.there,
forward.here,
);
// Reaped so it does not become a zombie while this server is still its
// parent; the health poll and the record are what say whether the
@@ -163,12 +178,18 @@ impl LlamaDriver {
let _ = child.wait().await;
});
let record = process::Record::of(pid, process::Detail::Http { port })
// The *near* port, because that is the one anything reaching this
// server has to dial -- including a later run of this backend,
// which adopts the record without knowing which machine the server
// is on. For a remote session the recorded pid is the ssh
// client's, which is the process this machine owns and which holds
// the tunnel open for exactly as long as the far server lives.
let record = process::Record::of(pid, process::Detail::Http { port: forward.here })
.context("llama-server was gone before its start time could be read")?;
process::write(session_dir, &record);
Ok(Self::attached(
format!("http://127.0.0.1:{port}"),
format!("http://127.0.0.1:{}", forward.here),
meta,
model,
transcript,
@@ -202,7 +223,7 @@ impl LlamaDriver {
let endpoint = endpoint.clone();
let model = model.to_string();
let session_dir = session_dir.to_path_buf();
std::thread::spawn(move || match wait_until_ready(&endpoint) {
std::thread::spawn(move || match wait_until_ready(&endpoint, &session_dir) {
Ok(()) => {
tracing::info!("{model} loaded and answering at {endpoint}");
let _ = sink.send(Event::Status {
@@ -499,17 +520,70 @@ fn model_path(models_dir: &Path, key: &str) -> Result<PathBuf> {
Ok(path)
}
/// An unused loopback port, by asking the OS for one and letting it go. Racy in
/// principle, but nothing on this machine is hunting for ports, and the
/// alternative -- parsing the port back out of the server's log -- couples us to
/// its output format for no real gain.
fn free_port() -> Result<u16> {
let listener = std::net::TcpListener::bind("127.0.0.1:0")?;
Ok(listener.local_addr()?.port())
/// The model file's path **on the machine that will serve it**, confirmed to be
/// there.
///
/// One function rather than a local check and hope for the other case: the same
/// question has to be asked of two filesystems. The remote answer is measured
/// for the reason the local one is -- a missing file otherwise becomes a
/// `llama-server` that starts, fails to load, and reports as a session that
/// never became ready, which reads as the machine being slow.
///
/// One blocking round trip on a remote spawn, which is what the spawn is
/// already paying to start ssh. The alternative is a path built here from a `~`
/// this machine cannot expand.
fn model_on(transport: &Transport, models_dir: &Path, key: &str) -> Result<String> {
let Transport::Ssh { name, .. } = transport else {
return Ok(model_path(models_dir, key)?.to_string_lossy().into_owned());
};
// The same directory the spawn screen listed for this machine, and one
// function for the same reason: a list from one place and a load from
// another is a model that appears and then fails.
let dir = crate::models::dir_on(transport, models_dir);
// Checked here rather than in the script: `..` in a key would walk out of
// the models directory on a machine this server can start processes on,
// and the phone is where the key comes from.
for part in key.split('/') {
if part.is_empty() || part == "." || part == ".." {
bail!("\"{key}\" is not a model key this can resolve");
}
}
let path = format!("{}/{key}", dir.trim_end_matches('/'));
// `$HOME` on the far side, which is the only machine that knows what it is,
// and the resolved path printed back so the launch hands `llama-server`
// something absolute. "Not there" is answered rather than failed, because a
// machine that could not be asked at all has to say so in its own words --
// it would otherwise arrive as this same sentence about a missing model.
let script = "p=$1; case $p in \"~\") p=$HOME;; \"~/\"*) p=$HOME/${p#\"~/\"};; esac; \
[ -f \"$p\" ] && printf 'at\\t%s\\n' \"$p\" || printf 'missing\\n'"
.to_string();
let launch = Launch::new(
"sh",
vec!["-c".to_string(), script, "sh".to_string(), path.clone()],
None,
);
let answer = transport
.capture_blocking(&launch)
.with_context(|| format!("couldn't ask {name} where its models are"))?;
match answer.trim().split_once('\t') {
Some(("at", resolved)) => Ok(resolved.to_string()),
_ => bail!(
"{name} has no model at {path}. A llama.cpp session serves the file from the \
machine it runs on, so the model has to be on {name} -- what this backend has \
downloaded is somewhere else."
),
}
}
/// Polls until the server says it is ready, or gives up.
fn wait_until_ready(endpoint: &str) -> Result<()> {
///
/// Watches the process as well as the port, because the two failures need
/// different words and one of them is common: a model that will not load,
/// a port already taken on the far machine, a `llama-server` too old for
/// a flag. All of those exit within a second and none of them will ever
/// answer `/health`, so waiting out the timeout turns a server that said
/// exactly what was wrong into "gave up after 300s".
fn wait_until_ready(endpoint: &str, session_dir: &Path) -> Result<()> {
let deadline = std::time::Instant::now() + READY_TIMEOUT;
let url = format!("{endpoint}/health");
loop {
@@ -518,13 +592,48 @@ fn wait_until_ready(endpoint: &str) -> Result<()> {
{
return Ok(());
}
// `None` is the session having been stopped or deleted while this
// waited, which is nobody's fault and still not worth waiting on.
match process::recorded(session_dir) {
Some((_, process::Liveness::Alive | process::Liveness::Unknown)) => {}
Some((_, process::Liveness::Dead)) | None => {
bail!("it exited before it answered.{}", log_tail(session_dir));
}
}
if std::time::Instant::now() > deadline {
bail!("gave up after {}s", READY_TIMEOUT.as_secs());
bail!(
"gave up after {}s.{}",
READY_TIMEOUT.as_secs(),
log_tail(session_dir)
);
}
std::thread::sleep(std::time::Duration::from_millis(250));
}
}
/// The end of `llama-server`'s own log, for a failure message.
///
/// Its account of what went wrong is the useful half -- "failed to load
/// model", "bind: Address already in use" -- and on a remote session it
/// is the only half, since nobody reading the phone can open a file on
/// that machine. Bounded, because this ends up in an event a phone draws.
fn log_tail(session_dir: &Path) -> String {
let Ok(text) = std::fs::read_to_string(session_dir.join(SERVER_LOG)) else {
return String::new();
};
let tail: Vec<&str> = text.lines().rev().take(LOG_TAIL_LINES).collect();
if tail.is_empty() {
return String::new();
}
format!(
" It last said: {}",
tail.into_iter().rev().collect::<Vec<_>>().join(" / ")
)
}
/// How much of that log to carry into a message somebody reads on a phone.
const LOG_TAIL_LINES: usize = 6;
/// One streamed completion: posts the conversation, emits each delta as it
/// arrives. Emits rather than returns, because the transcript those events land
/// in is what the next turn reads back.
+86 -24
View File
@@ -135,6 +135,17 @@ pub struct SessionInfo {
/// that happens to be big" are different answers.
#[serde(skip_serializing_if = "Option::is_none")]
pub max_image_edge: Option<u32>,
/// Which of `GET /usage`'s snapshots reports on this session, and
/// absent where nothing meters it -- see
/// [`DriverKind::usage_provider`].
///
/// Reported for the same reason `keeps_own_transcript` is: it is a
/// fact about the provider's *kind*, and the phone has only its name.
/// Pairing by machine alone was the bug it exists to fix -- one
/// machine runs echo and the Claude CLI, so every echo session drew
/// the CLI's five-hour window as if it were its own.
#[serde(skip_serializing_if = "Option::is_none")]
pub usage_provider: Option<&'static str>,
/// Whether this session announces itself -- reported for the same
/// reason `permission_mode` is: a switch that guesses its own position
/// is how you turn something off while believing you are reading it.
@@ -438,6 +449,7 @@ impl LiveSession {
context_tokens: *self.shared.context_tokens.lock().unwrap(),
notify: *self.shared.notify.lock().unwrap(),
max_image_edge: kind.and_then(DriverKind::max_image_edge),
usage_provider: kind.and_then(DriverKind::usage_provider),
imported,
keeps_own_transcript: kind.is_some_and(DriverKind::keeps_own_transcript),
cwd: cwd.map(Path::to_path_buf),
@@ -470,6 +482,11 @@ pub struct SessionManager {
/// What to mark sessions spawned here as -- see
/// [`SessionManager::marking_new_sessions_throwaway`].
spawn_throwaway: bool,
/// The invented rate-limit answer an echo session's `/usage` sets,
/// shared with the usage monitor that serves it. Held here because
/// every echo driver this manager builds is handed a clone -- see
/// [`SessionManager::reporting_usage_fixture`].
usage_fixture: crate::usage::Fixture,
inner: RwLock<Inner>,
}
@@ -485,6 +502,11 @@ impl SessionManager {
wg_app_link::private::create_dir(&data_dir)?;
let (notifications, _) = broadcast::channel(NOTIFICATION_BUFFER);
// Made here rather than passed in, and handed *out* to the usage
// monitor by whoever wires the two together: every echo driver
// this manager builds gets a clone, including the ones built
// below, so it has to exist before the first session does.
let usage_fixture = crate::usage::Fixture::new();
let mut live = HashMap::new();
for meta in &config.sessions {
// One unlaunchable session -- a corrupt transcript, an
@@ -495,8 +517,11 @@ impl SessionManager {
meta.clone(),
&setup,
&provider,
&data_dir,
&models_dir,
Env {
data_dir: &data_dir,
models_dir: &models_dir,
usage: &usage_fixture,
},
notifications.clone(),
// Nothing is started here; see `Launching`.
Launching::Restart,
@@ -517,11 +542,41 @@ impl SessionManager {
notifications,
pending: Arc::new(pending::Registry::default()),
spawn_throwaway: false,
usage_fixture,
inner: RwLock::new(Inner { config, live }),
};
Ok(manager)
}
/// Where this backend's own model downloads live. The machine a
/// session runs on may keep its elsewhere -- see `models::dir_on`.
pub fn models_dir(&self) -> &Path {
&self.models_dir
}
/// What this manager lends a session it launches. Borrowed from the
/// manager rather than cloned, so there is one answer to where things
/// are kept.
fn env(&self) -> Env<'_> {
Env {
data_dir: &self.data_dir,
models_dir: &self.models_dir,
usage: &self.usage_fixture,
}
}
/// The invented rate-limit answer this manager's echo sessions set
/// with `/usage`, for the usage monitor to serve.
///
/// Handed out rather than taken in because the drivers built inside
/// the constructor need it, and because the direction is the one the
/// layering allows: `usage` sits below the session layer, so a
/// session can hold one of its types while it holds nothing of a
/// session's.
pub fn usage_fixture(&self) -> crate::usage::Fixture {
self.usage_fixture.clone()
}
/// Marks every session spawned from here on as one whose process is
/// stopped when this server exits. Set from `--throwaway-sessions`,
/// which a debug build defaults to on. It decides only what a *new*
@@ -861,6 +916,8 @@ impl SessionManager {
context_tokens: None,
max_image_edge: kind_of(&inner.config, &meta.setup, &meta.provider)
.and_then(DriverKind::max_image_edge),
usage_provider: kind_of(&inner.config, &meta.setup, &meta.provider)
.and_then(DriverKind::usage_provider),
notify: meta.notify,
imported: import::read_cursor(&self.data_dir.join(&meta.id)).is_some(),
keeps_own_transcript: keeps_own_transcript(
@@ -974,8 +1031,7 @@ impl SessionManager {
meta.clone(),
&setup,
&provider,
&self.data_dir,
&self.models_dir,
self.env(),
self.notifications.clone(),
Launching::Asked(seed),
)?;
@@ -1372,7 +1428,7 @@ impl SessionManager {
&meta,
&setup,
&provider,
&self.models_dir,
self.env(),
session.dir(),
session.transcript_path(),
&session.sink,
@@ -1385,8 +1441,7 @@ impl SessionManager {
meta,
&setup,
&provider,
&self.data_dir,
&self.models_dir,
self.env(),
self.notifications.clone(),
Launching::Asked(None),
)?;
@@ -1760,6 +1815,19 @@ enum Launching {
Restart,
}
/// What the server around a session lends it: where sessions and models
/// are kept, and the usage fixture an echo session's `/usage` sets.
///
/// One parameter rather than three because they travel together through
/// every launch path and none of them is a fact about the session --
/// they are this server's belongings, handed down.
#[derive(Clone, Copy)]
struct Env<'a> {
data_dir: &'a Path,
models_dir: &'a Path,
usage: &'a crate::usage::Fixture,
}
/// Creates the session directory, opens its transcript (continuing the
/// sequence numbering if one exists), settles what the session is doing,
/// and spawns the event pump -- with a driver behind it where there is a
@@ -1768,12 +1836,11 @@ fn launch(
meta: SessionConfig,
setup: &SetupConfig,
provider: &ProviderConfig,
data_dir: &Path,
models_dir: &Path,
env: Env<'_>,
notifications: broadcast::Sender<Notification>,
why: Launching,
) -> Result<Arc<LiveSession>> {
let dir = data_dir.join(&meta.id);
let dir = env.data_dir.join(&meta.id);
wg_app_link::private::create_dir(&dir)?;
let transcript_path = dir.join("transcript.jsonl");
let mut transcript = Transcript::open(&transcript_path)?;
@@ -1887,17 +1954,7 @@ fn launch(
let driver = Arc::new(Mutex::new(
driving
.then(|| {
make_driver(
&meta,
setup,
provider,
models_dir,
&dir,
&transcript_path,
&sink,
)
})
.then(|| make_driver(&meta, setup, provider, env, &dir, &transcript_path, &sink))
.transpose()?,
));
@@ -1939,18 +1996,22 @@ fn make_driver(
meta: &SessionConfig,
setup: &SetupConfig,
provider: &ProviderConfig,
models_dir: &Path,
env: Env<'_>,
dir: &Path,
transcript_path: &Path,
sink: &EventSink,
) -> Result<Arc<dyn Driver>> {
Ok(match provider.kind {
DriverKind::Echo => Arc::new(EchoDriver::new(sink.clone(), dir.to_path_buf())),
DriverKind::Echo => Arc::new(EchoDriver::new(
sink.clone(),
dir.to_path_buf(),
env.usage.clone(),
)),
DriverKind::LlamaCpp => Arc::new(LlamaDriver::launch(
meta,
provider,
&Transport::for_setup(setup),
models_dir,
env.models_dir,
transcript_path,
dir,
sink.clone(),
@@ -2283,6 +2344,7 @@ mod tests {
driver: Arc::new(Mutex::new(Some(Arc::new(EchoDriver::new(
sink.clone(),
dir.path().to_path_buf(),
crate::usage::Fixture::new(),
))))),
sink,
waiting: Mutex::new(VecDeque::new()),
+71 -10
View File
@@ -11,9 +11,13 @@
//! `crate::ssh`'s: this module decides *which* transport, that one knows what a
//! correct ssh invocation is.
//!
//! Known second operation, not built because nothing needs it yet: a managed
//! `llama-server` is spawned as a process but then spoken to over HTTP, so a
//! remote one needs a forwarded port (`ssh -L`) as well.
//! A transport is therefore two operations rather than one: **run this** and
//! **reach this port**. The second is what a managed `llama-server` needs -- it
//! is spawned as a process and then spoken to over HTTP -- and it is a no-op
//! locally, where the port a program binds is already one this machine can
//! dial. Over ssh it is an `-L` tunnel on the same connection that runs the
//! command, so the model server binds loopback on the far machine and is never
//! exposed to its network. See [`Transport::reserve_port`].
use std::path::{Path, PathBuf};
use std::process::Stdio;
@@ -22,14 +26,26 @@ use anyhow::{Context, Result};
use tokio::process::Child;
use crate::config::SshConfig;
pub use crate::ssh::Forward;
/// What a driver needs run in order to exist as a process. Deliberately just
/// the three things every transport can carry; anything a particular machine
/// needs is the transport's own configuration, not something a driver states.
/// what every transport can carry -- the command, where it runs, and a port the
/// caller needs to reach; anything a particular machine needs is the
/// transport's own configuration, not something a driver states.
pub struct Launch {
pub program: String,
pub args: Vec<String>,
pub cwd: Option<PathBuf>,
/// A port this program will listen on, and the port that reaches it
/// from here -- see [`Transport::reserve_port`], which is the only
/// thing that should produce one.
///
/// On the launch rather than in [`Transport::spawn`]'s signature
/// because it is part of what is being run: a caller that needs to
/// reach the process it is starting says so once, where it says
/// everything else about it, and every transport reads it the same
/// way.
pub forward: Option<Forward>,
}
impl Launch {
@@ -38,8 +54,16 @@ impl Launch {
program: program.into(),
args,
cwd: cwd.map(Path::to_path_buf),
forward: None,
}
}
/// Says that this program serves `forward.there`, and that the caller
/// will reach it at `forward.here`.
pub fn reaching(mut self, forward: Forward) -> Self {
self.forward = Some(forward);
self
}
}
/// How a launched process's standard streams are connected.
@@ -102,6 +126,7 @@ impl Transport {
&launch.program,
&launch.args,
launch.cwd.as_deref(),
launch.forward,
));
match streams {
Streams::Piped => {
@@ -153,12 +178,15 @@ impl Transport {
Self::Here => None,
Self::Ssh { ssh, .. } => Some(ssh),
};
let output =
crate::ssh::command(host, &launch.program, &launch.args, launch.cwd.as_deref())
let output = crate::ssh::command(
host,
&launch.program,
&launch.args,
launch.cwd.as_deref(),
launch.forward,
)
.output()
.with_context(|| {
format!("couldn't run \"{}\" {}", launch.program, self.describe())
})?;
.with_context(|| format!("couldn't run \"{}\" {}", launch.program, self.describe()))?;
if !output.status.success() {
let stderr = String::from_utf8_lossy(&output.stderr).trim().to_string();
anyhow::bail!(if stderr.is_empty() {
@@ -217,6 +245,34 @@ impl Transport {
})
}
/// Picks a port for a launched program to serve on, and the port that
/// reaches it from here.
///
/// The "reach this port" half of what a transport is. Locally there is
/// one port and the OS chooses it, by binding and letting go -- racy
/// in principle, and nothing on this machine is hunting for ports.
///
/// Over ssh the near end is chosen the same way and the far end is a
/// guess, because there is no portable way to ask a machine for a free
/// port that does not race with binding it anyway. It is taken from
/// [`FAR_PORTS`], below the range Linux hands out to outgoing
/// connections, so a collision means something else deliberately
/// listening there. That is not silent: the program fails to bind and
/// exits, and `session::llama` reports what its log said rather than
/// waiting out its readiness timeout.
pub fn reserve_port(&self) -> Result<Forward> {
let listener = std::net::TcpListener::bind("127.0.0.1:0")
.context("asking this machine for a free port")?;
let here = listener.local_addr()?.port();
Ok(match self {
Self::Here => Forward { there: here, here },
Self::Ssh { .. } => Forward {
there: rand::random_range(FAR_PORTS),
here,
},
})
}
/// How to say where this runs, for a log line a person reads.
pub fn describe(&self) -> String {
match self {
@@ -226,6 +282,11 @@ impl Transport {
}
}
/// Where a port on another machine is guessed from: high enough to be out
/// of the way of services, and below the 32768-60999 Linux hands out to
/// outgoing connections, which is where a guess would most often collide.
const FAR_PORTS: std::ops::Range<u16> = 20000..30000;
/// What a command is given on its standard input.
///
/// Three cases rather than an `Option<Stdio>` because they are three genuinely
+4 -1
View File
@@ -25,7 +25,10 @@ use crate::session::transport::{Launch, Transport};
/// session stores.
const PROBES: &[(&str, &str, DriverKind)] = &[
("claude-cli", "claude", DriverKind::ClaudeCli),
("local-llama", "llama-server", DriverKind::LlamaCpp),
// Named for the program rather than for where it runs: it runs
// wherever the setup is, and "local" was true only while a llama
// session could not be spawned on another machine.
("llama-cpp", "llama-server", DriverKind::LlamaCpp),
];
/// Models offered for a discovered Claude CLI. A shortcut list for the spawn
+101 -5
View File
@@ -14,6 +14,22 @@ use std::process::Command;
use crate::config::SshConfig;
/// A port on the machine a command runs on, and the port that reaches it from
/// the backend.
///
/// The second half of what a transport is (PLAN.md's SSH section): "run this"
/// plus "reach this port". Locally the two numbers are one and nothing is
/// forwarded; over ssh the connection carries an `-L` tunnel, so a model server
/// binds loopback on the far machine and is never exposed to its network.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct Forward {
/// What the launched program should listen on, on its own machine.
pub there: u16,
/// What this machine connects to. The same number as `there` when the
/// program runs here.
pub here: u16,
}
/// Options forced onto every connection. `BatchMode` makes a missing key fail
/// immediately with a readable message instead of hanging on a password prompt
/// nothing can answer; the keepalives turn a silently dropped link into a
@@ -39,6 +55,7 @@ pub fn command(
program: &str,
args: &[String],
cwd: Option<&Path>,
forward: Option<Forward>,
) -> Command {
let Some(ssh) = remote else {
let mut command = Command::new(program);
@@ -55,9 +72,31 @@ pub fn command(
};
let mut command = Command::new("ssh");
// -T: no pty. This carries JSONL, and a pty would rewrite it (echo, CRLF
// translation, ^C handling) into something the parser can't read.
if let Some(forward) = forward {
// A forwarded process is not spoken to over stdio, and that changes how
// it is shut down. Everything else here is a CLI reading its stdin, so
// killing the ssh client ends it; a `llama-server` never reads its own,
// so the same kill left it running on the far machine with the model
// loaded -- measured 2026-09-04, an orphan per stopped session. A pty
// is what makes sshd hang the far side up. `-tt` because this client
// has no terminal to inherit one from. The cost is a log that arrives
// through a line discipline, which nothing parses.
command.arg("-tt");
// Loopback at both ends: the far side binds 127.0.0.1, so what it
// serves is reachable only through this connection.
command.args([
"-L",
&format!("127.0.0.1:{}:127.0.0.1:{}", forward.here, forward.there),
]);
// Without this a forward that cannot be set up is a warning on stderr
// and a session that runs anyway, answering nothing -- which would
// arrive as "the model never became ready".
command.args(["-o", "ExitOnForwardFailure=yes"]);
} else {
// -T: no pty. This carries JSONL, and a pty would rewrite it (echo,
// CRLF translation, ^C handling) into something the parser can't read.
command.arg("-T");
}
for option in SSH_OPTIONS {
command.args(["-o", option]);
}
@@ -181,6 +220,7 @@ mod tests {
port: None,
identity_file: None,
options: vec![],
models_dir: None,
attachments_dir: None,
}
}
@@ -192,6 +232,7 @@ mod tests {
"claude",
&args(["-p", "--verbose"]),
Some(Path::new("/tmp/x")),
None,
);
assert_eq!(argv(&command), ["claude", "-p", "--verbose"]);
assert_eq!(command.get_current_dir(), Some(Path::new("/tmp/x")));
@@ -204,6 +245,7 @@ mod tests {
port: Some(2222),
identity_file: Some("/home/me/.ssh/id_ai".into()),
options: vec!["StrictHostKeyChecking=accept-new".to_string()],
models_dir: None,
attachments_dir: None,
};
let rendered = argv(&command(
@@ -211,6 +253,7 @@ mod tests {
"claude",
&args(["-p", "--model", "haiku"]),
Some(Path::new("/home/bob/work")),
None,
));
assert_eq!(rendered[0], "ssh");
@@ -231,12 +274,59 @@ mod tests {
#[test]
fn a_remote_command_without_a_cwd_just_execs() {
let ssh = bare_host();
let rendered = argv(&command(Some(&ssh), "claude", &args(["-p"]), None));
let rendered = argv(&command(Some(&ssh), "claude", &args(["-p"]), None, None));
assert_eq!(rendered.last().unwrap(), "exec 'claude' '-p'");
// No -i means no IdentitiesOnly: ~/.ssh/config decides instead.
assert!(!rendered.contains(&"IdentitiesOnly=yes".to_string()));
}
/// The second half of a transport: the connection that runs the command also
/// carries the port that reaches it.
///
/// Both ends are pinned to loopback, which is what keeps a model server off
/// the far machine's network -- asserted rather than trusted, because
/// dropping the addresses is a one-word edit that still works on a machine
/// nobody else can reach.
#[test]
fn a_forwarded_port_rides_the_same_connection_as_the_command() {
let ssh = bare_host();
let rendered = argv(&command(
Some(&ssh),
"llama-server",
&args(["--port", "24242"]),
None,
Some(Forward {
there: 24242,
here: 41000,
}),
));
let forward = rendered
.iter()
.position(|arg| arg == "-L")
.expect("a forward");
assert_eq!(rendered[forward + 1], "127.0.0.1:41000:127.0.0.1:24242");
assert!(rendered.contains(&"ExitOnForwardFailure=yes".to_string()));
// The half that is easy to lose: without a pty the far process outlives
// the connection, because nothing closes a stdin it never reads.
assert!(rendered.contains(&"-tt".to_string()));
assert!(!rendered.contains(&"-T".to_string()));
// Options come before the host, or ssh reads them as part of the
// remote command.
assert!(forward < rendered.len() - 2);
assert_eq!(
rendered.last().unwrap(),
"exec 'llama-server' '--port' '24242'"
);
// Nothing forwarded is nothing added: every other session is one of
// these, and an -L on it would bind a port for no reason.
let plain = argv(&command(Some(&ssh), "claude", &args(["-p"]), None, None));
assert!(!plain.contains(&"-L".to_string()));
// And a session that *is* spoken to over stdio keeps its raw pipe.
assert!(plain.contains(&"-T".to_string()));
assert!(!plain.contains(&"-tt".to_string()));
}
/// The one character quoting must not swallow. A working directory typed as
/// `~/repos/ai-app` was arriving as the literal directory `~`, and the
/// remote shell reported it missing -- which reads as the path being wrong
@@ -277,7 +367,13 @@ mod tests {
assert_eq!(expand_home(Path::new("/tmp/~/x")), Path::new("/tmp/~/x"));
assert_eq!(expand_home(Path::new("~user/x")), Path::new("~user/x"));
let local = command(None, "claude", &args(["-p"]), Some(Path::new("~/work")));
let local = command(
None,
"claude",
&args(["-p"]),
Some(Path::new("~/work")),
None,
);
assert_eq!(local.get_current_dir(), Some(home.join("work").as_path()));
}
@@ -301,7 +397,7 @@ mod tests {
// tries to close the quote and start a new command.
let ssh = bare_host();
let evil = Path::new("/tmp/'; touch /tmp/pwned; '");
let rendered = argv(&command(Some(&ssh), "claude", &[], Some(evil)));
let rendered = argv(&command(Some(&ssh), "claude", &[], Some(evil), None));
let script = rendered.last().unwrap();
assert_eq!(
script,
+337 -20
View File
@@ -30,13 +30,13 @@
//! already has over that machine.
use std::collections::HashMap;
use std::sync::Mutex;
use std::sync::{Arc, Mutex};
use std::time::{Duration, Instant};
use serde::Serialize;
use serde_json::Value;
use crate::config::{DriverKind, SetupConfig};
use crate::config::SetupConfig;
use crate::session::transport::{Launch, Transport};
const USAGE_URL: &str = "https://api.anthropic.com/api/oauth/usage";
@@ -107,10 +107,30 @@ pub struct UsageSnapshot {
pub fetched_at: f64,
}
/// The name of each meter, said in one place because two lists have to
/// agree on it: [`UsageSnapshot::provider`], which is what `GET /usage`
/// labels a row with, and [`crate::config::DriverKind::usage_provider`],
/// which is how a session says which of those rows is about it.
pub const CLAUDE: &str = "claude";
/// The invented one, for testing the screens that draw these -- see
/// [`Fixture`].
pub const ECHO: &str = "echo";
pub trait UsageProvider: Send + Sync {
fn name(&self) -> &'static str;
/// Blocking -- call off the async workers.
fn fetch(&self) -> UsageSnapshot;
/// How long an answer from this one may be reused.
///
/// A property of the provider rather than of the cache, because what
/// sets it is what asking costs: [`ClaudeUsage`] makes a network call
/// against an endpoint that rate-limits impatient callers, and the
/// fixture below reads a mutex. Caching the fixture for three minutes
/// would mean a test setting a number and watching the old one for
/// most of that, which reads exactly like the command not working.
fn poll_interval(&self) -> Duration {
MIN_POLL_INTERVAL
}
}
/// Reads the numbers behind Claude Code's `/usage` from one machine, using the
@@ -170,7 +190,7 @@ impl ClaudeUsage {
impl UsageProvider for ClaudeUsage {
fn name(&self) -> &'static str {
"claude"
CLAUDE
}
fn fetch(&self) -> UsageSnapshot {
@@ -278,24 +298,259 @@ fn parse_windows(body: &Value) -> Vec<UsageWindow> {
.collect()
}
/// An invented answer, so the screens that draw these can be exercised
/// without an account.
///
/// Every state the usage bar and the usage dialog can be in is otherwise
/// reachable only by spending somebody's quota or by breaking a machine:
/// a number near the top, a machine nobody has logged into, one that
/// cannot be reached, a window between blocks with no reset time. Those
/// are exactly the states worth looking at, and the ones nobody looks at
/// because arranging them costs real turns. An echo session sets this
/// with `/usage` (see `session::echo`), which is the same bargain the
/// rest of that driver makes: the fixture is invented, what is real is
/// the path it travels.
///
/// Shared by the session layer, which writes it, and [`UsageMonitor`],
/// which reads it. Empty until something sets it, and an empty fixture
/// produces no snapshot at all -- an echo session meters nothing, and
/// nothing is what the phone should draw.
#[derive(Clone, Default)]
pub struct Fixture {
said: Arc<Mutex<Option<Reported>>>,
}
/// What a meter answered: which of the four states it is in, and whatever
/// windows go with it. Empty for every state but [`UsageState::Ok`].
type Reported = (UsageState, Vec<UsageWindow>);
/// How long the invented five-hour window has left, when nothing says.
const FIXTURE_MINUTES: i64 = 125;
impl Fixture {
pub fn new() -> Self {
Self::default()
}
fn is_set(&self) -> bool {
self.said.lock().unwrap().is_some()
}
fn read(&self) -> Option<Reported> {
self.said.lock().unwrap().clone()
}
/// Acts on the words typed after `/usage`, and says what it did.
///
/// The vocabulary lives here rather than in the echo driver because
/// these are this module's states: a driver spelling them out would
/// be a second place that has to learn about a fifth one.
pub fn command(&self, words: &str) -> String {
let mut words = words.split_whitespace();
let Some(first) = words.next() else {
return match self.read() {
Some((state, windows)) => format!("usage fixture: {}", describe(&state, &windows)),
None => "usage fixture: unset, so this session meters nothing. \
`/usage 42` puts up a five-hour window at 42%."
.to_string(),
};
};
let rest: Vec<&str> = words.collect();
let detail = || {
if rest.is_empty() {
"set by /usage".to_string()
} else {
rest.join(" ")
}
};
let (state, windows) = match first {
"off" | "none" | "clear" => {
*self.said.lock().unwrap() = None;
return "usage fixture cleared: this session meters nothing again".to_string();
}
"notloggedin" | "logged-out" => (UsageState::NotLoggedIn, Vec::new()),
"unreachable" => (UsageState::Unreachable { detail: detail() }, Vec::new()),
"failed" => (UsageState::Failed { detail: detail() }, Vec::new()),
percent => match percent.parse::<f64>() {
Ok(percent) => (
UsageState::Ok,
fixture_windows(percent.clamp(0.0, 100.0), rest.first().copied()),
),
Err(_) => {
return format!(
"\"{percent}\" is not one of this fixture's answers. Say a percentage \
(`/usage 42`, optionally with `90` minutes left, `never` for a window \
between blocks, or `unreadable` for a reset time that cannot be read), \
or one of `notloggedin`, `unreachable`, `failed`, `off`."
);
}
},
};
let said = describe(&state, &windows);
*self.said.lock().unwrap() = Some((state, windows));
format!("usage fixture set: {said}")
}
}
/// The three windows Claude reports today, invented around one number.
///
/// Three rather than one because the bar under a session header reads the
/// five-hour window and the dialog behind the button draws all of them,
/// and a fixture with one window leaves half the screen untested. The
/// weekly ones are derived from the same figure so that the worst of them
/// -- which is what colours the button -- is still the one asked for.
fn fixture_windows(percent: f64, reset: Option<&str>) -> Vec<UsageWindow> {
let resets_at = match reset {
// The state a real response is in between blocks: there is no
// window running, so there is nothing to reset. It is not a
// missing value, and the phone words it differently.
Some("never") | Some("none") => None,
// A timestamp that arrives and cannot be read, which is the one
// case that really is "we could not find out".
Some("unreadable") | Some("bad") => Some("whenever it feels like it".to_string()),
other => Some(reset_in(
other
.and_then(|word| word.parse().ok())
.unwrap_or(FIXTURE_MINUTES),
)),
};
vec![
UsageWindow {
kind: "session".to_string(),
label: "5-hour window".to_string(),
percent,
resets_at: resets_at.clone(),
active: true,
},
UsageWindow {
kind: "weekly_all".to_string(),
label: "Weekly (all models)".to_string(),
percent: percent / 2.0,
resets_at: resets_at.as_ref().map(|_| reset_in(FIXTURE_MINUTES * 40)),
active: false,
},
UsageWindow {
kind: "weekly_scoped".to_string(),
label: "Weekly (Echo)".to_string(),
percent: percent / 4.0,
resets_at: resets_at.as_ref().map(|_| reset_in(FIXTURE_MINUTES * 40)),
active: false,
},
]
}
/// `minutes` from now, in the format the real endpoint sends.
fn reset_in(minutes: i64) -> String {
let at = time::OffsetDateTime::now_utc() + time::Duration::minutes(minutes);
at.format(&time::format_description::well_known::Rfc3339)
// Formatting a timestamp cannot fail for any input this builds;
// saying so beats a fixture that silently has no reset time.
.unwrap_or_else(|_| "unformattable".to_string())
}
/// One line naming what a fixture is currently claiming, for the reply
/// the echo session writes back.
fn describe(state: &UsageState, windows: &[UsageWindow]) -> String {
match state {
UsageState::Ok => match windows.first() {
Some(window) => format!(
"{}% of the five-hour window, {}",
window.percent,
match &window.resets_at {
Some(at) => format!("resetting at {at}"),
None => "with no reset time (the between-blocks state)".to_string(),
}
),
None => "no windows at all".to_string(),
},
UsageState::NotLoggedIn => "nobody is logged in on this machine".to_string(),
UsageState::Unreachable { detail } => format!("machine unreachable ({detail})"),
UsageState::Failed { detail } => format!("the meter failed ({detail})"),
}
}
/// The fixture, as a provider, so it travels the same route and the same
/// cache as a real meter rather than being spliced in at the screen.
struct EchoUsage {
setup: String,
setup_name: String,
fixture: Fixture,
}
impl UsageProvider for EchoUsage {
fn name(&self) -> &'static str {
ECHO
}
fn fetch(&self) -> UsageSnapshot {
let (state, windows) = self
.fixture
.read()
// Only ever built for a fixture that is set; a race with
// `/usage off` between the two reads lands here, and "the
// machine could not be asked" is the honest word for it.
.unwrap_or((
UsageState::Unreachable {
detail: "the usage fixture was cleared".to_string(),
},
Vec::new(),
));
UsageSnapshot {
provider: self.name().to_string(),
setup: self.setup.clone(),
setup_name: self.setup_name.clone(),
state,
windows,
fetched_at: crate::session::now(),
}
}
/// Read from memory, and set by somebody who is about to look at the
/// screen it changes.
fn poll_interval(&self) -> Duration {
Duration::ZERO
}
}
/// Which paid services a machine can be asked about.
///
/// Derived from what the setup says it can run, so a machine with no Claude
/// provider is not asked about Claude limits -- it has none, and a row saying
/// so would be a fact about nothing. A second service later adds a branch here
/// and an impl beside [`ClaudeUsage`], not a screen.
fn providers_for(setup: &SetupConfig) -> Vec<Box<dyn UsageProvider>> {
/// so would be a fact about nothing.
///
/// Which meter a provider has is [`DriverKind::usage_provider`]'s answer rather
/// than a second match on kinds here, because the phone pairs a session with
/// one of these rows by that same name: two lists that disagreed would leave a
/// session looking for a snapshot nothing produces. A second service later is a
/// name there and an impl beside [`ClaudeUsage`], not a screen.
fn providers_for(setup: &SetupConfig, fixture: &Fixture) -> Vec<Box<dyn UsageProvider>> {
let mut found: Vec<Box<dyn UsageProvider>> = Vec::new();
if setup
.providers
.iter()
.any(|provider| provider.kind == DriverKind::ClaudeCli)
{
found.push(Box::new(ClaudeUsage {
for provider in &setup.providers {
let Some(name) = provider.kind.usage_provider() else {
continue;
};
// A machine offering two Claude providers has one account, not
// two: the meter belongs to the machine and the service, which is
// exactly what the cache is keyed by.
if found.iter().any(|already| already.name() == name) {
continue;
}
match name {
CLAUDE => found.push(Box::new(ClaudeUsage {
setup: setup.id.clone(),
setup_name: setup.name.clone(),
transport: Transport::for_setup(setup),
}));
})),
// Nothing at all until a test has asked for something: an
// echo session costs nothing, so the honest answer is no row
// rather than a row saying zero.
ECHO if fixture.is_set() => found.push(Box::new(EchoUsage {
setup: setup.id.clone(),
setup_name: setup.name.clone(),
fixture: fixture.clone(),
})),
_ => {}
}
}
found
}
@@ -313,11 +568,18 @@ type Cached = HashMap<(String, &'static str), (Instant, UsageSnapshot)>;
/// machine per service per [`MIN_POLL_INTERVAL`], however often the phone asks.
pub struct UsageMonitor {
cache: Mutex<Cached>,
/// The invented meter an echo session can put up; empty unless one
/// has. Shared with the session layer, which is where the command
/// that sets it is typed -- see [`Fixture`].
fixture: Fixture,
}
impl UsageMonitor {
pub fn new() -> Self {
Self::default()
pub fn new(fixture: Fixture) -> Self {
Self {
cache: Mutex::new(Cached::new()),
fixture,
}
}
/// One snapshot per machine that offers a paid service, in the order the
@@ -329,10 +591,10 @@ impl UsageMonitor {
pub fn snapshots(&self, setups: &[SetupConfig]) -> Vec<UsageSnapshot> {
let mut fresh = Vec::new();
for setup in setups {
for provider in providers_for(setup) {
for provider in providers_for(setup, &self.fixture) {
let key = (setup.id.clone(), provider.name());
if let Some((fetched, snapshot)) = self.cache.lock().unwrap().get(&key)
&& fetched.elapsed() < MIN_POLL_INTERVAL
&& fetched.elapsed() < provider.poll_interval()
{
// Cached numbers, but the machine's *name* is read fresh: a
// rename should show immediately rather than waiting out a
@@ -367,6 +629,7 @@ impl UsageMonitor {
#[cfg(test)]
mod tests {
use super::*;
use crate::config::DriverKind;
#[test]
fn parses_the_limits_array_defensively() {
@@ -404,6 +667,7 @@ mod tests {
port: None,
identity_file: None,
options: vec!["ConnectTimeout=1".to_string()],
models_dir: None,
attachments_dir: None,
}),
providers: vec![crate::config::ProviderConfig {
@@ -475,9 +739,62 @@ mod tests {
models: vec![],
}];
// A machine with no Claude on it has no Claude limits, and a row
// reporting on it would be a fact about nothing.
assert!(providers_for(&echo_only).is_empty());
assert_eq!(providers_for(&unreachable_setup()).len(), 1);
// reporting on it would be a fact about nothing. Echo included:
// an echo session spends nothing, so until a fixture says
// otherwise there is no meter to report.
let unset = Fixture::new();
assert!(providers_for(&echo_only, &unset).is_empty());
assert_eq!(providers_for(&unreachable_setup(), &unset).len(), 1);
// And with one set, that machine has exactly the invented meter
// -- under the name the session's `usageProvider` will name.
let fixture = Fixture::new();
fixture.command("42");
let found = providers_for(&echo_only, &fixture);
assert_eq!(found.len(), 1);
assert_eq!(found[0].name(), ECHO);
assert_eq!(DriverKind::Echo.usage_provider(), Some(ECHO));
assert_eq!(DriverKind::ClaudeCli.usage_provider(), Some(CLAUDE));
// A local model costs nothing to run, so it meters nothing.
assert_eq!(DriverKind::LlamaCpp.usage_provider(), None);
}
/// The states the fixture exists to make reachable, and the one thing
/// it must not do: invent a reset time for a window that has none.
#[test]
fn the_fixture_says_each_state_the_screens_have_to_draw() {
let fixture = Fixture::new();
assert!(fixture.read().is_none(), "unset until somebody sets it");
fixture.command("42 90");
let (state, windows) = fixture.read().expect("set");
assert_eq!(state, UsageState::Ok);
assert_eq!(windows[0].kind, "session");
assert_eq!(windows[0].percent, 42.0);
assert!(windows[0].resets_at.is_some());
// Between blocks: no reset time, which the phone words as the
// window not running rather than as a time it could not read.
fixture.command("42 never");
assert_eq!(fixture.read().expect("set").1[0].resets_at, None);
fixture.command("unreachable no route to host");
assert!(matches!(
fixture.read().expect("set").0,
UsageState::Unreachable { detail } if detail == "no route to host",
));
fixture.command("off");
assert!(fixture.read().is_none());
// A word it does not know changes nothing and says what it takes.
fixture.command("42");
let refused = fixture.command("sideways");
assert!(
refused.contains("not one of this fixture's answers"),
"{refused}"
);
assert_eq!(fixture.read().expect("still set").1[0].percent, 42.0);
}
#[test]