diff --git a/AGENTS.md b/AGENTS.md index 1f49443..d46e571 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -203,15 +203,44 @@ images both ways), and the usage screen. host, `session::transport` turns that into an `ssh host …` invocation, and the driver never learns which it got. -**Phase 4 (llama.cpp)** works end to end, phone included (2026-08-28). -Models are browsed and downloaded from HuggingFace (`models.rs`, resumable -and verified), and `session::llama` runs one through `llama-server` over -its OpenAI-compatible streaming endpoint. Two things are deliberate and -easy to undo by accident: the conversation is rebuilt from the +**Phase 4 (llama.cpp)** works end to end, phone included (2026-08-28), +**on any machine a setup names** (2026-09-04). Models are browsed and +downloaded from HuggingFace (`models.rs`, resumable and verified), and +`session::llama` runs one through `llama-server` over its +OpenAI-compatible streaming endpoint. The conversation is rebuilt from the **transcript** rather than kept in the driver, because driver memory is -invisible to a second device; and a llama session is refused on an ssh -host, because the model is reached over HTTP and forwarding that port is -not built. +invisible to a second device -- deliberate, and easy to undo by accident. + +A remote llama session is the same command through the same transport +plus the second half of what a transport is: `Transport::reserve_port` +hands back a port the server binds *there* and a port that reaches it +*here*, and the ssh connection carrying the command carries the `-L` +tunnel between them (`llama-server` binds loopback on the far machine, so +nothing is served to its network). Three things that came out of building +it, each of which is easy to get wrong again: + +- **A forwarded launch gets a pty (`-tt`); every other one keeps `-T`.** + Killing the ssh client ends a CLI because it closes the stdin that CLI + is reading. `llama-server` never reads its stdin, so the same kill left + it running on the far machine holding the model in memory -- measured + 2026-09-04, one orphan per stopped session. A pty is what makes sshd + hang the far side up. Its log then arrives through a line discipline, + which nothing parses. +- **The model is looked for on the machine that will serve it**, at that + machine's own models directory (`SshConfig::models_dir`, defaulting to + `~/.local/share/ai-app/models` expanded *there*). What this backend has + downloaded is on that machine only when they are the same machine, so + `GET /setups/{id}/models` is what the spawn screen offers rather than + `GET /models`, and a model that is not there is refused at the spawn + with a sentence saying so. Downloading *to* another machine is not + built; the file gets there however anything else does. +- **A readiness poll watches the process, not only the port.** A model + that will not load, a port already taken, a flag an older build does not + know: all of them exit within a second and none will ever answer + `/health`, so waiting out the 300s timeout turned the server's own + account of the problem into "gave up". The failure now carries the last + few lines of `llama-server.log`, which on a remote session is the only + copy anybody reading the phone can see. Setups — machines, each carrying what it can run — are added, renamed, re-probed and removed from the app; providers are **discovered by asking @@ -226,12 +255,25 @@ symlink into `~/.local/bin` before a setup finds it. The escape hatch for anything odder is editing `config.ron` on the backend, deliberately the one authority the phone does not have. -**Testing llama.cpp here:** the prebuilt CPU build lives outside the repo -at `~/.local/opt/llama.cpp` (the 15 MB `ubuntu-x64` release asset). It -needs its own directory on `LD_LIBRARY_PATH`, so start the server as -`LD_LIBRARY_PATH=~/.local/opt/llama.cpp ai-server …` and point a provider's -`command` at `~/.local/opt/llama.cpp/llama-server`. A 0.6B Q8_0 answers at -usable speed on this VM's 8 cores. **Do not test with a 2-bit quant**: the +**llama.cpp is set up in this VM** (2026-09-04) and needs nothing typed: +the prebuilt CPU build is at `~/.local/opt/llama.cpp` (the 15 MB +`ubuntu-x64` release asset), symlinked as `/usr/local/bin/llama-server` so +that **discovery finds it over ssh too** -- `~/.local/bin` is not on the +PATH a non-interactive ssh session gets, which is why the symlink is +there and not only in `~/.local/bin`. It resolves its own libraries +through `$ORIGIN`, so no `LD_LIBRARY_PATH` is needed. One model is +downloaded, `unsloth/Qwen3-0.6B-GGUF/Qwen3-0.6B-Q8_0.gguf` (639 MB, under +`~/.local/share/ai-app/models`), which answers at usable speed on this +VM's 8 cores. + +**And the ssh path is exercisable here**, because this VM can ssh to +itself: the key is `~/.config/ai-app/ssh-self` (its public half is in +`~/.ssh/authorized_keys`, labelled removable), and a setup naming +`bob@127.0.0.1` with that `identityFile` plus +`options: ["StrictHostKeyChecking=no", "UserKnownHostsFile=/tmp/ai-app-known-hosts"]` +discovers `claude-cli` and `llama-cpp` on it. That is the whole rig for +"does a remote llama session work", since the far machine is this one and +the model file is the same file. **Do not test with a 2-bit quant**: the IQ2_XXS of that model produces fluent nonsense, which reads exactly like a broken driver — `llama-cli` produces the same from the file directly, which is how to tell the two apart in a hurry. @@ -292,6 +334,24 @@ first if a remote spawn ever mangles an argument. Dev Updater lists every variant under `build/outputs/apk`, so pick `release` there; a phone still holding the debug build has to uninstall it first, since the two are signed differently. +- **A rate-limit bar belongs to a session's provider, not to its + machine.** One machine offers echo, the Claude CLI and a local model at + once and only the CLI spends anything, so a session says which meter + reports on it (`usageProvider`, from `DriverKind::usage_provider`, which + `usage::providers_for` reads too so the two lists cannot disagree) and + the phone matches a snapshot on machine *and* provider. Nothing meters + a llama or echo session, and the phone draws **nothing at all** for one + -- not a zero, and not "unknown". The bar also draws nothing while the + first fetch is out: "checking" under a session that turns out to meter + nothing is a row the screen then has to withdraw. +- **`/usage` in an echo session puts up an invented meter**, which is how + those screens' states are reached without spending quota: + `/usage 42`, `/usage 95 20` (minutes left), `/usage 42 never` (the + between-blocks window with no reset time), `/usage 42 unreadable`, + `/usage notloggedin`, `/usage unreachable`, `/usage failed`, + `/usage off`. The vocabulary is `usage::Fixture`'s, since those are its + states; with none set an echo session meters nothing, which is the + ordinary case. - **A row something is happening to is dimmed, drained of colour, inert, and says which operation in a word** -- `BusyItem`, used by both the session list and the import list so the appearance is learned once. The diff --git a/PLAN.md b/PLAN.md index f83f603..7cdcd55 100644 --- a/PLAN.md +++ b/PLAN.md @@ -684,7 +684,22 @@ host) and **hosts**. The manager runs at most one llama-server per prompt-replayed by pi against the new endpoint). - Remote llama-server output is only reachable from the backend host, and binds localhost on the remote side with an SSH local port forward - (`ssh -L`) held by the manager — no LAN-exposed inference ports. + (`ssh -L`) held by the manager — no LAN-exposed inference ports. Built + 2026-09-04, held by the session's own ssh client rather than by a + manager: there is one server per session (not per `(host, model)`), so + the process that runs it is the process that owns the tunnel, and the + two die together. +- **The model file lives on the machine that serves it** (2026-09-04). + Each setup names its own models directory (`SshConfig::models_dir`, + default `~/.local/share/ai-app/models` expanded on that machine), and a + spawn resolves the key there — one round trip that answers "at + /abs/path" or "missing", so a model that is not there is refused at the + spawn instead of becoming a server that never becomes ready. The spawn + screen offers `GET /setups/{id}/models`, which is that machine's list, + rather than `GET /models`, which is the backend's downloads. Downloading + *to* another machine is deliberately not built: it would be a + multi-gigabyte transfer with no progress anywhere, and the file gets + there however anything else on that machine got there. ### SSH @@ -712,7 +727,23 @@ host) and **hosts**. The manager runs at most one llama-server per as a process but then spoken to over HTTP, so a remote one needs a forwarded port (`ssh -L`) as well as a spawned process. A transport is therefore "run this" plus "reach this port", and the second operation is - a no-op locally. + a no-op locally. **Built 2026-09-04**: `Transport::reserve_port` returns + a `Forward { there, here }` — the port the program binds on its own + machine and the port that reaches it from the backend, the same number + when that machine is this one — and `Launch::reaching` carries it, so + the connection that runs the command also carries the tunnel. The far + end is a guess from a range below the ephemeral one, because no + portable way to ask a machine for a free port avoids racing with the + bind anyway; a collision is not silent, since the program fails to bind + and the readiness poll reports what its log said. +- **A forwarded launch gets a pty and every other one does not** (measured + 2026-09-04). Killing the ssh client ends a CLI because it closes the + stdin that CLI is reading; `llama-server` never reads its stdin, so the + same kill left it running on the far machine with the model loaded — + one orphan per stopped session. With `-tt` the far side takes SIGHUP + when the connection goes. Its log then arrives through a line + discipline, which nothing parses. `-T` stays everywhere else, where a + pty would rewrite the JSONL. - Images need no file transfer, contrary to what this section said before: `attachment_block` base64s an uploaded image into the stream-json message itself, and produced images come back the same way @@ -746,6 +777,26 @@ optional and degrades rather than erroring. Structure it as one `UsageProvider` per paid service so a second service later is a new impl, not a parallel screen (rule 9). +**Per provider, not per machine (decided 2026-09-04).** A machine is not +what is metered; the provider a session runs is. One machine offers echo, +the Claude CLI and a local model side by side, and only the second of them +spends anything — so pairing a session with a snapshot by machine alone +drew the CLI's five-hour window under every echo session on it, reporting +a quota that session cannot spend and could never run down. A session now +names its meter (`usageProvider`, from `DriverKind::usage_provider`, which +`usage::providers_for` also reads so the two lists cannot disagree), and +`GET /usage` is matched on machine *and* provider. `None` is a session +that meters nothing, and the phone draws nothing at all for it — not a +zero, and not "unknown". + +`DriverKind::Echo` names a meter of its own, and it exists only when a +test has asked for one: `/usage` in an echo session sets an invented +answer (`usage::Fixture`), and with none set there is no snapshot and no +bar. That is what makes the states of those screens reachable — a number +near the top, a window between blocks with no reset time, a machine +nobody logged into, one that could not be reached — without spending real +quota to arrange them, which is why none of them had ever been looked at. + **Per machine, not per backend (decided 2026-08-29).** The credential store that matters is the one on the machine the session runs on, because that is the account being billed. Reading this machine's was right only while the @@ -789,7 +840,8 @@ POST /sessions/:id/compact (llama sessions) POST /sessions/:id/attachments multipart upload → id (referenced by /message) GET /sessions/:id/files/:ref images the session produced or was sent DELETE /sessions/:id kill process, release llama-server, delete transcript+files -GET /usage cached usage windows +GET /usage cached usage windows, per machine and provider +GET /setups/:id/models GGUFs on that machine, for a llama session there GET /setups/:id/dir?path=P entries of directory P, and P resolved GET /setups/:id/file?path=P content of file P, or why not PUT /setups/:id/file {path, content, ifSha256}; 409 if it moved on @@ -1140,8 +1192,10 @@ window just fills. shell-quoted). Attachment shipping turned out to be unnecessary for images — they ride the stdio JSONL as base64 in both directions, so nothing needs `scp` — and was built on 2026-09-03 for files, which - are attached by path (see "Transport" above). Still outstanding: - remote llama-server with its port forward, which comes with phase 4. + are attached by path (see "Transport" above). Remote llama-server with + its port forward landed 2026-09-04 — see "Transport" above for the + forward and the pty, and "llama-server management" for where the model + file has to be. Two things learned doing it: a remote session inherits ssh's non-login PATH, which is narrower than an interactive shell's (point `command` at an absolute path if a CLI isn't found), and the remote command is run diff --git a/app/androidApp/src/main/kotlin/com/example/aiapp/Api.kt b/app/androidApp/src/main/kotlin/com/example/aiapp/Api.kt index c47e2c3..7528178 100644 --- a/app/androidApp/src/main/kotlin/com/example/aiapp/Api.kt +++ b/app/androidApp/src/main/kotlin/com/example/aiapp/Api.kt @@ -193,6 +193,17 @@ data class SessionSummary( * server because that is where a provider's kind is known -- see `uploadPickedImage`. */ val maxImageEdge: Int?, + /** + * Which of `GET /usage`'s snapshots is about this session, and null where nothing meters it. + * + * The rate-limit bar answers a question about an *account*, and what decides which account -- + * if any -- is the provider this session runs, not the machine it runs on. Pairing by machine + * alone drew the Claude CLI's five-hour window under every echo session on a machine that also + * has the CLI: a quota that session cannot spend and could never run down. Decided by the + * server for the same reason [maxImageEdge] is -- it is a fact about the provider's kind, and + * this app has only its name. + */ + val usageProvider: String?, val status: String, val lastActivity: Double, ) @@ -213,6 +224,7 @@ private fun parseSession(session: JSONObject) = contextTokens = if (session.has("contextTokens")) session.getLong("contextTokens") else null, maxImageEdge = session.optInt("maxImageEdge", 0).takeIf { it > 0 }, + usageProvider = session.optString("usageProvider").ifEmpty { null }, status = session.getString("status"), lastActivity = session.getDouble("lastActivity"), ) @@ -402,6 +414,12 @@ data class SshDetails( * Where files attached from here land on that machine; null for the session's own directory. */ val attachmentsDir: String? = null, + /** + * Where that machine keeps its GGUF models; null for the same place the backend keeps its own + * (`~/.local/share/ai-app/models`, read on that machine). A llama.cpp session serves the file + * from the machine it runs on, so this is where its models are looked for and listed. + */ + val modelsDir: String? = null, ) private fun SshDetails.toJson() = @@ -409,6 +427,7 @@ private fun SshDetails.toJson() = if (port != null) put("port", port) if (!identityFile.isNullOrBlank()) put("identityFile", identityFile) if (!attachmentsDir.isNullOrBlank()) put("attachmentsDir", attachmentsDir) + if (!modelsDir.isNullOrBlank()) put("modelsDir", modelsDir) } /** What a machine turns out to have, without saving anything. */ @@ -1107,6 +1126,26 @@ private fun parseDownload(o: JSONObject) = error = if (o.has("error")) o.getString("error") else null, ) +/** + * The models on one machine, which is the list a llama.cpp session there can choose from. + * + * Not [fetchModels], which is what the *backend* has downloaded. A session serves its model from + * the machine it runs on, so for a machine reached over ssh those are two different lists -- and + * offering the backend's would name files that are not there, turning a choice that cannot work + * into a session that fails when it tries to load one. + */ +fun fetchSetupModels(settings: ServerSettings, setupId: String): List = + requestFromServer(settings, "/setups/${setupId.urlEncoded()}/models") { connection -> + JSONArray(connection.inputStream.bufferedReader().readText()).mapObjects { m -> + LocalModel( + key = m.getString("key"), + repo = m.getString("repo"), + file = m.getString("file"), + bytes = m.getLong("bytes"), + ) + } + } + fun fetchModels(settings: ServerSettings): Models = requestFromServer(settings, "/models") { connection -> val body = JSONObject(connection.inputStream.bufferedReader().readText()) diff --git a/app/androidApp/src/main/kotlin/com/example/aiapp/SessionScreen.kt b/app/androidApp/src/main/kotlin/com/example/aiapp/SessionScreen.kt index b108c10..75ada87 100644 --- a/app/androidApp/src/main/kotlin/com/example/aiapp/SessionScreen.kt +++ b/app/androidApp/src/main/kotlin/com/example/aiapp/SessionScreen.kt @@ -1193,7 +1193,7 @@ fun SessionScreen( // One poll for the machines' limits, read by everything on this screen that reports them: // the bar under the header, the colour of the button that opens the dialog, and the dialog. val usageFeed = rememberUsageFeed(settings) - val usage = usageFeed.forSetup(summary.setup) + val usage = usageFeed.forSession(summary) RecordFrames() var usageOpen by remember { mutableStateOf(false) } var settingsOpen by remember { mutableStateOf(false) } diff --git a/app/androidApp/src/main/kotlin/com/example/aiapp/SessionUsageBar.kt b/app/androidApp/src/main/kotlin/com/example/aiapp/SessionUsageBar.kt index c2b3695..5b61d62 100644 --- a/app/androidApp/src/main/kotlin/com/example/aiapp/SessionUsageBar.kt +++ b/app/androidApp/src/main/kotlin/com/example/aiapp/SessionUsageBar.kt @@ -74,13 +74,22 @@ class UsageFeed( /** Ask the backend again now. The dialog's refresh button; the poll does it on its own. */ val refresh: () -> Unit, ) { - /** What [setup]'s own limits came back as. See [usageFor] for why the states are these. */ - fun forSetup(setup: String): SessionUsage = - when (val state = snapshots) { + /** + * What meters [session], and what that meter came back as. See [usageFor] for the states. + * + * A session rather than a machine, because a machine is not what is metered: one machine runs + * the Claude CLI and an echo session side by side, and only the first of them spends anything. + */ + fun forSession(session: SessionSummary): SessionUsage { + // Settled without asking anybody: a session nothing meters has nothing to check, and + // "checking" is what the fetch's own states would say about it for as long as one is out. + val provider = session.usageProvider ?: return SessionUsage.NotMetered + return when (val state = snapshots) { is LoadState.Loading -> SessionUsage.Waiting is LoadState.Error -> SessionUsage.Unavailable(state.message) - is LoadState.Loaded -> usageFor(state.value, setup) + is LoadState.Loaded -> usageFor(state.value, session.setup, provider) } + } } /** @@ -165,9 +174,16 @@ fun SessionUsageBar(usage: SessionUsage, modifier: Modifier = Modifier) { } } - // Nothing at all for a machine that meters nothing: a row saying "unknown" there would + // Nothing at all for a session that meters nothing: a row saying "unknown" there would // report a problem about a setup somebody chose, on every screen, forever. - if (usage is SessionUsage.NotMetered) { + // + // And nothing while the first fetch is out, which is not the same kind of silence. A + // request in flight is not a state to report -- and the session that meters nothing is + // exactly the one this cannot yet tell apart, so "5-hour usage: checking" appeared under + // an echo session for half a second and was then taken away. A row that has to be + // withdrawn is worse than one that arrives late, and this is the only state here whose + // wrongness is a matter of timing rather than of fact. + if (usage is SessionUsage.NotMetered || usage is SessionUsage.Waiting) { return } @@ -178,9 +194,10 @@ fun SessionUsageBar(usage: SessionUsage, modifier: Modifier = Modifier) { // Words, not a colour and not an empty bar: every one of these is a different kind of // answer from "this much is used", and only words carry a difference in kind. when (val state = usage) { - SessionUsage.NotMetered -> Unit + // Both handled above, before the row exists at all. + SessionUsage.NotMetered, + SessionUsage.Waiting -> Unit is SessionUsage.Unavailable -> UsageNote("5-hour usage unknown -- ${state.why}") - SessionUsage.Waiting -> UsageNote("5-hour usage: checking") is SessionUsage.Known -> { val window = state.windows.firstOrNull { it.kind == "session" } if (window == null) { @@ -242,17 +259,23 @@ private fun fiveHourLabel(window: UsageWindow, now: OffsetDateTime): String { } /** - * One machine's snapshot, out of every machine's. + * One meter's snapshot, out of every machine's: [setup]'s row for [provider]. + * + * Both halves are needed to pick it. A machine can hold more than one meter -- the Claude CLI's + * account and, while a test has one set, an echo session's invented one -- and a snapshot is one + * service on one machine. * * Every way of having *failed* to get numbers is [SessionUsage.Unavailable] with the reason in it: * a machine nobody logged into, one that could not be reached, a snapshot that came back empty. * None of them may look like zero, and none may look like [SessionUsage.NotMetered], which is the * machine having no quota rather than the question going unanswered. */ -fun usageFor(snapshots: List, setup: String): SessionUsage { - // No snapshot at all means the backend never asked, which it only does for a machine with - // nothing metered on it. That is a different answer from having asked and failed. - val mine = snapshots.firstOrNull { it.setup == setup } ?: return SessionUsage.NotMetered +fun usageFor(snapshots: List, setup: String, provider: String): SessionUsage { + // No snapshot at all means the backend never asked, which it only does where there is nothing + // to ask about. That is a different answer from having asked and failed. + val mine = + snapshots.firstOrNull { it.setup == setup && it.provider == provider } + ?: return SessionUsage.NotMetered if (mine.state != "ok") { return SessionUsage.Unavailable(mine.detail ?: mine.state) } diff --git a/app/androidApp/src/main/kotlin/com/example/aiapp/SetupsScreen.kt b/app/androidApp/src/main/kotlin/com/example/aiapp/SetupsScreen.kt index 23c2dc0..5fb4561 100644 --- a/app/androidApp/src/main/kotlin/com/example/aiapp/SetupsScreen.kt +++ b/app/androidApp/src/main/kotlin/com/example/aiapp/SetupsScreen.kt @@ -237,6 +237,7 @@ private fun AddSetupDialog( var address by remember { mutableStateOf("") } var identity by remember { mutableStateOf("") } var attachmentsDir by remember { mutableStateOf("") } + var modelsDir by remember { mutableStateOf("") } var tested by remember { mutableStateOf(null) } var testing by remember { mutableStateOf(false) } @@ -251,6 +252,7 @@ private fun AddSetupDialog( port = typedPort, identityFile = identity.trim().ifEmpty { null }, attachmentsDir = attachmentsDir.trim().ifEmpty { null }, + modelsDir = modelsDir.trim().ifEmpty { null }, ) } @@ -296,6 +298,14 @@ private fun AddSetupDialog( label = { Text("Folder for attached files (optional)") }, singleLine = true, ) + // Where that machine's GGUFs are, for a llama.cpp session on it. Blank means + // the same place this backend keeps its own downloads, read on that machine. + OutlinedTextField( + value = modelsDir, + onValueChange = { modelsDir = it }, + label = { Text("Folder for models (optional)") }, + singleLine = true, + ) tested?.let { Spacer(Modifier.height(8.dp)) Text(it, style = MaterialTheme.typography.bodySmall) diff --git a/app/androidApp/src/main/kotlin/com/example/aiapp/SpawnScreen.kt b/app/androidApp/src/main/kotlin/com/example/aiapp/SpawnScreen.kt index 699c81f..172c861 100644 --- a/app/androidApp/src/main/kotlin/com/example/aiapp/SpawnScreen.kt +++ b/app/androidApp/src/main/kotlin/com/example/aiapp/SpawnScreen.kt @@ -70,9 +70,10 @@ fun SpawnScreen( // one leaves a filled-in form worth keeping, and that one leaves // nothing to fill in. var spawnError by remember { mutableStateOf(null) } - // Downloaded models, for a llama provider to choose between. Fetched - // beside the setups but kept separate: a Claude session needs none, so - // failing to list them must not stop the screen rendering. + // The models on the *chosen machine*, for a llama provider to choose between. Kept separate + // from the setups: a Claude session needs none, so failing to list them must not stop the + // screen rendering. Refetched when the machine changes, because a model is a file on one + // machine -- see [fetchSetupModels]. var models by remember { mutableStateOf>(emptyList()) } var modelKey by remember { mutableStateOf(null) } var contextSize by remember { mutableStateOf("") } @@ -89,9 +90,6 @@ fun SpawnScreen( } catch (e: ApiException) { LoadState.failed(e) } - models = - runCatching { withContext(Dispatchers.IO) { fetchModels(settings).local } } - .getOrDefault(emptyList()) } Column(Modifier.fillMaxSize().verticalScroll(rememberScrollState()).padding(16.dp)) { @@ -122,6 +120,17 @@ fun SpawnScreen( is LoadState.Loaded -> state.value } val setup = setups.firstOrNull { it.name == setupName } + // Whichever machine is chosen now, asked again when that changes. The old machine's list + // is dropped first rather than left on screen: a file name from another machine looks + // exactly like one from this one. + LaunchedEffect(setup?.id) { + models = emptyList() + modelKey = null + val id = setup?.id ?: return@LaunchedEffect + models = + runCatching { withContext(Dispatchers.IO) { fetchSetupModels(settings, id) } } + .getOrDefault(emptyList()) + } val current = setup?.providers?.firstOrNull { it.name == providerName } // Only the Claude CLI has models, a working directory and // permission modes; keying the extra fields on the kind rather @@ -183,13 +192,14 @@ fun SpawnScreen( ) if (isLlama) { - // A llama session names one of the models this backend has - // downloaded, so the choice is that list rather than free - // text -- there is nothing sensible to type here, and a name - // that is not on disk is a session that cannot start. + // A llama session names one of the models on the machine it will run on, so the + // choice is that list rather than free text -- there is nothing sensible to type + // here, and a name that is not on that machine's disk is a session that cannot + // start. if (models.isEmpty()) { Text( - "No models downloaded yet. Get one from the Models screen first.", + "No models on ${setup?.name ?: "this machine"}. The Models screen downloads " + + "to the backend; another machine needs the file put there itself.", style = MaterialTheme.typography.bodyMedium, color = MaterialTheme.colorScheme.onSurfaceVariant, ) diff --git a/server/Cargo.lock b/server/Cargo.lock index ad173ae..a6d5600 100644 --- a/server/Cargo.lock +++ b/server/Cargo.lock @@ -35,6 +35,7 @@ dependencies = [ "sha2", "tempfile", "thiserror", + "time", "tokio", "tokio-stream", "tower", diff --git a/server/Cargo.toml b/server/Cargo.toml index 6c941ef..b993455 100644 --- a/server/Cargo.toml +++ b/server/Cargo.toml @@ -47,6 +47,12 @@ ureq = { version = "3", features = ["json"] } # both in the graph rustls refuses to auto-select one. rustls = "0.23" libc = "0.2.189" +# One ISO-8601 timestamp: the reset time on the invented rate-limit window +# an echo session's `/usage` puts up. Already in the tree behind the +# certificate machinery, so this is a direct name for what is compiled +# anyway rather than a new crate -- and the alternative was hand-rolling a +# civil-from-days conversion to print one line. +time = { version = "0.3", features = ["formatting"] } [dev-dependencies] tempfile = "3" diff --git a/server/src/config.rs b/server/src/config.rs index 9760d8f..81b26a7 100644 --- a/server/src/config.rs +++ b/server/src/config.rs @@ -106,6 +106,19 @@ pub struct SshConfig { /// Extra `-o` settings, each written as `Key=value`. #[serde(default, skip_serializing_if = "Vec::is_empty")] pub options: Vec, + /// Where this machine keeps the GGUF models it can serve, absent for + /// the same default this backend uses (`~/.local/share/ai-app/models` + /// -- `$XDG_DATA_HOME` is not read on the far side, since it is this + /// machine's environment that would answer). A `~` prefix is the + /// remote home. + /// + /// Here rather than on the provider because it is a fact about the + /// machine, and because a machine reached over ssh is where the model + /// has to be: a llama.cpp session serves the file from the machine + /// that runs `llama-server`, and this backend's own downloads are on + /// whichever machine that is only when they are the same one. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub models_dir: Option, /// Where a file attached from the phone is put on this machine so the /// session can read it. Absent means the session's own working /// directory, or the login home for a session that has none. A `~` @@ -167,6 +180,38 @@ impl DriverKind { } } + /// Which paid service meters a session of this kind, and `None` for + /// one that costs nothing. + /// + /// The rate-limit bars answer a question about an *account*, and what + /// decides which account -- if any -- is the provider a session runs, + /// not the machine it runs on. Those were the same thing only for as + /// long as a machine ran one kind of session: an echo session on a + /// laptop that also has the Claude CLI was drawn with that CLI's + /// five-hour window under its header, reporting a quota it cannot + /// spend and could not run down. A llama.cpp session is the same + /// story with the model on the far side. + /// + /// [`DriverKind::Echo`] names a meter of its own, which exists only + /// when a test has asked for one (`/usage` in `session::echo`). That + /// is what makes the bar's states -- a number, a machine nobody + /// logged into, one that could not be reached -- reachable without an + /// account and without spending a turn on somebody else's. With no + /// fixture set there is no snapshot for it, which the phone draws as + /// nothing at all. + /// + /// The string is a [`crate::usage::UsageProvider::name`], and it is + /// what pairs a session with one of the snapshots `GET /usage` + /// returns; the two lists have to agree, so `usage::providers_for` + /// reads this rather than matching on kinds a second time. + pub fn usage_provider(self) -> Option<&'static str> { + match self { + Self::ClaudeCli => Some(crate::usage::CLAUDE), + Self::Echo => Some(crate::usage::ECHO), + Self::LlamaCpp => None, + } + } + /// Whether the conversation exists outside this app, so that deleting /// the session here does not end it. /// @@ -442,6 +487,7 @@ mod tests { port: Some(2222), identity_file: None, options: Vec::new(), + models_dir: None, attachments_dir: None, }), providers: vec![ProviderConfig { diff --git a/server/src/main.rs b/server/src/main.rs index f85d4f5..5c0a586 100644 --- a/server/src/main.rs +++ b/server/src/main.rs @@ -284,7 +284,9 @@ async fn main() -> Result<()> { // -- so a machine added from the phone reports its limits without a // restart, and the backend's own account stops standing in for every // machine's. - let monitor = Arc::new(usage::UsageMonitor::new()); + // The fixture is the manager's, because that is where the `/usage` + // command that sets it is typed; the monitor is what serves it. + let monitor = Arc::new(usage::UsageMonitor::new(manager.usage_fixture())); // The bearer-token middleware wraps the entire router -- routes and // fallback alike -- here and only here, so a new route can't forget diff --git a/server/src/models.rs b/server/src/models.rs index 6f3ae7a..5434bd2 100644 --- a/server/src/models.rs +++ b/server/src/models.rs @@ -35,6 +35,8 @@ use serde::Serialize; use wg_app_link::private; +use crate::session::transport::{Launch, Transport}; + /// Identifies this client to HuggingFace. They ask for one, and a request /// without it is more likely to be rate-limited. const USER_AGENT: &str = concat!("ai-server/", env!("CARGO_PKG_VERSION")); @@ -551,6 +553,83 @@ fn collect(root: &Path, dir: &Path, found: &mut Vec) { } } +/// Where a machine reached over ssh keeps its models, when its setup does +/// not say. +/// +/// The same place this backend puts its own downloads, written out rather +/// than derived: `$XDG_DATA_HOME` here describes *this* machine's +/// environment, and the far machine's is the far machine's business. A +/// setup whose models are elsewhere says so (`SshConfig::models_dir`). +const FAR_MODELS_DIR: &str = "~/.local/share/ai-app/models"; + +/// Which directory holds the models on the machine `transport` reaches. +/// +/// One answer, because two things ask: the list a spawn screen offers, +/// and the path a session hands `llama-server`. A machine that listed one +/// directory and served from another would offer models that then failed +/// to load, which reads as the model being broken. +pub fn dir_on(transport: &Transport, local: &Path) -> String { + match transport { + Transport::Here => local.to_string_lossy().into_owned(), + Transport::Ssh { ssh, .. } => ssh + .models_dir + .as_ref() + .map_or(FAR_MODELS_DIR.to_string(), |dir| { + dir.to_string_lossy().into_owned() + }), + } +} + +/// Every GGUF on the machine a setup names, which is the machine that +/// would have to serve it. +/// +/// The local half of this is [`ModelStore::list`], reading the same shape +/// off this machine's disk; a caller picks by transport, since a setup +/// with no ssh *is* this machine and asking a shell about it would be a +/// slower way to the same answer. What must not happen is offering this +/// backend's downloads for a session on another machine: the file has to +/// be where `llama-server` runs, and a list that says otherwise is a +/// claim about the wrong filesystem. +/// +/// `dir` is that machine's models directory, `~` included -- expanded on +/// the far side, which is the only place that knows what it is. A +/// directory that is not there is an empty list rather than a failure: a +/// machine that has never had a model put on it is an ordinary state, and +/// the same one as a machine whose directory exists and is empty. +pub async fn on_machine(transport: &Transport, dir: &str) -> Result> { + let script = "p=$1; case $p in \"~\") p=$HOME;; \"~/\"*) p=$HOME/${p#\"~/\"};; esac; \ + [ -d \"$p\" ] || exit 0; \ + find \"$p\" -type f -name '*.gguf' -printf '%s\\t%P\\0'"; + let launch = Launch::new( + "sh", + vec![ + "-c".to_string(), + script.to_string(), + "sh".to_string(), + dir.to_string(), + ], + None, + ); + let out = transport.capture(&launch).await?; + let mut found: Vec = out + .split('\0') + .filter(|record| !record.is_empty()) + // Two fields, and the name last, so a `\t` in a filename survives. + .filter_map(|record| record.split_once('\t')) + .filter_map(|(bytes, key)| { + let (repo, file) = key.rsplit_once('/')?; + Some(LocalModel { + key: key.to_string(), + repo: repo.to_string(), + file: file.to_string(), + bytes: bytes.trim().parse().unwrap_or(0), + }) + }) + .collect(); + found.sort_by(|a, b| a.key.cmp(&b.key)); + Ok(found) +} + /// A model repository on HuggingFace, as the browse screen shows it. #[derive(Debug, Clone, Serialize)] #[serde(rename_all = "camelCase")] diff --git a/server/src/routes.rs b/server/src/routes.rs index 6cbdf15..a9986d9 100644 --- a/server/src/routes.rs +++ b/server/src/routes.rs @@ -7,6 +7,7 @@ //! POST /setups add {name, ssh?} -- providers are discovered //! POST /setups/probe dry run {ssh?}: what would be found there //! GET /setups/{id} one machine, for refetching after a change +//! GET /setups/{id}/models GGUFs on that machine, for a llama session //! GET /setups/{id}/dir?path=P entries of directory P, and P resolved //! GET /setups/{id}/file?path=P content of file P, or why not //! PUT /setups/{id}/file {path, content, ifSha256} -> new size/mtime/sha256 @@ -102,6 +103,7 @@ pub fn router(manager: Arc) -> Router { // `crate::files`. Under the setup rather than under a session // because a filesystem is a property of a machine; a session only // says where to start looking. + .route("/setups/{id}/models", get(setup_models)) .route("/setups/{id}/dir", get(list_dir).post(create_dir)) .route( "/setups/{id}/file", @@ -285,6 +287,9 @@ struct SshRequest { /// Where attached files land on that machine; see `SshConfig`. #[serde(default)] attachments_dir: Option, + /// Where that machine keeps its GGUF models; see `SshConfig`. + #[serde(default)] + models_dir: Option, } impl SshRequest { @@ -315,6 +320,14 @@ impl SshRequest { .map(str::trim) .filter(|dir| !dir.is_empty()) .map(std::path::PathBuf::from), + // The same rule, and for the same reason: this directory is + // on the other machine, so a `~` in it is that machine's home. + models_dir: self + .models_dir + .as_deref() + .map(str::trim) + .filter(|dir| !dir.is_empty()) + .map(std::path::PathBuf::from), }) } } @@ -499,6 +512,27 @@ struct PathQuery { path: String, } +/// The models **that machine** has, which is the list a llama.cpp session +/// on it can choose from. +/// +/// Not `GET /models`, which is this backend's own downloads: those are on +/// the machine a session runs on only when they are the same machine. A +/// spawn screen offering this backend's list for a remote setup would be +/// naming files that are not there, and the session would fail at the +/// point of loading rather than at the point of choosing. +async fn setup_models( + State(manager): State>, + UrlPath(id): UrlPath, +) -> Result>, ApiError> { + let setup = setup_by_id(&manager, &id)?; + let transport = crate::session::transport::Transport::for_setup(&setup); + let dir = crate::models::dir_on(&transport, manager.models_dir()); + crate::models::on_machine(&transport, &dir) + .await + .map(axum::Json) + .map_err(from_machine) +} + /// What is in a directory, and what that directory resolved to. async fn list_dir( State(manager): State>, diff --git a/server/src/session/echo.rs b/server/src/session/echo.rs index e731b99..8e8189b 100644 --- a/server/src/session/echo.rs +++ b/server/src/session/echo.rs @@ -28,6 +28,13 @@ //! - `/error [text]` -- a failure, which is otherwise awkward to cause. //! - `/peer [text]` -- a message from another agent, which otherwise takes //! two live sessions and one of them deciding to write. +//! - `/usage [what]` -- puts up an invented rate-limit answer, or takes +//! it away again (`/usage off`). An echo session meters nothing, so it +//! draws no usage bar at all until this is set; what it exists for is +//! the states the bar can be in, which otherwise cost real quota to +//! reach. `/usage 42`, `/usage 95 20`, `/usage 42 never`, +//! `/usage notloggedin`, `/usage unreachable`, `/usage failed`. The +//! vocabulary is `usage::Fixture`'s, which is where the states live. //! - `/compact` -- a compaction, start to finish. Typed rather than //! pressed, because the real dialects take it as a typed command too and //! the phone no longer has a button for it. @@ -106,6 +113,11 @@ pub struct EchoDriver { /// way AskUserQuestion does, and the turn resumes when the last of /// them is answered rather than the first. pending_questions: Mutex>, + /// The invented rate-limit answer `/usage` sets, shared with the + /// usage monitor that serves it. An echo session meters nothing, so + /// this is unset until a test asks for something -- see + /// [`crate::usage::Fixture`]. + usage: crate::usage::Fixture, /// A pretend context, so the status row has something that behaves the /// way a real one does: it grows with each turn, drops to what the /// compaction says it recovered, and a clear leaves it unmeasured. The @@ -345,6 +357,29 @@ impl EchoDriver { return; } + // Answered here rather than in the turn below, because it is not + // a turn: nothing is generated, and what is being exercised is + // the *other* screens -- the bar under the header, the button + // beside it and the dialog it opens, all of which read the usage + // route rather than this transcript. + if let Some(rest) = text.strip_prefix("/usage") { + if announce { + self.emit(Event::MessageTaken { + id: None, + text: text.clone(), + attachments, + }); + } + let said = self.usage.command(rest); + self.emit(Event::AssistantText { + delta: format!("{said}\n"), + }); + self.emit(Event::Status { + state: SessionStatus::Idle, + }); + return; + } + // The same word the real CLI takes, so a phone drives both the same // way. `Driver::compact` is what the manager's own route calls; // this is the typed path onto it. @@ -651,7 +686,7 @@ impl EchoDriver { }); } - pub fn new(sink: EventSink, session_dir: PathBuf) -> Self { + pub fn new(sink: EventSink, session_dir: PathBuf, usage: crate::usage::Fixture) -> Self { let driver = Self { sink, pending_questions: Mutex::new(Vec::new()), @@ -659,6 +694,7 @@ impl EchoDriver { busy: Arc::new(AtomicBool::new(false)), queued: Arc::new(Mutex::new(Vec::new())), session_dir, + usage, }; driver.emit(Event::Status { state: SessionStatus::Idle, diff --git a/server/src/session/llama.rs b/server/src/session/llama.rs index dca5eb3..0e86154 100644 --- a/server/src/session/llama.rs +++ b/server/src/session/llama.rs @@ -7,10 +7,24 @@ //! //! **It is spawned but not spoken to over stdio.** The process is started //! through the same [`Transport`] as any other, and then reached over -//! HTTP on a loopback port. That is the case the transport's doc comment -//! flags: a remote llama-server would need its port forwarded as well as -//! its command wrapped, which is not built, so a session on an ssh host -//! is refused rather than silently talking to the wrong machine. +//! HTTP on a loopback port. That is the second half of what a transport +//! is -- "run this" plus "reach this port" -- and it is what lets a +//! session run on another machine: [`Transport::reserve_port`] hands back +//! a port the server binds *there* and a port that reaches it *here*, and +//! the ssh connection carrying the command carries the tunnel between +//! them. The far `llama-server` binds loopback only, so a model is never +//! served to that machine's network. +//! +//! **The model file is the far machine's, not this one's.** A session +//! serves a GGUF from the machine that runs `llama-server`, so a remote +//! setup names its own models directory (`SshConfig::models_dir`, +//! defaulting to the same place this backend keeps its own downloads). +//! What this backend has downloaded is on that machine only when they are +//! the same machine -- so the file is looked for *there*, and a session +//! that names a model the machine does not have says so instead of +//! starting a server that will never load one. Downloading to another +//! machine is not built; the model gets there however anything else +//! gets there. //! //! **The server is stateless between requests**, so the whole //! conversation goes with every one. It is rebuilt from the session's @@ -85,16 +99,10 @@ impl LlamaDriver { session_dir: &Path, sink: EventSink, ) -> Result { - if !matches!(transport, Transport::Here) { - bail!( - "llama.cpp sessions can only run on this machine for now: the model is served \ - over HTTP, and forwarding that port to another host isn't built yet." - ); - } let model = meta.model.as_deref().context( "a llama.cpp session needs a model -- one of the downloaded ones, by its key", )?; - let path = model_path(models_dir, model)?; + let path = model_on(transport, models_dir, model)?; // Already loaded and still running: keep talking to it. The // health poll below is what confirms it is really answering, so @@ -120,14 +128,21 @@ impl LlamaDriver { )); } - let port = free_port().context("finding a port for llama-server")?; + // Where it listens on its own machine, and where that is reached + // from here -- the same number when that machine is this one. + let forward = transport + .reserve_port() + .context("finding a port for llama-server")?; let mut args: Vec = vec![ "-m".into(), - path.to_string_lossy().into_owned(), + path.clone(), + // Loopback there, whichever machine there is: what reaches it + // from outside that machine is the ssh tunnel and nothing + // else. "--host".into(), "127.0.0.1".into(), "--port".into(), - port.to_string(), + forward.there.to_string(), ]; // Settings that belong to the server because they decide how the // model is loaded; the sampling ones ride on each request instead, @@ -144,7 +159,7 @@ impl LlamaDriver { } let program = provider.command.as_deref().unwrap_or("llama-server"); - let launch = Launch::new(program, args, meta.cwd.as_deref()); + let launch = Launch::new(program, args, meta.cwd.as_deref()).reaching(forward); // Its output goes to files, not pipes. Not only so the process can // outlive this server: nothing ever read those pipes, so a chatty // llama-server filled the 64 KB buffer and blocked mid-load with @@ -161,8 +176,12 @@ impl LlamaDriver { .id() .context("llama-server exited before it could be recorded")?; tracing::info!( - "session {} running {program} for {model} on 127.0.0.1:{port} as pid {pid}", - meta.id + "session {} running {program} for {model} {} on 127.0.0.1:{} there, \ + reached at 127.0.0.1:{} here, as pid {pid}", + meta.id, + transport.describe(), + forward.there, + forward.here, ); // Reaped so it does not become a zombie while this server is still // its parent; the health poll and the record are what actually say @@ -173,12 +192,18 @@ impl LlamaDriver { let _ = child.wait().await; }); - let record = process::Record::of(pid, process::Detail::Http { port }) + // The *near* port, because that is the one anything reaching this + // server has to dial -- including a later run of this backend, + // which adopts the record without knowing which machine the server + // is on. For a remote session the recorded pid is the ssh + // client's, which is the process this machine owns and which holds + // the tunnel open for exactly as long as the far server lives. + let record = process::Record::of(pid, process::Detail::Http { port: forward.here }) .context("llama-server was gone before its start time could be read")?; process::write(session_dir, &record); Ok(Self::attached( - format!("http://127.0.0.1:{port}"), + format!("http://127.0.0.1:{}", forward.here), meta, model, transcript, @@ -212,7 +237,7 @@ impl LlamaDriver { let endpoint = endpoint.clone(); let model = model.to_string(); let session_dir = session_dir.to_path_buf(); - std::thread::spawn(move || match wait_until_ready(&endpoint) { + std::thread::spawn(move || match wait_until_ready(&endpoint, &session_dir) { Ok(()) => { tracing::info!("{model} loaded and answering at {endpoint}"); let _ = sink.send(Event::Status { @@ -518,19 +543,76 @@ fn model_path(models_dir: &Path, key: &str) -> Result { Ok(path) } -/// An unused loopback port, by asking the OS for one and letting it go. +/// The model file's path **on the machine that will serve it**, confirmed +/// to be there. /// -/// Racy in principle: something else could take it between here and -/// llama-server binding. In practice nothing on this machine is hunting -/// for ports, and the alternative -- parsing the port back out of the -/// server's log -- couples us to its output format for no real gain. -fn free_port() -> Result { - let listener = std::net::TcpListener::bind("127.0.0.1:0")?; - Ok(listener.local_addr()?.port()) +/// Local and remote answer the same question and it has to be asked of +/// two different filesystems, which is why this is one function rather +/// than a check beside the local path and hope for the other case. The +/// remote answer is measured for the same reason the local one is: a +/// missing file otherwise becomes a `llama-server` that starts, fails to +/// load, and reports as a session that never became ready -- which reads +/// as the machine being slow. +/// +/// One blocking round trip on a remote spawn, which is the same cost the +/// spawn is already paying to start ssh. The alternative is a path built +/// here from a `~` this machine cannot expand. +fn model_on(transport: &Transport, models_dir: &Path, key: &str) -> Result { + let Transport::Ssh { name, .. } = transport else { + return Ok(model_path(models_dir, key)?.to_string_lossy().into_owned()); + }; + // The same directory the spawn screen listed for this machine, and + // for the same reason it is one function: a list from one place and a + // load from another is a model that appears and then fails. + let dir = crate::models::dir_on(transport, models_dir); + // Checked here rather than in the script: `..` in a key would walk + // out of the models directory on a machine this server can start + // processes on, and the phone is where the key comes from. + for part in key.split('/') { + if part.is_empty() || part == "." || part == ".." { + bail!("\"{key}\" is not a model key this can resolve"); + } + } + let path = format!("{}/{key}", dir.trim_end_matches('/')); + // `$HOME` on the far side, which is the only machine that knows what + // it is -- and the resolved path is printed back so the launch below + // hands `llama-server` something absolute. + // + // "the file is not there" is answered rather than failed, because the + // two are different things to a reader and only one of them is a + // fault: a machine that could not be asked at all has to say so in + // its own words, and it would otherwise arrive as this same sentence + // about a missing model. + let script = "p=$1; case $p in \"~\") p=$HOME;; \"~/\"*) p=$HOME/${p#\"~/\"};; esac; \ + [ -f \"$p\" ] && printf 'at\\t%s\\n' \"$p\" || printf 'missing\\n'" + .to_string(); + let launch = Launch::new( + "sh", + vec!["-c".to_string(), script, "sh".to_string(), path.clone()], + None, + ); + let answer = transport + .capture_blocking(&launch) + .with_context(|| format!("couldn't ask {name} where its models are"))?; + match answer.trim().split_once('\t') { + Some(("at", resolved)) => Ok(resolved.to_string()), + _ => bail!( + "{name} has no model at {path}. A llama.cpp session serves the file from the \ + machine it runs on, so the model has to be on {name} -- what this backend has \ + downloaded is somewhere else." + ), + } } /// Polls until the server says it is ready, or gives up. -fn wait_until_ready(endpoint: &str) -> Result<()> { +/// +/// Watches the process as well as the port, because the two failures need +/// different words and one of them is common: a model that will not load, +/// a port already taken on the far machine, a `llama-server` too old for +/// a flag. All of those exit within a second and none of them will ever +/// answer `/health`, so waiting out the timeout turns a server that said +/// exactly what was wrong into "gave up after 300s". +fn wait_until_ready(endpoint: &str, session_dir: &Path) -> Result<()> { let deadline = std::time::Instant::now() + READY_TIMEOUT; let url = format!("{endpoint}/health"); loop { @@ -539,13 +621,48 @@ fn wait_until_ready(endpoint: &str) -> Result<()> { { return Ok(()); } + // `None` is the session having been stopped or deleted while this + // waited, which is nobody's fault and still not worth waiting on. + match process::recorded(session_dir) { + Some((_, process::Liveness::Alive | process::Liveness::Unknown)) => {} + Some((_, process::Liveness::Dead)) | None => { + bail!("it exited before it answered.{}", log_tail(session_dir)); + } + } if std::time::Instant::now() > deadline { - bail!("gave up after {}s", READY_TIMEOUT.as_secs()); + bail!( + "gave up after {}s.{}", + READY_TIMEOUT.as_secs(), + log_tail(session_dir) + ); } std::thread::sleep(std::time::Duration::from_millis(250)); } } +/// The end of `llama-server`'s own log, for a failure message. +/// +/// Its account of what went wrong is the useful half -- "failed to load +/// model", "bind: Address already in use" -- and on a remote session it +/// is the only half, since nobody reading the phone can open a file on +/// that machine. Bounded, because this ends up in an event a phone draws. +fn log_tail(session_dir: &Path) -> String { + let Ok(text) = std::fs::read_to_string(session_dir.join(SERVER_LOG)) else { + return String::new(); + }; + let tail: Vec<&str> = text.lines().rev().take(LOG_TAIL_LINES).collect(); + if tail.is_empty() { + return String::new(); + } + format!( + " It last said: {}", + tail.into_iter().rev().collect::>().join(" / ") + ) +} + +/// How much of that log to carry into a message somebody reads on a phone. +const LOG_TAIL_LINES: usize = 6; + /// One streamed completion: posts the conversation, emits each delta as it /// arrives. Emits rather than returns: the transcript those events land /// in is what the next turn reads back, so there is nothing to hand up. diff --git a/server/src/session/mod.rs b/server/src/session/mod.rs index f21241e..10d21ed 100644 --- a/server/src/session/mod.rs +++ b/server/src/session/mod.rs @@ -163,6 +163,17 @@ pub struct SessionInfo { /// answers and only one of them stays true. #[serde(skip_serializing_if = "Option::is_none")] pub max_image_edge: Option, + /// Which of `GET /usage`'s snapshots reports on this session, and + /// absent where nothing meters it -- see + /// [`DriverKind::usage_provider`]. + /// + /// Reported for the same reason `keeps_own_transcript` is: it is a + /// fact about the provider's *kind*, and the phone has only its name. + /// Pairing by machine alone was the bug it exists to fix -- one + /// machine runs echo and the Claude CLI, so every echo session drew + /// the CLI's five-hour window as if it were its own. + #[serde(skip_serializing_if = "Option::is_none")] + pub usage_provider: Option<&'static str>, /// Whether this session announces itself -- reported for the same /// reason `permission_mode` is: a switch that guesses its own position /// is how you turn something off while believing you are reading it. @@ -549,6 +560,7 @@ impl LiveSession { context_tokens: *self.shared.context_tokens.lock().unwrap(), notify: *self.shared.notify.lock().unwrap(), max_image_edge: kind.and_then(DriverKind::max_image_edge), + usage_provider: kind.and_then(DriverKind::usage_provider), imported, keeps_own_transcript: kind.is_some_and(DriverKind::keeps_own_transcript), cwd: cwd.map(Path::to_path_buf), @@ -585,6 +597,11 @@ pub struct SessionManager { /// [`SessionManager::marking_new_sessions_throwaway`] and /// [`SessionConfig::throwaway`]. spawn_throwaway: bool, + /// The invented rate-limit answer an echo session's `/usage` sets, + /// shared with the usage monitor that serves it. Held here because + /// every echo driver this manager builds is handed a clone -- see + /// [`SessionManager::reporting_usage_fixture`]. + usage_fixture: crate::usage::Fixture, inner: RwLock, } @@ -603,6 +620,11 @@ impl SessionManager { wg_app_link::private::create_dir(&data_dir)?; let (notifications, _) = broadcast::channel(NOTIFICATION_BUFFER); + // Made here rather than passed in, and handed *out* to the usage + // monitor by whoever wires the two together: every echo driver + // this manager builds gets a clone, including the ones built + // below, so it has to exist before the first session does. + let usage_fixture = crate::usage::Fixture::new(); let mut live = HashMap::new(); for meta in &config.sessions { // One unlaunchable session -- a corrupt transcript, an @@ -614,8 +636,11 @@ impl SessionManager { meta.clone(), &setup, &provider, - &data_dir, - &models_dir, + Env { + data_dir: &data_dir, + models_dir: &models_dir, + usage: &usage_fixture, + }, notifications.clone(), // Nothing is started here. See `Launching`: a restart // picks up the processes that are still running and @@ -638,11 +663,41 @@ impl SessionManager { notifications, pending: Arc::new(pending::Registry::default()), spawn_throwaway: false, + usage_fixture, inner: RwLock::new(Inner { config, live }), }; Ok(manager) } + /// Where this backend's own model downloads live. The machine a + /// session runs on may keep its elsewhere -- see `models::dir_on`. + pub fn models_dir(&self) -> &Path { + &self.models_dir + } + + /// What this manager lends a session it launches. Borrowed from the + /// manager rather than cloned, so there is one answer to where things + /// are kept. + fn env(&self) -> Env<'_> { + Env { + data_dir: &self.data_dir, + models_dir: &self.models_dir, + usage: &self.usage_fixture, + } + } + + /// The invented rate-limit answer this manager's echo sessions set + /// with `/usage`, for the usage monitor to serve. + /// + /// Handed out rather than taken in because the drivers built inside + /// the constructor need it, and because the direction is the one the + /// layering allows: `usage` sits below the session layer, so a + /// session can hold one of its types while it holds nothing of a + /// session's. + pub fn usage_fixture(&self) -> crate::usage::Fixture { + self.usage_fixture.clone() + } + /// Marks every session spawned from here on as one whose process is /// stopped when this server exits -- see [`SessionConfig::throwaway`] /// and [`SessionManager::stop_throwaway_sessions`]. @@ -1035,6 +1090,8 @@ impl SessionManager { context_tokens: None, max_image_edge: kind_of(&inner.config, &meta.setup, &meta.provider) .and_then(DriverKind::max_image_edge), + usage_provider: kind_of(&inner.config, &meta.setup, &meta.provider) + .and_then(DriverKind::usage_provider), notify: meta.notify, imported: import::read_cursor(&self.data_dir.join(&meta.id)).is_some(), keeps_own_transcript: keeps_own_transcript( @@ -1162,8 +1219,7 @@ impl SessionManager { meta.clone(), &setup, &provider, - &self.data_dir, - &self.models_dir, + self.env(), self.notifications.clone(), Launching::Asked(seed), )?; @@ -1605,7 +1661,7 @@ impl SessionManager { &meta, &setup, &provider, - &self.models_dir, + self.env(), session.dir(), session.transcript_path(), &session.sink, @@ -1619,8 +1675,7 @@ impl SessionManager { meta, &setup, &provider, - &self.data_dir, - &self.models_dir, + self.env(), self.notifications.clone(), Launching::Asked(None), )?; @@ -2020,6 +2075,19 @@ enum Launching { Restart, } +/// What the server around a session lends it: where sessions and models +/// are kept, and the usage fixture an echo session's `/usage` sets. +/// +/// One parameter rather than three because they travel together through +/// every launch path and none of them is a fact about the session -- +/// they are this server's belongings, handed down. +#[derive(Clone, Copy)] +struct Env<'a> { + data_dir: &'a Path, + models_dir: &'a Path, + usage: &'a crate::usage::Fixture, +} + /// Creates the session directory, opens its transcript (continuing the /// sequence numbering if one exists), settles what the session is doing, /// and spawns the event pump -- with a driver behind it where there is a @@ -2028,12 +2096,11 @@ fn launch( meta: SessionConfig, setup: &SetupConfig, provider: &ProviderConfig, - data_dir: &Path, - models_dir: &Path, + env: Env<'_>, notifications: broadcast::Sender, why: Launching, ) -> Result> { - let dir = data_dir.join(&meta.id); + let dir = env.data_dir.join(&meta.id); wg_app_link::private::create_dir(&dir)?; let transcript_path = dir.join("transcript.jsonl"); let mut transcript = Transcript::open(&transcript_path)?; @@ -2163,17 +2230,7 @@ fn launch( let driver = Arc::new(Mutex::new( driving - .then(|| { - make_driver( - &meta, - setup, - provider, - models_dir, - &dir, - &transcript_path, - &sink, - ) - }) + .then(|| make_driver(&meta, setup, provider, env, &dir, &transcript_path, &sink)) .transpose()?, )); @@ -2216,18 +2273,22 @@ fn make_driver( meta: &SessionConfig, setup: &SetupConfig, provider: &ProviderConfig, - models_dir: &Path, + env: Env<'_>, dir: &Path, transcript_path: &Path, sink: &EventSink, ) -> Result> { Ok(match provider.kind { - DriverKind::Echo => Arc::new(EchoDriver::new(sink.clone(), dir.to_path_buf())), + DriverKind::Echo => Arc::new(EchoDriver::new( + sink.clone(), + dir.to_path_buf(), + env.usage.clone(), + )), DriverKind::LlamaCpp => Arc::new(LlamaDriver::launch( meta, provider, &Transport::for_setup(setup), - models_dir, + env.models_dir, transcript_path, dir, sink.clone(), @@ -2584,6 +2645,7 @@ mod tests { driver: Arc::new(Mutex::new(Some(Arc::new(EchoDriver::new( sink.clone(), dir.path().to_path_buf(), + crate::usage::Fixture::new(), ))))), sink, waiting: Mutex::new(VecDeque::new()), diff --git a/server/src/session/transport.rs b/server/src/session/transport.rs index cf2262b..aad4f4e 100644 --- a/server/src/session/transport.rs +++ b/server/src/session/transport.rs @@ -13,11 +13,14 @@ //! this module decides *which* transport, that one knows what a correct //! ssh invocation is. //! -//! Known second operation, not built because nothing needs it yet: a -//! managed `llama-server` is spawned as a process but then spoken to over -//! HTTP, so a remote one needs a forwarded port (`ssh -L`) as well. A -//! transport is eventually "run this" plus "reach this port", where the -//! second is a no-op locally. See PLAN.md's SSH section. +//! A transport is therefore two operations rather than one: **run this**, +//! and **reach this port**. The second is what a managed `llama-server` +//! needs -- it is spawned as a process and then spoken to over HTTP -- and +//! it is a no-op locally, where the port a program binds is already a port +//! this machine can dial. Over ssh it is an `-L` tunnel carried by the +//! same connection that runs the command, so the model server binds +//! loopback on the far machine and is never exposed to its network. See +//! [`Transport::reserve_port`] and PLAN.md's SSH section. use std::path::{Path, PathBuf}; use std::process::Stdio; @@ -26,16 +29,28 @@ use anyhow::{Context, Result}; use tokio::process::Child; use crate::config::SshConfig; +pub use crate::ssh::Forward; /// What a driver needs run in order to exist as a process. /// -/// Deliberately just the three things every transport can carry. Anything -/// a particular machine needs -- a port, a key, extra ssh options -- is +/// Deliberately just what every transport can carry: the command, where +/// it runs, and a port the caller needs to reach. Anything a particular +/// machine needs -- a key, extra ssh options, which address to dial -- is /// the transport's own configuration, not something a driver states. pub struct Launch { pub program: String, pub args: Vec, pub cwd: Option, + /// A port this program will listen on, and the port that reaches it + /// from here -- see [`Transport::reserve_port`], which is the only + /// thing that should produce one. + /// + /// On the launch rather than in [`Transport::spawn`]'s signature + /// because it is part of what is being run: a caller that needs to + /// reach the process it is starting says so once, where it says + /// everything else about it, and every transport reads it the same + /// way. + pub forward: Option, } impl Launch { @@ -44,8 +59,16 @@ impl Launch { program: program.into(), args, cwd: cwd.map(Path::to_path_buf), + forward: None, } } + + /// Says that this program serves `forward.there`, and that the caller + /// will reach it at `forward.here`. + pub fn reaching(mut self, forward: Forward) -> Self { + self.forward = Some(forward); + self + } } /// How a launched process's standard streams are connected. @@ -113,6 +136,7 @@ impl Transport { &launch.program, &launch.args, launch.cwd.as_deref(), + launch.forward, )); match streams { Streams::Piped => { @@ -169,12 +193,15 @@ impl Transport { Self::Here => None, Self::Ssh { ssh, .. } => Some(ssh), }; - let output = - crate::ssh::command(host, &launch.program, &launch.args, launch.cwd.as_deref()) - .output() - .with_context(|| { - format!("couldn't run \"{}\" {}", launch.program, self.describe()) - })?; + let output = crate::ssh::command( + host, + &launch.program, + &launch.args, + launch.cwd.as_deref(), + launch.forward, + ) + .output() + .with_context(|| format!("couldn't run \"{}\" {}", launch.program, self.describe()))?; if !output.status.success() { let stderr = String::from_utf8_lossy(&output.stderr).trim().to_string(); anyhow::bail!(if stderr.is_empty() { @@ -237,6 +264,34 @@ impl Transport { }) } + /// Picks a port for a launched program to serve on, and the port that + /// reaches it from here. + /// + /// The "reach this port" half of what a transport is. Locally there is + /// one port and the OS chooses it, by binding and letting go -- racy + /// in principle, and nothing on this machine is hunting for ports. + /// + /// Over ssh the near end is chosen the same way and the far end is a + /// guess, because there is no portable way to ask a machine for a free + /// port that does not race with binding it anyway. It is taken from + /// [`FAR_PORTS`], below the range Linux hands out to outgoing + /// connections, so a collision means something else deliberately + /// listening there. That is not silent: the program fails to bind and + /// exits, and `session::llama` reports what its log said rather than + /// waiting out its readiness timeout. + pub fn reserve_port(&self) -> Result { + let listener = std::net::TcpListener::bind("127.0.0.1:0") + .context("asking this machine for a free port")?; + let here = listener.local_addr()?.port(); + Ok(match self { + Self::Here => Forward { there: here, here }, + Self::Ssh { .. } => Forward { + there: rand::random_range(FAR_PORTS), + here, + }, + }) + } + /// How to say where this runs, for a log line a person reads. pub fn describe(&self) -> String { match self { @@ -246,6 +301,11 @@ impl Transport { } } +/// Where a port on another machine is guessed from: high enough to be out +/// of the way of services, and below the 32768-60999 Linux hands out to +/// outgoing connections, which is where a guess would most often collide. +const FAR_PORTS: std::ops::Range = 20000..30000; + /// What a command is given on its standard input. /// /// Three cases rather than an `Option` because they are three diff --git a/server/src/setups.rs b/server/src/setups.rs index 0937bdb..24e3437 100644 --- a/server/src/setups.rs +++ b/server/src/setups.rs @@ -30,7 +30,10 @@ use crate::session::transport::{Launch, Transport}; /// what the phone shows and what a session stores. const PROBES: &[(&str, &str, DriverKind)] = &[ ("claude-cli", "claude", DriverKind::ClaudeCli), - ("local-llama", "llama-server", DriverKind::LlamaCpp), + // Named for the program rather than for where it runs: it runs + // wherever the setup is, and "local" was true only while a llama + // session could not be spawned on another machine. + ("llama-cpp", "llama-server", DriverKind::LlamaCpp), ]; /// Models offered for a discovered Claude CLI. A shortcut list for the diff --git a/server/src/ssh.rs b/server/src/ssh.rs index e47a511..f811227 100644 --- a/server/src/ssh.rs +++ b/server/src/ssh.rs @@ -15,6 +15,23 @@ use std::process::Command; use crate::config::SshConfig; +/// A port on the machine a command runs on, and the port that reaches it +/// from the backend. +/// +/// The second half of what a transport is (PLAN.md's SSH section): "run +/// this" plus "reach this port". Locally the two numbers are the same one +/// and nothing is forwarded; over ssh the connection carries an `-L` +/// tunnel, so a model server binds loopback on the far machine and is +/// never exposed to its network. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct Forward { + /// What the launched program should listen on, on its own machine. + pub there: u16, + /// What this machine connects to. The same number as `there` when the + /// program runs here. + pub here: u16, +} + /// Options forced onto every connection. `BatchMode` makes a missing key /// fail immediately with a readable message instead of hanging on a /// password prompt that nothing can answer; the keepalives turn a silently @@ -45,6 +62,7 @@ pub fn command( program: &str, args: &[String], cwd: Option<&Path>, + forward: Option, ) -> Command { let Some(ssh) = remote else { let mut command = Command::new(program); @@ -64,9 +82,42 @@ pub fn command( }; let mut command = Command::new("ssh"); - // -T: no pty. This carries JSONL, and a pty would rewrite it (echo, - // CRLF translation, ^C handling) into something the parser can't read. - command.arg("-T"); + if let Some(forward) = forward { + // A forwarded process is not spoken to over stdio, and that + // changes how it has to be shut down. Everything else here is a + // CLI reading its stdin, so killing the ssh client closes that + // stdin and the far process ends; a `llama-server` never reads + // its own, so the same kill left it running on the far machine + // holding the model in memory -- measured 2026-09-04, an orphan + // per stopped session. A pty is what makes sshd hang the far side + // up: when the connection goes, the master closes and the session + // takes SIGHUP. `-tt` because this client has no terminal of its + // own to inherit one from. + // + // The cost is that its log arrives through a line discipline + // (CRLF, and whatever the program does when it thinks it is on a + // terminal). Nothing parses that log, so it is a fair trade for a + // process that reliably goes away. + command.arg("-tt"); + // Loopback on both ends: the far side binds 127.0.0.1, so the + // port it serves is reachable only through this connection and + // never from that machine's network -- and the near end is bound + // to this host alone for the same reason. + command.args([ + "-L", + &format!("127.0.0.1:{}:127.0.0.1:{}", forward.here, forward.there), + ]); + // Without this a forward that cannot be set up is a warning on + // stderr and a session that runs anyway, answering nothing: the + // failure would arrive as "the model never became ready", which + // is the wrong thing to go looking at. + command.args(["-o", "ExitOnForwardFailure=yes"]); + } else { + // -T: no pty. This carries JSONL, and a pty would rewrite it + // (echo, CRLF translation, ^C handling) into something the parser + // can't read. + command.arg("-T"); + } for option in SSH_OPTIONS { command.args(["-o", option]); } @@ -200,6 +251,7 @@ mod tests { port: None, identity_file: None, options: vec![], + models_dir: None, attachments_dir: None, } } @@ -211,6 +263,7 @@ mod tests { "claude", &args(["-p", "--verbose"]), Some(Path::new("/tmp/x")), + None, ); assert_eq!(argv(&command), ["claude", "-p", "--verbose"]); assert_eq!(command.get_current_dir(), Some(Path::new("/tmp/x"))); @@ -223,6 +276,7 @@ mod tests { port: Some(2222), identity_file: Some("/home/me/.ssh/id_ai".into()), options: vec!["StrictHostKeyChecking=accept-new".to_string()], + models_dir: None, attachments_dir: None, }; let rendered = argv(&command( @@ -230,6 +284,7 @@ mod tests { "claude", &args(["-p", "--model", "haiku"]), Some(Path::new("/home/bob/work")), + None, )); assert_eq!(rendered[0], "ssh"); @@ -250,12 +305,60 @@ mod tests { #[test] fn a_remote_command_without_a_cwd_just_execs() { let ssh = bare_host(); - let rendered = argv(&command(Some(&ssh), "claude", &args(["-p"]), None)); + let rendered = argv(&command(Some(&ssh), "claude", &args(["-p"]), None, None)); assert_eq!(rendered.last().unwrap(), "exec 'claude' '-p'"); // No -i means no IdentitiesOnly: ~/.ssh/config decides instead. assert!(!rendered.contains(&"IdentitiesOnly=yes".to_string())); } + /// The second half of a transport: the connection that runs the + /// command also carries the port that reaches it. + /// + /// Both ends are pinned to loopback, which is the property that keeps + /// a model server off the far machine's network -- asserted here + /// rather than trusted, because dropping the addresses is a one-word + /// edit that still works on a machine nobody else can reach. + #[test] + fn a_forwarded_port_rides_the_same_connection_as_the_command() { + let ssh = bare_host(); + let rendered = argv(&command( + Some(&ssh), + "llama-server", + &args(["--port", "24242"]), + None, + Some(Forward { + there: 24242, + here: 41000, + }), + )); + let forward = rendered + .iter() + .position(|arg| arg == "-L") + .expect("a forward"); + assert_eq!(rendered[forward + 1], "127.0.0.1:41000:127.0.0.1:24242"); + assert!(rendered.contains(&"ExitOnForwardFailure=yes".to_string())); + // The half that is easy to lose: without a pty the far process + // outlives the connection, because nothing closes a stdin it + // never reads. + assert!(rendered.contains(&"-tt".to_string())); + assert!(!rendered.contains(&"-T".to_string())); + // Options come before the host, or ssh reads them as part of the + // remote command. + assert!(forward < rendered.len() - 2); + assert_eq!( + rendered.last().unwrap(), + "exec 'llama-server' '--port' '24242'" + ); + + // Nothing forwarded is nothing added: every other session is one + // of these, and an -L on it would bind a port for no reason. + let plain = argv(&command(Some(&ssh), "claude", &args(["-p"]), None, None)); + assert!(!plain.contains(&"-L".to_string())); + // And a session that *is* spoken to over stdio keeps its raw pipe. + assert!(plain.contains(&"-T".to_string())); + assert!(!plain.contains(&"-tt".to_string())); + } + /// The one character quoting must not swallow. /// /// A working directory typed as `~/repos/ai-app` was arriving as the @@ -300,7 +403,13 @@ mod tests { assert_eq!(expand_home(Path::new("/tmp/~/x")), Path::new("/tmp/~/x")); assert_eq!(expand_home(Path::new("~user/x")), Path::new("~user/x")); - let local = command(None, "claude", &args(["-p"]), Some(Path::new("~/work"))); + let local = command( + None, + "claude", + &args(["-p"]), + Some(Path::new("~/work")), + None, + ); assert_eq!(local.get_current_dir(), Some(home.join("work").as_path())); } @@ -324,7 +433,7 @@ mod tests { // that tries to close the quote and start a new command. let ssh = bare_host(); let evil = Path::new("/tmp/'; touch /tmp/pwned; '"); - let rendered = argv(&command(Some(&ssh), "claude", &[], Some(evil))); + let rendered = argv(&command(Some(&ssh), "claude", &[], Some(evil), None)); let script = rendered.last().unwrap(); assert_eq!( script, diff --git a/server/src/usage.rs b/server/src/usage.rs index 929f855..a63acc7 100644 --- a/server/src/usage.rs +++ b/server/src/usage.rs @@ -35,13 +35,13 @@ //! on it). use std::collections::HashMap; -use std::sync::Mutex; +use std::sync::{Arc, Mutex}; use std::time::{Duration, Instant}; use serde::Serialize; use serde_json::Value; -use crate::config::{DriverKind, SetupConfig}; +use crate::config::SetupConfig; use crate::session::transport::{Launch, Transport}; const USAGE_URL: &str = "https://api.anthropic.com/api/oauth/usage"; @@ -116,10 +116,30 @@ pub struct UsageSnapshot { pub fetched_at: f64, } +/// The name of each meter, said in one place because two lists have to +/// agree on it: [`UsageSnapshot::provider`], which is what `GET /usage` +/// labels a row with, and [`crate::config::DriverKind::usage_provider`], +/// which is how a session says which of those rows is about it. +pub const CLAUDE: &str = "claude"; +/// The invented one, for testing the screens that draw these -- see +/// [`Fixture`]. +pub const ECHO: &str = "echo"; + pub trait UsageProvider: Send + Sync { fn name(&self) -> &'static str; /// Blocking -- call off the async workers. fn fetch(&self) -> UsageSnapshot; + /// How long an answer from this one may be reused. + /// + /// A property of the provider rather than of the cache, because what + /// sets it is what asking costs: [`ClaudeUsage`] makes a network call + /// against an endpoint that rate-limits impatient callers, and the + /// fixture below reads a mutex. Caching the fixture for three minutes + /// would mean a test setting a number and watching the old one for + /// most of that, which reads exactly like the command not working. + fn poll_interval(&self) -> Duration { + MIN_POLL_INTERVAL + } } /// Reads the numbers behind Claude Code's `/usage` from one machine, using @@ -182,7 +202,7 @@ impl ClaudeUsage { impl UsageProvider for ClaudeUsage { fn name(&self) -> &'static str { - "claude" + CLAUDE } fn fetch(&self) -> UsageSnapshot { @@ -293,24 +313,260 @@ fn parse_windows(body: &Value) -> Vec { .collect() } +/// An invented answer, so the screens that draw these can be exercised +/// without an account. +/// +/// Every state the usage bar and the usage dialog can be in is otherwise +/// reachable only by spending somebody's quota or by breaking a machine: +/// a number near the top, a machine nobody has logged into, one that +/// cannot be reached, a window between blocks with no reset time. Those +/// are exactly the states worth looking at, and the ones nobody looks at +/// because arranging them costs real turns. An echo session sets this +/// with `/usage` (see `session::echo`), which is the same bargain the +/// rest of that driver makes: the fixture is invented, what is real is +/// the path it travels. +/// +/// Shared by the session layer, which writes it, and [`UsageMonitor`], +/// which reads it. Empty until something sets it, and an empty fixture +/// produces no snapshot at all -- an echo session meters nothing, and +/// nothing is what the phone should draw. +#[derive(Clone, Default)] +pub struct Fixture { + said: Arc>>, +} + +/// What a meter answered: which of the four states it is in, and whatever +/// windows go with it. Empty for every state but [`UsageState::Ok`]. +type Reported = (UsageState, Vec); + +/// How long the invented five-hour window has left, when nothing says. +const FIXTURE_MINUTES: i64 = 125; + +impl Fixture { + pub fn new() -> Self { + Self::default() + } + + fn is_set(&self) -> bool { + self.said.lock().unwrap().is_some() + } + + fn read(&self) -> Option { + self.said.lock().unwrap().clone() + } + + /// Acts on the words typed after `/usage`, and says what it did. + /// + /// The vocabulary lives here rather than in the echo driver because + /// these are this module's states: a driver spelling them out would + /// be a second place that has to learn about a fifth one. + pub fn command(&self, words: &str) -> String { + let mut words = words.split_whitespace(); + let Some(first) = words.next() else { + return match self.read() { + Some((state, windows)) => format!("usage fixture: {}", describe(&state, &windows)), + None => "usage fixture: unset, so this session meters nothing. \ + `/usage 42` puts up a five-hour window at 42%." + .to_string(), + }; + }; + let rest: Vec<&str> = words.collect(); + let detail = || { + if rest.is_empty() { + "set by /usage".to_string() + } else { + rest.join(" ") + } + }; + let (state, windows) = match first { + "off" | "none" | "clear" => { + *self.said.lock().unwrap() = None; + return "usage fixture cleared: this session meters nothing again".to_string(); + } + "notloggedin" | "logged-out" => (UsageState::NotLoggedIn, Vec::new()), + "unreachable" => (UsageState::Unreachable { detail: detail() }, Vec::new()), + "failed" => (UsageState::Failed { detail: detail() }, Vec::new()), + percent => match percent.parse::() { + Ok(percent) => ( + UsageState::Ok, + fixture_windows(percent.clamp(0.0, 100.0), rest.first().copied()), + ), + Err(_) => { + return format!( + "\"{percent}\" is not one of this fixture's answers. Say a percentage \ + (`/usage 42`, optionally with `90` minutes left, `never` for a window \ + between blocks, or `unreadable` for a reset time that cannot be read), \ + or one of `notloggedin`, `unreachable`, `failed`, `off`." + ); + } + }, + }; + let said = describe(&state, &windows); + *self.said.lock().unwrap() = Some((state, windows)); + format!("usage fixture set: {said}") + } +} + +/// The three windows Claude reports today, invented around one number. +/// +/// Three rather than one because the bar under a session header reads the +/// five-hour window and the dialog behind the button draws all of them, +/// and a fixture with one window leaves half the screen untested. The +/// weekly ones are derived from the same figure so that the worst of them +/// -- which is what colours the button -- is still the one asked for. +fn fixture_windows(percent: f64, reset: Option<&str>) -> Vec { + let resets_at = match reset { + // The state a real response is in between blocks: there is no + // window running, so there is nothing to reset. It is not a + // missing value, and the phone words it differently. + Some("never") | Some("none") => None, + // A timestamp that arrives and cannot be read, which is the one + // case that really is "we could not find out". + Some("unreadable") | Some("bad") => Some("whenever it feels like it".to_string()), + other => Some(reset_in( + other + .and_then(|word| word.parse().ok()) + .unwrap_or(FIXTURE_MINUTES), + )), + }; + vec![ + UsageWindow { + kind: "session".to_string(), + label: "5-hour window".to_string(), + percent, + resets_at: resets_at.clone(), + active: true, + }, + UsageWindow { + kind: "weekly_all".to_string(), + label: "Weekly (all models)".to_string(), + percent: percent / 2.0, + resets_at: resets_at.as_ref().map(|_| reset_in(FIXTURE_MINUTES * 40)), + active: false, + }, + UsageWindow { + kind: "weekly_scoped".to_string(), + label: "Weekly (Echo)".to_string(), + percent: percent / 4.0, + resets_at: resets_at.as_ref().map(|_| reset_in(FIXTURE_MINUTES * 40)), + active: false, + }, + ] +} + +/// `minutes` from now, in the format the real endpoint sends. +fn reset_in(minutes: i64) -> String { + let at = time::OffsetDateTime::now_utc() + time::Duration::minutes(minutes); + at.format(&time::format_description::well_known::Rfc3339) + // Formatting a timestamp cannot fail for any input this builds; + // saying so beats a fixture that silently has no reset time. + .unwrap_or_else(|_| "unformattable".to_string()) +} + +/// One line naming what a fixture is currently claiming, for the reply +/// the echo session writes back. +fn describe(state: &UsageState, windows: &[UsageWindow]) -> String { + match state { + UsageState::Ok => match windows.first() { + Some(window) => format!( + "{}% of the five-hour window, {}", + window.percent, + match &window.resets_at { + Some(at) => format!("resetting at {at}"), + None => "with no reset time (the between-blocks state)".to_string(), + } + ), + None => "no windows at all".to_string(), + }, + UsageState::NotLoggedIn => "nobody is logged in on this machine".to_string(), + UsageState::Unreachable { detail } => format!("machine unreachable ({detail})"), + UsageState::Failed { detail } => format!("the meter failed ({detail})"), + } +} + +/// The fixture, as a provider, so it travels the same route and the same +/// cache as a real meter rather than being spliced in at the screen. +struct EchoUsage { + setup: String, + setup_name: String, + fixture: Fixture, +} + +impl UsageProvider for EchoUsage { + fn name(&self) -> &'static str { + ECHO + } + + fn fetch(&self) -> UsageSnapshot { + let (state, windows) = self + .fixture + .read() + // Only ever built for a fixture that is set; a race with + // `/usage off` between the two reads lands here, and "the + // machine could not be asked" is the honest word for it. + .unwrap_or(( + UsageState::Unreachable { + detail: "the usage fixture was cleared".to_string(), + }, + Vec::new(), + )); + UsageSnapshot { + provider: self.name().to_string(), + setup: self.setup.clone(), + setup_name: self.setup_name.clone(), + state, + windows, + fetched_at: crate::session::now(), + } + } + + /// Read from memory, and set by somebody who is about to look at the + /// screen it changes. + fn poll_interval(&self) -> Duration { + Duration::ZERO + } +} + /// Which paid services a machine can be asked about. /// /// Derived from what the setup says it can run, so a machine with no /// Claude provider is not asked about Claude limits -- it has none, and a -/// row saying so would be a fact about nothing. A second service later -/// adds a branch here and an impl beside [`ClaudeUsage`], not a screen. -fn providers_for(setup: &SetupConfig) -> Vec> { +/// row saying so would be a fact about nothing. +/// +/// Which meter a provider has is [`DriverKind::usage_provider`]'s answer +/// rather than a second match on kinds here, because the phone pairs a +/// session with one of these rows by that same name: two lists that +/// disagree would leave a session looking for a snapshot nothing +/// produces, and nothing on screen could say why. A second service later +/// is a name there and an impl beside [`ClaudeUsage`], not a screen. +fn providers_for(setup: &SetupConfig, fixture: &Fixture) -> Vec> { let mut found: Vec> = Vec::new(); - if setup - .providers - .iter() - .any(|provider| provider.kind == DriverKind::ClaudeCli) - { - found.push(Box::new(ClaudeUsage { - setup: setup.id.clone(), - setup_name: setup.name.clone(), - transport: Transport::for_setup(setup), - })); + for provider in &setup.providers { + let Some(name) = provider.kind.usage_provider() else { + continue; + }; + // A machine offering two Claude providers has one account, not + // two: the meter belongs to the machine and the service, which is + // exactly what the cache is keyed by. + if found.iter().any(|already| already.name() == name) { + continue; + } + match name { + CLAUDE => found.push(Box::new(ClaudeUsage { + setup: setup.id.clone(), + setup_name: setup.name.clone(), + transport: Transport::for_setup(setup), + })), + // Nothing at all until a test has asked for something: an + // echo session costs nothing, so the honest answer is no row + // rather than a row saying zero. + ECHO if fixture.is_set() => found.push(Box::new(EchoUsage { + setup: setup.id.clone(), + setup_name: setup.name.clone(), + fixture: fixture.clone(), + })), + _ => {} + } } found } @@ -330,11 +586,18 @@ type Cached = HashMap<(String, &'static str), (Instant, UsageSnapshot)>; #[derive(Default)] pub struct UsageMonitor { cache: Mutex, + /// The invented meter an echo session can put up; empty unless one + /// has. Shared with the session layer, which is where the command + /// that sets it is typed -- see [`Fixture`]. + fixture: Fixture, } impl UsageMonitor { - pub fn new() -> Self { - Self::default() + pub fn new(fixture: Fixture) -> Self { + Self { + cache: Mutex::new(Cached::new()), + fixture, + } } /// One snapshot per machine that offers a paid service, in the order @@ -346,10 +609,10 @@ impl UsageMonitor { pub fn snapshots(&self, setups: &[SetupConfig]) -> Vec { let mut fresh = Vec::new(); for setup in setups { - for provider in providers_for(setup) { + for provider in providers_for(setup, &self.fixture) { let key = (setup.id.clone(), provider.name()); if let Some((fetched, snapshot)) = self.cache.lock().unwrap().get(&key) - && fetched.elapsed() < MIN_POLL_INTERVAL + && fetched.elapsed() < provider.poll_interval() { // Cached numbers, but the machine's *name* is read // fresh: a rename should show immediately rather than @@ -386,6 +649,7 @@ impl UsageMonitor { #[cfg(test)] mod tests { use super::*; + use crate::config::DriverKind; #[test] fn parses_the_limits_array_defensively() { @@ -423,6 +687,7 @@ mod tests { port: None, identity_file: None, options: vec!["ConnectTimeout=1".to_string()], + models_dir: None, attachments_dir: None, }), providers: vec![crate::config::ProviderConfig { @@ -494,9 +759,62 @@ mod tests { models: vec![], }]; // A machine with no Claude on it has no Claude limits, and a row - // reporting on it would be a fact about nothing. - assert!(providers_for(&echo_only).is_empty()); - assert_eq!(providers_for(&unreachable_setup()).len(), 1); + // reporting on it would be a fact about nothing. Echo included: + // an echo session spends nothing, so until a fixture says + // otherwise there is no meter to report. + let unset = Fixture::new(); + assert!(providers_for(&echo_only, &unset).is_empty()); + assert_eq!(providers_for(&unreachable_setup(), &unset).len(), 1); + + // And with one set, that machine has exactly the invented meter + // -- under the name the session's `usageProvider` will name. + let fixture = Fixture::new(); + fixture.command("42"); + let found = providers_for(&echo_only, &fixture); + assert_eq!(found.len(), 1); + assert_eq!(found[0].name(), ECHO); + assert_eq!(DriverKind::Echo.usage_provider(), Some(ECHO)); + assert_eq!(DriverKind::ClaudeCli.usage_provider(), Some(CLAUDE)); + // A local model costs nothing to run, so it meters nothing. + assert_eq!(DriverKind::LlamaCpp.usage_provider(), None); + } + + /// The states the fixture exists to make reachable, and the one thing + /// it must not do: invent a reset time for a window that has none. + #[test] + fn the_fixture_says_each_state_the_screens_have_to_draw() { + let fixture = Fixture::new(); + assert!(fixture.read().is_none(), "unset until somebody sets it"); + + fixture.command("42 90"); + let (state, windows) = fixture.read().expect("set"); + assert_eq!(state, UsageState::Ok); + assert_eq!(windows[0].kind, "session"); + assert_eq!(windows[0].percent, 42.0); + assert!(windows[0].resets_at.is_some()); + + // Between blocks: no reset time, which the phone words as the + // window not running rather than as a time it could not read. + fixture.command("42 never"); + assert_eq!(fixture.read().expect("set").1[0].resets_at, None); + + fixture.command("unreachable no route to host"); + assert!(matches!( + fixture.read().expect("set").0, + UsageState::Unreachable { detail } if detail == "no route to host", + )); + + fixture.command("off"); + assert!(fixture.read().is_none()); + + // A word it does not know changes nothing and says what it takes. + fixture.command("42"); + let refused = fixture.command("sideways"); + assert!( + refused.contains("not one of this fixture's answers"), + "{refused}" + ); + assert_eq!(fixture.read().expect("still set").1[0].percent, 42.0); } #[test]