Compare commits
3
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
e4f0935f98 | ||
|
|
3c0214ece8 | ||
|
|
127b25e60a |
No files matched your search
@@ -26,6 +26,19 @@ Module-by-module intent is in PLAN.md's "Backend layout".
|
||||
|
||||
- `server/` — the Rust backend (`ai-server`). `routes.rs`'s module doc
|
||||
comment is the HTTP table and the surface's source of truth.
|
||||
**A llama.cpp session runs on whatever machine its setup names** (built
|
||||
2026-09-04, the last of phase 5): `Transport::reserve_port` returns the
|
||||
port the server binds *there* and the port that reaches it *here*, and
|
||||
`Launch::reaching` puts the `-L` tunnel on the connection already carrying
|
||||
the command. Three things fell out of it and are easy to get wrong again —
|
||||
a forwarded launch gets a pty (`-tt`) and every other one keeps `-T`,
|
||||
because `llama-server` never reads the stdin whose closing ends a CLI and
|
||||
the same kill left it loaded on the far machine; the model is looked for on
|
||||
the machine that will serve it, so the spawn screen offers
|
||||
`GET /setups/{id}/models` rather than the backend's own downloads; and the
|
||||
readiness poll watches the process as well as the port, since a model that
|
||||
will not load exits in a second and was being reported as "gave up after
|
||||
300s". See PLAN.md's "Transport" and "llama-server management".
|
||||
- `app/` — the Compose app, package `com.example.aiapp`, label "AI Sessions".
|
||||
`AppRoot.kt` is the navigation `when`; `MainScreen.kt` the root's four tabs
|
||||
(sessions, import, models, setups); `Api.kt`/`EventStream.kt` the REST + SSE
|
||||
@@ -193,6 +206,13 @@ Each exists because something was invisible without it.
|
||||
conversation somebody may still be in. **A transcript never goes in this
|
||||
repository**: they hold whatever was said, read and written in that
|
||||
session, and `~/repos` is shared with the host besides.
|
||||
- **`/usage` in an echo session puts up an invented meter**, which is how the
|
||||
rate-limit screens' states are reached without spending quota: `/usage 42`,
|
||||
`/usage 95 20` (minutes left), `/usage 42 never` (the between-blocks window
|
||||
with no reset time), `/usage 42 unreadable`, `/usage notloggedin`,
|
||||
`/usage unreachable`, `/usage failed`, `/usage off`. The vocabulary is
|
||||
`usage::Fixture`'s, since those are its states. With none set an echo
|
||||
session meters nothing, which is the ordinary case and draws no bar.
|
||||
- **A fake CLI exercises the process lifecycle without a token.** Point a
|
||||
`claude_cli` provider's `command` at a two-line script — `#!/bin/sh` and
|
||||
`cat > /dev/null` — and it behaves the way the lifecycle code cares about:
|
||||
@@ -263,24 +283,32 @@ moment you use it — `ANDROID_SERIAL=$(emu serial) ./gradlew …`.
|
||||
|
||||
### Testing llama.cpp and ssh here
|
||||
|
||||
The prebuilt CPU llama.cpp lives outside the repo at
|
||||
`~/.local/opt/llama.cpp` (the 15 MB `ubuntu-x64` release asset). It needs its
|
||||
own directory on `LD_LIBRARY_PATH`, so start the server as
|
||||
`LD_LIBRARY_PATH=~/.local/opt/llama.cpp ai-server …` and point a provider's
|
||||
`command` at `~/.local/opt/llama.cpp/llama-server`. A 0.6B Q8_0 answers at
|
||||
usable speed on this VM's 8 cores. **Do not test with a 2-bit quant**: the
|
||||
**Both are set up here as of 2026-09-04** and need nothing typed. The
|
||||
prebuilt CPU llama.cpp lives outside the repo at `~/.local/opt/llama.cpp`
|
||||
(the 15 MB `ubuntu-x64` release asset) and is symlinked as
|
||||
`/usr/local/bin/llama-server`, which is what makes **discovery find it over
|
||||
ssh**: `~/.local/bin` is not on the PATH a non-interactive ssh session gets.
|
||||
It resolves its own libraries through `$ORIGIN`, so no `LD_LIBRARY_PATH` is
|
||||
needed. One model is downloaded — `unsloth/Qwen3-0.6B-GGUF/Qwen3-0.6B-Q8_0.gguf`,
|
||||
639 MB under `~/.local/share/ai-app/models` — and answers at usable speed on
|
||||
this VM's 8 cores. **Do not test with a 2-bit quant**: the
|
||||
IQ2_XXS of that model produces fluent nonsense, which reads exactly like a
|
||||
broken driver — `llama-cli` produces the same from the file directly, which
|
||||
is how to tell the two apart in a hurry.
|
||||
|
||||
There is no second machine, so **ssh this VM to itself**: generate a
|
||||
throwaway key, append the public half to `~/.ssh/authorized_keys`, and
|
||||
configure a host of `bob@127.0.0.1` with `identityFile` pointing at it plus
|
||||
`options: ["StrictHostKeyChecking=no", "UserKnownHostsFile=…"]` so it touches
|
||||
nothing real. Point a provider's `command` at something harmless like
|
||||
`/bin/echo` rather than at `claude`: the transport is what is under test, the
|
||||
process exiting immediately is the signal, and it costs no tokens. **Take the
|
||||
key back out afterwards.** The remote login shell here is **fish**; the
|
||||
There is no second machine, so **ssh this VM to itself**. That is set up
|
||||
too: the key is `~/.config/ai-app/ssh-self` (its public half is in
|
||||
`~/.ssh/authorized_keys`, labelled removable), and the real config carries a
|
||||
setup called **"this vm over ssh"** — `bob@127.0.0.1` with that
|
||||
`identityFile` plus
|
||||
`options: ["StrictHostKeyChecking=no", "UserKnownHostsFile=/tmp/ai-app-known-hosts"]`
|
||||
so it touches nothing real — offering `claude-cli` and `llama-cpp`. It is the
|
||||
whole rig for "does a remote llama session work", since the far machine is
|
||||
this one and the model file is the same file. For a throwaway setup of your
|
||||
own, point a provider's `command` at something harmless like `/bin/echo`
|
||||
rather than at `claude`: the transport is what is under test, the process
|
||||
exiting immediately is the signal, and it costs no tokens. The remote login
|
||||
shell here is **fish**; the
|
||||
remote script and `ssh.rs`'s POSIX quoting happen to mean the same thing in
|
||||
both, but that is luck rather than design, and a shell that is neither is the
|
||||
thing to suspect first if a remote spawn ever mangles an argument.
|
||||
@@ -379,6 +407,17 @@ where it was instead of half-deleted.
|
||||
swallowed the drag along with the tap, so a list could not be scrolled
|
||||
while anything in it was busy.
|
||||
|
||||
- **A rate-limit bar belongs to a session's provider, not to its machine.**
|
||||
One machine offers echo, the Claude CLI and a local model at once and only
|
||||
the CLI spends anything, so a session says which meter reports on it
|
||||
(`usageProvider`, from `DriverKind::usage_provider`, which
|
||||
`usage::providers_for` reads too so the two lists cannot disagree) and the
|
||||
phone matches a snapshot on machine *and* provider. Nothing meters a llama
|
||||
or echo session, and the phone draws **nothing** for one — not a zero, and
|
||||
not "unknown". Nothing while the first fetch is out either: "checking"
|
||||
under a session that turns out to meter nothing is a row the screen then
|
||||
has to withdraw.
|
||||
|
||||
## Things that have bitten
|
||||
|
||||
Project-specific only — a lesson that would bite any project on this machine
|
||||
|
||||
@@ -169,10 +169,33 @@ deliberate and easy to undo by accident:
|
||||
when the process restarts. That leaves the Claude driver as the odd one
|
||||
out rather than this one — the CLI's memory is a cache in front of the same
|
||||
transcript. Resolve any inconsistency in this direction.
|
||||
- **A llama session on an ssh host is refused.** The model is reached over
|
||||
HTTP and forwarding that port is not built, so refusing beats silently
|
||||
talking to the wrong machine. A transport is "run this" plus "reach this
|
||||
port", and only the first half exists.
|
||||
- **A llama session runs on whatever machine its setup names** (2026-09-04,
|
||||
the last of phase 5). A transport is "run this" plus "reach this port", and
|
||||
the second half is `Transport::reserve_port` — the port the server binds
|
||||
*there* and the port that reaches it *here*, the same number locally —
|
||||
carried by `Launch::reaching` onto the connection that already runs the
|
||||
command. `llama-server` binds loopback on the far machine, so nothing is
|
||||
served to its network. The far port is a guess from a range below the
|
||||
ephemeral one, because no portable way to ask a machine for a free port
|
||||
avoids racing the bind anyway; a collision is not silent, since the server
|
||||
fails to bind and the readiness poll reports what its log said.
|
||||
- **The model file lives on the machine that serves it** (2026-09-04). Each
|
||||
setup names its own models directory (`SshConfig::models_dir`, default
|
||||
`~/.local/share/ai-app/models` expanded *there*), and a spawn resolves the
|
||||
key on that machine — one round trip answering "at /abs/path" or "missing",
|
||||
so a model that is not there is refused at the spawn rather than becoming a
|
||||
server that never becomes ready. The spawn screen offers
|
||||
`GET /setups/{id}/models`, that machine's list, rather than `GET /models`,
|
||||
which is this backend's downloads. Downloading *to* another machine is
|
||||
deliberately not built: a multi-gigabyte transfer with no progress
|
||||
anywhere, and the file gets there however anything else on that machine
|
||||
did.
|
||||
- **The readiness poll watches the process, not only the port.** A model that
|
||||
will not load, a port already taken, a flag an older build does not know:
|
||||
all exit within a second and none will ever answer `/health`, so waiting
|
||||
out the 300s timeout turned the server's own account of the problem into
|
||||
"gave up". The failure carries the tail of `llama-server.log`, which on a
|
||||
remote session is the only copy anybody reading the phone can see.
|
||||
|
||||
### Models (2026-08-28)
|
||||
|
||||
@@ -203,6 +226,13 @@ deliberate and easy to undo by accident:
|
||||
A driver says what to run; something above it turns that into a process.
|
||||
Otherwise transport knowledge sits inside a translator whose job is a wire
|
||||
format, and every future driver has to remember to do the same.
|
||||
- **A forwarded launch gets a pty and every other one does not** (measured
|
||||
2026-09-04). Killing the ssh client ends a CLI because it closes the stdin
|
||||
that CLI is reading; `llama-server` never reads its stdin, so the same kill
|
||||
left it running on the far machine with the model loaded — one orphan per
|
||||
stopped session. With `-tt` the far side takes SIGHUP when the connection
|
||||
goes. Its log then arrives through a line discipline, which nothing parses.
|
||||
`-T` stays everywhere else, where a pty would rewrite the JSONL.
|
||||
- **`command -v` follows ssh's non-login PATH**, which is narrower than an
|
||||
interactive shell's, so a binary somewhere unusual is invisible to
|
||||
discovery. Point `command` at an absolute path.
|
||||
@@ -499,6 +529,25 @@ rate-limited bucket). Poll at ≥180 s, only while a Claude session exists or
|
||||
the usage screen is open, and cache the last answer. It is undocumented, so
|
||||
`usage.rs` treats every field as optional and degrades rather than erroring.
|
||||
|
||||
**Per provider, not per machine (2026-09-04).** A machine is not what is
|
||||
metered; the provider a session runs is. One machine offers echo, the Claude
|
||||
CLI and a local model side by side, and only the second spends anything — so
|
||||
pairing a session with a snapshot by machine alone drew the CLI's five-hour
|
||||
window under every echo session on it, a quota that session cannot spend. A
|
||||
session now names its meter (`usageProvider`, from
|
||||
`DriverKind::usage_provider`, which `usage::providers_for` reads too, so the
|
||||
two lists cannot disagree) and `GET /usage` is matched on machine *and*
|
||||
provider. `None` is a session that meters nothing, and the phone draws
|
||||
nothing at all for it — not a zero, and not "unknown".
|
||||
|
||||
`DriverKind::Echo` names a meter of its own that exists only when a test has
|
||||
asked for one: `/usage` in an echo session sets an invented answer
|
||||
(`usage::Fixture`), and with none set there is no snapshot and no bar. That
|
||||
is what makes those screens' states reachable — a number near the top, a
|
||||
window between blocks with no reset time, a machine nobody logged into, one
|
||||
that could not be reached — without spending real quota to arrange them,
|
||||
which is why none of them had ever been looked at.
|
||||
|
||||
**Per machine, not per backend (2026-08-29).** The credential store that
|
||||
matters is the one on the machine the session runs on, because that is the
|
||||
account being billed — and in the layout this aims at, `ai-server` is on the
|
||||
@@ -800,8 +849,9 @@ Noticed and deliberately not fixed, so they are not re-found from scratch.
|
||||
|
||||
Phases 1–3 (the skeleton pipe, the full Claude driver, the usage screen) done
|
||||
2026-08-24. Phase 4 (llama.cpp: model browsing, downloads, and `llama-server`
|
||||
through its OpenAI-compatible endpoint) and phase 5 (ssh) done 2026-08-28.
|
||||
The file explorer and the transcript cache followed in September. What is
|
||||
through its OpenAI-compatible endpoint) and phase 5 (ssh) done 2026-08-28,
|
||||
except for the remote `llama-server` and its port forward, which landed
|
||||
2026-09-04. The file explorer and the transcript cache followed in September. What is
|
||||
left is real-phone/WireGuard bring-up, which is operational rather than code.
|
||||
|
||||
Each phase ended runnable and verified against the real thing. The backend
|
||||
|
||||
@@ -178,6 +178,17 @@ data class SessionSummary(
|
||||
* server because that is where a provider's kind is known.
|
||||
*/
|
||||
val maxImageEdge: Int?,
|
||||
/**
|
||||
* Which of `GET /usage`'s snapshots is about this session, and null where nothing meters it.
|
||||
*
|
||||
* The rate-limit bar answers a question about an *account*, and what decides which account --
|
||||
* if any -- is the provider this session runs, not the machine it runs on. Pairing by machine
|
||||
* alone drew the Claude CLI's five-hour window under every echo session on a machine that also
|
||||
* has the CLI: a quota that session cannot spend and could never run down. Decided by the
|
||||
* server for the same reason [maxImageEdge] is -- it is a fact about the provider's kind, and
|
||||
* this app has only its name.
|
||||
*/
|
||||
val usageProvider: String?,
|
||||
val status: String,
|
||||
val lastActivity: Double,
|
||||
)
|
||||
@@ -198,6 +209,7 @@ private fun parseSession(session: JSONObject) =
|
||||
contextTokens =
|
||||
if (session.has("contextTokens")) session.getLong("contextTokens") else null,
|
||||
maxImageEdge = session.optInt("maxImageEdge", 0).takeIf { it > 0 },
|
||||
usageProvider = session.optString("usageProvider").ifEmpty { null },
|
||||
status = session.getString("status"),
|
||||
lastActivity = session.getDouble("lastActivity"),
|
||||
)
|
||||
@@ -375,6 +387,12 @@ data class SshDetails(
|
||||
* Where files attached from here land on that machine; null for the session's own directory.
|
||||
*/
|
||||
val attachmentsDir: String? = null,
|
||||
/**
|
||||
* Where that machine keeps its GGUF models; null for the same place the backend keeps its own
|
||||
* (`~/.local/share/ai-app/models`, read on that machine). A llama.cpp session serves the file
|
||||
* from the machine it runs on, so this is where its models are looked for and listed.
|
||||
*/
|
||||
val modelsDir: String? = null,
|
||||
)
|
||||
|
||||
private fun SshDetails.toJson() =
|
||||
@@ -382,6 +400,7 @@ private fun SshDetails.toJson() =
|
||||
if (port != null) put("port", port)
|
||||
if (!identityFile.isNullOrBlank()) put("identityFile", identityFile)
|
||||
if (!attachmentsDir.isNullOrBlank()) put("attachmentsDir", attachmentsDir)
|
||||
if (!modelsDir.isNullOrBlank()) put("modelsDir", modelsDir)
|
||||
}
|
||||
|
||||
/** What a machine turns out to have, without saving anything. */
|
||||
@@ -1074,6 +1093,26 @@ private fun parseDownload(o: JSONObject) =
|
||||
error = if (o.has("error")) o.getString("error") else null,
|
||||
)
|
||||
|
||||
/**
|
||||
* The models on one machine, which is the list a llama.cpp session there can choose from.
|
||||
*
|
||||
* Not [fetchModels], which is what the *backend* has downloaded. A session serves its model from
|
||||
* the machine it runs on, so for a machine reached over ssh those are two different lists -- and
|
||||
* offering the backend's would name files that are not there, turning a choice that cannot work
|
||||
* into a session that fails when it tries to load one.
|
||||
*/
|
||||
fun fetchSetupModels(settings: ServerSettings, setupId: String): List<LocalModel> =
|
||||
requestFromServer(settings, "/setups/${setupId.urlEncoded()}/models") { connection ->
|
||||
JSONArray(connection.inputStream.bufferedReader().readText()).mapObjects { m ->
|
||||
LocalModel(
|
||||
key = m.getString("key"),
|
||||
repo = m.getString("repo"),
|
||||
file = m.getString("file"),
|
||||
bytes = m.getLong("bytes"),
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
fun fetchModels(settings: ServerSettings): Models =
|
||||
requestFromServer(settings, "/models") { connection ->
|
||||
val body = JSONObject(connection.inputStream.bufferedReader().readText())
|
||||
|
||||
@@ -1149,7 +1149,7 @@ fun SessionScreen(
|
||||
|
||||
// One poll for the machines' limits, read by everything on this screen that reports them.
|
||||
val usageFeed = rememberUsageFeed(settings)
|
||||
val usage = usageFeed.forSetup(summary.setup)
|
||||
val usage = usageFeed.forSession(summary)
|
||||
RecordFrames()
|
||||
var usageOpen by remember { mutableStateOf(false) }
|
||||
var settingsOpen by remember { mutableStateOf(false) }
|
||||
|
||||
@@ -70,13 +70,22 @@ class UsageFeed(
|
||||
/** Ask the backend again now. The dialog's refresh button; the poll does it on its own. */
|
||||
val refresh: () -> Unit,
|
||||
) {
|
||||
/** What [setup]'s own limits came back as. See [usageFor] for why the states are these. */
|
||||
fun forSetup(setup: String): SessionUsage =
|
||||
when (val state = snapshots) {
|
||||
/**
|
||||
* What meters [session], and what that meter came back as. See [usageFor] for the states.
|
||||
*
|
||||
* A session rather than a machine, because a machine is not what is metered: one machine runs
|
||||
* the Claude CLI and an echo session side by side, and only the first of them spends anything.
|
||||
*/
|
||||
fun forSession(session: SessionSummary): SessionUsage {
|
||||
// Settled without asking anybody: a session nothing meters has nothing to check, and
|
||||
// "checking" is what the fetch's own states would say about it for as long as one is out.
|
||||
val provider = session.usageProvider ?: return SessionUsage.NotMetered
|
||||
return when (val state = snapshots) {
|
||||
is LoadState.Loading -> SessionUsage.Waiting
|
||||
is LoadState.Error -> SessionUsage.Unavailable(state.message)
|
||||
is LoadState.Loaded -> usageFor(state.value, setup)
|
||||
is LoadState.Loaded -> usageFor(state.value, session.setup, provider)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -158,9 +167,15 @@ fun SessionUsageBar(usage: SessionUsage, modifier: Modifier = Modifier) {
|
||||
}
|
||||
}
|
||||
|
||||
// Nothing at all for a machine that meters nothing: a row saying "unknown" there would report a
|
||||
// problem about a setup somebody chose, on every screen, forever.
|
||||
if (usage is SessionUsage.NotMetered) {
|
||||
// Nothing at all for a session that meters nothing: a row saying "unknown" there would report
|
||||
// a problem about a setup somebody chose, on every screen, forever.
|
||||
//
|
||||
// And nothing while the first fetch is out, which is a different silence. A request in flight
|
||||
// is not a state to report -- and the session that meters nothing is exactly the one this
|
||||
// cannot yet tell apart, so "5-hour usage: checking" appeared under an echo session for half a
|
||||
// second and was then taken away. A row that has to be withdrawn is worse than one that
|
||||
// arrives late.
|
||||
if (usage is SessionUsage.NotMetered || usage is SessionUsage.Waiting) {
|
||||
return
|
||||
}
|
||||
|
||||
@@ -171,9 +186,10 @@ fun SessionUsageBar(usage: SessionUsage, modifier: Modifier = Modifier) {
|
||||
// Words, not a colour and not an empty bar: every one of these is a different kind of
|
||||
// answer from "this much is used", and only words carry a difference in kind.
|
||||
when (val state = usage) {
|
||||
SessionUsage.NotMetered -> Unit
|
||||
// Both handled above, before the row exists at all.
|
||||
SessionUsage.NotMetered,
|
||||
SessionUsage.Waiting -> Unit
|
||||
is SessionUsage.Unavailable -> UsageNote("5-hour usage unknown -- ${state.why}")
|
||||
SessionUsage.Waiting -> UsageNote("5-hour usage: checking")
|
||||
is SessionUsage.Known -> {
|
||||
val window = state.windows.firstOrNull { it.kind == "session" }
|
||||
if (window == null) {
|
||||
@@ -234,16 +250,22 @@ private fun fiveHourLabel(window: UsageWindow, now: OffsetDateTime): String {
|
||||
}
|
||||
|
||||
/**
|
||||
* One machine's snapshot, out of every machine's.
|
||||
* One meter's snapshot, out of every machine's: [setup]'s row for [provider].
|
||||
*
|
||||
* Both halves are needed to pick it. A machine can hold more than one meter -- the Claude CLI's
|
||||
* account and, while a test has one set, an echo session's invented one -- and a snapshot is one
|
||||
* service on one machine.
|
||||
*
|
||||
* Every way of having *failed* to get numbers is [SessionUsage.Unavailable] with the reason in it.
|
||||
* None of them may look like zero, and none may look like [SessionUsage.NotMetered], which is the
|
||||
* machine having no quota rather than the question going unanswered.
|
||||
*/
|
||||
fun usageFor(snapshots: List<UsageSnapshot>, setup: String): SessionUsage {
|
||||
// No snapshot at all means the backend never asked, which it only does for a machine with
|
||||
// nothing metered on it. That is a different answer from having asked and failed.
|
||||
val mine = snapshots.firstOrNull { it.setup == setup } ?: return SessionUsage.NotMetered
|
||||
fun usageFor(snapshots: List<UsageSnapshot>, setup: String, provider: String): SessionUsage {
|
||||
// No snapshot at all means the backend never asked, which it only does where there is nothing
|
||||
// to ask about. That is a different answer from having asked and failed.
|
||||
val mine =
|
||||
snapshots.firstOrNull { it.setup == setup && it.provider == provider }
|
||||
?: return SessionUsage.NotMetered
|
||||
if (mine.state != "ok") {
|
||||
return SessionUsage.Unavailable(mine.detail ?: mine.state)
|
||||
}
|
||||
|
||||
@@ -236,6 +236,7 @@ private fun AddSetupDialog(
|
||||
var address by remember { mutableStateOf("") }
|
||||
var identity by remember { mutableStateOf("") }
|
||||
var attachmentsDir by remember { mutableStateOf("") }
|
||||
var modelsDir by remember { mutableStateOf("") }
|
||||
var tested by remember { mutableStateOf<String?>(null) }
|
||||
var testing by remember { mutableStateOf(false) }
|
||||
|
||||
@@ -250,6 +251,7 @@ private fun AddSetupDialog(
|
||||
port = typedPort,
|
||||
identityFile = identity.trim().ifEmpty { null },
|
||||
attachmentsDir = attachmentsDir.trim().ifEmpty { null },
|
||||
modelsDir = modelsDir.trim().ifEmpty { null },
|
||||
)
|
||||
}
|
||||
|
||||
@@ -293,6 +295,14 @@ private fun AddSetupDialog(
|
||||
label = { Text("Folder for attached files (optional)") },
|
||||
singleLine = true,
|
||||
)
|
||||
// Where that machine's GGUFs are, for a llama.cpp session on it. Blank means
|
||||
// the same place this backend keeps its own downloads, read on that machine.
|
||||
OutlinedTextField(
|
||||
value = modelsDir,
|
||||
onValueChange = { modelsDir = it },
|
||||
label = { Text("Folder for models (optional)") },
|
||||
singleLine = true,
|
||||
)
|
||||
tested?.let {
|
||||
Spacer(Modifier.height(8.dp))
|
||||
Text(it, style = MaterialTheme.typography.bodySmall)
|
||||
|
||||
@@ -65,8 +65,10 @@ fun SpawnScreen(
|
||||
// Only the spawn's own failure. The fetch's lives in `options`: this one leaves a filled-in
|
||||
// form worth keeping, and that one leaves nothing to fill in.
|
||||
var spawnError by remember { mutableStateOf<String?>(null) }
|
||||
// Downloaded models, for a llama provider to choose between. Kept separate from the setups: a
|
||||
// Claude session needs none, so failing to list them must not stop the screen rendering.
|
||||
// The models on the *chosen machine*, for a llama provider to choose between. Kept separate
|
||||
// from the setups: a Claude session needs none, so failing to list them must not stop the
|
||||
// screen rendering. Refetched when the machine changes, because a model is a file on one
|
||||
// machine -- see [fetchSetupModels].
|
||||
var models by remember { mutableStateOf<List<LocalModel>>(emptyList()) }
|
||||
var modelKey by remember { mutableStateOf<String?>(null) }
|
||||
var contextSize by remember { mutableStateOf("") }
|
||||
@@ -83,9 +85,6 @@ fun SpawnScreen(
|
||||
} catch (e: ApiException) {
|
||||
LoadState.failed(e)
|
||||
}
|
||||
models =
|
||||
runCatching { withContext(Dispatchers.IO) { fetchModels(settings).local } }
|
||||
.getOrDefault(emptyList())
|
||||
}
|
||||
|
||||
Column(Modifier.fillMaxSize().verticalScroll(rememberScrollState()).padding(16.dp)) {
|
||||
@@ -115,6 +114,17 @@ fun SpawnScreen(
|
||||
is LoadState.Loaded -> state.value
|
||||
}
|
||||
val setup = setups.firstOrNull { it.name == setupName }
|
||||
// Whichever machine is chosen now, asked again when that changes. The old machine's list
|
||||
// is dropped first rather than left on screen: a file name from another machine looks
|
||||
// exactly like one from this one.
|
||||
LaunchedEffect(setup?.id) {
|
||||
models = emptyList()
|
||||
modelKey = null
|
||||
val id = setup?.id ?: return@LaunchedEffect
|
||||
models =
|
||||
runCatching { withContext(Dispatchers.IO) { fetchSetupModels(settings, id) } }
|
||||
.getOrDefault(emptyList())
|
||||
}
|
||||
val current = setup?.providers?.firstOrNull { it.name == providerName }
|
||||
// Only the Claude CLI has models, a working directory and permission modes; keying the
|
||||
// extra fields on the kind rather than the provider name keeps a second Claude provider
|
||||
@@ -175,12 +185,13 @@ fun SpawnScreen(
|
||||
)
|
||||
|
||||
if (isLlama) {
|
||||
// A llama session names one of the models this backend has downloaded, so the choice is
|
||||
// that list rather than free text -- a name that is not on disk is a session that
|
||||
// cannot start.
|
||||
// A llama session names one of the models on the machine it will run on, so the
|
||||
// choice is that list rather than free text -- a name that is not on that machine's
|
||||
// disk is a session that cannot start.
|
||||
if (models.isEmpty()) {
|
||||
Text(
|
||||
"No models downloaded yet. Get one from the Models screen first.",
|
||||
"No models on ${setup?.name ?: "this machine"}. The Models screen downloads " +
|
||||
"to the backend; another machine needs the file put there itself.",
|
||||
style = MaterialTheme.typography.bodyMedium,
|
||||
color = MaterialTheme.colorScheme.onSurfaceVariant,
|
||||
)
|
||||
|
||||
Generated
+1
@@ -35,6 +35,7 @@ dependencies = [
|
||||
"sha2",
|
||||
"tempfile",
|
||||
"thiserror",
|
||||
"time",
|
||||
"tokio",
|
||||
"tokio-stream",
|
||||
"tower",
|
||||
|
||||
@@ -53,6 +53,12 @@ ureq = { version = "3", features = ["json"] }
|
||||
# both in the graph rustls refuses to auto-select one.
|
||||
rustls = "0.23"
|
||||
libc = "0.2.189"
|
||||
# One ISO-8601 timestamp: the reset time on the invented rate-limit window
|
||||
# an echo session's `/usage` puts up. Already in the tree behind the
|
||||
# certificate machinery, so this is a direct name for what is compiled
|
||||
# anyway rather than a new crate -- and the alternative was hand-rolling a
|
||||
# civil-from-days conversion to print one line.
|
||||
time = { version = "0.3", features = ["formatting"] }
|
||||
|
||||
[dev-dependencies]
|
||||
tempfile = "3"
|
||||
|
||||
@@ -207,6 +207,20 @@ mod tests {
|
||||
/// like any other.
|
||||
#[tokio::test]
|
||||
async fn a_spooled_enrollment_is_adopted_on_first_use() {
|
||||
// Under a subscriber, like every other exercise of this middleware.
|
||||
// `tracing` caches a callsite's interest process-wide the first time it
|
||||
// is reached, so the refusal at the end of this test -- reached with no
|
||||
// subscriber on this thread -- could cache the rejection warning as
|
||||
// never-enabled and make the tripwire above see an empty log. That
|
||||
// failed about one full-suite run in ten, in the test that exists to
|
||||
// notice a credential leak, which is the worst place for a flake.
|
||||
let _guard = tracing::subscriber::set_default(
|
||||
tracing_subscriber::fmt()
|
||||
.with_max_level(tracing::Level::TRACE)
|
||||
.with_writer(std::io::sink)
|
||||
.finish(),
|
||||
);
|
||||
|
||||
let dir = tempfile::tempdir().expect("tempdir");
|
||||
let manager = manager_with_token(dir.path(), "first");
|
||||
let spooled = generate_token();
|
||||
|
||||
@@ -94,6 +94,19 @@ pub struct SshConfig {
|
||||
/// Extra `-o` settings, each written as `Key=value`.
|
||||
#[serde(default, skip_serializing_if = "Vec::is_empty")]
|
||||
pub options: Vec<String>,
|
||||
/// Where this machine keeps the GGUF models it can serve, absent for
|
||||
/// the same default this backend uses (`~/.local/share/ai-app/models`
|
||||
/// -- `$XDG_DATA_HOME` is not read on the far side, since it is this
|
||||
/// machine's environment that would answer). A `~` prefix is the
|
||||
/// remote home.
|
||||
///
|
||||
/// Here rather than on the provider because it is a fact about the
|
||||
/// machine, and because a machine reached over ssh is where the model
|
||||
/// has to be: a llama.cpp session serves the file from the machine
|
||||
/// that runs `llama-server`, and this backend's own downloads are on
|
||||
/// whichever machine that is only when they are the same one.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub models_dir: Option<PathBuf>,
|
||||
/// Where a file attached from the phone is put on this machine so the
|
||||
/// session can read it. Absent means the session's own working directory,
|
||||
/// or the login home for a session that has none. A `~` prefix is the
|
||||
@@ -145,6 +158,30 @@ impl DriverKind {
|
||||
}
|
||||
}
|
||||
|
||||
/// Which paid service meters a session of this kind, and `None` for one
|
||||
/// that costs nothing.
|
||||
///
|
||||
/// What decides which account -- if any -- a rate-limit bar is about is the
|
||||
/// provider a session runs, not the machine it runs on: an echo session on
|
||||
/// a machine that also has the Claude CLI was drawn with that CLI's
|
||||
/// five-hour window, a quota it cannot spend.
|
||||
///
|
||||
/// Echo names a meter of its own that exists only when a test has asked for
|
||||
/// one (`/usage` in `session::echo`), which is how the bar's states are
|
||||
/// reached without an account. With none set there is no snapshot, and the
|
||||
/// phone draws nothing.
|
||||
///
|
||||
/// The string is a [`crate::usage::UsageProvider::name`], and it is what
|
||||
/// pairs a session with one of `GET /usage`'s snapshots -- so
|
||||
/// `usage::providers_for` reads this rather than matching on kinds again.
|
||||
pub fn usage_provider(self) -> Option<&'static str> {
|
||||
match self {
|
||||
Self::ClaudeCli => Some(crate::usage::CLAUDE),
|
||||
Self::Echo => Some(crate::usage::ECHO),
|
||||
Self::LlamaCpp => None,
|
||||
}
|
||||
}
|
||||
|
||||
/// Whether the conversation exists outside this app, so that deleting the
|
||||
/// session here does not end it.
|
||||
///
|
||||
@@ -385,6 +422,7 @@ mod tests {
|
||||
port: Some(2222),
|
||||
identity_file: None,
|
||||
options: Vec::new(),
|
||||
models_dir: None,
|
||||
attachments_dir: None,
|
||||
}),
|
||||
providers: vec![ProviderConfig {
|
||||
|
||||
+3
-1
@@ -264,7 +264,9 @@ async fn main() -> Result<()> {
|
||||
// No providers listed here any more: which machines can be asked, and about
|
||||
// what, comes from the setups at the moment the screen is opened -- so a
|
||||
// machine added from the phone reports its limits without a restart.
|
||||
let monitor = Arc::new(usage::UsageMonitor::new());
|
||||
// The fixture is the manager's, because that is where the `/usage` command
|
||||
// that sets it is typed; the monitor is what serves it.
|
||||
let monitor = Arc::new(usage::UsageMonitor::new(manager.usage_fixture()));
|
||||
|
||||
// The bearer-token middleware wraps the entire router -- routes and fallback
|
||||
// alike -- here and only here, so a new route can't forget auth.
|
||||
|
||||
@@ -29,6 +29,8 @@ use serde::Serialize;
|
||||
|
||||
use wg_app_link::private;
|
||||
|
||||
use crate::session::transport::{Launch, Transport};
|
||||
|
||||
/// Identifies this client to HuggingFace. They ask for one, and a request
|
||||
/// without it is more likely to be rate-limited.
|
||||
const USER_AGENT: &str = concat!("ai-server/", env!("CARGO_PKG_VERSION"));
|
||||
@@ -521,6 +523,83 @@ fn collect(root: &Path, dir: &Path, found: &mut Vec<LocalModel>) {
|
||||
}
|
||||
}
|
||||
|
||||
/// Where a machine reached over ssh keeps its models, when its setup does
|
||||
/// not say.
|
||||
///
|
||||
/// The same place this backend puts its own downloads, written out rather
|
||||
/// than derived: `$XDG_DATA_HOME` here describes *this* machine's
|
||||
/// environment, and the far machine's is the far machine's business. A
|
||||
/// setup whose models are elsewhere says so (`SshConfig::models_dir`).
|
||||
const FAR_MODELS_DIR: &str = "~/.local/share/ai-app/models";
|
||||
|
||||
/// Which directory holds the models on the machine `transport` reaches.
|
||||
///
|
||||
/// One answer, because two things ask: the list a spawn screen offers,
|
||||
/// and the path a session hands `llama-server`. A machine that listed one
|
||||
/// directory and served from another would offer models that then failed
|
||||
/// to load, which reads as the model being broken.
|
||||
pub fn dir_on(transport: &Transport, local: &Path) -> String {
|
||||
match transport {
|
||||
Transport::Here => local.to_string_lossy().into_owned(),
|
||||
Transport::Ssh { ssh, .. } => ssh
|
||||
.models_dir
|
||||
.as_ref()
|
||||
.map_or(FAR_MODELS_DIR.to_string(), |dir| {
|
||||
dir.to_string_lossy().into_owned()
|
||||
}),
|
||||
}
|
||||
}
|
||||
|
||||
/// Every GGUF on the machine a setup names, which is the machine that
|
||||
/// would have to serve it.
|
||||
///
|
||||
/// The local half of this is [`ModelStore::list`], reading the same shape
|
||||
/// off this machine's disk; a caller picks by transport, since a setup
|
||||
/// with no ssh *is* this machine and asking a shell about it would be a
|
||||
/// slower way to the same answer. What must not happen is offering this
|
||||
/// backend's downloads for a session on another machine: the file has to
|
||||
/// be where `llama-server` runs, and a list that says otherwise is a
|
||||
/// claim about the wrong filesystem.
|
||||
///
|
||||
/// `dir` is that machine's models directory, `~` included -- expanded on
|
||||
/// the far side, which is the only place that knows what it is. A
|
||||
/// directory that is not there is an empty list rather than a failure: a
|
||||
/// machine that has never had a model put on it is an ordinary state, and
|
||||
/// the same one as a machine whose directory exists and is empty.
|
||||
pub async fn on_machine(transport: &Transport, dir: &str) -> Result<Vec<LocalModel>> {
|
||||
let script = "p=$1; case $p in \"~\") p=$HOME;; \"~/\"*) p=$HOME/${p#\"~/\"};; esac; \
|
||||
[ -d \"$p\" ] || exit 0; \
|
||||
find \"$p\" -type f -name '*.gguf' -printf '%s\\t%P\\0'";
|
||||
let launch = Launch::new(
|
||||
"sh",
|
||||
vec![
|
||||
"-c".to_string(),
|
||||
script.to_string(),
|
||||
"sh".to_string(),
|
||||
dir.to_string(),
|
||||
],
|
||||
None,
|
||||
);
|
||||
let out = transport.capture(&launch).await?;
|
||||
let mut found: Vec<LocalModel> = out
|
||||
.split('\0')
|
||||
.filter(|record| !record.is_empty())
|
||||
// Two fields, and the name last, so a `\t` in a filename survives.
|
||||
.filter_map(|record| record.split_once('\t'))
|
||||
.filter_map(|(bytes, key)| {
|
||||
let (repo, file) = key.rsplit_once('/')?;
|
||||
Some(LocalModel {
|
||||
key: key.to_string(),
|
||||
repo: repo.to_string(),
|
||||
file: file.to_string(),
|
||||
bytes: bytes.trim().parse().unwrap_or(0),
|
||||
})
|
||||
})
|
||||
.collect();
|
||||
found.sort_by(|a, b| a.key.cmp(&b.key));
|
||||
Ok(found)
|
||||
}
|
||||
|
||||
/// A model repository on HuggingFace, as the browse screen shows it.
|
||||
#[derive(Debug, Clone, Serialize)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
|
||||
@@ -7,6 +7,7 @@
|
||||
//! POST /setups add {name, ssh?} -- providers are discovered
|
||||
//! POST /setups/probe dry run {ssh?}: what would be found there
|
||||
//! GET /setups/{id} one machine, for refetching after a change
|
||||
//! GET /setups/{id}/models GGUFs on that machine, for a llama session
|
||||
//! GET /setups/{id}/dir?path=P entries of directory P, and P resolved
|
||||
//! GET /setups/{id}/file?path=P content of file P, or why not
|
||||
//! PUT /setups/{id}/file {path, content, ifSha256} -> new size/mtime/sha256
|
||||
@@ -109,6 +110,8 @@ pub fn router(manager: Arc<SessionManager>) -> Router {
|
||||
"/setups/{id}",
|
||||
get(read_setup).put(update_setup).delete(delete_setup),
|
||||
)
|
||||
// The models on the machine a setup names, for a llama session there.
|
||||
.route("/setups/{id}/models", get(setup_models))
|
||||
// The filesystem of the machine a setup names. Under the setup
|
||||
// rather than under a session because a filesystem is a property of
|
||||
// a machine; a session only says where to start looking.
|
||||
@@ -291,6 +294,9 @@ struct SshRequest {
|
||||
/// Where attached files land on that machine; see `SshConfig`.
|
||||
#[serde(default)]
|
||||
attachments_dir: Option<String>,
|
||||
/// Where that machine keeps its GGUF models; see `SshConfig`.
|
||||
#[serde(default)]
|
||||
models_dir: Option<String>,
|
||||
}
|
||||
|
||||
impl SshRequest {
|
||||
@@ -320,6 +326,14 @@ impl SshRequest {
|
||||
.map(str::trim)
|
||||
.filter(|dir| !dir.is_empty())
|
||||
.map(std::path::PathBuf::from),
|
||||
// The same rule, and for the same reason: this directory is
|
||||
// on the other machine, so a `~` in it is that machine's home.
|
||||
models_dir: self
|
||||
.models_dir
|
||||
.as_deref()
|
||||
.map(str::trim)
|
||||
.filter(|dir| !dir.is_empty())
|
||||
.map(std::path::PathBuf::from),
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -495,6 +509,27 @@ struct PathQuery {
|
||||
path: String,
|
||||
}
|
||||
|
||||
/// The models **that machine** has, which is the list a llama.cpp session
|
||||
/// on it can choose from.
|
||||
///
|
||||
/// Not `GET /models`, which is this backend's own downloads: those are on
|
||||
/// the machine a session runs on only when they are the same machine. A
|
||||
/// spawn screen offering this backend's list for a remote setup would be
|
||||
/// naming files that are not there, and the session would fail at the
|
||||
/// point of loading rather than at the point of choosing.
|
||||
async fn setup_models(
|
||||
State(manager): State<Arc<SessionManager>>,
|
||||
UrlPath(id): UrlPath<String>,
|
||||
) -> Result<axum::Json<Vec<crate::models::LocalModel>>, ApiError> {
|
||||
let setup = setup_by_id(&manager, &id)?;
|
||||
let transport = crate::session::transport::Transport::for_setup(&setup);
|
||||
let dir = crate::models::dir_on(&transport, manager.models_dir());
|
||||
crate::models::on_machine(&transport, &dir)
|
||||
.await
|
||||
.map(axum::Json)
|
||||
.map_err(from_machine)
|
||||
}
|
||||
|
||||
/// What is in a directory, and what that directory resolved to.
|
||||
async fn list_dir(
|
||||
State(manager): State<Arc<SessionManager>>,
|
||||
|
||||
@@ -26,6 +26,13 @@
|
||||
//! - `/error [text]` -- a failure, which is otherwise awkward to cause.
|
||||
//! - `/peer [text]`, `/peer-turn` -- a message from another agent, in the
|
||||
//! in-place and the live shapes.
|
||||
//! - `/usage [what]` -- an invented rate-limit answer, or `/usage off` to take
|
||||
//! it away. An echo session meters nothing, so it draws no usage bar until
|
||||
//! this is set; what it exists for is the states that bar can be in, which
|
||||
//! otherwise cost real quota to reach. `/usage 42`, `/usage 95 20`,
|
||||
//! `/usage 42 never`, `/usage notloggedin`, `/usage unreachable`,
|
||||
//! `/usage failed`. The vocabulary is `usage::Fixture`'s, where the states
|
||||
//! live.
|
||||
//! - `/compact` -- a compaction, start to finish.
|
||||
//! - `/stream N` -- one long answer in N small pieces, 50ms apart: the shape a
|
||||
//! real model's reply arrives in, and the one where the row a reader is
|
||||
@@ -85,6 +92,10 @@ pub struct EchoDriver {
|
||||
/// because `/ask` puts up to four on one tool call, and the turn resumes
|
||||
/// when the last is answered rather than the first.
|
||||
pending_questions: Mutex<Vec<PendingQuestion>>,
|
||||
/// The invented rate-limit answer `/usage` sets, shared with the usage
|
||||
/// monitor that serves it. An echo session meters nothing, so this is unset
|
||||
/// until a test asks for something -- see [`crate::usage::Fixture`].
|
||||
usage: crate::usage::Fixture,
|
||||
/// A pretend context, so the status row has something that behaves the way
|
||||
/// a real one does: it grows with each turn, drops to what the compaction
|
||||
/// says it recovered, and a clear leaves it unmeasured. What is real is
|
||||
@@ -311,6 +322,28 @@ impl EchoDriver {
|
||||
return;
|
||||
}
|
||||
|
||||
// Answered here rather than in the turn below, because it is not a
|
||||
// turn: nothing is generated, and what is being exercised is the
|
||||
// *other* screens -- the bar under the header, the button beside it and
|
||||
// the dialog it opens, which read the usage route, not this transcript.
|
||||
if let Some(rest) = text.strip_prefix("/usage") {
|
||||
if announce {
|
||||
self.emit(Event::MessageTaken {
|
||||
id: None,
|
||||
text: text.clone(),
|
||||
attachments,
|
||||
});
|
||||
}
|
||||
let said = self.usage.command(rest);
|
||||
self.emit(Event::AssistantText {
|
||||
delta: format!("{said}\n"),
|
||||
});
|
||||
self.emit(Event::Status {
|
||||
state: SessionStatus::Idle,
|
||||
});
|
||||
return;
|
||||
}
|
||||
|
||||
// The same word the real CLI takes, so a phone drives both the same way.
|
||||
// `Driver::compact` is what the manager's route calls; this is the typed
|
||||
// path onto it.
|
||||
@@ -606,7 +639,7 @@ impl EchoDriver {
|
||||
});
|
||||
}
|
||||
|
||||
pub fn new(sink: EventSink, session_dir: PathBuf) -> Self {
|
||||
pub fn new(sink: EventSink, session_dir: PathBuf, usage: crate::usage::Fixture) -> Self {
|
||||
let driver = Self {
|
||||
sink,
|
||||
pending_questions: Mutex::new(Vec::new()),
|
||||
@@ -614,6 +647,7 @@ impl EchoDriver {
|
||||
busy: Arc::new(AtomicBool::new(false)),
|
||||
queued: Arc::new(Mutex::new(Vec::new())),
|
||||
session_dir,
|
||||
usage,
|
||||
};
|
||||
driver.emit(Event::Status {
|
||||
state: SessionStatus::Idle,
|
||||
|
||||
+137
-28
@@ -5,9 +5,19 @@
|
||||
//!
|
||||
//! **It is spawned but not spoken to over stdio.** The process is started
|
||||
//! through the same [`Transport`] as any other and then reached over HTTP on a
|
||||
//! loopback port. A remote llama-server would need its port forwarded as well
|
||||
//! as its command wrapped, which is not built, so a session on an ssh host is
|
||||
//! refused rather than silently talking to the wrong machine.
|
||||
//! loopback port. That is the second half of what a transport is -- "run this"
|
||||
//! plus "reach this port" -- and it is what lets a session run on another
|
||||
//! machine: [`Transport::reserve_port`] hands back a port the server binds
|
||||
//! *there* and one that reaches it *here*, and the ssh connection carrying the
|
||||
//! command carries the tunnel between them. The far `llama-server` binds
|
||||
//! loopback only, so a model is never served to that machine's network.
|
||||
//!
|
||||
//! **The model file is the far machine's, not this one's.** A remote setup
|
||||
//! names its own models directory (`SshConfig::models_dir`, defaulting to where
|
||||
//! this backend keeps its downloads), and the file is looked for *there* -- so
|
||||
//! a session naming a model that machine does not have says so, instead of
|
||||
//! starting a server that will never load one. Downloading to another machine
|
||||
//! is not built; the model gets there however anything else does.
|
||||
//!
|
||||
//! **The server is stateless between requests**, so the whole conversation goes
|
||||
//! with every one. It is rebuilt from the session's transcript rather than kept
|
||||
@@ -76,16 +86,10 @@ impl LlamaDriver {
|
||||
session_dir: &Path,
|
||||
sink: EventSink,
|
||||
) -> Result<Self> {
|
||||
if !matches!(transport, Transport::Here) {
|
||||
bail!(
|
||||
"llama.cpp sessions can only run on this machine for now: the model is served \
|
||||
over HTTP, and forwarding that port to another host isn't built yet."
|
||||
);
|
||||
}
|
||||
let model = meta.model.as_deref().context(
|
||||
"a llama.cpp session needs a model -- one of the downloaded ones, by its key",
|
||||
)?;
|
||||
let path = model_path(models_dir, model)?;
|
||||
let path = model_on(transport, models_dir, model)?;
|
||||
|
||||
// Already loaded and still running: keep talking to it. The health poll
|
||||
// below confirms it is really answering, so adopting a pid whose server
|
||||
@@ -111,14 +115,21 @@ impl LlamaDriver {
|
||||
));
|
||||
}
|
||||
|
||||
let port = free_port().context("finding a port for llama-server")?;
|
||||
// Where it listens on its own machine, and where that is reached
|
||||
// from here -- the same number when that machine is this one.
|
||||
let forward = transport
|
||||
.reserve_port()
|
||||
.context("finding a port for llama-server")?;
|
||||
let mut args: Vec<String> = vec![
|
||||
"-m".into(),
|
||||
path.to_string_lossy().into_owned(),
|
||||
path.clone(),
|
||||
// Loopback there, whichever machine there is: what reaches it
|
||||
// from outside that machine is the ssh tunnel and nothing
|
||||
// else.
|
||||
"--host".into(),
|
||||
"127.0.0.1".into(),
|
||||
"--port".into(),
|
||||
port.to_string(),
|
||||
forward.there.to_string(),
|
||||
];
|
||||
// Settings that belong to the server because they decide how the model
|
||||
// is loaded; the sampling ones ride on each request instead, so changing
|
||||
@@ -135,7 +146,7 @@ impl LlamaDriver {
|
||||
}
|
||||
|
||||
let program = provider.command.as_deref().unwrap_or("llama-server");
|
||||
let launch = Launch::new(program, args, meta.cwd.as_deref());
|
||||
let launch = Launch::new(program, args, meta.cwd.as_deref()).reaching(forward);
|
||||
// Its output goes to files, not pipes. Not only so the process can
|
||||
// outlive this server: nothing ever read those pipes, so a chatty
|
||||
// llama-server filled the 64 KB buffer and blocked mid-load with no sign
|
||||
@@ -152,8 +163,12 @@ impl LlamaDriver {
|
||||
.id()
|
||||
.context("llama-server exited before it could be recorded")?;
|
||||
tracing::info!(
|
||||
"session {} running {program} for {model} on 127.0.0.1:{port} as pid {pid}",
|
||||
meta.id
|
||||
"session {} running {program} for {model} {} on 127.0.0.1:{} there, \
|
||||
reached at 127.0.0.1:{} here, as pid {pid}",
|
||||
meta.id,
|
||||
transport.describe(),
|
||||
forward.there,
|
||||
forward.here,
|
||||
);
|
||||
// Reaped so it does not become a zombie while this server is still its
|
||||
// parent; the health poll and the record are what say whether the
|
||||
@@ -163,12 +178,18 @@ impl LlamaDriver {
|
||||
let _ = child.wait().await;
|
||||
});
|
||||
|
||||
let record = process::Record::of(pid, process::Detail::Http { port })
|
||||
// The *near* port, because that is the one anything reaching this
|
||||
// server has to dial -- including a later run of this backend,
|
||||
// which adopts the record without knowing which machine the server
|
||||
// is on. For a remote session the recorded pid is the ssh
|
||||
// client's, which is the process this machine owns and which holds
|
||||
// the tunnel open for exactly as long as the far server lives.
|
||||
let record = process::Record::of(pid, process::Detail::Http { port: forward.here })
|
||||
.context("llama-server was gone before its start time could be read")?;
|
||||
process::write(session_dir, &record);
|
||||
|
||||
Ok(Self::attached(
|
||||
format!("http://127.0.0.1:{port}"),
|
||||
format!("http://127.0.0.1:{}", forward.here),
|
||||
meta,
|
||||
model,
|
||||
transcript,
|
||||
@@ -202,7 +223,7 @@ impl LlamaDriver {
|
||||
let endpoint = endpoint.clone();
|
||||
let model = model.to_string();
|
||||
let session_dir = session_dir.to_path_buf();
|
||||
std::thread::spawn(move || match wait_until_ready(&endpoint) {
|
||||
std::thread::spawn(move || match wait_until_ready(&endpoint, &session_dir) {
|
||||
Ok(()) => {
|
||||
tracing::info!("{model} loaded and answering at {endpoint}");
|
||||
let _ = sink.send(Event::Status {
|
||||
@@ -499,17 +520,70 @@ fn model_path(models_dir: &Path, key: &str) -> Result<PathBuf> {
|
||||
Ok(path)
|
||||
}
|
||||
|
||||
/// An unused loopback port, by asking the OS for one and letting it go. Racy in
|
||||
/// principle, but nothing on this machine is hunting for ports, and the
|
||||
/// alternative -- parsing the port back out of the server's log -- couples us to
|
||||
/// its output format for no real gain.
|
||||
fn free_port() -> Result<u16> {
|
||||
let listener = std::net::TcpListener::bind("127.0.0.1:0")?;
|
||||
Ok(listener.local_addr()?.port())
|
||||
/// The model file's path **on the machine that will serve it**, confirmed to be
|
||||
/// there.
|
||||
///
|
||||
/// One function rather than a local check and hope for the other case: the same
|
||||
/// question has to be asked of two filesystems. The remote answer is measured
|
||||
/// for the reason the local one is -- a missing file otherwise becomes a
|
||||
/// `llama-server` that starts, fails to load, and reports as a session that
|
||||
/// never became ready, which reads as the machine being slow.
|
||||
///
|
||||
/// One blocking round trip on a remote spawn, which is what the spawn is
|
||||
/// already paying to start ssh. The alternative is a path built here from a `~`
|
||||
/// this machine cannot expand.
|
||||
fn model_on(transport: &Transport, models_dir: &Path, key: &str) -> Result<String> {
|
||||
let Transport::Ssh { name, .. } = transport else {
|
||||
return Ok(model_path(models_dir, key)?.to_string_lossy().into_owned());
|
||||
};
|
||||
// The same directory the spawn screen listed for this machine, and one
|
||||
// function for the same reason: a list from one place and a load from
|
||||
// another is a model that appears and then fails.
|
||||
let dir = crate::models::dir_on(transport, models_dir);
|
||||
// Checked here rather than in the script: `..` in a key would walk out of
|
||||
// the models directory on a machine this server can start processes on,
|
||||
// and the phone is where the key comes from.
|
||||
for part in key.split('/') {
|
||||
if part.is_empty() || part == "." || part == ".." {
|
||||
bail!("\"{key}\" is not a model key this can resolve");
|
||||
}
|
||||
}
|
||||
let path = format!("{}/{key}", dir.trim_end_matches('/'));
|
||||
// `$HOME` on the far side, which is the only machine that knows what it is,
|
||||
// and the resolved path printed back so the launch hands `llama-server`
|
||||
// something absolute. "Not there" is answered rather than failed, because a
|
||||
// machine that could not be asked at all has to say so in its own words --
|
||||
// it would otherwise arrive as this same sentence about a missing model.
|
||||
let script = "p=$1; case $p in \"~\") p=$HOME;; \"~/\"*) p=$HOME/${p#\"~/\"};; esac; \
|
||||
[ -f \"$p\" ] && printf 'at\\t%s\\n' \"$p\" || printf 'missing\\n'"
|
||||
.to_string();
|
||||
let launch = Launch::new(
|
||||
"sh",
|
||||
vec!["-c".to_string(), script, "sh".to_string(), path.clone()],
|
||||
None,
|
||||
);
|
||||
let answer = transport
|
||||
.capture_blocking(&launch)
|
||||
.with_context(|| format!("couldn't ask {name} where its models are"))?;
|
||||
match answer.trim().split_once('\t') {
|
||||
Some(("at", resolved)) => Ok(resolved.to_string()),
|
||||
_ => bail!(
|
||||
"{name} has no model at {path}. A llama.cpp session serves the file from the \
|
||||
machine it runs on, so the model has to be on {name} -- what this backend has \
|
||||
downloaded is somewhere else."
|
||||
),
|
||||
}
|
||||
}
|
||||
|
||||
/// Polls until the server says it is ready, or gives up.
|
||||
fn wait_until_ready(endpoint: &str) -> Result<()> {
|
||||
///
|
||||
/// Watches the process as well as the port, because the two failures need
|
||||
/// different words and one of them is common: a model that will not load,
|
||||
/// a port already taken on the far machine, a `llama-server` too old for
|
||||
/// a flag. All of those exit within a second and none of them will ever
|
||||
/// answer `/health`, so waiting out the timeout turns a server that said
|
||||
/// exactly what was wrong into "gave up after 300s".
|
||||
fn wait_until_ready(endpoint: &str, session_dir: &Path) -> Result<()> {
|
||||
let deadline = std::time::Instant::now() + READY_TIMEOUT;
|
||||
let url = format!("{endpoint}/health");
|
||||
loop {
|
||||
@@ -518,13 +592,48 @@ fn wait_until_ready(endpoint: &str) -> Result<()> {
|
||||
{
|
||||
return Ok(());
|
||||
}
|
||||
// `None` is the session having been stopped or deleted while this
|
||||
// waited, which is nobody's fault and still not worth waiting on.
|
||||
match process::recorded(session_dir) {
|
||||
Some((_, process::Liveness::Alive | process::Liveness::Unknown)) => {}
|
||||
Some((_, process::Liveness::Dead)) | None => {
|
||||
bail!("it exited before it answered.{}", log_tail(session_dir));
|
||||
}
|
||||
}
|
||||
if std::time::Instant::now() > deadline {
|
||||
bail!("gave up after {}s", READY_TIMEOUT.as_secs());
|
||||
bail!(
|
||||
"gave up after {}s.{}",
|
||||
READY_TIMEOUT.as_secs(),
|
||||
log_tail(session_dir)
|
||||
);
|
||||
}
|
||||
std::thread::sleep(std::time::Duration::from_millis(250));
|
||||
}
|
||||
}
|
||||
|
||||
/// The end of `llama-server`'s own log, for a failure message.
|
||||
///
|
||||
/// Its account of what went wrong is the useful half -- "failed to load
|
||||
/// model", "bind: Address already in use" -- and on a remote session it
|
||||
/// is the only half, since nobody reading the phone can open a file on
|
||||
/// that machine. Bounded, because this ends up in an event a phone draws.
|
||||
fn log_tail(session_dir: &Path) -> String {
|
||||
let Ok(text) = std::fs::read_to_string(session_dir.join(SERVER_LOG)) else {
|
||||
return String::new();
|
||||
};
|
||||
let tail: Vec<&str> = text.lines().rev().take(LOG_TAIL_LINES).collect();
|
||||
if tail.is_empty() {
|
||||
return String::new();
|
||||
}
|
||||
format!(
|
||||
" It last said: {}",
|
||||
tail.into_iter().rev().collect::<Vec<_>>().join(" / ")
|
||||
)
|
||||
}
|
||||
|
||||
/// How much of that log to carry into a message somebody reads on a phone.
|
||||
const LOG_TAIL_LINES: usize = 6;
|
||||
|
||||
/// One streamed completion: posts the conversation, emits each delta as it
|
||||
/// arrives. Emits rather than returns, because the transcript those events land
|
||||
/// in is what the next turn reads back.
|
||||
|
||||
+86
-24
@@ -135,6 +135,17 @@ pub struct SessionInfo {
|
||||
/// that happens to be big" are different answers.
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
pub max_image_edge: Option<u32>,
|
||||
/// Which of `GET /usage`'s snapshots reports on this session, and
|
||||
/// absent where nothing meters it -- see
|
||||
/// [`DriverKind::usage_provider`].
|
||||
///
|
||||
/// Reported for the same reason `keeps_own_transcript` is: it is a
|
||||
/// fact about the provider's *kind*, and the phone has only its name.
|
||||
/// Pairing by machine alone was the bug it exists to fix -- one
|
||||
/// machine runs echo and the Claude CLI, so every echo session drew
|
||||
/// the CLI's five-hour window as if it were its own.
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
pub usage_provider: Option<&'static str>,
|
||||
/// Whether this session announces itself -- reported for the same
|
||||
/// reason `permission_mode` is: a switch that guesses its own position
|
||||
/// is how you turn something off while believing you are reading it.
|
||||
@@ -438,6 +449,7 @@ impl LiveSession {
|
||||
context_tokens: *self.shared.context_tokens.lock().unwrap(),
|
||||
notify: *self.shared.notify.lock().unwrap(),
|
||||
max_image_edge: kind.and_then(DriverKind::max_image_edge),
|
||||
usage_provider: kind.and_then(DriverKind::usage_provider),
|
||||
imported,
|
||||
keeps_own_transcript: kind.is_some_and(DriverKind::keeps_own_transcript),
|
||||
cwd: cwd.map(Path::to_path_buf),
|
||||
@@ -470,6 +482,11 @@ pub struct SessionManager {
|
||||
/// What to mark sessions spawned here as -- see
|
||||
/// [`SessionManager::marking_new_sessions_throwaway`].
|
||||
spawn_throwaway: bool,
|
||||
/// The invented rate-limit answer an echo session's `/usage` sets,
|
||||
/// shared with the usage monitor that serves it. Held here because
|
||||
/// every echo driver this manager builds is handed a clone -- see
|
||||
/// [`SessionManager::reporting_usage_fixture`].
|
||||
usage_fixture: crate::usage::Fixture,
|
||||
inner: RwLock<Inner>,
|
||||
}
|
||||
|
||||
@@ -485,6 +502,11 @@ impl SessionManager {
|
||||
wg_app_link::private::create_dir(&data_dir)?;
|
||||
|
||||
let (notifications, _) = broadcast::channel(NOTIFICATION_BUFFER);
|
||||
// Made here rather than passed in, and handed *out* to the usage
|
||||
// monitor by whoever wires the two together: every echo driver
|
||||
// this manager builds gets a clone, including the ones built
|
||||
// below, so it has to exist before the first session does.
|
||||
let usage_fixture = crate::usage::Fixture::new();
|
||||
let mut live = HashMap::new();
|
||||
for meta in &config.sessions {
|
||||
// One unlaunchable session -- a corrupt transcript, an
|
||||
@@ -495,8 +517,11 @@ impl SessionManager {
|
||||
meta.clone(),
|
||||
&setup,
|
||||
&provider,
|
||||
&data_dir,
|
||||
&models_dir,
|
||||
Env {
|
||||
data_dir: &data_dir,
|
||||
models_dir: &models_dir,
|
||||
usage: &usage_fixture,
|
||||
},
|
||||
notifications.clone(),
|
||||
// Nothing is started here; see `Launching`.
|
||||
Launching::Restart,
|
||||
@@ -517,11 +542,41 @@ impl SessionManager {
|
||||
notifications,
|
||||
pending: Arc::new(pending::Registry::default()),
|
||||
spawn_throwaway: false,
|
||||
usage_fixture,
|
||||
inner: RwLock::new(Inner { config, live }),
|
||||
};
|
||||
Ok(manager)
|
||||
}
|
||||
|
||||
/// Where this backend's own model downloads live. The machine a
|
||||
/// session runs on may keep its elsewhere -- see `models::dir_on`.
|
||||
pub fn models_dir(&self) -> &Path {
|
||||
&self.models_dir
|
||||
}
|
||||
|
||||
/// What this manager lends a session it launches. Borrowed from the
|
||||
/// manager rather than cloned, so there is one answer to where things
|
||||
/// are kept.
|
||||
fn env(&self) -> Env<'_> {
|
||||
Env {
|
||||
data_dir: &self.data_dir,
|
||||
models_dir: &self.models_dir,
|
||||
usage: &self.usage_fixture,
|
||||
}
|
||||
}
|
||||
|
||||
/// The invented rate-limit answer this manager's echo sessions set
|
||||
/// with `/usage`, for the usage monitor to serve.
|
||||
///
|
||||
/// Handed out rather than taken in because the drivers built inside
|
||||
/// the constructor need it, and because the direction is the one the
|
||||
/// layering allows: `usage` sits below the session layer, so a
|
||||
/// session can hold one of its types while it holds nothing of a
|
||||
/// session's.
|
||||
pub fn usage_fixture(&self) -> crate::usage::Fixture {
|
||||
self.usage_fixture.clone()
|
||||
}
|
||||
|
||||
/// Marks every session spawned from here on as one whose process is
|
||||
/// stopped when this server exits. Set from `--throwaway-sessions`,
|
||||
/// which a debug build defaults to on. It decides only what a *new*
|
||||
@@ -861,6 +916,8 @@ impl SessionManager {
|
||||
context_tokens: None,
|
||||
max_image_edge: kind_of(&inner.config, &meta.setup, &meta.provider)
|
||||
.and_then(DriverKind::max_image_edge),
|
||||
usage_provider: kind_of(&inner.config, &meta.setup, &meta.provider)
|
||||
.and_then(DriverKind::usage_provider),
|
||||
notify: meta.notify,
|
||||
imported: import::read_cursor(&self.data_dir.join(&meta.id)).is_some(),
|
||||
keeps_own_transcript: keeps_own_transcript(
|
||||
@@ -974,8 +1031,7 @@ impl SessionManager {
|
||||
meta.clone(),
|
||||
&setup,
|
||||
&provider,
|
||||
&self.data_dir,
|
||||
&self.models_dir,
|
||||
self.env(),
|
||||
self.notifications.clone(),
|
||||
Launching::Asked(seed),
|
||||
)?;
|
||||
@@ -1372,7 +1428,7 @@ impl SessionManager {
|
||||
&meta,
|
||||
&setup,
|
||||
&provider,
|
||||
&self.models_dir,
|
||||
self.env(),
|
||||
session.dir(),
|
||||
session.transcript_path(),
|
||||
&session.sink,
|
||||
@@ -1385,8 +1441,7 @@ impl SessionManager {
|
||||
meta,
|
||||
&setup,
|
||||
&provider,
|
||||
&self.data_dir,
|
||||
&self.models_dir,
|
||||
self.env(),
|
||||
self.notifications.clone(),
|
||||
Launching::Asked(None),
|
||||
)?;
|
||||
@@ -1760,6 +1815,19 @@ enum Launching {
|
||||
Restart,
|
||||
}
|
||||
|
||||
/// What the server around a session lends it: where sessions and models
|
||||
/// are kept, and the usage fixture an echo session's `/usage` sets.
|
||||
///
|
||||
/// One parameter rather than three because they travel together through
|
||||
/// every launch path and none of them is a fact about the session --
|
||||
/// they are this server's belongings, handed down.
|
||||
#[derive(Clone, Copy)]
|
||||
struct Env<'a> {
|
||||
data_dir: &'a Path,
|
||||
models_dir: &'a Path,
|
||||
usage: &'a crate::usage::Fixture,
|
||||
}
|
||||
|
||||
/// Creates the session directory, opens its transcript (continuing the
|
||||
/// sequence numbering if one exists), settles what the session is doing,
|
||||
/// and spawns the event pump -- with a driver behind it where there is a
|
||||
@@ -1768,12 +1836,11 @@ fn launch(
|
||||
meta: SessionConfig,
|
||||
setup: &SetupConfig,
|
||||
provider: &ProviderConfig,
|
||||
data_dir: &Path,
|
||||
models_dir: &Path,
|
||||
env: Env<'_>,
|
||||
notifications: broadcast::Sender<Notification>,
|
||||
why: Launching,
|
||||
) -> Result<Arc<LiveSession>> {
|
||||
let dir = data_dir.join(&meta.id);
|
||||
let dir = env.data_dir.join(&meta.id);
|
||||
wg_app_link::private::create_dir(&dir)?;
|
||||
let transcript_path = dir.join("transcript.jsonl");
|
||||
let mut transcript = Transcript::open(&transcript_path)?;
|
||||
@@ -1887,17 +1954,7 @@ fn launch(
|
||||
|
||||
let driver = Arc::new(Mutex::new(
|
||||
driving
|
||||
.then(|| {
|
||||
make_driver(
|
||||
&meta,
|
||||
setup,
|
||||
provider,
|
||||
models_dir,
|
||||
&dir,
|
||||
&transcript_path,
|
||||
&sink,
|
||||
)
|
||||
})
|
||||
.then(|| make_driver(&meta, setup, provider, env, &dir, &transcript_path, &sink))
|
||||
.transpose()?,
|
||||
));
|
||||
|
||||
@@ -1939,18 +1996,22 @@ fn make_driver(
|
||||
meta: &SessionConfig,
|
||||
setup: &SetupConfig,
|
||||
provider: &ProviderConfig,
|
||||
models_dir: &Path,
|
||||
env: Env<'_>,
|
||||
dir: &Path,
|
||||
transcript_path: &Path,
|
||||
sink: &EventSink,
|
||||
) -> Result<Arc<dyn Driver>> {
|
||||
Ok(match provider.kind {
|
||||
DriverKind::Echo => Arc::new(EchoDriver::new(sink.clone(), dir.to_path_buf())),
|
||||
DriverKind::Echo => Arc::new(EchoDriver::new(
|
||||
sink.clone(),
|
||||
dir.to_path_buf(),
|
||||
env.usage.clone(),
|
||||
)),
|
||||
DriverKind::LlamaCpp => Arc::new(LlamaDriver::launch(
|
||||
meta,
|
||||
provider,
|
||||
&Transport::for_setup(setup),
|
||||
models_dir,
|
||||
env.models_dir,
|
||||
transcript_path,
|
||||
dir,
|
||||
sink.clone(),
|
||||
@@ -2283,6 +2344,7 @@ mod tests {
|
||||
driver: Arc::new(Mutex::new(Some(Arc::new(EchoDriver::new(
|
||||
sink.clone(),
|
||||
dir.path().to_path_buf(),
|
||||
crate::usage::Fixture::new(),
|
||||
))))),
|
||||
sink,
|
||||
waiting: Mutex::new(VecDeque::new()),
|
||||
|
||||
@@ -11,9 +11,13 @@
|
||||
//! `crate::ssh`'s: this module decides *which* transport, that one knows what a
|
||||
//! correct ssh invocation is.
|
||||
//!
|
||||
//! Known second operation, not built because nothing needs it yet: a managed
|
||||
//! `llama-server` is spawned as a process but then spoken to over HTTP, so a
|
||||
//! remote one needs a forwarded port (`ssh -L`) as well.
|
||||
//! A transport is therefore two operations rather than one: **run this** and
|
||||
//! **reach this port**. The second is what a managed `llama-server` needs -- it
|
||||
//! is spawned as a process and then spoken to over HTTP -- and it is a no-op
|
||||
//! locally, where the port a program binds is already one this machine can
|
||||
//! dial. Over ssh it is an `-L` tunnel on the same connection that runs the
|
||||
//! command, so the model server binds loopback on the far machine and is never
|
||||
//! exposed to its network. See [`Transport::reserve_port`].
|
||||
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::process::Stdio;
|
||||
@@ -22,14 +26,26 @@ use anyhow::{Context, Result};
|
||||
use tokio::process::Child;
|
||||
|
||||
use crate::config::SshConfig;
|
||||
pub use crate::ssh::Forward;
|
||||
|
||||
/// What a driver needs run in order to exist as a process. Deliberately just
|
||||
/// the three things every transport can carry; anything a particular machine
|
||||
/// needs is the transport's own configuration, not something a driver states.
|
||||
/// what every transport can carry -- the command, where it runs, and a port the
|
||||
/// caller needs to reach; anything a particular machine needs is the
|
||||
/// transport's own configuration, not something a driver states.
|
||||
pub struct Launch {
|
||||
pub program: String,
|
||||
pub args: Vec<String>,
|
||||
pub cwd: Option<PathBuf>,
|
||||
/// A port this program will listen on, and the port that reaches it
|
||||
/// from here -- see [`Transport::reserve_port`], which is the only
|
||||
/// thing that should produce one.
|
||||
///
|
||||
/// On the launch rather than in [`Transport::spawn`]'s signature
|
||||
/// because it is part of what is being run: a caller that needs to
|
||||
/// reach the process it is starting says so once, where it says
|
||||
/// everything else about it, and every transport reads it the same
|
||||
/// way.
|
||||
pub forward: Option<Forward>,
|
||||
}
|
||||
|
||||
impl Launch {
|
||||
@@ -38,8 +54,16 @@ impl Launch {
|
||||
program: program.into(),
|
||||
args,
|
||||
cwd: cwd.map(Path::to_path_buf),
|
||||
forward: None,
|
||||
}
|
||||
}
|
||||
|
||||
/// Says that this program serves `forward.there`, and that the caller
|
||||
/// will reach it at `forward.here`.
|
||||
pub fn reaching(mut self, forward: Forward) -> Self {
|
||||
self.forward = Some(forward);
|
||||
self
|
||||
}
|
||||
}
|
||||
|
||||
/// How a launched process's standard streams are connected.
|
||||
@@ -102,6 +126,7 @@ impl Transport {
|
||||
&launch.program,
|
||||
&launch.args,
|
||||
launch.cwd.as_deref(),
|
||||
launch.forward,
|
||||
));
|
||||
match streams {
|
||||
Streams::Piped => {
|
||||
@@ -153,12 +178,15 @@ impl Transport {
|
||||
Self::Here => None,
|
||||
Self::Ssh { ssh, .. } => Some(ssh),
|
||||
};
|
||||
let output =
|
||||
crate::ssh::command(host, &launch.program, &launch.args, launch.cwd.as_deref())
|
||||
.output()
|
||||
.with_context(|| {
|
||||
format!("couldn't run \"{}\" {}", launch.program, self.describe())
|
||||
})?;
|
||||
let output = crate::ssh::command(
|
||||
host,
|
||||
&launch.program,
|
||||
&launch.args,
|
||||
launch.cwd.as_deref(),
|
||||
launch.forward,
|
||||
)
|
||||
.output()
|
||||
.with_context(|| format!("couldn't run \"{}\" {}", launch.program, self.describe()))?;
|
||||
if !output.status.success() {
|
||||
let stderr = String::from_utf8_lossy(&output.stderr).trim().to_string();
|
||||
anyhow::bail!(if stderr.is_empty() {
|
||||
@@ -217,6 +245,34 @@ impl Transport {
|
||||
})
|
||||
}
|
||||
|
||||
/// Picks a port for a launched program to serve on, and the port that
|
||||
/// reaches it from here.
|
||||
///
|
||||
/// The "reach this port" half of what a transport is. Locally there is
|
||||
/// one port and the OS chooses it, by binding and letting go -- racy
|
||||
/// in principle, and nothing on this machine is hunting for ports.
|
||||
///
|
||||
/// Over ssh the near end is chosen the same way and the far end is a
|
||||
/// guess, because there is no portable way to ask a machine for a free
|
||||
/// port that does not race with binding it anyway. It is taken from
|
||||
/// [`FAR_PORTS`], below the range Linux hands out to outgoing
|
||||
/// connections, so a collision means something else deliberately
|
||||
/// listening there. That is not silent: the program fails to bind and
|
||||
/// exits, and `session::llama` reports what its log said rather than
|
||||
/// waiting out its readiness timeout.
|
||||
pub fn reserve_port(&self) -> Result<Forward> {
|
||||
let listener = std::net::TcpListener::bind("127.0.0.1:0")
|
||||
.context("asking this machine for a free port")?;
|
||||
let here = listener.local_addr()?.port();
|
||||
Ok(match self {
|
||||
Self::Here => Forward { there: here, here },
|
||||
Self::Ssh { .. } => Forward {
|
||||
there: rand::random_range(FAR_PORTS),
|
||||
here,
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
/// How to say where this runs, for a log line a person reads.
|
||||
pub fn describe(&self) -> String {
|
||||
match self {
|
||||
@@ -226,6 +282,11 @@ impl Transport {
|
||||
}
|
||||
}
|
||||
|
||||
/// Where a port on another machine is guessed from: high enough to be out
|
||||
/// of the way of services, and below the 32768-60999 Linux hands out to
|
||||
/// outgoing connections, which is where a guess would most often collide.
|
||||
const FAR_PORTS: std::ops::Range<u16> = 20000..30000;
|
||||
|
||||
/// What a command is given on its standard input.
|
||||
///
|
||||
/// Three cases rather than an `Option<Stdio>` because they are three genuinely
|
||||
|
||||
@@ -25,7 +25,10 @@ use crate::session::transport::{Launch, Transport};
|
||||
/// session stores.
|
||||
const PROBES: &[(&str, &str, DriverKind)] = &[
|
||||
("claude-cli", "claude", DriverKind::ClaudeCli),
|
||||
("local-llama", "llama-server", DriverKind::LlamaCpp),
|
||||
// Named for the program rather than for where it runs: it runs
|
||||
// wherever the setup is, and "local" was true only while a llama
|
||||
// session could not be spawned on another machine.
|
||||
("llama-cpp", "llama-server", DriverKind::LlamaCpp),
|
||||
];
|
||||
|
||||
/// Models offered for a discovered Claude CLI. A shortcut list for the spawn
|
||||
|
||||
+102
-6
@@ -14,6 +14,22 @@ use std::process::Command;
|
||||
|
||||
use crate::config::SshConfig;
|
||||
|
||||
/// A port on the machine a command runs on, and the port that reaches it from
|
||||
/// the backend.
|
||||
///
|
||||
/// The second half of what a transport is (PLAN.md's SSH section): "run this"
|
||||
/// plus "reach this port". Locally the two numbers are one and nothing is
|
||||
/// forwarded; over ssh the connection carries an `-L` tunnel, so a model server
|
||||
/// binds loopback on the far machine and is never exposed to its network.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub struct Forward {
|
||||
/// What the launched program should listen on, on its own machine.
|
||||
pub there: u16,
|
||||
/// What this machine connects to. The same number as `there` when the
|
||||
/// program runs here.
|
||||
pub here: u16,
|
||||
}
|
||||
|
||||
/// Options forced onto every connection. `BatchMode` makes a missing key fail
|
||||
/// immediately with a readable message instead of hanging on a password prompt
|
||||
/// nothing can answer; the keepalives turn a silently dropped link into a
|
||||
@@ -39,6 +55,7 @@ pub fn command(
|
||||
program: &str,
|
||||
args: &[String],
|
||||
cwd: Option<&Path>,
|
||||
forward: Option<Forward>,
|
||||
) -> Command {
|
||||
let Some(ssh) = remote else {
|
||||
let mut command = Command::new(program);
|
||||
@@ -55,9 +72,31 @@ pub fn command(
|
||||
};
|
||||
|
||||
let mut command = Command::new("ssh");
|
||||
// -T: no pty. This carries JSONL, and a pty would rewrite it (echo, CRLF
|
||||
// translation, ^C handling) into something the parser can't read.
|
||||
command.arg("-T");
|
||||
if let Some(forward) = forward {
|
||||
// A forwarded process is not spoken to over stdio, and that changes how
|
||||
// it is shut down. Everything else here is a CLI reading its stdin, so
|
||||
// killing the ssh client ends it; a `llama-server` never reads its own,
|
||||
// so the same kill left it running on the far machine with the model
|
||||
// loaded -- measured 2026-09-04, an orphan per stopped session. A pty
|
||||
// is what makes sshd hang the far side up. `-tt` because this client
|
||||
// has no terminal to inherit one from. The cost is a log that arrives
|
||||
// through a line discipline, which nothing parses.
|
||||
command.arg("-tt");
|
||||
// Loopback at both ends: the far side binds 127.0.0.1, so what it
|
||||
// serves is reachable only through this connection.
|
||||
command.args([
|
||||
"-L",
|
||||
&format!("127.0.0.1:{}:127.0.0.1:{}", forward.here, forward.there),
|
||||
]);
|
||||
// Without this a forward that cannot be set up is a warning on stderr
|
||||
// and a session that runs anyway, answering nothing -- which would
|
||||
// arrive as "the model never became ready".
|
||||
command.args(["-o", "ExitOnForwardFailure=yes"]);
|
||||
} else {
|
||||
// -T: no pty. This carries JSONL, and a pty would rewrite it (echo,
|
||||
// CRLF translation, ^C handling) into something the parser can't read.
|
||||
command.arg("-T");
|
||||
}
|
||||
for option in SSH_OPTIONS {
|
||||
command.args(["-o", option]);
|
||||
}
|
||||
@@ -181,6 +220,7 @@ mod tests {
|
||||
port: None,
|
||||
identity_file: None,
|
||||
options: vec![],
|
||||
models_dir: None,
|
||||
attachments_dir: None,
|
||||
}
|
||||
}
|
||||
@@ -192,6 +232,7 @@ mod tests {
|
||||
"claude",
|
||||
&args(["-p", "--verbose"]),
|
||||
Some(Path::new("/tmp/x")),
|
||||
None,
|
||||
);
|
||||
assert_eq!(argv(&command), ["claude", "-p", "--verbose"]);
|
||||
assert_eq!(command.get_current_dir(), Some(Path::new("/tmp/x")));
|
||||
@@ -204,6 +245,7 @@ mod tests {
|
||||
port: Some(2222),
|
||||
identity_file: Some("/home/me/.ssh/id_ai".into()),
|
||||
options: vec!["StrictHostKeyChecking=accept-new".to_string()],
|
||||
models_dir: None,
|
||||
attachments_dir: None,
|
||||
};
|
||||
let rendered = argv(&command(
|
||||
@@ -211,6 +253,7 @@ mod tests {
|
||||
"claude",
|
||||
&args(["-p", "--model", "haiku"]),
|
||||
Some(Path::new("/home/bob/work")),
|
||||
None,
|
||||
));
|
||||
|
||||
assert_eq!(rendered[0], "ssh");
|
||||
@@ -231,12 +274,59 @@ mod tests {
|
||||
#[test]
|
||||
fn a_remote_command_without_a_cwd_just_execs() {
|
||||
let ssh = bare_host();
|
||||
let rendered = argv(&command(Some(&ssh), "claude", &args(["-p"]), None));
|
||||
let rendered = argv(&command(Some(&ssh), "claude", &args(["-p"]), None, None));
|
||||
assert_eq!(rendered.last().unwrap(), "exec 'claude' '-p'");
|
||||
// No -i means no IdentitiesOnly: ~/.ssh/config decides instead.
|
||||
assert!(!rendered.contains(&"IdentitiesOnly=yes".to_string()));
|
||||
}
|
||||
|
||||
/// The second half of a transport: the connection that runs the command also
|
||||
/// carries the port that reaches it.
|
||||
///
|
||||
/// Both ends are pinned to loopback, which is what keeps a model server off
|
||||
/// the far machine's network -- asserted rather than trusted, because
|
||||
/// dropping the addresses is a one-word edit that still works on a machine
|
||||
/// nobody else can reach.
|
||||
#[test]
|
||||
fn a_forwarded_port_rides_the_same_connection_as_the_command() {
|
||||
let ssh = bare_host();
|
||||
let rendered = argv(&command(
|
||||
Some(&ssh),
|
||||
"llama-server",
|
||||
&args(["--port", "24242"]),
|
||||
None,
|
||||
Some(Forward {
|
||||
there: 24242,
|
||||
here: 41000,
|
||||
}),
|
||||
));
|
||||
let forward = rendered
|
||||
.iter()
|
||||
.position(|arg| arg == "-L")
|
||||
.expect("a forward");
|
||||
assert_eq!(rendered[forward + 1], "127.0.0.1:41000:127.0.0.1:24242");
|
||||
assert!(rendered.contains(&"ExitOnForwardFailure=yes".to_string()));
|
||||
// The half that is easy to lose: without a pty the far process outlives
|
||||
// the connection, because nothing closes a stdin it never reads.
|
||||
assert!(rendered.contains(&"-tt".to_string()));
|
||||
assert!(!rendered.contains(&"-T".to_string()));
|
||||
// Options come before the host, or ssh reads them as part of the
|
||||
// remote command.
|
||||
assert!(forward < rendered.len() - 2);
|
||||
assert_eq!(
|
||||
rendered.last().unwrap(),
|
||||
"exec 'llama-server' '--port' '24242'"
|
||||
);
|
||||
|
||||
// Nothing forwarded is nothing added: every other session is one of
|
||||
// these, and an -L on it would bind a port for no reason.
|
||||
let plain = argv(&command(Some(&ssh), "claude", &args(["-p"]), None, None));
|
||||
assert!(!plain.contains(&"-L".to_string()));
|
||||
// And a session that *is* spoken to over stdio keeps its raw pipe.
|
||||
assert!(plain.contains(&"-T".to_string()));
|
||||
assert!(!plain.contains(&"-tt".to_string()));
|
||||
}
|
||||
|
||||
/// The one character quoting must not swallow. A working directory typed as
|
||||
/// `~/repos/ai-app` was arriving as the literal directory `~`, and the
|
||||
/// remote shell reported it missing -- which reads as the path being wrong
|
||||
@@ -277,7 +367,13 @@ mod tests {
|
||||
assert_eq!(expand_home(Path::new("/tmp/~/x")), Path::new("/tmp/~/x"));
|
||||
assert_eq!(expand_home(Path::new("~user/x")), Path::new("~user/x"));
|
||||
|
||||
let local = command(None, "claude", &args(["-p"]), Some(Path::new("~/work")));
|
||||
let local = command(
|
||||
None,
|
||||
"claude",
|
||||
&args(["-p"]),
|
||||
Some(Path::new("~/work")),
|
||||
None,
|
||||
);
|
||||
assert_eq!(local.get_current_dir(), Some(home.join("work").as_path()));
|
||||
}
|
||||
|
||||
@@ -301,7 +397,7 @@ mod tests {
|
||||
// tries to close the quote and start a new command.
|
||||
let ssh = bare_host();
|
||||
let evil = Path::new("/tmp/'; touch /tmp/pwned; '");
|
||||
let rendered = argv(&command(Some(&ssh), "claude", &[], Some(evil)));
|
||||
let rendered = argv(&command(Some(&ssh), "claude", &[], Some(evil), None));
|
||||
let script = rendered.last().unwrap();
|
||||
assert_eq!(
|
||||
script,
|
||||
|
||||
+340
-23
@@ -30,13 +30,13 @@
|
||||
//! already has over that machine.
|
||||
|
||||
use std::collections::HashMap;
|
||||
use std::sync::Mutex;
|
||||
use std::sync::{Arc, Mutex};
|
||||
use std::time::{Duration, Instant};
|
||||
|
||||
use serde::Serialize;
|
||||
use serde_json::Value;
|
||||
|
||||
use crate::config::{DriverKind, SetupConfig};
|
||||
use crate::config::SetupConfig;
|
||||
use crate::session::transport::{Launch, Transport};
|
||||
|
||||
const USAGE_URL: &str = "https://api.anthropic.com/api/oauth/usage";
|
||||
@@ -107,10 +107,30 @@ pub struct UsageSnapshot {
|
||||
pub fetched_at: f64,
|
||||
}
|
||||
|
||||
/// The name of each meter, said in one place because two lists have to
|
||||
/// agree on it: [`UsageSnapshot::provider`], which is what `GET /usage`
|
||||
/// labels a row with, and [`crate::config::DriverKind::usage_provider`],
|
||||
/// which is how a session says which of those rows is about it.
|
||||
pub const CLAUDE: &str = "claude";
|
||||
/// The invented one, for testing the screens that draw these -- see
|
||||
/// [`Fixture`].
|
||||
pub const ECHO: &str = "echo";
|
||||
|
||||
pub trait UsageProvider: Send + Sync {
|
||||
fn name(&self) -> &'static str;
|
||||
/// Blocking -- call off the async workers.
|
||||
fn fetch(&self) -> UsageSnapshot;
|
||||
/// How long an answer from this one may be reused.
|
||||
///
|
||||
/// A property of the provider rather than of the cache, because what
|
||||
/// sets it is what asking costs: [`ClaudeUsage`] makes a network call
|
||||
/// against an endpoint that rate-limits impatient callers, and the
|
||||
/// fixture below reads a mutex. Caching the fixture for three minutes
|
||||
/// would mean a test setting a number and watching the old one for
|
||||
/// most of that, which reads exactly like the command not working.
|
||||
fn poll_interval(&self) -> Duration {
|
||||
MIN_POLL_INTERVAL
|
||||
}
|
||||
}
|
||||
|
||||
/// Reads the numbers behind Claude Code's `/usage` from one machine, using the
|
||||
@@ -170,7 +190,7 @@ impl ClaudeUsage {
|
||||
|
||||
impl UsageProvider for ClaudeUsage {
|
||||
fn name(&self) -> &'static str {
|
||||
"claude"
|
||||
CLAUDE
|
||||
}
|
||||
|
||||
fn fetch(&self) -> UsageSnapshot {
|
||||
@@ -278,24 +298,259 @@ fn parse_windows(body: &Value) -> Vec<UsageWindow> {
|
||||
.collect()
|
||||
}
|
||||
|
||||
/// An invented answer, so the screens that draw these can be exercised
|
||||
/// without an account.
|
||||
///
|
||||
/// Every state the usage bar and the usage dialog can be in is otherwise
|
||||
/// reachable only by spending somebody's quota or by breaking a machine:
|
||||
/// a number near the top, a machine nobody has logged into, one that
|
||||
/// cannot be reached, a window between blocks with no reset time. Those
|
||||
/// are exactly the states worth looking at, and the ones nobody looks at
|
||||
/// because arranging them costs real turns. An echo session sets this
|
||||
/// with `/usage` (see `session::echo`), which is the same bargain the
|
||||
/// rest of that driver makes: the fixture is invented, what is real is
|
||||
/// the path it travels.
|
||||
///
|
||||
/// Shared by the session layer, which writes it, and [`UsageMonitor`],
|
||||
/// which reads it. Empty until something sets it, and an empty fixture
|
||||
/// produces no snapshot at all -- an echo session meters nothing, and
|
||||
/// nothing is what the phone should draw.
|
||||
#[derive(Clone, Default)]
|
||||
pub struct Fixture {
|
||||
said: Arc<Mutex<Option<Reported>>>,
|
||||
}
|
||||
|
||||
/// What a meter answered: which of the four states it is in, and whatever
|
||||
/// windows go with it. Empty for every state but [`UsageState::Ok`].
|
||||
type Reported = (UsageState, Vec<UsageWindow>);
|
||||
|
||||
/// How long the invented five-hour window has left, when nothing says.
|
||||
const FIXTURE_MINUTES: i64 = 125;
|
||||
|
||||
impl Fixture {
|
||||
pub fn new() -> Self {
|
||||
Self::default()
|
||||
}
|
||||
|
||||
fn is_set(&self) -> bool {
|
||||
self.said.lock().unwrap().is_some()
|
||||
}
|
||||
|
||||
fn read(&self) -> Option<Reported> {
|
||||
self.said.lock().unwrap().clone()
|
||||
}
|
||||
|
||||
/// Acts on the words typed after `/usage`, and says what it did.
|
||||
///
|
||||
/// The vocabulary lives here rather than in the echo driver because
|
||||
/// these are this module's states: a driver spelling them out would
|
||||
/// be a second place that has to learn about a fifth one.
|
||||
pub fn command(&self, words: &str) -> String {
|
||||
let mut words = words.split_whitespace();
|
||||
let Some(first) = words.next() else {
|
||||
return match self.read() {
|
||||
Some((state, windows)) => format!("usage fixture: {}", describe(&state, &windows)),
|
||||
None => "usage fixture: unset, so this session meters nothing. \
|
||||
`/usage 42` puts up a five-hour window at 42%."
|
||||
.to_string(),
|
||||
};
|
||||
};
|
||||
let rest: Vec<&str> = words.collect();
|
||||
let detail = || {
|
||||
if rest.is_empty() {
|
||||
"set by /usage".to_string()
|
||||
} else {
|
||||
rest.join(" ")
|
||||
}
|
||||
};
|
||||
let (state, windows) = match first {
|
||||
"off" | "none" | "clear" => {
|
||||
*self.said.lock().unwrap() = None;
|
||||
return "usage fixture cleared: this session meters nothing again".to_string();
|
||||
}
|
||||
"notloggedin" | "logged-out" => (UsageState::NotLoggedIn, Vec::new()),
|
||||
"unreachable" => (UsageState::Unreachable { detail: detail() }, Vec::new()),
|
||||
"failed" => (UsageState::Failed { detail: detail() }, Vec::new()),
|
||||
percent => match percent.parse::<f64>() {
|
||||
Ok(percent) => (
|
||||
UsageState::Ok,
|
||||
fixture_windows(percent.clamp(0.0, 100.0), rest.first().copied()),
|
||||
),
|
||||
Err(_) => {
|
||||
return format!(
|
||||
"\"{percent}\" is not one of this fixture's answers. Say a percentage \
|
||||
(`/usage 42`, optionally with `90` minutes left, `never` for a window \
|
||||
between blocks, or `unreadable` for a reset time that cannot be read), \
|
||||
or one of `notloggedin`, `unreachable`, `failed`, `off`."
|
||||
);
|
||||
}
|
||||
},
|
||||
};
|
||||
let said = describe(&state, &windows);
|
||||
*self.said.lock().unwrap() = Some((state, windows));
|
||||
format!("usage fixture set: {said}")
|
||||
}
|
||||
}
|
||||
|
||||
/// The three windows Claude reports today, invented around one number.
|
||||
///
|
||||
/// Three rather than one because the bar under a session header reads the
|
||||
/// five-hour window and the dialog behind the button draws all of them,
|
||||
/// and a fixture with one window leaves half the screen untested. The
|
||||
/// weekly ones are derived from the same figure so that the worst of them
|
||||
/// -- which is what colours the button -- is still the one asked for.
|
||||
fn fixture_windows(percent: f64, reset: Option<&str>) -> Vec<UsageWindow> {
|
||||
let resets_at = match reset {
|
||||
// The state a real response is in between blocks: there is no
|
||||
// window running, so there is nothing to reset. It is not a
|
||||
// missing value, and the phone words it differently.
|
||||
Some("never") | Some("none") => None,
|
||||
// A timestamp that arrives and cannot be read, which is the one
|
||||
// case that really is "we could not find out".
|
||||
Some("unreadable") | Some("bad") => Some("whenever it feels like it".to_string()),
|
||||
other => Some(reset_in(
|
||||
other
|
||||
.and_then(|word| word.parse().ok())
|
||||
.unwrap_or(FIXTURE_MINUTES),
|
||||
)),
|
||||
};
|
||||
vec![
|
||||
UsageWindow {
|
||||
kind: "session".to_string(),
|
||||
label: "5-hour window".to_string(),
|
||||
percent,
|
||||
resets_at: resets_at.clone(),
|
||||
active: true,
|
||||
},
|
||||
UsageWindow {
|
||||
kind: "weekly_all".to_string(),
|
||||
label: "Weekly (all models)".to_string(),
|
||||
percent: percent / 2.0,
|
||||
resets_at: resets_at.as_ref().map(|_| reset_in(FIXTURE_MINUTES * 40)),
|
||||
active: false,
|
||||
},
|
||||
UsageWindow {
|
||||
kind: "weekly_scoped".to_string(),
|
||||
label: "Weekly (Echo)".to_string(),
|
||||
percent: percent / 4.0,
|
||||
resets_at: resets_at.as_ref().map(|_| reset_in(FIXTURE_MINUTES * 40)),
|
||||
active: false,
|
||||
},
|
||||
]
|
||||
}
|
||||
|
||||
/// `minutes` from now, in the format the real endpoint sends.
|
||||
fn reset_in(minutes: i64) -> String {
|
||||
let at = time::OffsetDateTime::now_utc() + time::Duration::minutes(minutes);
|
||||
at.format(&time::format_description::well_known::Rfc3339)
|
||||
// Formatting a timestamp cannot fail for any input this builds;
|
||||
// saying so beats a fixture that silently has no reset time.
|
||||
.unwrap_or_else(|_| "unformattable".to_string())
|
||||
}
|
||||
|
||||
/// One line naming what a fixture is currently claiming, for the reply
|
||||
/// the echo session writes back.
|
||||
fn describe(state: &UsageState, windows: &[UsageWindow]) -> String {
|
||||
match state {
|
||||
UsageState::Ok => match windows.first() {
|
||||
Some(window) => format!(
|
||||
"{}% of the five-hour window, {}",
|
||||
window.percent,
|
||||
match &window.resets_at {
|
||||
Some(at) => format!("resetting at {at}"),
|
||||
None => "with no reset time (the between-blocks state)".to_string(),
|
||||
}
|
||||
),
|
||||
None => "no windows at all".to_string(),
|
||||
},
|
||||
UsageState::NotLoggedIn => "nobody is logged in on this machine".to_string(),
|
||||
UsageState::Unreachable { detail } => format!("machine unreachable ({detail})"),
|
||||
UsageState::Failed { detail } => format!("the meter failed ({detail})"),
|
||||
}
|
||||
}
|
||||
|
||||
/// The fixture, as a provider, so it travels the same route and the same
|
||||
/// cache as a real meter rather than being spliced in at the screen.
|
||||
struct EchoUsage {
|
||||
setup: String,
|
||||
setup_name: String,
|
||||
fixture: Fixture,
|
||||
}
|
||||
|
||||
impl UsageProvider for EchoUsage {
|
||||
fn name(&self) -> &'static str {
|
||||
ECHO
|
||||
}
|
||||
|
||||
fn fetch(&self) -> UsageSnapshot {
|
||||
let (state, windows) = self
|
||||
.fixture
|
||||
.read()
|
||||
// Only ever built for a fixture that is set; a race with
|
||||
// `/usage off` between the two reads lands here, and "the
|
||||
// machine could not be asked" is the honest word for it.
|
||||
.unwrap_or((
|
||||
UsageState::Unreachable {
|
||||
detail: "the usage fixture was cleared".to_string(),
|
||||
},
|
||||
Vec::new(),
|
||||
));
|
||||
UsageSnapshot {
|
||||
provider: self.name().to_string(),
|
||||
setup: self.setup.clone(),
|
||||
setup_name: self.setup_name.clone(),
|
||||
state,
|
||||
windows,
|
||||
fetched_at: crate::session::now(),
|
||||
}
|
||||
}
|
||||
|
||||
/// Read from memory, and set by somebody who is about to look at the
|
||||
/// screen it changes.
|
||||
fn poll_interval(&self) -> Duration {
|
||||
Duration::ZERO
|
||||
}
|
||||
}
|
||||
|
||||
/// Which paid services a machine can be asked about.
|
||||
///
|
||||
/// Derived from what the setup says it can run, so a machine with no Claude
|
||||
/// provider is not asked about Claude limits -- it has none, and a row saying
|
||||
/// so would be a fact about nothing. A second service later adds a branch here
|
||||
/// and an impl beside [`ClaudeUsage`], not a screen.
|
||||
fn providers_for(setup: &SetupConfig) -> Vec<Box<dyn UsageProvider>> {
|
||||
/// so would be a fact about nothing.
|
||||
///
|
||||
/// Which meter a provider has is [`DriverKind::usage_provider`]'s answer rather
|
||||
/// than a second match on kinds here, because the phone pairs a session with
|
||||
/// one of these rows by that same name: two lists that disagreed would leave a
|
||||
/// session looking for a snapshot nothing produces. A second service later is a
|
||||
/// name there and an impl beside [`ClaudeUsage`], not a screen.
|
||||
fn providers_for(setup: &SetupConfig, fixture: &Fixture) -> Vec<Box<dyn UsageProvider>> {
|
||||
let mut found: Vec<Box<dyn UsageProvider>> = Vec::new();
|
||||
if setup
|
||||
.providers
|
||||
.iter()
|
||||
.any(|provider| provider.kind == DriverKind::ClaudeCli)
|
||||
{
|
||||
found.push(Box::new(ClaudeUsage {
|
||||
setup: setup.id.clone(),
|
||||
setup_name: setup.name.clone(),
|
||||
transport: Transport::for_setup(setup),
|
||||
}));
|
||||
for provider in &setup.providers {
|
||||
let Some(name) = provider.kind.usage_provider() else {
|
||||
continue;
|
||||
};
|
||||
// A machine offering two Claude providers has one account, not
|
||||
// two: the meter belongs to the machine and the service, which is
|
||||
// exactly what the cache is keyed by.
|
||||
if found.iter().any(|already| already.name() == name) {
|
||||
continue;
|
||||
}
|
||||
match name {
|
||||
CLAUDE => found.push(Box::new(ClaudeUsage {
|
||||
setup: setup.id.clone(),
|
||||
setup_name: setup.name.clone(),
|
||||
transport: Transport::for_setup(setup),
|
||||
})),
|
||||
// Nothing at all until a test has asked for something: an
|
||||
// echo session costs nothing, so the honest answer is no row
|
||||
// rather than a row saying zero.
|
||||
ECHO if fixture.is_set() => found.push(Box::new(EchoUsage {
|
||||
setup: setup.id.clone(),
|
||||
setup_name: setup.name.clone(),
|
||||
fixture: fixture.clone(),
|
||||
})),
|
||||
_ => {}
|
||||
}
|
||||
}
|
||||
found
|
||||
}
|
||||
@@ -313,11 +568,18 @@ type Cached = HashMap<(String, &'static str), (Instant, UsageSnapshot)>;
|
||||
/// machine per service per [`MIN_POLL_INTERVAL`], however often the phone asks.
|
||||
pub struct UsageMonitor {
|
||||
cache: Mutex<Cached>,
|
||||
/// The invented meter an echo session can put up; empty unless one
|
||||
/// has. Shared with the session layer, which is where the command
|
||||
/// that sets it is typed -- see [`Fixture`].
|
||||
fixture: Fixture,
|
||||
}
|
||||
|
||||
impl UsageMonitor {
|
||||
pub fn new() -> Self {
|
||||
Self::default()
|
||||
pub fn new(fixture: Fixture) -> Self {
|
||||
Self {
|
||||
cache: Mutex::new(Cached::new()),
|
||||
fixture,
|
||||
}
|
||||
}
|
||||
|
||||
/// One snapshot per machine that offers a paid service, in the order the
|
||||
@@ -329,10 +591,10 @@ impl UsageMonitor {
|
||||
pub fn snapshots(&self, setups: &[SetupConfig]) -> Vec<UsageSnapshot> {
|
||||
let mut fresh = Vec::new();
|
||||
for setup in setups {
|
||||
for provider in providers_for(setup) {
|
||||
for provider in providers_for(setup, &self.fixture) {
|
||||
let key = (setup.id.clone(), provider.name());
|
||||
if let Some((fetched, snapshot)) = self.cache.lock().unwrap().get(&key)
|
||||
&& fetched.elapsed() < MIN_POLL_INTERVAL
|
||||
&& fetched.elapsed() < provider.poll_interval()
|
||||
{
|
||||
// Cached numbers, but the machine's *name* is read fresh: a
|
||||
// rename should show immediately rather than waiting out a
|
||||
@@ -367,6 +629,7 @@ impl UsageMonitor {
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::config::DriverKind;
|
||||
|
||||
#[test]
|
||||
fn parses_the_limits_array_defensively() {
|
||||
@@ -404,6 +667,7 @@ mod tests {
|
||||
port: None,
|
||||
identity_file: None,
|
||||
options: vec!["ConnectTimeout=1".to_string()],
|
||||
models_dir: None,
|
||||
attachments_dir: None,
|
||||
}),
|
||||
providers: vec![crate::config::ProviderConfig {
|
||||
@@ -475,9 +739,62 @@ mod tests {
|
||||
models: vec![],
|
||||
}];
|
||||
// A machine with no Claude on it has no Claude limits, and a row
|
||||
// reporting on it would be a fact about nothing.
|
||||
assert!(providers_for(&echo_only).is_empty());
|
||||
assert_eq!(providers_for(&unreachable_setup()).len(), 1);
|
||||
// reporting on it would be a fact about nothing. Echo included:
|
||||
// an echo session spends nothing, so until a fixture says
|
||||
// otherwise there is no meter to report.
|
||||
let unset = Fixture::new();
|
||||
assert!(providers_for(&echo_only, &unset).is_empty());
|
||||
assert_eq!(providers_for(&unreachable_setup(), &unset).len(), 1);
|
||||
|
||||
// And with one set, that machine has exactly the invented meter
|
||||
// -- under the name the session's `usageProvider` will name.
|
||||
let fixture = Fixture::new();
|
||||
fixture.command("42");
|
||||
let found = providers_for(&echo_only, &fixture);
|
||||
assert_eq!(found.len(), 1);
|
||||
assert_eq!(found[0].name(), ECHO);
|
||||
assert_eq!(DriverKind::Echo.usage_provider(), Some(ECHO));
|
||||
assert_eq!(DriverKind::ClaudeCli.usage_provider(), Some(CLAUDE));
|
||||
// A local model costs nothing to run, so it meters nothing.
|
||||
assert_eq!(DriverKind::LlamaCpp.usage_provider(), None);
|
||||
}
|
||||
|
||||
/// The states the fixture exists to make reachable, and the one thing
|
||||
/// it must not do: invent a reset time for a window that has none.
|
||||
#[test]
|
||||
fn the_fixture_says_each_state_the_screens_have_to_draw() {
|
||||
let fixture = Fixture::new();
|
||||
assert!(fixture.read().is_none(), "unset until somebody sets it");
|
||||
|
||||
fixture.command("42 90");
|
||||
let (state, windows) = fixture.read().expect("set");
|
||||
assert_eq!(state, UsageState::Ok);
|
||||
assert_eq!(windows[0].kind, "session");
|
||||
assert_eq!(windows[0].percent, 42.0);
|
||||
assert!(windows[0].resets_at.is_some());
|
||||
|
||||
// Between blocks: no reset time, which the phone words as the
|
||||
// window not running rather than as a time it could not read.
|
||||
fixture.command("42 never");
|
||||
assert_eq!(fixture.read().expect("set").1[0].resets_at, None);
|
||||
|
||||
fixture.command("unreachable no route to host");
|
||||
assert!(matches!(
|
||||
fixture.read().expect("set").0,
|
||||
UsageState::Unreachable { detail } if detail == "no route to host",
|
||||
));
|
||||
|
||||
fixture.command("off");
|
||||
assert!(fixture.read().is_none());
|
||||
|
||||
// A word it does not know changes nothing and says what it takes.
|
||||
fixture.command("42");
|
||||
let refused = fixture.command("sideways");
|
||||
assert!(
|
||||
refused.contains("not one of this fixture's answers"),
|
||||
"{refused}"
|
||||
);
|
||||
assert_eq!(fixture.read().expect("still set").1[0].percent, 42.0);
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
||||
Reference in new issue
Block a user