Compare commits
3
Commits
1fcaa2d72d
...
e4f0935f98
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
e4f0935f98 | ||
|
|
3c0214ece8 | ||
|
|
127b25e60a |
No files matched your search
@@ -26,6 +26,19 @@ Module-by-module intent is in PLAN.md's "Backend layout".
|
|||||||
|
|
||||||
- `server/` — the Rust backend (`ai-server`). `routes.rs`'s module doc
|
- `server/` — the Rust backend (`ai-server`). `routes.rs`'s module doc
|
||||||
comment is the HTTP table and the surface's source of truth.
|
comment is the HTTP table and the surface's source of truth.
|
||||||
|
**A llama.cpp session runs on whatever machine its setup names** (built
|
||||||
|
2026-09-04, the last of phase 5): `Transport::reserve_port` returns the
|
||||||
|
port the server binds *there* and the port that reaches it *here*, and
|
||||||
|
`Launch::reaching` puts the `-L` tunnel on the connection already carrying
|
||||||
|
the command. Three things fell out of it and are easy to get wrong again —
|
||||||
|
a forwarded launch gets a pty (`-tt`) and every other one keeps `-T`,
|
||||||
|
because `llama-server` never reads the stdin whose closing ends a CLI and
|
||||||
|
the same kill left it loaded on the far machine; the model is looked for on
|
||||||
|
the machine that will serve it, so the spawn screen offers
|
||||||
|
`GET /setups/{id}/models` rather than the backend's own downloads; and the
|
||||||
|
readiness poll watches the process as well as the port, since a model that
|
||||||
|
will not load exits in a second and was being reported as "gave up after
|
||||||
|
300s". See PLAN.md's "Transport" and "llama-server management".
|
||||||
- `app/` — the Compose app, package `com.example.aiapp`, label "AI Sessions".
|
- `app/` — the Compose app, package `com.example.aiapp`, label "AI Sessions".
|
||||||
`AppRoot.kt` is the navigation `when`; `MainScreen.kt` the root's four tabs
|
`AppRoot.kt` is the navigation `when`; `MainScreen.kt` the root's four tabs
|
||||||
(sessions, import, models, setups); `Api.kt`/`EventStream.kt` the REST + SSE
|
(sessions, import, models, setups); `Api.kt`/`EventStream.kt` the REST + SSE
|
||||||
@@ -193,6 +206,13 @@ Each exists because something was invisible without it.
|
|||||||
conversation somebody may still be in. **A transcript never goes in this
|
conversation somebody may still be in. **A transcript never goes in this
|
||||||
repository**: they hold whatever was said, read and written in that
|
repository**: they hold whatever was said, read and written in that
|
||||||
session, and `~/repos` is shared with the host besides.
|
session, and `~/repos` is shared with the host besides.
|
||||||
|
- **`/usage` in an echo session puts up an invented meter**, which is how the
|
||||||
|
rate-limit screens' states are reached without spending quota: `/usage 42`,
|
||||||
|
`/usage 95 20` (minutes left), `/usage 42 never` (the between-blocks window
|
||||||
|
with no reset time), `/usage 42 unreadable`, `/usage notloggedin`,
|
||||||
|
`/usage unreachable`, `/usage failed`, `/usage off`. The vocabulary is
|
||||||
|
`usage::Fixture`'s, since those are its states. With none set an echo
|
||||||
|
session meters nothing, which is the ordinary case and draws no bar.
|
||||||
- **A fake CLI exercises the process lifecycle without a token.** Point a
|
- **A fake CLI exercises the process lifecycle without a token.** Point a
|
||||||
`claude_cli` provider's `command` at a two-line script — `#!/bin/sh` and
|
`claude_cli` provider's `command` at a two-line script — `#!/bin/sh` and
|
||||||
`cat > /dev/null` — and it behaves the way the lifecycle code cares about:
|
`cat > /dev/null` — and it behaves the way the lifecycle code cares about:
|
||||||
@@ -263,24 +283,32 @@ moment you use it — `ANDROID_SERIAL=$(emu serial) ./gradlew …`.
|
|||||||
|
|
||||||
### Testing llama.cpp and ssh here
|
### Testing llama.cpp and ssh here
|
||||||
|
|
||||||
The prebuilt CPU llama.cpp lives outside the repo at
|
**Both are set up here as of 2026-09-04** and need nothing typed. The
|
||||||
`~/.local/opt/llama.cpp` (the 15 MB `ubuntu-x64` release asset). It needs its
|
prebuilt CPU llama.cpp lives outside the repo at `~/.local/opt/llama.cpp`
|
||||||
own directory on `LD_LIBRARY_PATH`, so start the server as
|
(the 15 MB `ubuntu-x64` release asset) and is symlinked as
|
||||||
`LD_LIBRARY_PATH=~/.local/opt/llama.cpp ai-server …` and point a provider's
|
`/usr/local/bin/llama-server`, which is what makes **discovery find it over
|
||||||
`command` at `~/.local/opt/llama.cpp/llama-server`. A 0.6B Q8_0 answers at
|
ssh**: `~/.local/bin` is not on the PATH a non-interactive ssh session gets.
|
||||||
usable speed on this VM's 8 cores. **Do not test with a 2-bit quant**: the
|
It resolves its own libraries through `$ORIGIN`, so no `LD_LIBRARY_PATH` is
|
||||||
|
needed. One model is downloaded — `unsloth/Qwen3-0.6B-GGUF/Qwen3-0.6B-Q8_0.gguf`,
|
||||||
|
639 MB under `~/.local/share/ai-app/models` — and answers at usable speed on
|
||||||
|
this VM's 8 cores. **Do not test with a 2-bit quant**: the
|
||||||
IQ2_XXS of that model produces fluent nonsense, which reads exactly like a
|
IQ2_XXS of that model produces fluent nonsense, which reads exactly like a
|
||||||
broken driver — `llama-cli` produces the same from the file directly, which
|
broken driver — `llama-cli` produces the same from the file directly, which
|
||||||
is how to tell the two apart in a hurry.
|
is how to tell the two apart in a hurry.
|
||||||
|
|
||||||
There is no second machine, so **ssh this VM to itself**: generate a
|
There is no second machine, so **ssh this VM to itself**. That is set up
|
||||||
throwaway key, append the public half to `~/.ssh/authorized_keys`, and
|
too: the key is `~/.config/ai-app/ssh-self` (its public half is in
|
||||||
configure a host of `bob@127.0.0.1` with `identityFile` pointing at it plus
|
`~/.ssh/authorized_keys`, labelled removable), and the real config carries a
|
||||||
`options: ["StrictHostKeyChecking=no", "UserKnownHostsFile=…"]` so it touches
|
setup called **"this vm over ssh"** — `bob@127.0.0.1` with that
|
||||||
nothing real. Point a provider's `command` at something harmless like
|
`identityFile` plus
|
||||||
`/bin/echo` rather than at `claude`: the transport is what is under test, the
|
`options: ["StrictHostKeyChecking=no", "UserKnownHostsFile=/tmp/ai-app-known-hosts"]`
|
||||||
process exiting immediately is the signal, and it costs no tokens. **Take the
|
so it touches nothing real — offering `claude-cli` and `llama-cpp`. It is the
|
||||||
key back out afterwards.** The remote login shell here is **fish**; the
|
whole rig for "does a remote llama session work", since the far machine is
|
||||||
|
this one and the model file is the same file. For a throwaway setup of your
|
||||||
|
own, point a provider's `command` at something harmless like `/bin/echo`
|
||||||
|
rather than at `claude`: the transport is what is under test, the process
|
||||||
|
exiting immediately is the signal, and it costs no tokens. The remote login
|
||||||
|
shell here is **fish**; the
|
||||||
remote script and `ssh.rs`'s POSIX quoting happen to mean the same thing in
|
remote script and `ssh.rs`'s POSIX quoting happen to mean the same thing in
|
||||||
both, but that is luck rather than design, and a shell that is neither is the
|
both, but that is luck rather than design, and a shell that is neither is the
|
||||||
thing to suspect first if a remote spawn ever mangles an argument.
|
thing to suspect first if a remote spawn ever mangles an argument.
|
||||||
@@ -379,6 +407,17 @@ where it was instead of half-deleted.
|
|||||||
swallowed the drag along with the tap, so a list could not be scrolled
|
swallowed the drag along with the tap, so a list could not be scrolled
|
||||||
while anything in it was busy.
|
while anything in it was busy.
|
||||||
|
|
||||||
|
- **A rate-limit bar belongs to a session's provider, not to its machine.**
|
||||||
|
One machine offers echo, the Claude CLI and a local model at once and only
|
||||||
|
the CLI spends anything, so a session says which meter reports on it
|
||||||
|
(`usageProvider`, from `DriverKind::usage_provider`, which
|
||||||
|
`usage::providers_for` reads too so the two lists cannot disagree) and the
|
||||||
|
phone matches a snapshot on machine *and* provider. Nothing meters a llama
|
||||||
|
or echo session, and the phone draws **nothing** for one — not a zero, and
|
||||||
|
not "unknown". Nothing while the first fetch is out either: "checking"
|
||||||
|
under a session that turns out to meter nothing is a row the screen then
|
||||||
|
has to withdraw.
|
||||||
|
|
||||||
## Things that have bitten
|
## Things that have bitten
|
||||||
|
|
||||||
Project-specific only — a lesson that would bite any project on this machine
|
Project-specific only — a lesson that would bite any project on this machine
|
||||||
|
|||||||
@@ -169,10 +169,33 @@ deliberate and easy to undo by accident:
|
|||||||
when the process restarts. That leaves the Claude driver as the odd one
|
when the process restarts. That leaves the Claude driver as the odd one
|
||||||
out rather than this one — the CLI's memory is a cache in front of the same
|
out rather than this one — the CLI's memory is a cache in front of the same
|
||||||
transcript. Resolve any inconsistency in this direction.
|
transcript. Resolve any inconsistency in this direction.
|
||||||
- **A llama session on an ssh host is refused.** The model is reached over
|
- **A llama session runs on whatever machine its setup names** (2026-09-04,
|
||||||
HTTP and forwarding that port is not built, so refusing beats silently
|
the last of phase 5). A transport is "run this" plus "reach this port", and
|
||||||
talking to the wrong machine. A transport is "run this" plus "reach this
|
the second half is `Transport::reserve_port` — the port the server binds
|
||||||
port", and only the first half exists.
|
*there* and the port that reaches it *here*, the same number locally —
|
||||||
|
carried by `Launch::reaching` onto the connection that already runs the
|
||||||
|
command. `llama-server` binds loopback on the far machine, so nothing is
|
||||||
|
served to its network. The far port is a guess from a range below the
|
||||||
|
ephemeral one, because no portable way to ask a machine for a free port
|
||||||
|
avoids racing the bind anyway; a collision is not silent, since the server
|
||||||
|
fails to bind and the readiness poll reports what its log said.
|
||||||
|
- **The model file lives on the machine that serves it** (2026-09-04). Each
|
||||||
|
setup names its own models directory (`SshConfig::models_dir`, default
|
||||||
|
`~/.local/share/ai-app/models` expanded *there*), and a spawn resolves the
|
||||||
|
key on that machine — one round trip answering "at /abs/path" or "missing",
|
||||||
|
so a model that is not there is refused at the spawn rather than becoming a
|
||||||
|
server that never becomes ready. The spawn screen offers
|
||||||
|
`GET /setups/{id}/models`, that machine's list, rather than `GET /models`,
|
||||||
|
which is this backend's downloads. Downloading *to* another machine is
|
||||||
|
deliberately not built: a multi-gigabyte transfer with no progress
|
||||||
|
anywhere, and the file gets there however anything else on that machine
|
||||||
|
did.
|
||||||
|
- **The readiness poll watches the process, not only the port.** A model that
|
||||||
|
will not load, a port already taken, a flag an older build does not know:
|
||||||
|
all exit within a second and none will ever answer `/health`, so waiting
|
||||||
|
out the 300s timeout turned the server's own account of the problem into
|
||||||
|
"gave up". The failure carries the tail of `llama-server.log`, which on a
|
||||||
|
remote session is the only copy anybody reading the phone can see.
|
||||||
|
|
||||||
### Models (2026-08-28)
|
### Models (2026-08-28)
|
||||||
|
|
||||||
@@ -203,6 +226,13 @@ deliberate and easy to undo by accident:
|
|||||||
A driver says what to run; something above it turns that into a process.
|
A driver says what to run; something above it turns that into a process.
|
||||||
Otherwise transport knowledge sits inside a translator whose job is a wire
|
Otherwise transport knowledge sits inside a translator whose job is a wire
|
||||||
format, and every future driver has to remember to do the same.
|
format, and every future driver has to remember to do the same.
|
||||||
|
- **A forwarded launch gets a pty and every other one does not** (measured
|
||||||
|
2026-09-04). Killing the ssh client ends a CLI because it closes the stdin
|
||||||
|
that CLI is reading; `llama-server` never reads its stdin, so the same kill
|
||||||
|
left it running on the far machine with the model loaded — one orphan per
|
||||||
|
stopped session. With `-tt` the far side takes SIGHUP when the connection
|
||||||
|
goes. Its log then arrives through a line discipline, which nothing parses.
|
||||||
|
`-T` stays everywhere else, where a pty would rewrite the JSONL.
|
||||||
- **`command -v` follows ssh's non-login PATH**, which is narrower than an
|
- **`command -v` follows ssh's non-login PATH**, which is narrower than an
|
||||||
interactive shell's, so a binary somewhere unusual is invisible to
|
interactive shell's, so a binary somewhere unusual is invisible to
|
||||||
discovery. Point `command` at an absolute path.
|
discovery. Point `command` at an absolute path.
|
||||||
@@ -499,6 +529,25 @@ rate-limited bucket). Poll at ≥180 s, only while a Claude session exists or
|
|||||||
the usage screen is open, and cache the last answer. It is undocumented, so
|
the usage screen is open, and cache the last answer. It is undocumented, so
|
||||||
`usage.rs` treats every field as optional and degrades rather than erroring.
|
`usage.rs` treats every field as optional and degrades rather than erroring.
|
||||||
|
|
||||||
|
**Per provider, not per machine (2026-09-04).** A machine is not what is
|
||||||
|
metered; the provider a session runs is. One machine offers echo, the Claude
|
||||||
|
CLI and a local model side by side, and only the second spends anything — so
|
||||||
|
pairing a session with a snapshot by machine alone drew the CLI's five-hour
|
||||||
|
window under every echo session on it, a quota that session cannot spend. A
|
||||||
|
session now names its meter (`usageProvider`, from
|
||||||
|
`DriverKind::usage_provider`, which `usage::providers_for` reads too, so the
|
||||||
|
two lists cannot disagree) and `GET /usage` is matched on machine *and*
|
||||||
|
provider. `None` is a session that meters nothing, and the phone draws
|
||||||
|
nothing at all for it — not a zero, and not "unknown".
|
||||||
|
|
||||||
|
`DriverKind::Echo` names a meter of its own that exists only when a test has
|
||||||
|
asked for one: `/usage` in an echo session sets an invented answer
|
||||||
|
(`usage::Fixture`), and with none set there is no snapshot and no bar. That
|
||||||
|
is what makes those screens' states reachable — a number near the top, a
|
||||||
|
window between blocks with no reset time, a machine nobody logged into, one
|
||||||
|
that could not be reached — without spending real quota to arrange them,
|
||||||
|
which is why none of them had ever been looked at.
|
||||||
|
|
||||||
**Per machine, not per backend (2026-08-29).** The credential store that
|
**Per machine, not per backend (2026-08-29).** The credential store that
|
||||||
matters is the one on the machine the session runs on, because that is the
|
matters is the one on the machine the session runs on, because that is the
|
||||||
account being billed — and in the layout this aims at, `ai-server` is on the
|
account being billed — and in the layout this aims at, `ai-server` is on the
|
||||||
@@ -800,8 +849,9 @@ Noticed and deliberately not fixed, so they are not re-found from scratch.
|
|||||||
|
|
||||||
Phases 1–3 (the skeleton pipe, the full Claude driver, the usage screen) done
|
Phases 1–3 (the skeleton pipe, the full Claude driver, the usage screen) done
|
||||||
2026-08-24. Phase 4 (llama.cpp: model browsing, downloads, and `llama-server`
|
2026-08-24. Phase 4 (llama.cpp: model browsing, downloads, and `llama-server`
|
||||||
through its OpenAI-compatible endpoint) and phase 5 (ssh) done 2026-08-28.
|
through its OpenAI-compatible endpoint) and phase 5 (ssh) done 2026-08-28,
|
||||||
The file explorer and the transcript cache followed in September. What is
|
except for the remote `llama-server` and its port forward, which landed
|
||||||
|
2026-09-04. The file explorer and the transcript cache followed in September. What is
|
||||||
left is real-phone/WireGuard bring-up, which is operational rather than code.
|
left is real-phone/WireGuard bring-up, which is operational rather than code.
|
||||||
|
|
||||||
Each phase ended runnable and verified against the real thing. The backend
|
Each phase ended runnable and verified against the real thing. The backend
|
||||||
|
|||||||
@@ -178,6 +178,17 @@ data class SessionSummary(
|
|||||||
* server because that is where a provider's kind is known.
|
* server because that is where a provider's kind is known.
|
||||||
*/
|
*/
|
||||||
val maxImageEdge: Int?,
|
val maxImageEdge: Int?,
|
||||||
|
/**
|
||||||
|
* Which of `GET /usage`'s snapshots is about this session, and null where nothing meters it.
|
||||||
|
*
|
||||||
|
* The rate-limit bar answers a question about an *account*, and what decides which account --
|
||||||
|
* if any -- is the provider this session runs, not the machine it runs on. Pairing by machine
|
||||||
|
* alone drew the Claude CLI's five-hour window under every echo session on a machine that also
|
||||||
|
* has the CLI: a quota that session cannot spend and could never run down. Decided by the
|
||||||
|
* server for the same reason [maxImageEdge] is -- it is a fact about the provider's kind, and
|
||||||
|
* this app has only its name.
|
||||||
|
*/
|
||||||
|
val usageProvider: String?,
|
||||||
val status: String,
|
val status: String,
|
||||||
val lastActivity: Double,
|
val lastActivity: Double,
|
||||||
)
|
)
|
||||||
@@ -198,6 +209,7 @@ private fun parseSession(session: JSONObject) =
|
|||||||
contextTokens =
|
contextTokens =
|
||||||
if (session.has("contextTokens")) session.getLong("contextTokens") else null,
|
if (session.has("contextTokens")) session.getLong("contextTokens") else null,
|
||||||
maxImageEdge = session.optInt("maxImageEdge", 0).takeIf { it > 0 },
|
maxImageEdge = session.optInt("maxImageEdge", 0).takeIf { it > 0 },
|
||||||
|
usageProvider = session.optString("usageProvider").ifEmpty { null },
|
||||||
status = session.getString("status"),
|
status = session.getString("status"),
|
||||||
lastActivity = session.getDouble("lastActivity"),
|
lastActivity = session.getDouble("lastActivity"),
|
||||||
)
|
)
|
||||||
@@ -375,6 +387,12 @@ data class SshDetails(
|
|||||||
* Where files attached from here land on that machine; null for the session's own directory.
|
* Where files attached from here land on that machine; null for the session's own directory.
|
||||||
*/
|
*/
|
||||||
val attachmentsDir: String? = null,
|
val attachmentsDir: String? = null,
|
||||||
|
/**
|
||||||
|
* Where that machine keeps its GGUF models; null for the same place the backend keeps its own
|
||||||
|
* (`~/.local/share/ai-app/models`, read on that machine). A llama.cpp session serves the file
|
||||||
|
* from the machine it runs on, so this is where its models are looked for and listed.
|
||||||
|
*/
|
||||||
|
val modelsDir: String? = null,
|
||||||
)
|
)
|
||||||
|
|
||||||
private fun SshDetails.toJson() =
|
private fun SshDetails.toJson() =
|
||||||
@@ -382,6 +400,7 @@ private fun SshDetails.toJson() =
|
|||||||
if (port != null) put("port", port)
|
if (port != null) put("port", port)
|
||||||
if (!identityFile.isNullOrBlank()) put("identityFile", identityFile)
|
if (!identityFile.isNullOrBlank()) put("identityFile", identityFile)
|
||||||
if (!attachmentsDir.isNullOrBlank()) put("attachmentsDir", attachmentsDir)
|
if (!attachmentsDir.isNullOrBlank()) put("attachmentsDir", attachmentsDir)
|
||||||
|
if (!modelsDir.isNullOrBlank()) put("modelsDir", modelsDir)
|
||||||
}
|
}
|
||||||
|
|
||||||
/** What a machine turns out to have, without saving anything. */
|
/** What a machine turns out to have, without saving anything. */
|
||||||
@@ -1074,6 +1093,26 @@ private fun parseDownload(o: JSONObject) =
|
|||||||
error = if (o.has("error")) o.getString("error") else null,
|
error = if (o.has("error")) o.getString("error") else null,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The models on one machine, which is the list a llama.cpp session there can choose from.
|
||||||
|
*
|
||||||
|
* Not [fetchModels], which is what the *backend* has downloaded. A session serves its model from
|
||||||
|
* the machine it runs on, so for a machine reached over ssh those are two different lists -- and
|
||||||
|
* offering the backend's would name files that are not there, turning a choice that cannot work
|
||||||
|
* into a session that fails when it tries to load one.
|
||||||
|
*/
|
||||||
|
fun fetchSetupModels(settings: ServerSettings, setupId: String): List<LocalModel> =
|
||||||
|
requestFromServer(settings, "/setups/${setupId.urlEncoded()}/models") { connection ->
|
||||||
|
JSONArray(connection.inputStream.bufferedReader().readText()).mapObjects { m ->
|
||||||
|
LocalModel(
|
||||||
|
key = m.getString("key"),
|
||||||
|
repo = m.getString("repo"),
|
||||||
|
file = m.getString("file"),
|
||||||
|
bytes = m.getLong("bytes"),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
fun fetchModels(settings: ServerSettings): Models =
|
fun fetchModels(settings: ServerSettings): Models =
|
||||||
requestFromServer(settings, "/models") { connection ->
|
requestFromServer(settings, "/models") { connection ->
|
||||||
val body = JSONObject(connection.inputStream.bufferedReader().readText())
|
val body = JSONObject(connection.inputStream.bufferedReader().readText())
|
||||||
|
|||||||
@@ -1149,7 +1149,7 @@ fun SessionScreen(
|
|||||||
|
|
||||||
// One poll for the machines' limits, read by everything on this screen that reports them.
|
// One poll for the machines' limits, read by everything on this screen that reports them.
|
||||||
val usageFeed = rememberUsageFeed(settings)
|
val usageFeed = rememberUsageFeed(settings)
|
||||||
val usage = usageFeed.forSetup(summary.setup)
|
val usage = usageFeed.forSession(summary)
|
||||||
RecordFrames()
|
RecordFrames()
|
||||||
var usageOpen by remember { mutableStateOf(false) }
|
var usageOpen by remember { mutableStateOf(false) }
|
||||||
var settingsOpen by remember { mutableStateOf(false) }
|
var settingsOpen by remember { mutableStateOf(false) }
|
||||||
|
|||||||
@@ -70,12 +70,21 @@ class UsageFeed(
|
|||||||
/** Ask the backend again now. The dialog's refresh button; the poll does it on its own. */
|
/** Ask the backend again now. The dialog's refresh button; the poll does it on its own. */
|
||||||
val refresh: () -> Unit,
|
val refresh: () -> Unit,
|
||||||
) {
|
) {
|
||||||
/** What [setup]'s own limits came back as. See [usageFor] for why the states are these. */
|
/**
|
||||||
fun forSetup(setup: String): SessionUsage =
|
* What meters [session], and what that meter came back as. See [usageFor] for the states.
|
||||||
when (val state = snapshots) {
|
*
|
||||||
|
* A session rather than a machine, because a machine is not what is metered: one machine runs
|
||||||
|
* the Claude CLI and an echo session side by side, and only the first of them spends anything.
|
||||||
|
*/
|
||||||
|
fun forSession(session: SessionSummary): SessionUsage {
|
||||||
|
// Settled without asking anybody: a session nothing meters has nothing to check, and
|
||||||
|
// "checking" is what the fetch's own states would say about it for as long as one is out.
|
||||||
|
val provider = session.usageProvider ?: return SessionUsage.NotMetered
|
||||||
|
return when (val state = snapshots) {
|
||||||
is LoadState.Loading -> SessionUsage.Waiting
|
is LoadState.Loading -> SessionUsage.Waiting
|
||||||
is LoadState.Error -> SessionUsage.Unavailable(state.message)
|
is LoadState.Error -> SessionUsage.Unavailable(state.message)
|
||||||
is LoadState.Loaded -> usageFor(state.value, setup)
|
is LoadState.Loaded -> usageFor(state.value, session.setup, provider)
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -158,9 +167,15 @@ fun SessionUsageBar(usage: SessionUsage, modifier: Modifier = Modifier) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// Nothing at all for a machine that meters nothing: a row saying "unknown" there would report a
|
// Nothing at all for a session that meters nothing: a row saying "unknown" there would report
|
||||||
// problem about a setup somebody chose, on every screen, forever.
|
// a problem about a setup somebody chose, on every screen, forever.
|
||||||
if (usage is SessionUsage.NotMetered) {
|
//
|
||||||
|
// And nothing while the first fetch is out, which is a different silence. A request in flight
|
||||||
|
// is not a state to report -- and the session that meters nothing is exactly the one this
|
||||||
|
// cannot yet tell apart, so "5-hour usage: checking" appeared under an echo session for half a
|
||||||
|
// second and was then taken away. A row that has to be withdrawn is worse than one that
|
||||||
|
// arrives late.
|
||||||
|
if (usage is SessionUsage.NotMetered || usage is SessionUsage.Waiting) {
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -171,9 +186,10 @@ fun SessionUsageBar(usage: SessionUsage, modifier: Modifier = Modifier) {
|
|||||||
// Words, not a colour and not an empty bar: every one of these is a different kind of
|
// Words, not a colour and not an empty bar: every one of these is a different kind of
|
||||||
// answer from "this much is used", and only words carry a difference in kind.
|
// answer from "this much is used", and only words carry a difference in kind.
|
||||||
when (val state = usage) {
|
when (val state = usage) {
|
||||||
SessionUsage.NotMetered -> Unit
|
// Both handled above, before the row exists at all.
|
||||||
|
SessionUsage.NotMetered,
|
||||||
|
SessionUsage.Waiting -> Unit
|
||||||
is SessionUsage.Unavailable -> UsageNote("5-hour usage unknown -- ${state.why}")
|
is SessionUsage.Unavailable -> UsageNote("5-hour usage unknown -- ${state.why}")
|
||||||
SessionUsage.Waiting -> UsageNote("5-hour usage: checking")
|
|
||||||
is SessionUsage.Known -> {
|
is SessionUsage.Known -> {
|
||||||
val window = state.windows.firstOrNull { it.kind == "session" }
|
val window = state.windows.firstOrNull { it.kind == "session" }
|
||||||
if (window == null) {
|
if (window == null) {
|
||||||
@@ -234,16 +250,22 @@ private fun fiveHourLabel(window: UsageWindow, now: OffsetDateTime): String {
|
|||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* One machine's snapshot, out of every machine's.
|
* One meter's snapshot, out of every machine's: [setup]'s row for [provider].
|
||||||
|
*
|
||||||
|
* Both halves are needed to pick it. A machine can hold more than one meter -- the Claude CLI's
|
||||||
|
* account and, while a test has one set, an echo session's invented one -- and a snapshot is one
|
||||||
|
* service on one machine.
|
||||||
*
|
*
|
||||||
* Every way of having *failed* to get numbers is [SessionUsage.Unavailable] with the reason in it.
|
* Every way of having *failed* to get numbers is [SessionUsage.Unavailable] with the reason in it.
|
||||||
* None of them may look like zero, and none may look like [SessionUsage.NotMetered], which is the
|
* None of them may look like zero, and none may look like [SessionUsage.NotMetered], which is the
|
||||||
* machine having no quota rather than the question going unanswered.
|
* machine having no quota rather than the question going unanswered.
|
||||||
*/
|
*/
|
||||||
fun usageFor(snapshots: List<UsageSnapshot>, setup: String): SessionUsage {
|
fun usageFor(snapshots: List<UsageSnapshot>, setup: String, provider: String): SessionUsage {
|
||||||
// No snapshot at all means the backend never asked, which it only does for a machine with
|
// No snapshot at all means the backend never asked, which it only does where there is nothing
|
||||||
// nothing metered on it. That is a different answer from having asked and failed.
|
// to ask about. That is a different answer from having asked and failed.
|
||||||
val mine = snapshots.firstOrNull { it.setup == setup } ?: return SessionUsage.NotMetered
|
val mine =
|
||||||
|
snapshots.firstOrNull { it.setup == setup && it.provider == provider }
|
||||||
|
?: return SessionUsage.NotMetered
|
||||||
if (mine.state != "ok") {
|
if (mine.state != "ok") {
|
||||||
return SessionUsage.Unavailable(mine.detail ?: mine.state)
|
return SessionUsage.Unavailable(mine.detail ?: mine.state)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -236,6 +236,7 @@ private fun AddSetupDialog(
|
|||||||
var address by remember { mutableStateOf("") }
|
var address by remember { mutableStateOf("") }
|
||||||
var identity by remember { mutableStateOf("") }
|
var identity by remember { mutableStateOf("") }
|
||||||
var attachmentsDir by remember { mutableStateOf("") }
|
var attachmentsDir by remember { mutableStateOf("") }
|
||||||
|
var modelsDir by remember { mutableStateOf("") }
|
||||||
var tested by remember { mutableStateOf<String?>(null) }
|
var tested by remember { mutableStateOf<String?>(null) }
|
||||||
var testing by remember { mutableStateOf(false) }
|
var testing by remember { mutableStateOf(false) }
|
||||||
|
|
||||||
@@ -250,6 +251,7 @@ private fun AddSetupDialog(
|
|||||||
port = typedPort,
|
port = typedPort,
|
||||||
identityFile = identity.trim().ifEmpty { null },
|
identityFile = identity.trim().ifEmpty { null },
|
||||||
attachmentsDir = attachmentsDir.trim().ifEmpty { null },
|
attachmentsDir = attachmentsDir.trim().ifEmpty { null },
|
||||||
|
modelsDir = modelsDir.trim().ifEmpty { null },
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -293,6 +295,14 @@ private fun AddSetupDialog(
|
|||||||
label = { Text("Folder for attached files (optional)") },
|
label = { Text("Folder for attached files (optional)") },
|
||||||
singleLine = true,
|
singleLine = true,
|
||||||
)
|
)
|
||||||
|
// Where that machine's GGUFs are, for a llama.cpp session on it. Blank means
|
||||||
|
// the same place this backend keeps its own downloads, read on that machine.
|
||||||
|
OutlinedTextField(
|
||||||
|
value = modelsDir,
|
||||||
|
onValueChange = { modelsDir = it },
|
||||||
|
label = { Text("Folder for models (optional)") },
|
||||||
|
singleLine = true,
|
||||||
|
)
|
||||||
tested?.let {
|
tested?.let {
|
||||||
Spacer(Modifier.height(8.dp))
|
Spacer(Modifier.height(8.dp))
|
||||||
Text(it, style = MaterialTheme.typography.bodySmall)
|
Text(it, style = MaterialTheme.typography.bodySmall)
|
||||||
|
|||||||
@@ -65,8 +65,10 @@ fun SpawnScreen(
|
|||||||
// Only the spawn's own failure. The fetch's lives in `options`: this one leaves a filled-in
|
// Only the spawn's own failure. The fetch's lives in `options`: this one leaves a filled-in
|
||||||
// form worth keeping, and that one leaves nothing to fill in.
|
// form worth keeping, and that one leaves nothing to fill in.
|
||||||
var spawnError by remember { mutableStateOf<String?>(null) }
|
var spawnError by remember { mutableStateOf<String?>(null) }
|
||||||
// Downloaded models, for a llama provider to choose between. Kept separate from the setups: a
|
// The models on the *chosen machine*, for a llama provider to choose between. Kept separate
|
||||||
// Claude session needs none, so failing to list them must not stop the screen rendering.
|
// from the setups: a Claude session needs none, so failing to list them must not stop the
|
||||||
|
// screen rendering. Refetched when the machine changes, because a model is a file on one
|
||||||
|
// machine -- see [fetchSetupModels].
|
||||||
var models by remember { mutableStateOf<List<LocalModel>>(emptyList()) }
|
var models by remember { mutableStateOf<List<LocalModel>>(emptyList()) }
|
||||||
var modelKey by remember { mutableStateOf<String?>(null) }
|
var modelKey by remember { mutableStateOf<String?>(null) }
|
||||||
var contextSize by remember { mutableStateOf("") }
|
var contextSize by remember { mutableStateOf("") }
|
||||||
@@ -83,9 +85,6 @@ fun SpawnScreen(
|
|||||||
} catch (e: ApiException) {
|
} catch (e: ApiException) {
|
||||||
LoadState.failed(e)
|
LoadState.failed(e)
|
||||||
}
|
}
|
||||||
models =
|
|
||||||
runCatching { withContext(Dispatchers.IO) { fetchModels(settings).local } }
|
|
||||||
.getOrDefault(emptyList())
|
|
||||||
}
|
}
|
||||||
|
|
||||||
Column(Modifier.fillMaxSize().verticalScroll(rememberScrollState()).padding(16.dp)) {
|
Column(Modifier.fillMaxSize().verticalScroll(rememberScrollState()).padding(16.dp)) {
|
||||||
@@ -115,6 +114,17 @@ fun SpawnScreen(
|
|||||||
is LoadState.Loaded -> state.value
|
is LoadState.Loaded -> state.value
|
||||||
}
|
}
|
||||||
val setup = setups.firstOrNull { it.name == setupName }
|
val setup = setups.firstOrNull { it.name == setupName }
|
||||||
|
// Whichever machine is chosen now, asked again when that changes. The old machine's list
|
||||||
|
// is dropped first rather than left on screen: a file name from another machine looks
|
||||||
|
// exactly like one from this one.
|
||||||
|
LaunchedEffect(setup?.id) {
|
||||||
|
models = emptyList()
|
||||||
|
modelKey = null
|
||||||
|
val id = setup?.id ?: return@LaunchedEffect
|
||||||
|
models =
|
||||||
|
runCatching { withContext(Dispatchers.IO) { fetchSetupModels(settings, id) } }
|
||||||
|
.getOrDefault(emptyList())
|
||||||
|
}
|
||||||
val current = setup?.providers?.firstOrNull { it.name == providerName }
|
val current = setup?.providers?.firstOrNull { it.name == providerName }
|
||||||
// Only the Claude CLI has models, a working directory and permission modes; keying the
|
// Only the Claude CLI has models, a working directory and permission modes; keying the
|
||||||
// extra fields on the kind rather than the provider name keeps a second Claude provider
|
// extra fields on the kind rather than the provider name keeps a second Claude provider
|
||||||
@@ -175,12 +185,13 @@ fun SpawnScreen(
|
|||||||
)
|
)
|
||||||
|
|
||||||
if (isLlama) {
|
if (isLlama) {
|
||||||
// A llama session names one of the models this backend has downloaded, so the choice is
|
// A llama session names one of the models on the machine it will run on, so the
|
||||||
// that list rather than free text -- a name that is not on disk is a session that
|
// choice is that list rather than free text -- a name that is not on that machine's
|
||||||
// cannot start.
|
// disk is a session that cannot start.
|
||||||
if (models.isEmpty()) {
|
if (models.isEmpty()) {
|
||||||
Text(
|
Text(
|
||||||
"No models downloaded yet. Get one from the Models screen first.",
|
"No models on ${setup?.name ?: "this machine"}. The Models screen downloads " +
|
||||||
|
"to the backend; another machine needs the file put there itself.",
|
||||||
style = MaterialTheme.typography.bodyMedium,
|
style = MaterialTheme.typography.bodyMedium,
|
||||||
color = MaterialTheme.colorScheme.onSurfaceVariant,
|
color = MaterialTheme.colorScheme.onSurfaceVariant,
|
||||||
)
|
)
|
||||||
|
|||||||
Generated
+1
@@ -35,6 +35,7 @@ dependencies = [
|
|||||||
"sha2",
|
"sha2",
|
||||||
"tempfile",
|
"tempfile",
|
||||||
"thiserror",
|
"thiserror",
|
||||||
|
"time",
|
||||||
"tokio",
|
"tokio",
|
||||||
"tokio-stream",
|
"tokio-stream",
|
||||||
"tower",
|
"tower",
|
||||||
|
|||||||
@@ -53,6 +53,12 @@ ureq = { version = "3", features = ["json"] }
|
|||||||
# both in the graph rustls refuses to auto-select one.
|
# both in the graph rustls refuses to auto-select one.
|
||||||
rustls = "0.23"
|
rustls = "0.23"
|
||||||
libc = "0.2.189"
|
libc = "0.2.189"
|
||||||
|
# One ISO-8601 timestamp: the reset time on the invented rate-limit window
|
||||||
|
# an echo session's `/usage` puts up. Already in the tree behind the
|
||||||
|
# certificate machinery, so this is a direct name for what is compiled
|
||||||
|
# anyway rather than a new crate -- and the alternative was hand-rolling a
|
||||||
|
# civil-from-days conversion to print one line.
|
||||||
|
time = { version = "0.3", features = ["formatting"] }
|
||||||
|
|
||||||
[dev-dependencies]
|
[dev-dependencies]
|
||||||
tempfile = "3"
|
tempfile = "3"
|
||||||
|
|||||||
@@ -207,6 +207,20 @@ mod tests {
|
|||||||
/// like any other.
|
/// like any other.
|
||||||
#[tokio::test]
|
#[tokio::test]
|
||||||
async fn a_spooled_enrollment_is_adopted_on_first_use() {
|
async fn a_spooled_enrollment_is_adopted_on_first_use() {
|
||||||
|
// Under a subscriber, like every other exercise of this middleware.
|
||||||
|
// `tracing` caches a callsite's interest process-wide the first time it
|
||||||
|
// is reached, so the refusal at the end of this test -- reached with no
|
||||||
|
// subscriber on this thread -- could cache the rejection warning as
|
||||||
|
// never-enabled and make the tripwire above see an empty log. That
|
||||||
|
// failed about one full-suite run in ten, in the test that exists to
|
||||||
|
// notice a credential leak, which is the worst place for a flake.
|
||||||
|
let _guard = tracing::subscriber::set_default(
|
||||||
|
tracing_subscriber::fmt()
|
||||||
|
.with_max_level(tracing::Level::TRACE)
|
||||||
|
.with_writer(std::io::sink)
|
||||||
|
.finish(),
|
||||||
|
);
|
||||||
|
|
||||||
let dir = tempfile::tempdir().expect("tempdir");
|
let dir = tempfile::tempdir().expect("tempdir");
|
||||||
let manager = manager_with_token(dir.path(), "first");
|
let manager = manager_with_token(dir.path(), "first");
|
||||||
let spooled = generate_token();
|
let spooled = generate_token();
|
||||||
|
|||||||
@@ -94,6 +94,19 @@ pub struct SshConfig {
|
|||||||
/// Extra `-o` settings, each written as `Key=value`.
|
/// Extra `-o` settings, each written as `Key=value`.
|
||||||
#[serde(default, skip_serializing_if = "Vec::is_empty")]
|
#[serde(default, skip_serializing_if = "Vec::is_empty")]
|
||||||
pub options: Vec<String>,
|
pub options: Vec<String>,
|
||||||
|
/// Where this machine keeps the GGUF models it can serve, absent for
|
||||||
|
/// the same default this backend uses (`~/.local/share/ai-app/models`
|
||||||
|
/// -- `$XDG_DATA_HOME` is not read on the far side, since it is this
|
||||||
|
/// machine's environment that would answer). A `~` prefix is the
|
||||||
|
/// remote home.
|
||||||
|
///
|
||||||
|
/// Here rather than on the provider because it is a fact about the
|
||||||
|
/// machine, and because a machine reached over ssh is where the model
|
||||||
|
/// has to be: a llama.cpp session serves the file from the machine
|
||||||
|
/// that runs `llama-server`, and this backend's own downloads are on
|
||||||
|
/// whichever machine that is only when they are the same one.
|
||||||
|
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||||
|
pub models_dir: Option<PathBuf>,
|
||||||
/// Where a file attached from the phone is put on this machine so the
|
/// Where a file attached from the phone is put on this machine so the
|
||||||
/// session can read it. Absent means the session's own working directory,
|
/// session can read it. Absent means the session's own working directory,
|
||||||
/// or the login home for a session that has none. A `~` prefix is the
|
/// or the login home for a session that has none. A `~` prefix is the
|
||||||
@@ -145,6 +158,30 @@ impl DriverKind {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Which paid service meters a session of this kind, and `None` for one
|
||||||
|
/// that costs nothing.
|
||||||
|
///
|
||||||
|
/// What decides which account -- if any -- a rate-limit bar is about is the
|
||||||
|
/// provider a session runs, not the machine it runs on: an echo session on
|
||||||
|
/// a machine that also has the Claude CLI was drawn with that CLI's
|
||||||
|
/// five-hour window, a quota it cannot spend.
|
||||||
|
///
|
||||||
|
/// Echo names a meter of its own that exists only when a test has asked for
|
||||||
|
/// one (`/usage` in `session::echo`), which is how the bar's states are
|
||||||
|
/// reached without an account. With none set there is no snapshot, and the
|
||||||
|
/// phone draws nothing.
|
||||||
|
///
|
||||||
|
/// The string is a [`crate::usage::UsageProvider::name`], and it is what
|
||||||
|
/// pairs a session with one of `GET /usage`'s snapshots -- so
|
||||||
|
/// `usage::providers_for` reads this rather than matching on kinds again.
|
||||||
|
pub fn usage_provider(self) -> Option<&'static str> {
|
||||||
|
match self {
|
||||||
|
Self::ClaudeCli => Some(crate::usage::CLAUDE),
|
||||||
|
Self::Echo => Some(crate::usage::ECHO),
|
||||||
|
Self::LlamaCpp => None,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
/// Whether the conversation exists outside this app, so that deleting the
|
/// Whether the conversation exists outside this app, so that deleting the
|
||||||
/// session here does not end it.
|
/// session here does not end it.
|
||||||
///
|
///
|
||||||
@@ -385,6 +422,7 @@ mod tests {
|
|||||||
port: Some(2222),
|
port: Some(2222),
|
||||||
identity_file: None,
|
identity_file: None,
|
||||||
options: Vec::new(),
|
options: Vec::new(),
|
||||||
|
models_dir: None,
|
||||||
attachments_dir: None,
|
attachments_dir: None,
|
||||||
}),
|
}),
|
||||||
providers: vec![ProviderConfig {
|
providers: vec![ProviderConfig {
|
||||||
|
|||||||
+3
-1
@@ -264,7 +264,9 @@ async fn main() -> Result<()> {
|
|||||||
// No providers listed here any more: which machines can be asked, and about
|
// No providers listed here any more: which machines can be asked, and about
|
||||||
// what, comes from the setups at the moment the screen is opened -- so a
|
// what, comes from the setups at the moment the screen is opened -- so a
|
||||||
// machine added from the phone reports its limits without a restart.
|
// machine added from the phone reports its limits without a restart.
|
||||||
let monitor = Arc::new(usage::UsageMonitor::new());
|
// The fixture is the manager's, because that is where the `/usage` command
|
||||||
|
// that sets it is typed; the monitor is what serves it.
|
||||||
|
let monitor = Arc::new(usage::UsageMonitor::new(manager.usage_fixture()));
|
||||||
|
|
||||||
// The bearer-token middleware wraps the entire router -- routes and fallback
|
// The bearer-token middleware wraps the entire router -- routes and fallback
|
||||||
// alike -- here and only here, so a new route can't forget auth.
|
// alike -- here and only here, so a new route can't forget auth.
|
||||||
|
|||||||
@@ -29,6 +29,8 @@ use serde::Serialize;
|
|||||||
|
|
||||||
use wg_app_link::private;
|
use wg_app_link::private;
|
||||||
|
|
||||||
|
use crate::session::transport::{Launch, Transport};
|
||||||
|
|
||||||
/// Identifies this client to HuggingFace. They ask for one, and a request
|
/// Identifies this client to HuggingFace. They ask for one, and a request
|
||||||
/// without it is more likely to be rate-limited.
|
/// without it is more likely to be rate-limited.
|
||||||
const USER_AGENT: &str = concat!("ai-server/", env!("CARGO_PKG_VERSION"));
|
const USER_AGENT: &str = concat!("ai-server/", env!("CARGO_PKG_VERSION"));
|
||||||
@@ -521,6 +523,83 @@ fn collect(root: &Path, dir: &Path, found: &mut Vec<LocalModel>) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Where a machine reached over ssh keeps its models, when its setup does
|
||||||
|
/// not say.
|
||||||
|
///
|
||||||
|
/// The same place this backend puts its own downloads, written out rather
|
||||||
|
/// than derived: `$XDG_DATA_HOME` here describes *this* machine's
|
||||||
|
/// environment, and the far machine's is the far machine's business. A
|
||||||
|
/// setup whose models are elsewhere says so (`SshConfig::models_dir`).
|
||||||
|
const FAR_MODELS_DIR: &str = "~/.local/share/ai-app/models";
|
||||||
|
|
||||||
|
/// Which directory holds the models on the machine `transport` reaches.
|
||||||
|
///
|
||||||
|
/// One answer, because two things ask: the list a spawn screen offers,
|
||||||
|
/// and the path a session hands `llama-server`. A machine that listed one
|
||||||
|
/// directory and served from another would offer models that then failed
|
||||||
|
/// to load, which reads as the model being broken.
|
||||||
|
pub fn dir_on(transport: &Transport, local: &Path) -> String {
|
||||||
|
match transport {
|
||||||
|
Transport::Here => local.to_string_lossy().into_owned(),
|
||||||
|
Transport::Ssh { ssh, .. } => ssh
|
||||||
|
.models_dir
|
||||||
|
.as_ref()
|
||||||
|
.map_or(FAR_MODELS_DIR.to_string(), |dir| {
|
||||||
|
dir.to_string_lossy().into_owned()
|
||||||
|
}),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Every GGUF on the machine a setup names, which is the machine that
|
||||||
|
/// would have to serve it.
|
||||||
|
///
|
||||||
|
/// The local half of this is [`ModelStore::list`], reading the same shape
|
||||||
|
/// off this machine's disk; a caller picks by transport, since a setup
|
||||||
|
/// with no ssh *is* this machine and asking a shell about it would be a
|
||||||
|
/// slower way to the same answer. What must not happen is offering this
|
||||||
|
/// backend's downloads for a session on another machine: the file has to
|
||||||
|
/// be where `llama-server` runs, and a list that says otherwise is a
|
||||||
|
/// claim about the wrong filesystem.
|
||||||
|
///
|
||||||
|
/// `dir` is that machine's models directory, `~` included -- expanded on
|
||||||
|
/// the far side, which is the only place that knows what it is. A
|
||||||
|
/// directory that is not there is an empty list rather than a failure: a
|
||||||
|
/// machine that has never had a model put on it is an ordinary state, and
|
||||||
|
/// the same one as a machine whose directory exists and is empty.
|
||||||
|
pub async fn on_machine(transport: &Transport, dir: &str) -> Result<Vec<LocalModel>> {
|
||||||
|
let script = "p=$1; case $p in \"~\") p=$HOME;; \"~/\"*) p=$HOME/${p#\"~/\"};; esac; \
|
||||||
|
[ -d \"$p\" ] || exit 0; \
|
||||||
|
find \"$p\" -type f -name '*.gguf' -printf '%s\\t%P\\0'";
|
||||||
|
let launch = Launch::new(
|
||||||
|
"sh",
|
||||||
|
vec![
|
||||||
|
"-c".to_string(),
|
||||||
|
script.to_string(),
|
||||||
|
"sh".to_string(),
|
||||||
|
dir.to_string(),
|
||||||
|
],
|
||||||
|
None,
|
||||||
|
);
|
||||||
|
let out = transport.capture(&launch).await?;
|
||||||
|
let mut found: Vec<LocalModel> = out
|
||||||
|
.split('\0')
|
||||||
|
.filter(|record| !record.is_empty())
|
||||||
|
// Two fields, and the name last, so a `\t` in a filename survives.
|
||||||
|
.filter_map(|record| record.split_once('\t'))
|
||||||
|
.filter_map(|(bytes, key)| {
|
||||||
|
let (repo, file) = key.rsplit_once('/')?;
|
||||||
|
Some(LocalModel {
|
||||||
|
key: key.to_string(),
|
||||||
|
repo: repo.to_string(),
|
||||||
|
file: file.to_string(),
|
||||||
|
bytes: bytes.trim().parse().unwrap_or(0),
|
||||||
|
})
|
||||||
|
})
|
||||||
|
.collect();
|
||||||
|
found.sort_by(|a, b| a.key.cmp(&b.key));
|
||||||
|
Ok(found)
|
||||||
|
}
|
||||||
|
|
||||||
/// A model repository on HuggingFace, as the browse screen shows it.
|
/// A model repository on HuggingFace, as the browse screen shows it.
|
||||||
#[derive(Debug, Clone, Serialize)]
|
#[derive(Debug, Clone, Serialize)]
|
||||||
#[serde(rename_all = "camelCase")]
|
#[serde(rename_all = "camelCase")]
|
||||||
|
|||||||
@@ -7,6 +7,7 @@
|
|||||||
//! POST /setups add {name, ssh?} -- providers are discovered
|
//! POST /setups add {name, ssh?} -- providers are discovered
|
||||||
//! POST /setups/probe dry run {ssh?}: what would be found there
|
//! POST /setups/probe dry run {ssh?}: what would be found there
|
||||||
//! GET /setups/{id} one machine, for refetching after a change
|
//! GET /setups/{id} one machine, for refetching after a change
|
||||||
|
//! GET /setups/{id}/models GGUFs on that machine, for a llama session
|
||||||
//! GET /setups/{id}/dir?path=P entries of directory P, and P resolved
|
//! GET /setups/{id}/dir?path=P entries of directory P, and P resolved
|
||||||
//! GET /setups/{id}/file?path=P content of file P, or why not
|
//! GET /setups/{id}/file?path=P content of file P, or why not
|
||||||
//! PUT /setups/{id}/file {path, content, ifSha256} -> new size/mtime/sha256
|
//! PUT /setups/{id}/file {path, content, ifSha256} -> new size/mtime/sha256
|
||||||
@@ -109,6 +110,8 @@ pub fn router(manager: Arc<SessionManager>) -> Router {
|
|||||||
"/setups/{id}",
|
"/setups/{id}",
|
||||||
get(read_setup).put(update_setup).delete(delete_setup),
|
get(read_setup).put(update_setup).delete(delete_setup),
|
||||||
)
|
)
|
||||||
|
// The models on the machine a setup names, for a llama session there.
|
||||||
|
.route("/setups/{id}/models", get(setup_models))
|
||||||
// The filesystem of the machine a setup names. Under the setup
|
// The filesystem of the machine a setup names. Under the setup
|
||||||
// rather than under a session because a filesystem is a property of
|
// rather than under a session because a filesystem is a property of
|
||||||
// a machine; a session only says where to start looking.
|
// a machine; a session only says where to start looking.
|
||||||
@@ -291,6 +294,9 @@ struct SshRequest {
|
|||||||
/// Where attached files land on that machine; see `SshConfig`.
|
/// Where attached files land on that machine; see `SshConfig`.
|
||||||
#[serde(default)]
|
#[serde(default)]
|
||||||
attachments_dir: Option<String>,
|
attachments_dir: Option<String>,
|
||||||
|
/// Where that machine keeps its GGUF models; see `SshConfig`.
|
||||||
|
#[serde(default)]
|
||||||
|
models_dir: Option<String>,
|
||||||
}
|
}
|
||||||
|
|
||||||
impl SshRequest {
|
impl SshRequest {
|
||||||
@@ -320,6 +326,14 @@ impl SshRequest {
|
|||||||
.map(str::trim)
|
.map(str::trim)
|
||||||
.filter(|dir| !dir.is_empty())
|
.filter(|dir| !dir.is_empty())
|
||||||
.map(std::path::PathBuf::from),
|
.map(std::path::PathBuf::from),
|
||||||
|
// The same rule, and for the same reason: this directory is
|
||||||
|
// on the other machine, so a `~` in it is that machine's home.
|
||||||
|
models_dir: self
|
||||||
|
.models_dir
|
||||||
|
.as_deref()
|
||||||
|
.map(str::trim)
|
||||||
|
.filter(|dir| !dir.is_empty())
|
||||||
|
.map(std::path::PathBuf::from),
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -495,6 +509,27 @@ struct PathQuery {
|
|||||||
path: String,
|
path: String,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// The models **that machine** has, which is the list a llama.cpp session
|
||||||
|
/// on it can choose from.
|
||||||
|
///
|
||||||
|
/// Not `GET /models`, which is this backend's own downloads: those are on
|
||||||
|
/// the machine a session runs on only when they are the same machine. A
|
||||||
|
/// spawn screen offering this backend's list for a remote setup would be
|
||||||
|
/// naming files that are not there, and the session would fail at the
|
||||||
|
/// point of loading rather than at the point of choosing.
|
||||||
|
async fn setup_models(
|
||||||
|
State(manager): State<Arc<SessionManager>>,
|
||||||
|
UrlPath(id): UrlPath<String>,
|
||||||
|
) -> Result<axum::Json<Vec<crate::models::LocalModel>>, ApiError> {
|
||||||
|
let setup = setup_by_id(&manager, &id)?;
|
||||||
|
let transport = crate::session::transport::Transport::for_setup(&setup);
|
||||||
|
let dir = crate::models::dir_on(&transport, manager.models_dir());
|
||||||
|
crate::models::on_machine(&transport, &dir)
|
||||||
|
.await
|
||||||
|
.map(axum::Json)
|
||||||
|
.map_err(from_machine)
|
||||||
|
}
|
||||||
|
|
||||||
/// What is in a directory, and what that directory resolved to.
|
/// What is in a directory, and what that directory resolved to.
|
||||||
async fn list_dir(
|
async fn list_dir(
|
||||||
State(manager): State<Arc<SessionManager>>,
|
State(manager): State<Arc<SessionManager>>,
|
||||||
|
|||||||
@@ -26,6 +26,13 @@
|
|||||||
//! - `/error [text]` -- a failure, which is otherwise awkward to cause.
|
//! - `/error [text]` -- a failure, which is otherwise awkward to cause.
|
||||||
//! - `/peer [text]`, `/peer-turn` -- a message from another agent, in the
|
//! - `/peer [text]`, `/peer-turn` -- a message from another agent, in the
|
||||||
//! in-place and the live shapes.
|
//! in-place and the live shapes.
|
||||||
|
//! - `/usage [what]` -- an invented rate-limit answer, or `/usage off` to take
|
||||||
|
//! it away. An echo session meters nothing, so it draws no usage bar until
|
||||||
|
//! this is set; what it exists for is the states that bar can be in, which
|
||||||
|
//! otherwise cost real quota to reach. `/usage 42`, `/usage 95 20`,
|
||||||
|
//! `/usage 42 never`, `/usage notloggedin`, `/usage unreachable`,
|
||||||
|
//! `/usage failed`. The vocabulary is `usage::Fixture`'s, where the states
|
||||||
|
//! live.
|
||||||
//! - `/compact` -- a compaction, start to finish.
|
//! - `/compact` -- a compaction, start to finish.
|
||||||
//! - `/stream N` -- one long answer in N small pieces, 50ms apart: the shape a
|
//! - `/stream N` -- one long answer in N small pieces, 50ms apart: the shape a
|
||||||
//! real model's reply arrives in, and the one where the row a reader is
|
//! real model's reply arrives in, and the one where the row a reader is
|
||||||
@@ -85,6 +92,10 @@ pub struct EchoDriver {
|
|||||||
/// because `/ask` puts up to four on one tool call, and the turn resumes
|
/// because `/ask` puts up to four on one tool call, and the turn resumes
|
||||||
/// when the last is answered rather than the first.
|
/// when the last is answered rather than the first.
|
||||||
pending_questions: Mutex<Vec<PendingQuestion>>,
|
pending_questions: Mutex<Vec<PendingQuestion>>,
|
||||||
|
/// The invented rate-limit answer `/usage` sets, shared with the usage
|
||||||
|
/// monitor that serves it. An echo session meters nothing, so this is unset
|
||||||
|
/// until a test asks for something -- see [`crate::usage::Fixture`].
|
||||||
|
usage: crate::usage::Fixture,
|
||||||
/// A pretend context, so the status row has something that behaves the way
|
/// A pretend context, so the status row has something that behaves the way
|
||||||
/// a real one does: it grows with each turn, drops to what the compaction
|
/// a real one does: it grows with each turn, drops to what the compaction
|
||||||
/// says it recovered, and a clear leaves it unmeasured. What is real is
|
/// says it recovered, and a clear leaves it unmeasured. What is real is
|
||||||
@@ -311,6 +322,28 @@ impl EchoDriver {
|
|||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Answered here rather than in the turn below, because it is not a
|
||||||
|
// turn: nothing is generated, and what is being exercised is the
|
||||||
|
// *other* screens -- the bar under the header, the button beside it and
|
||||||
|
// the dialog it opens, which read the usage route, not this transcript.
|
||||||
|
if let Some(rest) = text.strip_prefix("/usage") {
|
||||||
|
if announce {
|
||||||
|
self.emit(Event::MessageTaken {
|
||||||
|
id: None,
|
||||||
|
text: text.clone(),
|
||||||
|
attachments,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
let said = self.usage.command(rest);
|
||||||
|
self.emit(Event::AssistantText {
|
||||||
|
delta: format!("{said}\n"),
|
||||||
|
});
|
||||||
|
self.emit(Event::Status {
|
||||||
|
state: SessionStatus::Idle,
|
||||||
|
});
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
// The same word the real CLI takes, so a phone drives both the same way.
|
// The same word the real CLI takes, so a phone drives both the same way.
|
||||||
// `Driver::compact` is what the manager's route calls; this is the typed
|
// `Driver::compact` is what the manager's route calls; this is the typed
|
||||||
// path onto it.
|
// path onto it.
|
||||||
@@ -606,7 +639,7 @@ impl EchoDriver {
|
|||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
pub fn new(sink: EventSink, session_dir: PathBuf) -> Self {
|
pub fn new(sink: EventSink, session_dir: PathBuf, usage: crate::usage::Fixture) -> Self {
|
||||||
let driver = Self {
|
let driver = Self {
|
||||||
sink,
|
sink,
|
||||||
pending_questions: Mutex::new(Vec::new()),
|
pending_questions: Mutex::new(Vec::new()),
|
||||||
@@ -614,6 +647,7 @@ impl EchoDriver {
|
|||||||
busy: Arc::new(AtomicBool::new(false)),
|
busy: Arc::new(AtomicBool::new(false)),
|
||||||
queued: Arc::new(Mutex::new(Vec::new())),
|
queued: Arc::new(Mutex::new(Vec::new())),
|
||||||
session_dir,
|
session_dir,
|
||||||
|
usage,
|
||||||
};
|
};
|
||||||
driver.emit(Event::Status {
|
driver.emit(Event::Status {
|
||||||
state: SessionStatus::Idle,
|
state: SessionStatus::Idle,
|
||||||
|
|||||||
+137
-28
@@ -5,9 +5,19 @@
|
|||||||
//!
|
//!
|
||||||
//! **It is spawned but not spoken to over stdio.** The process is started
|
//! **It is spawned but not spoken to over stdio.** The process is started
|
||||||
//! through the same [`Transport`] as any other and then reached over HTTP on a
|
//! through the same [`Transport`] as any other and then reached over HTTP on a
|
||||||
//! loopback port. A remote llama-server would need its port forwarded as well
|
//! loopback port. That is the second half of what a transport is -- "run this"
|
||||||
//! as its command wrapped, which is not built, so a session on an ssh host is
|
//! plus "reach this port" -- and it is what lets a session run on another
|
||||||
//! refused rather than silently talking to the wrong machine.
|
//! machine: [`Transport::reserve_port`] hands back a port the server binds
|
||||||
|
//! *there* and one that reaches it *here*, and the ssh connection carrying the
|
||||||
|
//! command carries the tunnel between them. The far `llama-server` binds
|
||||||
|
//! loopback only, so a model is never served to that machine's network.
|
||||||
|
//!
|
||||||
|
//! **The model file is the far machine's, not this one's.** A remote setup
|
||||||
|
//! names its own models directory (`SshConfig::models_dir`, defaulting to where
|
||||||
|
//! this backend keeps its downloads), and the file is looked for *there* -- so
|
||||||
|
//! a session naming a model that machine does not have says so, instead of
|
||||||
|
//! starting a server that will never load one. Downloading to another machine
|
||||||
|
//! is not built; the model gets there however anything else does.
|
||||||
//!
|
//!
|
||||||
//! **The server is stateless between requests**, so the whole conversation goes
|
//! **The server is stateless between requests**, so the whole conversation goes
|
||||||
//! with every one. It is rebuilt from the session's transcript rather than kept
|
//! with every one. It is rebuilt from the session's transcript rather than kept
|
||||||
@@ -76,16 +86,10 @@ impl LlamaDriver {
|
|||||||
session_dir: &Path,
|
session_dir: &Path,
|
||||||
sink: EventSink,
|
sink: EventSink,
|
||||||
) -> Result<Self> {
|
) -> Result<Self> {
|
||||||
if !matches!(transport, Transport::Here) {
|
|
||||||
bail!(
|
|
||||||
"llama.cpp sessions can only run on this machine for now: the model is served \
|
|
||||||
over HTTP, and forwarding that port to another host isn't built yet."
|
|
||||||
);
|
|
||||||
}
|
|
||||||
let model = meta.model.as_deref().context(
|
let model = meta.model.as_deref().context(
|
||||||
"a llama.cpp session needs a model -- one of the downloaded ones, by its key",
|
"a llama.cpp session needs a model -- one of the downloaded ones, by its key",
|
||||||
)?;
|
)?;
|
||||||
let path = model_path(models_dir, model)?;
|
let path = model_on(transport, models_dir, model)?;
|
||||||
|
|
||||||
// Already loaded and still running: keep talking to it. The health poll
|
// Already loaded and still running: keep talking to it. The health poll
|
||||||
// below confirms it is really answering, so adopting a pid whose server
|
// below confirms it is really answering, so adopting a pid whose server
|
||||||
@@ -111,14 +115,21 @@ impl LlamaDriver {
|
|||||||
));
|
));
|
||||||
}
|
}
|
||||||
|
|
||||||
let port = free_port().context("finding a port for llama-server")?;
|
// Where it listens on its own machine, and where that is reached
|
||||||
|
// from here -- the same number when that machine is this one.
|
||||||
|
let forward = transport
|
||||||
|
.reserve_port()
|
||||||
|
.context("finding a port for llama-server")?;
|
||||||
let mut args: Vec<String> = vec![
|
let mut args: Vec<String> = vec![
|
||||||
"-m".into(),
|
"-m".into(),
|
||||||
path.to_string_lossy().into_owned(),
|
path.clone(),
|
||||||
|
// Loopback there, whichever machine there is: what reaches it
|
||||||
|
// from outside that machine is the ssh tunnel and nothing
|
||||||
|
// else.
|
||||||
"--host".into(),
|
"--host".into(),
|
||||||
"127.0.0.1".into(),
|
"127.0.0.1".into(),
|
||||||
"--port".into(),
|
"--port".into(),
|
||||||
port.to_string(),
|
forward.there.to_string(),
|
||||||
];
|
];
|
||||||
// Settings that belong to the server because they decide how the model
|
// Settings that belong to the server because they decide how the model
|
||||||
// is loaded; the sampling ones ride on each request instead, so changing
|
// is loaded; the sampling ones ride on each request instead, so changing
|
||||||
@@ -135,7 +146,7 @@ impl LlamaDriver {
|
|||||||
}
|
}
|
||||||
|
|
||||||
let program = provider.command.as_deref().unwrap_or("llama-server");
|
let program = provider.command.as_deref().unwrap_or("llama-server");
|
||||||
let launch = Launch::new(program, args, meta.cwd.as_deref());
|
let launch = Launch::new(program, args, meta.cwd.as_deref()).reaching(forward);
|
||||||
// Its output goes to files, not pipes. Not only so the process can
|
// Its output goes to files, not pipes. Not only so the process can
|
||||||
// outlive this server: nothing ever read those pipes, so a chatty
|
// outlive this server: nothing ever read those pipes, so a chatty
|
||||||
// llama-server filled the 64 KB buffer and blocked mid-load with no sign
|
// llama-server filled the 64 KB buffer and blocked mid-load with no sign
|
||||||
@@ -152,8 +163,12 @@ impl LlamaDriver {
|
|||||||
.id()
|
.id()
|
||||||
.context("llama-server exited before it could be recorded")?;
|
.context("llama-server exited before it could be recorded")?;
|
||||||
tracing::info!(
|
tracing::info!(
|
||||||
"session {} running {program} for {model} on 127.0.0.1:{port} as pid {pid}",
|
"session {} running {program} for {model} {} on 127.0.0.1:{} there, \
|
||||||
meta.id
|
reached at 127.0.0.1:{} here, as pid {pid}",
|
||||||
|
meta.id,
|
||||||
|
transport.describe(),
|
||||||
|
forward.there,
|
||||||
|
forward.here,
|
||||||
);
|
);
|
||||||
// Reaped so it does not become a zombie while this server is still its
|
// Reaped so it does not become a zombie while this server is still its
|
||||||
// parent; the health poll and the record are what say whether the
|
// parent; the health poll and the record are what say whether the
|
||||||
@@ -163,12 +178,18 @@ impl LlamaDriver {
|
|||||||
let _ = child.wait().await;
|
let _ = child.wait().await;
|
||||||
});
|
});
|
||||||
|
|
||||||
let record = process::Record::of(pid, process::Detail::Http { port })
|
// The *near* port, because that is the one anything reaching this
|
||||||
|
// server has to dial -- including a later run of this backend,
|
||||||
|
// which adopts the record without knowing which machine the server
|
||||||
|
// is on. For a remote session the recorded pid is the ssh
|
||||||
|
// client's, which is the process this machine owns and which holds
|
||||||
|
// the tunnel open for exactly as long as the far server lives.
|
||||||
|
let record = process::Record::of(pid, process::Detail::Http { port: forward.here })
|
||||||
.context("llama-server was gone before its start time could be read")?;
|
.context("llama-server was gone before its start time could be read")?;
|
||||||
process::write(session_dir, &record);
|
process::write(session_dir, &record);
|
||||||
|
|
||||||
Ok(Self::attached(
|
Ok(Self::attached(
|
||||||
format!("http://127.0.0.1:{port}"),
|
format!("http://127.0.0.1:{}", forward.here),
|
||||||
meta,
|
meta,
|
||||||
model,
|
model,
|
||||||
transcript,
|
transcript,
|
||||||
@@ -202,7 +223,7 @@ impl LlamaDriver {
|
|||||||
let endpoint = endpoint.clone();
|
let endpoint = endpoint.clone();
|
||||||
let model = model.to_string();
|
let model = model.to_string();
|
||||||
let session_dir = session_dir.to_path_buf();
|
let session_dir = session_dir.to_path_buf();
|
||||||
std::thread::spawn(move || match wait_until_ready(&endpoint) {
|
std::thread::spawn(move || match wait_until_ready(&endpoint, &session_dir) {
|
||||||
Ok(()) => {
|
Ok(()) => {
|
||||||
tracing::info!("{model} loaded and answering at {endpoint}");
|
tracing::info!("{model} loaded and answering at {endpoint}");
|
||||||
let _ = sink.send(Event::Status {
|
let _ = sink.send(Event::Status {
|
||||||
@@ -499,17 +520,70 @@ fn model_path(models_dir: &Path, key: &str) -> Result<PathBuf> {
|
|||||||
Ok(path)
|
Ok(path)
|
||||||
}
|
}
|
||||||
|
|
||||||
/// An unused loopback port, by asking the OS for one and letting it go. Racy in
|
/// The model file's path **on the machine that will serve it**, confirmed to be
|
||||||
/// principle, but nothing on this machine is hunting for ports, and the
|
/// there.
|
||||||
/// alternative -- parsing the port back out of the server's log -- couples us to
|
///
|
||||||
/// its output format for no real gain.
|
/// One function rather than a local check and hope for the other case: the same
|
||||||
fn free_port() -> Result<u16> {
|
/// question has to be asked of two filesystems. The remote answer is measured
|
||||||
let listener = std::net::TcpListener::bind("127.0.0.1:0")?;
|
/// for the reason the local one is -- a missing file otherwise becomes a
|
||||||
Ok(listener.local_addr()?.port())
|
/// `llama-server` that starts, fails to load, and reports as a session that
|
||||||
|
/// never became ready, which reads as the machine being slow.
|
||||||
|
///
|
||||||
|
/// One blocking round trip on a remote spawn, which is what the spawn is
|
||||||
|
/// already paying to start ssh. The alternative is a path built here from a `~`
|
||||||
|
/// this machine cannot expand.
|
||||||
|
fn model_on(transport: &Transport, models_dir: &Path, key: &str) -> Result<String> {
|
||||||
|
let Transport::Ssh { name, .. } = transport else {
|
||||||
|
return Ok(model_path(models_dir, key)?.to_string_lossy().into_owned());
|
||||||
|
};
|
||||||
|
// The same directory the spawn screen listed for this machine, and one
|
||||||
|
// function for the same reason: a list from one place and a load from
|
||||||
|
// another is a model that appears and then fails.
|
||||||
|
let dir = crate::models::dir_on(transport, models_dir);
|
||||||
|
// Checked here rather than in the script: `..` in a key would walk out of
|
||||||
|
// the models directory on a machine this server can start processes on,
|
||||||
|
// and the phone is where the key comes from.
|
||||||
|
for part in key.split('/') {
|
||||||
|
if part.is_empty() || part == "." || part == ".." {
|
||||||
|
bail!("\"{key}\" is not a model key this can resolve");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
let path = format!("{}/{key}", dir.trim_end_matches('/'));
|
||||||
|
// `$HOME` on the far side, which is the only machine that knows what it is,
|
||||||
|
// and the resolved path printed back so the launch hands `llama-server`
|
||||||
|
// something absolute. "Not there" is answered rather than failed, because a
|
||||||
|
// machine that could not be asked at all has to say so in its own words --
|
||||||
|
// it would otherwise arrive as this same sentence about a missing model.
|
||||||
|
let script = "p=$1; case $p in \"~\") p=$HOME;; \"~/\"*) p=$HOME/${p#\"~/\"};; esac; \
|
||||||
|
[ -f \"$p\" ] && printf 'at\\t%s\\n' \"$p\" || printf 'missing\\n'"
|
||||||
|
.to_string();
|
||||||
|
let launch = Launch::new(
|
||||||
|
"sh",
|
||||||
|
vec!["-c".to_string(), script, "sh".to_string(), path.clone()],
|
||||||
|
None,
|
||||||
|
);
|
||||||
|
let answer = transport
|
||||||
|
.capture_blocking(&launch)
|
||||||
|
.with_context(|| format!("couldn't ask {name} where its models are"))?;
|
||||||
|
match answer.trim().split_once('\t') {
|
||||||
|
Some(("at", resolved)) => Ok(resolved.to_string()),
|
||||||
|
_ => bail!(
|
||||||
|
"{name} has no model at {path}. A llama.cpp session serves the file from the \
|
||||||
|
machine it runs on, so the model has to be on {name} -- what this backend has \
|
||||||
|
downloaded is somewhere else."
|
||||||
|
),
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Polls until the server says it is ready, or gives up.
|
/// Polls until the server says it is ready, or gives up.
|
||||||
fn wait_until_ready(endpoint: &str) -> Result<()> {
|
///
|
||||||
|
/// Watches the process as well as the port, because the two failures need
|
||||||
|
/// different words and one of them is common: a model that will not load,
|
||||||
|
/// a port already taken on the far machine, a `llama-server` too old for
|
||||||
|
/// a flag. All of those exit within a second and none of them will ever
|
||||||
|
/// answer `/health`, so waiting out the timeout turns a server that said
|
||||||
|
/// exactly what was wrong into "gave up after 300s".
|
||||||
|
fn wait_until_ready(endpoint: &str, session_dir: &Path) -> Result<()> {
|
||||||
let deadline = std::time::Instant::now() + READY_TIMEOUT;
|
let deadline = std::time::Instant::now() + READY_TIMEOUT;
|
||||||
let url = format!("{endpoint}/health");
|
let url = format!("{endpoint}/health");
|
||||||
loop {
|
loop {
|
||||||
@@ -518,13 +592,48 @@ fn wait_until_ready(endpoint: &str) -> Result<()> {
|
|||||||
{
|
{
|
||||||
return Ok(());
|
return Ok(());
|
||||||
}
|
}
|
||||||
|
// `None` is the session having been stopped or deleted while this
|
||||||
|
// waited, which is nobody's fault and still not worth waiting on.
|
||||||
|
match process::recorded(session_dir) {
|
||||||
|
Some((_, process::Liveness::Alive | process::Liveness::Unknown)) => {}
|
||||||
|
Some((_, process::Liveness::Dead)) | None => {
|
||||||
|
bail!("it exited before it answered.{}", log_tail(session_dir));
|
||||||
|
}
|
||||||
|
}
|
||||||
if std::time::Instant::now() > deadline {
|
if std::time::Instant::now() > deadline {
|
||||||
bail!("gave up after {}s", READY_TIMEOUT.as_secs());
|
bail!(
|
||||||
|
"gave up after {}s.{}",
|
||||||
|
READY_TIMEOUT.as_secs(),
|
||||||
|
log_tail(session_dir)
|
||||||
|
);
|
||||||
}
|
}
|
||||||
std::thread::sleep(std::time::Duration::from_millis(250));
|
std::thread::sleep(std::time::Duration::from_millis(250));
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// The end of `llama-server`'s own log, for a failure message.
|
||||||
|
///
|
||||||
|
/// Its account of what went wrong is the useful half -- "failed to load
|
||||||
|
/// model", "bind: Address already in use" -- and on a remote session it
|
||||||
|
/// is the only half, since nobody reading the phone can open a file on
|
||||||
|
/// that machine. Bounded, because this ends up in an event a phone draws.
|
||||||
|
fn log_tail(session_dir: &Path) -> String {
|
||||||
|
let Ok(text) = std::fs::read_to_string(session_dir.join(SERVER_LOG)) else {
|
||||||
|
return String::new();
|
||||||
|
};
|
||||||
|
let tail: Vec<&str> = text.lines().rev().take(LOG_TAIL_LINES).collect();
|
||||||
|
if tail.is_empty() {
|
||||||
|
return String::new();
|
||||||
|
}
|
||||||
|
format!(
|
||||||
|
" It last said: {}",
|
||||||
|
tail.into_iter().rev().collect::<Vec<_>>().join(" / ")
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// How much of that log to carry into a message somebody reads on a phone.
|
||||||
|
const LOG_TAIL_LINES: usize = 6;
|
||||||
|
|
||||||
/// One streamed completion: posts the conversation, emits each delta as it
|
/// One streamed completion: posts the conversation, emits each delta as it
|
||||||
/// arrives. Emits rather than returns, because the transcript those events land
|
/// arrives. Emits rather than returns, because the transcript those events land
|
||||||
/// in is what the next turn reads back.
|
/// in is what the next turn reads back.
|
||||||
|
|||||||
+86
-24
@@ -135,6 +135,17 @@ pub struct SessionInfo {
|
|||||||
/// that happens to be big" are different answers.
|
/// that happens to be big" are different answers.
|
||||||
#[serde(skip_serializing_if = "Option::is_none")]
|
#[serde(skip_serializing_if = "Option::is_none")]
|
||||||
pub max_image_edge: Option<u32>,
|
pub max_image_edge: Option<u32>,
|
||||||
|
/// Which of `GET /usage`'s snapshots reports on this session, and
|
||||||
|
/// absent where nothing meters it -- see
|
||||||
|
/// [`DriverKind::usage_provider`].
|
||||||
|
///
|
||||||
|
/// Reported for the same reason `keeps_own_transcript` is: it is a
|
||||||
|
/// fact about the provider's *kind*, and the phone has only its name.
|
||||||
|
/// Pairing by machine alone was the bug it exists to fix -- one
|
||||||
|
/// machine runs echo and the Claude CLI, so every echo session drew
|
||||||
|
/// the CLI's five-hour window as if it were its own.
|
||||||
|
#[serde(skip_serializing_if = "Option::is_none")]
|
||||||
|
pub usage_provider: Option<&'static str>,
|
||||||
/// Whether this session announces itself -- reported for the same
|
/// Whether this session announces itself -- reported for the same
|
||||||
/// reason `permission_mode` is: a switch that guesses its own position
|
/// reason `permission_mode` is: a switch that guesses its own position
|
||||||
/// is how you turn something off while believing you are reading it.
|
/// is how you turn something off while believing you are reading it.
|
||||||
@@ -438,6 +449,7 @@ impl LiveSession {
|
|||||||
context_tokens: *self.shared.context_tokens.lock().unwrap(),
|
context_tokens: *self.shared.context_tokens.lock().unwrap(),
|
||||||
notify: *self.shared.notify.lock().unwrap(),
|
notify: *self.shared.notify.lock().unwrap(),
|
||||||
max_image_edge: kind.and_then(DriverKind::max_image_edge),
|
max_image_edge: kind.and_then(DriverKind::max_image_edge),
|
||||||
|
usage_provider: kind.and_then(DriverKind::usage_provider),
|
||||||
imported,
|
imported,
|
||||||
keeps_own_transcript: kind.is_some_and(DriverKind::keeps_own_transcript),
|
keeps_own_transcript: kind.is_some_and(DriverKind::keeps_own_transcript),
|
||||||
cwd: cwd.map(Path::to_path_buf),
|
cwd: cwd.map(Path::to_path_buf),
|
||||||
@@ -470,6 +482,11 @@ pub struct SessionManager {
|
|||||||
/// What to mark sessions spawned here as -- see
|
/// What to mark sessions spawned here as -- see
|
||||||
/// [`SessionManager::marking_new_sessions_throwaway`].
|
/// [`SessionManager::marking_new_sessions_throwaway`].
|
||||||
spawn_throwaway: bool,
|
spawn_throwaway: bool,
|
||||||
|
/// The invented rate-limit answer an echo session's `/usage` sets,
|
||||||
|
/// shared with the usage monitor that serves it. Held here because
|
||||||
|
/// every echo driver this manager builds is handed a clone -- see
|
||||||
|
/// [`SessionManager::reporting_usage_fixture`].
|
||||||
|
usage_fixture: crate::usage::Fixture,
|
||||||
inner: RwLock<Inner>,
|
inner: RwLock<Inner>,
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -485,6 +502,11 @@ impl SessionManager {
|
|||||||
wg_app_link::private::create_dir(&data_dir)?;
|
wg_app_link::private::create_dir(&data_dir)?;
|
||||||
|
|
||||||
let (notifications, _) = broadcast::channel(NOTIFICATION_BUFFER);
|
let (notifications, _) = broadcast::channel(NOTIFICATION_BUFFER);
|
||||||
|
// Made here rather than passed in, and handed *out* to the usage
|
||||||
|
// monitor by whoever wires the two together: every echo driver
|
||||||
|
// this manager builds gets a clone, including the ones built
|
||||||
|
// below, so it has to exist before the first session does.
|
||||||
|
let usage_fixture = crate::usage::Fixture::new();
|
||||||
let mut live = HashMap::new();
|
let mut live = HashMap::new();
|
||||||
for meta in &config.sessions {
|
for meta in &config.sessions {
|
||||||
// One unlaunchable session -- a corrupt transcript, an
|
// One unlaunchable session -- a corrupt transcript, an
|
||||||
@@ -495,8 +517,11 @@ impl SessionManager {
|
|||||||
meta.clone(),
|
meta.clone(),
|
||||||
&setup,
|
&setup,
|
||||||
&provider,
|
&provider,
|
||||||
&data_dir,
|
Env {
|
||||||
&models_dir,
|
data_dir: &data_dir,
|
||||||
|
models_dir: &models_dir,
|
||||||
|
usage: &usage_fixture,
|
||||||
|
},
|
||||||
notifications.clone(),
|
notifications.clone(),
|
||||||
// Nothing is started here; see `Launching`.
|
// Nothing is started here; see `Launching`.
|
||||||
Launching::Restart,
|
Launching::Restart,
|
||||||
@@ -517,11 +542,41 @@ impl SessionManager {
|
|||||||
notifications,
|
notifications,
|
||||||
pending: Arc::new(pending::Registry::default()),
|
pending: Arc::new(pending::Registry::default()),
|
||||||
spawn_throwaway: false,
|
spawn_throwaway: false,
|
||||||
|
usage_fixture,
|
||||||
inner: RwLock::new(Inner { config, live }),
|
inner: RwLock::new(Inner { config, live }),
|
||||||
};
|
};
|
||||||
Ok(manager)
|
Ok(manager)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Where this backend's own model downloads live. The machine a
|
||||||
|
/// session runs on may keep its elsewhere -- see `models::dir_on`.
|
||||||
|
pub fn models_dir(&self) -> &Path {
|
||||||
|
&self.models_dir
|
||||||
|
}
|
||||||
|
|
||||||
|
/// What this manager lends a session it launches. Borrowed from the
|
||||||
|
/// manager rather than cloned, so there is one answer to where things
|
||||||
|
/// are kept.
|
||||||
|
fn env(&self) -> Env<'_> {
|
||||||
|
Env {
|
||||||
|
data_dir: &self.data_dir,
|
||||||
|
models_dir: &self.models_dir,
|
||||||
|
usage: &self.usage_fixture,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The invented rate-limit answer this manager's echo sessions set
|
||||||
|
/// with `/usage`, for the usage monitor to serve.
|
||||||
|
///
|
||||||
|
/// Handed out rather than taken in because the drivers built inside
|
||||||
|
/// the constructor need it, and because the direction is the one the
|
||||||
|
/// layering allows: `usage` sits below the session layer, so a
|
||||||
|
/// session can hold one of its types while it holds nothing of a
|
||||||
|
/// session's.
|
||||||
|
pub fn usage_fixture(&self) -> crate::usage::Fixture {
|
||||||
|
self.usage_fixture.clone()
|
||||||
|
}
|
||||||
|
|
||||||
/// Marks every session spawned from here on as one whose process is
|
/// Marks every session spawned from here on as one whose process is
|
||||||
/// stopped when this server exits. Set from `--throwaway-sessions`,
|
/// stopped when this server exits. Set from `--throwaway-sessions`,
|
||||||
/// which a debug build defaults to on. It decides only what a *new*
|
/// which a debug build defaults to on. It decides only what a *new*
|
||||||
@@ -861,6 +916,8 @@ impl SessionManager {
|
|||||||
context_tokens: None,
|
context_tokens: None,
|
||||||
max_image_edge: kind_of(&inner.config, &meta.setup, &meta.provider)
|
max_image_edge: kind_of(&inner.config, &meta.setup, &meta.provider)
|
||||||
.and_then(DriverKind::max_image_edge),
|
.and_then(DriverKind::max_image_edge),
|
||||||
|
usage_provider: kind_of(&inner.config, &meta.setup, &meta.provider)
|
||||||
|
.and_then(DriverKind::usage_provider),
|
||||||
notify: meta.notify,
|
notify: meta.notify,
|
||||||
imported: import::read_cursor(&self.data_dir.join(&meta.id)).is_some(),
|
imported: import::read_cursor(&self.data_dir.join(&meta.id)).is_some(),
|
||||||
keeps_own_transcript: keeps_own_transcript(
|
keeps_own_transcript: keeps_own_transcript(
|
||||||
@@ -974,8 +1031,7 @@ impl SessionManager {
|
|||||||
meta.clone(),
|
meta.clone(),
|
||||||
&setup,
|
&setup,
|
||||||
&provider,
|
&provider,
|
||||||
&self.data_dir,
|
self.env(),
|
||||||
&self.models_dir,
|
|
||||||
self.notifications.clone(),
|
self.notifications.clone(),
|
||||||
Launching::Asked(seed),
|
Launching::Asked(seed),
|
||||||
)?;
|
)?;
|
||||||
@@ -1372,7 +1428,7 @@ impl SessionManager {
|
|||||||
&meta,
|
&meta,
|
||||||
&setup,
|
&setup,
|
||||||
&provider,
|
&provider,
|
||||||
&self.models_dir,
|
self.env(),
|
||||||
session.dir(),
|
session.dir(),
|
||||||
session.transcript_path(),
|
session.transcript_path(),
|
||||||
&session.sink,
|
&session.sink,
|
||||||
@@ -1385,8 +1441,7 @@ impl SessionManager {
|
|||||||
meta,
|
meta,
|
||||||
&setup,
|
&setup,
|
||||||
&provider,
|
&provider,
|
||||||
&self.data_dir,
|
self.env(),
|
||||||
&self.models_dir,
|
|
||||||
self.notifications.clone(),
|
self.notifications.clone(),
|
||||||
Launching::Asked(None),
|
Launching::Asked(None),
|
||||||
)?;
|
)?;
|
||||||
@@ -1760,6 +1815,19 @@ enum Launching {
|
|||||||
Restart,
|
Restart,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// What the server around a session lends it: where sessions and models
|
||||||
|
/// are kept, and the usage fixture an echo session's `/usage` sets.
|
||||||
|
///
|
||||||
|
/// One parameter rather than three because they travel together through
|
||||||
|
/// every launch path and none of them is a fact about the session --
|
||||||
|
/// they are this server's belongings, handed down.
|
||||||
|
#[derive(Clone, Copy)]
|
||||||
|
struct Env<'a> {
|
||||||
|
data_dir: &'a Path,
|
||||||
|
models_dir: &'a Path,
|
||||||
|
usage: &'a crate::usage::Fixture,
|
||||||
|
}
|
||||||
|
|
||||||
/// Creates the session directory, opens its transcript (continuing the
|
/// Creates the session directory, opens its transcript (continuing the
|
||||||
/// sequence numbering if one exists), settles what the session is doing,
|
/// sequence numbering if one exists), settles what the session is doing,
|
||||||
/// and spawns the event pump -- with a driver behind it where there is a
|
/// and spawns the event pump -- with a driver behind it where there is a
|
||||||
@@ -1768,12 +1836,11 @@ fn launch(
|
|||||||
meta: SessionConfig,
|
meta: SessionConfig,
|
||||||
setup: &SetupConfig,
|
setup: &SetupConfig,
|
||||||
provider: &ProviderConfig,
|
provider: &ProviderConfig,
|
||||||
data_dir: &Path,
|
env: Env<'_>,
|
||||||
models_dir: &Path,
|
|
||||||
notifications: broadcast::Sender<Notification>,
|
notifications: broadcast::Sender<Notification>,
|
||||||
why: Launching,
|
why: Launching,
|
||||||
) -> Result<Arc<LiveSession>> {
|
) -> Result<Arc<LiveSession>> {
|
||||||
let dir = data_dir.join(&meta.id);
|
let dir = env.data_dir.join(&meta.id);
|
||||||
wg_app_link::private::create_dir(&dir)?;
|
wg_app_link::private::create_dir(&dir)?;
|
||||||
let transcript_path = dir.join("transcript.jsonl");
|
let transcript_path = dir.join("transcript.jsonl");
|
||||||
let mut transcript = Transcript::open(&transcript_path)?;
|
let mut transcript = Transcript::open(&transcript_path)?;
|
||||||
@@ -1887,17 +1954,7 @@ fn launch(
|
|||||||
|
|
||||||
let driver = Arc::new(Mutex::new(
|
let driver = Arc::new(Mutex::new(
|
||||||
driving
|
driving
|
||||||
.then(|| {
|
.then(|| make_driver(&meta, setup, provider, env, &dir, &transcript_path, &sink))
|
||||||
make_driver(
|
|
||||||
&meta,
|
|
||||||
setup,
|
|
||||||
provider,
|
|
||||||
models_dir,
|
|
||||||
&dir,
|
|
||||||
&transcript_path,
|
|
||||||
&sink,
|
|
||||||
)
|
|
||||||
})
|
|
||||||
.transpose()?,
|
.transpose()?,
|
||||||
));
|
));
|
||||||
|
|
||||||
@@ -1939,18 +1996,22 @@ fn make_driver(
|
|||||||
meta: &SessionConfig,
|
meta: &SessionConfig,
|
||||||
setup: &SetupConfig,
|
setup: &SetupConfig,
|
||||||
provider: &ProviderConfig,
|
provider: &ProviderConfig,
|
||||||
models_dir: &Path,
|
env: Env<'_>,
|
||||||
dir: &Path,
|
dir: &Path,
|
||||||
transcript_path: &Path,
|
transcript_path: &Path,
|
||||||
sink: &EventSink,
|
sink: &EventSink,
|
||||||
) -> Result<Arc<dyn Driver>> {
|
) -> Result<Arc<dyn Driver>> {
|
||||||
Ok(match provider.kind {
|
Ok(match provider.kind {
|
||||||
DriverKind::Echo => Arc::new(EchoDriver::new(sink.clone(), dir.to_path_buf())),
|
DriverKind::Echo => Arc::new(EchoDriver::new(
|
||||||
|
sink.clone(),
|
||||||
|
dir.to_path_buf(),
|
||||||
|
env.usage.clone(),
|
||||||
|
)),
|
||||||
DriverKind::LlamaCpp => Arc::new(LlamaDriver::launch(
|
DriverKind::LlamaCpp => Arc::new(LlamaDriver::launch(
|
||||||
meta,
|
meta,
|
||||||
provider,
|
provider,
|
||||||
&Transport::for_setup(setup),
|
&Transport::for_setup(setup),
|
||||||
models_dir,
|
env.models_dir,
|
||||||
transcript_path,
|
transcript_path,
|
||||||
dir,
|
dir,
|
||||||
sink.clone(),
|
sink.clone(),
|
||||||
@@ -2283,6 +2344,7 @@ mod tests {
|
|||||||
driver: Arc::new(Mutex::new(Some(Arc::new(EchoDriver::new(
|
driver: Arc::new(Mutex::new(Some(Arc::new(EchoDriver::new(
|
||||||
sink.clone(),
|
sink.clone(),
|
||||||
dir.path().to_path_buf(),
|
dir.path().to_path_buf(),
|
||||||
|
crate::usage::Fixture::new(),
|
||||||
))))),
|
))))),
|
||||||
sink,
|
sink,
|
||||||
waiting: Mutex::new(VecDeque::new()),
|
waiting: Mutex::new(VecDeque::new()),
|
||||||
|
|||||||
@@ -11,9 +11,13 @@
|
|||||||
//! `crate::ssh`'s: this module decides *which* transport, that one knows what a
|
//! `crate::ssh`'s: this module decides *which* transport, that one knows what a
|
||||||
//! correct ssh invocation is.
|
//! correct ssh invocation is.
|
||||||
//!
|
//!
|
||||||
//! Known second operation, not built because nothing needs it yet: a managed
|
//! A transport is therefore two operations rather than one: **run this** and
|
||||||
//! `llama-server` is spawned as a process but then spoken to over HTTP, so a
|
//! **reach this port**. The second is what a managed `llama-server` needs -- it
|
||||||
//! remote one needs a forwarded port (`ssh -L`) as well.
|
//! is spawned as a process and then spoken to over HTTP -- and it is a no-op
|
||||||
|
//! locally, where the port a program binds is already one this machine can
|
||||||
|
//! dial. Over ssh it is an `-L` tunnel on the same connection that runs the
|
||||||
|
//! command, so the model server binds loopback on the far machine and is never
|
||||||
|
//! exposed to its network. See [`Transport::reserve_port`].
|
||||||
|
|
||||||
use std::path::{Path, PathBuf};
|
use std::path::{Path, PathBuf};
|
||||||
use std::process::Stdio;
|
use std::process::Stdio;
|
||||||
@@ -22,14 +26,26 @@ use anyhow::{Context, Result};
|
|||||||
use tokio::process::Child;
|
use tokio::process::Child;
|
||||||
|
|
||||||
use crate::config::SshConfig;
|
use crate::config::SshConfig;
|
||||||
|
pub use crate::ssh::Forward;
|
||||||
|
|
||||||
/// What a driver needs run in order to exist as a process. Deliberately just
|
/// What a driver needs run in order to exist as a process. Deliberately just
|
||||||
/// the three things every transport can carry; anything a particular machine
|
/// what every transport can carry -- the command, where it runs, and a port the
|
||||||
/// needs is the transport's own configuration, not something a driver states.
|
/// caller needs to reach; anything a particular machine needs is the
|
||||||
|
/// transport's own configuration, not something a driver states.
|
||||||
pub struct Launch {
|
pub struct Launch {
|
||||||
pub program: String,
|
pub program: String,
|
||||||
pub args: Vec<String>,
|
pub args: Vec<String>,
|
||||||
pub cwd: Option<PathBuf>,
|
pub cwd: Option<PathBuf>,
|
||||||
|
/// A port this program will listen on, and the port that reaches it
|
||||||
|
/// from here -- see [`Transport::reserve_port`], which is the only
|
||||||
|
/// thing that should produce one.
|
||||||
|
///
|
||||||
|
/// On the launch rather than in [`Transport::spawn`]'s signature
|
||||||
|
/// because it is part of what is being run: a caller that needs to
|
||||||
|
/// reach the process it is starting says so once, where it says
|
||||||
|
/// everything else about it, and every transport reads it the same
|
||||||
|
/// way.
|
||||||
|
pub forward: Option<Forward>,
|
||||||
}
|
}
|
||||||
|
|
||||||
impl Launch {
|
impl Launch {
|
||||||
@@ -38,8 +54,16 @@ impl Launch {
|
|||||||
program: program.into(),
|
program: program.into(),
|
||||||
args,
|
args,
|
||||||
cwd: cwd.map(Path::to_path_buf),
|
cwd: cwd.map(Path::to_path_buf),
|
||||||
|
forward: None,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Says that this program serves `forward.there`, and that the caller
|
||||||
|
/// will reach it at `forward.here`.
|
||||||
|
pub fn reaching(mut self, forward: Forward) -> Self {
|
||||||
|
self.forward = Some(forward);
|
||||||
|
self
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/// How a launched process's standard streams are connected.
|
/// How a launched process's standard streams are connected.
|
||||||
@@ -102,6 +126,7 @@ impl Transport {
|
|||||||
&launch.program,
|
&launch.program,
|
||||||
&launch.args,
|
&launch.args,
|
||||||
launch.cwd.as_deref(),
|
launch.cwd.as_deref(),
|
||||||
|
launch.forward,
|
||||||
));
|
));
|
||||||
match streams {
|
match streams {
|
||||||
Streams::Piped => {
|
Streams::Piped => {
|
||||||
@@ -153,12 +178,15 @@ impl Transport {
|
|||||||
Self::Here => None,
|
Self::Here => None,
|
||||||
Self::Ssh { ssh, .. } => Some(ssh),
|
Self::Ssh { ssh, .. } => Some(ssh),
|
||||||
};
|
};
|
||||||
let output =
|
let output = crate::ssh::command(
|
||||||
crate::ssh::command(host, &launch.program, &launch.args, launch.cwd.as_deref())
|
host,
|
||||||
|
&launch.program,
|
||||||
|
&launch.args,
|
||||||
|
launch.cwd.as_deref(),
|
||||||
|
launch.forward,
|
||||||
|
)
|
||||||
.output()
|
.output()
|
||||||
.with_context(|| {
|
.with_context(|| format!("couldn't run \"{}\" {}", launch.program, self.describe()))?;
|
||||||
format!("couldn't run \"{}\" {}", launch.program, self.describe())
|
|
||||||
})?;
|
|
||||||
if !output.status.success() {
|
if !output.status.success() {
|
||||||
let stderr = String::from_utf8_lossy(&output.stderr).trim().to_string();
|
let stderr = String::from_utf8_lossy(&output.stderr).trim().to_string();
|
||||||
anyhow::bail!(if stderr.is_empty() {
|
anyhow::bail!(if stderr.is_empty() {
|
||||||
@@ -217,6 +245,34 @@ impl Transport {
|
|||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Picks a port for a launched program to serve on, and the port that
|
||||||
|
/// reaches it from here.
|
||||||
|
///
|
||||||
|
/// The "reach this port" half of what a transport is. Locally there is
|
||||||
|
/// one port and the OS chooses it, by binding and letting go -- racy
|
||||||
|
/// in principle, and nothing on this machine is hunting for ports.
|
||||||
|
///
|
||||||
|
/// Over ssh the near end is chosen the same way and the far end is a
|
||||||
|
/// guess, because there is no portable way to ask a machine for a free
|
||||||
|
/// port that does not race with binding it anyway. It is taken from
|
||||||
|
/// [`FAR_PORTS`], below the range Linux hands out to outgoing
|
||||||
|
/// connections, so a collision means something else deliberately
|
||||||
|
/// listening there. That is not silent: the program fails to bind and
|
||||||
|
/// exits, and `session::llama` reports what its log said rather than
|
||||||
|
/// waiting out its readiness timeout.
|
||||||
|
pub fn reserve_port(&self) -> Result<Forward> {
|
||||||
|
let listener = std::net::TcpListener::bind("127.0.0.1:0")
|
||||||
|
.context("asking this machine for a free port")?;
|
||||||
|
let here = listener.local_addr()?.port();
|
||||||
|
Ok(match self {
|
||||||
|
Self::Here => Forward { there: here, here },
|
||||||
|
Self::Ssh { .. } => Forward {
|
||||||
|
there: rand::random_range(FAR_PORTS),
|
||||||
|
here,
|
||||||
|
},
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
/// How to say where this runs, for a log line a person reads.
|
/// How to say where this runs, for a log line a person reads.
|
||||||
pub fn describe(&self) -> String {
|
pub fn describe(&self) -> String {
|
||||||
match self {
|
match self {
|
||||||
@@ -226,6 +282,11 @@ impl Transport {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Where a port on another machine is guessed from: high enough to be out
|
||||||
|
/// of the way of services, and below the 32768-60999 Linux hands out to
|
||||||
|
/// outgoing connections, which is where a guess would most often collide.
|
||||||
|
const FAR_PORTS: std::ops::Range<u16> = 20000..30000;
|
||||||
|
|
||||||
/// What a command is given on its standard input.
|
/// What a command is given on its standard input.
|
||||||
///
|
///
|
||||||
/// Three cases rather than an `Option<Stdio>` because they are three genuinely
|
/// Three cases rather than an `Option<Stdio>` because they are three genuinely
|
||||||
|
|||||||
@@ -25,7 +25,10 @@ use crate::session::transport::{Launch, Transport};
|
|||||||
/// session stores.
|
/// session stores.
|
||||||
const PROBES: &[(&str, &str, DriverKind)] = &[
|
const PROBES: &[(&str, &str, DriverKind)] = &[
|
||||||
("claude-cli", "claude", DriverKind::ClaudeCli),
|
("claude-cli", "claude", DriverKind::ClaudeCli),
|
||||||
("local-llama", "llama-server", DriverKind::LlamaCpp),
|
// Named for the program rather than for where it runs: it runs
|
||||||
|
// wherever the setup is, and "local" was true only while a llama
|
||||||
|
// session could not be spawned on another machine.
|
||||||
|
("llama-cpp", "llama-server", DriverKind::LlamaCpp),
|
||||||
];
|
];
|
||||||
|
|
||||||
/// Models offered for a discovered Claude CLI. A shortcut list for the spawn
|
/// Models offered for a discovered Claude CLI. A shortcut list for the spawn
|
||||||
|
|||||||
+101
-5
@@ -14,6 +14,22 @@ use std::process::Command;
|
|||||||
|
|
||||||
use crate::config::SshConfig;
|
use crate::config::SshConfig;
|
||||||
|
|
||||||
|
/// A port on the machine a command runs on, and the port that reaches it from
|
||||||
|
/// the backend.
|
||||||
|
///
|
||||||
|
/// The second half of what a transport is (PLAN.md's SSH section): "run this"
|
||||||
|
/// plus "reach this port". Locally the two numbers are one and nothing is
|
||||||
|
/// forwarded; over ssh the connection carries an `-L` tunnel, so a model server
|
||||||
|
/// binds loopback on the far machine and is never exposed to its network.
|
||||||
|
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||||
|
pub struct Forward {
|
||||||
|
/// What the launched program should listen on, on its own machine.
|
||||||
|
pub there: u16,
|
||||||
|
/// What this machine connects to. The same number as `there` when the
|
||||||
|
/// program runs here.
|
||||||
|
pub here: u16,
|
||||||
|
}
|
||||||
|
|
||||||
/// Options forced onto every connection. `BatchMode` makes a missing key fail
|
/// Options forced onto every connection. `BatchMode` makes a missing key fail
|
||||||
/// immediately with a readable message instead of hanging on a password prompt
|
/// immediately with a readable message instead of hanging on a password prompt
|
||||||
/// nothing can answer; the keepalives turn a silently dropped link into a
|
/// nothing can answer; the keepalives turn a silently dropped link into a
|
||||||
@@ -39,6 +55,7 @@ pub fn command(
|
|||||||
program: &str,
|
program: &str,
|
||||||
args: &[String],
|
args: &[String],
|
||||||
cwd: Option<&Path>,
|
cwd: Option<&Path>,
|
||||||
|
forward: Option<Forward>,
|
||||||
) -> Command {
|
) -> Command {
|
||||||
let Some(ssh) = remote else {
|
let Some(ssh) = remote else {
|
||||||
let mut command = Command::new(program);
|
let mut command = Command::new(program);
|
||||||
@@ -55,9 +72,31 @@ pub fn command(
|
|||||||
};
|
};
|
||||||
|
|
||||||
let mut command = Command::new("ssh");
|
let mut command = Command::new("ssh");
|
||||||
// -T: no pty. This carries JSONL, and a pty would rewrite it (echo, CRLF
|
if let Some(forward) = forward {
|
||||||
// translation, ^C handling) into something the parser can't read.
|
// A forwarded process is not spoken to over stdio, and that changes how
|
||||||
|
// it is shut down. Everything else here is a CLI reading its stdin, so
|
||||||
|
// killing the ssh client ends it; a `llama-server` never reads its own,
|
||||||
|
// so the same kill left it running on the far machine with the model
|
||||||
|
// loaded -- measured 2026-09-04, an orphan per stopped session. A pty
|
||||||
|
// is what makes sshd hang the far side up. `-tt` because this client
|
||||||
|
// has no terminal to inherit one from. The cost is a log that arrives
|
||||||
|
// through a line discipline, which nothing parses.
|
||||||
|
command.arg("-tt");
|
||||||
|
// Loopback at both ends: the far side binds 127.0.0.1, so what it
|
||||||
|
// serves is reachable only through this connection.
|
||||||
|
command.args([
|
||||||
|
"-L",
|
||||||
|
&format!("127.0.0.1:{}:127.0.0.1:{}", forward.here, forward.there),
|
||||||
|
]);
|
||||||
|
// Without this a forward that cannot be set up is a warning on stderr
|
||||||
|
// and a session that runs anyway, answering nothing -- which would
|
||||||
|
// arrive as "the model never became ready".
|
||||||
|
command.args(["-o", "ExitOnForwardFailure=yes"]);
|
||||||
|
} else {
|
||||||
|
// -T: no pty. This carries JSONL, and a pty would rewrite it (echo,
|
||||||
|
// CRLF translation, ^C handling) into something the parser can't read.
|
||||||
command.arg("-T");
|
command.arg("-T");
|
||||||
|
}
|
||||||
for option in SSH_OPTIONS {
|
for option in SSH_OPTIONS {
|
||||||
command.args(["-o", option]);
|
command.args(["-o", option]);
|
||||||
}
|
}
|
||||||
@@ -181,6 +220,7 @@ mod tests {
|
|||||||
port: None,
|
port: None,
|
||||||
identity_file: None,
|
identity_file: None,
|
||||||
options: vec![],
|
options: vec![],
|
||||||
|
models_dir: None,
|
||||||
attachments_dir: None,
|
attachments_dir: None,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -192,6 +232,7 @@ mod tests {
|
|||||||
"claude",
|
"claude",
|
||||||
&args(["-p", "--verbose"]),
|
&args(["-p", "--verbose"]),
|
||||||
Some(Path::new("/tmp/x")),
|
Some(Path::new("/tmp/x")),
|
||||||
|
None,
|
||||||
);
|
);
|
||||||
assert_eq!(argv(&command), ["claude", "-p", "--verbose"]);
|
assert_eq!(argv(&command), ["claude", "-p", "--verbose"]);
|
||||||
assert_eq!(command.get_current_dir(), Some(Path::new("/tmp/x")));
|
assert_eq!(command.get_current_dir(), Some(Path::new("/tmp/x")));
|
||||||
@@ -204,6 +245,7 @@ mod tests {
|
|||||||
port: Some(2222),
|
port: Some(2222),
|
||||||
identity_file: Some("/home/me/.ssh/id_ai".into()),
|
identity_file: Some("/home/me/.ssh/id_ai".into()),
|
||||||
options: vec!["StrictHostKeyChecking=accept-new".to_string()],
|
options: vec!["StrictHostKeyChecking=accept-new".to_string()],
|
||||||
|
models_dir: None,
|
||||||
attachments_dir: None,
|
attachments_dir: None,
|
||||||
};
|
};
|
||||||
let rendered = argv(&command(
|
let rendered = argv(&command(
|
||||||
@@ -211,6 +253,7 @@ mod tests {
|
|||||||
"claude",
|
"claude",
|
||||||
&args(["-p", "--model", "haiku"]),
|
&args(["-p", "--model", "haiku"]),
|
||||||
Some(Path::new("/home/bob/work")),
|
Some(Path::new("/home/bob/work")),
|
||||||
|
None,
|
||||||
));
|
));
|
||||||
|
|
||||||
assert_eq!(rendered[0], "ssh");
|
assert_eq!(rendered[0], "ssh");
|
||||||
@@ -231,12 +274,59 @@ mod tests {
|
|||||||
#[test]
|
#[test]
|
||||||
fn a_remote_command_without_a_cwd_just_execs() {
|
fn a_remote_command_without_a_cwd_just_execs() {
|
||||||
let ssh = bare_host();
|
let ssh = bare_host();
|
||||||
let rendered = argv(&command(Some(&ssh), "claude", &args(["-p"]), None));
|
let rendered = argv(&command(Some(&ssh), "claude", &args(["-p"]), None, None));
|
||||||
assert_eq!(rendered.last().unwrap(), "exec 'claude' '-p'");
|
assert_eq!(rendered.last().unwrap(), "exec 'claude' '-p'");
|
||||||
// No -i means no IdentitiesOnly: ~/.ssh/config decides instead.
|
// No -i means no IdentitiesOnly: ~/.ssh/config decides instead.
|
||||||
assert!(!rendered.contains(&"IdentitiesOnly=yes".to_string()));
|
assert!(!rendered.contains(&"IdentitiesOnly=yes".to_string()));
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// The second half of a transport: the connection that runs the command also
|
||||||
|
/// carries the port that reaches it.
|
||||||
|
///
|
||||||
|
/// Both ends are pinned to loopback, which is what keeps a model server off
|
||||||
|
/// the far machine's network -- asserted rather than trusted, because
|
||||||
|
/// dropping the addresses is a one-word edit that still works on a machine
|
||||||
|
/// nobody else can reach.
|
||||||
|
#[test]
|
||||||
|
fn a_forwarded_port_rides_the_same_connection_as_the_command() {
|
||||||
|
let ssh = bare_host();
|
||||||
|
let rendered = argv(&command(
|
||||||
|
Some(&ssh),
|
||||||
|
"llama-server",
|
||||||
|
&args(["--port", "24242"]),
|
||||||
|
None,
|
||||||
|
Some(Forward {
|
||||||
|
there: 24242,
|
||||||
|
here: 41000,
|
||||||
|
}),
|
||||||
|
));
|
||||||
|
let forward = rendered
|
||||||
|
.iter()
|
||||||
|
.position(|arg| arg == "-L")
|
||||||
|
.expect("a forward");
|
||||||
|
assert_eq!(rendered[forward + 1], "127.0.0.1:41000:127.0.0.1:24242");
|
||||||
|
assert!(rendered.contains(&"ExitOnForwardFailure=yes".to_string()));
|
||||||
|
// The half that is easy to lose: without a pty the far process outlives
|
||||||
|
// the connection, because nothing closes a stdin it never reads.
|
||||||
|
assert!(rendered.contains(&"-tt".to_string()));
|
||||||
|
assert!(!rendered.contains(&"-T".to_string()));
|
||||||
|
// Options come before the host, or ssh reads them as part of the
|
||||||
|
// remote command.
|
||||||
|
assert!(forward < rendered.len() - 2);
|
||||||
|
assert_eq!(
|
||||||
|
rendered.last().unwrap(),
|
||||||
|
"exec 'llama-server' '--port' '24242'"
|
||||||
|
);
|
||||||
|
|
||||||
|
// Nothing forwarded is nothing added: every other session is one of
|
||||||
|
// these, and an -L on it would bind a port for no reason.
|
||||||
|
let plain = argv(&command(Some(&ssh), "claude", &args(["-p"]), None, None));
|
||||||
|
assert!(!plain.contains(&"-L".to_string()));
|
||||||
|
// And a session that *is* spoken to over stdio keeps its raw pipe.
|
||||||
|
assert!(plain.contains(&"-T".to_string()));
|
||||||
|
assert!(!plain.contains(&"-tt".to_string()));
|
||||||
|
}
|
||||||
|
|
||||||
/// The one character quoting must not swallow. A working directory typed as
|
/// The one character quoting must not swallow. A working directory typed as
|
||||||
/// `~/repos/ai-app` was arriving as the literal directory `~`, and the
|
/// `~/repos/ai-app` was arriving as the literal directory `~`, and the
|
||||||
/// remote shell reported it missing -- which reads as the path being wrong
|
/// remote shell reported it missing -- which reads as the path being wrong
|
||||||
@@ -277,7 +367,13 @@ mod tests {
|
|||||||
assert_eq!(expand_home(Path::new("/tmp/~/x")), Path::new("/tmp/~/x"));
|
assert_eq!(expand_home(Path::new("/tmp/~/x")), Path::new("/tmp/~/x"));
|
||||||
assert_eq!(expand_home(Path::new("~user/x")), Path::new("~user/x"));
|
assert_eq!(expand_home(Path::new("~user/x")), Path::new("~user/x"));
|
||||||
|
|
||||||
let local = command(None, "claude", &args(["-p"]), Some(Path::new("~/work")));
|
let local = command(
|
||||||
|
None,
|
||||||
|
"claude",
|
||||||
|
&args(["-p"]),
|
||||||
|
Some(Path::new("~/work")),
|
||||||
|
None,
|
||||||
|
);
|
||||||
assert_eq!(local.get_current_dir(), Some(home.join("work").as_path()));
|
assert_eq!(local.get_current_dir(), Some(home.join("work").as_path()));
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -301,7 +397,7 @@ mod tests {
|
|||||||
// tries to close the quote and start a new command.
|
// tries to close the quote and start a new command.
|
||||||
let ssh = bare_host();
|
let ssh = bare_host();
|
||||||
let evil = Path::new("/tmp/'; touch /tmp/pwned; '");
|
let evil = Path::new("/tmp/'; touch /tmp/pwned; '");
|
||||||
let rendered = argv(&command(Some(&ssh), "claude", &[], Some(evil)));
|
let rendered = argv(&command(Some(&ssh), "claude", &[], Some(evil), None));
|
||||||
let script = rendered.last().unwrap();
|
let script = rendered.last().unwrap();
|
||||||
assert_eq!(
|
assert_eq!(
|
||||||
script,
|
script,
|
||||||
|
|||||||
+337
-20
@@ -30,13 +30,13 @@
|
|||||||
//! already has over that machine.
|
//! already has over that machine.
|
||||||
|
|
||||||
use std::collections::HashMap;
|
use std::collections::HashMap;
|
||||||
use std::sync::Mutex;
|
use std::sync::{Arc, Mutex};
|
||||||
use std::time::{Duration, Instant};
|
use std::time::{Duration, Instant};
|
||||||
|
|
||||||
use serde::Serialize;
|
use serde::Serialize;
|
||||||
use serde_json::Value;
|
use serde_json::Value;
|
||||||
|
|
||||||
use crate::config::{DriverKind, SetupConfig};
|
use crate::config::SetupConfig;
|
||||||
use crate::session::transport::{Launch, Transport};
|
use crate::session::transport::{Launch, Transport};
|
||||||
|
|
||||||
const USAGE_URL: &str = "https://api.anthropic.com/api/oauth/usage";
|
const USAGE_URL: &str = "https://api.anthropic.com/api/oauth/usage";
|
||||||
@@ -107,10 +107,30 @@ pub struct UsageSnapshot {
|
|||||||
pub fetched_at: f64,
|
pub fetched_at: f64,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// The name of each meter, said in one place because two lists have to
|
||||||
|
/// agree on it: [`UsageSnapshot::provider`], which is what `GET /usage`
|
||||||
|
/// labels a row with, and [`crate::config::DriverKind::usage_provider`],
|
||||||
|
/// which is how a session says which of those rows is about it.
|
||||||
|
pub const CLAUDE: &str = "claude";
|
||||||
|
/// The invented one, for testing the screens that draw these -- see
|
||||||
|
/// [`Fixture`].
|
||||||
|
pub const ECHO: &str = "echo";
|
||||||
|
|
||||||
pub trait UsageProvider: Send + Sync {
|
pub trait UsageProvider: Send + Sync {
|
||||||
fn name(&self) -> &'static str;
|
fn name(&self) -> &'static str;
|
||||||
/// Blocking -- call off the async workers.
|
/// Blocking -- call off the async workers.
|
||||||
fn fetch(&self) -> UsageSnapshot;
|
fn fetch(&self) -> UsageSnapshot;
|
||||||
|
/// How long an answer from this one may be reused.
|
||||||
|
///
|
||||||
|
/// A property of the provider rather than of the cache, because what
|
||||||
|
/// sets it is what asking costs: [`ClaudeUsage`] makes a network call
|
||||||
|
/// against an endpoint that rate-limits impatient callers, and the
|
||||||
|
/// fixture below reads a mutex. Caching the fixture for three minutes
|
||||||
|
/// would mean a test setting a number and watching the old one for
|
||||||
|
/// most of that, which reads exactly like the command not working.
|
||||||
|
fn poll_interval(&self) -> Duration {
|
||||||
|
MIN_POLL_INTERVAL
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Reads the numbers behind Claude Code's `/usage` from one machine, using the
|
/// Reads the numbers behind Claude Code's `/usage` from one machine, using the
|
||||||
@@ -170,7 +190,7 @@ impl ClaudeUsage {
|
|||||||
|
|
||||||
impl UsageProvider for ClaudeUsage {
|
impl UsageProvider for ClaudeUsage {
|
||||||
fn name(&self) -> &'static str {
|
fn name(&self) -> &'static str {
|
||||||
"claude"
|
CLAUDE
|
||||||
}
|
}
|
||||||
|
|
||||||
fn fetch(&self) -> UsageSnapshot {
|
fn fetch(&self) -> UsageSnapshot {
|
||||||
@@ -278,24 +298,259 @@ fn parse_windows(body: &Value) -> Vec<UsageWindow> {
|
|||||||
.collect()
|
.collect()
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// An invented answer, so the screens that draw these can be exercised
|
||||||
|
/// without an account.
|
||||||
|
///
|
||||||
|
/// Every state the usage bar and the usage dialog can be in is otherwise
|
||||||
|
/// reachable only by spending somebody's quota or by breaking a machine:
|
||||||
|
/// a number near the top, a machine nobody has logged into, one that
|
||||||
|
/// cannot be reached, a window between blocks with no reset time. Those
|
||||||
|
/// are exactly the states worth looking at, and the ones nobody looks at
|
||||||
|
/// because arranging them costs real turns. An echo session sets this
|
||||||
|
/// with `/usage` (see `session::echo`), which is the same bargain the
|
||||||
|
/// rest of that driver makes: the fixture is invented, what is real is
|
||||||
|
/// the path it travels.
|
||||||
|
///
|
||||||
|
/// Shared by the session layer, which writes it, and [`UsageMonitor`],
|
||||||
|
/// which reads it. Empty until something sets it, and an empty fixture
|
||||||
|
/// produces no snapshot at all -- an echo session meters nothing, and
|
||||||
|
/// nothing is what the phone should draw.
|
||||||
|
#[derive(Clone, Default)]
|
||||||
|
pub struct Fixture {
|
||||||
|
said: Arc<Mutex<Option<Reported>>>,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// What a meter answered: which of the four states it is in, and whatever
|
||||||
|
/// windows go with it. Empty for every state but [`UsageState::Ok`].
|
||||||
|
type Reported = (UsageState, Vec<UsageWindow>);
|
||||||
|
|
||||||
|
/// How long the invented five-hour window has left, when nothing says.
|
||||||
|
const FIXTURE_MINUTES: i64 = 125;
|
||||||
|
|
||||||
|
impl Fixture {
|
||||||
|
pub fn new() -> Self {
|
||||||
|
Self::default()
|
||||||
|
}
|
||||||
|
|
||||||
|
fn is_set(&self) -> bool {
|
||||||
|
self.said.lock().unwrap().is_some()
|
||||||
|
}
|
||||||
|
|
||||||
|
fn read(&self) -> Option<Reported> {
|
||||||
|
self.said.lock().unwrap().clone()
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Acts on the words typed after `/usage`, and says what it did.
|
||||||
|
///
|
||||||
|
/// The vocabulary lives here rather than in the echo driver because
|
||||||
|
/// these are this module's states: a driver spelling them out would
|
||||||
|
/// be a second place that has to learn about a fifth one.
|
||||||
|
pub fn command(&self, words: &str) -> String {
|
||||||
|
let mut words = words.split_whitespace();
|
||||||
|
let Some(first) = words.next() else {
|
||||||
|
return match self.read() {
|
||||||
|
Some((state, windows)) => format!("usage fixture: {}", describe(&state, &windows)),
|
||||||
|
None => "usage fixture: unset, so this session meters nothing. \
|
||||||
|
`/usage 42` puts up a five-hour window at 42%."
|
||||||
|
.to_string(),
|
||||||
|
};
|
||||||
|
};
|
||||||
|
let rest: Vec<&str> = words.collect();
|
||||||
|
let detail = || {
|
||||||
|
if rest.is_empty() {
|
||||||
|
"set by /usage".to_string()
|
||||||
|
} else {
|
||||||
|
rest.join(" ")
|
||||||
|
}
|
||||||
|
};
|
||||||
|
let (state, windows) = match first {
|
||||||
|
"off" | "none" | "clear" => {
|
||||||
|
*self.said.lock().unwrap() = None;
|
||||||
|
return "usage fixture cleared: this session meters nothing again".to_string();
|
||||||
|
}
|
||||||
|
"notloggedin" | "logged-out" => (UsageState::NotLoggedIn, Vec::new()),
|
||||||
|
"unreachable" => (UsageState::Unreachable { detail: detail() }, Vec::new()),
|
||||||
|
"failed" => (UsageState::Failed { detail: detail() }, Vec::new()),
|
||||||
|
percent => match percent.parse::<f64>() {
|
||||||
|
Ok(percent) => (
|
||||||
|
UsageState::Ok,
|
||||||
|
fixture_windows(percent.clamp(0.0, 100.0), rest.first().copied()),
|
||||||
|
),
|
||||||
|
Err(_) => {
|
||||||
|
return format!(
|
||||||
|
"\"{percent}\" is not one of this fixture's answers. Say a percentage \
|
||||||
|
(`/usage 42`, optionally with `90` minutes left, `never` for a window \
|
||||||
|
between blocks, or `unreadable` for a reset time that cannot be read), \
|
||||||
|
or one of `notloggedin`, `unreachable`, `failed`, `off`."
|
||||||
|
);
|
||||||
|
}
|
||||||
|
},
|
||||||
|
};
|
||||||
|
let said = describe(&state, &windows);
|
||||||
|
*self.said.lock().unwrap() = Some((state, windows));
|
||||||
|
format!("usage fixture set: {said}")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The three windows Claude reports today, invented around one number.
|
||||||
|
///
|
||||||
|
/// Three rather than one because the bar under a session header reads the
|
||||||
|
/// five-hour window and the dialog behind the button draws all of them,
|
||||||
|
/// and a fixture with one window leaves half the screen untested. The
|
||||||
|
/// weekly ones are derived from the same figure so that the worst of them
|
||||||
|
/// -- which is what colours the button -- is still the one asked for.
|
||||||
|
fn fixture_windows(percent: f64, reset: Option<&str>) -> Vec<UsageWindow> {
|
||||||
|
let resets_at = match reset {
|
||||||
|
// The state a real response is in between blocks: there is no
|
||||||
|
// window running, so there is nothing to reset. It is not a
|
||||||
|
// missing value, and the phone words it differently.
|
||||||
|
Some("never") | Some("none") => None,
|
||||||
|
// A timestamp that arrives and cannot be read, which is the one
|
||||||
|
// case that really is "we could not find out".
|
||||||
|
Some("unreadable") | Some("bad") => Some("whenever it feels like it".to_string()),
|
||||||
|
other => Some(reset_in(
|
||||||
|
other
|
||||||
|
.and_then(|word| word.parse().ok())
|
||||||
|
.unwrap_or(FIXTURE_MINUTES),
|
||||||
|
)),
|
||||||
|
};
|
||||||
|
vec![
|
||||||
|
UsageWindow {
|
||||||
|
kind: "session".to_string(),
|
||||||
|
label: "5-hour window".to_string(),
|
||||||
|
percent,
|
||||||
|
resets_at: resets_at.clone(),
|
||||||
|
active: true,
|
||||||
|
},
|
||||||
|
UsageWindow {
|
||||||
|
kind: "weekly_all".to_string(),
|
||||||
|
label: "Weekly (all models)".to_string(),
|
||||||
|
percent: percent / 2.0,
|
||||||
|
resets_at: resets_at.as_ref().map(|_| reset_in(FIXTURE_MINUTES * 40)),
|
||||||
|
active: false,
|
||||||
|
},
|
||||||
|
UsageWindow {
|
||||||
|
kind: "weekly_scoped".to_string(),
|
||||||
|
label: "Weekly (Echo)".to_string(),
|
||||||
|
percent: percent / 4.0,
|
||||||
|
resets_at: resets_at.as_ref().map(|_| reset_in(FIXTURE_MINUTES * 40)),
|
||||||
|
active: false,
|
||||||
|
},
|
||||||
|
]
|
||||||
|
}
|
||||||
|
|
||||||
|
/// `minutes` from now, in the format the real endpoint sends.
|
||||||
|
fn reset_in(minutes: i64) -> String {
|
||||||
|
let at = time::OffsetDateTime::now_utc() + time::Duration::minutes(minutes);
|
||||||
|
at.format(&time::format_description::well_known::Rfc3339)
|
||||||
|
// Formatting a timestamp cannot fail for any input this builds;
|
||||||
|
// saying so beats a fixture that silently has no reset time.
|
||||||
|
.unwrap_or_else(|_| "unformattable".to_string())
|
||||||
|
}
|
||||||
|
|
||||||
|
/// One line naming what a fixture is currently claiming, for the reply
|
||||||
|
/// the echo session writes back.
|
||||||
|
fn describe(state: &UsageState, windows: &[UsageWindow]) -> String {
|
||||||
|
match state {
|
||||||
|
UsageState::Ok => match windows.first() {
|
||||||
|
Some(window) => format!(
|
||||||
|
"{}% of the five-hour window, {}",
|
||||||
|
window.percent,
|
||||||
|
match &window.resets_at {
|
||||||
|
Some(at) => format!("resetting at {at}"),
|
||||||
|
None => "with no reset time (the between-blocks state)".to_string(),
|
||||||
|
}
|
||||||
|
),
|
||||||
|
None => "no windows at all".to_string(),
|
||||||
|
},
|
||||||
|
UsageState::NotLoggedIn => "nobody is logged in on this machine".to_string(),
|
||||||
|
UsageState::Unreachable { detail } => format!("machine unreachable ({detail})"),
|
||||||
|
UsageState::Failed { detail } => format!("the meter failed ({detail})"),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The fixture, as a provider, so it travels the same route and the same
|
||||||
|
/// cache as a real meter rather than being spliced in at the screen.
|
||||||
|
struct EchoUsage {
|
||||||
|
setup: String,
|
||||||
|
setup_name: String,
|
||||||
|
fixture: Fixture,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl UsageProvider for EchoUsage {
|
||||||
|
fn name(&self) -> &'static str {
|
||||||
|
ECHO
|
||||||
|
}
|
||||||
|
|
||||||
|
fn fetch(&self) -> UsageSnapshot {
|
||||||
|
let (state, windows) = self
|
||||||
|
.fixture
|
||||||
|
.read()
|
||||||
|
// Only ever built for a fixture that is set; a race with
|
||||||
|
// `/usage off` between the two reads lands here, and "the
|
||||||
|
// machine could not be asked" is the honest word for it.
|
||||||
|
.unwrap_or((
|
||||||
|
UsageState::Unreachable {
|
||||||
|
detail: "the usage fixture was cleared".to_string(),
|
||||||
|
},
|
||||||
|
Vec::new(),
|
||||||
|
));
|
||||||
|
UsageSnapshot {
|
||||||
|
provider: self.name().to_string(),
|
||||||
|
setup: self.setup.clone(),
|
||||||
|
setup_name: self.setup_name.clone(),
|
||||||
|
state,
|
||||||
|
windows,
|
||||||
|
fetched_at: crate::session::now(),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Read from memory, and set by somebody who is about to look at the
|
||||||
|
/// screen it changes.
|
||||||
|
fn poll_interval(&self) -> Duration {
|
||||||
|
Duration::ZERO
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
/// Which paid services a machine can be asked about.
|
/// Which paid services a machine can be asked about.
|
||||||
///
|
///
|
||||||
/// Derived from what the setup says it can run, so a machine with no Claude
|
/// Derived from what the setup says it can run, so a machine with no Claude
|
||||||
/// provider is not asked about Claude limits -- it has none, and a row saying
|
/// provider is not asked about Claude limits -- it has none, and a row saying
|
||||||
/// so would be a fact about nothing. A second service later adds a branch here
|
/// so would be a fact about nothing.
|
||||||
/// and an impl beside [`ClaudeUsage`], not a screen.
|
///
|
||||||
fn providers_for(setup: &SetupConfig) -> Vec<Box<dyn UsageProvider>> {
|
/// Which meter a provider has is [`DriverKind::usage_provider`]'s answer rather
|
||||||
|
/// than a second match on kinds here, because the phone pairs a session with
|
||||||
|
/// one of these rows by that same name: two lists that disagreed would leave a
|
||||||
|
/// session looking for a snapshot nothing produces. A second service later is a
|
||||||
|
/// name there and an impl beside [`ClaudeUsage`], not a screen.
|
||||||
|
fn providers_for(setup: &SetupConfig, fixture: &Fixture) -> Vec<Box<dyn UsageProvider>> {
|
||||||
let mut found: Vec<Box<dyn UsageProvider>> = Vec::new();
|
let mut found: Vec<Box<dyn UsageProvider>> = Vec::new();
|
||||||
if setup
|
for provider in &setup.providers {
|
||||||
.providers
|
let Some(name) = provider.kind.usage_provider() else {
|
||||||
.iter()
|
continue;
|
||||||
.any(|provider| provider.kind == DriverKind::ClaudeCli)
|
};
|
||||||
{
|
// A machine offering two Claude providers has one account, not
|
||||||
found.push(Box::new(ClaudeUsage {
|
// two: the meter belongs to the machine and the service, which is
|
||||||
|
// exactly what the cache is keyed by.
|
||||||
|
if found.iter().any(|already| already.name() == name) {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
match name {
|
||||||
|
CLAUDE => found.push(Box::new(ClaudeUsage {
|
||||||
setup: setup.id.clone(),
|
setup: setup.id.clone(),
|
||||||
setup_name: setup.name.clone(),
|
setup_name: setup.name.clone(),
|
||||||
transport: Transport::for_setup(setup),
|
transport: Transport::for_setup(setup),
|
||||||
}));
|
})),
|
||||||
|
// Nothing at all until a test has asked for something: an
|
||||||
|
// echo session costs nothing, so the honest answer is no row
|
||||||
|
// rather than a row saying zero.
|
||||||
|
ECHO if fixture.is_set() => found.push(Box::new(EchoUsage {
|
||||||
|
setup: setup.id.clone(),
|
||||||
|
setup_name: setup.name.clone(),
|
||||||
|
fixture: fixture.clone(),
|
||||||
|
})),
|
||||||
|
_ => {}
|
||||||
|
}
|
||||||
}
|
}
|
||||||
found
|
found
|
||||||
}
|
}
|
||||||
@@ -313,11 +568,18 @@ type Cached = HashMap<(String, &'static str), (Instant, UsageSnapshot)>;
|
|||||||
/// machine per service per [`MIN_POLL_INTERVAL`], however often the phone asks.
|
/// machine per service per [`MIN_POLL_INTERVAL`], however often the phone asks.
|
||||||
pub struct UsageMonitor {
|
pub struct UsageMonitor {
|
||||||
cache: Mutex<Cached>,
|
cache: Mutex<Cached>,
|
||||||
|
/// The invented meter an echo session can put up; empty unless one
|
||||||
|
/// has. Shared with the session layer, which is where the command
|
||||||
|
/// that sets it is typed -- see [`Fixture`].
|
||||||
|
fixture: Fixture,
|
||||||
}
|
}
|
||||||
|
|
||||||
impl UsageMonitor {
|
impl UsageMonitor {
|
||||||
pub fn new() -> Self {
|
pub fn new(fixture: Fixture) -> Self {
|
||||||
Self::default()
|
Self {
|
||||||
|
cache: Mutex::new(Cached::new()),
|
||||||
|
fixture,
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/// One snapshot per machine that offers a paid service, in the order the
|
/// One snapshot per machine that offers a paid service, in the order the
|
||||||
@@ -329,10 +591,10 @@ impl UsageMonitor {
|
|||||||
pub fn snapshots(&self, setups: &[SetupConfig]) -> Vec<UsageSnapshot> {
|
pub fn snapshots(&self, setups: &[SetupConfig]) -> Vec<UsageSnapshot> {
|
||||||
let mut fresh = Vec::new();
|
let mut fresh = Vec::new();
|
||||||
for setup in setups {
|
for setup in setups {
|
||||||
for provider in providers_for(setup) {
|
for provider in providers_for(setup, &self.fixture) {
|
||||||
let key = (setup.id.clone(), provider.name());
|
let key = (setup.id.clone(), provider.name());
|
||||||
if let Some((fetched, snapshot)) = self.cache.lock().unwrap().get(&key)
|
if let Some((fetched, snapshot)) = self.cache.lock().unwrap().get(&key)
|
||||||
&& fetched.elapsed() < MIN_POLL_INTERVAL
|
&& fetched.elapsed() < provider.poll_interval()
|
||||||
{
|
{
|
||||||
// Cached numbers, but the machine's *name* is read fresh: a
|
// Cached numbers, but the machine's *name* is read fresh: a
|
||||||
// rename should show immediately rather than waiting out a
|
// rename should show immediately rather than waiting out a
|
||||||
@@ -367,6 +629,7 @@ impl UsageMonitor {
|
|||||||
#[cfg(test)]
|
#[cfg(test)]
|
||||||
mod tests {
|
mod tests {
|
||||||
use super::*;
|
use super::*;
|
||||||
|
use crate::config::DriverKind;
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn parses_the_limits_array_defensively() {
|
fn parses_the_limits_array_defensively() {
|
||||||
@@ -404,6 +667,7 @@ mod tests {
|
|||||||
port: None,
|
port: None,
|
||||||
identity_file: None,
|
identity_file: None,
|
||||||
options: vec!["ConnectTimeout=1".to_string()],
|
options: vec!["ConnectTimeout=1".to_string()],
|
||||||
|
models_dir: None,
|
||||||
attachments_dir: None,
|
attachments_dir: None,
|
||||||
}),
|
}),
|
||||||
providers: vec![crate::config::ProviderConfig {
|
providers: vec![crate::config::ProviderConfig {
|
||||||
@@ -475,9 +739,62 @@ mod tests {
|
|||||||
models: vec![],
|
models: vec![],
|
||||||
}];
|
}];
|
||||||
// A machine with no Claude on it has no Claude limits, and a row
|
// A machine with no Claude on it has no Claude limits, and a row
|
||||||
// reporting on it would be a fact about nothing.
|
// reporting on it would be a fact about nothing. Echo included:
|
||||||
assert!(providers_for(&echo_only).is_empty());
|
// an echo session spends nothing, so until a fixture says
|
||||||
assert_eq!(providers_for(&unreachable_setup()).len(), 1);
|
// otherwise there is no meter to report.
|
||||||
|
let unset = Fixture::new();
|
||||||
|
assert!(providers_for(&echo_only, &unset).is_empty());
|
||||||
|
assert_eq!(providers_for(&unreachable_setup(), &unset).len(), 1);
|
||||||
|
|
||||||
|
// And with one set, that machine has exactly the invented meter
|
||||||
|
// -- under the name the session's `usageProvider` will name.
|
||||||
|
let fixture = Fixture::new();
|
||||||
|
fixture.command("42");
|
||||||
|
let found = providers_for(&echo_only, &fixture);
|
||||||
|
assert_eq!(found.len(), 1);
|
||||||
|
assert_eq!(found[0].name(), ECHO);
|
||||||
|
assert_eq!(DriverKind::Echo.usage_provider(), Some(ECHO));
|
||||||
|
assert_eq!(DriverKind::ClaudeCli.usage_provider(), Some(CLAUDE));
|
||||||
|
// A local model costs nothing to run, so it meters nothing.
|
||||||
|
assert_eq!(DriverKind::LlamaCpp.usage_provider(), None);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The states the fixture exists to make reachable, and the one thing
|
||||||
|
/// it must not do: invent a reset time for a window that has none.
|
||||||
|
#[test]
|
||||||
|
fn the_fixture_says_each_state_the_screens_have_to_draw() {
|
||||||
|
let fixture = Fixture::new();
|
||||||
|
assert!(fixture.read().is_none(), "unset until somebody sets it");
|
||||||
|
|
||||||
|
fixture.command("42 90");
|
||||||
|
let (state, windows) = fixture.read().expect("set");
|
||||||
|
assert_eq!(state, UsageState::Ok);
|
||||||
|
assert_eq!(windows[0].kind, "session");
|
||||||
|
assert_eq!(windows[0].percent, 42.0);
|
||||||
|
assert!(windows[0].resets_at.is_some());
|
||||||
|
|
||||||
|
// Between blocks: no reset time, which the phone words as the
|
||||||
|
// window not running rather than as a time it could not read.
|
||||||
|
fixture.command("42 never");
|
||||||
|
assert_eq!(fixture.read().expect("set").1[0].resets_at, None);
|
||||||
|
|
||||||
|
fixture.command("unreachable no route to host");
|
||||||
|
assert!(matches!(
|
||||||
|
fixture.read().expect("set").0,
|
||||||
|
UsageState::Unreachable { detail } if detail == "no route to host",
|
||||||
|
));
|
||||||
|
|
||||||
|
fixture.command("off");
|
||||||
|
assert!(fixture.read().is_none());
|
||||||
|
|
||||||
|
// A word it does not know changes nothing and says what it takes.
|
||||||
|
fixture.command("42");
|
||||||
|
let refused = fixture.command("sideways");
|
||||||
|
assert!(
|
||||||
|
refused.contains("not one of this fixture's answers"),
|
||||||
|
"{refused}"
|
||||||
|
);
|
||||||
|
assert_eq!(fixture.read().expect("still set").1[0].percent, 42.0);
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
|
|||||||
Reference in new issue
Block a user