irisandClaude Opus 5
127b25e60a
The rate-limit bar answered a question about an account, and picked the
answer by machine. One machine runs echo, the Claude CLI and a local
model side by side, so every echo session on it drew the CLI's five-hour
window: a quota that session cannot spend and could never run down. A
session now names its meter (`usageProvider`, from
`DriverKind::usage_provider`, which `usage::providers_for` reads too so
the two lists cannot disagree), and the phone matches on machine *and*
provider. Nothing meters echo or llama, and nothing at all is drawn --
including while the first fetch is out, since "checking" under a session
that turns out to meter nothing is a row the screen then withdraws.
Echo gets a meter it can be *told* about instead: `/usage 42`,
`/usage 95 20`, `/usage 42 never`, `/usage notloggedin`,
`/usage unreachable`, `/usage failed`, `/usage off`. Those states cost
real quota to arrange, which is why none of them had been looked at.
And llama.cpp runs wherever a setup says, which was the last of phase 5.
`Transport::reserve_port` is the second half of what a transport is --
"run this" plus "reach this port" -- returning the port the server binds
there and the port that reaches it here, and `Launch::reaching` puts the
`-L` tunnel on the connection that already carries the command. Three
things that came out of building it:
- A forwarded launch gets a pty and every other one keeps `-T`. Killing
the ssh client ends a CLI by closing the stdin it reads; llama-server
never reads its stdin, so the same kill left it running on the far
machine with the model loaded -- one orphan per stopped session.
- The model is looked for on the machine that will serve it, at that
machine's own models directory, so `GET /setups/{id}/models` is what
the spawn screen offers rather than the backend's own downloads.
- The readiness poll watches the process, not only the port: a model
that will not load exits in a second and would otherwise have been
reported as "gave up after 300s". The failure carries the log's tail.
Exercised end to end against this VM over ssh to itself: spawn, load,
answer, outlive a backend restart, be adopted, answer again, and stop --
with both the ssh client and the far llama-server gone afterwards. The
local path, the Claude bar and the spawn screen checked on the emulator.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
61 lines
2.7 KiB
TOML
61 lines
2.7 KiB
TOML
[package]
|
|
name = "ai-server"
|
|
version = "0.1.0"
|
|
edition = "2024"
|
|
|
|
[[bin]]
|
|
name = "ai-server"
|
|
path = "src/main.rs"
|
|
|
|
[dependencies]
|
|
# The link both this and dev-updater need in order to be reached from a
|
|
# phone: wg binding, the pinned CA, QR enrollment, owner-only files, and
|
|
# the RON house rules. Extracted from the two copies that had drifted --
|
|
# see that repo's README for the evidence and the bug the extraction found.
|
|
wg-app-link = { path = "../wg-app-link/server" }
|
|
axum = { version = "0.8", features = ["json", "multipart"] }
|
|
axum-server = { version = "0.8", features = ["tls-rustls"] }
|
|
tokio = { version = "1", features = ["rt-multi-thread", "macros", "net", "sync", "time", "process", "io-util", "signal"] }
|
|
# `sync` for BroadcastStream: the notifications route turns the manager's
|
|
# broadcast channel straight into an SSE body, which is the one place here a
|
|
# broadcast receiver has to be a Stream rather than something to poll.
|
|
tokio-stream = { version = "0.1", features = ["sync"] }
|
|
tracing = "0.1"
|
|
tracing-subscriber = { version = "0.3", features = ["env-filter"] }
|
|
serde = { version = "1", features = ["derive"] }
|
|
serde_json = "1"
|
|
# The config file's format. Not JSON, because this file is written and read
|
|
# by hand and RON says a sum type as syntax. The two house rules both
|
|
# projects write it under live in wg-app-link; this is here for the error
|
|
# types the schema's own signatures name.
|
|
ron = "0.12.2"
|
|
clap = { version = "4", features = ["derive"] }
|
|
anyhow = "1"
|
|
thiserror = "2"
|
|
# Verifying a downloaded model against HuggingFace's published digest.
|
|
sha2 = "0.11"
|
|
# Naming a session directory, and an attachment inside one.
|
|
rand = "0.10"
|
|
# Decoding the images a phone attaches, and encoding them for a driver.
|
|
base64 = "0.23"
|
|
# Outbound HTTPS for the usage endpoint. A small blocking client fits an
|
|
# every-few-minutes poll better than pulling in reqwest's tower stack;
|
|
# rustls-backed like the rest of the TLS here.
|
|
ureq = { version = "3", features = ["json"] }
|
|
# Direct dependency only to pick the process-level CryptoProvider in main:
|
|
# ureq pulls rustls-with-ring, axum-server rustls-with-aws-lc-rs, and with
|
|
# both in the graph rustls refuses to auto-select one.
|
|
rustls = "0.23"
|
|
libc = "0.2.189"
|
|
# One ISO-8601 timestamp: the reset time on the invented rate-limit window
|
|
# an echo session's `/usage` puts up. Already in the tree behind the
|
|
# certificate machinery, so this is a direct name for what is compiled
|
|
# anyway rather than a new crate -- and the alternative was hand-rolling a
|
|
# civil-from-days conversion to print one line.
|
|
time = { version = "0.3", features = ["formatting"] }
|
|
|
|
[dev-dependencies]
|
|
tempfile = "3"
|
|
# ServiceExt::oneshot, to drive the auth middleware without a socket.
|
|
tower = { version = "0.5", features = ["util"] }
|