diff --git a/AGENTS.md b/AGENTS.md index 670359f..983c67f 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -30,6 +30,12 @@ can't come apart). Read dev-updater's `README.md` and `AGENTS.md` for the conventions before diverging from them; module-by-module intent for this repo is in PLAN.md's "Backend layout" section. +- `server/src/models.rs` — downloaded GGUF models and the HuggingFace + browsing behind them. Downloads are keyed by the model rather than by + who asked, so any device can watch one; they resume through HTTP Range, + refuse to resume onto a partial from a different revision, and are + checked against HuggingFace's published sha256 before the file gets its + real name. - `server/` — Rust backend (`ai-server`). `main.rs` bootstraps (TLS, the auth layer, token/QR enrollment, wg0 binding), `routes.rs` has the HTTP table in its module doc comment, `auth.rs` the bearer-token middleware, @@ -61,9 +67,31 @@ crash recovery, images both ways), and the usage screen. **Phase 5 (SSH) is written and now exercised** (2026-08-28). A session names a host, `session::transport` turns that into an `ssh host …` -invocation, and the driver never learns which it got. Phase 4 (llama.cpp) -is being built now, no longer deferred. What is left is -real-phone/WireGuard bring-up, which is operational rather than code. +invocation, and the driver never learns which it got. + +**Phase 4 (llama.cpp) works on the server side** (2026-08-28). Models are +browsed and downloaded from HuggingFace (`models.rs`, resumable and +verified), and `session::llama` runs one through `llama-server`, talking +to its OpenAI-compatible streaming endpoint. Two things about it are +deliberate and easy to undo by accident: the conversation is rebuilt from +the **transcript** rather than kept in the driver, because driver memory +is invisible to a second device; and a llama session is refused on an ssh +host, because the model is reached over HTTP and forwarding that port is +not built. No app screen yet — models are driven through the routes. + +What is left is the models UI, and real-phone/WireGuard bring-up, which is +operational rather than code. + +**Testing llama.cpp here:** the prebuilt CPU build lives outside the repo +at `~/.local/opt/llama.cpp` (the 15 MB `ubuntu-x64` release asset — no +compiling, and it runs fine on Arch). It needs its own directory on +`LD_LIBRARY_PATH`, so start the server as +`LD_LIBRARY_PATH=~/.local/opt/llama.cpp ai-server …` and point a provider's +`command` at `~/.local/opt/llama.cpp/llama-server`. A 0.6B Q8_0 answers at +usable speed on this VM's 8 cores. **Do not test with a 2-bit quant**: the +IQ2_XXS of that model produces fluent nonsense, which reads exactly like a +broken driver — `llama-cli` produces the same from the file directly, which +is how to tell the two apart in a hurry. **How to test SSH here, since there is no second machine:** ssh this VM to itself. Generate a throwaway key, append the public half to diff --git a/PLAN.md b/PLAN.md index a42b87a..0170ed8 100644 --- a/PLAN.md +++ b/PLAN.md @@ -222,6 +222,28 @@ turn. Claude's dialect: a `user` message on stdin mid-stream; pi's: `steer`. endpoint changes. - pi's session JSONL gives resume-after-restart, same as Claude's. +### Models (built 2026-08-28) + +Bryan asked for listing and downloading models from HuggingFace and running +them with different parameters, which makes model management part of the +feature rather than something done by hand beforehand. + +- **A download belongs to the model, not to the request.** Keyed by + `owner/repo/file.gguf` and owned by the server, so a second device can + watch one it did not start, and so an hour-long fetch survives a phone + locking its screen. Every run has an id and its outcome outlives it, + because "not downloading" otherwise means finished, never started, or + someone else's run ended while you were away. +- **Progress is measured.** `total` is Content-Length, or Content-Range's + last field on a resumed request, and absent when the server says + nothing — never an estimate. +- **Resume is guarded by identity, not by hope.** A partial carries the + ETag it was written against; a mismatch discards it. `If-Range` would be + the tidy mechanism but HuggingFace's CDN ignores it (probed + 2026-08-28). The published sha256 is checked before the file is renamed. +- Parameters reach a driver as an untyped `params` map on the session, so + the shared schema does not grow llama.cpp's vocabulary. + ### llama-server management `config.ron` lists **models** (name → GGUF path or llama-server args, per