Write down what the llama.cpp work actually does

Phase 4 is no longer deferred and phase 5 is exercised, so the status
section says so. The two decisions worth not undoing by accident get
named: the conversation lives in the transcript rather than the driver,
and a llama session is refused on an ssh host rather than half-working.

Also the local testing recipe, including the trap that cost me twenty
minutes -- a 2-bit quant produces fluent nonsense that reads exactly like
a broken driver, and llama-cli on the same file is how to tell the two
apart.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017xn8nHw1tw1R6PtiY1eEtw
This commit is contained in:
irisandClaude Opus 5 committed 2026-08-28 05:23:47 -04:00
1 parent 3deeffd1e7
commit 3be25c2f64
2 files changed
+53 -3

No files matched your search

+22
View File
@@ -222,6 +222,28 @@ turn. Claude's dialect: a `user` message on stdin mid-stream; pi's: `steer`.
endpoint changes.
- pi's session JSONL gives resume-after-restart, same as Claude's.
### Models (built 2026-08-28)
Bryan asked for listing and downloading models from HuggingFace and running
them with different parameters, which makes model management part of the
feature rather than something done by hand beforehand.
- **A download belongs to the model, not to the request.** Keyed by
`owner/repo/file.gguf` and owned by the server, so a second device can
watch one it did not start, and so an hour-long fetch survives a phone
locking its screen. Every run has an id and its outcome outlives it,
because "not downloading" otherwise means finished, never started, or
someone else's run ended while you were away.
- **Progress is measured.** `total` is Content-Length, or Content-Range's
last field on a resumed request, and absent when the server says
nothing — never an estimate.
- **Resume is guarded by identity, not by hope.** A partial carries the
ETag it was written against; a mismatch discards it. `If-Range` would be
the tidy mechanism but HuggingFace's CDN ignores it (probed
2026-08-28). The published sha256 is checked before the file is renamed.
- Parameters reach a driver as an untyped `params` map on the session, so
the shared schema does not grow llama.cpp's vocabulary.
### llama-server management
`config.ron` lists **models** (name → GGUF path or llama-server args, per