Write down what the llama.cpp work actually does
Phase 4 is no longer deferred and phase 5 is exercised, so the status section says so. The two decisions worth not undoing by accident get named: the conversation lives in the transcript rather than the driver, and a llama session is refused on an ssh host rather than half-working. Also the local testing recipe, including the trap that cost me twenty minutes -- a 2-bit quant produces fluent nonsense that reads exactly like a broken driver, and llama-cli on the same file is how to tell the two apart. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017xn8nHw1tw1R6PtiY1eEtw
This commit is contained in:
1 parent
3deeffd1e7
commit
3be25c2f64
2 files changed
+53
-3
No files matched your search
@@ -222,6 +222,28 @@ turn. Claude's dialect: a `user` message on stdin mid-stream; pi's: `steer`.
|
||||
endpoint changes.
|
||||
- pi's session JSONL gives resume-after-restart, same as Claude's.
|
||||
|
||||
### Models (built 2026-08-28)
|
||||
|
||||
Bryan asked for listing and downloading models from HuggingFace and running
|
||||
them with different parameters, which makes model management part of the
|
||||
feature rather than something done by hand beforehand.
|
||||
|
||||
- **A download belongs to the model, not to the request.** Keyed by
|
||||
`owner/repo/file.gguf` and owned by the server, so a second device can
|
||||
watch one it did not start, and so an hour-long fetch survives a phone
|
||||
locking its screen. Every run has an id and its outcome outlives it,
|
||||
because "not downloading" otherwise means finished, never started, or
|
||||
someone else's run ended while you were away.
|
||||
- **Progress is measured.** `total` is Content-Length, or Content-Range's
|
||||
last field on a resumed request, and absent when the server says
|
||||
nothing — never an estimate.
|
||||
- **Resume is guarded by identity, not by hope.** A partial carries the
|
||||
ETag it was written against; a mismatch discards it. `If-Range` would be
|
||||
the tidy mechanism but HuggingFace's CDN ignores it (probed
|
||||
2026-08-28). The published sha256 is checked before the file is renamed.
|
||||
- Parameters reach a driver as an untyped `params` map on the session, so
|
||||
the shared schema does not grow llama.cpp's vocabulary.
|
||||
|
||||
### llama-server management
|
||||
|
||||
`config.ron` lists **models** (name → GGUF path or llama-server args, per
|
||||
|
||||
Reference in new issue
Block a user