Download a model onto the machine that will serve it

The Models tab was about this backend's own disk, which is the wrong disk
for every session that runs anywhere else: llama.cpp reads the file where
it runs. So the models of a machine live under that machine's llama.cpp
provider now, beside the settings deciding how each is loaded, and the
download that produces one happens there.

A download is a detached `curl` on that machine, started by a script this
server writes and never spoken to again. Its state is a file beside the
partial, so nothing about it is held here: it survives the app closing,
this backend restarting and a second device watching, and the progress is
`wc -c` of the partial against the size HuggingFace published rather than
anything remembered. A run whose process is gone is reported failed, since
`kill -0` is asked at each listing, and there is no "finished" state -- a
download that finished is a model, in the list beside the ones still
going. Resuming is guarded by the published sha256, which is also checked
before the file takes its real name.

Two other things the same screens wanted:

A provider is drawn as a card rather than as a line of text, bordered
against the machine card it sits in -- the tint it had was one step along
the surface ladder and rendered as one flat block -- with room to tap and
no chevron.

Nothing in a raw block wraps any more; the block scrolls sideways
instead, one offset for all its lines, so a diff or a column-aligned test
run still reads as one.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
iris-aiandClaude Opus 5 committed 2026-09-19 18:55:51 -04:00
1 parent 8c323fc7a9
commit 81c30dcda1
14 files changed
+1224 -1073

No files matched your search

+50 -22
View File
@@ -35,7 +35,7 @@ backend (Rust/Axum, desktop)
│ each driver's process is spawned through a Transport,
│ locally or as `ssh host …`, decided by the configured machine
├─ usage.rs (provider usage meters, per machine)
├─ models.rs (HuggingFace browsing and GGUF downloads)
├─ models.rs (HuggingFace browsing; a machine's GGUFs and downloads)
├─ files.rs (the file explorer's half of the backend)
└─ config.ron + per-session transcript files
```
@@ -92,7 +92,9 @@ axum 0.8, axum-server + rustls, tokio, serde, clap, tracing. Rust edition
- `machines.rs` — machines and provider discovery.
- `files.rs` — the file explorer (`EXPLORER.md`).
- `usage.rs` — provider usage polling, per machine.
- `models.rs` — HuggingFace browsing and GGUF downloads.
- `models.rs` — HuggingFace browsing, and the GGUFs on a machine with the
downloads putting them there. Addressed by transport and directory, never
by this process's own disk.
- `media.rs` — the image media-type/extension table, shared by the four
places that must agree: storing an upload, serving it back, handing one to
a driver, and saving one a tool produced.
@@ -424,11 +426,12 @@ deliberate and easy to undo by accident:
key on that machine — one round trip answering "at /abs/path" or "missing",
so a model that is not there is refused at the spawn rather than becoming a
server that never becomes ready. The spawn screen offers
`GET /machines/{id}/models`, that machine's list, rather than `GET /models`,
which is this backend's downloads. Downloading *to* another machine is
deliberately not built: a multi-gigabyte transfer with no progress
anywhere, and the file gets there however anything else on that machine
did.
`GET /machines/{id}/providers/{p}/models`, that machine's list, rather than
anything about this backend's own disk. **Downloading to that machine was
deliberately not built until 2026-09-19** -- the objection was a
multi-gigabyte transfer with no progress anywhere -- and what changed is
that the transfer happens *on* that machine and reports progress: see
"Models" below.
- **The readiness poll watches the process, not only the port.** A model that
will not load, a port already taken, a flag an older build does not know:
all exit within a second and none will ever answer `/health`, so waiting
@@ -506,21 +509,35 @@ deliberate and easy to undo by accident:
of the file — on the machine that will serve it, in the round trip the spawn
was already making — and `params["speculative"] = "off"` is the way out.
### Models (2026-08-28)
### Models (2026-08-28, rebuilt per machine 2026-09-19)
- **A download belongs to the model, not to the request.** Keyed by
`owner/repo/file.gguf` and owned by the server, so a second device can
watch one it did not start and an hour-long fetch survives a locked screen.
Every run has an id and its outcome outlives it, because "not downloading"
otherwise means finished, never started, or someone else's run ended while
you were away.
- **Progress is measured**, never estimated: `total` is Content-Length, or
Content-Range's last field on a resume, and absent when the server says
nothing.
- **Resume is guarded by identity, not by hope.** A partial carries the ETag
it was written against and a mismatch discards it. `If-Range` would be the
tidy mechanism but HuggingFace's CDN ignores it (probed 2026-08-28). The
published sha256 is checked before the file is renamed.
`owner/repo/file.gguf`, so a second device can watch one it did not start
and an hour-long fetch survives a locked screen. Asking for one already
going joins it rather than starting a second writer.
- **A download runs on the machine that will serve the file** (2026-09-19),
because that is where `llama-server` has to read it from -- so it is a
detached `curl` started by a script this backend writes over the same
transport everything else about a machine goes through. What this replaced
was a fetch onto the backend's own disk, offered under a Models tab, which
could not put a model on any other machine at all.
- **Its state is a file beside the partial**, `x.gguf.download`, holding the
worker's pid, the sha it is downloading against, the published size and a
state word. Nothing about a run is held in this process, which is what
makes it survive a backend restart, and what lets the answer be read off
the disk that has the file rather than remembered about it.
- **Progress is measured**, never estimated: the bytes are `wc -c` of the
partial and the total is the size HuggingFace published, absent when it
published none. A run whose pid is gone is reported failed rather than
left saying "running" for ever -- `kill -0` at each listing is the check --
and there is no `finished` state, because a download that finished is a
model and is in the list beside the ones still going.
- **Resume is guarded by identity, not by hope.** The state file records the
sha256 the partial is a piece of, and a partial written against a different
one is deleted rather than resumed onto. The same hash is checked, on that
machine, before the file takes its real name. The earlier ETag scheme went
with the local fetch; `If-Range` was never usable, since HuggingFace's CDN
ignores it (probed 2026-08-28).
- Sampling parameters reach a driver as an untyped `params` map, so the
shared schema does not grow llama.cpp's vocabulary.
@@ -1326,8 +1343,19 @@ dev-updater (Kotlin 2.4.x, CMP 1.11.x, JDK 21).
not been imported, and tapping it starts a second CLI on the same
transcript. That makes rows below slide up under the reader's finger, so a
row that has just moved ignores taps for `SETTLE_MS`.
3. **Models** and **Machines** — browsing and downloading GGUFs; adding,
renaming, re-probing and removing machines.
3. **Machines** — adding, renaming, re-probing and removing machines, and
what each one can run.
**A machine's models are that machine's**, so browsing and downloading
GGUFs lives inside its llama.cpp provider rather than in a tab of its own
(2026-09-19, `MachineModels.kt`). A "Models" tab was a claim that there is
one such set; there is one per machine, and the screen deciding how a
model is loaded is the screen that should be able to fetch one. A download
in flight is a card above the models, with a measured bar, and it keeps
going when the app is closed because it is a process on that machine.
**A provider's card is a card**: bordered against the machine's own card
rather than tinted a step away from it, since two adjacent surfaces render
as one flat block, and with no chevron — a card that reads as a card does
not need an arrow to say it opens.
**A provider is a card that opens** (2026-09-19, `ProviderScreen.kt`).
Settings that belong to a *machine* had nowhere to live until one
`llama-server` came to serve every session on one: how each of its models