Download GGUF models from HuggingFace, watchably

The first half of the llama.cpp work Bryan asked for: browse HuggingFace,
fetch a model, and see how far it has got from any device.

The design is dev-updater's build-progress shape with the four changes its
author recommended after living with it, since a model download is an hour
where a build is two minutes:

- **A run has an id.** Without one "not downloading" means three different
  things -- finished, never started, or someone else's run ended while you
  were away -- and over an hour that ambiguity is certain rather than
  theoretical. A device compares the run it was watching to the run
  reported now.
- **Outcomes outlive their run**, so a phone that was asleep at the moment
  of completion can still find out what happened.
- **Cancel exists.** Retrofitting cancellation into a blocking loop is
  miserable, and several gigabytes over someone's data plan is not
  something to have no answer for.
- **Progress is bytes, not a parsed marker.** We own the loop, so it counts
  directly; `total` is whatever Content-Length said and nothing else, and
  stays absent when the server sends none rather than becoming a bar drawn
  from a guess.

The download owns its own thread rather than the blocking pool, which
exists for short work. It resumes through HTTP Range, and trusts the 206
rather than the request -- a server that ignores Range answers 200 with the
whole file, and appending to that would corrupt it. `truncate(false)` on
the open is load-bearing for the same reason and says so.

Searching is proxied through the server rather than done from the phone,
because the app trusts exactly one certificate -- this one -- and the
machine that must do the downloading is also the one whose view of what
exists matters.

Verified against the real HuggingFace, not a mock: searched, listed a
repository's GGUFs, downloaded 234 MB with live byte progress, cancelled
mid-flight, confirmed the partial survived, restarted and watched it resume
at 162 MB rather than 0, and let it finish. The result's sha256 matches the
one HuggingFace publishes for that file, so the resume is byte-correct and
not merely the right length. llama.cpp then loaded it and ran inference.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017xn8nHw1tw1R6PtiY1eEtw
This commit is contained in:
irisandClaude Opus 5 committed 2026-08-28 05:01:53 -04:00
1 parent 4004183acf
commit 9d29776f02
3 files changed
+658

No files matched your search

+109
View File
@@ -456,3 +456,112 @@ async fn send_event(
tx.send(SseEvent::default().id(entry.seq.to_string()).data(data))
.await
}
/// Separate router because its state is the model store, like `usage`'s.
///
/// Keys are `owner/repo/file.gguf` and so contain slashes, which is why
/// nothing here puts one in the path: a key travels in the body or a query
/// string, and the routes stay addressable without escaping rules nobody
/// would get right from a phone.
pub fn models_router(store: Arc<crate::models::ModelStore>) -> Router {
Router::new()
.route("/models", get(list_models))
.route("/models/search", get(search_models))
.route("/models/files", get(repo_files))
.route("/models/download", post(start_download))
.route("/models/cancel", post(cancel_download))
.route("/models/delete", post(delete_model))
.with_state(store)
}
/// What this machine has and what it is fetching, in one answer.
///
/// Both together deliberately: a phone showing the model list needs both
/// to draw one screen, and two routes would let it render a model as
/// absent while its download sits at 99%.
#[derive(serde::Serialize)]
#[serde(rename_all = "camelCase")]
struct ModelsResponse {
local: Vec<crate::models::LocalModel>,
downloads: Vec<crate::models::DownloadStatus>,
}
async fn list_models(
State(store): State<Arc<crate::models::ModelStore>>,
) -> Result<axum::Json<ModelsResponse>, ApiError> {
let listing = tokio::task::spawn_blocking(move || ModelsResponse {
local: store.list(),
downloads: store.downloads(),
})
.await
.context("listing models panicked")?;
Ok(axum::Json(listing))
}
#[derive(Deserialize)]
struct SearchQuery {
q: String,
}
async fn search_models(
Query(query): Query<SearchQuery>,
) -> Result<axum::Json<Vec<crate::models::RemoteRepo>>, ApiError> {
// Blocking HTTP, like the usage fetch: off the request workers.
let found = tokio::task::spawn_blocking(move || crate::models::search(&query.q))
.await
.context("model search panicked")?
.map_err(bad_request)?;
Ok(axum::Json(found))
}
#[derive(Deserialize)]
struct RepoQuery {
repo: String,
}
async fn repo_files(
State(store): State<Arc<crate::models::ModelStore>>,
Query(query): Query<RepoQuery>,
) -> Result<axum::Json<Vec<crate::models::RemoteFile>>, ApiError> {
let files = tokio::task::spawn_blocking(move || crate::models::files(&query.repo, &store))
.await
.context("listing repository files panicked")?
.map_err(bad_request)?;
Ok(axum::Json(files))
}
#[derive(Deserialize)]
struct DownloadRequest {
repo: String,
file: String,
}
/// Starts a download, or rejoins the one already running for that model.
async fn start_download(
State(store): State<Arc<crate::models::ModelStore>>,
axum::Json(body): axum::Json<DownloadRequest>,
) -> Result<axum::Json<crate::models::DownloadStatus>, ApiError> {
let status = store.start(&body.repo, &body.file).map_err(bad_request)?;
Ok(axum::Json(status))
}
#[derive(Deserialize)]
struct KeyRequest {
key: String,
}
async fn cancel_download(
State(store): State<Arc<crate::models::ModelStore>>,
axum::Json(body): axum::Json<KeyRequest>,
) -> Result<axum::Json<crate::models::DownloadStatus>, ApiError> {
let status = store.cancel(&body.key).map_err(bad_request)?;
Ok(axum::Json(status))
}
async fn delete_model(
State(store): State<Arc<crate::models::ModelStore>>,
axum::Json(body): axum::Json<KeyRequest>,
) -> Result<StatusCode, ApiError> {
store.delete(&body.key).map_err(bad_request)?;
Ok(StatusCode::NO_CONTENT)
}