diff --git a/AGENTS.md b/AGENTS.md index ba1e4d2..cfbd3b8 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -61,7 +61,8 @@ Module-by-module intent is in PLAN.md's "Backend layout". because `llama-server` never reads the stdin whose closing ends a CLI and the same kill left it loaded on the far machine; the model is looked for on the machine that will serve it, so the spawn screen offers - `GET /machines/{id}/models` rather than the backend's own downloads; and the + `GET /machines/{id}/providers/{p}/models` rather than any list of this + backend's own; and the readiness poll watches the process as well as the port, since a model that will not load exits in a second and was being reported as "gave up after 300s". See PLAN.md's "Transport" and "llama-server management". @@ -119,14 +120,27 @@ Module-by-module intent is in PLAN.md's "Backend layout". for a provider's **models**, drawn in the machines tab's provider view — the settings that decide how a model is loaded, which belong to the machine because one loaded copy answers every session using it. + **A model is downloaded onto the machine that will serve it** (2026-09-19, + replacing the fetch this backend used to do onto its own disk, and the + Models tab that went with it). `models.rs` writes a script and a detached + `curl` runs it *there*; the state of a run is a file beside the partial + (`x.gguf.download`), so nothing about it is held in this process — it + survives the phone closing, this backend restarting and a second device + watching, and `kill -0` at each listing is what stops a machine that was + rebooted from leaving a download claiming to be running. The progress is + `wc -c` of the partial against the size HuggingFace published, the sha256 + it publishes is what makes a resume safe, and a finished download is not a + state: it is a model, in the list beside the one still going. Codex is one persistent `codex app-server --stdio` process per session; its driver uses native turn steering and interruption, persists the protocol state and thread id, and reads subscription limits through the same CLI protocol. - `app/` — the Compose app, package `com.example.aiapp`, label "AI Sessions". - `AppRoot.kt` is the navigation `when`; `MainScreen.kt` the root's four tabs - (sessions, import, models, machines); `Api.kt`/`EventStream.kt` the REST + SSE - clients; `Events.kt` the event model mirror; `ServerConfig.kt` settings and + `AppRoot.kt` is the navigation `when`; `MainScreen.kt` the root's three tabs + (sessions, import, machines); `MachineModels.kt` the models on one machine + and the downloads putting them there, drawn inside `ProviderScreen.kt` for a + provider that serves files off that machine's disk; `Api.kt`/`EventStream.kt` + the REST + SSE clients; `Events.kt` the event model mirror; `ServerConfig.kt` settings and the Keystore-sealed token. - `wg-app-link/` — a **git submodule** shared with dev-updater: the pinned CA and leaf (`certs`), QR enrollment and the bearer token (`enroll`), wg0 diff --git a/PLAN.md b/PLAN.md index 67adc2a..4e18973 100644 --- a/PLAN.md +++ b/PLAN.md @@ -35,7 +35,7 @@ backend (Rust/Axum, desktop) │ each driver's process is spawned through a Transport, │ locally or as `ssh host …`, decided by the configured machine ├─ usage.rs (provider usage meters, per machine) - ├─ models.rs (HuggingFace browsing and GGUF downloads) + ├─ models.rs (HuggingFace browsing; a machine's GGUFs and downloads) ├─ files.rs (the file explorer's half of the backend) └─ config.ron + per-session transcript files ``` @@ -92,7 +92,9 @@ axum 0.8, axum-server + rustls, tokio, serde, clap, tracing. Rust edition - `machines.rs` — machines and provider discovery. - `files.rs` — the file explorer (`EXPLORER.md`). - `usage.rs` — provider usage polling, per machine. -- `models.rs` — HuggingFace browsing and GGUF downloads. +- `models.rs` — HuggingFace browsing, and the GGUFs on a machine with the + downloads putting them there. Addressed by transport and directory, never + by this process's own disk. - `media.rs` — the image media-type/extension table, shared by the four places that must agree: storing an upload, serving it back, handing one to a driver, and saving one a tool produced. @@ -424,11 +426,12 @@ deliberate and easy to undo by accident: key on that machine — one round trip answering "at /abs/path" or "missing", so a model that is not there is refused at the spawn rather than becoming a server that never becomes ready. The spawn screen offers - `GET /machines/{id}/models`, that machine's list, rather than `GET /models`, - which is this backend's downloads. Downloading *to* another machine is - deliberately not built: a multi-gigabyte transfer with no progress - anywhere, and the file gets there however anything else on that machine - did. + `GET /machines/{id}/providers/{p}/models`, that machine's list, rather than + anything about this backend's own disk. **Downloading to that machine was + deliberately not built until 2026-09-19** -- the objection was a + multi-gigabyte transfer with no progress anywhere -- and what changed is + that the transfer happens *on* that machine and reports progress: see + "Models" below. - **The readiness poll watches the process, not only the port.** A model that will not load, a port already taken, a flag an older build does not know: all exit within a second and none will ever answer `/health`, so waiting @@ -506,21 +509,35 @@ deliberate and easy to undo by accident: of the file — on the machine that will serve it, in the round trip the spawn was already making — and `params["speculative"] = "off"` is the way out. -### Models (2026-08-28) +### Models (2026-08-28, rebuilt per machine 2026-09-19) - **A download belongs to the model, not to the request.** Keyed by - `owner/repo/file.gguf` and owned by the server, so a second device can - watch one it did not start and an hour-long fetch survives a locked screen. - Every run has an id and its outcome outlives it, because "not downloading" - otherwise means finished, never started, or someone else's run ended while - you were away. -- **Progress is measured**, never estimated: `total` is Content-Length, or - Content-Range's last field on a resume, and absent when the server says - nothing. -- **Resume is guarded by identity, not by hope.** A partial carries the ETag - it was written against and a mismatch discards it. `If-Range` would be the - tidy mechanism but HuggingFace's CDN ignores it (probed 2026-08-28). The - published sha256 is checked before the file is renamed. + `owner/repo/file.gguf`, so a second device can watch one it did not start + and an hour-long fetch survives a locked screen. Asking for one already + going joins it rather than starting a second writer. +- **A download runs on the machine that will serve the file** (2026-09-19), + because that is where `llama-server` has to read it from -- so it is a + detached `curl` started by a script this backend writes over the same + transport everything else about a machine goes through. What this replaced + was a fetch onto the backend's own disk, offered under a Models tab, which + could not put a model on any other machine at all. +- **Its state is a file beside the partial**, `x.gguf.download`, holding the + worker's pid, the sha it is downloading against, the published size and a + state word. Nothing about a run is held in this process, which is what + makes it survive a backend restart, and what lets the answer be read off + the disk that has the file rather than remembered about it. +- **Progress is measured**, never estimated: the bytes are `wc -c` of the + partial and the total is the size HuggingFace published, absent when it + published none. A run whose pid is gone is reported failed rather than + left saying "running" for ever -- `kill -0` at each listing is the check -- + and there is no `finished` state, because a download that finished is a + model and is in the list beside the ones still going. +- **Resume is guarded by identity, not by hope.** The state file records the + sha256 the partial is a piece of, and a partial written against a different + one is deleted rather than resumed onto. The same hash is checked, on that + machine, before the file takes its real name. The earlier ETag scheme went + with the local fetch; `If-Range` was never usable, since HuggingFace's CDN + ignores it (probed 2026-08-28). - Sampling parameters reach a driver as an untyped `params` map, so the shared schema does not grow llama.cpp's vocabulary. @@ -1326,8 +1343,19 @@ dev-updater (Kotlin 2.4.x, CMP 1.11.x, JDK 21). not been imported, and tapping it starts a second CLI on the same transcript. That makes rows below slide up under the reader's finger, so a row that has just moved ignores taps for `SETTLE_MS`. -3. **Models** and **Machines** — browsing and downloading GGUFs; adding, - renaming, re-probing and removing machines. +3. **Machines** — adding, renaming, re-probing and removing machines, and + what each one can run. + **A machine's models are that machine's**, so browsing and downloading + GGUFs lives inside its llama.cpp provider rather than in a tab of its own + (2026-09-19, `MachineModels.kt`). A "Models" tab was a claim that there is + one such set; there is one per machine, and the screen deciding how a + model is loaded is the screen that should be able to fetch one. A download + in flight is a card above the models, with a measured bar, and it keeps + going when the app is closed because it is a process on that machine. + **A provider's card is a card**: bordered against the machine's own card + rather than tinted a step away from it, since two adjacent surfaces render + as one flat block, and with no chevron — a card that reads as a card does + not need an arrow to say it opens. **A provider is a card that opens** (2026-09-19, `ProviderScreen.kt`). Settings that belong to a *machine* had nowhere to live until one `llama-server` came to serve every session on one: how each of its models diff --git a/app/androidApp/src/main/kotlin/com/example/aiapp/Api.kt b/app/androidApp/src/main/kotlin/com/example/aiapp/Api.kt index ce9b809..8c63ca9 100644 --- a/app/androidApp/src/main/kotlin/com/example/aiapp/Api.kt +++ b/app/androidApp/src/main/kotlin/com/example/aiapp/Api.kt @@ -350,6 +350,8 @@ fun deleteSubagents(settings: ServerSettings, sessionId: String, subagentIds: Li data class Provider( val name: String, val kind: String, + /** The program discovery found on that machine, or null for a provider that is not one. */ + val command: String?, val models: List, val permissionModes: List, val defaultPermissionMode: String?, @@ -399,6 +401,7 @@ private fun parseProvider(provider: JSONObject): Provider { return Provider( name = provider.getString("name"), kind = kind, + command = provider.optString("command").ifEmpty { null }, // Omitted entirely when the provider offers none. models = provider.optJSONArray("models")?.strings().orEmpty(), permissionModes = provider.optJSONArray("permissionModes")?.strings().orEmpty(), @@ -1397,7 +1400,9 @@ fun deleteSession(settings: ServerSettings, sessionId: String, deleteForeign: Bo requestFromServer(settings, "/sessions/$sessionId$query", method = "DELETE") {} } -// Models: what this backend has downloaded, what it is downloading, and what HuggingFace offers. +// Models: what one machine has, what it is fetching onto itself, and what HuggingFace offers. +// Under a machine because that is whose disk the file is on -- `llama-server` reads it where it +// runs, so a list or a download naming anywhere else would be about the wrong filesystem. // Browsing is proxied by the server rather than done here, because this app trusts exactly one // certificate and has no general internet trust to spend on huggingface.co. @@ -1415,12 +1420,14 @@ data class LocalModel( ) /** - * A download in flight or finished. [total] is null when the server never said how big the file is + * A download in flight or stopped. [total] is null when HuggingFace never said how big the file is * -- which must render as "not known", never as a bar at some invented position. + * + * There is no "finished": a download that finished is a model, and it is in [Models.local] beside + * this one. */ data class Download( val key: String, - val run: Long, val repo: String, val file: String, val state: String, @@ -1435,18 +1442,98 @@ data class RemoteRepo(val id: String, val downloads: Long, val likes: Long) data class RemoteFile(val path: String, val bytes: Long, val have: Boolean) -private fun parseDownload(o: JSONObject) = - Download( - key = o.getString("key"), - run = o.getLong("run"), - repo = o.getString("repo"), - file = o.getString("file"), - state = o.getString("state"), - done = o.getLong("done"), - // Absent rather than zero when unknown; see the field's comment. - total = if (o.has("total")) o.getLong("total") else null, - error = if (o.has("error")) o.getString("error") else null, - ) +private fun modelsPath(machineId: String, tail: String = "") = + "/machines/${machineId.urlEncoded()}/models$tail" + +/** Every GGUF on one machine, and every download putting one there. */ +fun fetchMachineModels(settings: ServerSettings, machineId: String): Models = + requestFromServer(settings, modelsPath(machineId), readTimeoutMs = 40000) { connection -> + val body = connection.jsonObject() + Models( + local = + body.getJSONArray("local").mapObjects { m -> + LocalModel( + key = m.getString("key"), + repo = m.getString("repo"), + file = m.getString("file"), + bytes = m.getLong("bytes"), + name = m.optString("name").ifEmpty { null }, + ) + }, + downloads = + body.getJSONArray("downloads").mapObjects { o -> + Download( + key = o.getString("key"), + repo = o.getString("repo"), + file = o.getString("file"), + state = o.getString("state"), + done = o.getLong("done"), + // Absent rather than zero when unknown; see the field's comment. + total = if (o.has("total")) o.getLong("total") else null, + error = if (o.has("error")) o.getString("error") else null, + ) + }, + ) + } + +fun searchModels(settings: ServerSettings, query: String): List = + requestFromServer(settings, "/models/search?q=${query.urlEncoded()}") { connection -> + connection.jsonObjects { r -> + RemoteRepo( + id = r.getString("id"), + downloads = r.getLong("downloads"), + likes = r.getLong("likes"), + ) + } + } + +fun fetchRepoFiles(settings: ServerSettings, machineId: String, repo: String): List = + requestFromServer( + settings, + modelsPath(machineId, "/files?repo=${repo.urlEncoded()}"), + readTimeoutMs = 40000, + ) { connection -> + connection.jsonObjects { f -> + RemoteFile( + path = f.getString("path"), + bytes = f.getLong("bytes"), + have = f.getBoolean("have"), + ) + } + } + +/** Starts one on that machine, or joins the run already going for the same model. */ +fun startDownload(settings: ServerSettings, machineId: String, repo: String, file: String) { + requestFromServer( + settings, + modelsPath(machineId, "/download"), + method = "POST", + jsonBody = JSONObject().put("repo", repo).put("file", file).toString(), + readTimeoutMs = 40000, + ) {} +} + +/** Stops one. The partial stays on the machine, so starting again carries on from there. */ +fun cancelDownload(settings: ServerSettings, machineId: String, key: String) { + requestFromServer( + settings, + modelsPath(machineId, "/cancel"), + method = "POST", + jsonBody = JSONObject().put("key", key).toString(), + readTimeoutMs = 40000, + ) {} +} + +/** Takes a model off that machine, downloaded or half-downloaded. */ +fun deleteModel(settings: ServerSettings, machineId: String, key: String) { + requestFromServer( + settings, + modelsPath(machineId, "/delete"), + method = "POST", + jsonBody = JSONObject().put("key", key).toString(), + readTimeoutMs = 40000, + ) {} +} /** * One model a picker can offer. @@ -1612,71 +1699,3 @@ fun unloadProviderModel( readTimeoutMs = 40000, ) {} } - -fun fetchModels(settings: ServerSettings): Models = - requestFromServer(settings, "/models") { connection -> - val body = JSONObject(connection.inputStream.bufferedReader().readText()) - Models( - local = - body.getJSONArray("local").mapObjects { m -> - LocalModel( - key = m.getString("key"), - repo = m.getString("repo"), - file = m.getString("file"), - bytes = m.getLong("bytes"), - name = m.optString("name").ifEmpty { null }, - ) - }, - downloads = body.getJSONArray("downloads").mapObjects(::parseDownload), - ) - } - -fun searchModels(settings: ServerSettings, query: String): List = - requestFromServer(settings, "/models/search?q=${query.urlEncoded()}") { connection -> - connection.jsonObjects { r -> - RemoteRepo( - id = r.getString("id"), - downloads = r.getLong("downloads"), - likes = r.getLong("likes"), - ) - } - } - -fun fetchRepoFiles(settings: ServerSettings, repo: String): List = - requestFromServer(settings, "/models/files?repo=${repo.urlEncoded()}") { connection -> - connection.jsonObjects { f -> - RemoteFile( - path = f.getString("path"), - bytes = f.getLong("bytes"), - have = f.getBoolean("have"), - ) - } - } - -fun startDownload(settings: ServerSettings, repo: String, file: String): Download = - requestFromServer( - settings, - "/models/download", - method = "POST", - jsonBody = JSONObject().put("repo", repo).put("file", file).toString(), - ) { connection -> - parseDownload(JSONObject(connection.inputStream.bufferedReader().readText())) - } - -fun cancelDownload(settings: ServerSettings, key: String) { - requestFromServer( - settings, - "/models/cancel", - method = "POST", - jsonBody = JSONObject().put("key", key).toString(), - ) {} -} - -fun deleteModel(settings: ServerSettings, key: String) { - requestFromServer( - settings, - "/models/delete", - method = "POST", - jsonBody = JSONObject().put("key", key).toString(), - ) {} -} diff --git a/app/androidApp/src/main/kotlin/com/example/aiapp/MachineModels.kt b/app/androidApp/src/main/kotlin/com/example/aiapp/MachineModels.kt new file mode 100644 index 0000000..820541a --- /dev/null +++ b/app/androidApp/src/main/kotlin/com/example/aiapp/MachineModels.kt @@ -0,0 +1,404 @@ +package com.example.aiapp + +import androidx.compose.foundation.layout.Column +import androidx.compose.foundation.layout.Row +import androidx.compose.foundation.layout.Spacer +import androidx.compose.foundation.layout.fillMaxWidth +import androidx.compose.foundation.layout.height +import androidx.compose.foundation.layout.padding +import androidx.compose.foundation.lazy.LazyListScope +import androidx.compose.foundation.text.KeyboardActions +import androidx.compose.foundation.text.KeyboardOptions +import androidx.compose.material3.Card +import androidx.compose.material3.CircularProgressIndicator +import androidx.compose.material3.LinearProgressIndicator +import androidx.compose.material3.MaterialTheme +import androidx.compose.material3.OutlinedTextField +import androidx.compose.material3.Text +import androidx.compose.material3.TextButton +import androidx.compose.runtime.Composable +import androidx.compose.runtime.LaunchedEffect +import androidx.compose.runtime.Stable +import androidx.compose.runtime.getValue +import androidx.compose.runtime.mutableStateOf +import androidx.compose.runtime.remember +import androidx.compose.runtime.rememberCoroutineScope +import androidx.compose.runtime.setValue +import androidx.compose.ui.Alignment +import androidx.compose.ui.Modifier +import androidx.compose.ui.platform.LocalSoftwareKeyboardController +import androidx.compose.ui.text.input.ImeAction +import androidx.compose.ui.text.style.TextOverflow +import androidx.compose.ui.unit.dp +import kotlinx.coroutines.CoroutineScope +import kotlinx.coroutines.Dispatchers +import kotlinx.coroutines.delay +import kotlinx.coroutines.launch +import kotlinx.coroutines.withContext + +/** + * The models on one machine, the downloads putting more there, and HuggingFace to find them in. + * + * This was a tab of its own, about the backend's own disk. It moved under the machine's llama.cpp + * provider on 2026-09-19, when a download came to run on the machine that will serve the file: + * there is no such thing as "the models", only this machine's, and the screen that decides how a + * model is loaded is the screen that should be able to fetch one. + * + * Everything here is the machine's state rather than this screen's. A download is a process on that + * machine with its progress written beside the partial file, so closing the app, locking the phone + * or restarting the backend does not touch it, and a second device watching sees the same numbers. + */ +@Stable +class MachineModelsState( + private val settings: ServerSettings, + private val machineId: String, + private val scope: CoroutineScope, +) { + var state by mutableStateOf>(LoadState.Loading) + private set + + var query by mutableStateOf("") + + var results by mutableStateOf>?>(null) + private set + + var openRepo by mutableStateOf(null) + private set + + var repoFiles by mutableStateOf>?>(null) + private set + + /** What the last action said went wrong, shown above the list that action was taken in. */ + var actionError by mutableStateOf(null) + private set + + val models: Models? + get() = (state as? LoadState.Loaded)?.value + + val downloads: List + get() = models?.downloads.orEmpty() + + /** How big each downloaded model is, by key, for the cards the provider screen draws. */ + val sizes: Map + get() = models?.local.orEmpty().associate { it.key to it.bytes } + + suspend fun reload() { + state = + try { + withContext(Dispatchers.IO) { + LoadState.Loaded(fetchMachineModels(settings, machineId)) + } + } catch (e: ApiException) { + LoadState.failed(e) + } + } + + /** Runs [action], says what it said if it failed, and asks the machine again either way. */ + private fun act(action: suspend () -> Unit) { + scope.launch { + actionError = + runCatching { withContext(Dispatchers.IO) { action() } }.exceptionOrNull()?.message + reload() + } + } + + fun search() { + openRepo = null + results = LoadState.Loading + scope.launch { + results = + try { + withContext(Dispatchers.IO) { LoadState.Loaded(searchModels(settings, query)) } + } catch (e: ApiException) { + LoadState.failed(e) + } + } + } + + fun toggleRepo(repo: String) { + if (openRepo == repo) { + openRepo = null + return + } + openRepo = repo + repoFiles = LoadState.Loading + scope.launch { + repoFiles = + try { + withContext(Dispatchers.IO) { + LoadState.Loaded(fetchRepoFiles(settings, machineId, repo)) + } + } catch (e: ApiException) { + LoadState.failed(e) + } + } + } + + fun download(repo: String, file: String) = act { + startDownload(settings, machineId, repo, file) + } + + fun cancel(key: String) = act { cancelDownload(settings, machineId, key) } + + fun remove(key: String) = act { deleteModel(settings, machineId, key) } +} + +/** + * One machine's models, asked for again while this screen is open. + * + * Polled rather than pushed: a download belongs to a machine, not to any session, so it has no + * event stream of its own. Faster while something is downloading, because that is the only thing + * here that changes by itself -- each ask is a round trip to that machine, and once a minute would + * be a progress bar that moved in jumps. + * + * [onLocalChange] fires when the set of models on the machine changes, which is how the screen + * around this learns that a download has become a model it must now draw settings for. + * + * [enabled] is false for a provider that holds no files of its own -- the Claude CLI names its + * models rather than storing them -- and then nothing is asked of the machine at all. Taken as a + * parameter rather than decided by the caller's `if`, so that this is composed unconditionally and + * keeps its search results across the moment the provider's kind arrives. + */ +@Composable +fun rememberMachineModels( + settings: ServerSettings, + machineId: String, + enabled: Boolean, + onLocalChange: () -> Unit, +): MachineModelsState { + val scope = rememberCoroutineScope() + val state = remember(settings, machineId) { MachineModelsState(settings, machineId, scope) } + LaunchedEffect(state, enabled) { + if (!enabled) return@LaunchedEffect + var known: List? = null + while (true) { + state.reload() + val local = state.models?.local?.map { it.key } + if (local != null) { + if (known != null && known != local) onLocalChange() + known = local + } + delay(if (state.downloads.any { it.state == "running" }) 1500 else 5000) + } + } + return state +} + +/** What is being fetched onto this machine, above the models it already has. */ +fun LazyListScope.downloadCards(state: MachineModelsState) { + uniqueItems(state.downloads, key = { "download:" + it.key }) { download -> + DownloadCard( + download = download, + onCancel = { state.cancel(download.key) }, + onResume = { state.download(download.repo, download.file) }, + onRemove = { state.remove(download.key) }, + ) + } +} + +/** + * Finding a model to fetch: a search, and what it found. + * + * Below the models this machine has rather than above them, because what is here is what the reader + * came for and getting another is the rarer errand. + */ +fun LazyListScope.modelSearch(state: MachineModelsState) { + item("search") { + Spacer(Modifier.height(16.dp)) + Text("Get another model", style = MaterialTheme.typography.titleSmall) + Text( + "Downloaded onto this machine, which is where llama.cpp reads it from.", + style = MaterialTheme.typography.bodySmall, + color = MaterialTheme.colorScheme.onSurfaceVariant, + ) + Spacer(Modifier.height(8.dp)) + val keyboard = LocalSoftwareKeyboardController.current + OutlinedTextField( + value = state.query, + onValueChange = { state.query = it }, + label = { Text("Search HuggingFace") }, + singleLine = true, + // The keyboard's own key searches, and puts itself away to show what it found. The + // button below this is under the keyboard while it is up, so without this the only + // way to press it is to dismiss the keyboard first -- which nothing on screen says. + keyboardOptions = KeyboardOptions(imeAction = ImeAction.Search), + keyboardActions = + KeyboardActions( + onSearch = { + keyboard?.hide() + state.search() + } + ), + modifier = Modifier.fillMaxWidth(), + ) + TextButton( + enabled = state.query.isNotBlank(), + onClick = { + keyboard?.hide() + state.search() + }, + ) { + Text("Search") + } + } + when (val found = state.results) { + null -> {} + is LoadState.Loading -> item("searching") { CircularProgressIndicator() } + is LoadState.Error -> + item("search-failed") { Text(found.message, color = MaterialTheme.colorScheme.error) } + is LoadState.Loaded -> + uniqueItems(found.value, key = { "repo:" + it.id }) { repo -> + val open = state.openRepo == repo.id + RepoRow(repo, expanded = open) { state.toggleRepo(repo.id) } + // Inside the expanded repository's own item rather than as a section after the + // list: drawn after every card, a repository's files read as belonging to + // whichever card happened to be last. + if (open) { + when (val files = state.repoFiles) { + null -> {} + is LoadState.Loading -> CircularProgressIndicator() + is LoadState.Error -> + Text(files.message, color = MaterialTheme.colorScheme.error) + is LoadState.Loaded -> + Column { + val busy = state.downloads.map { it.key }.toSet() + files.value.forEach { file -> + RepoFileRow( + file, + downloading = "${repo.id}/${file.path}" in busy, + ) { + state.download(repo.id, file.path) + } + } + } + } + } + } + } +} + +@Composable +private fun DownloadCard( + download: Download, + onCancel: () -> Unit, + onResume: () -> Unit, + onRemove: () -> Unit, +) { + val running = download.state == "running" || download.state == "verifying" + Card(Modifier.fillMaxWidth().padding(vertical = 4.dp)) { + Column(Modifier.padding(12.dp)) { + Text(download.file, style = MaterialTheme.typography.titleSmall) + Text( + download.repo, + style = MaterialTheme.typography.bodySmall, + color = MaterialTheme.colorScheme.onSurfaceVariant, + ) + Spacer(Modifier.height(8.dp)) + // A determinate bar only when the size is known. HuggingFace sends no size when it + // was never told one, and a bar drawn from a guess is worse than one that admits it + // is counting. + if (download.total != null && download.total > 0) { + LinearProgressIndicator( + progress = { download.done.toFloat() / download.total.toFloat() }, + // Blue at every value, unlike a quota bar: a download nearing its end is + // nearing success, and colouring it like a limit being approached would say + // the opposite. + color = progressColor, + modifier = Modifier.fillMaxWidth(), + ) + Text( + "${gigabytes(download.done)} of ${gigabytes(download.total)}", + style = MaterialTheme.typography.bodySmall, + ) + } else if (running) { + LinearProgressIndicator(color = progressColor, modifier = Modifier.fillMaxWidth()) + Text( + "${gigabytes(download.done)} so far, total size unknown", + style = MaterialTheme.typography.bodySmall, + ) + } + download.error?.let { + Text( + it, + style = MaterialTheme.typography.bodySmall, + color = MaterialTheme.colorScheme.error, + ) + } + Row(verticalAlignment = Alignment.CenterVertically) { + Text( + download.state, + style = MaterialTheme.typography.bodySmall, + modifier = Modifier.weight(1f), + ) + if (running) { + TextButton(onClick = onCancel) { Text("Cancel") } + } else { + // A stopped download kept its partial file, so carrying on is the cheap + // answer and starting again is not the only one offered. + TextButton(onClick = onResume) { Text("Resume") } + TextButton(onClick = onRemove) { Text("Remove") } + } + } + } + } +} + +@Composable +private fun RepoRow(repo: RemoteRepo, expanded: Boolean, onToggle: () -> Unit) { + Card(Modifier.fillMaxWidth().padding(vertical = 4.dp)) { + Row(Modifier.padding(12.dp), verticalAlignment = Alignment.CenterVertically) { + Column(Modifier.weight(1f)) { + Text( + repo.id, + style = MaterialTheme.typography.titleSmall, + maxLines = 1, + // The owner is the part that repeats; the model name at the end is what tells + // two entries apart. + overflow = TextOverflow.StartEllipsis, + ) + Text( + "${repo.downloads} downloads · ${repo.likes} likes", + style = MaterialTheme.typography.bodySmall, + color = MaterialTheme.colorScheme.onSurfaceVariant, + ) + } + TextButton(onClick = onToggle) { Text(if (expanded) "Hide" else "Files") } + } + } +} + +@Composable +private fun RepoFileRow(file: RemoteFile, downloading: Boolean, onDownload: () -> Unit) { + Row( + Modifier.fillMaxWidth().padding(start = 16.dp, top = 4.dp, bottom = 4.dp), + verticalAlignment = Alignment.CenterVertically, + ) { + Column(Modifier.weight(1f)) { + Text(file.path, style = MaterialTheme.typography.bodyMedium) + Text( + gigabytes(file.bytes), + style = MaterialTheme.typography.bodySmall, + color = MaterialTheme.colorScheme.onSurfaceVariant, + ) + } + // Disabled rather than absent, so the row reads the same whether this one is absent, + // already here, or on its way. Offering "Download" for a file that is downloading would be + // a button that does nothing anyone can see. + TextButton(enabled = !file.have && !downloading, onClick = onDownload) { + Text( + when { + file.have -> "Downloaded" + downloading -> "Downloading" + else -> "Download" + } + ) + } + } +} + +fun gigabytes(bytes: Long): String = + if (bytes >= 1_000_000_000) { + "%.2f GB".format(bytes / 1_000_000_000.0) + } else { + "%.0f MB".format(bytes / 1_000_000.0) + } diff --git a/app/androidApp/src/main/kotlin/com/example/aiapp/MachinesScreen.kt b/app/androidApp/src/main/kotlin/com/example/aiapp/MachinesScreen.kt index 6c5bf04..805efe1 100644 --- a/app/androidApp/src/main/kotlin/com/example/aiapp/MachinesScreen.kt +++ b/app/androidApp/src/main/kotlin/com/example/aiapp/MachinesScreen.kt @@ -1,5 +1,6 @@ package com.example.aiapp +import androidx.compose.foundation.BorderStroke import androidx.compose.foundation.clickable import androidx.compose.foundation.layout.Column import androidx.compose.foundation.layout.Row @@ -26,8 +27,11 @@ import androidx.compose.runtime.rememberCoroutineScope import androidx.compose.runtime.setValue import androidx.compose.ui.Alignment import androidx.compose.ui.Modifier +import androidx.compose.ui.graphics.Color import androidx.compose.ui.semantics.contentDescription import androidx.compose.ui.semantics.semantics +import androidx.compose.ui.text.font.FontFamily +import androidx.compose.ui.text.style.TextOverflow import androidx.compose.ui.unit.dp import kotlinx.coroutines.Dispatchers import kotlinx.coroutines.launch @@ -246,32 +250,50 @@ private fun MachineCard( machine.providers.forEach { provider -> // A card of its own rather than a line of text: a provider is where the // settings that belong to *this machine* live -- how each of its models is - // loaded, and the server holding them -- and those had nowhere to be until - // one llama-server came to serve every session on a machine. + // loaded, the models themselves, and the server holding them -- and those had + // nowhere to be until one llama-server came to serve every session on a + // machine. Sized by its own padding rather than by whatever control happened + // to be on its row, like the tool call cards it is built after. Card( - Modifier.fillMaxWidth().padding(vertical = 2.dp).clickable { - onProvider(provider) - }, - colors = - CardDefaults.cardColors( - containerColor = MaterialTheme.colorScheme.surfaceVariant - ), + Modifier.fillMaxWidth() + .padding(vertical = 4.dp) + .clickable { onProvider(provider) } + .semantics { contentDescription = "Open ${provider.name}" }, + // A border, and the machine card's own surface kept underneath it. + // The tint that was here before is one step along the surface ladder + // from the card it sits in, and two adjacent surfaces render as one flat + // block: these read as lines of text in a box rather than as things to + // open. One cue, and a visible one. + colors = CardDefaults.cardColors(containerColor = Color.Transparent), + border = BorderStroke(1.dp, MaterialTheme.colorScheme.outlineVariant), ) { Row( verticalAlignment = Alignment.CenterVertically, - modifier = Modifier.fillMaxWidth().padding(horizontal = 12.dp), + modifier = Modifier.fillMaxWidth().padding(12.dp), ) { - Text(provider.name, style = MaterialTheme.typography.bodyMedium) - Spacer(Modifier.weight(1f)) + Column(Modifier.weight(1f)) { + Text(provider.name, style = MaterialTheme.typography.titleSmall) + // What was actually found, which is the honest second line and + // the one thing here nobody can change. No arrow: a card that + // lifts off the one behind it already reads as something to open, + // and the chevron was the only thing making these look like rows + // of a list. + provider.command?.let { + Text( + it, + style = MaterialTheme.typography.bodySmall, + fontFamily = FontFamily.Monospace, + color = MaterialTheme.colorScheme.onSurfaceVariant, + maxLines = 1, + // A program is identified by its name, which is the tail + // of its path. + overflow = TextOverflow.StartEllipsis, + ) + } + } if (provider.kind == "claude_cli") { TextButton(onClick = { onSignIn(provider) }) { Text("Sign in") } } - Chevron( - Pointing.Right, - Modifier.padding(start = 4.dp).semantics { - contentDescription = "Settings for ${provider.name}" - }, - ) } } } diff --git a/app/androidApp/src/main/kotlin/com/example/aiapp/MainScreen.kt b/app/androidApp/src/main/kotlin/com/example/aiapp/MainScreen.kt index f759848..fb80665 100644 --- a/app/androidApp/src/main/kotlin/com/example/aiapp/MainScreen.kt +++ b/app/androidApp/src/main/kotlin/com/example/aiapp/MainScreen.kt @@ -26,18 +26,22 @@ import androidx.lifecycle.compose.LocalLifecycleOwner import androidx.lifecycle.repeatOnLifecycle /** - * The app's root: one title, and four views of the backend behind it. + * The app's root: one title, and three views of the backend behind it. * - * These were four screens reached by four words in a row under the title, and the row was already - * full. Tabs say the same thing in less space and say one more thing besides: that these are places - * to be rather than errands to run. Sessions, the machine's importable history, the models on it - * and the machines themselves are all *the same backend*, looked at four ways, and none is a step - * down from another. Settings still is, which is why it stays a pushed screen with its own Back. + * These were screens reached by words in a row under the title, and the row was already full. Tabs + * say the same thing in less space and say one more thing besides: that these are places to be + * rather than errands to run. Sessions, the machine's importable history and the machines + * themselves are all *the same backend*, looked at three ways, and none is a step down from + * another. Settings still is, which is why it stays a pushed screen with its own Back. + * + * Models were a fourth tab until 2026-09-19. They are a machine's models now -- downloaded onto the + * machine that has to serve them -- so they live under that machine's llama.cpp provider, beside + * the settings deciding how each one is loaded. A tab about "the models" was a claim that there is + * one such set, and there is one per machine. */ private enum class MainTab(val label: String) { Sessions("Sessions"), Import("Import"), - Models("Models"), Machines("Machines"), } @@ -145,7 +149,6 @@ fun MainScreen( ) MainTab.Import -> ImportScreen(settings = settings, reloadToken = token, onImported = onImported) - MainTab.Models -> ModelsScreen(settings = settings, reloadToken = token) MainTab.Machines -> MachinesScreen(settings = settings, reloadToken = token, onProvider = onProvider) } diff --git a/app/androidApp/src/main/kotlin/com/example/aiapp/ModelsScreen.kt b/app/androidApp/src/main/kotlin/com/example/aiapp/ModelsScreen.kt deleted file mode 100644 index 0521f9f..0000000 --- a/app/androidApp/src/main/kotlin/com/example/aiapp/ModelsScreen.kt +++ /dev/null @@ -1,374 +0,0 @@ -package com.example.aiapp - -import androidx.compose.foundation.layout.Column -import androidx.compose.foundation.layout.Row -import androidx.compose.foundation.layout.Spacer -import androidx.compose.foundation.layout.fillMaxSize -import androidx.compose.foundation.layout.fillMaxWidth -import androidx.compose.foundation.layout.height -import androidx.compose.foundation.layout.padding -import androidx.compose.foundation.lazy.LazyColumn -import androidx.compose.material3.Card -import androidx.compose.material3.CircularProgressIndicator -import androidx.compose.material3.LinearProgressIndicator -import androidx.compose.material3.MaterialTheme -import androidx.compose.material3.OutlinedTextField -import androidx.compose.material3.Text -import androidx.compose.material3.TextButton -import androidx.compose.runtime.Composable -import androidx.compose.runtime.LaunchedEffect -import androidx.compose.runtime.getValue -import androidx.compose.runtime.mutableStateOf -import androidx.compose.runtime.remember -import androidx.compose.runtime.rememberCoroutineScope -import androidx.compose.runtime.setValue -import androidx.compose.ui.Alignment -import androidx.compose.ui.Modifier -import androidx.compose.ui.text.style.TextOverflow -import androidx.compose.ui.unit.dp -import kotlinx.coroutines.Dispatchers -import kotlinx.coroutines.delay -import kotlinx.coroutines.launch -import kotlinx.coroutines.withContext - -/** - * Models on the backend, and HuggingFace to get more from. - * - * Everything here is the server's state rather than this screen's: what is downloaded, and what is - * downloading, are the same answers on every enrolled device, and a download started here keeps - * going when this screen closes. - */ -@Composable -fun ModelsScreen(settings: ServerSettings, reloadToken: Int) { - val scope = rememberCoroutineScope() - var state by remember { mutableStateOf>(LoadState.Loading) } - var query by remember { mutableStateOf("") } - var results by remember { mutableStateOf>?>(null) } - var openRepo by remember { mutableStateOf(null) } - var repoFiles by remember { mutableStateOf>?>(null) } - var actionError by remember { mutableStateOf(null) } - - suspend fun reload() { - state = - try { - withContext(Dispatchers.IO) { LoadState.Loaded(fetchModels(settings)) } - } catch (e: ApiException) { - LoadState.failed(e) - } - } - - // Polled rather than pushed: a download belongs to the machine, not to any session, so it has - // no event stream of its own. Keyed on the token as well, so the header's Refresh restarts the - // loop with a read now rather than leaving the reader watching for a second and a half. - LaunchedEffect(reloadToken) { - while (true) { - reload() - delay(1500) - } - } - - Column(Modifier.fillMaxSize().padding(16.dp)) { - actionError?.let { - Text(it, color = MaterialTheme.colorScheme.error) - Spacer(Modifier.height(8.dp)) - } - - OutlinedTextField( - value = query, - onValueChange = { query = it }, - label = { Text("Search HuggingFace") }, - singleLine = true, - modifier = Modifier.fillMaxWidth(), - ) - Spacer(Modifier.height(8.dp)) - TextButton( - enabled = query.isNotBlank(), - onClick = { - openRepo = null - results = LoadState.Loading - scope.launch { - results = - try { - withContext(Dispatchers.IO) { - LoadState.Loaded(searchModels(settings, query)) - } - } catch (e: ApiException) { - LoadState.failed(e) - } - } - }, - ) { - Text("Search") - } - - Spacer(Modifier.height(8.dp)) - LazyColumn(Modifier.fillMaxSize()) { - when (val current = state) { - is LoadState.Loading -> item { CircularProgressIndicator() } - is LoadState.Error -> - item { Text(current.message, color = MaterialTheme.colorScheme.error) } - is LoadState.Loaded -> { - if (current.value.downloads.isNotEmpty()) { - item { SectionLabel("Downloading") } - uniqueItems(current.value.downloads, key = { it.key + it.run }) { download - -> - DownloadCard(download) { - scope.launch { - actionError = - runCatching { - withContext(Dispatchers.IO) { - cancelDownload(settings, download.key) - } - } - .exceptionOrNull() - ?.message - } - } - } - } - item { SectionLabel("On the backend") } - if (current.value.local.isEmpty()) { - item { - Text( - "None yet. Search above to find one.", - style = MaterialTheme.typography.bodyMedium, - color = MaterialTheme.colorScheme.onSurfaceVariant, - ) - } - } - uniqueItems(current.value.local, key = { it.key }) { model -> - LocalModelCard(model) { - scope.launch { - actionError = - runCatching { - withContext(Dispatchers.IO) { - deleteModel(settings, model.key) - } - } - .exceptionOrNull() - ?.message - reload() - } - } - } - } - } - - results?.let { found -> - item { SectionLabel("HuggingFace") } - when (found) { - is LoadState.Loading -> item { CircularProgressIndicator() } - is LoadState.Error -> - item { Text(found.message, color = MaterialTheme.colorScheme.error) } - is LoadState.Loaded -> - uniqueItems(found.value, key = { it.id }) { repo -> - val open = openRepo == repo.id - RepoRow(repo, expanded = open) { - if (open) { - openRepo = null - } else { - openRepo = repo.id - repoFiles = LoadState.Loading - scope.launch { - repoFiles = - try { - withContext(Dispatchers.IO) { - LoadState.Loaded( - fetchRepoFiles(settings, repo.id) - ) - } - } catch (e: ApiException) { - LoadState.failed(e) - } - } - } - } - // Inside the expanded repository's own item rather than as a section - // after the list: drawn after every card, a repository's files read as - // belonging to whichever card happened to be last. - if (open) { - when (val files = repoFiles) { - null -> {} - is LoadState.Loading -> CircularProgressIndicator() - is LoadState.Error -> - Text(files.message, color = MaterialTheme.colorScheme.error) - is LoadState.Loaded -> - Column { - val busy = - (state as? LoadState.Loaded) - ?.value - ?.downloads - .orEmpty() - .filter { it.state == "running" } - .map { it.key } - .toSet() - files.value.forEach { file -> - RepoFileRow( - file, - downloading = "${repo.id}/${file.path}" in busy, - ) { - scope.launch { - actionError = - runCatching { - withContext(Dispatchers.IO) { - startDownload( - settings, - repo.id, - file.path, - ) - } - } - .exceptionOrNull() - ?.message - reload() - } - } - } - } - } - } - } - } - } - } - } -} - -@Composable -private fun SectionLabel(text: String) { - Spacer(Modifier.height(12.dp)) - Text(text, style = MaterialTheme.typography.titleSmall) - Spacer(Modifier.height(4.dp)) -} - -@Composable -private fun DownloadCard(download: Download, onCancel: () -> Unit) { - Card(Modifier.fillMaxWidth().padding(vertical = 4.dp)) { - Column(Modifier.padding(12.dp)) { - Text(download.file, style = MaterialTheme.typography.titleSmall) - Text( - download.repo, - style = MaterialTheme.typography.bodySmall, - color = MaterialTheme.colorScheme.onSurfaceVariant, - ) - Spacer(Modifier.height(8.dp)) - // A determinate bar only when the size is known. The server sends no total when it was - // never told one, and a bar drawn from a guess is worse than one that admits it is - // counting. - if (download.total != null && download.total > 0) { - LinearProgressIndicator( - progress = { download.done.toFloat() / download.total.toFloat() }, - // Blue at every value, unlike a quota bar: a download nearing its end is - // nearing success, and colouring it like a limit being approached would say the - // opposite. - color = progressColor, - modifier = Modifier.fillMaxWidth(), - ) - Text( - "${gigabytes(download.done)} of ${gigabytes(download.total)}", - style = MaterialTheme.typography.bodySmall, - ) - } else { - LinearProgressIndicator(color = progressColor, modifier = Modifier.fillMaxWidth()) - Text( - "${gigabytes(download.done)} so far, total size unknown", - style = MaterialTheme.typography.bodySmall, - ) - } - download.error?.let { - Text( - it, - style = MaterialTheme.typography.bodySmall, - color = MaterialTheme.colorScheme.error, - ) - } - Row { - Text( - download.state, - style = MaterialTheme.typography.bodySmall, - modifier = Modifier.weight(1f), - ) - if (download.state == "running") { - TextButton(onClick = onCancel) { Text("Cancel") } - } - } - } - } -} - -@Composable -private fun LocalModelCard(model: LocalModel, onDelete: () -> Unit) { - Card(Modifier.fillMaxWidth().padding(vertical = 4.dp)) { - Row(Modifier.padding(12.dp), verticalAlignment = Alignment.CenterVertically) { - Column(Modifier.weight(1f)) { - Text(model.file, style = MaterialTheme.typography.titleSmall) - Text( - "${model.repo} · ${gigabytes(model.bytes)}", - style = MaterialTheme.typography.bodySmall, - color = MaterialTheme.colorScheme.onSurfaceVariant, - ) - } - TextButton(onClick = onDelete) { Text("Delete") } - } - } -} - -@Composable -private fun RepoRow(repo: RemoteRepo, expanded: Boolean, onToggle: () -> Unit) { - Card(Modifier.fillMaxWidth().padding(vertical = 4.dp)) { - Row(Modifier.padding(12.dp), verticalAlignment = Alignment.CenterVertically) { - Column(Modifier.weight(1f)) { - Text( - repo.id, - style = MaterialTheme.typography.titleSmall, - maxLines = 1, - // The owner is the part that repeats; the model name at the end is what tells - // two entries apart. - overflow = TextOverflow.StartEllipsis, - ) - Text( - "${repo.downloads} downloads · ${repo.likes} likes", - style = MaterialTheme.typography.bodySmall, - color = MaterialTheme.colorScheme.onSurfaceVariant, - ) - } - TextButton(onClick = onToggle) { Text(if (expanded) "Hide" else "Files") } - } - } -} - -@Composable -private fun RepoFileRow(file: RemoteFile, downloading: Boolean, onDownload: () -> Unit) { - Row( - Modifier.fillMaxWidth().padding(start = 16.dp, top = 4.dp, bottom = 4.dp), - verticalAlignment = Alignment.CenterVertically, - ) { - Column(Modifier.weight(1f)) { - Text(file.path, style = MaterialTheme.typography.bodyMedium) - Text( - gigabytes(file.bytes), - style = MaterialTheme.typography.bodySmall, - color = MaterialTheme.colorScheme.onSurfaceVariant, - ) - } - // Disabled rather than absent, so the row reads the same whether this one is absent, - // already here, or on its way. Offering "Download" for a file that is downloading would be - // a button that does nothing anyone can see. - TextButton(enabled = !file.have && !downloading, onClick = onDownload) { - Text( - when { - file.have -> "Downloaded" - downloading -> "Downloading" - else -> "Download" - } - ) - } - } -} - -private fun gigabytes(bytes: Long): String = - if (bytes >= 1_000_000_000) { - "%.2f GB".format(bytes / 1_000_000_000.0) - } else { - "%.0f MB".format(bytes / 1_000_000.0) - } diff --git a/app/androidApp/src/main/kotlin/com/example/aiapp/ProviderScreen.kt b/app/androidApp/src/main/kotlin/com/example/aiapp/ProviderScreen.kt index 11bced5..14a2372 100644 --- a/app/androidApp/src/main/kotlin/com/example/aiapp/ProviderScreen.kt +++ b/app/androidApp/src/main/kotlin/com/example/aiapp/ProviderScreen.kt @@ -68,6 +68,21 @@ fun ProviderScreen( // when it did. Both here rather than per row: these act on the whole machine. var busy by remember { mutableStateOf(null) } var actionError by remember { mutableStateOf(null) } + var confirmingDelete by remember { mutableStateOf(null) } + + // The machine's own models and what is being fetched onto it. Only for a provider that serves + // files off that machine's disk -- everything else names its models rather than holding them, + // and a search for a GGUF under the Claude CLI would be an offer that leads nowhere. + val kind = (state as? LoadState.Loaded)?.value?.kind + val machineModels = + rememberMachineModels( + settings = settings, + machineId = machineId, + enabled = kind == "llama_cpp", + // A download that became a model is a model this screen has no settings for yet, so + // the view it is drawing is now one model short of the truth. + onLocalChange = { reload++ }, + ) LaunchedEffect(reload) { state = @@ -93,7 +108,9 @@ fun ProviderScreen( Unit } - Column(Modifier.fillMaxSize().padding(16.dp)) { + // The models search at the bottom takes the keyboard, and everything below the field it is + // typed in -- the Search button, the results -- is behind it without this. + Column(Modifier.fillMaxSize().imePadding().padding(16.dp)) { Row(verticalAlignment = Alignment.CenterVertically, modifier = Modifier.fillMaxWidth()) { TextButton(onClick = onBack) { Text("Back") } } @@ -163,10 +180,21 @@ fun ProviderScreen( Spacer(Modifier.height(8.dp)) } } + machineModels.actionError?.let { failure -> + item("models-error") { + Text(failure, color = MaterialTheme.colorScheme.error) + } + } + // Above the models: this is what is about to be one of them. + downloadCards(machineModels) + val sizes = machineModels.sizes uniqueItems(view.models, key = { it.id }) { model -> ModelCard( model = model, specs = view.modelParams, + bytes = sizes[model.id], + onDelete = + if (model.id in sizes) ({ confirmingDelete = model }) else null, // Tapping opens the settings; a provider whose models take none has // nothing to open, so the row is not a control. onEdit = @@ -187,6 +215,7 @@ fun ProviderScreen( enabled = busy == null, ) } + if (kind == "llama_cpp") modelSearch(machineModels) if (view.mcpServers.isNotEmpty()) { item("mcp") { Spacer(Modifier.height(12.dp)) @@ -219,6 +248,33 @@ fun ProviderScreen( ) } + confirmingDelete?.let { model -> + AlertDialog( + onDismissRequest = { confirmingDelete = null }, + title = { Text("Delete ${model.label}?") }, + text = { + Text( + "The file is removed from ${(state as? LoadState.Loaded)?.value?.machine ?: "this machine"}. " + + "Nothing here can get it back -- downloading it again is the whole file again. " + + "Sessions using it keep their conversations and cannot start it." + ) + }, + confirmButton = { + TextButton( + onClick = { + confirmingDelete = null + machineModels.remove(model.id) + } + ) { + Text("Delete") + } + }, + dismissButton = { + TextButton(onClick = { confirmingDelete = null }) { Text("Cancel") } + }, + ) + } + if (confirmingStop) { AlertDialog( onDismissRequest = { confirmingStop = false }, @@ -311,8 +367,11 @@ private fun ServerCard( private fun ModelCard( model: ProviderModel, specs: List, + /** How big the file is on the machine, for a provider whose models are files. */ + bytes: Long?, onEdit: (() -> Unit)?, onUnload: (() -> Unit)?, + onDelete: (() -> Unit)?, enabled: Boolean, ) { Card( @@ -321,7 +380,20 @@ private fun ModelCard( .then(if (onEdit != null && enabled) Modifier.clickable(onClick = onEdit) else Modifier) ) { Column(Modifier.padding(12.dp)) { - Text(model.label, style = MaterialTheme.typography.bodyMedium) + Row(verticalAlignment = Alignment.CenterVertically) { + Text( + model.label, + style = MaterialTheme.typography.bodyMedium, + modifier = Modifier.weight(1f), + ) + bytes?.let { + Text( + gigabytes(it), + style = MaterialTheme.typography.bodySmall, + color = MaterialTheme.colorScheme.onSurfaceVariant, + ) + } + } // What the server is doing with it, in its own word. Absent means nobody could ask -- // the server is not running -- and the line is left out rather than guessed at. model.status?.let { @@ -345,8 +417,15 @@ private fun ModelCard( color = MaterialTheme.colorScheme.onSurfaceVariant, ) } - if (onUnload != null) { - Row { TextButton(enabled = enabled, onClick = onUnload) { Text("Unload") } } + if (onUnload != null || onDelete != null) { + Row(verticalAlignment = Alignment.CenterVertically) { + // Both shown whenever this kind of model has them, disabled rather than + // absent: unloading frees memory and deleting frees disk, and a button that + // comes and goes makes its own absence the message. + onUnload?.let { TextButton(enabled = enabled, onClick = it) { Text("Unload") } } + Spacer(Modifier.weight(1f)) + onDelete?.let { TextButton(enabled = enabled, onClick = it) { Text("Delete") } } + } } } } diff --git a/app/androidApp/src/main/kotlin/com/example/aiapp/RawBlock.kt b/app/androidApp/src/main/kotlin/com/example/aiapp/RawBlock.kt index 2b3df2c..c59bc14 100644 --- a/app/androidApp/src/main/kotlin/com/example/aiapp/RawBlock.kt +++ b/app/androidApp/src/main/kotlin/com/example/aiapp/RawBlock.kt @@ -1,10 +1,12 @@ package com.example.aiapp import androidx.compose.foundation.background +import androidx.compose.foundation.horizontalScroll import androidx.compose.foundation.layout.Column import androidx.compose.foundation.layout.ColumnScope import androidx.compose.foundation.layout.fillMaxWidth import androidx.compose.foundation.layout.padding +import androidx.compose.foundation.rememberScrollState import androidx.compose.material3.MaterialTheme import androidx.compose.runtime.Composable import androidx.compose.ui.Modifier @@ -18,6 +20,14 @@ import androidx.compose.ui.unit.dp * monospace text drawn hard against the edge of a tinted block reads as a clipping fault, and three * copies of "clip, fill, pad" drift apart the first time one is adjusted. * + * **Nothing in here wraps; it scrolls sideways instead.** This is column-aligned far more often + * than it is prose -- a diff, a table, a test run, a command and its arguments -- and wrapping + * destroys exactly the alignment that was carrying the meaning, while turning one line into four + * and a run of them into a wall. The scroll belongs to the block rather than to each line so that + * the lines stay aligned with each other as it moves: one offset for the whole column is what makes + * a shifted diff still read as a diff. Every [Text] inside is therefore drawn with `softWrap = + * false`, which is the half of this a caller has to remember. + * * The colour is [rawSurface], which is also what a code block inside a reply is given. */ @Composable @@ -29,6 +39,10 @@ fun RawBlock(modifier: Modifier = Modifier, content: @Composable ColumnScope.() // rectangle drawn at the same radius as the one behind it reads as a misprint. .clip(MaterialTheme.shapes.extraSmall) .background(rawSurface) + // Clipped and filled before this, so the tint is the viewport and does not scroll away + // from under the text; padded after it, so the inset travels with the content and the + // last column does not end flush against the edge. + .horizontalScroll(rememberScrollState()) .padding(horizontal = 8.dp, vertical = 6.dp), content = content, ) diff --git a/app/androidApp/src/main/kotlin/com/example/aiapp/ToolInput.kt b/app/androidApp/src/main/kotlin/com/example/aiapp/ToolInput.kt index fe23274..0a2cc5e 100644 --- a/app/androidApp/src/main/kotlin/com/example/aiapp/ToolInput.kt +++ b/app/androidApp/src/main/kotlin/com/example/aiapp/ToolInput.kt @@ -1,9 +1,6 @@ package com.example.aiapp -import androidx.compose.foundation.horizontalScroll -import androidx.compose.foundation.layout.fillMaxWidth import androidx.compose.foundation.layout.padding -import androidx.compose.foundation.rememberScrollState import androidx.compose.material3.MaterialTheme import androidx.compose.material3.Text import androidx.compose.runtime.Composable @@ -150,7 +147,8 @@ fun ToolInputView(tool: String, input: String, modifier: Modifier = Modifier) { RawBlock(modifier) { parsed.subject?.let { subject -> // Not wrapped: a wrapped command hides where its arguments end, and the long one is the - // one being read closely. + // one being read closely. The sideways scroll that makes that readable is the block's, + // shared with the lines below -- see [RawBlock]. Text( // Not cached: a tool's subject is one command line, which lexes in microseconds -- // the cache exists for a fence with two hundred lines in it. @@ -158,7 +156,6 @@ fun ToolInputView(tool: String, input: String, modifier: Modifier = Modifier) { style = MaterialTheme.typography.bodySmall, fontFamily = FontFamily.Monospace, softWrap = false, - modifier = Modifier.fillMaxWidth().horizontalScroll(rememberScrollState()), ) } parsed.rest.forEach { @@ -167,6 +164,7 @@ fun ToolInputView(tool: String, input: String, modifier: Modifier = Modifier) { style = MaterialTheme.typography.bodySmall, fontFamily = FontFamily.Monospace, color = MaterialTheme.colorScheme.onSurfaceVariant, + softWrap = false, modifier = Modifier.padding(top = 2.dp), ) } diff --git a/app/androidApp/src/main/kotlin/com/example/aiapp/ToolRows.kt b/app/androidApp/src/main/kotlin/com/example/aiapp/ToolRows.kt index 105ba63..8caf121 100644 --- a/app/androidApp/src/main/kotlin/com/example/aiapp/ToolRows.kt +++ b/app/androidApp/src/main/kotlin/com/example/aiapp/ToolRows.kt @@ -431,7 +431,8 @@ fun ToolCard( // What the tool printed, on the surface everything verbatim gets and in the // face it was written for: this is column-aligned far more often than it is // prose, and a proportional font silently destroys the alignment that carried - // the meaning. + // the meaning. Unwrapped for the same reason, and scrolled sideways by the + // block around it -- see [RawBlock]. // // Its terminal styling applied and the rest of the escapes taken out: colour is // often the whole of what a diff or a test run is saying. Remembered against @@ -443,6 +444,7 @@ fun ToolCard( styled, style = MaterialTheme.typography.bodySmall, fontFamily = FontFamily.Monospace, + softWrap = false, ) } } diff --git a/server/src/main.rs b/server/src/main.rs index ec651a0..d02bd49 100644 --- a/server/src/main.rs +++ b/server/src/main.rs @@ -175,7 +175,6 @@ async fn main() -> Result<()> { let models_dir = args .models_dir .unwrap_or_else(|| data_home("ai-app").join("models")); - let models = Arc::new(models::ModelStore::new(models_dir.clone())); let manager = Arc::new( SessionManager::new(config_path.clone(), data_dir, models_dir.clone()) .with_context(|| format!("failed to load {}", config_path.display()))? @@ -290,7 +289,6 @@ async fn main() -> Result<()> { Arc::clone(&provider_logins), Arc::clone(&manager), )) - .merge(routes::models_router(Arc::clone(&models))) .layer(axum::middleware::from_fn_with_state( Arc::clone(&manager), auth::require_token, diff --git a/server/src/models.rs b/server/src/models.rs index 5278988..11f59c8 100644 --- a/server/src/models.rs +++ b/server/src/models.rs @@ -1,46 +1,45 @@ -//! GGUF models on this machine, and the downloads that produce them. +//! GGUF models on the machines this backend can run them on, and the +//! downloads that produce them. //! -//! The registry pattern again: one owner, one lock, so what is on disk and what -//! this server believes cannot come apart. Three things shape the design, all -//! consequences of a model file being gigabytes rather than kilobytes: +//! **A model file belongs to the machine that will serve it.** Everything +//! here is addressed by a transport and a directory rather than by this +//! process's own disk, because `llama-server` reads the file where it runs +//! and a list or a download naming anywhere else is a claim about the wrong +//! filesystem. //! -//! **A download belongs to the model, not to whoever asked for it.** It is -//! keyed by the model it produces and lives here, so any device can watch it -- -//! including one that did not start it. State in a per-connection channel would -//! not survive the phone locking its screen, which for an hour-long download is -//! the normal case. +//! **So a download runs there too** (2026-09-19, replacing the fetch this +//! server used to perform itself onto its own disk). It is a detached `curl` +//! on that machine, started by a script written here and never spoken to +//! again. Three things follow, and all three are why it is shaped this way: //! -//! **Every run has an id, and its outcome outlives it.** Without those, "not -//! downloading" is three answers at once -- it finished, it never started, or a -//! different run finished while you were away. -//! -//! **Progress is measured, never estimated.** `total` is whatever -//! `Content-Length` said and nothing else; when the server does not send one it -//! stays `None` and the phone shows that it does not know. +//! - **Its state is a file beside the model**, `x.gguf.download`, written by +//! that script at each step. Nothing about it is held in this process, so +//! one download survives the phone locking, this backend restarting, and +//! being watched from a second device -- and what is on that disk is the +//! answer rather than something this server remembers about it. +//! - **Progress is measured, never estimated.** The bytes are `wc -c` of the +//! partial file and the total is the size HuggingFace published for it. A +//! download whose process is gone is reported failed rather than left +//! saying "running" for ever, which is what `kill -0` at each listing is +//! for. +//! - **Resume is guarded by identity.** The partial records the sha256 the +//! file is meant to have; one written against a different sha is discarded +//! rather than resumed onto, and the finished file is checked against it +//! before it takes the real name. use std::collections::HashMap; -use std::io::{Read, Seek, SeekFrom, Write}; -use std::path::{Path, PathBuf}; -use std::sync::atomic::{AtomicBool, AtomicU64, Ordering}; -use std::sync::{Arc, Mutex}; +use std::path::Path; -use anyhow::{Context, Result, bail}; +use anyhow::{Context, Result}; use serde::Serialize; -use wg_app_link::private; - use crate::session::transport::{Launch, Transport}; /// Identifies this client to HuggingFace. They ask for one, and a request /// without it is more likely to be rate-limited. const USER_AGENT: &str = concat!("ai-server/", env!("CARGO_PKG_VERSION")); -/// Read size per loop iteration. Big enough that the syscall overhead is -/// nothing against a multi-gigabyte file, small enough that a cancel is noticed -/// promptly -- the flag is only checked between chunks. -const CHUNK: usize = 256 * 1024; - -/// A model file sitting on this machine, ready to run. +/// A model file sitting on a machine, ready to run there. #[derive(Debug, Clone, Serialize)] #[serde(rename_all = "camelCase")] pub struct LocalModel { @@ -107,486 +106,367 @@ pub fn labels(models: &[LocalModel]) -> Vec { .collect() } -/// The model's own name, read out of the file itself. +/// What a download is doing, or did. /// -/// Absent for every way of not finding out -- see [`crate::gguf`]. The file is -/// opened and read only as far as the name, which is the first few hundred -/// bytes, so this is affordable once per model per listing. -fn name_of(path: &Path) -> Option { - let mut file = std::fs::File::open(path).ok()?; - crate::gguf::name(&mut file) -} - -/// What a run is doing, or did. Flat rather than a tagged enum carrying its -/// message, because the phone switches on this and a string it can compare is -/// easier to render than a variant it has to destructure. +/// Flat rather than a tagged enum carrying its message, because the phone +/// switches on this and a word it can compare is easier to render than a +/// variant it has to destructure. There is no `finished`: a download that +/// finished is a model, and it is in the list beside this one -- an outcome +/// kept here as well would be the same fact stated twice, in two places that +/// could disagree. #[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize)] #[serde(rename_all = "snake_case")] pub enum DownloadState { Running, - /// Reading the finished file back to check it against the hash HuggingFace - /// publishes. Its own state because it takes real time on a multi-gigabyte - /// file and "still working" is the honest thing to show, rather than a bar - /// sitting at 100% for half a minute. + /// Reading the finished file back to check it against the hash + /// HuggingFace publishes. Its own state because it takes real time on a + /// multi-gigabyte file, and "still working" is the honest thing to show + /// where a bar sitting at 100% for half a minute is not. Verifying, - Finished, Failed, Cancelled, } -/// One download run, as the phone sees it. +impl DownloadState { + fn from_word(word: &str) -> Option { + match word { + "running" => Some(Self::Running), + "verifying" => Some(Self::Verifying), + "failed" => Some(Self::Failed), + "cancelled" => Some(Self::Cancelled), + _ => None, + } + } +} + +/// One download on one machine, as the phone sees it. #[derive(Debug, Clone, Serialize)] #[serde(rename_all = "camelCase")] -pub struct DownloadStatus { +pub struct Download { + /// `owner/repo/file.gguf`, the same key the finished model will have. + /// One download per model at a time, so this identifies it. pub key: String, - /// Distinguishes this run from any earlier one for the same model, so a - /// device that was watching run 3 can tell it is now looking at run 4. - pub run: u64, pub repo: String, pub file: String, pub state: DownloadState, pub done: u64, - /// What `Content-Length` said, or absent when the server did not say. - /// Absent means "unknown", never "zero". + /// What HuggingFace published as the file's size, or absent when it + /// published none. Absent means "not known", never "zero". #[serde(skip_serializing_if = "Option::is_none")] pub total: Option, #[serde(skip_serializing_if = "Option::is_none")] pub error: Option, - pub started: f64, - #[serde(skip_serializing_if = "Option::is_none")] - pub finished: Option, } -/// The mutable half of a run, behind one lock. -#[derive(Debug)] -struct Progress { - state: DownloadState, - done: u64, - total: Option, - error: Option, - started: f64, - finished: Option, +/// What HuggingFace says about one file before anything is fetched: how big +/// it is, and what it should hash to. +/// +/// Both read here rather than on the machine doing the download, because this +/// is the side with internet trust to spend (see [`search`]) and because the +/// sha is what makes a resume safe -- the machine is handed the answer rather +/// than asked to find it. +#[derive(Debug, Clone)] +pub struct Published { + pub bytes: Option, + /// The LFS object id, which for these repositories is the sha256 of the + /// content -- a free integrity check rather than a second source of + /// truth. Absent for a repository that publishes none, and then the file + /// is not verified and a partial is never resumed onto. + pub sha256: Option, } -/// A run, shared between the thread doing the work and everyone watching. -struct Run { - id: u64, - key: String, - repo: String, - file: String, - progress: Mutex, - /// Set by [`ModelStore::cancel`]; the download loop checks it between - /// chunks and stops, leaving the partial file for a later resume. - cancel: AtomicBool, -} - -impl Run { - fn status(&self) -> DownloadStatus { - let p = self.progress.lock().unwrap(); - DownloadStatus { - key: self.key.clone(), - run: self.id, - repo: self.repo.clone(), - file: self.file.clone(), - state: p.state, - done: p.done, - total: p.total, - error: p.error.clone(), - started: p.started, - finished: p.finished, +/// Refuses a key that is anything but `owner/repo/file.gguf`. +/// +/// The key comes from a phone, so it is treated as hostile: it reaches a +/// shell as a positional argument, which is what stops it being read as a +/// command, but nothing there stops `../..` from naming a file outside the +/// models directory. Rejected rather than rewritten, since a silently +/// corrected path downloads the right bytes to the wrong place. +fn checked(key: &str) -> Result<()> { + let bad = |why: &str| anyhow::anyhow!("\"{key}\" is not a model this can store: {why}"); + for part in key.split('/') { + if part.is_empty() || part == "." || part == ".." { + return Err(bad( + "a path component that would leave the models directory", + )); + } + if part.contains('\\') || part.chars().any(char::is_control) { + return Err(bad("a path component with a character no file here has")); } } - - fn finish(&self, state: DownloadState, error: Option) { - let mut p = self.progress.lock().unwrap(); - p.state = state; - p.error = error; - p.finished = Some(crate::session::now()); - } -} - -/// Every model this machine has, and every download in flight or finished. -pub struct ModelStore { - dir: PathBuf, - /// Keyed by model key: one run per model at a time, and the last run for a - /// model stays here after it ends so its outcome can still be read. - runs: Mutex>>, - next_run: AtomicU64, -} - -impl ModelStore { - pub fn new(dir: PathBuf) -> Self { - Self { - dir, - runs: Mutex::new(HashMap::new()), - next_run: AtomicU64::new(1), - } - } - - /// Where a model's file lives, refusing anything that would escape the - /// models directory. - /// - /// The repo and file come from a phone, so they are treated as hostile: - /// every component must be an ordinary name. Rejecting rather than - /// sanitising, since a silently rewritten path would download the right - /// bytes to the wrong place. - fn path_for(&self, repo: &str, file: &str) -> Result { - let mut path = self.dir.clone(); - for part in repo.split('/').chain(file.split('/')) { - if part.is_empty() || part == "." || part == ".." || part.contains('\\') { - bail!("\"{repo}/{file}\" is not a name this can store: \"{part}\""); - } - path.push(part); - } - Ok(path) - } - - pub fn key_for(repo: &str, file: &str) -> String { - format!("{repo}/{file}") - } - - /// Every `.gguf` found under the models directory, newest first. Read from - /// disk on each call rather than cached: a file deleted by hand should stop - /// being offered. - pub fn list(&self) -> Vec { - let mut found = Vec::new(); - collect(&self.dir, &self.dir, &mut found); - found.sort_by(|a, b| a.key.cmp(&b.key)); - found - } - - /// The status of every run this server remembers. - pub fn downloads(&self) -> Vec { - let runs = self.runs.lock().unwrap(); - let mut all: Vec<_> = runs.values().map(|run| run.status()).collect(); - all.sort_by_key(|status| std::cmp::Reverse(status.run)); - all - } - - /// Starts fetching `file` from `repo`, or returns the run already doing so. - /// Idempotent on purpose: a phone that lost its connection will press the - /// button again, and that must join the existing run rather than start a - /// second one writing the same file. - pub fn start(self: &Arc, repo: &str, file: &str) -> Result { - let key = Self::key_for(repo, file); - let target = self.path_for(repo, file)?; - if target.is_file() { - bail!("{key} is already downloaded"); - } - - let mut runs = self.runs.lock().unwrap(); - if let Some(existing) = runs.get(&key) - && existing.progress.lock().unwrap().state == DownloadState::Running - { - return Ok(existing.status()); - } - - let run = Arc::new(Run { - id: self.next_run.fetch_add(1, Ordering::Relaxed), - key: key.clone(), - repo: repo.to_string(), - file: file.to_string(), - progress: Mutex::new(Progress { - state: DownloadState::Running, - done: 0, - total: None, - error: None, - started: crate::session::now(), - finished: None, - }), - cancel: AtomicBool::new(false), - }); - runs.insert(key, Arc::clone(&run)); - let status = run.status(); - drop(runs); - - // A dedicated thread rather than the blocking pool: this holds its - // thread for as long as the download takes, which is minutes to hours, - // and the pool exists for short work. - let store = Arc::clone(self); - std::thread::spawn(move || { - let outcome = store.fetch(&run, &target); - match outcome { - Ok(()) if run.cancel.load(Ordering::Relaxed) => { - run.finish(DownloadState::Cancelled, None); - tracing::info!("download {} cancelled", run.key); - } - Ok(()) => { - run.finish(DownloadState::Finished, None); - tracing::info!("download {} finished", run.key); - } - Err(err) => { - let message = format!("{err:#}"); - tracing::warn!("download {} failed: {message}", run.key); - run.finish(DownloadState::Failed, Some(message)); - } - } - }); - Ok(status) - } - - /// Asks a running download to stop. The partial file stays, so starting - /// again resumes rather than refetching. - pub fn cancel(&self, key: &str) -> Result { - let runs = self.runs.lock().unwrap(); - let Some(run) = runs.get(key) else { - bail!("no download for {key}"); - }; - run.cancel.store(true, Ordering::Relaxed); - Ok(run.status()) - } - - /// Removes a downloaded model, and any partial file for it. - pub fn delete(&self, key: &str) -> Result<()> { - let (repo, file) = key.rsplit_once('/').context("a key is repo/file")?; - let target = self.path_for(repo, file)?; - let partial = partial_of(&target); - if !target.is_file() && !partial.is_file() { - bail!("{key} is not downloaded"); - } - for path in [&target, &partial] { - if path.is_file() { - std::fs::remove_file(path).with_context(|| format!("remove {}", path.display()))?; - } - } - self.runs.lock().unwrap().remove(key); - Ok(()) - } - - fn fetch(&self, run: &Run, target: &Path) -> Result<()> { - let partial = partial_of(target); - let identity = identity_of(target); - if let Some(parent) = target.parent() { - private::create_dir(parent)?; - } - - // What we have, and what it was part of. A partial with no recorded - // identity is not resumable -- it could be a fragment of any revision. - let known = std::fs::read_to_string(&identity) - .ok() - .map(|s| s.trim().to_string()); - let have = match known { - Some(_) => partial.metadata().map(|m| m.len()).unwrap_or(0), - None => 0, - }; - - let url = format!( - "https://huggingface.co/{}/resolve/main/{}", - run.repo, - run.file.replace(' ', "%20") - ); - let (mut response, mut resumed) = request(&url, have)?; - let mut etag = etag_of(&response); - - // HuggingFace's CDN ignores `If-Range` -- probed 2026-08-28: a - // deliberately stale validator still answers 206 with the ranged bytes. - // So the header cannot be relied on to restart us, and the check is done - // here instead: if what arrived is not the revision our partial belongs - // to, resuming would splice two files into something of exactly the - // right length and the wrong contents. - if resumed && etag.is_some() && etag != known { - tracing::info!( - "{} changed upstream since the partial was written -- starting again", - run.key, - ); - let (fresh, fresh_resumed) = request(&url, 0)?; - response = fresh; - resumed = fresh_resumed; - etag = etag_of(&response); - } - - // On a 206, Content-Length is the length of the *range*, not of the file - // -- taken at face value it would fill the bar at 72 MB of a 234 MB - // model. The whole size is the last field of Content-Range, which has - // the further merit of not depending on where the range began. - let total: Option = if resumed { - response - .headers() - .get("content-range") - .and_then(|v| v.to_str().ok()) - .and_then(|v| { - v.rsplit_once('/') - .map(|(_, whole)| whole.trim().to_string()) - }) - .and_then(|whole| whole.parse().ok()) - } else { - response - .headers() - .get("content-length") - .and_then(|v| v.to_str().ok()?.parse().ok()) - }; - let mut done = if resumed { have } else { 0 }; - { - let mut p = run.progress.lock().unwrap(); - p.done = done; - p.total = total; - } - - // `truncate(false)` is the whole resume story: the file is opened to be - // seeked into and appended to, and truncating would throw away exactly - // the bytes the Range request just asked the server not to send again. - // Stated rather than left to the default. - let mut file = std::fs::OpenOptions::new() - .create(true) - .write(true) - .truncate(false) - .open(&partial) - .with_context(|| format!("open {}", partial.display()))?; - if resumed { - file.seek(SeekFrom::Start(have)) - .context("seek to resume point")?; - } else { - file.set_len(0) - .context("truncate a partial we cannot resume onto")?; - } - // Written before the body, so an interrupted download leaves a partial - // that can still say which revision it belongs to. - if let Some(etag) = &etag { - std::fs::write(&identity, etag).ok(); - } - - let mut reader = response.body_mut().as_reader(); - let mut buffer = vec![0u8; CHUNK]; - loop { - if run.cancel.load(Ordering::Relaxed) { - file.flush().ok(); - return Ok(()); - } - let read = reader - .read(&mut buffer) - .context("reading from HuggingFace")?; - if read == 0 { - break; - } - file.write_all(&buffer[..read]) - .context("writing the model file")?; - done += read as u64; - run.progress.lock().unwrap().done = done; - } - file.flush().context("flushing the model file")?; - drop(file); - - // Checked before the rename, so a file that fails never gets the real - // name and `list` never offers it. With the identity check above this - // should not fire; it is here because a wrong model is the kind of - // failure that surfaces as bad output rather than as an error. - if let Some(expected) = published_sha256(&run.repo, &run.file) { - run.progress.lock().unwrap().state = DownloadState::Verifying; - let actual = sha256_of(&partial)?; - if actual != expected { - std::fs::remove_file(&partial).ok(); - std::fs::remove_file(&identity).ok(); - bail!( - "{} arrived corrupted -- HuggingFace publishes sha256 {expected}, what \ - arrived hashes to {actual}. It has been deleted; downloading again \ - starts clean.", - run.key, - ); - } - } - - // Renamed only once complete, so a file at its real name is always a - // whole model -- `list` needs no other way to tell. - std::fs::rename(&partial, target) - .with_context(|| format!("finish {}", target.display()))?; - std::fs::remove_file(&identity).ok(); - Ok(()) - } -} - -/// The sha256 of a file, read in chunks -- these are gigabytes, and reading one -/// into memory to hash it would be the largest allocation this server makes. -fn sha256_of(path: &Path) -> Result { - use sha2::{Digest, Sha256}; - let mut file = - std::fs::File::open(path).with_context(|| format!("reopen {}", path.display()))?; - let mut hasher = Sha256::new(); - let mut buffer = vec![0u8; CHUNK]; - loop { - let read = file.read(&mut buffer).context("reading back to verify")?; - if read == 0 { - break; - } - hasher.update(&buffer[..read]); - } - // Hex by hand, as `wg_app_link::enroll::token_hash_hex` also has to, since - // this sha2 version's output type does not implement LowerHex. - Ok(hasher - .finalize() - .iter() - .map(|byte| format!("{byte:02x}")) - .collect()) -} - -/// One GET, ranged when there is something to resume onto. -fn request(url: &str, from: u64) -> Result<(ureq::http::Response, bool)> { - let mut get = ureq::get(url).header("User-Agent", USER_AGENT); - if from > 0 { - get = get.header("Range", &format!("bytes={from}-")); - } - let response = get.call().with_context(|| format!("GET {url}"))?; - // Trust the status, not the request: a server that ignores Range answers - // 200 with the whole file, and appending to that would corrupt it. - let resumed = response.status() == 206; - Ok((response, resumed)) -} - -fn etag_of(response: &ureq::http::Response) -> Option { - Some( - response - .headers() - .get("etag")? - .to_str() - .ok()? - .trim() - .to_string(), - ) -} - -/// `x.gguf` -> `x.gguf.part.etag`, holding which revision the partial beside it -/// is a piece of. -fn identity_of(target: &Path) -> PathBuf { - let mut name = target.as_os_str().to_os_string(); - name.push(".part.etag"); - PathBuf::from(name) -} - -/// `x.gguf` -> `x.gguf.part`, the in-progress name. -fn partial_of(target: &Path) -> PathBuf { - let mut name = target.as_os_str().to_os_string(); - name.push(".part"); - PathBuf::from(name) -} - -/// Walks `dir` collecting `.gguf` files, keyed by their path under `root`. -fn collect(root: &Path, dir: &Path, found: &mut Vec) { - let Ok(entries) = std::fs::read_dir(dir) else { - return; + let Some((_, file)) = key.rsplit_once('/') else { + return Err(bad("a key is owner/repo/file.gguf")); }; - for entry in entries.flatten() { - let path = entry.path(); - if path.is_dir() { - collect(root, &path, found); - continue; - } - if path.extension().is_none_or(|e| e != "gguf") { - continue; - } - let Ok(relative) = path.strip_prefix(root) else { - continue; - }; - let key = relative.to_string_lossy().replace('\\', "/"); - let Some((repo, file)) = key.rsplit_once('/') else { - continue; - }; - found.push(LocalModel { - key: key.clone(), - repo: repo.to_string(), - file: file.to_string(), - bytes: entry.metadata().map(|m| m.len()).unwrap_or(0), - name: name_of(&path), - }); + if !file.ends_with(".gguf") { + return Err(bad("only .gguf files are models")); } + Ok(()) +} + +pub fn key_for(repo: &str, file: &str) -> String { + format!("{repo}/{file}") +} + +/// Expands a leading `~` the way that machine's own shell would, which is the +/// only place that knows what it stands for. Prefixed to every script here; +/// each takes the directory as `$1`. +const EXPAND: &str = "d=$1; case $d in \"~\") d=$HOME;; \"~/\"*) d=$HOME/${d#\"~/\"};; esac;"; + +/// The download itself, run detached on the machine that will hold the file. +/// +/// Passed to [`START`] as an argument rather than written inside it, so that +/// neither script has to quote the other: `sh -c "$worker"` takes this whole +/// text as one string the outer shell never parses. +/// +/// `-sS` rather than plain: curl's progress meter goes to stderr, and the +/// same file is what a failure is read out of -- a message buried in three +/// screens of redrawn bar is one the phone cannot show. Progress is read off +/// the partial file instead, which is the measurement rather than a report +/// of it. +/// +/// The state file is written whole and moved into place, because a listing +/// can arrive in the middle of any of these writes and a truncated one is a +/// download that blinks out of the phone's list for a poll. +/// +/// `curl` is waited on rather than run in the foreground, because a cancel +/// arrives as a signal and a shell blocked in `wait` is the one that can run +/// a trap for it. The trap writes nothing when the partial has already gone: +/// that is how [`REMOVE`] deletes a running download without the dying worker +/// recreating the state file behind it. +const WORKER: &str = r#" +t=$1; url=$2; sha=$3; total=$4; p=$t.part; s=$t.download; e=$t.part.err +st() { printf 'pid=%s\nsha=%s\ntotal=%s\nstate=%s\nerror=%s\n' "$$" "$sha" "$total" "$1" "$2" > "$s.tmp" && mv "$s.tmp" "$s"; } +trap '[ -n "$c" ] && kill "$c" 2>/dev/null; [ -f "$p" ] && st cancelled ""; exit 0' TERM INT +c= +st running "" +have=$(wc -c 2>/dev/null < "$p" | tr -d " ") +if [ -z "$total" ] || [ "$have" != "$total" ]; then + curl -fsSL --retry 3 --retry-delay 2 -C - -o "$p" "$url" 2> "$e" & + c=$! + if ! wait "$c"; then + st failed "$(tr -s '\n\t\r' ' ' < "$e" | tail -c 300)" + exit 1 + fi +fi +rm -f "$e" +if [ -n "$sha" ] && command -v sha256sum > /dev/null 2>&1; then + st verifying "" + got=$(sha256sum "$p" | cut -d' ' -f1) + if [ "$got" != "$sha" ]; then + rm -f "$p" + st failed "it arrived corrupted -- HuggingFace publishes sha256 $sha, what arrived hashes to $got. It has been deleted; downloading again starts clean." + exit 1 + fi +fi +mv "$p" "$t" && rm -f "$s" +"#; + +/// Starts [`WORKER`], or leaves alone whatever is already happening. +/// +/// Idempotent on purpose, in both directions: a phone that lost its +/// connection presses the button again, and that must join the run already +/// going rather than start a second one writing the same file. A model +/// already downloaded is likewise the answer rather than an error, which is +/// the ordinary case once two devices can both see the list. +const START: &str = r#" +k=$2; url=$3; sha=$4; total=$5; worker=$6 +t=$d/$k; s=$t.download +[ -f "$t" ] && exit 0 +pid=$(sed -n 's/^pid=//p' "$s" 2>/dev/null) +if [ -n "$pid" ] && kill -0 "$pid" 2>/dev/null; then exit 0; fi +command -v curl > /dev/null 2>&1 || { echo "there is no curl on this machine to download with" >&2; exit 1; } +mkdir -p "${t%/*}" || { echo "cannot make ${t%/*} to download into" >&2; exit 1; } +had=$(sed -n 's/^sha=//p' "$s" 2>/dev/null) +if [ -z "$sha" ] || [ "$had" != "$sha" ]; then rm -f "$t.part"; fi +nohup sh -c "$worker" sh "$t" "$url" "$sha" "$total" > /dev/null 2>&1 & +"#; + +/// Every download the machine is holding state for, with the bytes read off +/// the partial file rather than from anything remembered here. +/// +/// `kill -0` is why a stopped machine does not leave a download claiming to +/// be running for ever: a state file whose process is gone is the failure it +/// actually is. +/// +/// A state file beside a finished model is skipped, because the file is the +/// truth and the state file is only ever a report about producing it. That is +/// also what keeps the one race here self-correcting: two devices asking for +/// the same model in the same second get two workers writing one partial, +/// whose sha will not match -- and whichever of them loses the rename would +/// otherwise leave a failure recorded against a model that is sitting there. +const LIST: &str = r#" +[ -d "$d" ] || exit 0 +cd "$d" || exit 0 +find . -type f -name '*.download' | while read -r f; do + k=${f#./}; k=${k%.download} + [ -f "$k" ] && continue + size=$(wc -c 2>/dev/null < "$k.part" | tr -d " ") + [ -n "$size" ] || size=0 + pid=$(sed -n 's/^pid=//p' "$f") + alive=no + if [ -n "$pid" ] && kill -0 "$pid" 2>/dev/null; then alive=yes; fi + printf '%s\037%s\037%s\037%s\000' "$size" "$alive" "$(tr '\n' '\036' < "$f")" "$k" +done +"#; + +/// Asks a running download to stop; the partial stays, so starting again +/// resumes rather than refetching. +/// +/// A record whose process has already gone is written off here rather than +/// left to be reported as a failure, because somebody asking for it to stop +/// and it being stopped is not a fault. +const CANCEL: &str = r#" +k=$2; s=$d/$k.download +pid=$(sed -n 's/^pid=//p' "$s" 2>/dev/null) +[ -n "$pid" ] || { echo "nothing is downloading $k" >&2; exit 1; } +if ! kill -TERM "$pid" 2>/dev/null; then + sed 's/^state=.*/state=cancelled/' "$s" > "$s.tmp" && mv "$s.tmp" "$s" +fi +"#; + +/// Takes a model off a machine: the file, a partial, and the state beside it. +/// +/// The partial goes *before* the signal, so that a worker dying from it finds +/// nothing to write about and leaves no state file behind -- see [`WORKER`]'s +/// trap. +const REMOVE: &str = r#" +k=$2; t=$d/$k +if [ ! -f "$t" ] && [ ! -f "$t.part" ] && [ ! -f "$t.download" ]; then + echo "$k is not on this machine" >&2 + exit 1 +fi +rm -f "$t.part" +pid=$(sed -n 's/^pid=//p' "$t.download" 2>/dev/null) +[ -n "$pid" ] && kill -TERM "$pid" 2>/dev/null +rm -f "$t" "$t.part.err" "$t.download" "$t.download.tmp" +# The repository's own directory, and only that one: it was made by the +# download and is litter once empty. Never its parent, which for a key with +# one component would be the models directory itself. +rmdir "${t%/*}" 2>/dev/null +exit 0 +"#; + +/// Runs one of the scripts above on `transport`, against `dir`. +async fn run(transport: &Transport, script: &str, dir: &str, rest: &[&str]) -> Result { + let mut args = vec![ + "-c".to_string(), + format!("{EXPAND}{script}"), + "sh".to_string(), + dir.to_string(), + ]; + args.extend(rest.iter().map(|arg| (*arg).to_string())); + transport.capture(&Launch::new("sh", args, None)).await +} + +/// Starts fetching `file` from `repo` onto the machine `transport` reaches. +pub async fn start( + transport: &Transport, + dir: &str, + repo: &str, + file: &str, + published: &Published, +) -> Result<()> { + let key = key_for(repo, file); + checked(&key)?; + let url = format!( + "https://huggingface.co/{repo}/resolve/main/{}", + file.replace(' ', "%20") + ); + run( + transport, + START, + dir, + &[ + &key, + &url, + published.sha256.as_deref().unwrap_or(""), + &published.bytes.map(|b| b.to_string()).unwrap_or_default(), + WORKER, + ], + ) + .await?; + Ok(()) +} + +/// Every download in flight or stopped on that machine, in key order -- +/// which is the order the models they become are listed in. +pub async fn downloads(transport: &Transport, dir: &str) -> Result> { + let out = run(transport, LIST, dir, &[]).await?; + let mut found: Vec = out + .split('\0') + .filter(|record| !record.is_empty()) + // The key last, so a name with a separator in it cannot eat a field, + // and the state file's own lines separated by RS inside field three. + .filter_map(|record| { + let mut fields = record.splitn(4, '\u{1f}'); + let done: u64 = fields.next()?.trim().parse().unwrap_or(0); + let alive = fields.next()? == "yes"; + let status: HashMap<&str, &str> = fields + .next()? + .split('\u{1e}') + .filter_map(|line| line.split_once('=')) + .collect(); + let key = fields.next()?.trim().to_string(); + let (repo, file) = key.rsplit_once('/')?; + let recorded = status + .get("state") + .and_then(|word| DownloadState::from_word(word.trim())); + let error = status + .get("error") + .map(|text| text.trim()) + .filter(|text| !text.is_empty()) + .map(str::to_string); + // A word this server does not know means a state file written by + // a different version of the script. Whether there is a process + // behind it is measured either way, so that is what it is + // reported as -- dropping the record instead would make a + // download that is using disk invisible. + let recorded = recorded.unwrap_or(if alive { + DownloadState::Running + } else { + DownloadState::Failed + }); + // A record that says it is working and has no process behind it + // is the one case this has to correct: the machine was restarted, + // or the worker was killed by something other than a cancel. + let stopped = matches!(recorded, DownloadState::Running | DownloadState::Verifying); + Some(Download { + state: if stopped && !alive { + DownloadState::Failed + } else { + recorded + }, + error: error.or_else(|| { + (stopped && !alive).then(|| { + "it stopped before it finished -- the machine may have been restarted. \ + Downloading again carries on from here." + .to_string() + }) + }), + done, + total: status.get("total").and_then(|t| t.trim().parse().ok()), + repo: repo.to_string(), + file: file.to_string(), + key, + }) + }) + .collect(); + found.sort_by(|a, b| a.key.cmp(&b.key)); + Ok(found) +} + +/// Asks the machine to stop one download. The partial stays behind. +pub async fn cancel(transport: &Transport, dir: &str, key: &str) -> Result<()> { + checked(key)?; + run(transport, CANCEL, dir, &[key]).await?; + Ok(()) +} + +/// Removes a model from a machine, downloaded or half-downloaded. +pub async fn remove(transport: &Transport, dir: &str, key: &str) -> Result<()> { + checked(key)?; + run(transport, REMOVE, dir, &[key]).await?; + Ok(()) } /// Where a machine reached over ssh keeps its models, when its machine does @@ -707,8 +587,8 @@ pub struct RemoteRepo { pub struct RemoteFile { pub path: String, pub bytes: u64, - /// Already on this machine, so the phone can say so rather than offering to - /// fetch it again. + /// Already on the machine being looked at, so the phone can say so rather + /// than offering to fetch it again. pub have: bool, } @@ -745,27 +625,37 @@ pub fn search(query: &str) -> Result> { .collect()) } -/// The sha256 HuggingFace publishes for one file, if it publishes one. It is -/// the LFS object id, which for these repositories is the sha256 of the content -/// -- so it is a free integrity check rather than a second source of truth. -fn published_sha256(repo: &str, file: &str) -> Option { +/// Asks HuggingFace for both halves of [`Published`] in the one call they +/// come from. +pub fn published(repo: &str, file: &str) -> Result { let url = format!("https://huggingface.co/api/models/{repo}/tree/main?expand=true"); - let body = get_json(&url).ok()?; - body.as_array()?.iter().find_map(|f| { - (f.get("path")?.as_str()? == file) - .then(|| f.get("lfs")?.get("oid")?.as_str().map(str::to_string))? + let body = get_json(&url)?; + let entry = body + .as_array() + .and_then(|files| { + files + .iter() + .find(|f| f.get("path").and_then(serde_json::Value::as_str) == Some(file)) + }) + .with_context(|| format!("{repo} does not have a file called {file}"))?; + Ok(Published { + bytes: entry.get("size").and_then(serde_json::Value::as_u64), + sha256: entry + .get("lfs") + .and_then(|lfs| lfs.get("oid")) + .and_then(serde_json::Value::as_str) + .map(str::to_string), }) } /// The GGUF files in one repository, largest last, with the ones already /// downloaded marked. -pub fn files(repo: &str, store: &ModelStore) -> Result> { +pub fn files(repo: &str, have: &std::collections::HashSet) -> Result> { let url = format!("https://huggingface.co/api/models/{repo}/tree/main"); let body = get_json(&url)?; let list = body .as_array() .context("HuggingFace returned something that is not a list")?; - let have: std::collections::HashSet = store.list().into_iter().map(|m| m.key).collect(); let mut files: Vec = list .iter() .filter_map(|f| { @@ -778,7 +668,7 @@ pub fn files(repo: &str, store: &ModelStore) -> Result> { .get("size") .and_then(serde_json::Value::as_u64) .unwrap_or(0), - have: have.contains(&ModelStore::key_for(repo, &path)), + have: have.contains(&key_for(repo, &path)), path, }) }) diff --git a/server/src/routes.rs b/server/src/routes.rs index f2b3ac3..aeb42bd 100644 --- a/server/src/routes.rs +++ b/server/src/routes.rs @@ -18,6 +18,12 @@ //! GET /machines/{id}/providers/{provider}/auth/{attempt} sign-in state //! POST /machines/{id}/providers/{provider}/auth/{attempt}/code submit browser code //! DELETE /machines/{id}/providers/{provider}/auth/{attempt} cancel sign-in +//! GET /machines/{id}/models its GGUFs, and what is being fetched onto it +//! GET /machines/{id}/models/files?repo=R the GGUFs in one repository, marked +//! with the ones that machine already has +//! POST /machines/{id}/models/download {repo, file}; joins the run already going +//! POST /machines/{id}/models/cancel {key}; the partial stays, so starting again resumes +//! POST /machines/{id}/models/delete {key}; the model, a partial, and the state beside it //! GET /machines/{id}/dir?path=P entries of directory P, and P resolved //! GET /machines/{id}/file?path=P content of file P, or why not //! PUT /machines/{id}/file {path, content, ifSha256} -> new size/mtime/sha256 @@ -79,12 +85,7 @@ //! GET /defaults {effort} -- what a new session starts at //! POST /defaults {effort} -- null for the CLI's own default //! GET /usage cached usage windows per provider -//! GET /models downloaded GGUFs, and what is being fetched //! GET /models/search?q=Q HuggingFace repositories matching Q -//! GET /models/files?repo=R the GGUFs in one repository -//! POST /models/download {repo, file}; rejoins the run already going -//! POST /models/cancel {key}; the partial stays, so starting again resumes -//! POST /models/delete {key} //! ``` //! //! Everything here works purely in the common event model; nothing may @@ -137,6 +138,19 @@ pub fn router(manager: Arc) -> Router { "/machines/{id}", get(read_machine).put(update_machine).delete(delete_machine), ) + // The GGUFs on a machine, and the downloads putting them there. Under + // the machine because that is whose disk they are on: the file has to + // be where `llama-server` will read it, and one backend serves + // several machines. Keys are `owner/repo/file.gguf` and so contain + // slashes, which is why none of these puts one in the path -- a key + // travels in the body or a query string, and the routes stay + // addressable without escaping rules nobody would get right. + .route("/machines/{id}/models", get(machine_models)) + .route("/machines/{id}/models/files", get(repo_files)) + .route("/machines/{id}/models/download", post(start_download)) + .route("/machines/{id}/models/cancel", post(cancel_download)) + .route("/machines/{id}/models/delete", post(delete_model)) + .route("/models/search", get(search_models)) // The models on a configured machine, for a llama session there. .route( "/machines/{id}/providers/{provider}/models", @@ -325,6 +339,12 @@ struct MachineInfo { struct ProviderInfo { name: String, kind: crate::config::DriverKind, + /// The program discovery found, which is the honest answer to "what is + /// this" and is not something the phone may change -- see `machines`. + /// Shown on the provider's card, so that two machines offering the same + /// provider from different builds say so. + #[serde(skip_serializing_if = "Option::is_none")] + command: Option, models: Vec, permission_modes: Vec<&'static str>, #[serde(skip_serializing_if = "Option::is_none")] @@ -351,6 +371,7 @@ fn info_for(machine: crate::config::MachineConfig) -> MachineInfo { .map(|provider| ProviderInfo { name: provider.name, kind: provider.kind, + command: provider.command, models: provider.models, permission_modes: provider.kind.permission_modes().to_vec(), default_permission_mode: provider.kind.default_permission_mode(), @@ -458,6 +479,7 @@ async fn probe_machine( .map(|provider| ProviderInfo { name: provider.name, kind: provider.kind, + command: provider.command, models: provider.models, permission_modes: provider.kind.permission_modes().to_vec(), default_permission_mode: provider.kind.default_permission_mode(), @@ -733,7 +755,7 @@ async fn provider_view( machine: machine.name.clone(), name: provider.name.clone(), kind: provider.kind, - command: provider.command.clone(), + command: provider.command, model_params: provider.kind.model_params(), max_loaded: provider.max_loaded, models, @@ -2492,46 +2514,11 @@ async fn send_event( .await } -/// Separate router because its state is the model store, like `usage`'s. -/// -/// Keys are `owner/repo/file.gguf` and so contain slashes, which is why -/// nothing here puts one in the path: a key travels in the body or a query -/// string, and the routes stay addressable without escaping rules nobody would -/// get right from a phone. -pub fn models_router(store: Arc) -> Router { - Router::new() - .route("/models", get(list_models)) - .route("/models/search", get(search_models)) - .route("/models/files", get(repo_files)) - .route("/models/download", post(start_download)) - .route("/models/cancel", post(cancel_download)) - .route("/models/delete", post(delete_model)) - .with_state(store) -} - -/// What this machine has and what it is fetching, in one answer. Both -/// together deliberately: a phone showing the model list needs both to draw -/// one screen, and two routes would let it render a model as absent while its -/// download sits at 99%. -#[derive(serde::Serialize)] -#[serde(rename_all = "camelCase")] -struct ModelsResponse { - local: Vec, - downloads: Vec, -} - -async fn list_models( - State(store): State>, -) -> Result, ApiError> { - let listing = tokio::task::spawn_blocking(move || ModelsResponse { - local: store.list(), - downloads: store.downloads(), - }) - .await - .context("listing models panicked")?; - Ok(axum::Json(listing)) -} - +/// Searching HuggingFace is the one thing here that is about no machine in +/// particular, so it is the one route left at the top level. Proxied through +/// this server rather than called from the phone, which trusts exactly one +/// certificate -- this server's -- and has no general internet trust to spend +/// on huggingface.co. #[derive(Deserialize)] struct SearchQuery { q: String, @@ -2548,16 +2535,63 @@ async fn search_models( Ok(axum::Json(found)) } +/// One machine's models and its downloads, in one answer. +/// +/// Both together deliberately: this is one screen, and two routes would let +/// the phone draw a model as absent while its download sits at 99%. +#[derive(serde::Serialize)] +#[serde(rename_all = "camelCase")] +struct MachineModels { + local: Vec, + downloads: Vec, +} + +/// Where a machine keeps its models, and how to reach it. Every route below +/// needs both, and a machine that is not there is the same 404 each time. +fn models_on( + manager: &Arc, + id: &str, +) -> Result<(crate::session::transport::Transport, String), ApiError> { + let machine = machine_by_id(manager, id)?; + let transport = crate::session::transport::Transport::for_machine(&machine); + let dir = crate::models::dir_on(&transport, manager.models_dir()); + Ok((transport, dir)) +} + +async fn machine_models( + State(manager): State>, + UrlPath(id): UrlPath, +) -> Result, ApiError> { + let (transport, dir) = models_on(&manager, &id)?; + let local = crate::models::on_machine(&transport, &dir) + .await + .map_err(from_machine)?; + let downloads = crate::models::downloads(&transport, &dir) + .await + .map_err(from_machine)?; + Ok(axum::Json(MachineModels { local, downloads })) +} + #[derive(Deserialize)] struct RepoQuery { repo: String, } +/// The GGUFs in one repository, marked with what this machine already has -- +/// which is why this is under a machine and the search is not. async fn repo_files( - State(store): State>, + State(manager): State>, + UrlPath(id): UrlPath, Query(query): Query, ) -> Result>, ApiError> { - let files = tokio::task::spawn_blocking(move || crate::models::files(&query.repo, &store)) + let (transport, dir) = models_on(&manager, &id)?; + let have: std::collections::HashSet = crate::models::on_machine(&transport, &dir) + .await + .map_err(from_machine)? + .into_iter() + .map(|model| model.key) + .collect(); + let files = tokio::task::spawn_blocking(move || crate::models::files(&query.repo, &have)) .await .context("listing repository files panicked")? .map_err(bad_request)?; @@ -2570,13 +2604,25 @@ struct DownloadRequest { file: String, } -/// Starts a download, or rejoins the one already running for that model. +/// Starts a download on that machine, or leaves the one already running +/// alone. What HuggingFace publishes about the file is read here and handed +/// over, because this is the side with internet trust and because the machine +/// needs the sha to resume safely. async fn start_download( - State(store): State>, + State(manager): State>, + UrlPath(id): UrlPath, axum::Json(body): axum::Json, -) -> Result, ApiError> { - let status = store.start(&body.repo, &body.file).map_err(bad_request)?; - Ok(axum::Json(status)) +) -> Result { + let (transport, dir) = models_on(&manager, &id)?; + let (repo, file) = (body.repo.clone(), body.file.clone()); + let published = tokio::task::spawn_blocking(move || crate::models::published(&repo, &file)) + .await + .context("asking HuggingFace about a file panicked")? + .map_err(bad_request)?; + crate::models::start(&transport, &dir, &body.repo, &body.file, &published) + .await + .map_err(from_machine)?; + Ok(StatusCode::NO_CONTENT) } #[derive(Deserialize)] @@ -2585,17 +2631,25 @@ struct KeyRequest { } async fn cancel_download( - State(store): State>, + State(manager): State>, + UrlPath(id): UrlPath, axum::Json(body): axum::Json, -) -> Result, ApiError> { - let status = store.cancel(&body.key).map_err(bad_request)?; - Ok(axum::Json(status)) +) -> Result { + let (transport, dir) = models_on(&manager, &id)?; + crate::models::cancel(&transport, &dir, &body.key) + .await + .map_err(from_machine)?; + Ok(StatusCode::NO_CONTENT) } async fn delete_model( - State(store): State>, + State(manager): State>, + UrlPath(id): UrlPath, axum::Json(body): axum::Json, ) -> Result { - store.delete(&body.key).map_err(bad_request)?; + let (transport, dir) = models_on(&manager, &id)?; + crate::models::remove(&transport, &dir, &body.key) + .await + .map_err(from_machine)?; Ok(StatusCode::NO_CONTENT) }