The second half of the llama.cpp work: a session can now name a downloaded
model and talk to it. `llama-server` is spawned through the same transport
as any other driver, polled until the model is loaded, then driven over its
OpenAI-compatible streaming endpoint and translated into the same events
the Claude driver emits -- so the transcript, the SSE stream and the phone
need to know nothing new.
**The conversation is rebuilt from the transcript, not held in the driver.**
llama-server is stateless between requests, so the whole history goes with
every one, and the obvious place to keep it is a Vec in the driver. That
fails the requirement: memory in a driver is invisible to a second device
and gone on restart, and this app is meant to work across devices. Reading
it back also means the model is prompted with exactly what the phone was
shown -- including a reply that was interrupted half way, which is in the
transcript because the deltas were already emitted.
That leaves the Claude driver as the odd one out rather than this one: the
CLI's memory of a conversation is a cache in front of the same transcript,
not a second truth. Said so at the top of llama.rs, because it is the sort
of inconsistency that gets "fixed" in the wrong direction.
Session settings arrive as a driver-interpreted `params` map rather than
new typed fields, so the shared schema does not grow one dialect's
vocabulary. Context size, gpu layers and threads become server flags;
temperature and the rest ride on each request, so changing them need not
reload a model.
**Also fixes an orphan this feature would have created.** Drivers set
kill_on_drop, which covers a session being deleted -- but nothing drops on
the way out of a SIGTERM, so signalling the server left its children
running. For the Claude CLI that is untidy; for a llama-server holding a
model it is gigabytes belonging to nobody. The server now stops its
sessions on SIGTERM and SIGINT. Found by killing a test server and noticing
two 600 MB processes still resident.
Remote llama sessions are refused rather than half-working: the model is
reached over HTTP, and forwarding that port to an ssh host is the "reach
this port" operation the transport does not have yet.
Verified end to end against a real model: downloaded Qwen3-0.6B Q8_0
through the app's own download route, spawned a session on it, and held a
two-turn conversation -- "my favourite colour is teal" then "what is my
favourite colour?", answered "teal", which is the transcript replay doing
its job. Token counts arrive. An earlier attempt with the IQ2_XXS quant
produced fluent nonsense, which turned out to be the quantisation rather
than the pipeline: llama-cli produces the same from that file directly.
Four unit tests cover the fold and the path guard.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017xn8nHw1tw1R6PtiY1eEtw
The first half of the llama.cpp work Bryan asked for: browse HuggingFace,
fetch a model, and see how far it has got from any device.
The design is dev-updater's build-progress shape with the four changes its
author recommended after living with it, since a model download is an hour
where a build is two minutes:
- **A run has an id.** Without one "not downloading" means three different
things -- finished, never started, or someone else's run ended while you
were away -- and over an hour that ambiguity is certain rather than
theoretical. A device compares the run it was watching to the run
reported now.
- **Outcomes outlive their run**, so a phone that was asleep at the moment
of completion can still find out what happened.
- **Cancel exists.** Retrofitting cancellation into a blocking loop is
miserable, and several gigabytes over someone's data plan is not
something to have no answer for.
- **Progress is bytes, not a parsed marker.** We own the loop, so it counts
directly; `total` is whatever Content-Length said and nothing else, and
stays absent when the server sends none rather than becoming a bar drawn
from a guess.
The download owns its own thread rather than the blocking pool, which
exists for short work. It resumes through HTTP Range, and trusts the 206
rather than the request -- a server that ignores Range answers 200 with the
whole file, and appending to that would corrupt it. `truncate(false)` on
the open is load-bearing for the same reason and says so.
Searching is proxied through the server rather than done from the phone,
because the app trusts exactly one certificate -- this one -- and the
machine that must do the downloading is also the one whose view of what
exists matters.
Verified against the real HuggingFace, not a mock: searched, listed a
repository's GGUFs, downloaded 234 MB with live byte progress, cancelled
mid-flight, confirmed the partial survived, restarted and watched it resume
at 162 MB rather than 0, and let it finish. The result's sha256 matches the
one HuggingFace publishes for that file, so the resume is byte-correct and
not merely the right length. llama.cpp then loaded it and ran inference.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017xn8nHw1tw1R6PtiY1eEtw
The code was hand-formatted -- close to rustfmt's output but not it, mostly
in keeping chains and call arguments on one line where the formatter would
break them. That is a per-line decision every future change has to make
again, and reproducing it would mean a config whose only job is to preserve
how the code already looks.
So this is `cargo fmt` at its defaults, with no rustfmt.toml, which is
where the sibling dev-updater checkout already sits: it is clean at the
defaults today, so the two repos now agree on layout without either of them
configuring it.
Formatting only -- no behaviour, no renames, nothing reordered. Verified
after: cargo test (35 pass), cargo clippy --all-targets clean, cargo fmt
--check clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017xn8nHw1tw1R6PtiY1eEtw
The same move, for the same reason: this file is written and read by hand,
and JSON has no comments to say why a host is configured the way it is.
Both house rules come across with it, in config.rs's `format` module and
nowhere else -- a file is the *body* of the config, so no outer parentheses
and nothing indented for them, and `Some` is implicit, which is what makes
`skip_serializing_if` on every optional field load-bearing rather than
tidiness.
The switch is outright: there is no reader for the old format. That is
invisible everywhere except here, because this file holds the enrolled
token hashes -- starting empty leaves the phone unable to talk to the
server and looks, from the phone, like the config having been lost. So a
config.json left beside the new file is named in the log and left alone,
rather than read or deleted.
One wart, documented at DriverKind: the kebab-case spelling is the string
the phone compares against, so it stays, and the file pays for it with
`kind: r#claude-cli` -- a hyphen is not a RON identifier. Renaming the
variant would change what an already-installed build is talking to.
Verified: cargo test, cargo clippy --all-targets, and a real start against
a scratch state directory -- a hand-typed config with comments and a bare
`port: 2222` loads, and what the server writes back sits at column 0 with
no Some(...) in it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017xn8nHw1tw1R6PtiY1eEtw
Nothing behavioral except two status codes; mostly removing places where
the same rule was written down more than once and could drift.
- server/src/private.rs: the owner-only create/write helpers, which
config.rs, certs.rs, and the session dirs each had their own copy of
(certs.rs even duplicated the explanatory comment). One module owns the
modes now, so the "nothing this server writes is readable by anyone
else" property is checkable in one place.
- server/src/media.rs: the image media-type/extension table, which the
four places that have to agree on it each spelled out separately --
storing an upload, serving it back, building a content block, saving a
produced image. The differing *defaults* stay at the call sites with
the reasoning, since they genuinely differ by direction.
- routes.rs: a missing file was a 400 and an unreadable one a 400 with a
hand-rolled log line; they are now 404 and Internal respectively.
UnknownSession became NotFound, since it was the only 404-with-message.
- main.rs: xdg_dir takes the variable's value instead of reading the
environment, which drops the unsafe set_var from its test and lets the
test actually assert the relative-path rule.
- echo.rs had its own 4-byte hex generator beside session::random_hex.
- claude.rs: the two impl Translator blocks were one type's methods.
- Stale comments: phase-2 markers on shipped work, a permission-mode list
that had drifted from the CLI's, "dev-updater" as the leaf certificate's
fallback common name, a half-written sentence in build-apk.sh.
- App: the JSONArray walk written out in four fetchers, the four
near-identical BackHandlers in AppRoot, and SessionScreen's inline
fully-qualified names where the file otherwise imports.
- server/wg-test.log was committed by accident; *.log is ignored now, and
the gitignore comments describe where state actually lives.
- PLAN.md's backend layout gains the new modules and drops hosts.rs for
the ssh.rs that was built instead.
Verified: 35 server tests, clippy clean, app compiles warning-free, and a
scratch server driven over curl -- attachment upload/serve round-trip with
both a known and an unknown content type, the new 404s, transcript and
session-dir deletion, plus a real claude-cli session answering a prompt.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017xn8nHw1tw1R6PtiY1eEtw
It is no longer "local" -- it serves over WireGuard rather than the LAN --
and it is specifically for developing new apps. Renaming the references
here at the same time keeps one name to search for across both repos.
Also drops the last references to gen-dev-cert.sh, which the in-process
certificate generation replaced: the build script and the Gradle task now
say to start the server once, and test-wg-tunnel.sh reads the
certificates from the XDG directory rather than the repo.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017xn8nHw1tw1R6PtiY1eEtw
gen-dev-cert.sh is gone. The server ensures its own certificates on start,
which removes a setup step to remember, a dependency on whatever openssl
was installed, and a second place for the "which addresses?" answer to
live -- the leaf now covers every local IPv4 plus loopback and the
emulator's host alias, so nobody maintains a hardcoded IP.
The split that mattered in the script is kept and now enforced by tests:
the CA is generated once and left alone, because the app pins it and
replacing it strands every installed copy; the leaf is cheap and reissued
every start, so covering a new address is a restart. Both are written
owner-only into a directory outside the repo.
Two things the tests caught. DirBuilder's mode applies only when the
directory is created, so a directory that already existed kept whatever
permissions it had while holding a private key -- the mode is now set
explicitly, in the session directories too. And loading the leaf into the
real RustlsConfig needs the crypto provider installed, which main does but
tests don't.
Verified end to end: deleted the certs, started the server, watched it
generate a CA and warn that installed apps now pin the wrong one, rebuilt
the APK against the new CA, and reinstalled -- the emulator connects over
a certificate that never existed as a pasted constant.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017xn8nHw1tw1R6PtiY1eEtw
Nothing has run against the host's backend yet, so there is no old config
or transcript to keep working -- the XDG paths are simply where state
lives. Removing the fallback takes repo_root() with it, since finding the
repo from the running executable existed only to locate that legacy state.
AGENTS.md gets the arrangement that replaced it: the host builds and runs
from its own clone outside the shared mount, and code reaches it by push
to gitea, which this VM's key is not authorized for.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017xn8nHw1tw1R6PtiY1eEtw
The dev VM is treated as untrusted, and the repo is a read-write virtiofs
mount shared with the backend host -- so a CA private key sitting in it is
a key that machine can sign with, and a leaf signed by this CA is one the
phone's pinned app accepts without question. Pinning against a CA the
attacker holds is no pinning at all.
So certificates are now generated on the machine that serves them, into
$XDG_CONFIG_HOME/ai-app/certs at 0700 with 0600 keys (AI_APP_CERTS
overrides), and config.json and session transcripts move to the XDG config
and data directories. Transcripts move for a plainer reason than the keys:
they are whole conversations, and they were world-readable at 0644.
Two smaller things fall out. The host and VM stop sharing one config,
which had already put a test token on the production backend. And state
stops living where `git clean -xdf` would take the enrollment and every
transcript with it.
State that predates the move is still read from the repo, with a warning
naming where to move it, so an existing install keeps working rather than
silently coming up on an empty config -- the precedence is covered by a
test, since picking the wrong file would otherwise be silent.
Verified: 31 tests, clippy clean; the certificate script writing 0700/0600
into an overridden directory; and the server logging the fallback and
serving from it.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017xn8nHw1tw1R6PtiY1eEtw
A session now names a provider (what: driver kind, command, models) and
optionally a host (where: an ssh target). Keeping them independent is
what the real setup needs -- the backend runs where the phone can reach
it, which isn't where the CLI is installed -- and it means any provider
can be sent to any host rather than a machine being baked into one.
The first provider is claude-cli, named for the CLI rather than bare
"claude", which would suggest the credit-billed API. A fresh config is
seeded with it so a new install has something to spawn and a worked
example to edit; echo stays a built-in provider needing no config.
ssh.rs builds the child process either way: locally, or `ssh -T` with
BatchMode and keepalives, every argument single-quoted for the remote
shell (a working directory that tries to close the quote and start a
command is covered by a test), and `exec` so dropping the connection
takes the CLI down instead of orphaning it.
App: the spawn screen reads /providers and /hosts instead of hardcoded
lists, so config changes need no rebuild. Chip rows are FlowRow, fixing
the reported bug where a row of models that didn't fit wrapped *inside*
each chip -- one letter of "haiku" per line -- rather than onto a second
line.
Verified: 29 tests, clippy clean; the same claude-cli provider run once
locally and once over ssh, with the remote one visibly in a different
environment; an unknown host name refused with the configured list; and
the spawn screen on the emulator showing server-driven providers, hosts,
and models that wrap.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017xn8nHw1tw1R6PtiY1eEtw
wg-setup-host.sh sets up the tunnel on the backend host: keys generated
there and kept there, wg0.conf, wg-quick enabled, and the phone's config
printed as a scannable QR. Split tunnel (AllowedIPs is only the backend
subnet), single-address addressing per PLAN.md, and the three things it
can't do for you -- router UDP forward, DDNS, hairpin check -- spelled
out at the end.
test-wg-tunnel.sh stands up a real WireGuard tunnel between two network
namespaces inside one machine, so the production posture (bind wg0 and
nothing else) is testable with no router, phone, or internet exposure.
Verified: real handshake, server listening on 10.66.0.1:8443 only, and
an authorized request from inside the tunnel answering 200 over pinned
TLS -- the leaf's 10.66.0.1 SAN is what a phone will validate too.
repo_root() now resolves from the running executable before falling back
to the compiled-in path: the repo is shared host<->VM over virtiofs at
different absolute paths with a shared target/, so a binary built on one
side and run on the other looked for its config where nothing exists.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017xn8nHw1tw1R6PtiY1eEtw
GET /usage serves the numbers behind Claude Code's /usage, read with the
CLI's own stored OAuth credentials (nothing to configure). The endpoint
is undocumented, so parsing is defensive -- the generic limits[] array
becomes labeled window bars, unknown kinds surface under their raw name,
and any failure degrades to an 'unavailable' snapshot with the reason.
One UsageProvider per paid service behind a caching monitor that
enforces the >=180s minimum poll regardless of phone refreshes; no
background polling at all. ureq (rustls) does the outbound call, with
the process-level CryptoProvider now chosen explicitly in main -- ureq
brings ring while axum-server brings aws-lc-rs, and with both in the
graph rustls refuses to guess.
App: a Usage screen off the session list -- per-window bars colored by
utilization with relative reset times. Verified live: 74%/26%/16%
windows rendered against the real endpoint.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017xn8nHw1tw1R6PtiY1eEtw
The whole pipe behind one Driver trait and a common event model:
spawn/list/delete sessions, message + question answering, append-only
JSONL transcripts whose sequence numbers are the phone's resume cursor
(surviving backend restarts), bearer-token middleware wrapping every
route including the fallback, wg0-only binding that fails closed, and
first-run token enrollment via a terminal QR.
Verified: cargo test (10), clippy clean, and curl end-to-end over pinned
TLS -- auth rejection, spawn, streamed SSE replay/resume, /question
round trip, restart continuing seq numbers, delete removing everything.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017xn8nHw1tw1R6PtiY1eEtw
irisandClaude Opus 5