Say that llama.cpp reaches the phone now
The status section still said "server side" and "no app screen yet", both of which stopped being true today. Also records the discovery trade that would otherwise be rediscovered: command -v follows a non-interactive ssh PATH, so llama.cpp unpacked into ~/.local/opt is invisible until it is symlinked onto PATH. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017xn8nHw1tw1R6PtiY1eEtw
This commit is contained in:
1 parent
6187958de3
commit
4d162331e3
1 file changed
+17
-4
@@ -69,7 +69,7 @@ crash recovery, images both ways), and the usage screen.
|
|||||||
names a host, `session::transport` turns that into an `ssh host …`
|
names a host, `session::transport` turns that into an `ssh host …`
|
||||||
invocation, and the driver never learns which it got.
|
invocation, and the driver never learns which it got.
|
||||||
|
|
||||||
**Phase 4 (llama.cpp) works on the server side** (2026-08-28). Models are
|
**Phase 4 (llama.cpp) works end to end, phone included** (2026-08-28). Models are
|
||||||
browsed and downloaded from HuggingFace (`models.rs`, resumable and
|
browsed and downloaded from HuggingFace (`models.rs`, resumable and
|
||||||
verified), and `session::llama` runs one through `llama-server`, talking
|
verified), and `session::llama` runs one through `llama-server`, talking
|
||||||
to its OpenAI-compatible streaming endpoint. Two things about it are
|
to its OpenAI-compatible streaming endpoint. Two things about it are
|
||||||
@@ -77,10 +77,23 @@ deliberate and easy to undo by accident: the conversation is rebuilt from
|
|||||||
the **transcript** rather than kept in the driver, because driver memory
|
the **transcript** rather than kept in the driver, because driver memory
|
||||||
is invisible to a second device; and a llama session is refused on an ssh
|
is invisible to a second device; and a llama session is refused on an ssh
|
||||||
host, because the model is reached over HTTP and forwarding that port is
|
host, because the model is reached over HTTP and forwarding that port is
|
||||||
not built. No app screen yet — models are driven through the routes.
|
not built.
|
||||||
|
|
||||||
What is left is the models UI, and real-phone/WireGuard bring-up, which is
|
Models are browsed, downloaded and deleted from the app, and a llama
|
||||||
operational rather than code.
|
session picks one of them plus a context size and temperature. Setups —
|
||||||
|
machines, each carrying what it can run — are added, renamed, re-probed
|
||||||
|
and removed from the app too; providers are **discovered by asking the
|
||||||
|
machine**, never typed, so the enrolled token cannot introduce a command.
|
||||||
|
|
||||||
|
What is left is real-phone/WireGuard bring-up, which is operational rather
|
||||||
|
than code.
|
||||||
|
|
||||||
|
**`command -v` follows PATH under a non-interactive ssh session**, which is
|
||||||
|
not the PATH a login shell shows, so a binary somewhere unusual is
|
||||||
|
invisible to discovery — llama.cpp unpacked into `~/.local/opt` needs a
|
||||||
|
symlink into `~/.local/bin` before a setup finds it. The escape hatch for
|
||||||
|
anything odder is editing `config.ron` on the backend, which is
|
||||||
|
deliberately the one authority the phone does not have.
|
||||||
|
|
||||||
**Testing llama.cpp here:** the prebuilt CPU build lives outside the repo
|
**Testing llama.cpp here:** the prebuilt CPU build lives outside the repo
|
||||||
at `~/.local/opt/llama.cpp` (the 15 MB `ubuntu-x64` release asset — no
|
at `~/.local/opt/llama.cpp` (the 15 MB `ubuntu-x64` release asset — no
|
||||||
|
|||||||
Reference in new issue
Block a user