Let a machine have more than one llama.cpp
A model whose kernels are not upstream needs the fork that has them, and the ordinary models still want the ordinary build. Anything under `~/.local/share/ai-app/llama/<name>/` -- `llama-server`, or the `bin/llama-server` a `cmake --install --prefix` leaves -- is now discovered beside the one on PATH and becomes a provider called `llama-cpp-<name>`, with its own router, preset and model settings. That keeps the module's security property rather than bending it: the phone still names no command, because what runs is still decided by what somebody put on the machine. Each probe answer is tagged with what was asked for, since two of these are now the same program under different paths. Flash attention joins the model settings (`flash-attn` in the preset). llama.cpp's `auto` stays the default; the control is for a model whose publisher asks for `on` outright, which Prism ML's ternary Bonsai does. Verified against the fork built into that directory: discovery answers `llama-cpp-prism`, the child server is started with `--flash-attn on`, and Ternary-Bonsai-2-27B PTQ1_0 loads and answers through a session.
This commit is contained in:
1 parent
df48a334f7
commit
7e7910083c
6 files changed
+220
-16
No files matched your search
@@ -52,6 +52,16 @@ Module-by-module intent is in PLAN.md's "Backend layout".
|
||||
that model; and the preset is read back before every edit, because a router
|
||||
adopted from an earlier run is serving sections this process has never seen
|
||||
and rewriting without them unloads those.
|
||||
**A machine can have more than one llama.cpp** (2026-09-21,
|
||||
`machines.rs`): anything at `~/.local/share/ai-app/llama/<name>/llama-server`
|
||||
or `.../<name>/bin/llama-server` is discovered beside the one on PATH and
|
||||
becomes a provider called `llama-cpp-<name>`, with its own router, preset
|
||||
and model settings. That is how a model whose kernels are not upstream is
|
||||
served -- Prism ML's ternary Bonsai is the one here, built from the
|
||||
`prism` branch into `~/.local/share/ai-app/llama/prism` -- without the
|
||||
phone ever naming a command, which is the property this module exists for.
|
||||
Both providers offer the machine's whole models directory, since which
|
||||
build reads which packing is not answerable from the file.
|
||||
**A llama.cpp session runs on its configured machine** (built
|
||||
2026-09-04, the last of phase 5): `Transport::reserve_port` returns the
|
||||
port the server binds *there* and the port that reaches it *here*, and
|
||||
|
||||
Reference in new issue
Block a user