Let a machine have more than one llama.cpp

A model whose kernels are not upstream needs the fork that has them, and
the ordinary models still want the ordinary build. Anything under
`~/.local/share/ai-app/llama/<name>/` -- `llama-server`, or the
`bin/llama-server` a `cmake --install --prefix` leaves -- is now
discovered beside the one on PATH and becomes a provider called
`llama-cpp-<name>`, with its own router, preset and model settings.

That keeps the module's security property rather than bending it: the
phone still names no command, because what runs is still decided by what
somebody put on the machine. Each probe answer is tagged with what was
asked for, since two of these are now the same program under different
paths.

Flash attention joins the model settings (`flash-attn` in the preset).
llama.cpp's `auto` stays the default; the control is for a model whose
publisher asks for `on` outright, which Prism ML's ternary Bonsai does.

Verified against the fork built into that directory: discovery answers
`llama-cpp-prism`, the child server is started with `--flash-attn on`,
and Ternary-Bonsai-2-27B PTQ1_0 loads and answers through a session.
This commit is contained in:
iris-ai committed 2026-09-21 01:37:36 -04:00
1 parent df48a334f7
commit 7e7910083c
6 files changed
+220 -16

No files matched your search

+10
View File
@@ -52,6 +52,16 @@ Module-by-module intent is in PLAN.md's "Backend layout".
that model; and the preset is read back before every edit, because a router
adopted from an earlier run is serving sections this process has never seen
and rewriting without them unloads those.
**A machine can have more than one llama.cpp** (2026-09-21,
`machines.rs`): anything at `~/.local/share/ai-app/llama/<name>/llama-server`
or `.../<name>/bin/llama-server` is discovered beside the one on PATH and
becomes a provider called `llama-cpp-<name>`, with its own router, preset
and model settings. That is how a model whose kernels are not upstream is
served -- Prism ML's ternary Bonsai is the one here, built from the
`prism` branch into `~/.local/share/ai-app/llama/prism` -- without the
phone ever naming a command, which is the property this module exists for.
Both providers offer the machine's whole models directory, since which
build reads which packing is not answerable from the file.
**A llama.cpp session runs on its configured machine** (built
2026-09-04, the last of phase 5): `Transport::reserve_port` returns the
port the server binds *there* and the port that reaches it *here*, and