Let a machine have more than one llama.cpp

A model whose kernels are not upstream needs the fork that has them, and
the ordinary models still want the ordinary build. Anything under
`~/.local/share/ai-app/llama/<name>/` -- `llama-server`, or the
`bin/llama-server` a `cmake --install --prefix` leaves -- is now
discovered beside the one on PATH and becomes a provider called
`llama-cpp-<name>`, with its own router, preset and model settings.

That keeps the module's security property rather than bending it: the
phone still names no command, because what runs is still decided by what
somebody put on the machine. Each probe answer is tagged with what was
asked for, since two of these are now the same program under different
paths.

Flash attention joins the model settings (`flash-attn` in the preset).
llama.cpp's `auto` stays the default; the control is for a model whose
publisher asks for `on` outright, which Prism ML's ternary Bonsai does.

Verified against the fork built into that directory: discovery answers
`llama-cpp-prism`, the child server is started with `--flash-attn on`,
and Ternary-Bonsai-2-27B PTQ1_0 loads and answers through a session.
This commit is contained in:
iris-ai committed 2026-09-21 01:37:36 -04:00
1 parent df48a334f7
commit 7e7910083c
6 files changed
+220 -16

No files matched your search

+29
View File
@@ -74,6 +74,17 @@ a control that silently did nothing).
enrolled token cannot introduce a command. The escape hatch for a binary
somewhere unusual is editing `config.ron`, deliberately the one authority
the phone does not have.
- **A machine may have more than one llama.cpp** (2026-09-21). Anything under
`~/.local/share/ai-app/llama/<name>/``llama-server`, or the
`bin/llama-server` a `cmake --install --prefix` leaves — is discovered
beside the one on PATH and becomes a provider called `llama-cpp-<name>`.
This is the same authority as before, not a new one: what runs is still
decided by what somebody put on the machine, and the phone still names no
command. It exists because one build is not enough — a model whose kernels
are not upstream needs the fork that has them, and the ordinary models
should keep being served by the ordinary build. A provider of its own is
what that has to be, because a router process, its preset file and a
model's load settings all hang off the provider.
- **Migration code is deleted once the update carrying it is received.** The
providers/hosts migration ran on the one host there is and is gone. A file
in the old shape now fails to parse, which is correct because no such file
@@ -556,6 +567,24 @@ deliberate and easy to undo by accident:
server that cannot load it, and it stays in the machine's own model list,
which is where a file on a disk is managed.
- **A build that is not the machine's `llama-server` is another provider**
(2026-09-21, the case being Prism ML's ternary Bonsai, whose `PTQ1_0` and
`PQ2_0` packings stock llama.cpp rejects as unknown types). Nothing about
the fork is named anywhere: it is a directory under
`~/.local/share/ai-app/llama/`, found by the same probe that finds the
binaries on PATH, and everything downstream — the router, the preset file,
a model's load settings, a session — already hangs off the provider, so
there is nothing for a second build to special-case. The consequence worth
knowing is that both providers offer the machine's whole models directory,
including the files the other one is the answer for: which build reads
which packing is not something this side can tell from the file, and a
model that will not load says so in the transcript.
- **Flash attention is a model setting** (`flashAttention`, written as the
preset's `flash-attn`). llama.cpp's own `auto` remains the default and is
the right one; the control exists because a publisher can ask for `on`
outright, which is a statement about the file rather than about the
backend that would be deciding.
### Models (2026-08-28, rebuilt per machine 2026-09-19)
- **A download belongs to the model, not to the request.** Keyed by