Declare provider settings, and give the context figure a denominator
Two things a session could not say, and one it was saying wrongly.
**Every provider setting is reachable.** `-np 1`, the MTP draft depth, the
tool set, the sampling parameters -- most were hardcoded to what measured
best on this machine, which is right as a default and wrong as a constant:
the next machine has a different GPU and a different core count, and
nobody running this app can edit the source. `DriverKind::params` now
declares what a provider takes -- key, label, shape, what blank means, and
whether a change waits for a restart -- and the phone renders whatever
arrives, on the spawn form and in the session settings dialog. Adding a
setting to a driver is one entry in that table and no app change.
`POST /sessions/{id}/params` takes the whole map, so an absent key is the
instruction to unset; the sampling half applies at once and the session is
told in words which of the rest are waiting for a restart.
`tools` is one of them, because it is the biggest lever on a tight
context: the seven built-in definitions are ~1,300 tokens of every prompt
(2,191 against 887 with none). `"none"` omits the flag rather than passing
it on, since `--tools none` is `unknown tool "none"` and a server that
exits.
**The context figure has a denominator.** `Event::ContextWindow` carries
it, read from `llama-server`'s `/props` once the model is up -- the
measurement rather than the request, since a session that named no context
size gets the model's own. Neither coding CLI states its window, so those
keep the bare figure: "2,042" and "2,042 / 8,192" are deliberately
different-looking, and a missing ceiling is never drawn as a proportion of
an assumed one.
**And the numerator was wrong**, by the length of the last reply: it was
the prompt alone, so a five-word answer reported 2,042 against a slot
holding 2,355. It is the turn's total now, which matches `llama-server`'s
own `n_tokens` to within a token.
Two defects the review found, both of which would have shipped: changing
settings on a *stopped* session reported "no process running, so it can't
take new settings", when a stopped session is exactly when you would set
them for the next start; and `GET /tools` answers **403** rather than an
empty list on a server started without `--tools`, so reading it as a
failure made the no-tools session one that never started.
Verified against real models: settings spawned and changed live, the
restart note, a session with two tools and one with none, and the counter
checked against the server's own slot occupancy each time.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
1 parent
ac476ab0c9
commit
81ab564a09
18 files changed
+831
-115
No files matched your search
@@ -62,6 +62,15 @@ Module-by-module intent is in PLAN.md's "Backend layout".
|
||||
MTP draft head is a 50% speed-up or a 33% loss; and `--spec-type draft-mtp`
|
||||
is conditional on the file actually having a head, because asking for one
|
||||
that is not there makes `llama-server` **exit**.
|
||||
**Every one of those is a default rather than a constant** (2026-09-19):
|
||||
`DriverKind::params` declares what a provider takes — key, label, shape,
|
||||
and whether a change waits for a restart — and the phone renders whatever
|
||||
arrives, on the spawn form and in the session settings dialog. Adding a
|
||||
setting to a driver is one entry in that table and no app change. `tools`
|
||||
is in there too, because the seven built-in definitions are ~1,300 tokens
|
||||
of every prompt (2,191 against 887 with none), which on a small window is
|
||||
the difference between a usable session and one that overruns; `"none"`
|
||||
omits the flag, since `--tools none` is a server that exits.
|
||||
Codex is one persistent `codex app-server --stdio` process per session; its
|
||||
driver uses native turn steering and interruption, persists the protocol
|
||||
state and thread id, and reads subscription limits through the same CLI
|
||||
@@ -325,6 +334,11 @@ written, and the fold uses that same predicate to decide a reply is settled.
|
||||
|
||||
## Things that have bitten
|
||||
|
||||
- **A server started with no `--tools` answers 403 at `GET /tools`, not an
|
||||
empty list.** The route is off rather than empty, so reading that as a
|
||||
failure made "no tools" — the one setting whose entire purpose is to have
|
||||
none — a session that never started.
|
||||
|
||||
- **A llama session reports `loading`, and a message sent into it waits.**
|
||||
Before 2026-09-19 the session showed `running` from the moment the process
|
||||
started, so a minute of reading a model off disk was indistinguishable from
|
||||
|
||||
Reference in new issue
Block a user