Declare provider settings, and give the context figure a denominator

Two things a session could not say, and one it was saying wrongly.

**Every provider setting is reachable.** `-np 1`, the MTP draft depth, the
tool set, the sampling parameters -- most were hardcoded to what measured
best on this machine, which is right as a default and wrong as a constant:
the next machine has a different GPU and a different core count, and
nobody running this app can edit the source. `DriverKind::params` now
declares what a provider takes -- key, label, shape, what blank means, and
whether a change waits for a restart -- and the phone renders whatever
arrives, on the spawn form and in the session settings dialog. Adding a
setting to a driver is one entry in that table and no app change.
`POST /sessions/{id}/params` takes the whole map, so an absent key is the
instruction to unset; the sampling half applies at once and the session is
told in words which of the rest are waiting for a restart.

`tools` is one of them, because it is the biggest lever on a tight
context: the seven built-in definitions are ~1,300 tokens of every prompt
(2,191 against 887 with none). `"none"` omits the flag rather than passing
it on, since `--tools none` is `unknown tool "none"` and a server that
exits.

**The context figure has a denominator.** `Event::ContextWindow` carries
it, read from `llama-server`'s `/props` once the model is up -- the
measurement rather than the request, since a session that named no context
size gets the model's own. Neither coding CLI states its window, so those
keep the bare figure: "2,042" and "2,042 / 8,192" are deliberately
different-looking, and a missing ceiling is never drawn as a proportion of
an assumed one.

**And the numerator was wrong**, by the length of the last reply: it was
the prompt alone, so a five-word answer reported 2,042 against a slot
holding 2,355. It is the turn's total now, which matches `llama-server`'s
own `n_tokens` to within a token.

Two defects the review found, both of which would have shipped: changing
settings on a *stopped* session reported "no process running, so it can't
take new settings", when a stopped session is exactly when you would set
them for the next start; and `GET /tools` answers **403** rather than an
empty list on a server started without `--tools`, so reading it as a
failure made the no-tools session one that never started.

Verified against real models: settings spawned and changed live, the
restart note, a session with two tools and one with none, and the counter
checked against the server's own slot occupancy each time.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
iris-aiandClaude Opus 5 committed 2026-09-19 13:58:08 -04:00
1 parent ac476ab0c9
commit 81ab564a09
18 files changed
+831 -115

No files matched your search

@@ -204,6 +204,13 @@ data class SessionSummary(
* full.
*/
val contextTokens: Long?,
/**
* What [contextTokens] is out of, or null where this session's provider does not say. A third
* state, not a fourth reading of the same one: the occupancy is known and the ceiling is not.
*/
val contextLimit: Long?,
/** What this session's provider settings are set to; empty where it takes none. */
val params: Map<String, String>,
/**
* The longest edge an image should have when it reaches this session, or null where the
* provider has no limit.
@@ -262,6 +269,8 @@ private fun parseSession(session: JSONObject) =
cwd = session.optString("cwd").ifEmpty { null },
contextTokens =
if (session.has("contextTokens")) session.getLong("contextTokens") else null,
contextLimit = if (session.has("contextLimit")) session.getLong("contextLimit") else null,
params = session.optJSONObject("params").stringMap(),
maxImageEdge = session.optInt("maxImageEdge", 0).takeIf { it > 0 },
usageProvider = session.optString("usageProvider").ifEmpty { null },
status = session.getString("status"),
@@ -344,6 +353,27 @@ data class Provider(
val models: List<String>,
val permissionModes: List<String>,
val defaultPermissionMode: String?,
/** The extra settings this provider takes, in the order to draw them. See [ParamSpec]. */
val params: List<ParamSpec>,
)
/**
* One setting a provider takes, as the server describes it.
*
* Declared by the server rather than drawn here, so a driver that grows a setting gets a control
* with no app change — and, more to the point, so that a value measured on one machine can ship as
* a default without becoming a constant nobody else can reach.
*/
data class ParamSpec(
val key: String,
val label: String,
/** What leaving it blank means, in words, shown as the placeholder. */
val unset: String,
val kind: String,
/** For [kind] `"choice"`: the options, the first of which means "unset". */
val options: List<String>,
/** Whether a change waits for the session's process to start again. */
val restart: Boolean,
)
/**
@@ -359,6 +389,11 @@ data class Machine(
val providers: List<Provider>,
)
/** A JSON object of strings, and the empty map for one that is absent. */
private fun JSONObject?.stringMap(): Map<String, String> =
this?.let { object_ -> object_.keys().asSequence().associateWith { object_.getString(it) } }
?: emptyMap()
private fun parseProvider(provider: JSONObject): Provider {
val kind = provider.getString("kind")
return Provider(
@@ -368,6 +403,17 @@ private fun parseProvider(provider: JSONObject): Provider {
models = provider.optJSONArray("models")?.strings().orEmpty(),
permissionModes = provider.optJSONArray("permissionModes")?.strings().orEmpty(),
defaultPermissionMode = provider.optString("defaultPermissionMode").ifEmpty { null },
params =
provider.optJSONArray("params")?.mapObjects { spec ->
ParamSpec(
key = spec.getString("key"),
label = spec.getString("label"),
unset = spec.getString("unset"),
kind = spec.getString("kind"),
options = spec.optJSONArray("options")?.strings().orEmpty(),
restart = spec.optBoolean("restart"),
)
} ?: emptyList(),
)
}
@@ -481,6 +527,13 @@ data class Importable(
* history from before a compaction, which the model is no longer given.
*/
val contextTokens: Long?,
/**
* What [contextTokens] is out of, or null where this session's provider does not say. A third
* state, not a fourth reading of the same one: the occupancy is known and the ceiling is not.
*/
val contextLimit: Long?,
/** What this session's provider settings are set to; empty where it takes none. */
val params: Map<String, String>,
/** Whether [title] is a name somebody chose rather than the last thing said in the session. */
val named: Boolean,
/**
@@ -555,6 +608,10 @@ fun fetchImportable(settings: ServerSettings, machine: String): List<Importable>
contextTokens =
if (session.isNull("contextTokens")) null
else session.optLong("contextTokens").takeIf { it > 0L },
contextLimit =
if (session.isNull("contextLimit")) null
else session.optLong("contextLimit").takeIf { it > 0L },
params = session.optJSONObject("params").stringMap(),
// Absent means an older backend that cannot answer, which is what "unknown" says.
inUse = session.optString("inUse", "unknown"),
named = session.optBoolean("named", false),
@@ -1240,6 +1297,16 @@ fun setSessionEffort(settings: ServerSettings, sessionId: String, level: String?
}
/** Switches how much a running session asks before acting, also in place. */
/** Replaces a session's provider settings with [params] — the whole map, not a patch. */
fun setSessionParams(settings: ServerSettings, sessionId: String, params: Map<String, String>) {
requestFromServer(
settings,
"/sessions/$sessionId/params",
method = "POST",
jsonBody = JSONObject().put("params", JSONObject(params.toMap())).toString(),
) {}
}
fun setSessionPermissionMode(settings: ServerSettings, sessionId: String, mode: String) {
requestFromServer(
settings,