Declare provider settings, and give the context figure a denominator

Two things a session could not say, and one it was saying wrongly.

**Every provider setting is reachable.** `-np 1`, the MTP draft depth, the
tool set, the sampling parameters -- most were hardcoded to what measured
best on this machine, which is right as a default and wrong as a constant:
the next machine has a different GPU and a different core count, and
nobody running this app can edit the source. `DriverKind::params` now
declares what a provider takes -- key, label, shape, what blank means, and
whether a change waits for a restart -- and the phone renders whatever
arrives, on the spawn form and in the session settings dialog. Adding a
setting to a driver is one entry in that table and no app change.
`POST /sessions/{id}/params` takes the whole map, so an absent key is the
instruction to unset; the sampling half applies at once and the session is
told in words which of the rest are waiting for a restart.

`tools` is one of them, because it is the biggest lever on a tight
context: the seven built-in definitions are ~1,300 tokens of every prompt
(2,191 against 887 with none). `"none"` omits the flag rather than passing
it on, since `--tools none` is `unknown tool "none"` and a server that
exits.

**The context figure has a denominator.** `Event::ContextWindow` carries
it, read from `llama-server`'s `/props` once the model is up -- the
measurement rather than the request, since a session that named no context
size gets the model's own. Neither coding CLI states its window, so those
keep the bare figure: "2,042" and "2,042 / 8,192" are deliberately
different-looking, and a missing ceiling is never drawn as a proportion of
an assumed one.

**And the numerator was wrong**, by the length of the last reply: it was
the prompt alone, so a five-word answer reported 2,042 against a slot
holding 2,355. It is the turn's total now, which matches `llama-server`'s
own `n_tokens` to within a token.

Two defects the review found, both of which would have shipped: changing
settings on a *stopped* session reported "no process running, so it can't
take new settings", when a stopped session is exactly when you would set
them for the next start; and `GET /tools` answers **403** rather than an
empty list on a server started without `--tools`, so reading it as a
failure made the no-tools session one that never started.

Verified against real models: settings spawned and changed live, the
restart note, a session with two tools and one with none, and the counter
checked against the server's own slot occupancy each time.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
iris-aiandClaude Opus 5 committed 2026-09-19 13:58:08 -04:00
1 parent ac476ab0c9
commit 81ab564a09
18 files changed
+831 -115

No files matched your search

@@ -204,6 +204,13 @@ data class SessionSummary(
* full.
*/
val contextTokens: Long?,
/**
* What [contextTokens] is out of, or null where this session's provider does not say. A third
* state, not a fourth reading of the same one: the occupancy is known and the ceiling is not.
*/
val contextLimit: Long?,
/** What this session's provider settings are set to; empty where it takes none. */
val params: Map<String, String>,
/**
* The longest edge an image should have when it reaches this session, or null where the
* provider has no limit.
@@ -262,6 +269,8 @@ private fun parseSession(session: JSONObject) =
cwd = session.optString("cwd").ifEmpty { null },
contextTokens =
if (session.has("contextTokens")) session.getLong("contextTokens") else null,
contextLimit = if (session.has("contextLimit")) session.getLong("contextLimit") else null,
params = session.optJSONObject("params").stringMap(),
maxImageEdge = session.optInt("maxImageEdge", 0).takeIf { it > 0 },
usageProvider = session.optString("usageProvider").ifEmpty { null },
status = session.getString("status"),
@@ -344,6 +353,27 @@ data class Provider(
val models: List<String>,
val permissionModes: List<String>,
val defaultPermissionMode: String?,
/** The extra settings this provider takes, in the order to draw them. See [ParamSpec]. */
val params: List<ParamSpec>,
)
/**
* One setting a provider takes, as the server describes it.
*
* Declared by the server rather than drawn here, so a driver that grows a setting gets a control
* with no app change — and, more to the point, so that a value measured on one machine can ship as
* a default without becoming a constant nobody else can reach.
*/
data class ParamSpec(
val key: String,
val label: String,
/** What leaving it blank means, in words, shown as the placeholder. */
val unset: String,
val kind: String,
/** For [kind] `"choice"`: the options, the first of which means "unset". */
val options: List<String>,
/** Whether a change waits for the session's process to start again. */
val restart: Boolean,
)
/**
@@ -359,6 +389,11 @@ data class Machine(
val providers: List<Provider>,
)
/** A JSON object of strings, and the empty map for one that is absent. */
private fun JSONObject?.stringMap(): Map<String, String> =
this?.let { object_ -> object_.keys().asSequence().associateWith { object_.getString(it) } }
?: emptyMap()
private fun parseProvider(provider: JSONObject): Provider {
val kind = provider.getString("kind")
return Provider(
@@ -368,6 +403,17 @@ private fun parseProvider(provider: JSONObject): Provider {
models = provider.optJSONArray("models")?.strings().orEmpty(),
permissionModes = provider.optJSONArray("permissionModes")?.strings().orEmpty(),
defaultPermissionMode = provider.optString("defaultPermissionMode").ifEmpty { null },
params =
provider.optJSONArray("params")?.mapObjects { spec ->
ParamSpec(
key = spec.getString("key"),
label = spec.getString("label"),
unset = spec.getString("unset"),
kind = spec.getString("kind"),
options = spec.optJSONArray("options")?.strings().orEmpty(),
restart = spec.optBoolean("restart"),
)
} ?: emptyList(),
)
}
@@ -481,6 +527,13 @@ data class Importable(
* history from before a compaction, which the model is no longer given.
*/
val contextTokens: Long?,
/**
* What [contextTokens] is out of, or null where this session's provider does not say. A third
* state, not a fourth reading of the same one: the occupancy is known and the ceiling is not.
*/
val contextLimit: Long?,
/** What this session's provider settings are set to; empty where it takes none. */
val params: Map<String, String>,
/** Whether [title] is a name somebody chose rather than the last thing said in the session. */
val named: Boolean,
/**
@@ -555,6 +608,10 @@ fun fetchImportable(settings: ServerSettings, machine: String): List<Importable>
contextTokens =
if (session.isNull("contextTokens")) null
else session.optLong("contextTokens").takeIf { it > 0L },
contextLimit =
if (session.isNull("contextLimit")) null
else session.optLong("contextLimit").takeIf { it > 0L },
params = session.optJSONObject("params").stringMap(),
// Absent means an older backend that cannot answer, which is what "unknown" says.
inUse = session.optString("inUse", "unknown"),
named = session.optBoolean("named", false),
@@ -1240,6 +1297,16 @@ fun setSessionEffort(settings: ServerSettings, sessionId: String, level: String?
}
/** Switches how much a running session asks before acting, also in place. */
/** Replaces a session's provider settings with [params] — the whole map, not a patch. */
fun setSessionParams(settings: ServerSettings, sessionId: String, params: Map<String, String>) {
requestFromServer(
settings,
"/sessions/$sessionId/params",
method = "POST",
jsonBody = JSONObject().put("params", JSONObject(params.toMap())).toString(),
) {}
}
fun setSessionPermissionMode(settings: ServerSettings, sessionId: String, mode: String) {
requestFromServer(
settings,
@@ -60,3 +60,23 @@ fun compactingLabel(seconds: Long?): String =
seconds < 60 -> "compacting ${seconds}s"
else -> "compacting ${seconds / 60}m ${seconds % 60}s"
}
/**
* How full the session is, as the status row says it.
*
* Three states, not two, and the third is the one that needed the words: a session whose occupancy
* is known and whose ceiling is not. That one keeps the bare figure, and a session with a ceiling
* gets both — the reader can see which they are looking at. What must not happen is a missing
* ceiling drawn as a number, or as a proportion of some assumed window, which would be this screen
* inventing the very fact it does not have.
*
* A llama.cpp session always has one, since the window is a flag its own server was started with. A
* coding CLI's is the vendor's business and neither control protocol states it, so those keep the
* bare figure they have always had.
*/
fun contextLabel(held: Long?, limit: Long?): String =
when {
held == null -> "context unknown"
limit == null -> "context ${tokens(held)}"
else -> "context ${tokens(held)} / ${tokens(limit)}"
}
@@ -163,6 +163,9 @@ sealed class SessionEvent {
*/
data class UsageDelta(val tokens: Long, val context: Long?) : SessionEvent()
/** How much context this session's model has, which is what [UsageDelta.context] is out of. */
data class ContextWindow(val tokens: Long) : SessionEvent()
/**
* A compaction that finished, and how much context it recovered.
*
@@ -293,6 +296,7 @@ fun parseSeqEvent(json: String): SeqEvent {
model = body.optString("model").ifEmpty { null },
permissionMode = body.optString("permissionMode").ifEmpty { null },
)
"contextWindow" -> SessionEvent.ContextWindow(body.getLong("tokens"))
"usageDelta" ->
SessionEvent.UsageDelta(
body.getLong("tokens"),
@@ -363,3 +367,18 @@ fun contextAfter(current: Long?, event: SessionEvent): Long? =
is SessionEvent.Cleared -> null
else -> current
}
/**
* The context window after [event], mirroring the server's `context_limit_after` for the same
* reason [contextAfter] mirrors its neighbour: the screen has to keep up between page loads.
*
* A window belongs to the process, so a session whose process has exited has none — left standing,
* a session restarted on a different model would draw its occupancy against the old model's
* ceiling.
*/
fun contextLimitAfter(current: Long?, event: SessionEvent): Long? =
when (event) {
is SessionEvent.ContextWindow -> event.tokens
is SessionEvent.Status -> if (event.state == "exited") null else current
else -> current
}
@@ -0,0 +1,117 @@
package com.example.aiapp
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Spacer
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.height
import androidx.compose.foundation.text.KeyboardOptions
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.OutlinedTextField
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.ui.Modifier
import androidx.compose.ui.text.input.KeyboardType
import androidx.compose.ui.unit.dp
/**
* The controls for whatever settings a provider says it takes.
*
* One composable for both screens that offer them — the spawn form and the session settings dialog
* — and for every provider, because the server declares the list (see `DriverKind::params`) rather
* than this file knowing it. A driver that grows a setting gets a control here with no change to
* the app, which is the whole point: the values that suit one machine ship as defaults, and every
* one of them stays reachable from a phone.
*
* [values] is the whole map and [onChange] hands back the whole map. A key absent from it means the
* setting is unset, which is what every [ParamSpec.unset] describes — so clearing a field and never
* touching it are deliberately the same state.
*/
@Composable
fun ProviderParamFields(
specs: List<ParamSpec>,
values: Map<String, String>,
onChange: (Map<String, String>) -> Unit,
/**
* Whether to say which settings wait for a restart. False on a spawn form, where nothing is
* running yet and every setting is about to be read — saying it there would be a warning about
* a state the reader cannot be in.
*/
warnAboutRestart: Boolean,
modifier: Modifier = Modifier,
) {
if (specs.isEmpty()) return
Column(modifier.fillMaxWidth()) {
specs.forEach { spec ->
val set = { value: String ->
onChange(
// Blank clears rather than storing an empty string: the server reads an absent
// key as "use the default", and an empty one would be a value it then failed
// to parse.
if (value.isBlank()) values - spec.key else values + (spec.key to value)
)
}
when (spec.kind) {
"choice" -> {
// The first option is what unset means, so selecting it clears the key — see
// `ParamKind::Choice`. Without that the picker could show a default it could
// not return to.
val default = spec.options.firstOrNull().orEmpty()
ChipGroup(
label = spec.label + restartSuffix(spec, warnAboutRestart),
options = spec.options,
selected = values[spec.key] ?: default,
onSelect = { chosen -> set(if (chosen == default) "" else chosen) },
)
}
else ->
OutlinedTextField(
value = values[spec.key].orEmpty(),
onValueChange = set,
label = { Text(spec.label + restartSuffix(spec, warnAboutRestart)) },
placeholder = { Text(spec.unset) },
singleLine = true,
keyboardOptions = KeyboardOptions(keyboardType = keyboardFor(spec.kind)),
modifier = Modifier.fillMaxWidth(),
)
}
Spacer(Modifier.height(16.dp))
}
if (warnAboutRestart && specs.any { it.restart }) {
Text(
"A setting marked “on restart” is saved now and read when this session's process " +
"next starts.",
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
}
}
}
/**
* Marks a control whose value will not take effect yet.
*
* On the label rather than beside it, because the reader decides whether to change the thing before
* they touch it — a note underneath is read after the decision.
*/
private fun restartSuffix(spec: ParamSpec, warn: Boolean): String =
if (warn && spec.restart) " (on restart)" else ""
/**
* The keyboard for a value's shape. A number field that opens the letter keyboard is one every
* entry is made harder by, and these are nearly all numbers.
*/
private fun keyboardFor(kind: String): KeyboardType =
when (kind) {
"integer" -> KeyboardType.Number
"decimal" -> KeyboardType.Decimal
else -> KeyboardType.Text
}
/**
* How long typing has to stop before edited settings are sent.
*
* Long enough that a number is one request rather than one per digit, short enough that closing the
* dialog straight after typing still saves — the save runs on the screen behind it, which outlives
* the dialog, so this delay is not a window the value can be lost in.
*/
const val PARAM_SAVE_DELAY_MS = 700L
@@ -233,6 +233,8 @@ fun SessionScreen(
// that was measured.
var contextTokens by
remember(address) { mutableStateOf(if (isSubagent) null else summary.contextTokens) }
var contextLimit by
remember(address) { mutableStateOf(if (isSubagent) null else summary.contextLimit) }
// When the current compaction started. The moment comes off the `compacting` status event
// itself -- the server timestamps every transcript line -- rather than off this device noticing
// one, which is what makes it survive leaving the session and reopening it.
@@ -304,6 +306,14 @@ fun SessionScreen(
// hardcoded list is a claim about a machine.
var offeredModels by remember { mutableStateOf<List<OfferedModel>>(emptyList()) }
var offeredPermissionModes by remember { mutableStateOf<List<String>>(emptyList()) }
// The settings this session's provider takes, and what they are set to. The specs come from
// the provider and the values from the session, because "what can be set" and "what is set"
// are different questions with different answers.
var paramSpecs by remember { mutableStateOf<List<ParamSpec>>(emptyList()) }
var params by remember(summary.id) { mutableStateOf(summary.params) }
// What was last successfully saved, so the debounce below knows whether there is anything to
// send -- and so a failed save can put the controls back to what the server actually holds.
var savedParams by remember(summary.id) { mutableStateOf(summary.params) }
val lifecycleOwner = LocalLifecycleOwner.current
// The resume cursor, written from the stream's IO thread.
val lastSeq = remember { AtomicLong(0) }
@@ -489,6 +499,7 @@ fun SessionScreen(
// Before the rest, and for every event rather than only the usage ones: a compaction and a
// clear move this as much as a turn does. See `contextAfter`.
contextTokens = contextAfter(contextTokens, entry.event)
contextLimit = contextLimitAfter(contextLimit, entry.event)
when (val event = entry.event) {
// Nothing further: what it carries was folded into the context above.
is SessionEvent.UsageDelta -> {}
@@ -1205,6 +1216,30 @@ fun SessionScreen(
fun label(id: String?): String =
offeredModels.firstOrNull { it.id == id }?.label ?: modelLabel(id)
/**
* Saves edited provider settings once the typing stops.
*
* Debounced rather than sent per keystroke, because a text field over the tunnel would be a
* round trip and a config write per character. Here rather than in the settings dialog for the
* reason the delay exists at all: the dialog can be dismissed mid-edit, and this screen
* outlives it, so the last value typed is still saved.
*/
LaunchedEffect(params) {
if (params == savedParams) return@LaunchedEffect
delay(PARAM_SAVE_DELAY_MS)
val wanted = params
try {
withContext(Dispatchers.IO) { setSessionParams(settings, summary.id, wanted) }
savedParams = wanted
actionError = null
} catch (e: ApiException) {
// Back to what the server holds. A control left showing a value that was refused is
// stating something untrue about the session.
params = savedParams
actionError = e.message
}
}
// Only for the model picker, which a subagent does not have.
if (!isSubagent) {
LaunchedEffect(summary.machine, summary.provider) {
@@ -1218,6 +1253,7 @@ fun SessionScreen(
}
.getOrNull()
offeredPermissionModes = provider?.permissionModes.orEmpty()
paramSpecs = provider?.params.orEmpty()
offeredModels =
provider
?.let {
@@ -1916,6 +1952,7 @@ fun SessionScreen(
status = status,
compactingFor = compactingFor,
contextTokens = contextTokens,
contextLimit = contextLimit,
backgroundTasks = backgroundTasks,
subagent = isSubagent,
)
@@ -2197,6 +2234,9 @@ fun SessionScreen(
effort = effort.takeIf { summary.takesEffort },
takesEffort = summary.takesEffort,
onEffortChanged = { effort = it },
paramSpecs = paramSpecs,
params = params,
onParamsChanged = { params = it },
cachedBytes = cachedBytes,
// The purge finishes before the epoch moves, because the relaunched opening effect
// reads the same directory and would otherwise draw what is about to be deleted. The
@@ -2473,6 +2513,8 @@ private fun SessionStatusRow(
compactingFor: Long?,
/** Context the session is holding, or null where nothing has measured it. */
contextTokens: Long?,
/** What that is out of, or null where the provider does not say. */
contextLimit: Long?,
/** Provider-reported live background work; zero is deliberately not drawn. */
backgroundTasks: Int,
modifier: Modifier = Modifier,
@@ -2577,7 +2619,7 @@ private fun SessionStatusRow(
// reports usage, and one that has not run a turn all showed nothing at all, which reads as
// a conversation with room to spare.
Text(
contextTokens?.let { "context ${tokens(it)}" } ?: "context unknown",
contextLabel(contextTokens, contextLimit),
style = MaterialTheme.typography.labelSmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
@@ -74,6 +74,15 @@ fun SessionSettingsDialog(
onEffortChanged: (String?) -> Unit,
/** Whether a level does anything here; the row is left out entirely where it does not. */
takesEffort: Boolean,
/**
* The settings this session's provider takes, and what they are set to.
*
* Declared by the server rather than listed here -- see [ProviderParamFields]. Empty for a
* provider with none, which draws no section at all.
*/
paramSpecs: List<ParamSpec>,
params: Map<String, String>,
onParamsChanged: (Map<String, String>) -> Unit,
/**
* What this phone is holding of the conversation, or null while that is being measured -- see
* the Reload row below, which is what would discard it.
@@ -451,6 +460,23 @@ fun SessionSettingsDialog(
)
}
}
if (paramSpecs.isNotEmpty()) {
Spacer(Modifier.height(16.dp))
Text(
"Model settings",
style = MaterialTheme.typography.titleSmall,
)
Spacer(Modifier.height(8.dp))
// Edited here and saved by the screen behind this, which is what makes
// typing in a text field affordable: the save is debounced, and a dialog
// dismissed mid-edit would take an unsaved value with it.
ProviderParamFields(
specs = paramSpecs,
values = params,
onChange = onParamsChanged,
warnAboutRestart = true,
)
}
Spacer(Modifier.height(8.dp))
Row(
verticalAlignment = Alignment.CenterVertically,
@@ -73,13 +73,9 @@ fun SpawnScreen(
// Only the spawn's own failure. The fetch's lives in `options`: this one leaves a filled-in
// form worth keeping, and that one leaves nothing to fill in.
var spawnError by remember { mutableStateOf<String?>(null) }
var contextSize by remember { mutableStateOf("") }
var temperature by remember { mutableStateOf("") }
// Whether a model that carries a multi-token-prediction head drafts with it. Left to the
// server by default, which turns it on exactly where the file has one -- see `SPECULATIVE` in
// the llama driver. Here so a machine where drafting does not pay has a way out that is not
// an edit to config.ron.
var speculative by remember { mutableStateOf(SPECULATIVE_AUTO) }
// Whatever the chosen provider says it takes, by key. Empty until something is typed: an
// absent key means the server's own default, which is what every field's placeholder says.
var params by remember { mutableStateOf<Map<String, String>>(emptyMap()) }
LaunchedEffect(Unit) {
// Separate from the machines fetch below and deliberately not fatal: failing to learn the
@@ -145,6 +141,9 @@ fun SpawnScreen(
// machine looks exactly like one from this one.
LaunchedEffect(machine?.id, current?.name) {
model = ""
// A key from the previous provider would be a setting this one does not have, drawn
// by no control and sent at the spawn anyway.
params = emptyMap()
providerModels = emptyList()
providerModelsError = null
permissionMode = current?.defaultPermissionMode.orEmpty()
@@ -283,35 +282,14 @@ fun SpawnScreen(
Spacer(Modifier.height(16.dp))
}
if (isLlama) {
OutlinedTextField(
value = contextSize,
onValueChange = { contextSize = it },
label = { Text("Context size (blank = the model's default)") },
singleLine = true,
modifier = Modifier.fillMaxWidth(),
)
Spacer(Modifier.height(16.dp))
OutlinedTextField(
value = temperature,
onValueChange = { temperature = it },
label = { Text("Temperature (blank = llama.cpp's default)") },
singleLine = true,
modifier = Modifier.fillMaxWidth(),
)
Spacer(Modifier.height(16.dp))
// Said as what it is rather than as "MTP": the reader is choosing whether the session
// goes faster, and most models have nothing to turn on here at all.
ChipGroup(
label = "Speculative decoding (models that carry a draft head)",
options = listOf(SPECULATIVE_AUTO, SPECULATIVE_OFF),
selected = speculative,
onSelect = { speculative = it },
)
Spacer(Modifier.height(16.dp))
}
// Nothing is running yet, so nothing here waits for a restart -- every one of these is
// read by the process this form is about to start.
ProviderParamFields(
specs = current?.params.orEmpty(),
values = params,
onChange = { params = it },
warnAboutRestart = false,
)
// Every session whose tools act on files needs one, which is both kinds that have
// tools -- a llama session's built-in tools run in it exactly as a CLI's do.
@@ -387,28 +365,10 @@ fun SpawnScreen(
cwd = cwd.trim().takeIf { takesCwd },
permissionMode = permissionMode.takeIf { it.isNotEmpty() },
effort = effort.takeIf { isCodingCli },
// Sent only when set, so blank means "whatever llama.cpp does
// by default" rather than a zero.
params =
buildMap {
if (isLlama) {
contextSize
.trim()
.takeIf { it.isNotEmpty() }
?.let { put("contextSize", it) }
temperature
.trim()
.takeIf { it.isNotEmpty() }
?.let { put("temperature", it) }
// Only the choice that changes anything: "auto"
// is the absence of the setting, not a value of
// it, so a session spawned without an opinion
// carries none.
if (speculative == SPECULATIVE_OFF) {
put("speculative", "off")
}
}
},
// Already only the keys somebody set: a field left blank
// removes its key rather than sending an empty value, so
// "blank" reaches the server as "your default".
params = params,
)
}
onSpawned(spawned)
@@ -426,15 +386,6 @@ fun SpawnScreen(
}
}
/**
* Leave the draft head to the server, which uses one wherever the model file has one. Spelled the
* same as the absence of the `speculative` parameter, because that is what it means.
*/
private const val SPECULATIVE_AUTO = "auto"
/** The `speculative` parameter's only other value; see the llama driver's `SPECULATIVE`. */
private const val SPECULATIVE_OFF = "off"
/**
* A labeled row of choices that wraps onto as many lines as it needs.
*
@@ -555,6 +555,7 @@ fun foldEvent(items: List<TranscriptItem>, entry: SeqEvent): List<TranscriptItem
is SessionEvent.RetiredTaskNote -> items
// Screen-level state, not transcript rows -- see SessionScreen.
is SessionEvent.UsageDelta -> items
is SessionEvent.ContextWindow -> items
}
/**
@@ -75,6 +75,8 @@ class SessionOrderTest {
resumeAt = null,
cwd = null,
contextTokens = null,
contextLimit = null,
params = emptyMap(),
maxImageEdge = null,
usageProvider = null,
status = status,