Let a session choose how hard it thinks
Output is about an eighth of what a session costs and thinking is nearly all of it -- prose is ~1.5% of output tokens, measured over 27,015 requests of this account's own transcripts -- so the level is the largest saving available short of shortening the conversation itself. Shaped like the working directory rather than like the model: the CLI's only two setting control requests are `set_model` and `set_permission_mode` (checked against the 2.1.258 binary), so `--effort` is read when the process launches and cannot be asked of a running one. `set_session_effort` records the level and stops the process; the next message or Start launches one that has it. That is also why the picker is in the session settings dialog beside Move, and not on the bar beside the model and the mode, which take effect mid-turn. `None` is a level in its own right -- the CLI's own default -- so the picker can return to it, and a blank is normalized to it at the boundary rather than stored as a level the CLI would reject. Offered only where it means something: `DriverKind::takes_effort` reports the capability and the phone leaves the row out entirely, rather than the session-type branch this app does not have anywhere else. A llama session would otherwise get a control whose only effect is stopping its process. Verified on the emulator against the sandbox's fake CLI: the picker sets it, the server reports it, and an echo session's dialog is unchanged. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
1 parent
e4f0935f98
commit
1ff662c7c3
8 files changed
+329
-6
No files matched your search
@@ -42,6 +42,8 @@
|
||||
//! which starts again in the new one
|
||||
//! POST /sessions/{id}/model {model}
|
||||
//! POST /sessions/{id}/permission-mode {permissionMode}
|
||||
//! POST /sessions/{id}/effort {effort} -- null for the CLI's default;
|
||||
//! settled at launch, so this stops the process
|
||||
//! POST /sessions/{id}/command {text} -- /compact, /clear, /rename x, or the dialect's own
|
||||
//! (starts the process first if it has exited)
|
||||
//! POST /sessions/{id}/compact
|
||||
@@ -134,6 +136,7 @@ pub fn router(manager: Arc<SessionManager>) -> Router {
|
||||
.route("/sessions/{id}/cwd", post(set_cwd))
|
||||
.route("/sessions/{id}/model", post(set_model))
|
||||
.route("/sessions/{id}/permission-mode", post(set_permission_mode))
|
||||
.route("/sessions/{id}/effort", post(set_effort))
|
||||
.route("/sessions/{id}/notify", post(set_notify))
|
||||
.route("/notifications", get(notifications))
|
||||
.route("/sessions/{id}/compact", post(compact))
|
||||
@@ -644,6 +647,8 @@ struct SpawnRequest {
|
||||
cwd: Option<PathBuf>,
|
||||
#[serde(default)]
|
||||
permission_mode: Option<String>,
|
||||
#[serde(default)]
|
||||
effort: Option<String>,
|
||||
/// Whatever the chosen driver understands -- llama.cpp's context size and
|
||||
/// sampling. Opaque here on purpose: see `SessionConfig::params`.
|
||||
#[serde(default)]
|
||||
@@ -831,6 +836,7 @@ async fn start_import(
|
||||
model: body.model.clone(),
|
||||
cwd: None,
|
||||
permission_mode: body.permission_mode.clone(),
|
||||
effort: body.effort.clone(),
|
||||
params: std::collections::BTreeMap::new(),
|
||||
import: Some(session.clone()),
|
||||
};
|
||||
@@ -863,6 +869,8 @@ struct ImportRequest {
|
||||
model: Option<String>,
|
||||
#[serde(default)]
|
||||
permission_mode: Option<String>,
|
||||
#[serde(default)]
|
||||
effort: Option<String>,
|
||||
}
|
||||
|
||||
/// Runs `work` on the server, marked as in flight for as long as it takes.
|
||||
@@ -1013,6 +1021,7 @@ async fn spawn(manager: &Arc<SessionManager>, body: SpawnRequest) -> Result<Sess
|
||||
.filter(|cwd| cwd.as_os_str() != "")
|
||||
}),
|
||||
permission_mode: body.permission_mode,
|
||||
effort: body.effort,
|
||||
params: body.params,
|
||||
};
|
||||
|
||||
@@ -1346,6 +1355,30 @@ struct PermissionModeRequest {
|
||||
mode: String,
|
||||
}
|
||||
|
||||
#[derive(Deserialize)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
#[serde(deny_unknown_fields)]
|
||||
struct EffortRequest {
|
||||
/// Absent or null is the CLI's own default, which is a choice somebody can
|
||||
/// make rather than only a state to start in.
|
||||
#[serde(default)]
|
||||
effort: Option<String>,
|
||||
}
|
||||
|
||||
/// Records how hard this session thinks, and stops the process so the next one
|
||||
/// is launched with it -- `--effort` has no control request behind it. See
|
||||
/// [`SessionManager::set_session_effort`].
|
||||
async fn set_effort(
|
||||
State(manager): State<Arc<SessionManager>>,
|
||||
UrlPath(id): UrlPath<String>,
|
||||
axum::Json(body): axum::Json<EffortRequest>,
|
||||
) -> Result<StatusCode, ApiError> {
|
||||
manager
|
||||
.set_session_effort(&id, body.effort.as_deref())
|
||||
.map_err(bad_request)?;
|
||||
Ok(StatusCode::NO_CONTENT)
|
||||
}
|
||||
|
||||
async fn set_permission_mode(
|
||||
State(manager): State<Arc<SessionManager>>,
|
||||
UrlPath(id): UrlPath<String>,
|
||||
|
||||
Reference in new issue
Block a user