Let a session drop its context without ending
Adds `Event::Cleared`, `SessionCommand::Clear`, and `Driver::clear`, so
`POST /sessions/{id}/command {"text": "/clear"}` does for a session what
the CLI's own `/clear` does for a terminal.
The marker is a divider, not a truncation: everything above it stays in
the transcript, because that is the only copy of the conversation the
phone has and a person scrolling back is a different question from what
the model is given. It also makes clearing mean one thing across
drivers -- `claude` sends `/clear` and the CLI answers with a fresh
`init` whose new session_id the reader already persists as the resume
token, so the next launch resumes the cleared conversation with nothing
to keep in step; `llama` needs no state at all, since `conversation()`
already folds the transcript and now folds from the last marker; `echo`
emits the marker alone, so the phone's divider and scroll behaviour can
be exercised without spending a real session's context.
That fold is why `Cleared` is documented as load-bearing rather than
decorative. For any driver that rebuilds its conversation from the
transcript, this marker decides what the model sees, and treating it as
something only the phone draws would silently put the cleared
conversation back in front of the model at full price.
Clear rides the existing boundary pump like any other SessionCommand, so
one arriving mid-turn waits exactly as a compaction does, and nothing
grows a second way to wait.
Removes `--autocompact` in the same change, because clearing is the
cheaper answer to the problem it was added for and Bryan would rather
manage context that way. Keeping the measurement here, since it was the
reason for the constant and is worth more than the constant was:
context returned to 70-85k within ten calls of a compaction; a
compaction took 104,346 to 147,671 ms; compaction cost that session
2,655,508 tokens across six boundaries, of which the single automatic
one at the 1M ceiling was 1,696,870. Clearing costs nothing, because
nothing is sent.
70 tests, clippy clean, rustfmt clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VETa8afmpWaYezLCqJhDB8
This commit is contained in:
1 parent
797513bb86
commit
1e3eec8cf0
5 files changed
+122
-34
No files matched your search
@@ -411,11 +411,19 @@ impl Driver for LlamaDriver {
|
||||
|
||||
fn compact(&self) {
|
||||
let _ = self.sink.send(Event::Error {
|
||||
message: "llama.cpp has no compaction. When the context fills, start a new session."
|
||||
message: "llama.cpp has no compaction. Clear the session instead, which costs nothing."
|
||||
.to_string(),
|
||||
});
|
||||
}
|
||||
|
||||
fn clear(&self) {
|
||||
// All of it. `conversation` folds from the last of these, so
|
||||
// recording the marker *is* the reset -- there is no driver state
|
||||
// to keep in step with it, which is the same property that makes
|
||||
// a second device see the same conversation this one does.
|
||||
let _ = self.sink.send(Event::Cleared);
|
||||
}
|
||||
|
||||
/// Stops generating and leaves the server loaded.
|
||||
///
|
||||
/// Worth being deliberate about, because the cost is asymmetric and
|
||||
@@ -457,7 +465,14 @@ fn conversation(path: &Path) -> Vec<Message> {
|
||||
};
|
||||
let mut messages: Vec<Message> = Vec::new();
|
||||
let mut pending = String::new();
|
||||
for event in events {
|
||||
// Everything before the last clear is still in the transcript and is
|
||||
// deliberately not in the conversation. Folding from zero here would
|
||||
// put it back, which is the whole of what clearing had to undo.
|
||||
let events = match events.iter().rposition(|e| e.event == Event::Cleared) {
|
||||
Some(at) => &events[at + 1..],
|
||||
None => &events[..],
|
||||
};
|
||||
for event in events.iter().cloned() {
|
||||
match event.event {
|
||||
Event::UserMessage { text } => {
|
||||
if !pending.is_empty() {
|
||||
@@ -698,6 +713,50 @@ mod tests {
|
||||
assert_eq!(messages[1].content, "still here");
|
||||
}
|
||||
|
||||
#[test]
|
||||
/// Clearing decides what the *model* is given, not just what the
|
||||
/// phone draws. Everything above the marker stays in the transcript
|
||||
/// -- a person can still scroll back to it -- and none of it is sent.
|
||||
fn the_conversation_starts_after_the_last_clear() {
|
||||
let (_dir, path) = transcript_with(&[
|
||||
Event::UserMessage {
|
||||
text: "the long expensive conversation".into(),
|
||||
},
|
||||
Event::AssistantText {
|
||||
delta: "at length".into(),
|
||||
},
|
||||
Event::Cleared,
|
||||
Event::UserMessage {
|
||||
text: "a fresh start".into(),
|
||||
},
|
||||
Event::AssistantText {
|
||||
delta: "cheaply".into(),
|
||||
},
|
||||
]);
|
||||
let messages = conversation(&path);
|
||||
assert_eq!(messages.len(), 2);
|
||||
assert_eq!(messages[0].content, "a fresh start");
|
||||
assert_eq!(messages[1].content, "cheaply");
|
||||
}
|
||||
|
||||
#[test]
|
||||
/// The *last* one, so clearing twice does not resurrect what the
|
||||
/// first clear dropped.
|
||||
fn only_the_newest_clear_counts() {
|
||||
let (_dir, path) = transcript_with(&[
|
||||
Event::UserMessage { text: "one".into() },
|
||||
Event::Cleared,
|
||||
Event::UserMessage { text: "two".into() },
|
||||
Event::Cleared,
|
||||
Event::UserMessage {
|
||||
text: "three".into(),
|
||||
},
|
||||
]);
|
||||
let messages = conversation(&path);
|
||||
assert_eq!(messages.len(), 1);
|
||||
assert_eq!(messages[0].content, "three");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_model_key_cannot_climb_out_of_the_models_directory() {
|
||||
let dir = tempfile::tempdir().expect("tempdir");
|
||||
|
||||
Reference in new issue
Block a user