Let sessions outlive the backend, and never resume one twice
Three `claude` processes ended up running against this checkout on 2026-08-29, and the account hit its session limit. One cause, several ways in. An agent imported the Claude Code session it was *itself* running in. That is an ordinary import, and importing runs `--resume` -- so a second CLI attached to a file the first was still writing. The whole 65 MB conversation, 154 embedded screenshots included, was re-appended to the transcript under a new prompt id; both copies then read each other's writes as work done elsewhere, and the adopted one was billed for re-reading all of it. Meanwhile `shutdown_all` asked each session to stop and the process exited immediately, so the SIGKILL timer died with the runtime, the stop was unreliable, and whatever survived was orphaned with nothing written down to find it by. The processes leaked either way. So leak them on purpose, and be able to pick them back up. A session's process now outlives the backend and is adopted again on the way up, which is worth having for its own sake: restarting the server no longer ends a turn somebody is waiting on. Its stdio lives in the session directory -- a fifo opened read-write so the process is its own last writer and never reads EOF, plus stdout/stderr logs read from a byte offset. `session::process` records the pid *and* the kernel's start time for it, because a pid alone is reused and adopting a stranger's would mean never resuming the real conversation. That makes the fix structural rather than a check: everything goes through `ClaudeDriver::launch`, which adopts if it can and starts if it cannot, and `--resume` is reachable only on the second path. `Driver` gains two ways out where it had one -- `detach` (coming back) and `stop` (the session is being deleted, so the process must not survive). Importing a session that is open is now refused outright. Claude Code keeps `~/.claude/sessions/<pid>.json` for every live session, so this is a measurement rather than a guess; it reports no/yes/unknown, because a machine that keeps no such record cannot answer and "could not check" is not "nobody is using it". `SessionStatus` gains `Unknown` for the same reason. Also here, found on the way: - A reconnecting phone was sent the entire backlog. Opening a session was bounded to a page but reconnecting was not, so a long disconnect delivered thousands of events one frame at a time. Past `CATCH_UP_LIMIT` the stream sends a `reset` frame and the newest window, and the client rebuilds from it as it does on open -- without the reset the window is spliced onto rows no longer adjacent to it. - A session's status was assumed idle at launch. Read from the transcript instead, so a restart stops claiming an exited session is waiting for you. - `llama-server`'s stdout was piped and never drained, so a chatty one blocked on a full pipe buffer mid-load. It goes to a log now. - A turn that exited or errored never emitted `Idle`, so the queue stayed "running" for good: every later message was held forever and, since a message is only recorded when taken, vanished with nothing on screen. - Two doc comments had drifted onto the wrong functions. Verified by killing the server mid-turn: the process survived, finished its turn unattended (12.8 KB of output nothing was reading), and the restarted server adopted it -- one process, all 700 lines in the transcript, no hole, and it still took a new message afterwards. Deleting a session stops its process; a 266-event backlog resets while a 16-event one streams. 46 tests, clippy and rustfmt clean, app compiles and lints. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VETa8afmpWaYezLCqJhDB8
This commit is contained in:
1 parent
9791afcfd6
commit
362d436d4f
23 files changed
+1934
-345
No files matched your search
+115
-14
@@ -3,13 +3,28 @@
|
||||
//! interrupts -- before any AI is involved, and stays useful afterwards as
|
||||
//! a connectivity check that costs no tokens.
|
||||
//!
|
||||
//! Behavior: every message is echoed back as a few streamed text deltas. A
|
||||
//! message starting with `/tool` also emits a fake tool run, and one
|
||||
//! starting with `/question` asks one (exercising the answer path). This is
|
||||
//! exactly the event vocabulary the real drivers produce, so a UI that
|
||||
//! renders echo sessions correctly renders the real thing.
|
||||
//! Behavior: every message is echoed back as a few streamed text deltas.
|
||||
//! A leading word asks for something more specific:
|
||||
//!
|
||||
//! - `/tool [input]` -- a full tool run, start through end.
|
||||
//! - `/question [text]` -- a question, exercising the answer path.
|
||||
//! - `/slow [seconds]` -- a turn that stays running (default 30), so states that only
|
||||
//! exist *while* something is happening can be looked at.
|
||||
//! - `/error [text]` -- a failure, which is otherwise awkward to cause.
|
||||
//!
|
||||
//! This is exactly the event vocabulary the real drivers produce, so a UI
|
||||
//! that renders echo sessions correctly renders the real thing.
|
||||
//!
|
||||
//! `/slow` earns its place: a queued message, a Stop button, a spinner
|
||||
//! where the answer will go are all states that only exist mid-turn, and
|
||||
//! the obvious way to get one -- ask a real model to sleep -- does not
|
||||
//! work. It declines, reasonably, and answers instantly instead, so the
|
||||
//! state never arrives and the attempt still costs a turn on somebody's
|
||||
//! account. A driver that can be *told* to take its time costs nothing and
|
||||
//! is the same every run.
|
||||
|
||||
use std::sync::Mutex;
|
||||
use std::sync::atomic::{AtomicBool, Ordering};
|
||||
use std::sync::{Arc, Mutex};
|
||||
use std::time::Duration;
|
||||
|
||||
use super::driver::{Driver, Event, EventSink, ImageRef, SessionStatus};
|
||||
@@ -21,6 +36,15 @@ const DELTA_DELAY: Duration = Duration::from_millis(50);
|
||||
|
||||
pub struct EchoDriver {
|
||||
sink: EventSink,
|
||||
/// Whether a turn is in flight, and what arrived during it.
|
||||
///
|
||||
/// A real CLI holds a message sent mid-turn and injects it at the next
|
||||
/// tool boundary; echo used to answer it on the spot, which made it
|
||||
/// the wrong shape for testing anything about queueing -- the status
|
||||
/// dropped to idle immediately, so a phone had nothing to show as
|
||||
/// pending. Holding it here is what makes echo able to stand in.
|
||||
busy: Arc<AtomicBool>,
|
||||
queued: Arc<Mutex<Vec<String>>>,
|
||||
/// Id of the question currently awaiting an answer, if any. One at a
|
||||
/// time is all the echo behavior ever produces.
|
||||
pending_question: Mutex<Option<String>>,
|
||||
@@ -31,6 +55,8 @@ impl EchoDriver {
|
||||
let driver = Self {
|
||||
sink,
|
||||
pending_question: Mutex::new(None),
|
||||
busy: Arc::new(AtomicBool::new(false)),
|
||||
queued: Arc::new(Mutex::new(Vec::new())),
|
||||
};
|
||||
driver.emit(Event::Status {
|
||||
state: SessionStatus::Idle,
|
||||
@@ -50,6 +76,15 @@ impl Driver for EchoDriver {
|
||||
fn send_user_message(&self, text: String, _images: Vec<ImageRef>) {
|
||||
let sink = self.sink.clone();
|
||||
|
||||
// Mid-turn messages are held rather than answered, the way a real
|
||||
// CLI holds them until the next tool boundary. Without this the
|
||||
// session went idle the instant one arrived, and every state that
|
||||
// only exists while something is queued was untestable.
|
||||
if self.busy.load(Ordering::SeqCst) {
|
||||
self.queued.lock().unwrap().push(text);
|
||||
return;
|
||||
}
|
||||
|
||||
if let Some(rest) = text.strip_prefix("/question") {
|
||||
let id = format!("q-{}", super::random_hex());
|
||||
let prompt = if rest.trim().is_empty() {
|
||||
@@ -75,14 +110,82 @@ impl Driver for EchoDriver {
|
||||
let run_tool = text
|
||||
.strip_prefix("/tool")
|
||||
.map(|rest| rest.trim().to_string());
|
||||
// Seconds to stay running before answering, default 30. Clamped
|
||||
// rather than trusted: this is a test affordance, and a session
|
||||
// pinned running for an hour by a typo is a worse outcome than a
|
||||
// short wait.
|
||||
let linger = text.strip_prefix("/slow").map(|rest| {
|
||||
Duration::from_secs(rest.trim().parse::<u64>().unwrap_or(30).clamp(1, 600))
|
||||
});
|
||||
let fail = text
|
||||
.strip_prefix("/error")
|
||||
.map(|rest| rest.trim().to_string());
|
||||
let busy = Arc::clone(&self.busy);
|
||||
let queued = Arc::clone(&self.queued);
|
||||
busy.store(true, Ordering::SeqCst);
|
||||
tokio::spawn(async move {
|
||||
let send = |event: Event| {
|
||||
let _ = sink.send(event);
|
||||
};
|
||||
// Ending a turn is also when anything held during it is taken
|
||||
// up -- the moment a real CLI would have injected it. One
|
||||
// place, because a turn has several ways to end and every one
|
||||
// of them owes the same answer.
|
||||
let finish = || {
|
||||
let held = std::mem::take(&mut *queued.lock().unwrap());
|
||||
for text in held {
|
||||
// Announced before it is answered, in that order: a
|
||||
// phone showing the message as pending needs the
|
||||
// signal that it has been read, and the answer is
|
||||
// meaningless above a message still drawn as waiting.
|
||||
send(Event::MessageTaken { text: text.clone() });
|
||||
send(Event::AssistantText {
|
||||
delta: format!("\n(taken from the queue) You said: {text}"),
|
||||
});
|
||||
}
|
||||
busy.store(false, Ordering::SeqCst);
|
||||
send(Event::Status {
|
||||
state: SessionStatus::Idle,
|
||||
});
|
||||
};
|
||||
// Echo takes a message the instant it gets one, but it says so
|
||||
// anyway: a driver that skips this leaves the phone holding a
|
||||
// message it thinks is still queued, and the point of an echo
|
||||
// provider is that it behaves like the real ones.
|
||||
send(Event::MessageTaken { text: text.clone() });
|
||||
send(Event::Status {
|
||||
state: SessionStatus::Running,
|
||||
});
|
||||
|
||||
if let Some(linger) = linger {
|
||||
// A delta a second: visibly alive rather than merely slow,
|
||||
// which is what the states being looked at accompany.
|
||||
let seconds = linger.as_secs();
|
||||
for remaining in (1..=seconds).rev() {
|
||||
send(Event::AssistantText {
|
||||
delta: format!("still working, {remaining}s\n"),
|
||||
});
|
||||
tokio::time::sleep(Duration::from_secs(1)).await;
|
||||
}
|
||||
send(Event::AssistantText {
|
||||
delta: "done.".to_string(),
|
||||
});
|
||||
finish();
|
||||
return;
|
||||
}
|
||||
|
||||
if let Some(message) = fail {
|
||||
send(Event::Error {
|
||||
message: if message.is_empty() {
|
||||
"echo was asked to fail".to_string()
|
||||
} else {
|
||||
message
|
||||
},
|
||||
});
|
||||
finish();
|
||||
return;
|
||||
}
|
||||
|
||||
if let Some(input) = run_tool {
|
||||
let id = format!("t-{}", super::random_hex());
|
||||
send(Event::ToolStart {
|
||||
@@ -112,9 +215,7 @@ impl Driver for EchoDriver {
|
||||
send(Event::UsageDelta {
|
||||
tokens: text.split_whitespace().count() as u64,
|
||||
});
|
||||
send(Event::Status {
|
||||
state: SessionStatus::Idle,
|
||||
});
|
||||
finish();
|
||||
});
|
||||
}
|
||||
|
||||
@@ -163,9 +264,9 @@ impl Driver for EchoDriver {
|
||||
});
|
||||
}
|
||||
|
||||
fn shutdown(&self) {
|
||||
self.emit(Event::Status {
|
||||
state: SessionStatus::Exited,
|
||||
});
|
||||
}
|
||||
/// Nothing to detach from and nothing to stop: the echo driver has no
|
||||
/// process, so both halves of the way out are already done.
|
||||
fn detach(&self) {}
|
||||
|
||||
fn stop(&self) {}
|
||||
}
|
||||
Reference in new issue
Block a user