Let sessions outlive the backend, and never resume one twice
Three `claude` processes ended up running against this checkout on 2026-08-29, and the account hit its session limit. One cause, several ways in. An agent imported the Claude Code session it was *itself* running in. That is an ordinary import, and importing runs `--resume` -- so a second CLI attached to a file the first was still writing. The whole 65 MB conversation, 154 embedded screenshots included, was re-appended to the transcript under a new prompt id; both copies then read each other's writes as work done elsewhere, and the adopted one was billed for re-reading all of it. Meanwhile `shutdown_all` asked each session to stop and the process exited immediately, so the SIGKILL timer died with the runtime, the stop was unreliable, and whatever survived was orphaned with nothing written down to find it by. The processes leaked either way. So leak them on purpose, and be able to pick them back up. A session's process now outlives the backend and is adopted again on the way up, which is worth having for its own sake: restarting the server no longer ends a turn somebody is waiting on. Its stdio lives in the session directory -- a fifo opened read-write so the process is its own last writer and never reads EOF, plus stdout/stderr logs read from a byte offset. `session::process` records the pid *and* the kernel's start time for it, because a pid alone is reused and adopting a stranger's would mean never resuming the real conversation. That makes the fix structural rather than a check: everything goes through `ClaudeDriver::launch`, which adopts if it can and starts if it cannot, and `--resume` is reachable only on the second path. `Driver` gains two ways out where it had one -- `detach` (coming back) and `stop` (the session is being deleted, so the process must not survive). Importing a session that is open is now refused outright. Claude Code keeps `~/.claude/sessions/<pid>.json` for every live session, so this is a measurement rather than a guess; it reports no/yes/unknown, because a machine that keeps no such record cannot answer and "could not check" is not "nobody is using it". `SessionStatus` gains `Unknown` for the same reason. Also here, found on the way: - A reconnecting phone was sent the entire backlog. Opening a session was bounded to a page but reconnecting was not, so a long disconnect delivered thousands of events one frame at a time. Past `CATCH_UP_LIMIT` the stream sends a `reset` frame and the newest window, and the client rebuilds from it as it does on open -- without the reset the window is spliced onto rows no longer adjacent to it. - A session's status was assumed idle at launch. Read from the transcript instead, so a restart stops claiming an exited session is waiting for you. - `llama-server`'s stdout was piped and never drained, so a chatty one blocked on a full pipe buffer mid-load. It goes to a log now. - A turn that exited or errored never emitted `Idle`, so the queue stayed "running" for good: every later message was held forever and, since a message is only recorded when taken, vanished with nothing on screen. - Two doc comments had drifted onto the wrong functions. Verified by killing the server mid-turn: the process survived, finished its turn unattended (12.8 KB of output nothing was reading), and the restarted server adopted it -- one process, all 700 lines in the transcript, no hole, and it still took a new message afterwards. Deleting a session stops its process; a 266-event backlog resets while a 16-event one streams. 46 tests, clippy and rustfmt clean, app compiles and lints. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VETa8afmpWaYezLCqJhDB8
This commit is contained in:
1 parent
9791afcfd6
commit
362d436d4f
23 files changed
+1934
-345
No files matched your search
+60
-25
@@ -14,6 +14,7 @@ pub mod driver;
|
||||
pub mod echo;
|
||||
pub mod import;
|
||||
pub mod llama;
|
||||
pub mod process;
|
||||
pub mod transcript;
|
||||
pub mod transport;
|
||||
|
||||
@@ -136,18 +137,22 @@ struct Shared {
|
||||
}
|
||||
|
||||
impl LiveSession {
|
||||
/// Records the user's message in the transcript, then hands it to the
|
||||
/// driver -- which queues it for injection mid-run rather than at the
|
||||
/// end of the turn (the point of the whole app).
|
||||
/// Hands the user's message to the driver, which records it in the
|
||||
/// transcript by reporting that it has taken it -- see `MessageTaken`.
|
||||
///
|
||||
/// The message is deliberately not recorded here. Sent into a running
|
||||
/// turn it waits, and writing it down on the way past would put it
|
||||
/// above output that happened before the session ever saw it.
|
||||
pub fn send_message(&self, text: String, images: Vec<ImageRef>) {
|
||||
// Attachments render in the transcript like any produced image --
|
||||
// the files route serves uploads by the same ref.
|
||||
// Attachments are the exception, recorded on the way past: they
|
||||
// are uploaded whether or not the message waits, and the phone
|
||||
// fetches them by the same ref the files route serves. So a queued
|
||||
// message's picture appears a little before its text.
|
||||
for image in &images {
|
||||
let _ = self.sink.send(Event::Image {
|
||||
image: image.clone(),
|
||||
});
|
||||
}
|
||||
let _ = self.sink.send(Event::UserMessage { text: text.clone() });
|
||||
self.driver.send_user_message(text, images);
|
||||
}
|
||||
|
||||
@@ -163,9 +168,11 @@ impl LiveSession {
|
||||
self.driver.interrupt();
|
||||
}
|
||||
|
||||
/// Stops this session's process without deleting anything.
|
||||
pub fn shutdown(&self) {
|
||||
self.driver.shutdown();
|
||||
/// Leaves this session's process running and stops attending to it,
|
||||
/// for a server that is going away and means to come back. See
|
||||
/// [`Driver::detach`].
|
||||
pub fn detach(&self) {
|
||||
self.driver.detach();
|
||||
}
|
||||
|
||||
pub fn compact(&self) {
|
||||
@@ -460,22 +467,30 @@ impl SessionManager {
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Every session, in config order, with live status joined in. A
|
||||
/// session that failed to relaunch reports as exited.
|
||||
/// Stops every session's process, for a server that is going away.
|
||||
/// Lets go of every session's process, for a server that is going
|
||||
/// away and means to adopt them again when it comes back.
|
||||
///
|
||||
/// Drivers set `kill_on_drop`, which covers a session being deleted
|
||||
/// while the server keeps running -- but not the server itself being
|
||||
/// signalled, because nothing drops on the way out of a SIGTERM. That
|
||||
/// leaves the children orphaned, which for a `llama-server` holding a
|
||||
/// model means gigabytes of memory nobody owns any more. So exiting
|
||||
/// asks them all to stop first.
|
||||
pub fn shutdown_all(&self) {
|
||||
/// Deliberately not a shutdown, and this is the load-bearing half of
|
||||
/// it: a backend restart -- a rebuild, a service restart, a crash --
|
||||
/// must not end a turn somebody is waiting on. Each process keeps its
|
||||
/// record in the session directory, and `launch` finds it there rather
|
||||
/// than starting a second one against the same conversation.
|
||||
///
|
||||
/// What this did before was ask them all to stop and then exit
|
||||
/// immediately, which stopped nothing reliably -- the grace timer died
|
||||
/// with the runtime -- and orphaned whatever survived with nothing
|
||||
/// written down to find it by. Processes leaked either way; what is
|
||||
/// different now is that they are left on purpose and can be picked
|
||||
/// back up.
|
||||
pub fn detach_all(&self) {
|
||||
let inner = self.inner.read().unwrap();
|
||||
for session in inner.live.values() {
|
||||
session.shutdown();
|
||||
session.detach();
|
||||
}
|
||||
tracing::info!("stopped {} session process(es)", inner.live.len());
|
||||
tracing::info!(
|
||||
"left {} session process(es) running to be reattached to",
|
||||
inner.live.len()
|
||||
);
|
||||
}
|
||||
|
||||
/// The session already continuing `source`, if there is one.
|
||||
@@ -499,6 +514,8 @@ impl SessionManager {
|
||||
})
|
||||
}
|
||||
|
||||
/// Every session, in config order, with live status joined in. A
|
||||
/// session that failed to relaunch reports as exited.
|
||||
pub fn sessions(&self) -> Vec<SessionInfo> {
|
||||
let inner = self.inner.read().unwrap();
|
||||
inner
|
||||
@@ -695,7 +712,10 @@ impl SessionManager {
|
||||
candidate.save(&self.config_path)?;
|
||||
inner.config = candidate;
|
||||
if let Some(session) = inner.live.remove(id) {
|
||||
session.driver.shutdown();
|
||||
// Stopped, not detached: this is the one exit where the
|
||||
// process must not survive, because the conversation it
|
||||
// belongs to is being removed. See `Driver::stop`.
|
||||
session.driver.stop();
|
||||
}
|
||||
let dir = self.data_dir.join(id);
|
||||
if dir.exists() {
|
||||
@@ -878,7 +898,13 @@ fn launch(
|
||||
let (sink, source) = mpsc::unbounded_channel();
|
||||
let (events, _) = broadcast::channel(EVENT_BUFFER);
|
||||
let shared = Arc::new(Shared {
|
||||
status: Mutex::new(SessionStatus::Idle),
|
||||
// What it was last known to be doing, not an assumption. A driver
|
||||
// that has something to say corrects this within its first poll;
|
||||
// one adopting a process that has been quiet says nothing, and
|
||||
// this is then the only true answer available.
|
||||
status: Mutex::new(
|
||||
transcript::last_status(&transcript_path).unwrap_or(SessionStatus::Idle),
|
||||
),
|
||||
last_activity: Mutex::new(now()),
|
||||
model: Mutex::new(meta.model.clone()),
|
||||
permission_mode: Mutex::new(meta.permission_mode.clone()),
|
||||
@@ -901,15 +927,16 @@ fn launch(
|
||||
|
||||
let driver: Box<dyn Driver> = match provider.kind {
|
||||
DriverKind::Echo => Box::new(EchoDriver::new(sink.clone())),
|
||||
DriverKind::LlamaCpp => Box::new(LlamaDriver::spawn(
|
||||
DriverKind::LlamaCpp => Box::new(LlamaDriver::launch(
|
||||
&meta,
|
||||
provider,
|
||||
&Transport::for_setup(setup),
|
||||
models_dir,
|
||||
&transcript_path,
|
||||
&dir,
|
||||
sink.clone(),
|
||||
)?),
|
||||
DriverKind::ClaudeCli => Box::new(ClaudeDriver::spawn(
|
||||
DriverKind::ClaudeCli => Box::new(ClaudeDriver::launch(
|
||||
&meta,
|
||||
provider,
|
||||
&Transport::for_setup(setup),
|
||||
@@ -951,6 +978,14 @@ async fn pump(
|
||||
) {
|
||||
while let Some(event) = source.recv().await {
|
||||
let ts = now();
|
||||
// Taking a message is how it enters the conversation, and the
|
||||
// conversation is what a phone renders -- so the event becomes the
|
||||
// message here rather than being carried alongside it. One rule
|
||||
// for where a user's message sits: where the session read it.
|
||||
let event = match event {
|
||||
Event::MessageTaken { text } => Event::UserMessage { text },
|
||||
other => other,
|
||||
};
|
||||
match transcript.append(event, ts) {
|
||||
Ok(entry) => {
|
||||
if let Event::Status { state } = &entry.event {
|
||||
|
||||
Reference in new issue
Block a user