Show a replayed session's images instead of dropping them

An imported session showed no screenshots. `text_of` kept only `text`
blocks, so every image in the replayed tail was silently discarded -- while
the *live* translator has always saved them into the session's `files/` and
referenced them. Two readings of the same records, and the one used for
history was the lesser.

`save_image` moves out of `Translator` to a free function both paths call,
since the naming scheme for that directory should exist once. `events_from`
now takes the session directory to write into, which means the conversion
has to happen where that directory exists -- so `Seed` carries the raw
JSONL and `launch` turns it into events, rather than `routes` doing it
before the session is created.

Costs nothing in tokens, which is the point worth recording: this writes
into ai-app's own session directory and the phone fetches a reference only
when it draws one. Nothing here is ever written to the CLI's stdin -- it
reads its own session file, and the only things this app sends it are typed
messages, control requests and `/compact`.

Verified against the 133 MB session behind the 2026-08-29 incident: 45
images in the replayed tail, written as real PNGs and served over the files
route, with the transcript itself staying at 756 KB of references.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VETa8afmpWaYezLCqJhDB8
This commit is contained in:
irisandClaude Opus 5 committed 2026-08-29 05:01:48 -04:00
1 parent d96bc041a7
commit d7c692a4ec
6 files changed
+148 -50

No files matched your search

+5
View File
@@ -41,6 +41,11 @@ repo is in PLAN.md's "Backend layout" section.
`POST /setups`. Only the tail is replayed (`REPLAY_LINES`) because these
files reach tens of megabytes and the CLI reads the real one itself; what
crosses the tunnel is what a person reads, not what the model is given.
Images in the replayed tail are written into the session's `files/` by
the same function the live translator uses, and referenced -- so a
screenshot looks the same whether it was watched happening or replayed
afterwards, the phone fetches the bytes only when it draws one, and none
of it goes anywhere near the CLI, which reads its own file.
An imported session then **keeps itself level with that file**, so work
done at a terminal appears without anyone pressing anything. `--resume`
appends to the same transcript rather than forking — measured, but
+6 -6
View File
@@ -478,7 +478,7 @@ async fn spawn_session(
whole thing. Close it there first, then import it here."
)));
}
let events = crate::session::import::replay(&transport, &chosen.path)
let records = crate::session::import::read_tail(&transport, &chosen.path)
.await
.map_err(bad_request)?;
// The recorded directory can outlive itself; resuming into one
@@ -495,7 +495,7 @@ async fn spawn_session(
);
chosen.cwd = String::new();
}
Some((chosen, events))
Some((chosen, records))
}
None => None,
};
@@ -532,11 +532,11 @@ async fn spawn_session(
};
let info = match seed {
Some((chosen, events)) => {
Some((chosen, records)) => {
tracing::info!(
"importing Claude Code session {} ({} events replayed)",
"importing Claude Code session {} ({} lines replayed)",
chosen.id,
events.len()
records.lines().count()
);
manager.spawn_imported(
spec,
@@ -549,7 +549,7 @@ async fn spawn_session(
path: chosen.path,
lines: chosen.lines,
},
events,
records,
},
)
}
+1 -1
View File
@@ -96,7 +96,7 @@ fn tail_of(kept: &VecDeque<String>) -> String {
/// the conversation back up from Claude's own session files. Kept in the
/// session directory rather than config.ron so the shared schema stays
/// free of per-driver state.
mod translate;
pub(super) mod translate;
const RESUME_FILE: &str = "claude-session.json";
+33 -27
View File
@@ -13,7 +13,7 @@
//! without a process.
use std::collections::HashMap;
use std::path::PathBuf;
use std::path::{Path, PathBuf};
use serde_json::{Value, json};
@@ -307,7 +307,7 @@ impl Translator {
}
}
Some("image") => {
if let Some(name) = self.save_image(part) {
if let Some(name) = save_image(&self.session_dir, part) {
events.push(Event::Image { image: name });
}
}
@@ -328,33 +328,39 @@ impl Translator {
}
events
}
}
/// Decodes one base64 image block into `files/` and returns its ref.
fn save_image(&self, part: &Value) -> Option<String> {
let source = part.get("source")?;
let data = source.get("data")?.as_str()?;
use base64::Engine;
let bytes = base64::engine::general_purpose::STANDARD
.decode(data)
.ok()?;
// Screenshots are the overwhelming case, and they are PNG; an
// unrecognized type is more likely a dialect change than a JPEG.
let extension = source
.get("media_type")
.and_then(Value::as_str)
.and_then(crate::media::extension_for)
.unwrap_or("png");
let name = format!("{}.{extension}", super::super::random_hex());
let dir = self.session_dir.join("files");
if let Err(err) = wg_app_link::private::create_dir(&dir)
.map_err(std::io::Error::other)
.and_then(|()| std::fs::write(dir.join(&name), bytes))
{
tracing::error!("couldn't save produced image: {err}");
return None;
}
Some(name)
/// Decodes one base64 image block into `files/` and returns its ref.
///
/// A free function rather than a method because the import replay needs
/// exactly this too: a session's history carries the same image blocks as
/// its live output, and a reader who can see a screenshot while it happens
/// should still see it after a restart. Two copies of this would be two
/// naming schemes for one directory.
pub(in crate::session) fn save_image(session_dir: &Path, part: &Value) -> Option<String> {
let source = part.get("source")?;
let data = source.get("data")?.as_str()?;
use base64::Engine;
let bytes = base64::engine::general_purpose::STANDARD
.decode(data)
.ok()?;
// Screenshots are the overwhelming case, and they are PNG; an
// unrecognized type is more likely a dialect change than a JPEG.
let extension = source
.get("media_type")
.and_then(Value::as_str)
.and_then(crate::media::extension_for)
.unwrap_or("png");
let name = format!("{}.{extension}", super::super::random_hex());
let dir = session_dir.join("files");
if let Err(err) = wg_app_link::private::create_dir(&dir)
.map_err(std::io::Error::other)
.and_then(|()| std::fs::write(dir.join(&name), bytes))
{
tracing::error!("couldn't save produced image: {err}");
return None;
}
Some(name)
}
#[cfg(test)]
+93 -10
View File
@@ -317,21 +317,26 @@ pub async fn directory_exists(transport: &Transport, path: &str) -> bool {
transport.capture(&launch).await.is_ok()
}
/// Reads the tail of one session's file and turns it into events.
/// Reads the tail of one session's file, as the raw JSONL.
///
/// `tail` rather than the whole file, and as [`Launch`] arguments rather
/// than a shell string, so the path is an argument and never syntax.
pub async fn replay(transport: &Transport, path: &str) -> Result<Vec<Event>> {
///
/// Returns text rather than events because turning records into events has
/// a side effect -- writing out the images they carry -- and it needs the
/// session directory to write them into. That directory does not exist
/// until the session is created, which is after this runs, so the
/// conversion happens there instead. See [`events_from`].
pub async fn read_tail(transport: &Transport, path: &str) -> Result<String> {
let launch = Launch::new(
"tail",
vec!["-n".to_string(), REPLAY_LINES.to_string(), path.to_string()],
None,
);
let text = transport
transport
.capture(&launch)
.await
.with_context(|| format!("reading {path}"))?;
Ok(events_from(&text))
.with_context(|| format!("reading {path}"))
}
/// Claude Code's stored JSONL as this project's events.
@@ -339,7 +344,15 @@ pub async fn replay(transport: &Transport, path: &str) -> Result<Vec<Event>> {
/// A partial first line is expected and ignored: `tail -n` cuts at a line
/// boundary, but the *file* may have been appended to since, and a line
/// that does not parse is one this reader has no opinion about.
pub fn events_from(text: &str) -> Vec<Event> {
///
/// `session_dir` is where images found along the way are written, the same
/// place and by the same function the live translator uses -- so a
/// screenshot looks identical whether it was watched as it happened or
/// replayed afterwards. It is only the *reference* that reaches the phone;
/// the bytes are fetched from `/sessions/{id}/files/{ref}` when something
/// actually draws them, and none of this is ever sent back to the CLI,
/// which reads its own session file.
pub fn events_from(text: &str, session_dir: &std::path::Path) -> Vec<Event> {
let mut events = Vec::new();
for line in text.lines() {
let Ok(record) = serde_json::from_str::<Value>(line) else {
@@ -355,7 +368,7 @@ pub fn events_from(text: &str) -> Vec<Event> {
continue;
};
match record.get("type").and_then(Value::as_str) {
Some("user") => push_user(&mut events, content),
Some("user") => push_user(&mut events, content, session_dir),
Some("assistant") => push_assistant(&mut events, content),
_ => {}
}
@@ -363,16 +376,25 @@ pub fn events_from(text: &str) -> Vec<Event> {
events
}
fn push_user(events: &mut Vec<Event>, content: &Value) {
fn push_user(events: &mut Vec<Event>, content: &Value, session_dir: &std::path::Path) {
// A tool result arrives as a user record, because that is how the API
// models it -- but it is the other half of a tool call, not something
// a person said, and showing it as a message would put the reader's
// own words and a command's output in the same voice.
if let Value::Array(blocks) = content {
for block in blocks {
// A picture the person attached to their own message, rather
// than one a tool produced. Same block shape, one level up.
push_images(events, std::slice::from_ref(block), session_dir);
if block.get("type").and_then(Value::as_str) == Some("tool_result")
&& let Some(id) = block.get("tool_use_id").and_then(Value::as_str)
{
// Before the tool's own row, matching the live translator:
// a screenshot belongs to the call that took it, and after
// the result it reads as belonging to whatever came next.
if let Some(Value::Array(parts)) = block.get("content") {
push_images(events, parts, session_dir);
}
events.push(Event::ToolEnd {
id: id.to_string(),
output: text_of(block.get("content").unwrap_or(&Value::Null)),
@@ -386,6 +408,17 @@ fn push_user(events: &mut Vec<Event>, content: &Value) {
}
}
/// Saves every image block in `parts` and references each one.
fn push_images(events: &mut Vec<Event>, parts: &[Value], session_dir: &std::path::Path) {
for part in parts {
if part.get("type").and_then(Value::as_str) == Some("image")
&& let Some(name) = super::claude::translate::save_image(session_dir, part)
{
events.push(Event::Image { image: name });
}
}
}
fn push_assistant(events: &mut Vec<Event>, content: &Value) {
let Value::Array(blocks) = content else {
return;
@@ -501,7 +534,12 @@ pub async fn line_count(transport: &Transport, path: &str) -> Result<usize> {
/// Events from the lines after `after`, which is a 0-based count of lines
/// already accounted for.
pub async fn replay_after(transport: &Transport, path: &str, after: usize) -> Result<Vec<Event>> {
pub async fn replay_after(
transport: &Transport,
path: &str,
after: usize,
session_dir: &std::path::Path,
) -> Result<Vec<Event>> {
let launch = Launch::new(
"tail",
vec![format!("-n+{}", after + 1), path.to_string()],
@@ -511,5 +549,50 @@ pub async fn replay_after(transport: &Transport, path: &str, after: usize) -> Re
.capture(&launch)
.await
.with_context(|| format!("reading {path} from line {}", after + 1))?;
Ok(events_from(&text))
Ok(events_from(&text, session_dir))
}
#[cfg(test)]
mod tests {
use super::*;
/// A 1x1 PNG, base64 -- the smallest thing with a real header.
const PNG: &str = "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mNk\
YPhfDwAChwGA60e6kgAAAABJRU5ErkJggg==";
#[test]
fn replayed_screenshots_are_saved_and_referenced() {
let dir = tempfile::tempdir().expect("tempdir");
// The shape a screenshot actually has in these files: an image
// part inside a tool result, beside its text.
let line = format!(
r#"{{"type":"user","message":{{"role":"user","content":[{{"type":"tool_result","tool_use_id":"toolu_1","content":[{{"type":"text","text":"took a screenshot"}},{{"type":"image","source":{{"type":"base64","media_type":"image/png","data":"{PNG}"}}}}]}}]}}}}"#
);
let events = events_from(&line, dir.path());
let Some(Event::Image { image }) = events.first() else {
panic!("a replayed screenshot must become an image event: {events:?}");
};
assert!(image.ends_with(".png"));
// On disk, where the files route serves it from -- the phone
// fetches it only when something draws it.
assert!(dir.path().join("files").join(image).is_file());
// And it comes before the tool row it belongs to, so it does not
// read as belonging to whatever happened next.
assert!(
matches!(events.get(1), Some(Event::ToolEnd { .. })),
"{events:?}"
);
}
#[test]
fn a_record_with_no_image_writes_nothing() {
let dir = tempfile::tempdir().expect("tempdir");
let line = r#"{"type":"user","message":{"role":"user","content":[{"type":"tool_result","tool_use_id":"toolu_1","content":"plain output"}]}}"#;
let events = events_from(line, dir.path());
assert_eq!(events.len(), 1, "{events:?}");
// No stray directory for a session that never produced one.
assert!(!dir.path().join("files").exists());
}
}
+10 -6
View File
@@ -831,7 +831,7 @@ fn spawn_import_sync(
if lines <= cursor.lines {
continue;
}
match import::replay_after(&transport, &cursor.path, cursor.lines).await {
match import::replay_after(&transport, &cursor.path, cursor.lines, &dir).await {
Ok(events) => {
tracing::info!(
"{} grew by {} lines with nothing from here; replaying {} events",
@@ -866,10 +866,14 @@ pub struct Seed {
/// Where that session's file is and how much of it has been shown, so
/// the session can keep itself up to date afterwards.
pub cursor: import::Cursor,
/// Replayed into the transcript so the phone shows the conversation it
/// is joining. The CLI reads the real file itself, so this is what the
/// reader sees rather than what the model is given.
pub events: Vec<Event>,
/// The tail of that session's file, as the raw JSONL.
///
/// Turned into events in `launch`, not before, because doing so writes
/// out the images the records carry and that needs the session
/// directory to write them into -- which does not exist until the
/// session does. The CLI reads the real file itself, so this only ever
/// decides what the *reader* sees.
pub records: String,
}
fn launch(
@@ -890,7 +894,7 @@ fn launch(
claude::write_resume_token(&dir, &seed.resume);
import::write_cursor(&dir, &seed.cursor);
let at = now();
for event in seed.events {
for event in import::events_from(&seed.records, &dir) {
transcript.append(event, at)?;
}
}