Show a replayed session's images instead of dropping them

An imported session showed no screenshots. `text_of` kept only `text`
blocks, so every image in the replayed tail was silently discarded -- while
the *live* translator has always saved them into the session's `files/` and
referenced them. Two readings of the same records, and the one used for
history was the lesser.

`save_image` moves out of `Translator` to a free function both paths call,
since the naming scheme for that directory should exist once. `events_from`
now takes the session directory to write into, which means the conversion
has to happen where that directory exists -- so `Seed` carries the raw
JSONL and `launch` turns it into events, rather than `routes` doing it
before the session is created.

Costs nothing in tokens, which is the point worth recording: this writes
into ai-app's own session directory and the phone fetches a reference only
when it draws one. Nothing here is ever written to the CLI's stdin -- it
reads its own session file, and the only things this app sends it are typed
messages, control requests and `/compact`.

Verified against the 133 MB session behind the 2026-08-29 incident: 45
images in the replayed tail, written as real PNGs and served over the files
route, with the transcript itself staying at 756 KB of references.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VETa8afmpWaYezLCqJhDB8
This commit is contained in:
irisandClaude Opus 5 committed 2026-08-29 05:01:48 -04:00
1 parent d96bc041a7
commit d7c692a4ec
6 files changed
+148 -50

No files matched your search

+1 -1
View File
@@ -96,7 +96,7 @@ fn tail_of(kept: &VecDeque<String>) -> String {
/// the conversation back up from Claude's own session files. Kept in the
/// session directory rather than config.ron so the shared schema stays
/// free of per-driver state.
mod translate;
pub(super) mod translate;
const RESUME_FILE: &str = "claude-session.json";
+33 -27
View File
@@ -13,7 +13,7 @@
//! without a process.
use std::collections::HashMap;
use std::path::PathBuf;
use std::path::{Path, PathBuf};
use serde_json::{Value, json};
@@ -307,7 +307,7 @@ impl Translator {
}
}
Some("image") => {
if let Some(name) = self.save_image(part) {
if let Some(name) = save_image(&self.session_dir, part) {
events.push(Event::Image { image: name });
}
}
@@ -328,33 +328,39 @@ impl Translator {
}
events
}
}
/// Decodes one base64 image block into `files/` and returns its ref.
fn save_image(&self, part: &Value) -> Option<String> {
let source = part.get("source")?;
let data = source.get("data")?.as_str()?;
use base64::Engine;
let bytes = base64::engine::general_purpose::STANDARD
.decode(data)
.ok()?;
// Screenshots are the overwhelming case, and they are PNG; an
// unrecognized type is more likely a dialect change than a JPEG.
let extension = source
.get("media_type")
.and_then(Value::as_str)
.and_then(crate::media::extension_for)
.unwrap_or("png");
let name = format!("{}.{extension}", super::super::random_hex());
let dir = self.session_dir.join("files");
if let Err(err) = wg_app_link::private::create_dir(&dir)
.map_err(std::io::Error::other)
.and_then(|()| std::fs::write(dir.join(&name), bytes))
{
tracing::error!("couldn't save produced image: {err}");
return None;
}
Some(name)
/// Decodes one base64 image block into `files/` and returns its ref.
///
/// A free function rather than a method because the import replay needs
/// exactly this too: a session's history carries the same image blocks as
/// its live output, and a reader who can see a screenshot while it happens
/// should still see it after a restart. Two copies of this would be two
/// naming schemes for one directory.
pub(in crate::session) fn save_image(session_dir: &Path, part: &Value) -> Option<String> {
let source = part.get("source")?;
let data = source.get("data")?.as_str()?;
use base64::Engine;
let bytes = base64::engine::general_purpose::STANDARD
.decode(data)
.ok()?;
// Screenshots are the overwhelming case, and they are PNG; an
// unrecognized type is more likely a dialect change than a JPEG.
let extension = source
.get("media_type")
.and_then(Value::as_str)
.and_then(crate::media::extension_for)
.unwrap_or("png");
let name = format!("{}.{extension}", super::super::random_hex());
let dir = session_dir.join("files");
if let Err(err) = wg_app_link::private::create_dir(&dir)
.map_err(std::io::Error::other)
.and_then(|()| std::fs::write(dir.join(&name), bytes))
{
tracing::error!("couldn't save produced image: {err}");
return None;
}
Some(name)
}
#[cfg(test)]
+93 -10
View File
@@ -317,21 +317,26 @@ pub async fn directory_exists(transport: &Transport, path: &str) -> bool {
transport.capture(&launch).await.is_ok()
}
/// Reads the tail of one session's file and turns it into events.
/// Reads the tail of one session's file, as the raw JSONL.
///
/// `tail` rather than the whole file, and as [`Launch`] arguments rather
/// than a shell string, so the path is an argument and never syntax.
pub async fn replay(transport: &Transport, path: &str) -> Result<Vec<Event>> {
///
/// Returns text rather than events because turning records into events has
/// a side effect -- writing out the images they carry -- and it needs the
/// session directory to write them into. That directory does not exist
/// until the session is created, which is after this runs, so the
/// conversion happens there instead. See [`events_from`].
pub async fn read_tail(transport: &Transport, path: &str) -> Result<String> {
let launch = Launch::new(
"tail",
vec!["-n".to_string(), REPLAY_LINES.to_string(), path.to_string()],
None,
);
let text = transport
transport
.capture(&launch)
.await
.with_context(|| format!("reading {path}"))?;
Ok(events_from(&text))
.with_context(|| format!("reading {path}"))
}
/// Claude Code's stored JSONL as this project's events.
@@ -339,7 +344,15 @@ pub async fn replay(transport: &Transport, path: &str) -> Result<Vec<Event>> {
/// A partial first line is expected and ignored: `tail -n` cuts at a line
/// boundary, but the *file* may have been appended to since, and a line
/// that does not parse is one this reader has no opinion about.
pub fn events_from(text: &str) -> Vec<Event> {
///
/// `session_dir` is where images found along the way are written, the same
/// place and by the same function the live translator uses -- so a
/// screenshot looks identical whether it was watched as it happened or
/// replayed afterwards. It is only the *reference* that reaches the phone;
/// the bytes are fetched from `/sessions/{id}/files/{ref}` when something
/// actually draws them, and none of this is ever sent back to the CLI,
/// which reads its own session file.
pub fn events_from(text: &str, session_dir: &std::path::Path) -> Vec<Event> {
let mut events = Vec::new();
for line in text.lines() {
let Ok(record) = serde_json::from_str::<Value>(line) else {
@@ -355,7 +368,7 @@ pub fn events_from(text: &str) -> Vec<Event> {
continue;
};
match record.get("type").and_then(Value::as_str) {
Some("user") => push_user(&mut events, content),
Some("user") => push_user(&mut events, content, session_dir),
Some("assistant") => push_assistant(&mut events, content),
_ => {}
}
@@ -363,16 +376,25 @@ pub fn events_from(text: &str) -> Vec<Event> {
events
}
fn push_user(events: &mut Vec<Event>, content: &Value) {
fn push_user(events: &mut Vec<Event>, content: &Value, session_dir: &std::path::Path) {
// A tool result arrives as a user record, because that is how the API
// models it -- but it is the other half of a tool call, not something
// a person said, and showing it as a message would put the reader's
// own words and a command's output in the same voice.
if let Value::Array(blocks) = content {
for block in blocks {
// A picture the person attached to their own message, rather
// than one a tool produced. Same block shape, one level up.
push_images(events, std::slice::from_ref(block), session_dir);
if block.get("type").and_then(Value::as_str) == Some("tool_result")
&& let Some(id) = block.get("tool_use_id").and_then(Value::as_str)
{
// Before the tool's own row, matching the live translator:
// a screenshot belongs to the call that took it, and after
// the result it reads as belonging to whatever came next.
if let Some(Value::Array(parts)) = block.get("content") {
push_images(events, parts, session_dir);
}
events.push(Event::ToolEnd {
id: id.to_string(),
output: text_of(block.get("content").unwrap_or(&Value::Null)),
@@ -386,6 +408,17 @@ fn push_user(events: &mut Vec<Event>, content: &Value) {
}
}
/// Saves every image block in `parts` and references each one.
fn push_images(events: &mut Vec<Event>, parts: &[Value], session_dir: &std::path::Path) {
for part in parts {
if part.get("type").and_then(Value::as_str) == Some("image")
&& let Some(name) = super::claude::translate::save_image(session_dir, part)
{
events.push(Event::Image { image: name });
}
}
}
fn push_assistant(events: &mut Vec<Event>, content: &Value) {
let Value::Array(blocks) = content else {
return;
@@ -501,7 +534,12 @@ pub async fn line_count(transport: &Transport, path: &str) -> Result<usize> {
/// Events from the lines after `after`, which is a 0-based count of lines
/// already accounted for.
pub async fn replay_after(transport: &Transport, path: &str, after: usize) -> Result<Vec<Event>> {
pub async fn replay_after(
transport: &Transport,
path: &str,
after: usize,
session_dir: &std::path::Path,
) -> Result<Vec<Event>> {
let launch = Launch::new(
"tail",
vec![format!("-n+{}", after + 1), path.to_string()],
@@ -511,5 +549,50 @@ pub async fn replay_after(transport: &Transport, path: &str, after: usize) -> Re
.capture(&launch)
.await
.with_context(|| format!("reading {path} from line {}", after + 1))?;
Ok(events_from(&text))
Ok(events_from(&text, session_dir))
}
#[cfg(test)]
mod tests {
use super::*;
/// A 1x1 PNG, base64 -- the smallest thing with a real header.
const PNG: &str = "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mNk\
YPhfDwAChwGA60e6kgAAAABJRU5ErkJggg==";
#[test]
fn replayed_screenshots_are_saved_and_referenced() {
let dir = tempfile::tempdir().expect("tempdir");
// The shape a screenshot actually has in these files: an image
// part inside a tool result, beside its text.
let line = format!(
r#"{{"type":"user","message":{{"role":"user","content":[{{"type":"tool_result","tool_use_id":"toolu_1","content":[{{"type":"text","text":"took a screenshot"}},{{"type":"image","source":{{"type":"base64","media_type":"image/png","data":"{PNG}"}}}}]}}]}}}}"#
);
let events = events_from(&line, dir.path());
let Some(Event::Image { image }) = events.first() else {
panic!("a replayed screenshot must become an image event: {events:?}");
};
assert!(image.ends_with(".png"));
// On disk, where the files route serves it from -- the phone
// fetches it only when something draws it.
assert!(dir.path().join("files").join(image).is_file());
// And it comes before the tool row it belongs to, so it does not
// read as belonging to whatever happened next.
assert!(
matches!(events.get(1), Some(Event::ToolEnd { .. })),
"{events:?}"
);
}
#[test]
fn a_record_with_no_image_writes_nothing() {
let dir = tempfile::tempdir().expect("tempdir");
let line = r#"{"type":"user","message":{"role":"user","content":[{"type":"tool_result","tool_use_id":"toolu_1","content":"plain output"}]}}"#;
let events = events_from(line, dir.path());
assert_eq!(events.len(), 1, "{events:?}");
// No stray directory for a session that never produced one.
assert!(!dir.path().join("files").exists());
}
}
+10 -6
View File
@@ -831,7 +831,7 @@ fn spawn_import_sync(
if lines <= cursor.lines {
continue;
}
match import::replay_after(&transport, &cursor.path, cursor.lines).await {
match import::replay_after(&transport, &cursor.path, cursor.lines, &dir).await {
Ok(events) => {
tracing::info!(
"{} grew by {} lines with nothing from here; replaying {} events",
@@ -866,10 +866,14 @@ pub struct Seed {
/// Where that session's file is and how much of it has been shown, so
/// the session can keep itself up to date afterwards.
pub cursor: import::Cursor,
/// Replayed into the transcript so the phone shows the conversation it
/// is joining. The CLI reads the real file itself, so this is what the
/// reader sees rather than what the model is given.
pub events: Vec<Event>,
/// The tail of that session's file, as the raw JSONL.
///
/// Turned into events in `launch`, not before, because doing so writes
/// out the images the records carry and that needs the session
/// directory to write them into -- which does not exist until the
/// session does. The CLI reads the real file itself, so this only ever
/// decides what the *reader* sees.
pub records: String,
}
fn launch(
@@ -890,7 +894,7 @@ fn launch(
claude::write_resume_token(&dir, &seed.resume);
import::write_cursor(&dir, &seed.cursor);
let at = now();
for event in seed.events {
for event in import::events_from(&seed.records, &dir) {
transcript.append(event, at)?;
}
}