ai-app/TRANSCRIPT_CACHE.md

The transcript cache

Asked for by Iris on 2026-09-04: keep the transcripts of recently visited sessions on the phone, so reopening one does not download it again. It has to save data over the tunnel, it must not disturb a reply that is streaming when the screen is reopened, it must never skip an event, and session settings needs a manual reload for when the file on the machine has changed under it.

Built 2026-09-04. Like EXPLORER.md this records each decision with its reason and what was rejected, so that when one changes it is changed here rather than re-argued -- three of them changed during the building, and "What building it changed" at the foot says which and why. What it is not is the operational half: how to exercise it, and what has bitten, are in AGENTS.md with the rest of the working notes.

What it is, in one paragraph

A per-session file on the phone holding the exact JSON lines the server has already sent, in transcript order, with a record of which sequence numbers each run of lines covers. Everything the session screen fetches today -- the opening window, the pages it scrolls back through, the span an anchor restore reaches for -- is asked of the cache first and of the server only for what the cache does not hold, and everything that arrives from the server is written into it. The live stream then resumes from the newest cached event, exactly as it resumes today from the newest event on screen, so the server sends only what happened since. One tiny request checks that the cached tail is still what the server has before the stream is opened from it, and a button in session settings throws the cache away and rebuilds the screen as a cold open for the cases that check cannot see.

The invariants

Everything below is in service of four rules. When a decision looks arbitrary, it is one of these forcing it.

1. What is on screen is what the server's transcript says, in order, with nothing missing, for every sequence number the screen claims to show. The cache is a copy of server output and is never inferred, folded, or edited on the phone. Where the copy cannot be shown to be current, it is thrown away, not patched.
2. A cached line is never ahead of the live cursor, and the live cursor is never ahead of the cache. The stream resumes from the newest cached event, so a reply that was mid-stream when the screen closed picks up at its next delta and folds into the same row, as it does today when the phone merely lost the tunnel for a second.
3. The cache is never load-bearing. A missing, evicted, corrupt or unwritable cache degrades to today's behaviour -- a cold open -- and never to a blank or wrong screen. Every path that reads it has a network path beside it that produces the same result.
4. Data crosses the tunnel once. A line already on the phone is not fetched again unless the reader asks for that (the reload button) or the check in decision 3 says it must be.

Decisions

1. Raw server lines, on the phone, keyed by server and session

The cache stores the server's own JSON, one event per line, byte-for-byte as it arrived: the elements of the /transcript array and the data: payload of each SSE frame. Reading the cache means running the same parseSeqEvent the network path runs, so a cached transcript and a fetched one cannot draw differently, and an event type this build does not know (SessionEvent.Unknown) survives on disk for the build that will.

It lives under context.cacheDir -- <cacheDir>/transcripts/v1/<host>_<port>/<sessionId>/ -- because it is exactly what that directory is for: bytes the phone can regenerate from the server, which Android may delete under storage pressure without asking. Keyed by the server's host and port because two servers can hold a session with the same id (the sandbox and the real server, or a re-enrolment), and a line from one shown against the other is invariant 1 broken. ServerSettings has both fields; the key is "${settings.host}_${settings.port}" with : never appearing in it. The v1 segment is the format version: any change to the layout below bumps it, and a directory of another version is deleted on first use.

Rejected: a database (Room, SQLite). The access pattern is "the newest N lines" and "the lines before seq X", on files of tens of megabytes at most, and a JSONL file per contiguous run answers both by reading from its end. A database would be a new dependency for an index the file layout already provides.

Rejected: caching folded TranscriptItem rows instead of events. Rows are a rendering of events, and their shape changes when the fold changes; the cache would need invalidating on every app update that touched foldEvent, and would still have to keep raw seqs for the stream cursor. Events are the server's contract and the only thing that is stable.

2. Chunks with explicit coverage; one contiguous run behind the cursor

A page from the server is a set of lines and a claim about what they cover, and the two are not the same thing. A coalesced page (coalesce=true, which the scroll-back pager asks for) joins each run of assistantText deltas into one event carrying the seq of its oldest delta, so a page whose newest event has seq 1,200 may in fact cover every line up to the before it was asked with, say 1,650. Nothing in the lines themselves says so. So each stored chunk records its coverage as a half-open range [first, end), where first is the seq of its oldest event and end is the before the request was made with -- or, for a raw chunk, its newest seq plus one.

Chunks are files named by their coverage:

<first>-<end>.rows.jsonl     a coalesced page; end is the `before` it was fetched with
<first>-<end>.raw.jsonl      an uncoalesced page or a closed live run
<first>-open.raw.jsonl       the live run: appended to by the stream; end = last line's seq + 1

Two chunks are adjacent when one's end equals the other's first. The cache serves only the contiguous run of adjacent chunks that ends at the newest raw chunk (the suffix); chunks behind a gap are kept on disk, because the gap is usually filled (decision 4), but are never served across the gap.

The newest chunk is always raw. That is what makes the stream cursor and the check in decision 3 well defined: a raw chunk's last line is a real event at a real seq, and the server never coalesces the newest window ("the live cursor depends on real seqs", read_window). It holds by construction -- the opening window is fetched with no before, stream frames are raw, and a reset window is raw -- and is checked on read: if the newest chunk on disk is a .rows chunk (which can only happen if the app died between closing one live run and appending to the next), the session's cache is purged and the open is cold.

There is at most one open chunk. When a stream event arrives whose seq is not the open chunk's end -- which is what a reset looks like from here, see decision 6 -- the open chunk is closed by renaming it with its real end, and a new open chunk starts at the arriving seq. An event whose seq is below the open chunk's end is already covered and is not written (the SSE contract is seq > after, so this is a guard, not a path).

Rejected: one file per session, rewritten to prepend older pages. A 20 MB transcript would be rewritten on every page scrolled back to. The chunk directory costs a directory listing per open instead.

Rejected: trimming chunks to resolve overlaps. A coalesced event cannot be split at a seq inside its run, so an overlap between a coalesced page and an existing chunk has no clean cut. The cache therefore never stores a page that overlaps an existing chunk; decision 4 makes sure such a page is never fetched in the first place, and if one arrives anyway (a server without decision 4's change) it is used for display and not stored.

3. The cached tail is checked against the server before the stream opens from it

The screen must not resume a stream from a cached seq unless the server's event at that seq is the one in the cache. The transcript file on the machine is append-only in ordinary use, but it can be replaced or truncated -- a sandbox re-seeded with the same ids, a backup restored, a directory deleted and the session re-imported under the same name -- and catch_up on such a file would hand the phone a continuation of a different conversation, spliced onto the cached one with no seam. That is the worst thing this feature can do, and it is caught with one request.

The probe: GET /sessions/{id}/transcript?before=<cursor+1>&limit=1, where cursor is the seq of the cache's newest line. read_window with that before returns the single newest event with seq ≤ cursor, which is the event at the cursor when it exists. The probe passes when that response, parsed with parseSeqEvent, is == to the cached line parsed the same way -- data-class equality over seq, ts, and the whole event. It fails when the response is empty, is a different seq, or differs in any field.

That equality rested on an assumption this plan stated and did not check: that the two ways the server hands out a line agree bit for bit. They did not. serde_json's default float parser is not correctly rounded, so a ts of 1788546972.6030757 written to the transcript came back from /transcript as ...0755, while the SSE stream -- serializing the same struct -- sent the original. Measured on the emulator 2026-09-04: 23 of 330 cached lines differed from the server's answer in the last bit, so the probe would have failed on any session whose cached tail happened to be one of them, silently and only sometimes. That is a defect in the server independent of this feature -- two answers to "what is line 30" -- and it is fixed there, with float_roundtrip and a test (a_line_read_back_is_the_line_that_was_written) that fails the moment the feature is dropped. Comparing everything except ts was the other option and was rejected: a re-seeded fixture is identical in content and differs only in when it happened, which is exactly the case the probe exists for. A failed probe purges the session's cache and proceeds as a cold open. A probe that cannot be made (no route to the server) leaves the cached transcript on screen, shows the request's error on the stream banner where a connection failure shows today, and is retried on the stream loop's schedule (RECONNECT_DELAY_MS); the stream is never opened until a probe has passed once for this screen instance.

What the probe does not catch: a line changed in the middle of the file with the tail intact, or a file rewritten so that the event at the cursor happens to be identical. Those are what the reload button is for, and the button's caption says so.

Cost: one request of a few hundred bytes, one round trip, in the slot where the opening page's request is today -- so the round trips before the stream is live are unchanged at two, and the bytes fall from a page to a line. The cached rows are drawn before the probe returns, which is the whole point of the feature; a failed probe replaces them, the same appearance as a reset.

Rejected: a server-side check on the stream (events?after=N&ts=T, answered with a distinct frame when the event at N is not what the phone thinks). Strictly better coverage -- it would run on every reconnect, not only on open -- and no extra round trip. Not chosen because it puts a cache's validation into a protocol that otherwise knows nothing about caching, and because the reset frame already has to keep meaning "you are behind, your history is fine" (decision 6), so a second frame would be needed. Worth revisiting if the probe's round trip is ever measured as the thing making reopen slow; note it as the alternative here and in PLAN.md.

Rejected: trusting the cache without a check and relying on the reload button. Invariant 1 is not something a button restores after the fact.

Rejected: fetching the newest page as today and using it to validate the overlap. Zero saving on the opening page, which is the request paid on every open.

4. Pages ask the server only for the gap: after on /transcript

After a reader has been away, the cache holds [a, b) and the screen holds the newest window [W, …) with a gap between b and W. Paging back from W asks the server for a coalesced page before W, and that page may reach back past b -- a single reply is hundreds of lines, so forty rows can be thousands of seqs -- producing exactly the overlap decision 2 refuses to store. Left like that, every cached chunk would be overlapped and dropped in turn as the reader paged back through the gap, and the cache would save nothing for the sessions it exists for.

So the transcript route gains a lower bound. TranscriptQuery in server/src/routes.rs gets

/// Return nothing at or below this seq; the page stops here instead of at `limit`.
/// The phone passes the end of what it already holds, so a page never overlaps it.
#[serde(default)]
after: Option<u64>,

named to match the SSE route's after (exclusive, seq > after). read_window(path, before, after, limit, coalesce) in server/src/session/transcript.rs computes start = first_at_or_after(after + 1) and stops the walk there: the raw branch parses max(start, end - limit)..end; parse_coalesced takes a start and its while index > 0 becomes while index > start. A delta run cut at start is emitted as the partial it is, exactly as one cut by limit already is, and healSplitMessage welds it on the phone -- no new mechanism. The route's table comment in routes.rs gains the parameter, and transcript.rs gets a test beside a_window_is_the_events_before_a_cursor_and_nothing_else: with after set, the page's oldest seq is greater than after, and with after set inside a delta run the partial run's seq is the first delta above after.

The phone passes after = b - 1 where b is the end of the nearest chunk whose end ≤ before, and nothing when there is none. A page that comes back with first == b is adjacent, and the suffix now runs through the old chunks: the gap is closed with exactly the bytes it was wide, and the history behind it is served locally from then on.

Rejected: fetching the gap raw in one request (before=W&limit=W-b, which is what the anchor restore already does). Exact, but a gap of ten thousand lines is several megabytes downloaded to save re-downloading history the reader may never scroll to; the feature exists to save data. Paging as today with a bound saves the same bytes and fetches only what is read.

Rejected: dropping the cached run whenever a gap opens. Being more than CATCH_UP_LIMIT (200) events behind is the ordinary state of an active session revisited -- 200 raw events is one reply -- so this would empty the cache for exactly the sessions that are opened most.

5. A page is served locally in rows, mirroring the server's count

loadOlderPage asks for HISTORY_PAGE (40) rows when coalesce = true, and for a number of events otherwise (the anchor restore). Served from the cache, the events branch is the limit lines before before. The rows branch walks back from the line before before counting rows the way parse_coalesced does: every event that is not an assistantText is a row, and each maximal run of assistantText lines is one row; it stops only between rows, once limit rows are complete, and returns the raw lines oldest-first. It does not join the deltas -- the fold does that (foldEvent appends a delta to a preceding AssistantMsg), and the joined row keeps the seq of its first delta either way, so anchors and the next before land where they do today.

A cached page is allowed to be short: the suffix's oldest chunk starts at some first, and a walk that reaches it returns what it found. The caller already treats a short page as a page; only an empty page means "start of the conversation" (moreHistory = false), and the cache never returns an empty page -- it returns null (a miss) and the network is asked. The walk may cross a chunk boundary inside the suffix, since adjacent chunks are one run; a delta run straddling a boundary counts as one row, as it should.

A miss is before outside what the suffix covers continuously -- above its newest end, or at or below its oldest first. This plan first said a miss was "no chunk of the suffix ends at before", which is wrong in the commonest case there is: a warm open draws the newest eighty lines of the live run, so the cursor the reader then scrolls back from is in the middle of a chunk, not at a boundary. Under the narrower rule every warm open sent its first backwards page to the server, and that page -- reaching back past the run the phone already held -- overlapped it and could not be stored, so the same history was fetched again on every visit. The feature would have saved the opening window and nothing else.

The row rule is a copy of the server's, and copies drift. It is short (one comparison), it is pure, and it goes under a JVM unit test with the same fixture as the server's coalescing_counts_rows_and_joins_delta_runs -- the three cases are a run cut by the limit, a usageDelta inside a run (the server flushes the run there, so it is two rows), and a page that is all one run.

6. What a reset means for the cache: behind, not wrong

The server sends reset when the cursor is more than CATCH_UP_LIMIT events behind, then the newest 200 raw events. The screen already drops everything and rebuilds from that window. For the cache, a reset means the history is intact and there is a gap: the probe passed, the file is append-only, and the window's first seq is above the open chunk's end. The store learns this from the first window event's seq (decision 2: a seq that is not the open chunk's end closes it and opens a new chunk) and needs no signal from the screen; the gap is filled by paging (decision 4).

Two things the reset handler in SessionScreen does not clear today and must: queued and waitingCommands. Both are folded from events, and a messageQueued whose resolving userMessage fell in the gap would otherwise draw a waiting bubble for a message the session has long since read. This is a latent bug today, made likely by the cache because a cached tail is older than a fetched one. contextTokens needs no change: UsageDelta.context is absolute, so the window's first one corrects it.

7. Session state that is not the transcript comes from the list, not the cache

apply derives status, model, permissionMode and compactingSince from Status and Settings events. Replayed from a fetched page those are current; replayed from the cache they are as old as the last visit, while summary.status, summary.model and summary.permissionMode -- the row the reader just tapped -- were fetched moments ago. So the cache replay runs through apply for the transcript's sake (queued bubbles, context, rows) and then reassigns those four from summary, which is the newer of the two measurements; the stream's catch-up then makes them current. Without this a session that finished an hour ago would open saying "working" until the stream connected, which is a status row lying for a round trip.

8. Reload, in session settings

SessionSettingsDialog gains a row under the working directory:

[ Transcript ]  2.3 MB cached          [ Reload ]

The size is what the button discards, and it is the unknown state made visible: null while the directory is being measured (spinner, as the notifications switch does), "nothing cached" when the directory is absent or empty, else the size. A caption in the style of Move's, because the button costs something the reader cannot see:

Reload throws away this phone's copy and fetches the transcript from the
server again. Use it when what is shown here disagrees with the file on
the machine.

Pressing it: purge the session's cache directory, close the dialog, and rebuild the screen as a cold open -- the same sequence as reset plus a fresh opening fetch, with the reader put back where they were. The mechanism is an epoch counter (mutableIntStateOf(0)) added to the key of the opening effect and the stream effect; incrementing it cancels both (the stream's finally closes the socket) and relaunches them. State the relaunch must see cleared: items, replies.clear(), held, oldestSeq = 0, moreHistory = true, queued, waitingCommands, lastSeq.set(0), ready = false. savedAnchor becomes remember(summary.id, epoch) so the restore path reads the anchor saved at the reader's current position (the anchor saver writes on every settle, so it is there), and restoring is re-derived from it. The button is enabled whether or not anything is cached: "what I see disagrees with the machine" is a state an empty cache can also be in, and a control that comes and goes makes its own presence the signal.

Nothing is announced on success. The transcript shows the opening spinner and then the rows, which is what the screen already says about a reload. A failure is the opening fetch's, and lands on the stream banner where that failure lands today.

Rejected: a global "clear transcript cache" in the app's settings screen. Not asked for; eviction (decision 9) bounds the total, and the per-session button is where the reader is when they notice a problem. Easy to add as one more caller of TranscriptCache.purgeAll if wanted.

9. Budget, eviction, pruning

The cache is bounded three ways, each with its path out written beside the path in:

• Budget. CACHE_BUDGET_BYTES = 256 MB across all sessions of one server. Each open touches the session directory's mtime; after the opening replay, on Dispatchers.IO, the store sums the server's directories and deletes least-recently-touched session directories (never the one on screen) until under budget. 256 MB is a dozen of the largest transcripts seen in this VM (21 MB for 24,000 events) and a small fraction of a phone; it is a number to revisit against real use, not a measurement.
• Deleted sessions. SessionListScreen's delete calls cache.session(id).purge() after deleteSession succeeds, and every successful list fetch calls cache.retainOnly(ids) for that server, so a session deleted from another device or from the backend is pruned on the next visit to the list. Drafts.kt chose not to prune because its residue is bytes; here it is megabytes, so the pass is worth having.
• Android. cacheDir may be emptied under pressure at any moment, including while a screen is open. Every read tolerates a missing directory (cold open) and every write failure is swallowed once and disables writing for that screen instance (decision 10).

10. The cache never breaks the screen

Every store operation that touches the disk catches IOException and answers as if the cache were empty: null from a read, no-op from a write, with the failure logged once at Log.w("ai-app", …). After a write failure the SessionCache instance sets disabled = true and writes nothing more, so a full disk costs one log line rather than one per delta. A line at the end of an open chunk that does not parse -- the app died mid-write -- is dropped and the file truncated to the last good line before anything is served from it; a line that does not parse anywhere else purges the session's cache (that file was not written by this code). None of this is reported on screen: none of it changes what the screen shows, and the reader has nothing to do about it.

Layout on disk

<cacheDir>/transcripts/
  v1/
    10.0.2.2_8443/                      one directory per server (host_port)
      3f2c…/                            one per session id
        1-1650.rows.jsonl               coalesced page: covers seqs 1..1649
        1650-2001.rows.jsonl
        2001-2400.raw.jsonl             a closed live run
        2600-open.raw.jsonl             the live run; end = last line's seq + 1

Here 2400..2599 is a gap: the reader was away for two hundred events and the stream reset. The suffix is the single chunk 2600-open; the first backwards page asks the server for before=2600&after=2399&coalesce=true, and once a page comes back with first == 2400 the suffix runs to seq 1.

Each .jsonl is one JSON object per line, oldest first, exactly as the server sent it. No header, no index: coverage is in the name, order is the file's, and the seq is in every line.

What building it changed

Each of these contradicted something written above, and each was found by running it rather than by reading it. The decisions themselves are amended in place; this is the list of what moved, so that a reader who remembers the first version knows what to re-read.

• The probe's equality had a false premise -- decision 3. The server did not hand out the same line twice the same way. Fixed on the server.
• A cached page starts anywhere inside the run -- decision 5. Requiring a chunk boundary would have made the cache save the opening window and nothing else.
• The opening window is stored by append, not by storePage. The sketch below had storePage grow a special case for "this page is the new open chunk", decided by an implicit condition that a raw history page also satisfies. Appending each line instead is the mechanism that already exists, and the open chunk stays the one thing that grows.
• Chunks are read backwards, in blocks, and never whole. Every question the cache is asked is about the newest end, and a live run reaches the size of the conversation -- so reading a chunk to answer with eighty lines of it is the cost the server's own reader was rewritten to stop paying, arriving on the phone. Damage is therefore noticed when a read reaches it rather than up front, which is the better time: what is not read cannot be wrong.
• The stream waits for the opening effect's probe. The screen lifts ready before the probe returns -- that is the point of the cache -- so ready stopped being the whole gate, and the stream loop asked the same question a second time and raced its own answer. Two probes per warm open, visible in the server's log.
• SessionCache is synchronized. The stream appends live events from one IO thread while a reader scrolling back reads pages from another; the open chunk's name, its end and its writer must never be seen half-rotated.

What it cost, measured

On the emulator against app/ui-sandbox.sh, 2026-09-04, on a session of 505 events (three short exchanges and two 300-delta replies):

• Reopening it: one request, for one event. The probe, and nothing else -- including scrolling the whole conversation back to its first line. A cold open of the same session is two requests and 100 events.
• A reset after falling 300 events behind costs the gap and no more. The window arrived at seq 306, the phone held up to 202, and the first backwards page asked before=306&after=201 and came back with four coalesced rows covering 202..305 -- against the 104 raw events an unbounded page would have re-fetched and then thrown away.
• Every chunk is exactly what the server says for the range its name claims, checked line by line against /transcript for each chunk's own before/after/coalesce, across a reset and a gap-fill.
• Nothing about drawing changed, which is what a cache must not do: transcript-bench.sh before and after, same viewport content and the same gestures, reported p50 16.9ms both times and the transcript's own draw accounting at 0.33ms against 0.32ms.

Still to measure, in real use rather than here: the size the cache reaches against CACHE_BUDGET_BYTES, and whether the probe's round trip is ever what a reader waits on.

Open questions

• The probe on every reconnect, not only on open? Decision 3 probes once per screen instance. A file replaced while the screen is open is today's behaviour and not made worse, but the server-side check it rejects would close it. Decide after measuring how often the probe's round trip is what the reader waits on.
• A reset arriving during an anchor restore was an open worry when this was written, and was measured and closed on 2026-09-04 (see "The reconnect loop does not reproduce") before this landed. The cache makes the restore cheaper again -- a warm one is now the probe and nothing else -- so it can only have narrowed the window further. Worth re-measuring here only if a reader reports the screen reconnecting on reopen.
• Images. SessionImage fetches bytes from the files route on draw; they are not part of this cache and are re-downloaded per view. A separate, simpler cache (a directory of refs, no ordering) if the measurement above says the images are where the data goes.