ai-app/TRANSCRIPT_CACHE.md

The transcript cache

Asked for by Iris on 2026-09-04 and built the same day: keep the transcripts of recently visited sessions on the phone, so reopening one does not download it again. It has to save data over the tunnel, must not disturb a reply that is streaming when the screen is reopened, must never skip an event, and needs a manual reload for when the file on the machine has changed under it.

Like EXPLORER.md this records each decision with its reason and what was rejected, so that when one changes it is changed here rather than re-argued. "What building it changed" at the foot says which of them moved while it was being built. How to exercise it, and what has bitten, are in AGENTS.md.

What it is, in one paragraph

A per-session file on the phone holding the exact JSON lines the server has already sent, in transcript order, with a record of which sequence numbers each run of lines covers. Everything the session screen fetches — the opening window, the pages it scrolls back through, the span an anchor restore reaches for — is asked of the cache first and of the server only for what the cache does not hold, and everything that arrives from the server is written into it. The live stream then resumes from the newest cached event, exactly as it resumes from the newest event on screen, so the server sends only what happened since. One tiny request checks that the cached tail is still what the server has before the stream is opened from it, and a button in session settings throws the cache away and rebuilds the screen as a cold open for the cases that check cannot see.

The invariants

When a decision below looks arbitrary, it is one of these forcing it.

1. What is on screen is what the server's transcript says, in order, with nothing missing, for every sequence number the screen claims to show. The cache is a copy of server output and is never inferred, folded, or edited on the phone. Where the copy cannot be shown to be current, it is thrown away, not patched.
2. A cached line is never ahead of the live cursor, and the live cursor is never ahead of the cache. The stream resumes from the newest cached event, so a reply that was mid-stream when the screen closed picks up at its next delta and folds into the same row.
3. The cache is never load-bearing. A missing, evicted, corrupt or unwritable cache degrades to a cold open, never to a blank or wrong screen. Every path that reads it has a network path beside it producing the same result.
4. Data crosses the tunnel once. A line already on the phone is not fetched again unless the reader asks (the reload button) or the check in decision 3 says it must be.

Decisions

1. Raw server lines, on the phone, keyed by server and session

The cache stores the server's own JSON, one event per line, byte-for-byte as it arrived: the elements of the /transcript array and the data: payload of each SSE frame. Reading the cache runs the same parseSeqEvent the network path runs, so a cached transcript and a fetched one cannot draw differently, and an event type this build does not know (SessionEvent.Unknown) survives on disk for the build that will.

It lives under context.cacheDir, which is exactly what that directory is for: bytes the phone can regenerate from the server, which Android may delete under storage pressure without asking. Keyed by the server's host and port, because two servers can hold a session with the same id (the sandbox and the real server, or a re-enrolment) and a line from one shown against the other is invariant 1 broken. The v1 segment is the format version: any change to the layout below bumps it, and a directory of another version is deleted on first use.

Rejected: a database (Room, SQLite). The access pattern is "the newest N lines" and "the lines before seq X", on files of tens of megabytes at most, and a JSONL file per contiguous run answers both by reading from its end. A database would be a new dependency for an index the file layout provides.

Rejected: caching folded TranscriptItem rows instead of events. Rows are a rendering of events, and their shape changes when the fold changes; the cache would need invalidating on every app update that touched foldEvent, and would still have to keep raw seqs for the stream cursor. Events are the server's contract and the only thing that is stable.

2. Chunks with explicit coverage; one contiguous run behind the cursor

A page from the server is a set of lines and a claim about what they cover, and the two are not the same thing. A coalesced page joins each run of assistantText deltas into one event carrying the seq of its oldest delta, so a page whose newest event has seq 1,200 may in fact cover every line up to the before it was asked with, say 1,650. Nothing in the lines themselves says so. So each stored chunk records its coverage as a half-open range [first, end), where end is the before the request was made with — or, for a raw chunk, its newest seq plus one.

Chunks are files named by their coverage:

<first>-<end>.rows.jsonl     a coalesced page; end is the `before` it was fetched with
<first>-<end>.raw.jsonl      an uncoalesced page or a closed live run
<first>-open.raw.jsonl       the live run: appended to by the stream

Two chunks are adjacent when one's end equals the other's first. The cache serves only the contiguous run of adjacent chunks that ends at the newest raw chunk (the suffix); chunks behind a gap are kept on disk, because the gap is usually filled (decision 4), but are never served across it.

The newest chunk is always raw. That is what makes the stream cursor and the probe well defined: a raw chunk's last line is a real event at a real seq, and the server never coalesces the newest window. It holds by construction — the opening window is fetched with no before, stream frames are raw, and a reset window is raw — and is checked on read: a .rows chunk found newest (which can only happen if the app died between closing one live run and appending to the next) purges the session's cache.

There is at most one open chunk. A stream event whose seq is not the open chunk's end — which is what a reset looks like from here — closes it by renaming it with its real end and starts a new one. An event whose seq is below the open chunk's end is already covered and is not written; the SSE contract is seq > after, so that is a guard rather than a path.

Rejected: one file per session, rewritten to prepend older pages. A 20 MB transcript would be rewritten on every page scrolled back to. The chunk directory costs a directory listing per open instead.

Rejected: trimming chunks to resolve overlaps. A coalesced event cannot be split at a seq inside its run, so an overlap between a coalesced page and an existing chunk has no clean cut. The cache therefore never stores a page that overlaps an existing chunk; decision 4 makes sure such a page is never fetched, and one that arrives anyway is used for display and not stored.

3. The cached tail is checked against the server before the stream opens from it

The transcript file is append-only in ordinary use, but it can be replaced or truncated — a sandbox re-seeded with the same ids, a backup restored, a session deleted and re-imported — and catch_up on such a file would hand the phone a continuation of a different conversation, spliced onto the cached one with no seam. That is the worst thing this feature can do, and it is caught with one request.

The probe is GET /sessions/{id}/transcript?before=<cursor+1>&limit=1, where cursor is the seq of the cache's newest line. read_window with that before returns the single newest event with seq ≤ cursor, which is the event at the cursor when it exists. It passes when that response, parsed with parseSeqEvent, is == to the cached line parsed the same way — over seq, ts, and the whole event. It fails when the response is empty, is a different seq, or differs in any field.

That equality rested on an assumption this plan stated and did not check: that the two ways the server hands out a line agree bit for bit. They did not, and the server was fixed — see AGENTS.md's entry on float_roundtrip. Comparing everything except ts was the other option and was rejected: a re-seeded fixture is identical in content and differs only in when it happened, which is exactly the case the probe exists for.

A failed probe purges the session's cache and proceeds as a cold open. A probe that cannot be made leaves the cached transcript on screen, shows the error on the stream banner where a connection failure shows today, and is retried on the stream loop's schedule; the stream is never opened until a probe has passed once for this screen instance.

What the probe does not catch: a line changed in the middle of the file with the tail intact, or a file rewritten so that the event at the cursor happens to be identical. Those are what the reload button is for, and the button's caption says so.

Cost: one request of a few hundred bytes, in the slot where the opening page's request would be — so the round trips before the stream is live are unchanged at two, and the bytes fall from a page to a line. The cached rows are drawn before the probe returns, which is the whole point; a failed probe replaces them, with the same appearance as a reset.

Rejected: a server-side check on the stream, answered with a distinct frame when the event at N is not what the phone thinks. Strictly better coverage — it would run on every reconnect — and no extra round trip. Not chosen because it puts a cache's validation into a protocol that otherwise knows nothing about caching, and because the reset frame already has to keep meaning "you are behind, your history is fine". Worth revisiting if the probe's round trip is ever measured as the thing making reopen slow.

Rejected: trusting the cache and relying on the reload button. Invariant 1 is not something a button restores after the fact.

Rejected: fetching the newest page as before and using it to validate the overlap. Zero saving on the opening page, which is the request paid on every open.

4. Pages ask the server only for the gap: after on /transcript

After a reader has been away, the cache holds [a, b) and the screen holds the newest window [W, …) with a gap between b and W. Paging back from W asks for a coalesced page before W, and that page may reach back past b — a single reply is hundreds of lines, so forty rows can be thousands of seqs — producing exactly the overlap decision 2 refuses to store. Left like that, every cached chunk would be dropped in turn as the reader paged back through the gap, and the cache would save nothing for the sessions it exists for.

So the transcript route takes a lower bound, after, named to match the SSE route's (exclusive, seq > after). read_window starts the walk at first_at_or_after(after + 1) instead of at end - limit. A delta run cut at the start is emitted as the partial it is, exactly as one cut by limit already is, and healSplitMessage welds it on the phone — no new mechanism.

The phone passes after = b - 1 where b is the end of the nearest chunk whose end ≤ before, and nothing when there is none. A page that comes back with first == b is adjacent, and the suffix now runs through the old chunks: the gap is closed with exactly the bytes it was wide, and the history behind it is served locally from then on.

Rejected: fetching the gap raw in one request, which is what the anchor restore does. Exact, but a gap of ten thousand lines is several megabytes downloaded to save re-downloading history the reader may never scroll to.

Rejected: dropping the cached run whenever a gap opens. Being more than CATCH_UP_LIMIT (200) events behind is the ordinary state of an active session revisited — 200 raw events is one reply — so this would empty the cache for exactly the sessions that are opened most.

5. A page is served locally in rows, mirroring the server's count

loadOlderPage asks for HISTORY_PAGE (40) rows when coalescing and for a number of events otherwise (the anchor restore). Served from the cache, the events branch is the limit lines before before. The rows branch walks back counting rows the way parse_coalesced does — every event that is not an assistantText is a row, and each maximal run of assistantText lines is one row — stopping only between rows. It does not join the deltas; the fold does that, and the joined row keeps the seq of its first delta either way, so anchors and the next before land where they do on the network path.

A cached page is allowed to be short: a walk that reaches the suffix's oldest chunk returns what it found. The caller already treats a short page as a page; only an empty page means "start of the conversation", and the cache never returns one — it returns null (a miss) and the network is asked.

A miss is before outside what the suffix covers continuously — above its newest end, or at or below its oldest first. This plan first said a miss was "no chunk of the suffix ends at before", which is wrong in the commonest case there is: a warm open draws the newest eighty lines of the live run, so the cursor the reader then scrolls back from is in the middle of a chunk. Under the narrower rule every warm open sent its first backwards page to the server, and that page overlapped what the phone already held and could not be stored, so the same history was fetched again on every visit. The feature would have saved the opening window and nothing else.

The row rule is a copy of the server's, and copies drift. It is short, it is pure, and it is under a JVM unit test with the same fixture as the server's coalescing_counts_rows_and_joins_delta_runs — a run cut by the limit, a usageDelta inside a run (the server flushes the run there, so it is two rows), and a page that is all one run.

6. What a reset means for the cache: behind, not wrong

The server sends reset when the cursor is more than CATCH_UP_LIMIT events behind, then the newest 200 raw events. For the cache that means the history is intact and there is a gap: the probe passed, the file is append-only, and the window's first seq is above the open chunk's end. The store learns this from the first window event's seq and needs no signal from the screen; the gap is filled by paging.

The reset handler also clears queued and waitingCommands, which it did not originally. Both are folded from events, and a messageQueued whose resolving userMessage fell in the gap would otherwise draw a waiting bubble for a message the session has long since read. That was a latent bug made likely by the cache, because a cached tail is older than a fetched one. contextTokens needs no clearing: UsageDelta.context is absolute, so the window's first one corrects it.

7. Session state that is not the transcript comes from the list, not the cache

apply derives status, model, permissionMode and compactingSince from Status and Settings events. Replayed from a fetched page those are current; replayed from the cache they are as old as the last visit, while the list row the reader just tapped was fetched moments ago. So the cache replay runs through apply for the transcript's sake and then reassigns those four from summary, which is the newer of the two measurements; the stream's catch-up then makes them current. Without this a session that finished an hour ago would open saying "working" until the stream connected, which is a status row lying for a round trip.

8. Reload, in session settings

A row under the working directory showing what the button discards:

[ Transcript ]  2.3 MB cached          [ Reload ]

The size is the unknown state made visible — null while the directory is being measured (spinner, as the notifications switch does), "nothing cached" when the directory is absent or empty, else the size. The caption is in the style of Move's, because the button costs something the reader cannot see: "Reload throws away this phone's copy and fetches the transcript from the server again. Use it when what is shown here disagrees with the file on the machine."

Pressing it purges the session's cache directory, closes the dialog, and rebuilds the screen as a cold open, with the reader put back where they were. The mechanism is an epoch counter in the key of the opening effect and the stream effect; incrementing it cancels both and relaunches them. savedAnchor is keyed on the epoch too, so the restore reads the anchor saved at the reader's current position. The button is enabled whether or not anything is cached: "what I see disagrees with the machine" is a state an empty cache can also be in, and a control that comes and goes makes its own presence the signal.

Nothing is announced on success — the transcript shows the opening spinner and then the rows, which is what the screen already says about a reload. A failure is the opening fetch's, and lands on the stream banner.

Rejected: a global "clear transcript cache" in the app's settings. Not asked for; eviction bounds the total, and the per-session button is where the reader is when they notice a problem. Easy to add as one more caller of purgeAll.

9. Budget, eviction, pruning

Bounded three ways, each with its path out written beside the path in:

• Budget. CACHE_BUDGET_BYTES is 256 MB across all sessions of one server. Each open touches the session directory's mtime; after the opening replay, on Dispatchers.IO, the store sums the server's directories and deletes least-recently-touched ones (never the one on screen) until under budget. 256 MB is a dozen of the largest transcripts seen in this VM (21 MB for 24,000 events) and a small fraction of a phone; it is a number to revisit against real use, not a measurement.
• Deleted sessions. The list screen's delete purges after deleteSession succeeds, and every successful list fetch calls retainOnly(ids), so a session deleted from another device is pruned on the next visit to the list. Drafts.kt chose not to prune because its residue is bytes; here it is megabytes.
• Android. cacheDir may be emptied at any moment, including while a screen is open. Every read tolerates a missing directory and every write failure is swallowed once.

10. The cache never breaks the screen

Every store operation that touches the disk catches IOException and answers as if the cache were empty: null from a read, no-op from a write, logged once. After a write failure the instance stops writing, so a full disk costs one log line rather than one per delta. A line at the end of an open chunk that does not parse — the app died mid-write — is dropped and the file truncated to the last good line before anything is served from it; a line that does not parse anywhere else purges the session's cache, since that file was not written by this code. None of this is reported on screen: none of it changes what the screen shows, and the reader has nothing to do about it.

Layout on disk

<cacheDir>/transcripts/
  v1/
    10.0.2.2_8443/                      one directory per server (host_port)
      3f2c…/                            one per session id
        1-1650.rows.jsonl               coalesced page: covers seqs 1..1649
        1650-2001.rows.jsonl
        2001-2400.raw.jsonl             a closed live run
        2600-open.raw.jsonl             the live run

Here 2400..2599 is a gap: the reader was away for two hundred events and the stream reset. The suffix is the single chunk 2600-open; the first backwards page asks the server for before=2600&after=2399&coalesce=true, and once a page comes back with first == 2400 the suffix runs to seq 1.

Each .jsonl is one JSON object per line, oldest first, exactly as the server sent it. No header, no index: coverage is in the name, order is the file's, and the seq is in every line.

What building it changed

Each of these contradicted the plan, and each was found by running it rather than by reading it. The decisions above are amended in place; this is what moved, so a reader who remembers the first version knows what to re-read.

• The probe's equality had a false premise (decision 3). The server did not hand out the same line twice the same way. Fixed on the server.
• A cached page starts anywhere inside the run (decision 5). Requiring a chunk boundary would have made the cache save the opening window and nothing else.
• The opening window is stored by append, not by storePage. The sketch had storePage grow a special case for "this page is the new open chunk", decided by an implicit condition a raw history page also satisfies. Appending each line instead is the mechanism that already exists, and the open chunk stays the one thing that grows.
• Chunks are read backwards, in blocks, and never whole. Every question the cache is asked is about the newest end, and a live run reaches the size of the conversation — so reading a chunk to answer with eighty lines of it is the cost the server's own reader was rewritten to stop paying, arriving on the phone. Damage is therefore noticed when a read reaches it rather than up front, which is the better time: what is not read cannot be wrong.
• The stream waits for the opening effect's probe. The screen lifts ready before the probe returns — that is the point of the cache — so ready stopped being the whole gate, and the stream loop asked the same question a second time and raced its own answer. Two probes per warm open, visible in the server's log.
• SessionCache is synchronized. The stream appends live events from one IO thread while a reader scrolling back reads pages from another; the open chunk's name, its end and its writer must never be seen half-rotated.

What it cost, measured

On the emulator against app/ui-sandbox.sh, 2026-09-04, on a session of 505 events (three short exchanges and two 300-delta replies):

• Reopening it: one request, for one event. The probe, and nothing else — including scrolling the whole conversation back to its first line. A cold open of the same session is two requests and 100 events.
• A reset after falling 300 events behind costs the gap and no more. The window arrived at seq 306, the phone held up to 202, and the first backwards page asked before=306&after=201 and came back with four coalesced rows covering 202..305 — against the 104 raw events an unbounded page would have re-fetched and thrown away.
• Every chunk is exactly what the server says for the range its name claims, checked line by line against /transcript for each chunk's own before/after/coalesce, across a reset and a gap-fill.
• Nothing about drawing changed, which is what a cache must not do: transcript-bench.sh before and after, same viewport content and gestures, p50 16.9ms both times and the transcript's own draw accounting at 0.33ms against 0.32ms.

Still to measure, in real use rather than here: the size the cache reaches against CACHE_BUDGET_BYTES, and whether the probe's round trip is ever what a reader waits on.

Open questions

• The probe on every reconnect, not only on open? A file replaced while the screen is open is not made worse than it was, but the server-side check decision 3 rejects would close it. Decide after measuring how often the probe's round trip is what the reader waits on.
• Images. SessionImage fetches bytes from the files route on draw; they are not part of this cache and are re-downloaded per view. A separate, simpler cache (a directory of refs, no ordering) if the measurement above says the images are where the data goes.