ai-app/docs/TRANSCRIPT_CACHE.md

# The transcript cache

Asked for by Iris on 2026-09-04 and built the same day: keep the transcripts
of recently visited sessions on the phone, so reopening one does not download
it again. It has to save data over the tunnel, must not disturb a reply that
is streaming when the screen is reopened, must never skip an event, and needs
a manual reload for when the file on the machine has changed under it.

Like EXPLORER.md this records each decision with its reason and what was
rejected, so that when one changes it is changed here rather than re-argued.
"What building it changed" at the foot says which of them moved while it was
being built. How to exercise it, and what has bitten, are in AGENTS.md.

## What it is, in one paragraph

A per-session file on the phone holding the exact JSON lines the server has
already sent, in transcript order, with a record of which sequence numbers
each run of lines covers. Everything the session screen fetches — the opening
window, the pages it scrolls back through, the span an anchor restore reaches
for — is asked of the cache first and of the server only for what the cache
does not hold, and everything that arrives from the server is written into
it. The live stream then resumes from the newest cached event, exactly as it
resumes from the newest event on screen, so the server sends only what
happened since. One tiny request checks that the cached tail is still what
the server has before the stream is opened from it, and a button in session
settings throws the cache away and rebuilds the screen as a cold open for the
cases that check cannot see.

## The invariants

When a decision below looks arbitrary, it is one of these forcing it.

1. **What is on screen is what the server's transcript says, in order, with
   nothing missing, for every sequence number the screen claims to show.**
   The cache is a copy of server output and is never inferred, folded, or
   edited on the phone. Where the copy cannot be shown to be current, it is
   thrown away, not patched.
2. **A cached line is never ahead of the live cursor, and the live cursor is
   never ahead of the cache.** The stream resumes from the newest cached
   event, so a reply that was mid-stream when the screen closed picks up at
   its next delta and folds into the same row.
3. **The cache is never load-bearing.** A missing, evicted, corrupt or
   unwritable cache degrades to a cold open, never to a blank or wrong
   screen. Every path that reads it has a network path beside it producing
   the same result.
4. **Data crosses the tunnel once.** A line already on the phone is not
   fetched again unless the reader asks (the reload button) or the check in
   decision 3 says it must be.

## Decisions

### 1. Raw server lines, on the phone, keyed by server and session

The cache stores the server's own JSON, one event per line, byte-for-byte as
it arrived: the elements of the `/transcript` array and the `data:` payload
of each SSE frame. Reading the cache runs the same `parseSeqEvent` the
network path runs, so a cached transcript and a fetched one cannot draw
differently, and an event type this build does not know
(`SessionEvent.Unknown`) survives on disk for the build that will.

It lives under `context.cacheDir`, which is exactly what that directory is
for: bytes the phone can regenerate from the server, which Android may delete
under storage pressure without asking. Keyed by the server's host and port,
because two servers can hold a session with the same id (the sandbox and the
real server, or a re-enrolment) and a line from one shown against the other
is invariant 1 broken. The `v1` segment is the format version: any change to
the layout below bumps it, and a directory of another version is deleted on
first use.

Rejected: a database (Room, SQLite). The access pattern is "the newest N
lines" and "the lines before seq X", on files of tens of megabytes at most,
and a JSONL file per contiguous run answers both by reading from its end. A
database would be a new dependency for an index the file layout provides.

Rejected: caching folded `TranscriptItem` rows instead of events. Rows are a
*rendering* of events, and their shape changes when the fold changes; the
cache would need invalidating on every app update that touched `foldEvent`,
and would still have to keep raw seqs for the stream cursor. Events are the
server's contract and the only thing that is stable.

### 2. Chunks with explicit coverage; one contiguous run behind the cursor

A page from the server is a set of lines *and a claim about what they cover*,
and the two are not the same thing. A coalesced page joins each run of
`assistantText` deltas into one event carrying the seq of its *oldest* delta,
so a page whose newest event has seq 1,200 may in fact cover every line up to
the `before` it was asked with, say 1,650. Nothing in the lines themselves
says so. So each stored chunk records its coverage as a half-open range
`[first, end)`, where `end` is the `before` the request was made with — or,
for a raw chunk, its newest seq plus one.

Chunks are files named by their coverage:

    <first>-<end>.rows.jsonl     a coalesced page; end is the `before` it was fetched with
    <first>-<end>.raw.jsonl      an uncoalesced page or a closed live run
    <first>-open.raw.jsonl       the live run: appended to by the stream

Two chunks are **adjacent** when one's `end` equals the other's `first`. The
cache serves only the contiguous run of adjacent chunks that ends at the
newest raw chunk (the **suffix**); chunks behind a gap are kept on disk,
because the gap is usually filled (decision 4), but are never served across
it.

**The newest chunk is always raw.** That is what makes the stream cursor and
the probe well defined: a raw chunk's last line is a real event at a real
seq, and the server never coalesces the newest window. It holds by
construction — the opening window is fetched with no `before`, stream frames
are raw, and a `reset` window is raw — and is *checked* on read: a `.rows`
chunk found newest (which can only happen if the app died between closing one
live run and appending to the next) purges the session's cache.

There is at most one open chunk. A stream event whose seq is not the open
chunk's `end` — which is what a `reset` looks like from here — closes it by
renaming it with its real end and starts a new one. An event whose seq is
below the open chunk's `end` is already covered and is not written; the SSE
contract is `seq > after`, so that is a guard rather than a path.

Rejected: one file per session, rewritten to prepend older pages. A 20 MB
transcript would be rewritten on every page scrolled back to. The chunk
directory costs a directory listing per open instead.

Rejected: trimming chunks to resolve overlaps. A coalesced event cannot be
split at a seq inside its run, so an overlap between a coalesced page and an
existing chunk has no clean cut. The cache therefore **never stores a page
that overlaps an existing chunk**; decision 4 makes sure such a page is never
fetched, and one that arrives anyway is used for display and not stored.

### 3. The cached tail is checked against the server before the stream opens from it

The transcript file is append-only in ordinary use, but it can be replaced or
truncated — a sandbox re-seeded with the same ids, a backup restored, a
session deleted and re-imported — and `catch_up` on such a file would hand
the phone a continuation of a *different* conversation, spliced onto the
cached one with no seam. That is the worst thing this feature can do, and it
is caught with one request.

**The probe** is `GET /sessions/{id}/transcript?before=<cursor+1>&limit=1`,
where `cursor` is the seq of the cache's newest line. `read_window` with that
`before` returns the single newest event with seq ≤ cursor, which is the
event *at* the cursor when it exists. It passes when that response, parsed
with `parseSeqEvent`, is `==` to the cached line parsed the same way — over
seq, ts, and the whole event. It fails when the response is empty, is a
different seq, or differs in any field.

That equality rested on an assumption this plan stated and did not check:
that the two ways the server hands out a line agree bit for bit. **They did
not**, and the server was fixed — see AGENTS.md's entry on `float_roundtrip`.
Comparing everything *except* `ts` was the other option and was rejected: a
re-seeded fixture is identical in content and differs only in when it
happened, which is exactly the case the probe exists for.

A failed probe **purges the session's cache and proceeds as a cold open**. A
probe that cannot be made leaves the cached transcript on screen, shows the
error on the stream banner where a connection failure shows today, and is
retried on the stream loop's schedule; the stream is never opened until a
probe has passed once for this screen instance.

What the probe does *not* catch: a line changed in the middle of the file
with the tail intact, or a file rewritten so that the event at the cursor
happens to be identical. Those are what the reload button is for, and the
button's caption says so.

Cost: one request of a few hundred bytes, in the slot where the opening
page's request would be — so the round trips before the stream is live are
unchanged at two, and the bytes fall from a page to a line. The cached rows
are drawn *before* the probe returns, which is the whole point; a failed
probe replaces them, with the same appearance as a `reset`.

Rejected: a server-side check on the stream, answered with a distinct frame
when the event at N is not what the phone thinks. Strictly better coverage —
it would run on every reconnect — and no extra round trip. Not chosen because
it puts a cache's validation into a protocol that otherwise knows nothing
about caching, and because the reset frame already has to keep meaning "you
are behind, your history is fine". Worth revisiting if the probe's round trip
is ever measured as the thing making reopen slow.

Rejected: trusting the cache and relying on the reload button. Invariant 1 is
not something a button restores after the fact.

Rejected: fetching the newest page as before and using it to validate the
overlap. Zero saving on the opening page, which is the request paid on every
open.

### 4. Pages ask the server only for the gap: `after` on `/transcript`

After a reader has been away, the cache holds `[a, b)` and the screen holds
the newest window `[W, …)` with a gap between `b` and `W`. Paging back from
`W` asks for a coalesced page before `W`, and that page may reach back past
`b` — a single reply is hundreds of lines, so forty rows can be thousands of
seqs — producing exactly the overlap decision 2 refuses to store. Left like
that, every cached chunk would be dropped in turn as the reader paged back
through the gap, and the cache would save nothing for the sessions it exists
for.

So the transcript route takes a lower bound, `after`, named to match the SSE
route's (exclusive, `seq > after`). `read_window` starts the walk at
`first_at_or_after(after + 1)` instead of at `end - limit`. A delta run cut
at the start is emitted as the partial it is, exactly as one cut by `limit`
already is, and `healSplitMessage` welds it on the phone — no new mechanism.

The phone passes `after = b - 1` where `b` is the `end` of the nearest chunk
whose `end ≤ before`, and nothing when there is none. A page that comes back
with `first == b` is adjacent, and the suffix now runs through the old
chunks: the gap is closed with exactly the bytes it was wide, and the history
behind it is served locally from then on.

Rejected: fetching the gap raw in one request, which is what the anchor
restore does. Exact, but a gap of ten thousand lines is several megabytes
downloaded to save re-downloading history the reader may never scroll to.

Rejected: dropping the cached run whenever a gap opens. Being more than
`CATCH_UP_LIMIT` (200) events behind is the *ordinary* state of an active
session revisited — 200 raw events is one reply — so this would empty the
cache for exactly the sessions that are opened most.

### 5. A page is served locally in rows, mirroring the server's count

`loadOlderPage` asks for `HISTORY_PAGE` (40) **rows** when coalescing and for
a number of **events** otherwise (the anchor restore). Served from the cache,
the events branch is the `limit` lines before `before`. The rows branch walks
back counting rows the way `parse_coalesced` does — every event that is not
an `assistantText` is a row, and each maximal run of `assistantText` lines is
one row — stopping only between rows. It does not join the deltas; the fold
does that, and the joined row keeps the seq of its first delta either way, so
anchors and the next `before` land where they do on the network path.

A cached page is allowed to be **short**: a walk that reaches the suffix's
oldest chunk returns what it found. The caller already treats a short page as
a page; only an *empty* page means "start of the conversation", and the cache
never returns one — it returns `null` (a miss) and the network is asked.

A miss is `before` **outside what the suffix covers continuously** — above
its newest `end`, or at or below its oldest `first`. This plan first said a
miss was "no chunk of the suffix ends at `before`", which is wrong in the
commonest case there is: a warm open draws the newest eighty lines of the
live run, so the cursor the reader then scrolls back from is in the *middle*
of a chunk. Under the narrower rule every warm open sent its first backwards
page to the server, and that page overlapped what the phone already held and
could not be stored, so the same history was fetched again on every visit.
The feature would have saved the opening window and nothing else.

The row rule is a copy of the server's, and copies drift. It is short, it is
pure, and it is under a JVM unit test with the same fixture as the server's
`coalescing_counts_rows_and_joins_delta_runs` — a run cut by the limit, a
`usageDelta` inside a run (the server flushes the run there, so it is two
rows), and a page that is all one run.

### 6. What a `reset` means for the cache: behind, not wrong

The server sends `reset` when the cursor is more than `CATCH_UP_LIMIT` events
behind, then the newest 200 raw events. For the cache that means **the
history is intact and there is a gap**: the probe passed, the file is
append-only, and the window's first seq is above the open chunk's end. The
store learns this from the first window event's seq and needs no signal from
the screen; the gap is filled by paging.

The reset handler also clears `queued` and `waitingCommands`, which it did
not originally. Both are folded from events, and a `messageQueued` whose
resolving `userMessage` fell in the gap would otherwise draw a waiting bubble
for a message the session has long since read. That was a latent bug made
likely by the cache, because a cached tail is older than a fetched one.
`contextTokens` needs no clearing: `UsageDelta.context` is absolute, so the
window's first one corrects it.

### 7. Session state that is not the transcript comes from the list, not the cache

`apply` derives `status`, `model`, `permissionMode` and `compactingSince`
from `Status` and `Settings` events. Replayed from a fetched page those are
current; replayed from the cache they are as old as the last visit, while the
list row the reader just tapped was fetched moments ago. So the cache replay
runs through `apply` for the transcript's sake and then **reassigns those
four from `summary`**, which is the newer of the two measurements; the
stream's catch-up then makes them current. Without this a session that
finished an hour ago would open saying "working" until the stream connected,
which is a status row lying for a round trip.

### 8. Reload, in session settings

A row under the working directory showing what the button discards:

    [ Transcript ]  2.3 MB cached          [ Reload ]

The size is the unknown state made visible — `null` while the directory is
being measured (spinner, as the notifications switch does), "nothing cached"
when the directory is absent or empty, else the size. The caption is in the
style of Move's, because the button costs something the reader cannot see:
*"Reload throws away this phone's copy and fetches the transcript from the
server again. Use it when what is shown here disagrees with the file on the
machine."*

Pressing it purges the session's cache directory, closes the dialog, and
rebuilds the screen as a cold open, with the reader put back where they were.
The mechanism is an `epoch` counter in the key of the opening effect and the
stream effect; incrementing it cancels both and relaunches them. `savedAnchor`
is keyed on the epoch too, so the restore reads the anchor saved at the
reader's *current* position. The button is enabled whether or not anything is
cached: "what I see disagrees with the machine" is a state an empty cache can
also be in, and a control that comes and goes makes its own presence the
signal.

Nothing is announced on success — the transcript shows the opening spinner
and then the rows, which is what the screen already says about a reload. A
failure is the opening fetch's, and lands on the stream banner.

Rejected: a global "clear transcript cache" in the app's settings. Not asked
for; eviction bounds the total, and the per-session button is where the
reader is when they notice a problem. Easy to add as one more caller of
`purgeAll`.

### 9. Budget, eviction, pruning

Bounded three ways, each with its path out written beside the path in:

- **Budget.** `CACHE_BUDGET_BYTES` is 256 MB across all sessions of one
  server. Each open touches the session directory's mtime; after the opening
  replay, on `Dispatchers.IO`, the store sums the server's directories and
  deletes least-recently-touched ones (never the one on screen) until under
  budget. 256 MB is a dozen of the largest transcripts seen in this VM
  (21 MB for 24,000 events) and a small fraction of a phone; it is a number
  to revisit against real use, not a measurement.
- **Deleted sessions.** The list screen's delete purges after `deleteSession`
  succeeds, and every successful list fetch calls `retainOnly(ids)`, so a
  session deleted from another device is pruned on the next visit to the
  list. `Drafts.kt` chose not to prune because its residue is bytes; here it
  is megabytes.
- **Android.** `cacheDir` may be emptied at any moment, including while a
  screen is open. Every read tolerates a missing directory and every write
  failure is swallowed once.

### 10. The cache never breaks the screen

Every store operation that touches the disk catches `IOException` and answers
as if the cache were empty: `null` from a read, no-op from a write, logged
once. After a write failure the instance stops writing, so a full disk costs
one log line rather than one per delta. A line at the end of an open chunk
that does not parse — the app died mid-write — is dropped and the file
truncated to the last good line before anything is served from it; a line
that does not parse anywhere else purges the session's cache, since that file
was not written by this code. None of this is reported on screen: none of it
changes what the screen shows, and the reader has nothing to do about it.

## Layout on disk

    <cacheDir>/transcripts/
      v1/
        10.0.2.2_8443/                      one directory per server (host_port)
          3f2c…/                            one per session id
            1-1650.rows.jsonl               coalesced page: covers seqs 1..1649
            1650-2001.rows.jsonl
            2001-2400.raw.jsonl             a closed live run
            2600-open.raw.jsonl             the live run

Here 2400..2599 is a gap: the reader was away for two hundred events and the
stream reset. The suffix is the single chunk `2600-open`; the first backwards
page asks the server for `before=2600&after=2399&coalesce=true`, and once a
page comes back with `first == 2400` the suffix runs to seq 1.

Each `.jsonl` is one JSON object per line, oldest first, exactly as the
server sent it. No header, no index: coverage is in the name, order is the
file's, and the seq is in every line.

## What building it changed

Each of these contradicted the plan, and each was found by running it rather
than by reading it. The decisions above are amended in place; this is what
moved, so a reader who remembers the first version knows what to re-read.

- **The probe's equality had a false premise** (decision 3). The server did
  not hand out the same line twice the same way. Fixed on the server.
- **A cached page starts anywhere inside the run** (decision 5). Requiring a
  chunk boundary would have made the cache save the opening window and
  nothing else.
- **The opening window is stored by `append`, not by `storePage`.** The
  sketch had `storePage` grow a special case for "this page is the new open
  chunk", decided by an implicit condition a raw history page also satisfies.
  Appending each line instead is the mechanism that already exists, and the
  open chunk stays the one thing that grows.
- **Chunks are read backwards, in blocks, and never whole.** Every question
  the cache is asked is about the newest end, and a live run reaches the size
  of the conversation — so reading a chunk to answer with eighty lines of it
  is the cost the server's own reader was rewritten to stop paying, arriving
  on the phone. Damage is therefore noticed when a read reaches it rather
  than up front, which is the better time: what is not read cannot be wrong.
- **The stream waits for the opening effect's probe.** The screen lifts
  `ready` before the probe returns — that is the point of the cache — so
  `ready` stopped being the whole gate, and the stream loop asked the same
  question a second time and raced its own answer. Two probes per warm open,
  visible in the server's log.
- **`SessionCache` is synchronized.** The stream appends live events from one
  IO thread while a reader scrolling back reads pages from another; the open
  chunk's name, its end and its writer must never be seen half-rotated.

## What it cost, measured

On the emulator against `app/ui-sandbox.sh`, 2026-09-04, on a session of 505
events (three short exchanges and two 300-delta replies):

- **Reopening it: one request, for one event.** The probe, and nothing else —
  including scrolling the whole conversation back to its first line. A cold
  open of the same session is two requests and 100 events.
- **A reset after falling 300 events behind costs the gap and no more.** The
  window arrived at seq 306, the phone held up to 202, and the first
  backwards page asked `before=306&after=201` and came back with **four
  coalesced rows** covering 202..305 — against the 104 raw events an
  unbounded page would have re-fetched and thrown away.
- **Every chunk is exactly what the server says for the range its name
  claims**, checked line by line against `/transcript` for each chunk's own
  `before`/`after`/`coalesce`, across a reset and a gap-fill.
- **Nothing about drawing changed**, which is what a cache must not do:
  `transcript-bench.sh` before and after, same viewport content and gestures,
  p50 16.9ms both times and the transcript's own draw accounting at 0.33ms
  against 0.32ms.

Still to measure, in real use rather than here: the size the cache reaches
against `CACHE_BUDGET_BYTES`, and whether the probe's round trip is ever what
a reader waits on.

## Open questions

- **The probe on every reconnect, not only on open?** A file replaced *while*
  the screen is open is not made worse than it was, but the server-side check
  decision 3 rejects would close it. Decide after measuring how often the
  probe's round trip is what the reader waits on.
- **Images.** `SessionImage` fetches bytes from the files route on draw; they
  are not part of this cache and are re-downloaded per view. A separate,
  simpler cache (a directory of refs, no ordering) if the measurement above
  says the images are where the data goes.