ai-app/TRANSCRIPT_CACHE.md

# The transcript cache

Asked for by Iris on 2026-09-04: keep the transcripts of recently visited
sessions on the phone, so reopening one does not download it again. It has
to save data over the tunnel, it must not disturb a reply that is streaming
when the screen is reopened, it must never skip an event, and session
settings needs a manual reload for when the file on the machine has
changed under it.

Built 2026-09-04. Like EXPLORER.md this records each decision with its
reason and what was rejected, so that when one changes it is changed here
rather than re-argued -- three of them changed during the building, and
"What building it changed" at the foot says which and why. What it is *not*
is the operational half: how to exercise it, and what has bitten, are in
AGENTS.md with the rest of the working notes.

## What it is, in one paragraph

A per-session file on the phone holding the exact JSON lines the server has
already sent, in transcript order, with a record of which sequence numbers
each run of lines covers. Everything the session screen fetches today --
the opening window, the pages it scrolls back through, the span an anchor
restore reaches for -- is asked of the cache first and of the server only
for what the cache does not hold, and everything that arrives from the
server is written into it. The live stream then resumes from the newest
cached event, exactly as it resumes today from the newest event on screen,
so the server sends only what happened since. One tiny request checks that
the cached tail is still what the server has before the stream is opened
from it, and a button in session settings throws the cache away and
rebuilds the screen as a cold open for the cases that check cannot see.

## The invariants

Everything below is in service of four rules. When a decision looks
arbitrary, it is one of these forcing it.

1. **What is on screen is what the server's transcript says, in order,
   with nothing missing, for every sequence number the screen claims to
   show.** The cache is a copy of server output and is never inferred,
   folded, or edited on the phone. Where the copy cannot be shown to be
   current, it is thrown away, not patched.
2. **A cached line is never ahead of the live cursor, and the live cursor
   is never ahead of the cache.** The stream resumes from the newest cached
   event, so a reply that was mid-stream when the screen closed picks up at
   its next delta and folds into the same row, as it does today when the
   phone merely lost the tunnel for a second.
3. **The cache is never load-bearing.** A missing, evicted, corrupt or
   unwritable cache degrades to today's behaviour -- a cold open -- and
   never to a blank or wrong screen. Every path that reads it has a
   network path beside it that produces the same result.
4. **Data crosses the tunnel once.** A line already on the phone is not
   fetched again unless the reader asks for that (the reload button) or the
   check in decision 3 says it must be.

## Decisions

### 1. Raw server lines, on the phone, keyed by server and session

The cache stores the server's own JSON, one event per line, byte-for-byte
as it arrived: the elements of the `/transcript` array and the `data:`
payload of each SSE frame. Reading the cache means running the same
`parseSeqEvent` the network path runs, so a cached transcript and a fetched
one cannot draw differently, and an event type this build does not know
(`SessionEvent.Unknown`) survives on disk for the build that will.

It lives under `context.cacheDir` -- `<cacheDir>/transcripts/v1/<host>_<port>/<sessionId>/`
-- because it is exactly what that directory is for: bytes the phone can
regenerate from the server, which Android may delete under storage
pressure without asking. Keyed by the server's host and port because two
servers can hold a session with the same id (the sandbox and the real
server, or a re-enrolment), and a line from one shown against the other
is invariant 1 broken. `ServerSettings` has both fields; the key is
`"${settings.host}_${settings.port}"` with `:` never appearing in it.
The `v1` segment is the format version: any change to the layout below
bumps it, and a directory of another version is deleted on first use.

Rejected: a database (Room, SQLite). The access pattern is "the newest N
lines" and "the lines before seq X", on files of tens of megabytes at most,
and a JSONL file per contiguous run answers both by reading from its end.
A database would be a new dependency for an index the file layout already
provides.

Rejected: caching folded `TranscriptItem` rows instead of events. Rows are
a *rendering* of events, and their shape changes when the fold changes;
the cache would need invalidating on every app update that touched
`foldEvent`, and would still have to keep raw seqs for the stream cursor.
Events are the server's contract and the only thing that is stable.

### 2. Chunks with explicit coverage; one contiguous run behind the cursor

A page from the server is a set of lines *and a claim about what they
cover*, and the two are not the same thing. A coalesced page
(`coalesce=true`, which the scroll-back pager asks for) joins each run of
`assistantText` deltas into one event carrying the seq of its *oldest*
delta, so a page whose newest event has seq 1,200 may in fact cover every
line up to the `before` it was asked with, say 1,650. Nothing in the lines
themselves says so. So each stored chunk records its coverage as a
half-open range `[first, end)`, where `first` is the seq of its oldest
event and `end` is the `before` the request was made with -- or, for a
raw chunk, its newest seq plus one.

Chunks are files named by their coverage:

    <first>-<end>.rows.jsonl     a coalesced page; end is the `before` it was fetched with
    <first>-<end>.raw.jsonl      an uncoalesced page or a closed live run
    <first>-open.raw.jsonl       the live run: appended to by the stream; end = last line's seq + 1

Two chunks are **adjacent** when one's `end` equals the other's `first`.
The cache serves only the contiguous run of adjacent chunks that ends at
the newest raw chunk (the **suffix**); chunks behind a gap are kept on
disk, because the gap is usually filled (decision 4), but are never served
across the gap.

**The newest chunk is always raw.** That is what makes the stream cursor
and the check in decision 3 well defined: a raw chunk's last line is a real
event at a real seq, and the server never coalesces the newest window
("the live cursor depends on real seqs", `read_window`). It holds by
construction -- the opening window is fetched with no `before`, stream
frames are raw, and a `reset` window is raw -- and is *checked* on read:
if the newest chunk on disk is a `.rows` chunk (which can only happen if
the app died between closing one live run and appending to the next), the
session's cache is purged and the open is cold.

There is at most one open chunk. When a stream event arrives whose seq is
not the open chunk's `end` -- which is what a `reset` looks like from
here, see decision 6 -- the open chunk is closed by renaming it with its
real end, and a new open chunk starts at the arriving seq. An event whose
seq is below the open chunk's `end` is already covered and is not written
(the SSE contract is `seq > after`, so this is a guard, not a path).

Rejected: one file per session, rewritten to prepend older pages. A
20 MB transcript would be rewritten on every page scrolled back to. The
chunk directory costs a directory listing per open instead.

Rejected: trimming chunks to resolve overlaps. A coalesced event cannot be
split at a seq inside its run, so an overlap between a coalesced page and
an existing chunk has no clean cut. The cache therefore **never stores a
page that overlaps an existing chunk**; decision 4 makes sure such a page
is never fetched in the first place, and if one arrives anyway (a server
without decision 4's change) it is used for display and not stored.

### 3. The cached tail is checked against the server before the stream opens from it

The screen must not resume a stream from a cached seq unless the server's
event at that seq is the one in the cache. The transcript file on the
machine is append-only in ordinary use, but it can be replaced or
truncated -- a sandbox re-seeded with the same ids, a backup restored, a
directory deleted and the session re-imported under the same name -- and
`catch_up` on such a file would hand the phone a continuation of a
different conversation, spliced onto the cached one with no seam. That is
the worst thing this feature can do, and it is caught with one request.

**The probe:** `GET /sessions/{id}/transcript?before=<cursor+1>&limit=1`,
where `cursor` is the seq of the cache's newest line. `read_window` with
that `before` returns the single newest event with seq ≤ cursor, which is
the event *at* the cursor when it exists. The probe passes when that
response, parsed with `parseSeqEvent`, is `==` to the cached line parsed
the same way -- data-class equality over seq, ts, and the whole event. It
fails when the response is empty, is a different seq, or differs in any
field.

That equality rested on an assumption this plan stated and did not check:
that the two ways the server hands out a line agree bit for bit. **They did
not.** `serde_json`'s default float parser is not correctly rounded, so a
`ts` of `1788546972.6030757` written to the transcript came back from
`/transcript` as `...0755`, while the SSE stream -- serializing the same
struct -- sent the original. Measured on the emulator 2026-09-04: 23 of 330
cached lines differed from the server's answer in the last bit, so the probe
would have failed on any session whose cached tail happened to be one of
them, silently and only sometimes. That is a defect in the server
independent of this feature -- two answers to "what is line 30" -- and it is
fixed there, with `float_roundtrip` and a test
(`a_line_read_back_is_the_line_that_was_written`) that fails the moment the
feature is dropped. Comparing everything *except* `ts` was the other option
and was rejected: a re-seeded fixture is identical in content and differs
only in when it happened, which is exactly the case the probe exists for. A failed probe **purges the session's cache
and proceeds as a cold open**. A probe that cannot be made (no route to
the server) leaves the cached transcript on screen, shows the request's
error on the stream banner where a connection failure shows today, and is
retried on the stream loop's schedule (`RECONNECT_DELAY_MS`); the stream
is never opened until a probe has passed once for this screen instance.

What the probe does *not* catch: a line changed in the middle of the file
with the tail intact, or a file rewritten so that the event at the cursor
happens to be identical. Those are what the reload button is for, and the
button's caption says so.

Cost: one request of a few hundred bytes, one round trip, in the slot
where the opening page's request is today -- so the round trips before
the stream is live are unchanged at two, and the bytes fall from a page to
a line. The cached rows are drawn *before* the probe returns, which is the
whole point of the feature; a failed probe replaces them, the same
appearance as a `reset`.

Rejected: a server-side check on the stream (`events?after=N&ts=T`,
answered with a distinct frame when the event at N is not what the phone
thinks). Strictly better coverage -- it would run on every reconnect, not
only on open -- and no extra round trip. Not chosen because it puts a
cache's validation into a protocol that otherwise knows nothing about
caching, and because the reset frame already has to keep meaning "you are
behind, your history is fine" (decision 6), so a second frame would be
needed. Worth revisiting if the probe's round trip is ever measured as the
thing making reopen slow; note it as the alternative here and in PLAN.md.

Rejected: trusting the cache without a check and relying on the reload
button. Invariant 1 is not something a button restores after the fact.

Rejected: fetching the newest page as today and using it to validate the
overlap. Zero saving on the opening page, which is the request paid on
every open.

### 4. Pages ask the server only for the gap: `after` on `/transcript`

After a reader has been away, the cache holds `[a, b)` and the screen
holds the newest window `[W, …)` with a gap between `b` and `W`. Paging
back from `W` asks the server for a coalesced page before `W`, and that
page may reach back past `b` -- a single reply is hundreds of lines, so
forty rows can be thousands of seqs -- producing exactly the overlap
decision 2 refuses to store. Left like that, every cached chunk would be
overlapped and dropped in turn as the reader paged back through the gap,
and the cache would save nothing for the sessions it exists for.

So the transcript route gains a lower bound. `TranscriptQuery` in
`server/src/routes.rs` gets

    /// Return nothing at or below this seq; the page stops here instead of at `limit`.
    /// The phone passes the end of what it already holds, so a page never overlaps it.
    #[serde(default)]
    after: Option<u64>,

named to match the SSE route's `after` (exclusive, `seq > after`).
`read_window(path, before, after, limit, coalesce)` in
`server/src/session/transcript.rs` computes
`start = first_at_or_after(after + 1)` and stops the walk there: the raw
branch parses `max(start, end - limit)..end`; `parse_coalesced` takes a
`start` and its `while index > 0` becomes `while index > start`. A delta
run cut at `start` is emitted as the partial it is, exactly as one cut by
`limit` already is, and `healSplitMessage` welds it on the phone -- no new
mechanism. The route's table comment in `routes.rs` gains the parameter,
and `transcript.rs` gets a test beside
`a_window_is_the_events_before_a_cursor_and_nothing_else`: with `after`
set, the page's oldest seq is greater than `after`, and with `after` set
inside a delta run the partial run's seq is the first delta above `after`.

The phone passes `after = b - 1` where `b` is the `end` of the nearest
chunk whose `end ≤ before`, and nothing when there is none. A page that
comes back with `first == b` is adjacent, and the suffix now runs through
the old chunks: the gap is closed with exactly the bytes it was wide, and
the history behind it is served locally from then on.

Rejected: fetching the gap raw in one request (`before=W&limit=W-b`,
which is what the anchor restore already does). Exact, but a gap of ten
thousand lines is several megabytes downloaded to save re-downloading
history the reader may never scroll to; the feature exists to save data.
Paging as today with a bound saves the same bytes and fetches only what
is read.

Rejected: dropping the cached run whenever a gap opens. Being more than
`CATCH_UP_LIMIT` (200) events behind is the *ordinary* state of an active
session revisited -- 200 raw events is one reply -- so this would empty
the cache for exactly the sessions that are opened most.

### 5. A page is served locally in rows, mirroring the server's count

`loadOlderPage` asks for `HISTORY_PAGE` (40) **rows** when
`coalesce = true`, and for a number of **events** otherwise (the anchor
restore). Served from the cache, the events branch is the `limit` lines
before `before`. The rows branch walks back from the line before `before`
counting rows the way `parse_coalesced` does: every event that is not an
`assistantText` is a row, and each maximal run of `assistantText` lines is
one row; it stops only between rows, once `limit` rows are complete, and
returns the raw lines oldest-first. It does not join the deltas -- the
fold does that (`foldEvent` appends a delta to a preceding
`AssistantMsg`), and the joined row keeps the seq of its first delta either
way, so anchors and the next `before` land where they do today.

A cached page is allowed to be **short**: the suffix's oldest chunk starts
at some `first`, and a walk that reaches it returns what it found. The
caller already treats a short page as a page; only an *empty* page means
"start of the conversation" (`moreHistory = false`), and the cache never
returns an empty page -- it returns `null` (a miss) and the network is
asked. The walk may cross a chunk boundary inside the suffix, since adjacent
chunks are one run; a delta run straddling a boundary counts as one row, as
it should.

A miss is `before` **outside what the suffix covers continuously** -- above
its newest `end`, or at or below its oldest `first`. This plan first said a
miss was "no chunk of the suffix ends at `before`", which is wrong in the
commonest case there is: a warm open draws the newest eighty lines of the
live run, so the cursor the reader then scrolls back from is in the *middle*
of a chunk, not at a boundary. Under the narrower rule every warm open sent
its first backwards page to the server, and that page -- reaching back past
the run the phone already held -- overlapped it and could not be stored, so
the same history was fetched again on every visit. The feature would have
saved the opening window and nothing else.

The row rule is a copy of the server's, and copies drift. It is short
(one comparison), it is pure, and it goes under a JVM unit test with the
same fixture as the server's `coalescing_counts_rows_and_joins_delta_runs`
-- the three cases are a run cut by the limit, a `usageDelta` inside a run
(the server flushes the run there, so it is two rows), and a page that is
all one run.

### 6. What a `reset` means for the cache: behind, not wrong

The server sends `reset` when the cursor is more than `CATCH_UP_LIMIT`
events behind, then the newest 200 raw events. The screen already drops
everything and rebuilds from that window. For the cache, a reset means
**the history is intact and there is a gap**: the probe passed, the file
is append-only, and the window's first seq is above the open chunk's end.
The store learns this from the first window event's seq (decision 2:
a seq that is not the open chunk's `end` closes it and opens a new chunk)
and needs no signal from the screen; the gap is filled by paging
(decision 4).

Two things the reset handler in `SessionScreen` does not clear today and
must: `queued` and `waitingCommands`. Both are folded from events, and a
`messageQueued` whose resolving `userMessage` fell in the gap would
otherwise draw a waiting bubble for a message the session has long since
read. This is a latent bug today, made likely by the cache because a
cached tail is older than a fetched one. `contextTokens` needs no change:
`UsageDelta.context` is absolute, so the window's first one corrects it.

### 7. Session state that is not the transcript comes from the list, not the cache

`apply` derives `status`, `model`, `permissionMode` and `compactingSince`
from `Status` and `Settings` events. Replayed from a fetched page those are
current; replayed from the cache they are as old as the last visit, while
`summary.status`, `summary.model` and `summary.permissionMode` -- the row
the reader just tapped -- were fetched moments ago. So the cache replay
runs through `apply` for the transcript's sake (queued bubbles, context,
rows) and then **reassigns those four from `summary`**, which is the newer
of the two measurements; the stream's catch-up then makes them current.
Without this a session that finished an hour ago would open saying
"working" until the stream connected, which is a status row lying for a
round trip.

### 8. Reload, in session settings

`SessionSettingsDialog` gains a row under the working directory:

    [ Transcript ]  2.3 MB cached          [ Reload ]

The size is what the button discards, and it is the unknown state made
visible: `null` while the directory is being measured (spinner, as the
notifications switch does), "nothing cached" when the directory is absent
or empty, else the size. A caption in the style of Move's, because the
button costs something the reader cannot see:

    Reload throws away this phone's copy and fetches the transcript from the
    server again. Use it when what is shown here disagrees with the file on
    the machine.

Pressing it: purge the session's cache directory, close the dialog, and
rebuild the screen as a cold open -- the same sequence as `reset` plus a
fresh opening fetch, with the reader put back where they were. The
mechanism is an `epoch` counter (`mutableIntStateOf(0)`) added to the key
of the opening effect and the stream effect; incrementing it cancels both
(the stream's `finally` closes the socket) and relaunches them. State the
relaunch must see cleared: `items`, `replies.clear()`, `held`, `oldestSeq
= 0`, `moreHistory = true`, `queued`, `waitingCommands`, `lastSeq.set(0)`,
`ready = false`. `savedAnchor` becomes `remember(summary.id, epoch)` so
the restore path reads the anchor saved at the reader's *current*
position (the anchor saver writes on every settle, so it is there), and
`restoring` is re-derived from it. The button is enabled whether or not
anything is cached: "what I see disagrees with the machine" is a state an
empty cache can also be in, and a control that comes and goes makes its
own presence the signal.

Nothing is announced on success. The transcript shows the opening spinner
and then the rows, which is what the screen already says about a reload.
A failure is the opening fetch's, and lands on the stream banner where
that failure lands today.

Rejected: a global "clear transcript cache" in the app's settings screen.
Not asked for; eviction (decision 9) bounds the total, and the per-session
button is where the reader is when they notice a problem. Easy to add as
one more caller of `TranscriptCache.purgeAll` if wanted.

### 9. Budget, eviction, pruning

The cache is bounded three ways, each with its path out written beside
the path in:

- **Budget.** `CACHE_BUDGET_BYTES = 256 MB` across all sessions of one
  server. Each open touches the session directory's mtime; after the
  opening replay, on `Dispatchers.IO`, the store sums the server's
  directories and deletes least-recently-touched session directories
  (never the one on screen) until under budget. 256 MB is a dozen of the
  largest transcripts seen in this VM (21 MB for 24,000 events) and a
  small fraction of a phone; it is a number to revisit against real use,
  not a measurement.
- **Deleted sessions.** `SessionListScreen`'s delete calls
  `cache.session(id).purge()` after `deleteSession` succeeds, and every
  successful list fetch calls `cache.retainOnly(ids)` for that server, so
  a session deleted from another device or from the backend is pruned on
  the next visit to the list. `Drafts.kt` chose not to prune because its
  residue is bytes; here it is megabytes, so the pass is worth having.
- **Android.** `cacheDir` may be emptied under pressure at any moment,
  including while a screen is open. Every read tolerates a missing
  directory (cold open) and every write failure is swallowed once and
  disables writing for that screen instance (decision 10).

### 10. The cache never breaks the screen

Every store operation that touches the disk catches `IOException` and
answers as if the cache were empty: `null` from a read, no-op from a
write, with the failure logged once at `Log.w("ai-app", …)`. After a
write failure the `SessionCache` instance sets `disabled = true` and
writes nothing more, so a full disk costs one log line rather than one
per delta. A line at the end of an open chunk that does not parse -- the
app died mid-write -- is dropped and the file truncated to the last
good line before anything is served from it; a line that does not parse
anywhere else purges the session's cache (that file was not written by
this code). None of this is reported on screen: none of it changes what
the screen shows, and the reader has nothing to do about it.

## Layout on disk

    <cacheDir>/transcripts/
      v1/
        10.0.2.2_8443/                      one directory per server (host_port)
          3f2c…/                            one per session id
            1-1650.rows.jsonl               coalesced page: covers seqs 1..1649
            1650-2001.rows.jsonl
            2001-2400.raw.jsonl             a closed live run
            2600-open.raw.jsonl             the live run; end = last line's seq + 1

Here 2400..2599 is a gap: the reader was away for two hundred events and
the stream reset. The suffix is the single chunk `2600-open`; the first
backwards page asks the server for `before=2600&after=2399&coalesce=true`,
and once a page comes back with `first == 2400` the suffix runs to seq 1.

Each `.jsonl` is one JSON object per line, oldest first, exactly as the
server sent it. No header, no index: coverage is in the name, order is the
file's, and the seq is in every line.

## What building it changed

Each of these contradicted something written above, and each was found by
running it rather than by reading it. The decisions themselves are amended
in place; this is the list of what moved, so that a reader who remembers the
first version knows what to re-read.

- **The probe's equality had a false premise** -- decision 3. The server did
  not hand out the same line twice the same way. Fixed on the server.
- **A cached page starts anywhere inside the run** -- decision 5. Requiring
  a chunk boundary would have made the cache save the opening window and
  nothing else.
- **The opening window is stored by `append`, not by `storePage`.** The
  sketch below had `storePage` grow a special case for "this page is the new
  open chunk", decided by an implicit condition that a raw history page also
  satisfies. Appending each line instead is the mechanism that already
  exists, and the open chunk stays the one thing that grows.
- **Chunks are read backwards, in blocks, and never whole.** Every question
  the cache is asked is about the newest end, and a live run reaches the size
  of the conversation -- so reading a chunk to answer with eighty lines of it
  is the cost the server's own reader was rewritten to stop paying, arriving
  on the phone. Damage is therefore noticed when a read reaches it rather
  than up front, which is the better time: what is not read cannot be wrong.
- **The stream waits for the opening effect's probe.** The screen lifts
  `ready` before the probe returns -- that is the point of the cache -- so
  `ready` stopped being the whole gate, and the stream loop asked the same
  question a second time and raced its own answer. Two probes per warm open,
  visible in the server's log.
- **`SessionCache` is synchronized.** The stream appends live events from
  one IO thread while a reader scrolling back reads pages from another; the
  open chunk's name, its end and its writer must never be seen
  half-rotated.

## What it cost, measured

On the emulator against `app/ui-sandbox.sh`, 2026-09-04, on a session of
505 events (three short exchanges and two 300-delta replies):

- **Reopening it: one request, for one event.** The probe, and nothing else
  -- including scrolling the whole conversation back to its first line. A
  cold open of the same session is two requests and 100 events.
- **A reset after falling 300 events behind costs the gap and no more.**
  The window arrived at seq 306, the phone held up to 202, and the first
  backwards page asked `before=306&after=201` and came back with **four
  coalesced rows** covering 202..305 -- against the 104 raw events an
  unbounded page would have re-fetched and then thrown away.
- **Every chunk is exactly what the server says for the range its name
  claims**, checked line by line against `/transcript` for each chunk's own
  `before`/`after`/`coalesce`, across a reset and a gap-fill.
- **Nothing about drawing changed**, which is what a cache must not do:
  `transcript-bench.sh` before and after, same viewport content and the same
  gestures, reported p50 16.9ms both times and the transcript's own draw
  accounting at 0.33ms against 0.32ms.

Still to measure, in real use rather than here: the size the cache reaches
against `CACHE_BUDGET_BYTES`, and whether the probe's round trip is ever
what a reader waits on.

## Open questions

- **The probe on every reconnect, not only on open?** Decision 3 probes
  once per screen instance. A file replaced *while* the screen is open is
  today's behaviour and not made worse, but the server-side check it
  rejects would close it. Decide after measuring how often the probe's
  round trip is what the reader waits on.
- **A reset arriving during an anchor restore** was an open worry when this
  was written, and was measured and closed on 2026-09-04 (see "The reconnect
  loop does not reproduce") before this landed. The cache makes the restore
  cheaper again -- a warm one is now the probe and nothing else -- so it can
  only have narrowed the window further. Worth re-measuring here only if a
  reader reports the screen reconnecting on reopen.
- **Images.** `SessionImage` fetches bytes from the files route on draw;
  they are not part of this cache and are re-downloaded per view. A
  separate, simpler cache (a directory of refs, no ordering) if the
  measurement above says the images are where the data goes.