Plan the file explorer

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

EXPLORER.md

@@ -0,0 +1,453 @@
+# The file explorer
+
+Asked for by Bryan on 2026-09-03: replace the session screen's debug
+button with a folder icon that opens a file and directory viewer for the
+machine the session runs on. Browse directories, open files with the
+existing syntax highlighting, line numbers, no wrapping; edit a file behind
+a pencil icon; create files through a modal like the ones the app already
+has; work over ssh; open at the session's working directory.
+
+This is the plan. Like PLAN.md it records each decision with the reason and
+what was rejected, so that when one changes it is changed here rather than
+re-argued. Once built, the operational notes (how to test it, what bit)
+move to AGENTS.md and this file keeps only the design.
+
+## What it is, in one paragraph
+
+A machine's filesystem, seen from the phone through the backend. The
+explorer belongs to a **setup** (a machine), not to a session: a session
+only says where to start. Every operation -- list, read, write, create --
+is one shell script run through `Transport`, exactly the way the import
+listing and the usage fetch already work, so the local and the ssh case
+are one implementation and a machine the backend cannot reach fails with
+ssh's own message. The phone draws what came back: a listing, a file with
+its lines coloured by the scanner in `Highlighter.kt`, or an editor over
+the same text.
+
+## Decisions
+
+### 1. Keyed on the machine, opened from the session
+
+Routes live under `/setups/{id}/…`, beside `importable`, because a
+filesystem is a property of a machine. The session screen's folder button
+opens the explorer with the session's setup and its `cwd` as the starting
+directory; a session with no `cwd` opens at the machine's home, which the
+machine resolves (`cd` with no argument and `pwd -P`), never a path the
+phone guessed. Nothing in the explorer knows what a session is, so a later
+entry point from the setups tab is one more caller and no new code.
+
+Rejected: routes under `/sessions/{id}/`. The session would be a detour to
+find the setup, and "browse this machine" from anywhere but a session would
+need a session to exist first.
+
+### 2. One shell script per operation, over `Transport`, on both transports
+
+Each operation is a small POSIX shell script handed to `sh -c script sh
+"$path" …` through `Transport::capture` (or the stdin-carrying variant
+below). The path and every other value cross as **positional arguments**,
+never interpolated into the script -- the same rule `import::find` follows
+with `"$1"`, and the same reason `ssh::quote` exists: a path is
+attacker-adjacent input in a server whose job is running commands. A `~`
+prefix is handled by the same `quote_path`/`expand_home` pair every other
+path goes through; nothing new is invented for it.
+
+The scripts assume GNU coreutils and findutils (`find -printf`, `stat -c`,
+`sha256sum`, `chmod --reference`). That is already what `import.rs`
+assumes (`stat -c`, `/proc`), and both machines that exist are Linux. A
+machine without them fails with that tool's own message, which names what
+is missing.
+
+Rejected: `std::fs` for the local transport and scripts for ssh. Two
+implementations of "list a directory" drift -- the ordering of entries,
+what a symlink reports, how a permission error reads -- and the local one
+is the one that gets tested, so the remote one ships broken. The transport
+design exists so that a driver never learns which machine it got; the
+explorer is held to the same rule. The cost is a `sh` process per
+operation locally, which is under a millisecond.
+
+Rejected: a Rust SSH or SFTP library. PLAN.md rule 23 -- the system `ssh`
+inherits `~/.ssh/config`, agents and jump hosts, and there is one place to
+configure a connection. SFTP would need a second one.
+
+### 3. The token can now name a path, and that is written down
+
+AGENTS.md says of the import route: "the phone picks an **id**, never a
+path: the server resolves which file that is, so an enrolled token cannot
+become 'read me an arbitrary file'." The explorer's whole purpose is the
+path, so it takes one. This is recorded in PLAN.md's Security section as a
+change to the threat model paragraph, in these terms: the token already
+gates spawning a bypass-permissions agent in any directory on any machine
+a setup names, and that agent can already read and write every file its
+user can. The explorer is a shorter path to authority the token already
+holds, not new authority. The import route's rule stands where it is,
+because there a path was unnecessary and refusing it cost nothing.
+
+What is *not* changed: no route accepts a command. Listing, reading and
+writing are fixed scripts; the phone chooses only the path and the bytes.
+
+### 4. Paths are absolute or `~`-prefixed, and the machine answers with the real one
+
+Same rule as `POST /sessions/{id}/cwd`: a relative path is refused with
+the same wording, because where it would be depends on where nothing the
+reader can see. Every listing answers with `pwd -P` of the directory it
+listed, so the phone navigates on a resolved absolute path -- the parent
+of `/home/bob/repos/ai-app` is a string operation on that, and a `~` the
+session was spawned with is shown as what it turned out to be. The phone
+never resolves `..` itself.
+
+### 5. A read is capped and typed, and every state it can be in has a word
+
+`GET /setups/{id}/file` answers with one of:
+
+- `text` -- the content, with its size, mtime and sha256.
+- `binary` -- the content is not UTF-8. Size reported, nothing shown.
+- `tooBig` -- over `FILE_LIMIT` (1 MiB to start; see "Numbers to
+  measure"). Size reported so the reader knows what they are looking at.
+- an error -- no such file, permission denied, machine unreachable --
+  carrying the machine's message.
+
+Four outcomes rather than content-or-error, because a binary file drawn as
+text and a big file cut off silently are both wrong in ways the reader
+cannot see, and "couldn't read it" must not look like "it is empty". An
+empty file is `text` with empty content and is drawn as one empty line
+numbered 1, which is what it is.
+
+Not in the first cut: showing images (the phone has `isImageRef` and a
+viewer already; the route would serve bytes). Listed under "later".
+
+### 6. A write is conditional on what the reader saw
+
+`PUT /setups/{id}/file` carries the sha256 the read reported. The script
+compares it against the file as it is now and refuses with a distinct exit
+code if it differs; the server answers **409** with "changed on the machine
+since you opened it". Agents edit files while people read them; this is
+the common case, not the exotic one, and silently overwriting an agent's
+edit with a stale copy is the worst available outcome. The phone offers
+three ways out and says what each costs: **Overwrite** (theirs is lost),
+**Reload** (yours is lost), **Cancel** (keep editing, decide later).
+
+The write is `cat > "$1.ai-app-tmp" && chmod --reference="$1"
+"$1.ai-app-tmp" && mv -f -- "$1.ai-app-tmp" "$1"`, with the bytes on
+stdin. A temp file and a rename, so a connection dropped mid-write leaves
+the old file whole rather than a truncated one; `chmod --reference` keeps
+the mode, which a fresh file would otherwise lose (an executable script
+would stop being one). What this trades away: the inode changes, so a hard
+link elsewhere stops being the same file. Accepted; editors do the same.
+The check-then-write is not atomic against a writer landing between the
+two -- a window of microseconds on the same machine -- and that is accepted
+too, and noted at the script.
+
+The response carries the new size, mtime and sha256, so the editor's
+precondition is fresh without a second read.
+
+### 7. Create refuses to overwrite
+
+`POST /setups/{id}/file {path}` runs under `set -C` (noclobber) and
+`: > "$1"`, so a name that exists fails with the shell's own message rather
+than truncating somebody's file. `POST /setups/{id}/dir {path}` is `mkdir
+--` with the same property. The modal names one thing in the current
+directory and has a switch for "directory"; a created file opens straight
+into edit mode, because an empty file is not something to look at.
+
+Rejected: create-with-content in one request. The editor is the place
+content is typed, and a modal with a text area is a second editor.
+
+### 8. The viewer is a list of lines, coloured once
+
+The file is scanned once, off the main thread, by `scan` in
+`Highlighter.kt` with `rulesOf(language)`; the spans are bucketed per line
+in one pass, and each line's `AnnotatedString` is built when that line is
+composed. A `LazyColumn` of lines, not one `Text`: text layout is linear
+in the text, and a 20,000-line file in one `Text` measures all of it to
+draw a screenful. Lines are drawn with `softWrap = false` inside one
+shared `horizontalScroll` state, so the whole file scrolls sideways as a
+block and a line never wraps.
+
+Line numbers are a gutter in each row, right-aligned, with the gutter
+width taken from the digit count of the line count in the same monospace
+style -- so a 9-line file and a 12,000-line file each get exactly the
+width they need and nothing is measured by hand. Because nothing wraps, a
+logical line is one visual line, and the gutter cannot drift from the text
+it numbers. Gutter numbers take `onSurfaceVariant`; the text takes the
+scanner's palette on `rawSurface`, the surface every verbatim thing in the
+app already sits on.
+
+The language comes from the file's extension through the same table
+`fenceLanguage` reads (`FENCE_LANGUAGES` already keys on `kt`, `rs`,
+`py`, …). One function, `fileLanguage(name)`, takes the part after the
+last dot and asks that table; it is one table, not two, so a language
+added for fences is added for files. A file with no entry is drawn plain,
+for the reason the table's comment gives.
+
+Selection: the lines sit inside one `SelectionContainer`, as the
+transcript does, so a selection can run across lines.
+
+### 9. The editor is the legacy text field with a highlighting transformation
+
+Edit mode swaps the viewer for a `BasicTextField(TextFieldValue)` in the
+same monospace style, inside the same horizontal scroll so it does not
+wrap, with a `VisualTransformation` that returns the text unchanged and
+the scanner's spans as styles (`OffsetMapping.Identity`, since no
+character moves). This is the one Compose API that colours a field's text
+without replacing the field; the newer `TextFieldState` API has no hook
+for styles. The gutter is one `Text` of `1\n2\n…` in the same style beside
+the field, aligned for the same reason as the viewer: no wrap, one line
+each.
+
+Save is a glyph in the header, **disabled** until the text differs from
+what was loaded (never hidden -- a control that comes and goes makes its
+own absence the signal), and a `GlyphSpinner` while the write is out.
+Back with unsaved changes asks; the question says the edits will be lost.
+The keyboard: the explorer draws over the session, which deliberately has
+no `imePadding` (see `SessionScreen`'s layout note), so the explorer's own
+box adds it.
+
+Re-scanning on every keystroke is the cost to watch. For a file under
+`FILE_LIMIT` it is expected to be a few milliseconds (the scanner replaced
+a library that took 174ms on 200 lines; ours has not been measured on a
+1 MiB file). Measure before deciding whether edit mode needs a size below
+which highlighting is on -- see "Numbers to measure".
+
+### 10. The explorer draws over the session, and back closes it first
+
+`Screen.Session` in `AppRoot` gains a `files: FilesTarget?`. When set, the
+`FilesScreen` is composed **on top of** the session in the same `Box`, and
+the session stays composed under it: its event stream keeps flowing, its
+scroll position and draft stay where they were, and returning from a file
+costs nothing. Back -- the button, the platform gesture and `swipeBack` --
+clears `files` when it is set and goes to the list otherwise. Inside the
+explorer the same back steps one level: editor → viewer (with the unsaved
+question), viewer → listing, listing → parent directory it came from, and
+only from the starting directory does it close. "Back returns; it does not
+exit."
+
+Rejected: a `Screen.Files` beside `Screen.Session`. Every route back from
+a leaf screen goes to Main today, and a session disposed and re-created on
+each return refetches its transcript over the tunnel -- exactly the flip
+between "what did it change" and "what is it saying" this feature is for.
+The image viewer already made the same choice for the same reason.
+
+### 11. The listing is drawn as it came, sorted at display time
+
+Entries carry name, kind (`directory`, `file`, `other`), size, mtime, and
+whether the entry is a symlink (with the kind being the *target's*, from
+`find -printf '%Y'`, so a link to a directory navigates). Sorted on the
+phone, stably: directories first, then case-insensitive name. Dotfiles are
+shown -- in a repository they are half of what matters. A row is the
+glyph, the name, and the size for a file; tapping a directory descends,
+tapping a file opens it. Each directory's entries are kept for as long as
+the explorer is open, keyed by path, so returning to one does not refetch
+it; the header's refresh glyph refetches the current one on purpose, and a
+create refetches the directory it created into, since that is what the
+operation changed.
+
+An empty directory says "Nothing here". A listing that failed says why,
+in the machine's words, where the rows would be -- never an empty list.
+
+Entries are separated by `\0` in the script's output and by `\t` within a
+line (`find -printf '%y\t%Y\t%s\t%T@\t%f\0'`), so a filename with a
+newline or a tab in it survives; `parse_entries` is a unit test with
+exactly those names in it.
+
+### 12. Icons
+
+Added to `NerdIcons.kt` **and** `build-icon-font.sh`, then the script
+rerun and its output committed (it needs network):
+
+- `md-folder` U+F024B -- the header button, and directory rows. The same
+  codepoint dev-updater uses, and it must not drift from it, as the cog
+  and the refresh arrow already must not.
+- `md-plus` U+F0415 -- create. Also dev-updater's.
+- `md-pencil` -- edit.
+- `md-content_save` -- save.
+- `md-file_outline` -- file rows.
+
+The last three are verified against the Nerd Fonts cheat sheet when they
+are added, not copied from memory.
+
+### 13. The render report moves, and the benches move with it
+
+The speedometer goes. The report it copies is the standard measurement
+`transcript-bench.sh` and `stream-bench.sh` read from logcat, so it stays
+reachable: a "Copy render timings" row in `SessionSettingsDialog`, which
+is where the session's other about-the-session controls already are.
+
+**No script that drives the UI taps by coordinate, and moving this
+button is where that rule gets enforced** (Bryan, 2026-09-03). Both bench
+scripts press the button today as `ui-trace record --do 'tap 723 205'`, a
+position measured once by hand. Anything that moves the header -- this
+change, a font size, a density, another emulator -- makes that tap land on
+whatever now sits there, and the script then reports a number that was
+never measured, which reads exactly like a result. A control is found by
+the name it already carries for assistive technology (`GlyphButton`'s
+`label`, a row's text) and pressed at the bounds the screen reports at
+that moment.
+
+That belongs in the tool, not in each script: `ui-trace` in
+`~/repos/emulator-tools` gains a tap-by-label action (`tap 'Session
+settings'`, resolving the element's box from the same uiautomator tree
+`elements` already reads, at the moment of the gesture), and both benches
+move onto it in the same commit as the button -- cog, then "Copy render
+timings" -- so the measurement is never unavailable and never wrong
+quietly. `grep -n "tap [0-9]" app/*.sh` is the check that no coordinate
+tap is left, and it goes in the emulator-tools README beside the action.
+Once the action exists, this rule applies to every script that presses
+something on an Android screen, not only these two.
+
+## HTTP surface
+
+Added to the table in `routes.rs`'s module doc:
+
+```text
+GET  /setups/{id}/dir?path=P     entries of directory P, and P resolved
+GET  /setups/{id}/file?path=P    content of file P, or why not
+PUT  /setups/{id}/file           {path, content, ifSha256} -> new size/mtime/sha256
+                                 (409 when the file no longer matches ifSha256)
+POST /setups/{id}/file           {path} create empty; refused if it exists
+POST /setups/{id}/dir            {path} create; refused if it exists
+```
+
+Bodies use `deny_unknown_fields` like every other body here. Paths in the
+query string are URL-encoded by `Api.kt`'s existing helper.
+
+```json
+GET dir  -> {"path":"/home/bob/repos/ai-app",
+             "entries":[{"name":"app","kind":"directory","size":4096,"modified":1756900000,"link":false},
+                        {"name":"README.md","kind":"file","size":1234,"modified":1756900000,"link":false}]}
+GET file -> {"path":"/…/x.rs","kind":"text","size":1234,"modified":…,"sha256":"…","content":"…"}
+          | {"path":"/…/a.png","kind":"binary","size":45678,"modified":…}
+          | {"path":"/…/big.log","kind":"tooBig","size":12345678,"modified":…}
+PUT file -> {"size":1240,"modified":…,"sha256":"…"}
+```
+
+Errors: `BadRequest` with the machine's message for a path that is not
+there, not allowed or not absolute; the existing 409 variant for the
+precondition; `Internal` only for the server's own faults. The message is
+what the phone shows, in place, so it is written to be read there.
+
+## Server work (`server/src/files.rs`)
+
+One module, with the same shape as `setups.rs`: the scripts as constants,
+one `pub async fn` per operation taking `&Transport`, and the parsing as
+pure functions with tests.
+
+1. `Transport::capture_with_input(launch, stdin)` -- `capture` with bytes
+   on stdin. `ship_attachment` in `routes.rs` builds this by hand today
+   (an `ssh::command`, a `File` on stdin, `output().await`); it moves onto
+   the new helper in the same change, so there is one description of
+   "run this there with this on stdin" rather than two.
+2. `list(transport, path) -> Listing`: `cd -- "$1" && pwd -P && find .
+   -mindepth 1 -maxdepth 1 -printf '%y\t%Y\t%s\t%T@\t%f\0'`. First line is
+   the resolved path; the rest is entries. `parse_entries` tested with
+   names containing a tab, a newline, a leading dash and a `'`.
+3. `read(transport, path) -> Read`: `stat -c '%s %Y' -- "$1"`, refuse
+   above `FILE_LIMIT` before `cat` so a 2 GB log never crosses the
+   tunnel, then `sha256sum -- "$1"` and `cat -- "$1"`, header lines then
+   bytes; the server splits at the header and decides `text`/`binary` by
+   `String::from_utf8`.
+4. `write(transport, path, expected_sha256, bytes) -> Written`: the
+   script in decision 6, with a distinct exit code for the precondition
+   (`exit 3`) that the route maps to 409; anything else is the machine's
+   stderr.
+5. `create_file`, `create_dir`: decision 7.
+6. Routes in `routes.rs`, each resolving the setup with `setup_by_id` and
+   `Transport::for_setup` as `set_cwd` does. The path check (absolute or
+   `~`) is one function shared with `set_cwd`, which has it inline today.
+7. Tests: the parsers; the quoting (a path that tries to close the quote
+   ends up as one absurd argument -- `ssh.rs` has the pattern); and an
+   integration test running each script through `Transport::Here`
+   against a `tempfile` tree, which is cheap because `sh` is there
+   wherever `cargo test` runs. The precondition test writes the file
+   between the read and the write and asserts the 409 path.
+8. PLAN.md: the Security paragraph from decision 3, and an "Explorer"
+   section pointing here. AGENTS.md: the layout bullet for `files.rs`.
+
+## App work
+
+1. `Api.kt`: `fetchDir`, `fetchFile`, `writeFile`, `createFile`,
+   `createDir`, and the three data classes (`DirEntry`, `FileContent`
+   as a sealed class with the four kinds, `Written`).
+2. `NerdIcons.kt` + `build-icon-font.sh`: decision 12.
+3. `Languages.kt` (or `CodeFence.kt`, wherever `FENCE_LANGUAGES` sits):
+   `fileLanguage(name)`.
+4. `FileLines.kt`: the pure half of the viewer -- spans bucketed per line,
+   `lineOf(index) -> AnnotatedString` -- so it has a JVM unit test beside
+   `HighlighterTest`, the app's one existing test suite, covering a block
+   comment that spans lines and a file with no trailing newline.
+5. `FilesScreen.kt`: the listing, the navigation stack, the per-directory
+   cache, the create dialog (modelled on `AddSetupDialog`: fields, a busy
+   state, the failure shown inside the dialog beside the button that
+   caused it), and the header. `LoadState` for the listing.
+6. `FileViewer.kt`: decision 8. `FileEditor.kt`: decision 9, including
+   the conflict dialog.
+7. `AppRoot.kt`: decision 10. `SessionScreen.kt`: the folder glyph where
+   the speedometer was, `onFiles(setup, cwd)` out to the root.
+8. `SessionSettingsDialog.kt`: the render-report row. In
+   `~/repos/emulator-tools`, `ui-trace`'s tap-by-label action; then the
+   two bench scripts onto it, with no coordinate tap left in `app/*.sh`.
+
+## Testing
+
+- **Server**: `./run-tests.sh`, `cargo clippy --all-targets`, `cargo fmt`.
+- **Local transport, by hand**: `./ui-sandbox.sh api
+  "/setups/<id>/dir?path=~"` against the sandbox, whose `$HOME` is a
+  throwaway tree it is fine to write into. The sandbox gets a small
+  fixture directory with the states worth seeing: an empty directory, a
+  file with a tab in its name, a binary file, one over `FILE_LIMIT`, an
+  unreadable one (`chmod 000`), a symlink to a directory, and a source
+  file in each of a few languages.
+- **Remote transport**: the ssh-to-this-VM recipe in AGENTS.md ("How to
+  test SSH here"). The point of the exercise is the quoting and the
+  stdin path: write a file whose name has a `'` in it, and read it back.
+- **Phone**: `ui-trace`, not screenshots, for the things this feature is
+  made of -- that the gutter's number and its line share a baseline at
+  the first and the last row, that a long line's row is wider than the
+  viewport and does not grow the row height, that the editor's gutter
+  stays put while the text scrolls sideways. Screenshots for colour and
+  contrast on `rawSurface`.
+- **States to produce on purpose**, since the default state is the one
+  everybody looks at: a directory that fails to list (permission),
+  an unreachable machine (a setup pointing at a dead address), `binary`,
+  `tooBig`, the 409 conflict (edit the file with `sed -i` on the machine
+  between opening and saving), creating a name that exists, back with
+  unsaved edits, and the keyboard up over the editor.
+
+## Numbers to measure, before deciding
+
+- Scan time for a 1 MiB source file on the emulator, and on the phone
+  through the render report. That decides whether `FILE_LIMIT` is right
+  and whether edit mode highlights every keystroke or only below a size.
+- Time to first line for a 1 MiB file over the tunnel: the read, the
+  transfer, the scan, the first composition. If the transfer dominates,
+  the route gains nothing from streaming; if the scan does, it moves to
+  a worker with the plain text drawn first.
+- The `BasicTextField` at 20,000 lines: whether typing stays responsive.
+  If not, edit mode gets a lower cap than the viewer, stated in the
+  editor rather than discovered by a stuck keyboard.
+
+## Order of work
+
+Each step leaves the app working and is one commit.
+
+1. Server: `files.rs` with `list` and `read`, routes, tests. Half a day.
+2. App: icons, `Api.kt`, `FilesScreen` listing, `FileViewer`, the root
+   and session wiring, the render-report move with the benches. A day.
+3. Server: `write`, `create_file`, `create_dir`, the stdin helper and
+   `ship_attachment` onto it. Half a day.
+4. App: `FileEditor`, the create dialog, the conflict dialog. Half a day.
+5. Measurements above, the sandbox fixture, PLAN.md and AGENTS.md. Half a
+   day.
+
+## Later, deliberately not now
+
+- Delete, rename and move. Destructive controls belong here eventually,
+  shown and confirmed rather than hidden, but none of them is needed to
+  read or change a file.
+- Images in the viewer, through the existing `SessionImageViewer`.
+- Following an agent's edits live: a file open in the viewer refreshing
+  when a `Write`/`Edit` tool call on the same path lands in the
+  transcript. The transcript already knows the path.
+- Remembering the last directory per session.
+- Uploading from the phone into a directory. Attachments already do the
+  upload half; this would be the same route with a chosen destination.
+- Search within a file, and find-in-files.