From 32c57a47a5c9d3a4adce9f86e35541fcde78d4f1 Mon Sep 17 00:00:00 2001 From: iris <2+iris@noreply.localhost> Date: Thu, 3 Sep 2026 21:07:07 -0400 Subject: [PATCH] Plan the file explorer Co-Authored-By: Claude Fable 5.1 --- EXPLORER.md | 453 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 453 insertions(+) create mode 100644 EXPLORER.md diff --git a/EXPLORER.md b/EXPLORER.md new file mode 100644 index 0000000..7deb312 --- /dev/null +++ b/EXPLORER.md @@ -0,0 +1,453 @@ +# The file explorer + +Asked for by Bryan on 2026-09-03: replace the session screen's debug +button with a folder icon that opens a file and directory viewer for the +machine the session runs on. Browse directories, open files with the +existing syntax highlighting, line numbers, no wrapping; edit a file behind +a pencil icon; create files through a modal like the ones the app already +has; work over ssh; open at the session's working directory. + +This is the plan. Like PLAN.md it records each decision with the reason and +what was rejected, so that when one changes it is changed here rather than +re-argued. Once built, the operational notes (how to test it, what bit) +move to AGENTS.md and this file keeps only the design. + +## What it is, in one paragraph + +A machine's filesystem, seen from the phone through the backend. The +explorer belongs to a **setup** (a machine), not to a session: a session +only says where to start. Every operation -- list, read, write, create -- +is one shell script run through `Transport`, exactly the way the import +listing and the usage fetch already work, so the local and the ssh case +are one implementation and a machine the backend cannot reach fails with +ssh's own message. The phone draws what came back: a listing, a file with +its lines coloured by the scanner in `Highlighter.kt`, or an editor over +the same text. + +## Decisions + +### 1. Keyed on the machine, opened from the session + +Routes live under `/setups/{id}/…`, beside `importable`, because a +filesystem is a property of a machine. The session screen's folder button +opens the explorer with the session's setup and its `cwd` as the starting +directory; a session with no `cwd` opens at the machine's home, which the +machine resolves (`cd` with no argument and `pwd -P`), never a path the +phone guessed. Nothing in the explorer knows what a session is, so a later +entry point from the setups tab is one more caller and no new code. + +Rejected: routes under `/sessions/{id}/`. The session would be a detour to +find the setup, and "browse this machine" from anywhere but a session would +need a session to exist first. + +### 2. One shell script per operation, over `Transport`, on both transports + +Each operation is a small POSIX shell script handed to `sh -c script sh +"$path" …` through `Transport::capture` (or the stdin-carrying variant +below). The path and every other value cross as **positional arguments**, +never interpolated into the script -- the same rule `import::find` follows +with `"$1"`, and the same reason `ssh::quote` exists: a path is +attacker-adjacent input in a server whose job is running commands. A `~` +prefix is handled by the same `quote_path`/`expand_home` pair every other +path goes through; nothing new is invented for it. + +The scripts assume GNU coreutils and findutils (`find -printf`, `stat -c`, +`sha256sum`, `chmod --reference`). That is already what `import.rs` +assumes (`stat -c`, `/proc`), and both machines that exist are Linux. A +machine without them fails with that tool's own message, which names what +is missing. + +Rejected: `std::fs` for the local transport and scripts for ssh. Two +implementations of "list a directory" drift -- the ordering of entries, +what a symlink reports, how a permission error reads -- and the local one +is the one that gets tested, so the remote one ships broken. The transport +design exists so that a driver never learns which machine it got; the +explorer is held to the same rule. The cost is a `sh` process per +operation locally, which is under a millisecond. + +Rejected: a Rust SSH or SFTP library. PLAN.md rule 23 -- the system `ssh` +inherits `~/.ssh/config`, agents and jump hosts, and there is one place to +configure a connection. SFTP would need a second one. + +### 3. The token can now name a path, and that is written down + +AGENTS.md says of the import route: "the phone picks an **id**, never a +path: the server resolves which file that is, so an enrolled token cannot +become 'read me an arbitrary file'." The explorer's whole purpose is the +path, so it takes one. This is recorded in PLAN.md's Security section as a +change to the threat model paragraph, in these terms: the token already +gates spawning a bypass-permissions agent in any directory on any machine +a setup names, and that agent can already read and write every file its +user can. The explorer is a shorter path to authority the token already +holds, not new authority. The import route's rule stands where it is, +because there a path was unnecessary and refusing it cost nothing. + +What is *not* changed: no route accepts a command. Listing, reading and +writing are fixed scripts; the phone chooses only the path and the bytes. + +### 4. Paths are absolute or `~`-prefixed, and the machine answers with the real one + +Same rule as `POST /sessions/{id}/cwd`: a relative path is refused with +the same wording, because where it would be depends on where nothing the +reader can see. Every listing answers with `pwd -P` of the directory it +listed, so the phone navigates on a resolved absolute path -- the parent +of `/home/bob/repos/ai-app` is a string operation on that, and a `~` the +session was spawned with is shown as what it turned out to be. The phone +never resolves `..` itself. + +### 5. A read is capped and typed, and every state it can be in has a word + +`GET /setups/{id}/file` answers with one of: + +- `text` -- the content, with its size, mtime and sha256. +- `binary` -- the content is not UTF-8. Size reported, nothing shown. +- `tooBig` -- over `FILE_LIMIT` (1 MiB to start; see "Numbers to + measure"). Size reported so the reader knows what they are looking at. +- an error -- no such file, permission denied, machine unreachable -- + carrying the machine's message. + +Four outcomes rather than content-or-error, because a binary file drawn as +text and a big file cut off silently are both wrong in ways the reader +cannot see, and "couldn't read it" must not look like "it is empty". An +empty file is `text` with empty content and is drawn as one empty line +numbered 1, which is what it is. + +Not in the first cut: showing images (the phone has `isImageRef` and a +viewer already; the route would serve bytes). Listed under "later". + +### 6. A write is conditional on what the reader saw + +`PUT /setups/{id}/file` carries the sha256 the read reported. The script +compares it against the file as it is now and refuses with a distinct exit +code if it differs; the server answers **409** with "changed on the machine +since you opened it". Agents edit files while people read them; this is +the common case, not the exotic one, and silently overwriting an agent's +edit with a stale copy is the worst available outcome. The phone offers +three ways out and says what each costs: **Overwrite** (theirs is lost), +**Reload** (yours is lost), **Cancel** (keep editing, decide later). + +The write is `cat > "$1.ai-app-tmp" && chmod --reference="$1" +"$1.ai-app-tmp" && mv -f -- "$1.ai-app-tmp" "$1"`, with the bytes on +stdin. A temp file and a rename, so a connection dropped mid-write leaves +the old file whole rather than a truncated one; `chmod --reference` keeps +the mode, which a fresh file would otherwise lose (an executable script +would stop being one). What this trades away: the inode changes, so a hard +link elsewhere stops being the same file. Accepted; editors do the same. +The check-then-write is not atomic against a writer landing between the +two -- a window of microseconds on the same machine -- and that is accepted +too, and noted at the script. + +The response carries the new size, mtime and sha256, so the editor's +precondition is fresh without a second read. + +### 7. Create refuses to overwrite + +`POST /setups/{id}/file {path}` runs under `set -C` (noclobber) and +`: > "$1"`, so a name that exists fails with the shell's own message rather +than truncating somebody's file. `POST /setups/{id}/dir {path}` is `mkdir +--` with the same property. The modal names one thing in the current +directory and has a switch for "directory"; a created file opens straight +into edit mode, because an empty file is not something to look at. + +Rejected: create-with-content in one request. The editor is the place +content is typed, and a modal with a text area is a second editor. + +### 8. The viewer is a list of lines, coloured once + +The file is scanned once, off the main thread, by `scan` in +`Highlighter.kt` with `rulesOf(language)`; the spans are bucketed per line +in one pass, and each line's `AnnotatedString` is built when that line is +composed. A `LazyColumn` of lines, not one `Text`: text layout is linear +in the text, and a 20,000-line file in one `Text` measures all of it to +draw a screenful. Lines are drawn with `softWrap = false` inside one +shared `horizontalScroll` state, so the whole file scrolls sideways as a +block and a line never wraps. + +Line numbers are a gutter in each row, right-aligned, with the gutter +width taken from the digit count of the line count in the same monospace +style -- so a 9-line file and a 12,000-line file each get exactly the +width they need and nothing is measured by hand. Because nothing wraps, a +logical line is one visual line, and the gutter cannot drift from the text +it numbers. Gutter numbers take `onSurfaceVariant`; the text takes the +scanner's palette on `rawSurface`, the surface every verbatim thing in the +app already sits on. + +The language comes from the file's extension through the same table +`fenceLanguage` reads (`FENCE_LANGUAGES` already keys on `kt`, `rs`, +`py`, …). One function, `fileLanguage(name)`, takes the part after the +last dot and asks that table; it is one table, not two, so a language +added for fences is added for files. A file with no entry is drawn plain, +for the reason the table's comment gives. + +Selection: the lines sit inside one `SelectionContainer`, as the +transcript does, so a selection can run across lines. + +### 9. The editor is the legacy text field with a highlighting transformation + +Edit mode swaps the viewer for a `BasicTextField(TextFieldValue)` in the +same monospace style, inside the same horizontal scroll so it does not +wrap, with a `VisualTransformation` that returns the text unchanged and +the scanner's spans as styles (`OffsetMapping.Identity`, since no +character moves). This is the one Compose API that colours a field's text +without replacing the field; the newer `TextFieldState` API has no hook +for styles. The gutter is one `Text` of `1\n2\n…` in the same style beside +the field, aligned for the same reason as the viewer: no wrap, one line +each. + +Save is a glyph in the header, **disabled** until the text differs from +what was loaded (never hidden -- a control that comes and goes makes its +own absence the signal), and a `GlyphSpinner` while the write is out. +Back with unsaved changes asks; the question says the edits will be lost. +The keyboard: the explorer draws over the session, which deliberately has +no `imePadding` (see `SessionScreen`'s layout note), so the explorer's own +box adds it. + +Re-scanning on every keystroke is the cost to watch. For a file under +`FILE_LIMIT` it is expected to be a few milliseconds (the scanner replaced +a library that took 174ms on 200 lines; ours has not been measured on a +1 MiB file). Measure before deciding whether edit mode needs a size below +which highlighting is on -- see "Numbers to measure". + +### 10. The explorer draws over the session, and back closes it first + +`Screen.Session` in `AppRoot` gains a `files: FilesTarget?`. When set, the +`FilesScreen` is composed **on top of** the session in the same `Box`, and +the session stays composed under it: its event stream keeps flowing, its +scroll position and draft stay where they were, and returning from a file +costs nothing. Back -- the button, the platform gesture and `swipeBack` -- +clears `files` when it is set and goes to the list otherwise. Inside the +explorer the same back steps one level: editor → viewer (with the unsaved +question), viewer → listing, listing → parent directory it came from, and +only from the starting directory does it close. "Back returns; it does not +exit." + +Rejected: a `Screen.Files` beside `Screen.Session`. Every route back from +a leaf screen goes to Main today, and a session disposed and re-created on +each return refetches its transcript over the tunnel -- exactly the flip +between "what did it change" and "what is it saying" this feature is for. +The image viewer already made the same choice for the same reason. + +### 11. The listing is drawn as it came, sorted at display time + +Entries carry name, kind (`directory`, `file`, `other`), size, mtime, and +whether the entry is a symlink (with the kind being the *target's*, from +`find -printf '%Y'`, so a link to a directory navigates). Sorted on the +phone, stably: directories first, then case-insensitive name. Dotfiles are +shown -- in a repository they are half of what matters. A row is the +glyph, the name, and the size for a file; tapping a directory descends, +tapping a file opens it. Each directory's entries are kept for as long as +the explorer is open, keyed by path, so returning to one does not refetch +it; the header's refresh glyph refetches the current one on purpose, and a +create refetches the directory it created into, since that is what the +operation changed. + +An empty directory says "Nothing here". A listing that failed says why, +in the machine's words, where the rows would be -- never an empty list. + +Entries are separated by `\0` in the script's output and by `\t` within a +line (`find -printf '%y\t%Y\t%s\t%T@\t%f\0'`), so a filename with a +newline or a tab in it survives; `parse_entries` is a unit test with +exactly those names in it. + +### 12. Icons + +Added to `NerdIcons.kt` **and** `build-icon-font.sh`, then the script +rerun and its output committed (it needs network): + +- `md-folder` U+F024B -- the header button, and directory rows. The same + codepoint dev-updater uses, and it must not drift from it, as the cog + and the refresh arrow already must not. +- `md-plus` U+F0415 -- create. Also dev-updater's. +- `md-pencil` -- edit. +- `md-content_save` -- save. +- `md-file_outline` -- file rows. + +The last three are verified against the Nerd Fonts cheat sheet when they +are added, not copied from memory. + +### 13. The render report moves, and the benches move with it + +The speedometer goes. The report it copies is the standard measurement +`transcript-bench.sh` and `stream-bench.sh` read from logcat, so it stays +reachable: a "Copy render timings" row in `SessionSettingsDialog`, which +is where the session's other about-the-session controls already are. + +**No script that drives the UI taps by coordinate, and moving this +button is where that rule gets enforced** (Bryan, 2026-09-03). Both bench +scripts press the button today as `ui-trace record --do 'tap 723 205'`, a +position measured once by hand. Anything that moves the header -- this +change, a font size, a density, another emulator -- makes that tap land on +whatever now sits there, and the script then reports a number that was +never measured, which reads exactly like a result. A control is found by +the name it already carries for assistive technology (`GlyphButton`'s +`label`, a row's text) and pressed at the bounds the screen reports at +that moment. + +That belongs in the tool, not in each script: `ui-trace` in +`~/repos/emulator-tools` gains a tap-by-label action (`tap 'Session +settings'`, resolving the element's box from the same uiautomator tree +`elements` already reads, at the moment of the gesture), and both benches +move onto it in the same commit as the button -- cog, then "Copy render +timings" -- so the measurement is never unavailable and never wrong +quietly. `grep -n "tap [0-9]" app/*.sh` is the check that no coordinate +tap is left, and it goes in the emulator-tools README beside the action. +Once the action exists, this rule applies to every script that presses +something on an Android screen, not only these two. + +## HTTP surface + +Added to the table in `routes.rs`'s module doc: + +```text +GET /setups/{id}/dir?path=P entries of directory P, and P resolved +GET /setups/{id}/file?path=P content of file P, or why not +PUT /setups/{id}/file {path, content, ifSha256} -> new size/mtime/sha256 + (409 when the file no longer matches ifSha256) +POST /setups/{id}/file {path} create empty; refused if it exists +POST /setups/{id}/dir {path} create; refused if it exists +``` + +Bodies use `deny_unknown_fields` like every other body here. Paths in the +query string are URL-encoded by `Api.kt`'s existing helper. + +```json +GET dir -> {"path":"/home/bob/repos/ai-app", + "entries":[{"name":"app","kind":"directory","size":4096,"modified":1756900000,"link":false}, + {"name":"README.md","kind":"file","size":1234,"modified":1756900000,"link":false}]} +GET file -> {"path":"/…/x.rs","kind":"text","size":1234,"modified":…,"sha256":"…","content":"…"} + | {"path":"/…/a.png","kind":"binary","size":45678,"modified":…} + | {"path":"/…/big.log","kind":"tooBig","size":12345678,"modified":…} +PUT file -> {"size":1240,"modified":…,"sha256":"…"} +``` + +Errors: `BadRequest` with the machine's message for a path that is not +there, not allowed or not absolute; the existing 409 variant for the +precondition; `Internal` only for the server's own faults. The message is +what the phone shows, in place, so it is written to be read there. + +## Server work (`server/src/files.rs`) + +One module, with the same shape as `setups.rs`: the scripts as constants, +one `pub async fn` per operation taking `&Transport`, and the parsing as +pure functions with tests. + +1. `Transport::capture_with_input(launch, stdin)` -- `capture` with bytes + on stdin. `ship_attachment` in `routes.rs` builds this by hand today + (an `ssh::command`, a `File` on stdin, `output().await`); it moves onto + the new helper in the same change, so there is one description of + "run this there with this on stdin" rather than two. +2. `list(transport, path) -> Listing`: `cd -- "$1" && pwd -P && find . + -mindepth 1 -maxdepth 1 -printf '%y\t%Y\t%s\t%T@\t%f\0'`. First line is + the resolved path; the rest is entries. `parse_entries` tested with + names containing a tab, a newline, a leading dash and a `'`. +3. `read(transport, path) -> Read`: `stat -c '%s %Y' -- "$1"`, refuse + above `FILE_LIMIT` before `cat` so a 2 GB log never crosses the + tunnel, then `sha256sum -- "$1"` and `cat -- "$1"`, header lines then + bytes; the server splits at the header and decides `text`/`binary` by + `String::from_utf8`. +4. `write(transport, path, expected_sha256, bytes) -> Written`: the + script in decision 6, with a distinct exit code for the precondition + (`exit 3`) that the route maps to 409; anything else is the machine's + stderr. +5. `create_file`, `create_dir`: decision 7. +6. Routes in `routes.rs`, each resolving the setup with `setup_by_id` and + `Transport::for_setup` as `set_cwd` does. The path check (absolute or + `~`) is one function shared with `set_cwd`, which has it inline today. +7. Tests: the parsers; the quoting (a path that tries to close the quote + ends up as one absurd argument -- `ssh.rs` has the pattern); and an + integration test running each script through `Transport::Here` + against a `tempfile` tree, which is cheap because `sh` is there + wherever `cargo test` runs. The precondition test writes the file + between the read and the write and asserts the 409 path. +8. PLAN.md: the Security paragraph from decision 3, and an "Explorer" + section pointing here. AGENTS.md: the layout bullet for `files.rs`. + +## App work + +1. `Api.kt`: `fetchDir`, `fetchFile`, `writeFile`, `createFile`, + `createDir`, and the three data classes (`DirEntry`, `FileContent` + as a sealed class with the four kinds, `Written`). +2. `NerdIcons.kt` + `build-icon-font.sh`: decision 12. +3. `Languages.kt` (or `CodeFence.kt`, wherever `FENCE_LANGUAGES` sits): + `fileLanguage(name)`. +4. `FileLines.kt`: the pure half of the viewer -- spans bucketed per line, + `lineOf(index) -> AnnotatedString` -- so it has a JVM unit test beside + `HighlighterTest`, the app's one existing test suite, covering a block + comment that spans lines and a file with no trailing newline. +5. `FilesScreen.kt`: the listing, the navigation stack, the per-directory + cache, the create dialog (modelled on `AddSetupDialog`: fields, a busy + state, the failure shown inside the dialog beside the button that + caused it), and the header. `LoadState` for the listing. +6. `FileViewer.kt`: decision 8. `FileEditor.kt`: decision 9, including + the conflict dialog. +7. `AppRoot.kt`: decision 10. `SessionScreen.kt`: the folder glyph where + the speedometer was, `onFiles(setup, cwd)` out to the root. +8. `SessionSettingsDialog.kt`: the render-report row. In + `~/repos/emulator-tools`, `ui-trace`'s tap-by-label action; then the + two bench scripts onto it, with no coordinate tap left in `app/*.sh`. + +## Testing + +- **Server**: `./run-tests.sh`, `cargo clippy --all-targets`, `cargo fmt`. +- **Local transport, by hand**: `./ui-sandbox.sh api + "/setups//dir?path=~"` against the sandbox, whose `$HOME` is a + throwaway tree it is fine to write into. The sandbox gets a small + fixture directory with the states worth seeing: an empty directory, a + file with a tab in its name, a binary file, one over `FILE_LIMIT`, an + unreadable one (`chmod 000`), a symlink to a directory, and a source + file in each of a few languages. +- **Remote transport**: the ssh-to-this-VM recipe in AGENTS.md ("How to + test SSH here"). The point of the exercise is the quoting and the + stdin path: write a file whose name has a `'` in it, and read it back. +- **Phone**: `ui-trace`, not screenshots, for the things this feature is + made of -- that the gutter's number and its line share a baseline at + the first and the last row, that a long line's row is wider than the + viewport and does not grow the row height, that the editor's gutter + stays put while the text scrolls sideways. Screenshots for colour and + contrast on `rawSurface`. +- **States to produce on purpose**, since the default state is the one + everybody looks at: a directory that fails to list (permission), + an unreachable machine (a setup pointing at a dead address), `binary`, + `tooBig`, the 409 conflict (edit the file with `sed -i` on the machine + between opening and saving), creating a name that exists, back with + unsaved edits, and the keyboard up over the editor. + +## Numbers to measure, before deciding + +- Scan time for a 1 MiB source file on the emulator, and on the phone + through the render report. That decides whether `FILE_LIMIT` is right + and whether edit mode highlights every keystroke or only below a size. +- Time to first line for a 1 MiB file over the tunnel: the read, the + transfer, the scan, the first composition. If the transfer dominates, + the route gains nothing from streaming; if the scan does, it moves to + a worker with the plain text drawn first. +- The `BasicTextField` at 20,000 lines: whether typing stays responsive. + If not, edit mode gets a lower cap than the viewer, stated in the + editor rather than discovered by a stuck keyboard. + +## Order of work + +Each step leaves the app working and is one commit. + +1. Server: `files.rs` with `list` and `read`, routes, tests. Half a day. +2. App: icons, `Api.kt`, `FilesScreen` listing, `FileViewer`, the root + and session wiring, the render-report move with the benches. A day. +3. Server: `write`, `create_file`, `create_dir`, the stdin helper and + `ship_attachment` onto it. Half a day. +4. App: `FileEditor`, the create dialog, the conflict dialog. Half a day. +5. Measurements above, the sandbox fixture, PLAN.md and AGENTS.md. Half a + day. + +## Later, deliberately not now + +- Delete, rename and move. Destructive controls belong here eventually, + shown and confirmed rather than hidden, but none of them is needed to + read or change a file. +- Images in the viewer, through the existing `SessionImageViewer`. +- Following an agent's edits live: a file open in the viewer refreshing + when a `Write`/`Edit` tool call on the same path lands in the + transcript. The transcript already knows the path. +- Remembering the last directory per session. +- Uploading from the phone into a directory. Attachments already do the + upload half; this would be the same route with a chosen destination. +- Search within a file, and find-in-files.