Plan and working notes

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017xn8nHw1tw1R6PtiY1eEtw

PLAN.md

@@ -0,0 +1,430 @@
+# ai-app — plan
+
+A phone interface to AI coding sessions — Claude Code and llama.cpp for now —
+built to replace the Claude app for day-to-day use. Two motivations: local
+models need a front end at all, and owning the client means fixing the things
+the official app gets wrong (e.g. it won't deliver a typed message until the
+session fully finishes its turn, where the TUI injects it at the next tool
+boundary).
+
+Same shape as `../local-updater`: a Rust (Axum) backend on the desktop, a
+Kotlin/Compose Android app, pinned self-signed TLS between them.
+
+## The one idea everything hangs off
+
+Both session types are **a child process speaking JSONL over stdio**:
+
+- Claude Code: `claude -p --input-format stream-json --output-format stream-json`
+  — bidirectional streaming JSON. User messages sent while a turn is running
+  are injected at the next opportunity (the TUI behavior we want), a control
+  protocol carries interrupts and permission requests, `--resume <id>` picks a
+  session back up after a backend restart.
+- llama.cpp: **pi in RPC mode** (`pi --mode rpc`), pointed at a llama-server
+  endpoint. Same deal: JSONL on stdio, `prompt` (with images), `steer` for
+  mid-run injection, `abort`, `set_model`, `compact` / `set_auto_compaction`,
+  session files that survive restarts, structured events for streaming text
+  and tool executions.
+
+So the backend has one abstraction — spawn a process, translate its dialect to
+a common event stream, keep an append-only transcript — and two translators.
+SSH support falls out of the same shape: a remote session is the identical
+command run as `ssh <host> <cmd>`; stdio doesn't care.
+
+Decisions already made (2026-08-24):
+
+- llama.cpp harness: **pi RPC now**, with the session abstraction kept clean
+  enough that a custom Rust agent loop can be added as a third driver later.
+- The backend **manages llama-server itself** (start with a chosen GGUF, stop,
+  swap models), locally and over SSH.
+- Claude permission prompts are **interactive in the app**, with a per-session
+  permission mode chosen at spawn.
+- **One backend** on the main machine; the phone talks only to it, and it
+  reaches other hosts via SSH. Remote hosts need the CLIs installed but no
+  backend.
+
+## Architecture
+
+```
+Android app (Compose)
+   │  HTTPS (pinned CA) — REST for actions, SSE for live events
+   ▼
+backend (Rust/Axum, desktop)
+   ├─ SessionManager ── Session ── Driver (trait)
+   │                      ├─ ClaudeDriver   (claude stream-json)
+   │                      └─ PiDriver       (pi --mode rpc)
+   │        each driver's process is spawned locally or as `ssh host …`
+   ├─ LlamaServerManager  (llama-server lifecycle, local + SSH)
+   ├─ UsageMonitor        (Anthropic OAuth usage endpoint)
+   └─ config.json + per-session transcript files
+```
+
+### Backend layout (`server/`)
+
+Mirroring local-updater's stack: axum 0.8, axum-server + rustls, tokio, serde,
+clap, tracing. Rust edition 2024, warning-clean, clippy in CI habit.
+
+- `main.rs` — bootstrap, TLS listener.
+- `routes.rs` — the whole HTTP table in one module doc comment (as in
+  local-updater).
+- `session/mod.rs` — `SessionManager`: the live session registry, every
+  mutation funnels through it (the `registry.rs` pattern: in-memory and
+  on-disk state can't come apart).
+- `session/driver.rs` — the `Driver` trait and the common event model.
+- `session/claude.rs`, `session/pi.rs` — the two translators.
+- `session/transcript.rs` — append-only JSONL event log per session, with
+  monotonically increasing sequence numbers (the phone's resume cursor).
+- `llama.rs` — `LlamaServerManager`.
+- `hosts.rs` — host configs and the ssh command builder.
+- `usage.rs` — Anthropic usage polling.
+- `config.rs` — persisted schema.
+
+### The common event model
+
+Driver output, whatever the dialect, is normalized into one event enum before
+it touches the transcript or the phone:
+
+- `AssistantText { delta }` — streaming text (rendered as markdown).
+- `ToolStart / ToolUpdate / ToolEnd { tool, input, output }` — the "view tools
+  it's running" screen is just these.
+- `Image { ref }` — images in output (screenshots from tools, etc.) are saved
+  under the session dir and referenced by id; the phone fetches them by URL.
+- `Question { id, prompt, options }` — anything the session needs a human for:
+  Claude's AskUserQuestion, and **permission requests** (canUseTool) are the
+  same shape with approve/deny options. Answered via one endpoint.
+- `Status { state }` — idle / running / awaiting-input / compacting / exited.
+- `UsageDelta { tokens }` — per-turn token counts where the dialect reports
+  them (both do).
+- `Error { message }`.
+
+Every event is appended to the session's transcript file with a sequence
+number, then fanned out to any connected SSE subscribers. The phone renders
+purely from this stream: reconnecting means "give me events after seq N" —
+no separate "load history" path to drift from the live one.
+
+Inbound, the driver trait is small:
+
+```rust
+trait Driver {
+    fn send_user_message(&self, text: String, images: Vec<ImageRef>);
+    fn answer_question(&self, id: QuestionId, answer: Answer);
+    fn interrupt(&self);                       // stop mid-run, session survives
+    fn set_model(&self, model: &str);
+    fn compact(&self);                         // pi: native; claude: /compact
+    fn shutdown(&self);                        // graceful process exit
+}
+```
+
+`send_user_message` during a run is the point of the whole app: both dialects
+queue it for injection at the next tool boundary rather than the end of the
+turn. Claude's dialect: a `user` message on stdin mid-stream; pi's: `steer`.
+
+### Claude driver specifics
+
+- Spawn: `claude -p --verbose --input-format stream-json --output-format
+  stream-json --permission-mode <mode>` in the chosen working directory, plus
+  `--model` at spawn. Permission mode (default/plan/acceptEdits/
+  bypassPermissions) is chosen on the spawn screen.
+- Interactive permissions: run with the stream-json control protocol's
+  permission request flow (the same mechanism the Agent SDK's `canUseTool`
+  uses) so tool approvals arrive as control requests, become `Question`
+  events, and our answer goes back as the control response. **Verify the
+  exact control-request wire format against the current CLI early in
+  implementation** — it's the least-documented part of this plan.
+- Interrupt: control-protocol interrupt request.
+- Model change mid-session: try the control protocol's set-model; if the
+  installed CLI doesn't support it, fall back to `shutdown` + respawn with
+  `--resume <session_id> --model <new>` — cheap, since Claude persists
+  sessions in `~/.claude/projects` anyway. That same resume path is the crash
+  recovery story: a dead backend or a killed process loses nothing.
+- Images in: base64 image content blocks in the stream-json user message.
+- Working directory, host, and model are spawn-screen fields.
+
+### pi driver specifics
+
+- Spawn: `pi --mode rpc --provider openai-generic --model <name>` (endpoint =
+  the llama-server the LlamaServerManager provides), `--session-dir` under
+  our session storage so transcripts and pi's own session files live together.
+- Auto-compaction on by default (`set_auto_compaction`), threshold
+  configurable per session; manual `compact` exposed as a button.
+- `steer` for mid-run messages, `abort` for stop, `set_model` when the target
+  endpoint changes.
+- pi's session JSONL gives resume-after-restart, same as Claude's.
+
+### llama-server management
+
+`config.json` lists **models** (name → GGUF path or llama-server args, per
+host) and **hosts**. The manager runs at most one llama-server per
+`(host, model)`, spawned on demand when a session needs it:
+
+- Spawn (local or `ssh host llama-server …`) on an allocated port, wait on
+  `/health`, hand the endpoint to the pi driver.
+- Refcounted by sessions. The path out, written in the same change as the
+  spawn: the last session using an instance releasing it starts an idle
+  timer (configurable, e.g. 10 min), after which it's killed. Delete of the
+  last session kills it immediately.
+- "Change model" on a llama session = acquire the new model's server,
+  `set_model` on pi, release the old one. Context carries over (it's
+  prompt-replayed by pi against the new endpoint).
+- Remote llama-server output is only reachable from the backend host, and
+  binds localhost on the remote side with an SSH local port forward
+  (`ssh -L`) held by the manager — no LAN-exposed inference ports.
+
+### SSH
+
+- Host entries in `config.json`: name, `user@host`, optional ssh options,
+  which capabilities it has (claude / pi / llama-server, with paths if not on
+  PATH). Key-based auth only, using the system `ssh` client via
+  `tokio::process` — no Rust SSH library; this inherits `~/.ssh/config`,
+  agents, and jump hosts for free. (Rule 23: openssh is already here and
+  battle-tested; a library buys nothing but a second config surface.)
+- A remote session is exactly a local one with the command wrapped in
+  `ssh -T host …`. Process death ≙ connection death; the session shows as
+  `exited` and both dialects resume (`--resume` / pi session file) on respawn,
+  so a dropped SSH connection is an annoyance, not data loss.
+- Images and attachments for remote sessions are written to the remote
+  session dir via `scp`/stdin before the message referencing them is sent.
+
+### Usage limits (Claude)
+
+Poll `https://api.anthropic.com/api/oauth/usage` — the same endpoint behind
+Claude Code's `/usage` — with the OAuth access token from Claude Code's local
+credential store (`~/.claude/.credentials.json`), headers
+`anthropic-beta: oauth-2025-04-20` and `User-Agent: claude-code/<version>`
+(without the User-Agent it lands in an aggressively rate-limited bucket).
+Poll at ≥180 s, only while any Claude session exists or the usage screen is
+open, cache the last answer. Surface: 5-hour and weekly window utilization %
+and reset times. It's undocumented, so `usage.rs` treats every field as
+optional and degrades to "unavailable" rather than erroring. Structure it as
+one `UsageProvider` per paid service so a second service later is a new impl,
+not a parallel screen (rule 9).
+
+### HTTP surface (phone ⇄ backend)
+
+REST for actions, one SSE stream per open session screen for events, all over
+the pinned TLS listener. SSE over WebSocket because resume-by-cursor
+(`Last-Event-ID` = transcript seq) is native to it and the inbound direction
+is plain POSTs anyway.
+
+```
+GET  /sessions                        list (id, kind, title, host, model, status, last activity)
+POST /sessions                        spawn {kind, host, model, cwd, permission_mode, title}
+GET  /sessions/:id/events?after=N     SSE: transcript replay from N, then live
+POST /sessions/:id/message            {text, attachment_ids}
+POST /sessions/:id/answer             {question_id, answer}   (questions and permissions)
+POST /sessions/:id/interrupt
+POST /sessions/:id/model              {model}
+POST /sessions/:id/compact            (llama sessions)
+DELETE /sessions/:id                  kill process, release llama-server, delete transcript+files
+POST /attachments                     multipart upload → id (referenced by /message)
+GET  /files/:session/:id              images/files the session produced
+GET  /usage                           cached usage windows
+GET/PUT /hosts, /models               config editing from the phone
+```
+
+Sessions live in `config.json` + a per-session directory (transcript.jsonl,
+attachments, produced images). Deleting a session is the complete path out of
+everything spawning one created.
+
+### Security
+
+- TLS with a self-signed CA, pinned in the app — `gen-dev-cert.sh` and
+  `PinnedCert.kt` copied from local-updater, same idempotent-CA/reissued-leaf
+  scheme, same one-way-door caveat about regenerating the CA.
+- This server is strictly more dangerous than the updater: its API *is*
+  remote code execution (spawn a bypass-permissions Claude on any SSH host).
+  Pinning authenticates the server to the phone but not the phone to the
+  server, so a bearer token adds the other direction. Threat model: the token
+  gates LAN-reachable RCE; it does not (and cannot) defend a compromised
+  backend host or phone — those are inside the trust boundary, and a
+  compromised phone is handled by rotation.
+  - **Generation**: 256 bits from the OS CSPRNG on first run, base64url. A
+    machine credential, never typed twice, so unguessable costs nothing; at
+    this entropy no key stretching is needed.
+  - **Enrollment**: printed once as a terminal QR code (`qrcode` crate,
+    ANSI), encoding `aiapp://enroll?host=…&port=…&token=…`. The CA stays
+    embedded in the APK (`PinnedCert.kt` pattern), so the QR carries no
+    trust material — photographing the terminal leaks only the token
+    (rotatable), never a way to weaken pinning.
+  - **Storage**: server keeps only the SHA-256 in `config.json` (plain hash
+    is enough for high-entropy random input; buys that a leaked config
+    doesn't leak the credential). No "show token again" — lost means rotate.
+    Phone side: Keystore-backed encrypted preferences.
+  - **Transport**: `Authorization: Bearer` header on every request including
+    the SSE GET. Never a query parameter (URLs leak into logs). The tracing
+    layer must not log the header — covered by a test so a logging change
+    can't silently start leaking it.
+  - **Verification**: one middleware wrapping the entire router in `main.rs`,
+    never per-route, so a new route can't forget auth. Zero unauthenticated
+    endpoints, `/health` included. Hash-then-constant-time-compare
+    (`subtle`); failures logged with peer address plus a small fixed delay —
+    not against brute force (infeasible at 256 bits) but so scanners show up
+    in the log.
+  - **Rotation (the path out)**: `--rotate-token` regenerates, invalidates
+    the old hash immediately, reprints the QR. That's the whole lost-phone
+    story. Config stores a *list* of `{name, hash}` (of one, today) so
+    per-device tokens with individual revocation are a config entry later,
+    not a schema migration.
+  - **Why not mTLS**: stronger in theory (key never leaves the Keystore, no
+    bearer secret to exfiltrate), but given pinning the delta is only
+    "someone reads the token off a device already inside the trust
+    boundary", and it costs Android client-cert provisioning ceremony and a
+    worse new-phone story than a QR scan. Revisit if this outgrows
+    single-user-on-LAN.
+- **Off-network access: plain WireGuard** (decided 2026-08-24; no third
+  party). The backend binds to the WireGuard interface (`wg0`) only; the
+  phone runs the official WireGuard app (always-on VPN, per-app tunneling),
+  enrolled by scanning its config as a terminal QR
+  (`qrencode -t ansiutf8 < phone.conf` — same gesture as token enrollment).
+  The only internet-visible thing is one forwarded UDP port that is silent
+  to unauthenticated packets — scanners see it as closed — so the app's
+  pre-auth surface (rustls handshake, hyper parsing, auth middleware) is
+  reachable only from enrolled peers, and the token becomes defense in depth
+  rather than the sole gate. Addressing stays single-path: the phone reaches
+  the backend at its WireGuard address (e.g. `10.66.0.1`) from everywhere —
+  one address in the app, one SAN in the leaf cert (`SERVER_IP=`/SAN
+  override in the cert script), no home/away distinction. Another machine
+  later is one keypair + one `[Peer]` block.
+  - Operational needs, accepted: a public endpoint hostname. The home IP is
+    mostly static but not guaranteed, so the phone's endpoint is a DDNS name
+    (free, e.g. DuckDNS, or the router's built-in client; a curl cron on the
+    backend host works too) that tracks changes automatically. One WireGuard
+    nuance: the phone app resolves the endpoint hostname when the tunnel
+    comes up and does not re-resolve on its own, so on the rare IP change
+    the fix is toggling the tunnel off/on once DDNS has caught up (minutes).
+    The symptom is obvious (app can't reach the backend) and lossless — the
+    SSE cursor design means reconnects replay whatever was missed. Also:
+    at-home traffic rides NAT hairpinning on the router (verify early; most
+    support it, and the fallback is toggling the tunnel off at home).
+  - Rejected: **Tailscale** — same WireGuard underneath with easier setup
+    (no port forward, LAN peer discovery), but it adds a third-party
+    coordination service and account this setup doesn't need at two or
+    three devices; **Headscale** — self-hosting that coordination server is
+    strictly more moving parts than one wg config per peer at this scale;
+    **forwarding the HTTPS port directly** — puts every internet scanner
+    one pre-auth bug away from RCE on a machine holding SSH keys.
+  - The server still refuses to start without TLS — no plaintext listener
+    exists even inside the tunnel, so the token can't travel unencrypted by
+    misconfiguration, and interface binding failing closed (refuse to start
+    if `wg0` is absent, rather than falling back to 0.0.0.0) is part of the
+    same guarantee.
+- The bootstrap-over-HTTP trick from the updater is unnecessary here — the
+  app installs via Local Updater.
+
+## App (`app/`)
+
+Kotlin + Compose Multiplatform, single `:androidApp` module, same versions as
+local-updater (Kotlin 2.4.x, CMP 1.11.x, JDK 21). Screens:
+
+1. **Session list** — cards: kind icon, title, host, model, status
+   (running / awaiting answer / idle / exited), last activity. Spawn FAB;
+   swipe/long-press to delete (confirm). Sessions awaiting an answer sort to
+   the top — that's the "your turn" inbox.
+2. **Spawn** — kind, host (from config), model (Claude list is static+editable;
+   llama list from config), working directory, permission mode (Claude),
+   title.
+3. **Session screen** — the core:
+   - Transcript rendered from the event stream: markdown text, inline images,
+     collapsed-by-default tool cards (name + input summary, expandable to
+     output; a spinner while `ToolStart` has no matching `ToolEnd`).
+   - Question cards inline: option buttons for AskUserQuestion, allow/deny for
+     permissions, free-text where allowed.
+   - Input bar: text, attach (camera/gallery/file), send — **always enabled**;
+     mid-run sends become steering messages.
+   - Top bar: model chip (tap to change), stop button while running, token
+     count, compact button (llama), overflow → delete.
+4. **Usage** — window bars for the 5-hour and weekly limits with reset times.
+5. **Settings** — server address + token, hosts editor, llama model list
+   editor.
+
+Networking mirrors local-updater's app layer (`AppsApi.kt` style thin client +
+pinned transport), plus an SSE client with `after=` resume driven by
+connectivity/lifecycle. The app keeps no persistent transcript store — the
+backend's transcript is the source of truth; the app caches only for the
+screen it's showing.
+
+## Compaction: options explored
+
+Context: raw llama-server has no conversation memory management; the context
+window just fills.
+
+1. **pi's auto-compaction** — *chosen*. When the prompt nears the model's
+   context limit, pi summarizes older history with the model itself and
+   replaces it with a structured summary; threshold configurable; manual
+   `compact` also exposed. Battle-tested, zero work for us.
+2. **Manual compaction in a custom Rust loop** — *the later third driver*.
+   The design when we build it: every llama-server response reports prompt +
+   completion token counts; track them against `n_ctx` (from `/props`); at a
+   threshold (~75%), pause, run a summarization request over all but the last
+   few turns ("state of the task, decisions made, open items, relevant
+   file/tool state"), replace those turns with the summary as a system-adjacent
+   message, continue. Keep the full pre-compaction transcript on disk — the
+   phone view never loses history, only the model's view shrinks. Worth doing
+   eventually for control over the summarization prompt and for tool-loop
+   experiments pi doesn't allow.
+3. **llama-server `--context-shift`** — *rejected* as the strategy. It
+   truncates old KV cache entries: silent forgetting, no summary, and it
+   corrupts the harness's view of what the model knows. Fine as a server-side
+   safety net; not memory management.
+
+## Phases
+
+1. **Skeleton** — repo layout, cert script, TLS + token auth, wg0-bound
+   listener (fail closed if the interface is missing), config.json,
+   session registry with a fake `EchoDriver`, session list + session screen in
+   the app end-to-end over SSE. Proves the whole pipe before any AI is
+   involved.
+2. **Claude local** — ClaudeDriver: spawn, stream text/tools, mid-run send,
+   interrupt, permission questions, AskUserQuestion, images both ways, delete.
+   *Milestone: daily-drivable Claude replacement on localhost.*
+3. **Usage screen.**
+4. **llama.cpp** — LlamaServerManager (local), PiDriver, model change,
+   compaction controls.
+5. **SSH** — host config, remote spawn for both kinds, remote llama-server
+   with port forward, attachment shipping.
+6. **Polish** — reconnect edges, notification when a session awaits an answer
+   (the "your turn" push), transcript search, whatever daily use surfaces.
+
+Each phase ends runnable and verified against the real thing (rule 22); the
+backend gets tests where logic is pure (event normalization, transcript
+cursors, config persistence, refcounting) — the app is UI over the API and is
+verified by running it, matching local-updater's posture.
+
+## Open questions / risks
+
+- **Claude stream-json control protocol details** (permission requests,
+  set-model, interrupt wire format) are the least-documented dependency and
+  version-coupled to the installed CLI. Phase 2 starts by probing the
+  installed version and pinning what works; the `--resume` respawn fallback
+  covers whatever the control channel can't do.
+- The **usage endpoint is undocumented** and has changed rate-limit behavior
+  before; treat as best-effort.
+- **pi RPC schema drift** — pin a pi version; the translator is one file.
+- Whether **notifications** need FCM or a foreground-service polling
+  connection — decide in phase 6; the SSE cursor design already supports
+  either.
+- Claude sessions over SSH need the remote host **logged in to Claude**; usage
+  reporting reads only the backend host's credentials. Acceptable for now
+  (same account everywhere); revisit if not.
+
+## References
+
+Research behind the decisions above (verified 2026-08-24; re-check against
+installed versions when each phase starts):
+
+- pi RPC protocol: https://github.com/badlogic/pi-mono/blob/main/packages/coding-agent/docs/rpc.md
+  — commands (`prompt`, `steer`, `follow_up`, `abort`, `set_model`,
+  `compact`, `set_auto_compaction`, session ops) and the event stream.
+- pi + llama-server in practice: https://medium.com/@tolgaeren/running-pi-with-local-llms-c596aa14b062
+- llama-server API (`/health`, `/props`, OpenAI-compatible endpoints,
+  `--context-shift`): https://github.com/ggml-org/llama.cpp/blob/master/tools/server/README.md
+  and the offline-agentic-coding walkthrough:
+  https://github.com/ggml-org/llama.cpp/discussions/14758
+- Usage endpoint (`GET https://api.anthropic.com/api/oauth/usage`, bearer
+  token from `~/.claude/.credentials.json`, headers
+  `anthropic-beta: oauth-2025-04-20` + `User-Agent: claude-code/<version>`,
+  ≥180 s polling; wrong User-Agent → aggressive 429 bucket):
+  https://github.com/anthropics/claude-code/issues/31637 and
+  https://github.com/Maciek-roboblog/Claude-Code-Usage-Monitor/issues/202
+- Sibling project this repo's conventions mirror: `../local-updater`
+  (README.md + AGENTS.md — server/registry/routes layout, cert scheme,
+  testing posture, Android env notes).