# ai-app — plan

A phone interface to AI coding sessions — Claude Code and llama.cpp for now —
built to replace the Claude app for day-to-day use. Two motivations: local
models need a front end at all, and owning the client means fixing the things
the official app gets wrong (e.g. it won't deliver a typed message until the
session fully finishes its turn, where the TUI injects it at the next tool
boundary).

Same shape as `../local-updater`: a Rust (Axum) backend on the desktop, a
Kotlin/Compose Android app, pinned self-signed TLS between them.

## The one idea everything hangs off

Both session types are **a child process speaking JSONL over stdio**:

- Claude Code: `claude -p --input-format stream-json --output-format stream-json`
  — bidirectional streaming JSON. User messages sent while a turn is running
  are injected at the next opportunity (the TUI behavior we want), a control
  protocol carries interrupts and permission requests, `--resume <id>` picks a
  session back up after a backend restart.
- llama.cpp: **pi in RPC mode** (`pi --mode rpc`), pointed at a llama-server
  endpoint. Same deal: JSONL on stdio, `prompt` (with images), `steer` for
  mid-run injection, `abort`, `set_model`, `compact` / `set_auto_compaction`,
  session files that survive restarts, structured events for streaming text
  and tool executions.

So the backend has one abstraction — spawn a process, translate its dialect to
a common event stream, keep an append-only transcript — and two translators.
SSH support falls out of the same shape: a remote session is the identical
command run as `ssh <host> <cmd>`; stdio doesn't care.

Decisions already made (2026-08-24):

- llama.cpp harness: **pi RPC now**, with the session abstraction kept clean
  enough that a custom Rust agent loop can be added as a third driver later.
- The backend **manages llama-server itself** (start with a chosen GGUF, stop,
  swap models), locally and over SSH.
- Claude permission prompts are **interactive in the app**, with a per-session
  permission mode chosen at spawn.
- **One backend** on the main machine; the phone talks only to it, and it
  reaches other hosts via SSH. Remote hosts need the CLIs installed but no
  backend.

## Architecture

```
Android app (Compose)
   │  HTTPS (pinned CA) — REST for actions, SSE for live events
   ▼
backend (Rust/Axum, desktop)
   ├─ SessionManager ── Session ── Driver (trait)
   │                      ├─ ClaudeDriver   (claude stream-json)
   │                      └─ PiDriver       (pi --mode rpc)
   │        each driver's process is spawned locally or as `ssh host …`
   ├─ LlamaServerManager  (llama-server lifecycle, local + SSH)
   ├─ UsageMonitor        (Anthropic OAuth usage endpoint)
   └─ config.json + per-session transcript files
```

### Backend layout (`server/`)

Mirroring local-updater's stack: axum 0.8, axum-server + rustls, tokio, serde,
clap, tracing. Rust edition 2024, warning-clean, clippy in CI habit.

- `main.rs` — bootstrap, TLS listener.
- `routes.rs` — the whole HTTP table in one module doc comment (as in
  local-updater).
- `session/mod.rs` — `SessionManager`: the live session registry, every
  mutation funnels through it (the `registry.rs` pattern: in-memory and
  on-disk state can't come apart).
- `session/driver.rs` — the `Driver` trait and the common event model.
- `session/claude.rs`, `session/pi.rs` — the two translators.
- `session/transcript.rs` — append-only JSONL event log per session, with
  monotonically increasing sequence numbers (the phone's resume cursor).
- `llama.rs` — `LlamaServerManager`.
- `hosts.rs` — host configs and the ssh command builder.
- `usage.rs` — Anthropic usage polling.
- `config.rs` — persisted schema.

### The common event model

Driver output, whatever the dialect, is normalized into one event enum before
it touches the transcript or the phone:

- `AssistantText { delta }` — streaming text (rendered as markdown).
- `ToolStart / ToolUpdate / ToolEnd { tool, input, output }` — the "view tools
  it's running" screen is just these.
- `Image { ref }` — images in output (screenshots from tools, etc.) are saved
  under the session dir and referenced by id; the phone fetches them by URL.
- `Question { id, prompt, options }` — anything the session needs a human for:
  Claude's AskUserQuestion, and **permission requests** (canUseTool) are the
  same shape with approve/deny options. Answered via one endpoint.
- `Status { state }` — idle / running / awaiting-input / compacting / exited.
- `UsageDelta { tokens }` — per-turn token counts where the dialect reports
  them (both do).
- `Error { message }`.

Every event is appended to the session's transcript file with a sequence
number, then fanned out to any connected SSE subscribers. The phone renders
purely from this stream: reconnecting means "give me events after seq N" —
no separate "load history" path to drift from the live one.

Inbound, the driver trait is small:

```rust
trait Driver {
    fn send_user_message(&self, text: String, images: Vec<ImageRef>);
    fn answer_question(&self, id: QuestionId, answer: Answer);
    fn interrupt(&self);                       // stop mid-run, session survives
    fn set_model(&self, model: &str);
    fn compact(&self);                         // pi: native; claude: /compact
    fn shutdown(&self);                        // graceful process exit
}
```

`send_user_message` during a run is the point of the whole app: both dialects
queue it for injection at the next tool boundary rather than the end of the
turn. Claude's dialect: a `user` message on stdin mid-stream; pi's: `steer`.

### Claude driver specifics

- Spawn: `claude -p --verbose --input-format stream-json --output-format
  stream-json --permission-mode <mode>` in the chosen working directory, plus
  `--model` at spawn. Permission mode (default/plan/acceptEdits/
  bypassPermissions) is chosen on the spawn screen.
- Interactive permissions: run with the stream-json control protocol's
  permission request flow (the same mechanism the Agent SDK's `canUseTool`
  uses) so tool approvals arrive as control requests, become `Question`
  events, and our answer goes back as the control response. **Verify the
  exact control-request wire format against the current CLI early in
  implementation** — it's the least-documented part of this plan.
- Interrupt: control-protocol interrupt request.
- Model change mid-session: try the control protocol's set-model; if the
  installed CLI doesn't support it, fall back to `shutdown` + respawn with
  `--resume <session_id> --model <new>` — cheap, since Claude persists
  sessions in `~/.claude/projects` anyway. That same resume path is the crash
  recovery story: a dead backend or a killed process loses nothing.
- Images in: base64 image content blocks in the stream-json user message.
- Working directory, host, and model are spawn-screen fields.

### pi driver specifics

- Spawn: `pi --mode rpc --provider openai-generic --model <name>` (endpoint =
  the llama-server the LlamaServerManager provides), `--session-dir` under
  our session storage so transcripts and pi's own session files live together.
- Auto-compaction on by default (`set_auto_compaction`), threshold
  configurable per session; manual `compact` exposed as a button.
- `steer` for mid-run messages, `abort` for stop, `set_model` when the target
  endpoint changes.
- pi's session JSONL gives resume-after-restart, same as Claude's.

### llama-server management

`config.json` lists **models** (name → GGUF path or llama-server args, per
host) and **hosts**. The manager runs at most one llama-server per
`(host, model)`, spawned on demand when a session needs it:

- Spawn (local or `ssh host llama-server …`) on an allocated port, wait on
  `/health`, hand the endpoint to the pi driver.
- Refcounted by sessions. The path out, written in the same change as the
  spawn: the last session using an instance releasing it starts an idle
  timer (configurable, e.g. 10 min), after which it's killed. Delete of the
  last session kills it immediately.
- "Change model" on a llama session = acquire the new model's server,
  `set_model` on pi, release the old one. Context carries over (it's
  prompt-replayed by pi against the new endpoint).
- Remote llama-server output is only reachable from the backend host, and
  binds localhost on the remote side with an SSH local port forward
  (`ssh -L`) held by the manager — no LAN-exposed inference ports.

### SSH

- Host entries in `config.json`: name, `user@host`, optional ssh options,
  which capabilities it has (claude / pi / llama-server, with paths if not on
  PATH). Key-based auth only, using the system `ssh` client via
  `tokio::process` — no Rust SSH library; this inherits `~/.ssh/config`,
  agents, and jump hosts for free. (Rule 23: openssh is already here and
  battle-tested; a library buys nothing but a second config surface.)
- A remote session is exactly a local one with the command wrapped in
  `ssh -T host …`. Process death ≙ connection death; the session shows as
  `exited` and both dialects resume (`--resume` / pi session file) on respawn,
  so a dropped SSH connection is an annoyance, not data loss.
- Images and attachments for remote sessions are written to the remote
  session dir via `scp`/stdin before the message referencing them is sent.

### Usage limits (Claude)

Poll `https://api.anthropic.com/api/oauth/usage` — the same endpoint behind
Claude Code's `/usage` — with the OAuth access token from Claude Code's local
credential store (`~/.claude/.credentials.json`), headers
`anthropic-beta: oauth-2025-04-20` and `User-Agent: claude-code/<version>`
(without the User-Agent it lands in an aggressively rate-limited bucket).
Poll at ≥180 s, only while any Claude session exists or the usage screen is
open, cache the last answer. Surface: 5-hour and weekly window utilization %
and reset times. It's undocumented, so `usage.rs` treats every field as
optional and degrades to "unavailable" rather than erroring. Structure it as
one `UsageProvider` per paid service so a second service later is a new impl,
not a parallel screen (rule 9).

### HTTP surface (phone ⇄ backend)

REST for actions, one SSE stream per open session screen for events, all over
the pinned TLS listener. SSE over WebSocket because resume-by-cursor
(`Last-Event-ID` = transcript seq) is native to it and the inbound direction
is plain POSTs anyway.

```
GET  /sessions                        list (id, kind, title, host, model, status, last activity)
POST /sessions                        spawn {kind, host, model, cwd, permission_mode, title}
GET  /sessions/:id/events?after=N     SSE: transcript replay from N, then live
POST /sessions/:id/message            {text, attachment_ids}
POST /sessions/:id/answer             {question_id, answer}   (questions and permissions)
POST /sessions/:id/interrupt
POST /sessions/:id/model              {model}
POST /sessions/:id/compact            (llama sessions)
DELETE /sessions/:id                  kill process, release llama-server, delete transcript+files
POST /attachments                     multipart upload → id (referenced by /message)
GET  /files/:session/:id              images/files the session produced
GET  /usage                           cached usage windows
GET/PUT /hosts, /models               config editing from the phone
```

Sessions live in `config.json` + a per-session directory (transcript.jsonl,
attachments, produced images). Deleting a session is the complete path out of
everything spawning one created.

### Security

- TLS with a self-signed CA, pinned in the app — `gen-dev-cert.sh` and
  `PinnedCert.kt` copied from local-updater, same idempotent-CA/reissued-leaf
  scheme, same one-way-door caveat about regenerating the CA.
- This server is strictly more dangerous than the updater: its API *is*
  remote code execution (spawn a bypass-permissions Claude on any SSH host).
  Pinning authenticates the server to the phone but not the phone to the
  server, so a bearer token adds the other direction. Threat model: the token
  gates LAN-reachable RCE; it does not (and cannot) defend a compromised
  backend host or phone — those are inside the trust boundary, and a
  compromised phone is handled by rotation.
  - **Generation**: 256 bits from the OS CSPRNG on first run, base64url. A
    machine credential, never typed twice, so unguessable costs nothing; at
    this entropy no key stretching is needed.
  - **Enrollment**: printed once as a terminal QR code (`qrcode` crate,
    ANSI), encoding `aiapp://enroll?host=…&port=…&token=…`. The CA stays
    embedded in the APK (`PinnedCert.kt` pattern), so the QR carries no
    trust material — photographing the terminal leaks only the token
    (rotatable), never a way to weaken pinning.
  - **Storage**: server keeps only the SHA-256 in `config.json` (plain hash
    is enough for high-entropy random input; buys that a leaked config
    doesn't leak the credential). No "show token again" — lost means rotate.
    Phone side: Keystore-backed encrypted preferences.
  - **Transport**: `Authorization: Bearer` header on every request including
    the SSE GET. Never a query parameter (URLs leak into logs). The tracing
    layer must not log the header — covered by a test so a logging change
    can't silently start leaking it.
  - **Verification**: one middleware wrapping the entire router in `main.rs`,
    never per-route, so a new route can't forget auth. Zero unauthenticated
    endpoints, `/health` included. Hash-then-constant-time-compare
    (`subtle`); failures logged with peer address plus a small fixed delay —
    not against brute force (infeasible at 256 bits) but so scanners show up
    in the log.
  - **Rotation (the path out)**: `--rotate-token` regenerates, invalidates
    the old hash immediately, reprints the QR. That's the whole lost-phone
    story. Config stores a *list* of `{name, hash}` (of one, today) so
    per-device tokens with individual revocation are a config entry later,
    not a schema migration.
  - **Why not mTLS**: stronger in theory (key never leaves the Keystore, no
    bearer secret to exfiltrate), but given pinning the delta is only
    "someone reads the token off a device already inside the trust
    boundary", and it costs Android client-cert provisioning ceremony and a
    worse new-phone story than a QR scan. Revisit if this outgrows
    single-user-on-LAN.
- **Off-network access: plain WireGuard** (decided 2026-08-24; no third
  party). The backend binds to the WireGuard interface (`wg0`) only; the
  phone runs the official WireGuard app (always-on VPN, per-app tunneling),
  enrolled by scanning its config as a terminal QR
  (`qrencode -t ansiutf8 < phone.conf` — same gesture as token enrollment).
  The only internet-visible thing is one forwarded UDP port that is silent
  to unauthenticated packets — scanners see it as closed — so the app's
  pre-auth surface (rustls handshake, hyper parsing, auth middleware) is
  reachable only from enrolled peers, and the token becomes defense in depth
  rather than the sole gate. Addressing stays single-path: the phone reaches
  the backend at its WireGuard address (e.g. `10.66.0.1`) from everywhere —
  one address in the app, one SAN in the leaf cert (`SERVER_IP=`/SAN
  override in the cert script), no home/away distinction. Another machine
  later is one keypair + one `[Peer]` block.
  - Operational needs, accepted: a public endpoint hostname. The home IP is
    mostly static but not guaranteed, so the phone's endpoint is a DDNS name
    (free, e.g. DuckDNS, or the router's built-in client; a curl cron on the
    backend host works too) that tracks changes automatically. One WireGuard
    nuance: the phone app resolves the endpoint hostname when the tunnel
    comes up and does not re-resolve on its own, so on the rare IP change
    the fix is toggling the tunnel off/on once DDNS has caught up (minutes).
    The symptom is obvious (app can't reach the backend) and lossless — the
    SSE cursor design means reconnects replay whatever was missed. Also:
    at-home traffic rides NAT hairpinning on the router (verify early; most
    support it, and the fallback is toggling the tunnel off at home).
  - Rejected: **Tailscale** — same WireGuard underneath with easier setup
    (no port forward, LAN peer discovery), but it adds a third-party
    coordination service and account this setup doesn't need at two or
    three devices; **Headscale** — self-hosting that coordination server is
    strictly more moving parts than one wg config per peer at this scale;
    **forwarding the HTTPS port directly** — puts every internet scanner
    one pre-auth bug away from RCE on a machine holding SSH keys.
  - The server still refuses to start without TLS — no plaintext listener
    exists even inside the tunnel, so the token can't travel unencrypted by
    misconfiguration, and interface binding failing closed (refuse to start
    if `wg0` is absent, rather than falling back to 0.0.0.0) is part of the
    same guarantee.
- The bootstrap-over-HTTP trick from the updater is unnecessary here — the
  app installs via Local Updater.

## App (`app/`)

Kotlin + Compose Multiplatform, single `:androidApp` module, same versions as
local-updater (Kotlin 2.4.x, CMP 1.11.x, JDK 21). Screens:

1. **Session list** — cards: kind icon, title, host, model, status
   (running / awaiting answer / idle / exited), last activity. Spawn FAB;
   swipe/long-press to delete (confirm). Sessions awaiting an answer sort to
   the top — that's the "your turn" inbox.
2. **Spawn** — kind, host (from config), model (Claude list is static+editable;
   llama list from config), working directory, permission mode (Claude),
   title.
3. **Session screen** — the core:
   - Transcript rendered from the event stream: markdown text, inline images,
     collapsed-by-default tool cards (name + input summary, expandable to
     output; a spinner while `ToolStart` has no matching `ToolEnd`).
   - Question cards inline: option buttons for AskUserQuestion, allow/deny for
     permissions, free-text where allowed.
   - Input bar: text, attach (camera/gallery/file), send — **always enabled**;
     mid-run sends become steering messages.
   - Top bar: model chip (tap to change), stop button while running, token
     count, compact button (llama), overflow → delete.
4. **Usage** — window bars for the 5-hour and weekly limits with reset times.
5. **Settings** — server address + token, hosts editor, llama model list
   editor.

Networking mirrors local-updater's app layer (`AppsApi.kt` style thin client +
pinned transport), plus an SSE client with `after=` resume driven by
connectivity/lifecycle. The app keeps no persistent transcript store — the
backend's transcript is the source of truth; the app caches only for the
screen it's showing.

## Compaction: options explored

Context: raw llama-server has no conversation memory management; the context
window just fills.

1. **pi's auto-compaction** — *chosen*. When the prompt nears the model's
   context limit, pi summarizes older history with the model itself and
   replaces it with a structured summary; threshold configurable; manual
   `compact` also exposed. Battle-tested, zero work for us.
2. **Manual compaction in a custom Rust loop** — *the later third driver*.
   The design when we build it: every llama-server response reports prompt +
   completion token counts; track them against `n_ctx` (from `/props`); at a
   threshold (~75%), pause, run a summarization request over all but the last
   few turns ("state of the task, decisions made, open items, relevant
   file/tool state"), replace those turns with the summary as a system-adjacent
   message, continue. Keep the full pre-compaction transcript on disk — the
   phone view never loses history, only the model's view shrinks. Worth doing
   eventually for control over the summarization prompt and for tool-loop
   experiments pi doesn't allow.
3. **llama-server `--context-shift`** — *rejected* as the strategy. It
   truncates old KV cache entries: silent forgetting, no summary, and it
   corrupts the harness's view of what the model knows. Fine as a server-side
   safety net; not memory management.

## Phases

1. **Skeleton** — repo layout, cert script, TLS + token auth, wg0-bound
   listener (fail closed if the interface is missing), config.json,
   session registry with a fake `EchoDriver`, session list + session screen in
   the app end-to-end over SSE. Proves the whole pipe before any AI is
   involved.
2. **Claude local** — ClaudeDriver: spawn, stream text/tools, mid-run send,
   interrupt, permission questions, AskUserQuestion, images both ways, delete.
   *Milestone: daily-drivable Claude replacement on localhost.*
3. **Usage screen.**
4. **llama.cpp** — LlamaServerManager (local), PiDriver, model change,
   compaction controls.
5. **SSH** — host config, remote spawn for both kinds, remote llama-server
   with port forward, attachment shipping.
6. **Polish** — reconnect edges, notification when a session awaits an answer
   (the "your turn" push), transcript search, whatever daily use surfaces.

Each phase ends runnable and verified against the real thing (rule 22); the
backend gets tests where logic is pure (event normalization, transcript
cursors, config persistence, refcounting) — the app is UI over the API and is
verified by running it, matching local-updater's posture.

## Open questions / risks

- **Claude stream-json control protocol details** (permission requests,
  set-model, interrupt wire format) are the least-documented dependency and
  version-coupled to the installed CLI. Phase 2 starts by probing the
  installed version and pinning what works; the `--resume` respawn fallback
  covers whatever the control channel can't do.
- The **usage endpoint is undocumented** and has changed rate-limit behavior
  before; treat as best-effort.
- **pi RPC schema drift** — pin a pi version; the translator is one file.
- Whether **notifications** need FCM or a foreground-service polling
  connection — decide in phase 6; the SSE cursor design already supports
  either.
- Claude sessions over SSH need the remote host **logged in to Claude**; usage
  reporting reads only the backend host's credentials. Acceptable for now
  (same account everywhere); revisit if not.

## References

Research behind the decisions above (verified 2026-08-24; re-check against
installed versions when each phase starts):

- pi RPC protocol: https://github.com/badlogic/pi-mono/blob/main/packages/coding-agent/docs/rpc.md
  — commands (`prompt`, `steer`, `follow_up`, `abort`, `set_model`,
  `compact`, `set_auto_compaction`, session ops) and the event stream.
- pi + llama-server in practice: https://medium.com/@tolgaeren/running-pi-with-local-llms-c596aa14b062
- llama-server API (`/health`, `/props`, OpenAI-compatible endpoints,
  `--context-shift`): https://github.com/ggml-org/llama.cpp/blob/master/tools/server/README.md
  and the offline-agentic-coding walkthrough:
  https://github.com/ggml-org/llama.cpp/discussions/14758
- Usage endpoint (`GET https://api.anthropic.com/api/oauth/usage`, bearer
  token from `~/.claude/.credentials.json`, headers
  `anthropic-beta: oauth-2025-04-20` + `User-Agent: claude-code/<version>`,
  ≥180 s polling; wrong User-Agent → aggressive 429 bucket):
  https://github.com/anthropics/claude-code/issues/31637 and
  https://github.com/Maciek-roboblog/Claude-Code-Usage-Monitor/issues/202
- Sibling project this repo's conventions mirror: `../local-updater`
  (README.md + AGENTS.md — server/registry/routes layout, cert scheme,
  testing posture, Android env notes).