A multimodal model is loaded with the `mmproj` found beside its weights --
which is how a repository publishes the pair -- and an attached image rides
in the request as an `image_url` data URI, so it reaches a model on another
machine without the file going there. Nothing is done for a model without a
projector: no captioning, no OCR, no second model.
Whether a session takes pictures is measured rather than assumed:
`/props`'s `modalities.vision` from the server that loaded the model, in
three states, because a model still coming off disk has genuinely not said.
Unknown is offered rather than refused -- a control withheld because nobody
could ask goes missing from sessions that would have taken it. The answer
reaches the phone twice per model as `Event::Images`, so the photo button is
withdrawn the moment a model with vision is left rather than at whatever
later point the session row is fetched again.
A message carrying an image a model cannot read is stopped rather than
stripped: `llama-server` refuses the whole request over one image part, and
a message sent without its picture would be answered as though the picture
had never been mentioned. The phone will not attach one, and the driver
refuses it again at the three moments the answer can first exist -- at the
door, when a message queued behind a loading model is read, and at the tool
boundary a steer enters by. An earlier turn's image folds into a line of
words for a model without vision, so switching a conversation onto one does
not end it.
A projector is filtered out of the models a provider *offers*, since a
session started on one is a server that cannot load it; it stays in the
machine's own model list, where a file on a disk is managed.
Verified against ggml-org/SmolVLM-256M-Instruct-GGUF, local and over ssh:
"In this picture there is a red circle." Switching that session to
Qwen3-0.6B reports `refused`, refuses the next picture with the reason, and
still answers an ordinary message.
The count beside the status said how much work was going and never what,
so "3 bg tasks" was a number with no way to find out what it was about.
Drivers now report the tasks themselves rather than a size:
`Driver::background_tasks` returns `Vec<BackgroundTask>` -- id, the
provider's own description, and a kind -- served by
`GET /sessions/{id}/background`. It is runtime state, never persisted,
and `null` is "nobody has said", which is what a session with no process
answers and what the panel says in words rather than drawing as an empty
list. `description` is optional because Codex names a background terminal
by a process id, and a number drawn as a name is worse than admitting
there is none.
Claude's `background_tasks_changed` entries turn out to be objects
carrying `task_id`, `task_type` and `description`, so each is read rather
than counted -- and an `ambient` one is now dropped from the list and the
count alike, on the CLI's own instruction: a live-update watcher is not
activity, and counting one left a session reading `waiting` with nothing
to wait for.
The phone draws them in the right-hand panel above the subagents,
collapsed to "2 bg tasks running" and pushing the subagents down when
opened. Both lists are items of one lazy column, so neither can run off
the panel, and the section is refetched whenever the live count moves --
a card for work that has finished is exactly the stale measurement the
count exists not to be.
Verified against the real Claude CLI (2.1.261): a backgrounded `sleep 120`
came back as `{"id":"br16327wr","description":"Sleep for 120 seconds",
"kind":"command"}`, and on the emulator against the echo rig the section
appeared, expanded, and dropped a card as its task finished.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A resume token the CLI will not accept made the session unrecoverable
rather than merely failed: every later start passed the same `--resume`,
died the same way, and nothing ever forgot it, so the chat could not be
opened again from the phone.
The reader now recognises both refusals the CLI gives, forgets the token
and emits `Cleared`, which is what the Codex path already does for a
thread whose rollout has gone. The transcript is this server's and
survives; only the model's context restarts.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A message typed during a turn reaches the model at the next model call if
the turn has one left, and otherwise as the opening line of the next turn --
Claude's read out of the fifo after the turn ended, Codex's requeued when
turn/steer is refused. Read there it is indistinguishable from a reply, so
the model treats the answer it just gave as seen.
Both drivers now compose the text the CLI receives through
driver::message_body, which prefixes a note saying the message was written
without having seen the rest of that turn. The transcript still holds the
words that were typed; only the CLI's copy carries the note.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A turn started by something with no row of its own -- a subagent reporting
back, a peer message the CLI only owns up to at the end -- met the previous
reply with nothing between it, and the fold grew that reply rather than
starting a new one. Two answers were drawn as one paragraph, running together
mid-sentence with not even a space between them. The fold now refuses to grow
a settled reply, and `joinPages` carries the same rule across a page boundary.
The other half is the row. `Event::TaskNote` records a background task
reporting back -- a subagent that finished, or a backgrounded command -- with
its title, how it ended and what it said; `TaskNoteRow` draws it as a card,
since somebody said this, and its own row rather than an update to the Task
call's, which is above everything the session has said since. Reported once
however many of the CLI's two lifecycle shapes arrive.
`SessionStatus::Waiting` is a session whose own turn is over while work it
started is not. `Idle` means "waiting for a person" and this means the
opposite, so reporting it as idle sent a "finished" notification at the one
moment that was untrue. Drawn as "waiting" in `waitingColor`; the queue and
the held-command boundary release on either end-of-turn status, so a message
sent while a subagent runs is not held until it finishes.
And a usage limit the account hits inside a subagent now reaches the session
as well as the subagent's transcript. `resume.rs` can only schedule against a
session, and a background Task outliving its parent's turn is the ordinary
case, so auto-resume was doing nothing at all for it.
The status word and its colour were two `when`s on two screens, and the second
missed `waiting` silently; they are `sessionStatusWord`/`sessionStatusColour`
now. Echo's `/subagent n` reproduces the whole shape, staggered a second
apart. Verified on the emulator against the sandbox: 169 server tests, ktfmt,
clippy, rustfmt, Android lint and the JVM unit tests all clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A subagent is a second transcript owned by a session, in the same event
model, with no process and no controls. The claude translator routes lines
carrying parent_tool_use_id to a per-subagent translator and transcript
under <session>/subagents/<tool_use_id>; three routes expose the list, a
transcript page and the SSE stream. Echo grows /subagent [n] as the rig.
On the phone a card with subagents ends in a chevron expander, collapsed by
default, opening to outlined subcards styled like dev-updater's components;
a subcard opens SessionScreen in read-only form, addressed through
TranscriptAddress so paging, cache and stream are shared.
Design in SUBAGENTS.md; choices awaiting review in DECISIONS.md.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A 401 from the usage endpoint means the stored access token has expired.
Refreshing it here is not an option: Anthropic's OAuth rotates the refresh
token, so a second refresher invalidates the CLI's copy and forces a
re-login on a machine that usually has a live session on it. So run the CLI
there instead and re-read what it wrote.
`doctor` rather than `auth status`: probed against 2.1.258 with an invalid
token, `auth status` answers loggedIn:true from the file alone and never
reaches the network. The same probe showed a failed refresh blanks both
tokens, which is why this stays on the 401 path.
Also gives ProviderConfig one program() so the CLI's default path is not
written down twice.
Output is about an eighth of what a session costs and thinking is nearly
all of it -- prose is ~1.5% of output tokens, measured over 27,015 requests
of this account's own transcripts -- so the level is the largest saving
available short of shortening the conversation itself.
Shaped like the working directory rather than like the model: the CLI's
only two setting control requests are `set_model` and `set_permission_mode`
(checked against the 2.1.258 binary), so `--effort` is read when the process
launches and cannot be asked of a running one. `set_session_effort` records
the level and stops the process; the next message or Start launches one that
has it. That is also why the picker is in the session settings dialog beside
Move, and not on the bar beside the model and the mode, which take effect
mid-turn.
`None` is a level in its own right -- the CLI's own default -- so the picker
can return to it, and a blank is normalized to it at the boundary rather
than stored as a level the CLI would reject.
Offered only where it means something: `DriverKind::takes_effort` reports
the capability and the phone leaves the row out entirely, rather than the
session-type branch this app does not have anywhere else. A llama session
would otherwise get a control whose only effect is stopping its process.
Verified on the emulator against the sandbox's fake CLI: the picker sets it,
the server reports it, and an echo session's dialog is unchanged.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The markdown had accumulated a lot that was stale rather than wrong.
PLAN.md still described pi as the llama.cpp harness, a refcounted
LlamaServerManager, and a providers-by-hosts cross-product, all of which
were superseded or never built; it also carried a second copy of the HTTP
table that routes.rs owns. EXPLORER.md and TRANSCRIPT_CACHE.md held
implementation checklists for work that has since landed. AGENTS.md
restated most of PLAN.md's design instead of being the working-notes
layer it says it is. 3225 lines of markdown to 2180, with the stale
sections gone rather than reworded.
On the server, comments explaining what the code already says are out and
the ones recording a constraint, a measurement or an incident are kept but
cut to a few lines each: 5504 comment lines to 4586.
Four doc comments in session/mod.rs, and one each in process.rs and
usage.rs, had drifted onto the item above the one they describe --
functions were reordered without them, so `stop_session`'s doc sat on
`set_session_cwd`, `stat_of`'s on `struct Stat`, and `UsageMonitor`'s on
`type Cached`. Each is back on its own item.
routes.rs's module table also claimed later phases would add `/hosts`,
which setups replaced.
cargo test (127 passed), clippy --all-targets and fmt are clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Uploads no longer sit whole in memory anywhere: the phone writes the
multipart body chunked as it reads the picked file, and the server writes
each chunk to a `.part` file under the session and renames it when whole.
The per-request cap is 4 GB and bounds disk, not memory.
A file attached to a session on another machine is copied there in the
same request: one ssh invocation takes the bytes on stdin into the
setup's `attachmentsDir` (new, optional, on the machine form and in the
config), else the session's cwd, else the login home, and answers with
`pwd -P`, which is recorded beside the file as `<name>.remote` and is the
path the driver tells the CLI. A failed copy fails the upload and says
why, so no message ever names a file that is not there. The host keeps
its copy so transcripts can reference and fetch it. Measured against the
Gentoo test guest: a 40 MB file shared from the phone arrived there byte
for byte. The tilde in that setting is the remote home, so it is not
expanded on the server the way other setup paths are.
TRANSCRIPT_RENDERING.md records the week of transcript work -- the
measurements behind each decision, the harness, what was rejected, and
what to do next -- so a new session can start from it.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Attachments were images only. Now any file can be attached: from the file
chooser behind the "+" menu, or from Android's share sheet, which the app
is now in. An image still goes to the model as a picture; anything else is
stored under its own name (`<hex>-<name>`, cleaned by `safe_file_name`)
and the Claude driver ends the message with `Attached file: /abs/path`,
since the CLI reads files by path and a model cannot be shown a trace. The
user-message field is renamed `images` -> `attachments` on both sides,
with a serde alias reading the rows written before. A share arrives before
anyone has said which session it is for, so it is held in AppRoot with a
banner on the list until a session takes it; an open session takes it at
once. Unreadable shares are reported beside the composer, not thrown.
The tool card crashed the app when opened on a command holding a quoted
glob such as `-path '*/.git/*'`: highlights 1.1.0's shell lexer answers
`x '*/a/*'` with a span whose end is before its start, and AnnotatedString
refuses the range. Such spans are dropped; the library is the place for
the fix. The echo driver gains `/bash <command>` so a card with a given
command can be produced on the emulator.
ui-sandbox.sh's token salvage read the tokens block's close only at a line
start, ran past the compact `),],` the server writes, and copied `setups`
into the new config twice, which the server then refused.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
`Queue::close` reports the messages a dead process never read -- they
reached no transcript, so that error is the only place they are ever
mentioned -- but it left each one drawn as a bubble waiting to be read, by
a session that no longer exists. Nothing would ever clear it: the
`UserMessage` that resolves a queued bubble is exactly what is not coming.
Seen on the emulator as a grey bubble sitting under its own error message,
still saying "tap to take it back", on a session reporting `exited`.
It now sends the `MessageDropped` the unqueue route introduced, one per
lost message, alongside the error. The error says what happened and the
drop is what ends it, which is the same division of labour as the rest of
this path.
Two things a reader could not do to what is on screen.
**Selection.** Nothing in the transcript was selectable at all, so a
command, a path or an error message could be read and not copied. One
`SelectionContainer` around the whole list rather than one per row: a
transcript is one body of text to a reader, and a selection has to be able
to run from a reply into the tool output under it. Per row it also could
not, and whatever was drawn without a container would have been silently
unselectable -- a state nothing on screen reports. Rows keep their tap
handlers; checked on the emulator that expanding a tool call, scrolling and
flinging are all unaffected, since a selection is a long press.
**Taking a message back.** A message sent into a running turn sits as a
bubble waiting to be read, and there was no way to change your mind: it is
tappable now, and the server answers `POST /sessions/{id}/unqueue`.
The answer has three states, and the middle one is the point. Claude's
driver writes a steer into the CLI's stdin the instant it arrives -- that
is what makes it reach the model at the next tool boundary rather than at
the end of the turn, and it was measured -- so the line is already gone and
`AlreadySent` is the only honest answer it can give. Holding the write
until a boundary would make the drop real and cost a steer one model call,
which is the latency the immediate write exists to remove; rejected on that
trade, with the reasoning in PLAN.md. The refusal is drawn on the bubble
that was pressed rather than in the error row under the header, a screen
away from it.
Where a driver really does hold its queue -- echo today -- the message goes
for good, and it goes as an `Event::MessageDropped` rather than as a return
value: every device watching the session loses the bubble, and a phone that
reconnects and replays the `messageQueued` does not put back one that was
cancelled with nothing left to resolve it.
A Rust backend that owns the sessions and an Android app that reads them.
The server spawns and adopts CLI processes, normalises everything they emit
into one event model, keeps the transcript, and serves it over pinned TLS on
a WireGuard interface; the phone streams that, replies, sends images, and
imports conversations the machine already has.
`AGENTS.md` is the working guide -- what runs where, what has been measured,
and the faults that were expensive to find. `PLAN.md` is the design record.
History before this point was squashed away. It was a personal project's
running commentary and carried a name and a couple of machine paths that
have no business in a public repository; the tree is what mattered and the
tree is here.
iris-ai
iris