Steer a llama turn at its next tool boundary

A message typed into a running llama session waited for the turn to end and
then opened one of its own, so a turn spending minutes on a chain of tool
calls read nothing sent during it -- which is the one moment steering is for.
It now goes into the request the loop is about to build, prefixed with the
same note every other driver's steer carries.

The boundary being ours rather than the CLI's has two consequences worth
keeping: a waiting message can be taken back right up to the moment it is
read, and an interrupted turn deliberately takes nothing, since a request
that is not going out must not record a message as read.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
iris-aiandClaude Opus 5 committed 2026-09-19 19:30:35 -04:00
1 parent 81c30dcda1
commit c3c6ab0ecf
2 files changed
+133 -2

No files matched your search

+27
View File
@@ -679,6 +679,33 @@ It is prefixed on every steer rather than only on the ones that land late,
because the two are the same message until the CLI reads it, and the note is
true either way: a steer never saw the rest of the turn it was typed into.
### A llama session steers at its own tool boundary (2026-09-19)
A message typed into a running llama session is handed to the model at that
turn's **next tool call**, in the request the driver is about to build
(`session/llama/mod.rs`'s `take_steers`), rather than waiting for the turn to
end and opening one of its own. Before this it waited: a turn that spent two
minutes on a chain of tool calls read nothing sent during it, which is the
one moment steering is for.
This is the same landing place as the Claude CLI's, reached the other way
round. Claude's steer is written into stdin on arrival and *the CLI* decides
it lands at the next model call; here the loop is ours, so nothing is handed
over until the boundary is reached. Two things follow that the CLI cannot
offer: a waiting message can still be taken back right up to the moment it is
read, so `Driver::unqueue` keeps answering `Dropped` rather than
`AlreadySent`; and an interrupted turn deliberately takes nothing, leaving
the message in the queue to open the next turn, because a request that is not
going out must not record a message as read.
A turn with no tool call left still has no boundary to interject at -- the
server is generating until it returns -- so such a message opens the next
turn as it always did. The `STEERING_NOTE` is on the model's copy only, so a
steer is folded back out of the transcript as the words that were typed, and
`a_steer_taken_mid_turn_folds_back_between_the_two_replies` is what keeps the
fold matching what the turn built: the prompt cache depends on the next turn
rendering this one byte for byte.
### Session processes outlive the backend (2026-08-29)
A session's process is **left running when the backend stops and adopted