Steer a llama turn at its next tool boundary
A message typed into a running llama session waited for the turn to end and then opened one of its own, so a turn spending minutes on a chain of tool calls read nothing sent during it -- which is the one moment steering is for. It now goes into the request the loop is about to build, prefixed with the same note every other driver's steer carries. The boundary being ours rather than the CLI's has two consequences worth keeping: a waiting message can be taken back right up to the moment it is read, and an interrupted turn deliberately takes nothing, since a request that is not going out must not record a message as read. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
1 parent
81c30dcda1
commit
c3c6ab0ecf
2 files changed
+133
-2
No files matched your search
@@ -679,6 +679,33 @@ It is prefixed on every steer rather than only on the ones that land late,
|
||||
because the two are the same message until the CLI reads it, and the note is
|
||||
true either way: a steer never saw the rest of the turn it was typed into.
|
||||
|
||||
### A llama session steers at its own tool boundary (2026-09-19)
|
||||
|
||||
A message typed into a running llama session is handed to the model at that
|
||||
turn's **next tool call**, in the request the driver is about to build
|
||||
(`session/llama/mod.rs`'s `take_steers`), rather than waiting for the turn to
|
||||
end and opening one of its own. Before this it waited: a turn that spent two
|
||||
minutes on a chain of tool calls read nothing sent during it, which is the
|
||||
one moment steering is for.
|
||||
|
||||
This is the same landing place as the Claude CLI's, reached the other way
|
||||
round. Claude's steer is written into stdin on arrival and *the CLI* decides
|
||||
it lands at the next model call; here the loop is ours, so nothing is handed
|
||||
over until the boundary is reached. Two things follow that the CLI cannot
|
||||
offer: a waiting message can still be taken back right up to the moment it is
|
||||
read, so `Driver::unqueue` keeps answering `Dropped` rather than
|
||||
`AlreadySent`; and an interrupted turn deliberately takes nothing, leaving
|
||||
the message in the queue to open the next turn, because a request that is not
|
||||
going out must not record a message as read.
|
||||
|
||||
A turn with no tool call left still has no boundary to interject at -- the
|
||||
server is generating until it returns -- so such a message opens the next
|
||||
turn as it always did. The `STEERING_NOTE` is on the model's copy only, so a
|
||||
steer is folded back out of the transcript as the words that were typed, and
|
||||
`a_steer_taken_mid_turn_folds_back_between_the_two_replies` is what keeps the
|
||||
fold matching what the turn built: the prompt cache depends on the next turn
|
||||
rendering this one byte for byte.
|
||||
|
||||
### Session processes outlive the backend (2026-08-29)
|
||||
|
||||
A session's process is **left running when the backend stops and adopted
|
||||
|
||||
Reference in new issue
Block a user