Steer a llama turn at its next tool boundary

A message typed into a running llama session waited for the turn to end and
then opened one of its own, so a turn spending minutes on a chain of tool
calls read nothing sent during it -- which is the one moment steering is for.
It now goes into the request the loop is about to build, prefixed with the
same note every other driver's steer carries.

The boundary being ours rather than the CLI's has two consequences worth
keeping: a waiting message can be taken back right up to the moment it is
read, and an interrupted turn deliberately takes nothing, since a request
that is not going out must not record a message as read.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

server/src/session/llama/mod.rs

@@ -46,6 +46,11 @@
 //! [`tools`] is the catalog and the running; [`mcp`] is the half of it that
 //! this backend reaches rather than the model's machine.
 //!
+//! Steering falls out of owning the loop: a message typed during a turn is
+//! handed to the model at that turn's next tool call rather than opening one
+//! of its own -- [`take_steers`], and it is why a waiting message here can be
+//! taken back right up to the boundary that reads it.
+//!
 //! **Loading is a state, not a fast bit of starting.** A multi-gigabyte model
 //! takes a while to reach memory, and for that while the server refuses
 //! everything. It is [`SessionStatus::Loading`] on screen and a message sent
@@ -290,6 +295,10 @@ impl Serves {
 /// Whether a turn is running, and the messages written during it -- each with
 /// the id of the `MessageQueued` that announced it, so the `UserMessage` can
 /// say which bubble it resolves.
+///
+/// `waiting` is drained by whichever comes first: the running turn's next tool
+/// boundary, which steers them into it, or the turn ending, which starts one
+/// with the first of them.
 #[derive(Default)]
 struct Turns {
     running: bool,
@@ -955,6 +964,7 @@ fn converse(
             let output = run_call(shared, tools, call);
             messages.push(Message::result_of(&call.id, output));
         }
+        take_steers(shared, &mut messages);
     }
     // Said rather than left as a turn that simply stopped: a reply that ends
     // here and one that ends because the model was finished look identical on
@@ -969,6 +979,49 @@ fn converse(
     Ok(())
 }
 
+/// Moves whatever was typed during this turn into the request about to be
+/// built, which is what steering a llama session is.
+///
+/// Called at a tool boundary and nowhere else, because that is the only place
+/// this driver can interject: `llama-server` is generating until it returns,
+/// and the next thing it is asked is assembled here. A message typed during a
+/// turn with no tool call left therefore still opens the next turn, exactly as
+/// it did before -- [`LlamaDriver::take_next`] is that path.
+///
+/// What differs from the Claude CLI, which writes a steer into stdin on
+/// arrival, is that the boundary is ours: nothing is handed over until one is
+/// reached, so a waiting message can still be taken back right up to it and
+/// [`Driver::unqueue`] keeps answering [`Unqueued::Dropped`].
+///
+/// The note saying this is a steer goes only into the copy the model is
+/// handed; the transcript keeps the words that were typed. So it is gone by
+/// the next turn, which rebuilds the conversation out of the transcript --
+/// which is what the note says anyway, since by then the turn it was typed
+/// into is over.
+fn take_steers(shared: &Arc<Shared>, messages: &mut Vec<Message>) {
+    // An interrupted turn takes nothing: this request is not going out, so a
+    // message taken here would be recorded as read and then never answered.
+    // Left in the queue it opens the next turn instead.
+    if shared.cancel.load(Ordering::SeqCst) {
+        return;
+    }
+    let waiting = std::mem::take(&mut shared.turns.lock().unwrap().waiting);
+    for (id, text) in waiting {
+        // The message enters the transcript here, below the calls that had
+        // not read it and above the ones that will -- the same rule the
+        // Claude driver announces a steer by.
+        shared.emit(Event::MessageTaken {
+            id: Some(id),
+            text: text.clone(),
+            attachments: Vec::new(),
+        });
+        messages.push(Message::new(
+            "user",
+            super::driver::message_body(&text, &[], true),
+        ));
+    }
+}
+
 /// One tool call: announced, asked about if it has to be, run, and reported.
 ///
 /// Always returns something for the model to read, including when it was
@@ -1075,8 +1128,10 @@ impl Driver for LlamaDriver {
                 message: "this model can't be sent attachments or files".to_string(),
             });
         }
-        // A message written during a turn waits for it, rather than starting a
-        // second conversation against the same server. Nothing was queued here
+        // A message written during a turn does not start a second
+        // conversation against the same server: it waits for the turn's next
+        // tool boundary and steers it from there (`take_steers`), or for the
+        // turn to end if there is no boundary left. Nothing was queued here
         // until tools arrived and turns grew long enough for it to matter --
         // two turns interleaving their deltas into one transcript is what that
         // looked like.
@@ -1993,6 +2048,55 @@ mod tests {
         assert_eq!(messages[1].content, "one two");
     }
 
+    #[test]
+    /// A steer taken at a tool boundary has to fold back the way `converse`
+    /// built it: the call and its result belong to the reply that asked, and
+    /// the message sits between that and the reply that read it. Getting this
+    /// wrong is not visible on screen -- the transcript renders the same
+    /// either way -- and costs the whole prompt cache on the next turn. The
+    /// steering note is not here because it never was: only the copy the model
+    /// was handed carried it.
+    fn a_steer_taken_mid_turn_folds_back_between_the_two_replies() {
+        let (_dir, path) = transcript_with(&[
+            Event::UserMessage {
+                id: None,
+                text: "read it".into(),
+                attachments: Vec::new(),
+            },
+            Event::ToolStart {
+                id: "c1".into(),
+                tool: "read_file".into(),
+                input: json!({"path": "x"}),
+            },
+            Event::ToolEnd {
+                id: "c1".into(),
+                output: "contents".into(),
+            },
+            Event::UserMessage {
+                id: Some("q1".into()),
+                text: "the other one actually".into(),
+                attachments: Vec::new(),
+            },
+            Event::AssistantText {
+                delta: "right".into(),
+            },
+        ]);
+        let messages = conversation(&path);
+        assert_eq!(
+            messages
+                .iter()
+                .map(|m| (m.role.as_str(), m.content.as_str()))
+                .collect::<Vec<_>>(),
+            [
+                ("user", "read it"),
+                ("assistant", ""),
+                ("tool", "contents"),
+                ("user", "the other one actually"),
+                ("assistant", "right"),
+            ],
+        );
+    }
+
     #[test]
     /// The interrupted case, which decides what a resumed conversation is built
     /// from: whatever the phone was shown. The deltas that arrived before the