A turn waits on three things that look nowhere at all: a permission
question, a tool call with a minute to run, and the completion itself,
which says nothing while the prompt is read -- tens of seconds on a long
conversation. Setting a flag left the turn exactly where it was until
whichever it was came back.
The wait is now what ends, not the work. Each of those runs on a thread of
its own and the interrupt answers the wait; the abandoned thread finishes
into a channel nobody is reading. 43ms to end a turn in every state,
measured against a real model -- including mid prompt-processing, which
used to be a twenty-second wait. That makes cancellation a token per turn
rather than a flag on the session: the abandoned thread wakes up some time
later, and a flag the next turn had reset would let it write into a
conversation it is no longer part of. The open thinking block moves to
Shared for the same reason -- the thread that knows one is open is no
longer the thread that ends the turn.
Stop, meanwhile, did nothing at all to a llama session: it signals the
session's recorded process and process::stop refuses a Shared one, which
is the whole point of that record -- so the session sat at idle. A Shared
record routes to the driver now, because what stopping means for a session
that borrows the machine's process is the driver's to say. It ends the
turn, says exited itself, and gives up its claim on the model; each live
session claims the model it is on, and the model is unloaded when the last
claim goes. A model another session is using stays where it is.
iris-ai
386c1c4def