Draw a model's thinking, and what a reply cost to produce
A llama.cpp session's `reasoning_content` becomes `Event::Thinking` deltas closed by an `Event::ThinkingDone` carrying the span the driver measured, and the phone draws it as a card of its own: "Thinking" with the spinner a running command has, then "Thought for 12.4s". Deliberately not a tool call, so a run of calls cannot collapse the reasoning into "Called 6 tools"; the reasoning is also kept out of the next prompt, which `conversation` already ignored. `UsageDelta` gains `tokensPerSecond`, the provider's own figure or nothing -- llama.cpp reports `timings.predicted_per_second` and the coding CLIs report no such thing -- and a finished reply carries a small line under it saying when it was sent and, where there is one, how fast it came out: "3:00 PM · 149 tok/s". The compact usage bar drops the provider's name for the window and puts its length after the time left instead: "42% · 3h 20m left / 5h". Three things that had to come with it: the transcript coalesces runs of thinking deltas as it does reply deltas, so one block is one row of a page rather than a page of its own; `joinPages` welds a block cut by a page boundary (`healSplitThinking`), since the half with no ending spun for ever; and `UsageDelta` now reaches the fold, which is what carries the rate to the reply. Verified on the emulator against a real Qwen3-0.6B session and the echo rig's new `/think [seconds]`: the spinner while it runs, "Thought for 1.4s" and "2:54 PM · 149 tok/s" after, the reasoning on tapping the card, and the usage bar reading "42% · 3h 19m left / 5h". Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
1 parent
45f249ae91
commit
bb5ac1a242
18 files changed
+900
-101
No files matched your search
@@ -212,6 +212,34 @@ pub enum Event {
|
||||
AssistantTextFinal {
|
||||
text: String,
|
||||
},
|
||||
/// The model's working, streamed the same way its reply is: the
|
||||
/// reasoning it produced before -- or between -- the things it said.
|
||||
///
|
||||
/// Its own kind rather than [`Event::AssistantText`], because it is not
|
||||
/// what the session said. The phone draws it as a card of its own, shut,
|
||||
/// and no driver folds it back into the next prompt: a provider that
|
||||
/// wants its own reasoning back sends it back itself.
|
||||
///
|
||||
/// Only a provider that actually streams its working sends this.
|
||||
/// llama.cpp does, as `reasoning_content`; nothing is inferred for one
|
||||
/// that does not, since a card that appeared whenever a turn was slow
|
||||
/// would be a guess wearing a measurement's clothes.
|
||||
Thinking {
|
||||
delta: String,
|
||||
},
|
||||
/// The thinking immediately above this finished, having taken `ms`.
|
||||
///
|
||||
/// Measured by the driver rather than worked out by a reader from two
|
||||
/// event timestamps. A reader only knows when an event *arrived*, so the
|
||||
/// last delta of a block followed by a slow tool call is indistinguishable
|
||||
/// from thinking that went on that long -- and a transcript replayed on a
|
||||
/// phone has to reach the same figure the live stream showed.
|
||||
///
|
||||
/// A block with no end is one still being thought, which is what the card
|
||||
/// draws a spinner for.
|
||||
ThinkingDone {
|
||||
ms: u64,
|
||||
},
|
||||
ToolStart {
|
||||
id: String,
|
||||
tool: String,
|
||||
@@ -399,6 +427,17 @@ pub enum Event {
|
||||
/// able to draw.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
context: Option<u64>,
|
||||
/// How fast the reply came out, as the provider measured it.
|
||||
///
|
||||
/// `None` wherever nothing measured it, which is most providers: a
|
||||
/// coding CLI reports what a turn cost and never how long the model
|
||||
/// took over it, and dividing tokens by the wall time this server
|
||||
/// waited would count the network, the tool calls and the reader's own
|
||||
/// permission answers as generation. A figure that is right most of
|
||||
/// the time is no use here, because nothing on screen could say which
|
||||
/// times those were.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
tokens_per_second: Option<f64>,
|
||||
},
|
||||
/// A compaction that finished, and how much context it recovered.
|
||||
///
|
||||
@@ -850,6 +889,7 @@ mod tests {
|
||||
Event::UsageDelta {
|
||||
tokens: 12,
|
||||
context: Some(30_100),
|
||||
tokens_per_second: None,
|
||||
}
|
||||
),
|
||||
Some(30_100)
|
||||
@@ -889,6 +929,7 @@ mod tests {
|
||||
Event::UsageDelta {
|
||||
tokens: 12,
|
||||
context: None,
|
||||
tokens_per_second: None,
|
||||
}
|
||||
),
|
||||
Some(30_100)
|
||||
|
||||
Reference in new issue
Block a user