Draw a model's thinking, and what a reply cost to produce
A llama.cpp session's `reasoning_content` becomes `Event::Thinking` deltas closed by an `Event::ThinkingDone` carrying the span the driver measured, and the phone draws it as a card of its own: "Thinking" with the spinner a running command has, then "Thought for 12.4s". Deliberately not a tool call, so a run of calls cannot collapse the reasoning into "Called 6 tools"; the reasoning is also kept out of the next prompt, which `conversation` already ignored. `UsageDelta` gains `tokensPerSecond`, the provider's own figure or nothing -- llama.cpp reports `timings.predicted_per_second` and the coding CLIs report no such thing -- and a finished reply carries a small line under it saying when it was sent and, where there is one, how fast it came out: "3:00 PM · 149 tok/s". The compact usage bar drops the provider's name for the window and puts its length after the time left instead: "42% · 3h 20m left / 5h". Three things that had to come with it: the transcript coalesces runs of thinking deltas as it does reply deltas, so one block is one row of a page rather than a page of its own; `joinPages` welds a block cut by a page boundary (`healSplitThinking`), since the half with no ending spun for ever; and `UsageDelta` now reaches the fold, which is what carries the rate to the reply. Verified on the emulator against a real Qwen3-0.6B session and the echo rig's new `/think [seconds]`: the spinner while it runs, "Thought for 1.4s" and "2:54 PM · 149 tok/s" after, the reasoning on tapping the card, and the usage bar reading "42% · 3h 19m left / 5h". Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
1 parent
45f249ae91
commit
bb5ac1a242
18 files changed
+900
-101
No files matched your search
@@ -666,6 +666,12 @@ impl EchoDriver {
|
||||
let table = text
|
||||
.strip_prefix("/table")
|
||||
.map(|rest| rest.trim().parse::<usize>().unwrap_or(6).clamp(1, 12));
|
||||
// Seconds to spend thinking before the reply, default three. The rig
|
||||
// for the thinking card: a block that runs long enough to watch the
|
||||
// spinner, then ends with a duration to read.
|
||||
let think = text
|
||||
.strip_prefix("/think")
|
||||
.map(|rest| Duration::from_secs(rest.trim().parse::<u64>().unwrap_or(3).clamp(1, 600)));
|
||||
let linger = text.strip_prefix("/slow").map(|rest| {
|
||||
Duration::from_secs(rest.trim().parse::<u64>().unwrap_or(30).clamp(1, 600))
|
||||
});
|
||||
@@ -849,10 +855,31 @@ impl EchoDriver {
|
||||
});
|
||||
}
|
||||
|
||||
if let Some(think) = think {
|
||||
let started = std::time::Instant::now();
|
||||
for remaining in (1..=think.as_secs()).rev() {
|
||||
send(Event::Thinking {
|
||||
delta: format!(
|
||||
"Considering what to echo back, {remaining}s of it left. \
|
||||
The reply is the message, which took some working out.\n\n"
|
||||
),
|
||||
});
|
||||
tokio::time::sleep(Duration::from_secs(1)).await;
|
||||
}
|
||||
// Measured here for the same reason a driver measures it: the
|
||||
// phone can only see when an event arrived.
|
||||
send(Event::ThinkingDone {
|
||||
ms: started.elapsed().as_millis() as u64,
|
||||
});
|
||||
}
|
||||
|
||||
let streaming = std::time::Instant::now();
|
||||
let mut words = 0u64;
|
||||
for word in format!("You said: {text}").split_inclusive(' ') {
|
||||
send(Event::AssistantText {
|
||||
delta: word.to_string(),
|
||||
});
|
||||
words += 1;
|
||||
tokio::time::sleep(DELTA_DELAY).await;
|
||||
}
|
||||
// A conversation gets bigger, so the pretend context does too:
|
||||
@@ -861,6 +888,12 @@ impl EchoDriver {
|
||||
send(Event::UsageDelta {
|
||||
tokens: spent,
|
||||
context: Some(context.fetch_add(spent + 100, Ordering::SeqCst) + spent + 100),
|
||||
// A real measurement of a pretend model: what this driver
|
||||
// emitted, over how long it took. A rig owes the app a figure
|
||||
// of the shape a real one has, not an invented value.
|
||||
tokens_per_second: Some(streaming.elapsed().as_secs_f64())
|
||||
.filter(|elapsed| *elapsed > 0.0)
|
||||
.map(|elapsed| words as f64 / elapsed),
|
||||
});
|
||||
finish();
|
||||
});
|
||||
|
||||
Reference in new issue
Block a user