Draw a model's thinking, and what a reply cost to produce

A llama.cpp session's `reasoning_content` becomes `Event::Thinking` deltas
closed by an `Event::ThinkingDone` carrying the span the driver measured, and
the phone draws it as a card of its own: "Thinking" with the spinner a running
command has, then "Thought for 12.4s". Deliberately not a tool call, so a run
of calls cannot collapse the reasoning into "Called 6 tools"; the reasoning is
also kept out of the next prompt, which `conversation` already ignored.

`UsageDelta` gains `tokensPerSecond`, the provider's own figure or nothing --
llama.cpp reports `timings.predicted_per_second` and the coding CLIs report no
such thing -- and a finished reply carries a small line under it saying when it
was sent and, where there is one, how fast it came out: "3:00 PM · 149 tok/s".

The compact usage bar drops the provider's name for the window and puts its
length after the time left instead: "42% · 3h 20m left / 5h".

Three things that had to come with it: the transcript coalesces runs of
thinking deltas as it does reply deltas, so one block is one row of a page
rather than a page of its own; `joinPages` welds a block cut by a page boundary
(`healSplitThinking`), since the half with no ending spun for ever; and
`UsageDelta` now reaches the fold, which is what carries the rate to the reply.

Verified on the emulator against a real Qwen3-0.6B session and the echo rig's
new `/think [seconds]`: the spinner while it runs, "Thought for 1.4s" and
"2:54 PM · 149 tok/s" after, the reasoning on tapping the card, and the usage
bar reading "42% · 3h 19m left / 5h".

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
iris-aiandClaude Opus 5 committed 2026-09-19 15:07:25 -04:00
1 parent 45f249ae91
commit bb5ac1a242
18 files changed
+900 -101

No files matched your search

+74 -4
View File
@@ -1562,9 +1562,12 @@ fn context_window(endpoint: &str) -> Option<u64> {
/// what happens to them next -- asking, running, reporting -- is the caller's,
/// and a call is not in the transcript until it has actually been made.
///
/// `reasoning_content` is dropped, which is what the Claude driver does with
/// thinking deltas. A transcript is what was said, and this app does not draw
/// a model's working.
/// `reasoning_content` is a model's working, and it is emitted as
/// [`Event::Thinking`] rather than mixed into the reply -- its own card, and
/// deliberately not part of what the next prompt is built from (see
/// [`conversation`], which ignores it). The block is closed with
/// [`Event::ThinkingDone`] the moment the model says something else, which is
/// how long it thought for.
fn generate(
endpoint: &str,
messages: &[Message],
@@ -1619,6 +1622,24 @@ fn generate(
// worst direction for a figure somebody is watching to see how much room
// is left.
let mut context = None;
// The open thinking block: when it started, so its duration is measured
// where the deltas actually arrive rather than worked out later from two
// timestamps. `None` between blocks -- a turn can think, speak, call a
// tool and think again.
let mut thinking: Option<std::time::Instant> = None;
// What the server says its own generation ran at. Read from it rather than
// divided out of the wall time here, which would count the request, the
// prompt processing and this loop's own scheduling as generation.
let mut per_second = None;
// Closes the open block, which is anything the model says that is not more
// working: the first word of the reply, or a tool call.
let done_thinking = |thinking: &mut Option<std::time::Instant>| {
if let Some(started) = thinking.take() {
shared.emit(Event::ThinkingDone {
ms: started.elapsed().as_millis() as u64,
});
}
};
for line in std::io::BufRead::lines(reader) {
if shared.cancel.load(Ordering::SeqCst) {
break;
@@ -1642,12 +1663,27 @@ fn generate(
tokens = total;
context = Some(total);
}
if let Some(rate) = chunk
.pointer("/timings/predicted_per_second")
.and_then(Value::as_f64)
{
per_second = Some(rate);
}
let Some(delta) = chunk.pointer("/choices/0/delta") else {
continue;
};
if let Some(fragment) = delta.get("reasoning_content").and_then(Value::as_str)
&& !fragment.is_empty()
{
thinking.get_or_insert_with(std::time::Instant::now);
shared.emit(Event::Thinking {
delta: fragment.to_string(),
});
}
if let Some(fragment) = delta.get("content").and_then(Value::as_str)
&& !fragment.is_empty()
{
done_thinking(&mut thinking);
text.push_str(fragment);
shared.emit(Event::AssistantText {
delta: fragment.to_string(),
@@ -1659,11 +1695,20 @@ fn generate(
.into_iter()
.flatten()
{
done_thinking(&mut thinking);
absorb(&mut calls, fragment);
}
}
// A block left open by the end of the stream -- a model that thought and
// then said nothing, or a turn the reader cancelled -- is still a block
// that ended. Without this its card spins for ever.
done_thinking(&mut thinking);
if tokens > 0 {
shared.emit(Event::UsageDelta { tokens, context });
shared.emit(Event::UsageDelta {
tokens,
context,
tokens_per_second: per_second,
});
}
// A call whose name never arrived is not a call. It happens when a stream
// is cut mid-fragment, and running it would mean inventing what was asked
@@ -1815,6 +1860,30 @@ mod tests {
);
}
#[test]
/// A model's working is drawn and is deliberately not sent back to it: the
/// prompt is what was said, and feeding reasoning back costs the whole of
/// it in context for a model that never asked to see it again.
fn thinking_is_not_part_of_the_conversation() {
let (_dir, path) = transcript_with(&[
Event::UserMessage {
id: None,
text: "count".into(),
attachments: Vec::new(),
},
Event::Thinking {
delta: "the user wants".into(),
},
Event::ThinkingDone { ms: 1200 },
Event::AssistantText {
delta: "one two".into(),
},
]);
let messages = conversation(&path);
assert_eq!(messages.len(), 2);
assert_eq!(messages[1].content, "one two");
}
#[test]
/// The interrupted case, which decides what a resumed conversation is built
/// from: whatever the phone was shown. The deltas that arrived before the
@@ -1862,6 +1931,7 @@ mod tests {
Event::UsageDelta {
tokens: 12,
context: Some(12),
tokens_per_second: None,
},
]);
let messages = conversation(&path);