Draw a model's thinking, and what a reply cost to produce
A llama.cpp session's `reasoning_content` becomes `Event::Thinking` deltas closed by an `Event::ThinkingDone` carrying the span the driver measured, and the phone draws it as a card of its own: "Thinking" with the spinner a running command has, then "Thought for 12.4s". Deliberately not a tool call, so a run of calls cannot collapse the reasoning into "Called 6 tools"; the reasoning is also kept out of the next prompt, which `conversation` already ignored. `UsageDelta` gains `tokensPerSecond`, the provider's own figure or nothing -- llama.cpp reports `timings.predicted_per_second` and the coding CLIs report no such thing -- and a finished reply carries a small line under it saying when it was sent and, where there is one, how fast it came out: "3:00 PM · 149 tok/s". The compact usage bar drops the provider's name for the window and puts its length after the time left instead: "42% · 3h 20m left / 5h". Three things that had to come with it: the transcript coalesces runs of thinking deltas as it does reply deltas, so one block is one row of a page rather than a page of its own; `joinPages` welds a block cut by a page boundary (`healSplitThinking`), since the half with no ending spun for ever; and `UsageDelta` now reaches the fold, which is what carries the rate to the reply. Verified on the emulator against a real Qwen3-0.6B session and the echo rig's new `/think [seconds]`: the spinner while it runs, "Thought for 1.4s" and "2:54 PM · 149 tok/s" after, the reasoning on tapping the card, and the usage bar reading "42% · 3h 19m left / 5h". Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
1 parent
45f249ae91
commit
bb5ac1a242
18 files changed
+900
-101
No files matched your search
@@ -64,6 +64,21 @@ sealed class SessionEvent {
|
||||
/** The durable value of the open assistant message, replacing its provisional deltas. */
|
||||
data class AssistantTextFinal(val text: String) : SessionEvent()
|
||||
|
||||
/**
|
||||
* The model's working, streamed the way its reply is: its own card, and deliberately not part
|
||||
* of what the session said. Only a provider that actually streams its reasoning sends it.
|
||||
*/
|
||||
data class Thinking(val delta: String) : SessionEvent()
|
||||
|
||||
/**
|
||||
* The thinking above this finished, having taken [ms].
|
||||
*
|
||||
* Measured by the driver, because only it can see when the model stopped: this app knows when
|
||||
* an event *arrived*, and the last fragment of a block followed by a slow tool call looks
|
||||
* exactly like thinking that went on that long.
|
||||
*/
|
||||
data class ThinkingDone(val ms: Long) : SessionEvent()
|
||||
|
||||
data class ToolStart(val id: String, val tool: String, val input: String) : SessionEvent()
|
||||
|
||||
data class ToolUpdate(val id: String, val output: String) : SessionEvent()
|
||||
@@ -161,7 +176,16 @@ sealed class SessionEvent {
|
||||
* so adding turns up would report a figure the session stopped being true of. Null where the
|
||||
* dialect did not say, which leaves the context unmeasured rather than unchanged.
|
||||
*/
|
||||
data class UsageDelta(val tokens: Long, val context: Long?) : SessionEvent()
|
||||
data class UsageDelta(
|
||||
val tokens: Long,
|
||||
val context: Long?,
|
||||
/**
|
||||
* How fast the reply came out, where the provider measured it -- null everywhere else,
|
||||
* which is most of them. Never worked out here: the time this app watched a reply arrive
|
||||
* over includes the network and whatever the server was doing between tokens.
|
||||
*/
|
||||
val tokensPerSecond: Double? = null,
|
||||
) : SessionEvent()
|
||||
|
||||
/** How much context this session's model has, which is what [UsageDelta.context] is out of. */
|
||||
data class ContextWindow(val tokens: Long) : SessionEvent()
|
||||
@@ -237,6 +261,8 @@ fun parseSeqEvent(json: String): SeqEvent {
|
||||
"messageDropped" -> SessionEvent.MessageDropped(body.getString("id"))
|
||||
"assistantText" -> SessionEvent.AssistantText(body.getString("delta"))
|
||||
"assistantTextFinal" -> SessionEvent.AssistantTextFinal(body.getString("text"))
|
||||
"thinking" -> SessionEvent.Thinking(body.getString("delta"))
|
||||
"thinkingDone" -> SessionEvent.ThinkingDone(body.getLong("ms"))
|
||||
"toolStart" ->
|
||||
SessionEvent.ToolStart(
|
||||
id = body.getString("id"),
|
||||
@@ -301,6 +327,7 @@ fun parseSeqEvent(json: String): SeqEvent {
|
||||
SessionEvent.UsageDelta(
|
||||
body.getLong("tokens"),
|
||||
if (body.has("context")) body.getLong("context") else null,
|
||||
if (body.has("tokensPerSecond")) body.getDouble("tokensPerSecond") else null,
|
||||
)
|
||||
"compacted" ->
|
||||
SessionEvent.Compacted(
|
||||
|
||||
Reference in new issue
Block a user