Report what a reply spent reading its prompt, and pin the clock right
`UsageDelta` gains `prefillMs`, llama-server's own `timings.prompt_ms`, so the footer under a finished reply is "read 9.5s · 50.3 tok/s · 3:00 PM". Prefill is the half of a turn that was invisible and is often the larger: measured on the 0.6B here, 1m 4s for the first turn after a model loads against 22ms for the next, whose prompt the server still had cached. The clock moves to the end of the line. Everything in front of it is a provider's own measurement, so a session on another provider has fewer of them or none, and a reader who has learned where the time is should not have to find it again because the model changed. The costs grow leftwards into the space instead, and a test asserts every shape of the line ends with the same thing. Verified on the emulator against a real llama session: three replies reading "read 1m 4s · 193 tok/s · 3:54 PM", "read 25ms · 308 tok/s · 3:54 PM" and "read 22ms · 194 tok/s · 3:54 PM", with the clock in one column. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
1 parent
b660905098
commit
369b8f7e52
14 files changed
+135
-42
No files matched your search
@@ -169,13 +169,17 @@ seq N", so there is no separate history path to drift from the live one.
|
||||
waiting for itself and will speak again with nobody having typed anything.
|
||||
Reporting it as idle sent a "finished" notification at the one moment that
|
||||
was untrue.
|
||||
- `UsageDelta { tokens, context, tokensPerSecond }` — what a turn cost, how
|
||||
much the model was holding when it ended, and how fast it was generated.
|
||||
`tokensPerSecond` (2026-09-19) is the provider's own measurement or nothing:
|
||||
llama.cpp reports `timings.predicted_per_second`, and the coding CLIs report
|
||||
no such figure, so dividing what this server watched a reply arrive over
|
||||
would count the network, the tool calls and the reader's own permission
|
||||
answers as generation. The phone draws it under the reply it measured. `context` is prompt plus both cache figures,
|
||||
- `UsageDelta { tokens, context, tokensPerSecond, prefillMs }` — what a turn
|
||||
cost, how much the model was holding when it ended, how fast it was
|
||||
generated, and how long the provider spent reading the prompt first. The last
|
||||
two (2026-09-19) are the provider's own measurements or nothing: llama.cpp
|
||||
reports `timings.predicted_per_second` and `timings.prompt_ms`, and the
|
||||
coding CLIs report neither, so dividing what this server watched a reply
|
||||
arrive over would count the network, the tool calls and the reader's own
|
||||
permission answers as generation. The phone draws both under the reply they
|
||||
measured. `prefillMs` is small on a turn whose prompt the server still had
|
||||
cached -- 22ms against 64s for the first turn after a model loads, measured
|
||||
on the 0.6B -- which is a fact about the turn rather than a missing figure. `context` is prompt plus both cache figures,
|
||||
taken from the **last assistant message** rather than the turn's `result`:
|
||||
measured 2026-08-30 against CLI 2.1.237, the result adds a turn's messages
|
||||
up, so its cache read of 40,211 was the same conversation counted twice.
|
||||
@@ -1275,14 +1279,17 @@ dev-updater (Kotlin 2.4.x, CMP 1.11.x, JDK 21).
|
||||
the model stopped to think in the middle of it. A block cut by a page
|
||||
boundary is welded like a reply is (`healSplitThinking`), since the half
|
||||
with no ending would otherwise spin for the rest of the conversation.
|
||||
- **A finished reply carries a line under it saying when it was sent and,
|
||||
where the provider measured one, how fast it was generated** (2026-09-19)
|
||||
— "3:00 PM · 149 tok/s", small and set back, right-aligned because it
|
||||
closes the message rather than opening one. The time is the transcript's
|
||||
own timestamp, so every device draws the same one; the rate is the
|
||||
provider's own figure or nothing at all. It is a list unit of its own
|
||||
(`ReplyFoot`), because a settled reply *is* its blocks and there is no
|
||||
row left to hang it on.
|
||||
- **A finished reply carries a line under it saying what it cost to produce
|
||||
and when it was sent** (2026-09-19) — "read 9.5s · 50.3 tok/s · 3:00 PM",
|
||||
small and set back, right-aligned because it closes the message rather
|
||||
than opening one. **The clock is last**, so it sits against the right edge
|
||||
whatever else is on the line: the measurements in front of it belong to
|
||||
the provider, and a reader who has learned where the time is should not
|
||||
have to find it again because the session is on a different one. The time
|
||||
is the transcript's own timestamp, so every device draws the same one; the
|
||||
costs are the provider's own figures or nothing at all. It is a list unit
|
||||
of its own (`ReplyFoot`), because a settled reply *is* its blocks and
|
||||
there is no row left to hang it on.
|
||||
- **Anything that is a note *about* the conversation rather than a turn in
|
||||
it is closed by default** — a tool call, a peer message, a memory note,
|
||||
a thinking block.
|
||||
|
||||
Reference in new issue
Block a user