What's new
Analysis14 Nov 20265 min read

The first token is not the finished answer

In our fictional example, Route A starts after 0.2 seconds but finishes after about 20.2, while Route B starts after a second and finishes after about nine, so which is faster depends on what the user needs.

Chat windows start printing quickly, while a document extractor is judged on when it finishes. They are different experiences and need different measurements. If your results log keeps only a tokens-per-second figure, it cannot show how long somebody waited before useful work began, or how long the whole job took.

The Claude streaming guide, read on 14 November 2026, describes incremental events, content blocks and a final message-stop event. We suggest recording several timestamps around that flow, named for what actually happened.

Which event does the user actually wait for?

Three timestamps cover most cases: request submission, first useful answer text, and successful completion after your application checks the result. The first network event can be logged too, but separately, since an acknowledgement or a non-text block is not necessarily the start of the answer the person is waiting to read.

For an extractor, the useful event may be a complete validated object. For a long explanation, readable partial text may help sooner. The job decides the event, not whichever timestamp an SDK happens to expose.

Idealised 800-token output at constant emission rates. This is arithmetic, not a measured service comparison.
Fictional routeStart delay, secondsEmission, tokens/secondApproximate finish, seconds
A0.24020.2
B11009

The table assumes a constant emission rate for 800 tokens after a start delay, so the finish estimate is delay plus tokens divided by rate. That ignores the spacing of individual tokens. Neither route is a real provider, and the example says nothing about answer quality.

Output length changes what the timings mean

Only like workloads compare fairly. A short answer can finish earlier while being emitted more slowly, and a model that writes an unnecessary essay may show impressive output speed and still keep the user waiting longer. The actual output length and whether the answer was accepted belong in the record.

A length limit goes in the record with the run. A request that stops at its limit before completing the job is a failure, and it should not win a latency comparison just because it ended early.

In tool workflows, time spent in external operations can be separated from time waiting on model responses where your instrumentation allows. The user feels the whole delay. The split helps diagnose it but should not replace the total in a product-facing comparison.

Medians need the slow and failed requests beside them

Per-request timings should be kept, and the write-up should state how the median and any upper percentile were calculated and whether failures and timeouts are included or shown separately. Dropping the slowest unfinished requests can make the surviving set look better while the service gets less useful.

Every candidate needs the same stop condition. If one application declares success when the last token arrives and another waits for validation, their finish times describe different work; saving both timestamps lets you compare generation and application completion.

The arrival pattern should be realistic too. A quiet sequential trial shows behaviour under that trial's conditions, not what happens when a queue forms. Concurrency, request sizes and the time of the observation should be recorded so the comparison can be repeated.

The right route depends on the job

For interactive prose, the start of useful text matters, along with whether partial answers stay readable. For jobs consumed by software, accepted completion and the tail of the timing distribution matter more. In both cases price and acceptance should sit beside the timing instead of being folded into an unexplained speed score.

A results log can say that one route starts sooner, another completes sooner, and the evidence comes from a specified workload. That says more than calling one model fast without saying what finished.