These examples use stated assumptions. They are not model measurements. Each entry names its frozen data and an offline script; run the script with --check to reproduce it. Documentation is linked and briefly paraphrased, not collected as a dataset. No additional data licence is claimed.
Retrieval coverage = 80 / 100 = 80%. Accepted answers overall = 68 / 100 = 68%. Accepted answers among retrieved cases = 68 / 80 = 85%. Counts are nested assumptions.
Assumptions: 100 fictional answerable questions: evidence is retrieved for 80 and an accepted answer is produced for 68 of those. No system was evaluated.
Limits: This defines evidence availability for one question set, not document recall across a whole corpus. A retrieved passage can still be insufficient or misread.
- Anthropic: Contextual Retrieval, read 14 November 2026: The described retrieval workflow chunks documents, retrieves relevant passages and adds them to the generation prompt. Its hybrid example combines lexical and embedding retrieval.
Code: scripts/insights/practitioner-b4.mjs. Frozen tables: src/data/insights/2026-11-14-b4.json.
For document length L, chunk capacity C and overlap O, greedy full coverage uses 1 + ceil(max(0,L − C)/(C − O)) chunks. Stored tokens = L + (chunks − 1) × O. Example L=10,000, C=1,000, O=0,200,500.
Assumptions: One synthetic 10,000-token document is split into chunks of at most 1,000 tokens. Each adjacent pair overlaps by a fixed amount; the final chunk may be shorter.
Limits: Token counts are arithmetic, not tokenizer measurements. This ignores metadata and vector storage and does not demonstrate a retrieval-quality benefit.
- Anthropic: Contextual Retrieval, read 14 November 2026: The described retrieval workflow chunks documents, retrieves relevant passages and adds them to the generation prompt. Its hybrid example combines lexical and embedding retrieval.
Code: scripts/insights/practitioner-b4.mjs. Frozen tables: src/data/insights/2026-11-14-b4.json.
On 50 fictional queries: both retrieve evidence=39, old-only=6, new-only=3, neither=2. Old coverage=(39+6)/50=90%; new=(39+3)/50=84%. Migration text volume=100,000 chunks × 600 assumed tokens=60,000,000 tokens.
Assumptions: Two invented retrieval configurations share 50 questions. A separate planning example has 100,000 chunks averaging exactly 600 tokens. No embeddings were generated.
Limits: Equal vector dimensions do not establish model compatibility. The example supplies no real accuracy, rebuild duration, storage forecast or embedding price.
- Google: embedding task types, read 14 November 2026: Retrieval queries and retrieval documents have distinct task types. Embedding configuration is part of the retrieval setup.
Code: scripts/insights/practitioner-b4.mjs. Frozen tables: src/data/insights/2026-11-14-b4.json.
Idealised completion seconds = start delay + 800 output tokens / constant emission rate. A: 0.2+800/40=20.2 seconds; B: 1+800/100=9 seconds. This is a fluid approximation, not an observed stream.
Assumptions: Two fictional routes emit 800 tokens at constant rates after assumed start delays. No queue variation, thinking, tools, network jitter or rendering delay.
Limits: Token emission is not truly continuous. Real streams have varied block types and gaps; a first event need not contain useful answer text. No current speed ranking is made.
- Claude streaming messages, read 14 November 2026: Messages can stream incrementally as server-sent events. The event flow includes content blocks and a final message_stop; receiving initial content is not the same as a complete message.
Code: scripts/insights/practitioner-b5.mjs. Frozen tables: src/data/insights/2026-11-14-b5.json.
Illustrative steady throughput ceiling = min(RPM, input-token limit / input per request, output-token limit / output per request). With limits 60,120,000,30,000: workloads (1,000,200), (4,000,1,000), (4,000,2,000) give 60,30,15 requests/minute.
Assumptions: Hypothetical limits of 60 requests, 120,000 input tokens and 30,000 output tokens per minute. Fixed per-request sizes, no caching or bursts.
Limits: A ceiling is not a delivered throughput guarantee. Fractional capacity, shared limits, estimation rules, burst controls and other workloads can reduce useful throughput.
- Claude API rate limits, read 14 November 2026: Messages API rate limits include requests, input tokens and output tokens per minute. Short bursts can hit limits; published maxima are not guaranteed minimum throughput.
Code: scripts/insights/practitioner-b5.mjs. Frozen tables: src/data/insights/2026-11-14-b5.json.
Illustrative attempts = 100+20+5=125, costing 125×$0.01=$1.25. Accepted jobs =80+15+2=97. A maximum of three attempts per 100 jobs reserves up to 300×$0.01=$3 under the fixed-price assumption.
Assumptions: Fictional job ledger: 100 first attempts yield 80 accepted; 20 second attempts yield 15 more; five third attempts yield two more. Each attempt costs exactly $0.01, even when unsuccessful.
Limits: Real failure probabilities and bills differ. No independence or success forecast is assumed. Three attempts means the first plus at most two retries; transport failure does not by itself prove whether a side effect happened.
- Claude API errors, read 14 November 2026: The API distinguishes validation, rate-limit, internal and overload errors. Its guidance recommends exponential backoff for internal API errors; error type and context matter when choosing recovery.
Code: scripts/insights/practitioner-b5.mjs. Frozen tables: src/data/insights/2026-11-14-b5.json.
Assumed maximum model-call charge = (4,000×$1 + 1,000×$5)/1,000,000=$0.009. For caps of 5,10,20 calls, nominal reserves are $0.045,$0.090,$0.180. All billable output must fit the 1,000-token cap.
Assumptions: Hypothetical flat rates and enforceable per-call maxima of 4,000 input and 1,000 billable output tokens. Calls are serial, with no retries outside the ledger.
Limits: This is a model-call bound under fixed assumptions, not a whole-agent spending guarantee. Tools, storage, price tiers, hidden retries and unaccounted parallel calls require separate limits.
- Claude tool-use overview, read 14 November 2026: For client tools, the model emits structured tool calls and the application executes operations and returns results. The application therefore controls that execution boundary.
Code: scripts/insights/practitioner-b6.mjs. Frozen tables: src/data/insights/2026-11-14-b6.json.
Synthetic tool-result allowance =32,000 total −4,000 fixed input −8,000 reserved output=20,000 tokens. Ten results of 500,2,000,8,000 tokens occupy 5,000,20,000,80,000 tokens respectively.
Assumptions: One hypothetical 32,000-token request budget with fixed input and output reserves. Ten tool results are included once each, with no other history or overhead.
Limits: This is a text-capacity calculation, not a measured tokenizer result or a model specification. Repeated history and tool schemas need additional accounting. Shortening a result can remove needed evidence.
- Claude: handle tool calls, read 14 November 2026: Client tool results are returned to the model in tool_result blocks associated with tool calls. The returned content is part of the material the application supplies.
Code: scripts/insights/practitioner-b6.mjs. Frozen tables: src/data/insights/2026-11-14-b6.json.
Fictional screening flags 18/20 malicious fixtures=90%; two reach the model. Of two prohibited action proposals, an independent executor denies 2/2. Executed prohibited actions=0 in this constructed example.
Assumptions: Twenty invented malicious tool-result fixtures, eighteen detected, two passed through, two prohibited proposals and two executor denials. No attack or security assessment was run.
Limits: A constructed zero is not evidence of safety or a real attack success rate. Other permissions, data exposure and benign-task failures require separate tests. No defence is claimed complete.
- Claude: mitigate prompt injections, read 14 November 2026: Indirect injections can arrive through third-party documents and tool results. The guidance recommends separating untrusted content and applying least privilege, including narrowly scoped permissions and sandboxing.
Code: scripts/insights/practitioner-b6.mjs. Frozen tables: src/data/insights/2026-11-14-b6.json.
Synthetic audit: 100 claims, 95 valid pointers, 80 supported claims among those. Pointer validity=95%; support overall=80%; support among valid pointers=80/95×100≈84.2%. Valid but unsupported=95−80=15.
Assumptions: 100 invented claim-citation pairs, one citation per claim. Ninety-five point to the intended source; eighty of those support the complete claim. No real citation system was tested.
Limits: Valid pointers do not establish source truth, currentness or full claim support. Multiple-source claims need a separate completeness rule. This is not a provider citation-accuracy result.
- Claude citations, read 14 November 2026: The API returns source-location pointers and cited text. PDF citations use page locations; plain-text citations use character locations. These identify source material for verification.
Code: scripts/insights/practitioner-b7.mjs. Frozen tables: src/data/insights/2026-11-14-b7.json.
100 synthetic documents ×10 fields=1,000 fields. Twenty errors give 98% field accuracy in both scenarios. Concentrated in two documents:98/100 clean=98%. Spread over twenty documents:80/100 clean=80%.
Assumptions: Each document has exactly ten equally counted fields. A document passes only when all ten fields are correct. Twenty wrong fields are arranged in either two or twenty documents.
Limits: The scenarios are constructed and do not describe OCR performance. Real fields differ in importance; repeated values and dependent errors violate any simple independent-error interpretation.
- Google Document AI: handle a processing response, read 14 November 2026: Document AI responses provide extracted document structure and text anchors that connect structured output to document text. These fields can be inspected when validating extracted results.
Code: scripts/insights/practitioner-b7.mjs. Frozen tables: src/data/insights/2026-11-14-b7.json.
Synthetic monthly input =10,000 requests × assumed input tokens. At an assumed $1 per million input tokens, lengths 4,000,7,000,12,000 cost $40,$70,$120. No token counts were measured and no language ratio is claimed.
Assumptions: Three invented localised prompt versions have specified input lengths and the same 10,000-request volume. Flat input-only teaching rate of $1 per million tokens.
Limits: Versions are not named languages and are not claimed semantically equivalent by test. Output, caching, tools, token-estimation differences and quality checks are excluded.
- Claude token counting, read 14 November 2026: The token-counting endpoint accepts structured message input, including system prompts and supported tool/document inputs. Counting is an estimate; actual message usage can differ.
Code: scripts/insights/practitioner-b7.mjs. Frozen tables: src/data/insights/2026-11-14-b7.json.
Under independent identical trials with fixed failure probability p, probability of zero failures in n trials=(1−p)^n. At p=0.01 and n=50,100,300, this is about 60.5%,36.6%,4.9%. For zero observed failures, one-sided exact 95% binomial upper bound=1−0.05^(1/n).
Assumptions: A mathematical model of independent identical trials with a fixed 1% failure probability. The displayed upper bounds assume zero observed failures. No AI system was tested.
Limits: Related fixtures, changing prompts or workload drift can invalidate the trial model. A frequentist bound is not a 95% probability statement about a particular model being safe; it covers only the defined failure and sampling assumptions.
- NIST: binomial distribution, read 14 November 2026: The binomial model describes counts of binary outcomes over trials with a fixed probability. The zero-failure probability here follows by substituting zero into its probability formula.
Code: scripts/insights/practitioner-b8.mjs. Frozen tables: src/data/insights/2026-11-14-b8.json.
For 1,000 synthetic cases with ten positives: always-negative gives TP=0,FN=10,FP=0,TN=990, accuracy=99%,recall=0%,precision undefined. Candidate gives TP=8,FN=2,FP=20,TN=970, accuracy=97.8%,recall=80%,precision=8/28≈28.6%.
Assumptions: An invented binary classification task contains ten relevant positive cases and 990 negatives. Two constructed prediction sets are compared at fixed decisions.
Limits: The class mix and errors are synthetic. Precision and recall will depend on the real workload and decision threshold. No model ranking or universal trade-off is established.
- Google: accuracy, precision and recall, read 14 November 2026: Accuracy counts all correct classifications; recall counts detected actual positives; precision counts correct predicted positives. The guide explains why overall accuracy can mislead on imbalanced data.
Code: scripts/insights/practitioner-b8.mjs. Frozen tables: src/data/insights/2026-11-14-b8.json.
Synthetic 100-case table: both accept=80,reference accept/judge reject=10,reference reject/judge accept=5,both reject=5. Agreement=(80+5)/100=85%; disagreement=15%; false accepts among ten reference rejects=5/10=50%.
Assumptions: One hundred invented answers have binary reviewed reference labels and one automated judge label each. The human reference is the chosen test standard, not infallible truth.
Limits: No judge was run. Agreement is specific to the rubric, case mix and labels. A high overall rate can conceal poor agreement on a small important group; this example does not estimate real judging reliability.
- Claude: define success and build evaluations, read 14 November 2026: The evaluation guide describes code, human and LLM grading. It recommends clear rubrics and testing LLM-grading reliability before scaling.
Code: scripts/insights/practitioner-b8.mjs. Frozen tables: src/data/insights/2026-11-14-b8.json.