Zero failures in a small AI test does not mean zero risk
A process that fails 1% of the time has about a 36.6% chance of showing no failures in 100 independent trials, so a clean test run is evidence with limits, not a guarantee.
Zero failures in a hundred tests is a useful observation. It does not show that the next request cannot fail. A clean run should be reported with its size, the definition of failure and how the cases were chosen.
The NIST binomial reference, read on 14 November 2026, gives the probability formula for repeated yes-or-no outcomes at a fixed probability. We use that model for one narrow calculation: how often a process with a known hypothetical failure rate would produce a clean sample.
How likely is a clean run if failures exist?
Assume each trial fails with probability 1%, independently of the others. Its chance of passing is 99%. The chance that all n trials pass is 0.99 raised to n. That is a calculation under stated assumptions, not an estimate of any model's real behaviour.
| Independent trials | Chance of zero failures at p=1% | 95% upper bound after zero failures |
|---|---|---|
| 50 | 60.5% | 5.82% |
| 100 | 36.6% | 2.95% |
| 300 | 4.9% | 0.99% |
At fifty trials, the hypothetical process has about a 60.5% chance of a clean run. At a hundred it is about 36.6%, and at three hundred about 4.9%. A small clean sample can therefore sit comfortably alongside failures at the assumed rate.
What an upper bound does and does not say
If zero failures are observed, the same zero-failure probability can be turned round at 5% to give the one-sided exact 95% binomial upper bound: one minus 0.05 raised to one divided by n. At a hundred trials that upper limit is about 2.95% under the trial model.
This is a frequentist confidence procedure. It does not mean a 95% probability that a particular model's failure rate lies below the number. It also says nothing about failures you did not define or workloads you did not sample, and those limits should sit beside the bound rather than in a methodology link.
For a high-consequence decision, an appropriately designed evaluation and qualified statistical review are the right tools. The table is for teaching. It is not a certification rule, and it does not say three hundred tests are enough to deploy.
Rows are not the same as independent cases
A test set full of near-duplicate questions gives less distinct coverage than its row count suggests. Repeating an easy case answers a different question from sampling the intended workload widely, so the unique cases and the repetitions both need recording.
For this calculation to mean anything, the model, prompt, tool configuration and pass rule (the written test for what counts as acceptable) stay fixed within a run. Repairing the prompt after a failure and carrying on counting produces a record that spans several configurations. A new version should start a new count, with the earlier result kept.
Important categories deserve their own counts. A clean set of short classification tasks says little about long documents, and a category with only a handful of cases should be described that way even when the full set looks large.
What to do with a clean result
The original cases stay, and new ones go in wherever the workflow exposes fresh ways to fail. After an incident, a cleared or synthetic version of it makes a good regression case, meaning a test added so the same fault is caught if it returns. A next set made entirely of cases already used to tune the prompt would prove little.
Excluded or inconclusive results belong in the report too. A timeout that stops review is not a pass. How such outcomes enter the acceptance record is a policy choice, and it should be the same for every candidate.
A fair line in your own results log reads “zero defined failures in this many trials under this configuration”. With the denominator and assumptions left off, the number sounds stronger than the evidence behind it.