“Deterministic” has become a sales word. It is worth knowing what it means, because the word is true of some legal AI products, false of others, and the difference is the difference between a result you can defend and a result you can only repeat if you are lucky.
Deterministic means one thing: the same input produces the same output, every time. Not usually. Every time.
The thing vendors usually mean: temperature zero
Ask what makes the product deterministic and you will often be told that the model runs at temperature zero. Temperature controls the sampling step, so at zero the model is supposed to take the highest-probability token every time, which sounds like determinism and is not.
OpenAI’s own cookbook says it plainly for its API: with a fixed seed and parameters, the system will make a best effort to sample deterministically, outputs will be mostly identical, and “Determinism is not guaranteed” [1]. It tells developers to watch a system fingerprint value that changes when the backend configuration changes, which is an admission that the backend can move underneath you [1].
The reason is not mystical, and it is not simply that GPUs are parallel. A 2025 analysis from the Thinking Machines Lab traced the common cause to a lack of batch invariance: the numerical result for your request depends on the size and shape of the batch it happened to be processed in, which depends on what other traffic the server was handling at that moment [2]. Their measurement is the one to quote in a vendor meeting. Sampling 1,000 completions from one model at temperature zero, with the same prompt, produced 80 unique completions; the outputs were identical for the first 102 tokens and then diverged, with 992 continuing one way and 8 another [2]. With batch-invariant kernels in place, all 1,000 runs were identical [2].
Academic work points the same way for the settings a lawyer would call deterministic. A study of five models configured for determinism across eight tasks found accuracy varying by up to 15 percent across runs, and none of the models reliably producing identical output strings [3].
So temperature zero is a request, not a guarantee. The California bar’s guidance states the underlying property without hedging: generative AI may produce outputs that are probabilistic, not deterministic [4].
The thing that can actually be deterministic: the decision layer
Here is the distinction that matters when you buy.
A generator proposes text. It is probabilistic by construction, and no setting removes that.
A decision layer takes what the generator produced and applies rules to it: does this citation resolve to a document, does the quoted passage appear at that page, does this conclusion follow from the facts listed, does every rendered sentence carry a source. Those are checks, not predictions. They can be written as ordinary code, and ordinary code is deterministic: run it twice on the same draft and the same record, and you get the same verdicts, in the same order, with the same reasons.
A product can honestly say its enforcement is deterministic while saying nothing of the kind about its drafting. A product that says “our AI is deterministic” without drawing that line is either confused or hoping you are.
What to ask
Four questions, in this order.
- Which component is deterministic? The generator, the checks, or the end-to-end output. Only one of those three answers can be fully true.
- What does a rerun hold fixed? Same input, same rules version, same corpus snapshot, same model version. If the vendor cannot pin the corpus and the rule set, a rerun is a new run.
- What is logged per decision? For a real decision layer, each verdict should name the rule that fired and the passage it relied on, not just pass or fail.
- What happens when a check cannot be performed at all? A deterministic gate needs a defined answer for “unavailable”, and the safe answer is to fail closed and say so.
What a reproducible rerun proves, and what it does not
If you hand the system the same closed file next month and the verdicts come back identical, you have learned something narrow and valuable: the checking is a function of the input rather than of the day. That is what lets you test the tool once and rely on it for a class of work, which is the posture ABA Formal Opinion 512 describes when it says the appropriate amount of independent verification depends on the tool and the task, and points to prior testing as part of the basis for reliance [5].
It does not prove the rules are the right rules. A gate that reliably fails to notice a misquoted holding will fail to notice it every single time, with perfect consistency. Determinism is a property of the machinery, not a measure of judgment.
So run the test on a file where you already know the answer, twice, a month apart, and also read what the gate says about the two or three sentences you deliberately broke. This is how Apodicta asks to be evaluated: the enforcement is code outside the model, the same draft and record produce the same verdicts, and anything that cannot be proved is held and named rather than quietly rendered.