IIBA membership provides you with exclusive access to Member Articles.
Defining Done for AI Features That Never Give the Same Answer Twice
Key Takeaways
- The acceptance criterion is still the unit of work. It needs four extra fields for AI features (a threshold, an evidence base, a judge, and a guardrail floor), not a replacement.
- Classify before you specify. Most “AI features” decompose into deterministic, constrained, bounded, and open-ended parts, each needing a different standard of evidence.
- Thresholds are risk acceptance and belong to the accountable business owner; datasets and rubrics are elicitation and belong to the analyst.
- Never blend the release gates into one score. Capability, safety, consistency, economics, and operability are judged separately, each with named evidence and a named owner.
- Acceptance is a subscription, not an event. Models and inputs drift; every criterion needs a production signal and a trigger that reopens the decision.
AI features never give the same answer twice, so writing what "done" means becomes business analysis at its purest.
Disclaimer: The views and opinions expressed in this article are those of the author and may not reflect the perspectives of IIBA.
Every delivery method you use rests on one atomic unit: the acceptance criterion. Given a context, when an action occurs, then a specific result follows. The BABOK Guide lists acceptance and evaluation criteria among the core techniques of the profession, and the whole apparatus of delivery (stories, sprints, sign-off) assumes that “then” is singular.1
A story is either accepted or it isn't. The definition of done is a boolean.
AI features quietly dissolve that boolean. A large language model asked the same question twice may answer differently, and for the tasks organizations buy these systems for—summarize this document, draft this message, triage this exception—there’s no single correct output at all (only a distribution of acceptable ones). Engineering has built a rich evaluation literature in response. Almost none of it has reached the language business analysts use: user stories, acceptance criteria, release gates.
That translation is the analyst's job, because deciding what “done” means for a probabilistic feature is a business decision about acceptable risk, not a tooling decision. After 17 years of specifying investment data platforms, and two years taking AI features through acceptance in a domain where a bad output can touch client money, these are the three instruments I now use every time.
