Acceptance criteria for an analytics deliverable

A website has a screen. A client can open it, click things, and form a view. A tracking implementation, a pipeline or a semantic model has none of that. What the client sees is a number, and a number gives no clue whether it is correct.

So acceptance collapses into two failure modes. Either it never happens, and the milestone is invoiced against a deliverable nobody confirmed, or it happens six weeks later when somebody notices a figure they do not believe and the entire engagement is reopened.

The fix is not more rigour at demo time. It is writing acceptance as a test rather than as an impression. Atlassian's definition is the right starting point: acceptance criteria are the conditions a deliverable must satisfy to be accepted by the customer, which means conditions, plural, and satisfied rather than approved.

Two levels are needed and they are constantly conflated. Acceptance criteria are per deliverable and answer whether this particular thing does what it was supposed to. The definition of done applies to everything and covers the standing bar, which Atlassian describes as the shared agreement on what completion means for the team. Without the second, every deliverable arrives with a different idea of finished attached to it.

Every workable analytics acceptance criterion contains the same three things.

A named comparison source. Correct is meaningless on its own; correct against something is testable. Name the platform interface, the client's own finance figure, a call tracking export, or a manually counted sample. If there is no comparison source, you are not going to be able to prove anything and that is worth discovering during scoping.

A numeric tolerance. The comparison will not match exactly, and pretending it will is how a deliverable fails acceptance for a reason that was never within your control. Two figures from different systems are counting different events at different moments with different rules, and the reasons for that on the advertising side are set out in why Google Ads and GA4 numbers do not match. Write a tolerance because a tolerance is the honest description.

A window. Within what period, measured over how many days, and as of when. "Sessions match" is unbounded. "Daily session totals for the trailing 28 days differ by no more than 2 percent per day" is a query somebody can run and either passes or does not.

Put together: the deliverable is accepted when the warehouse figure for a named metric agrees with a named source, within a named tolerance, over a named window, verified by a named person. Five nouns and the argument is over before it starts.

The pipeline row is the one that gets skipped and the one that costs the most later. A pipeline that produces the right numbers once has not been accepted; a pipeline that produces the right numbers on a rerun has. Reruns happen constantly in real operation, and a load that duplicates on rerun will corrupt a dataset weeks after anybody remembers the deliverable.

The scheduled report row is unusual in that the test is elapsed time rather than an action. That is deliberate. Three consecutive clean runs is the smallest evidence that the schedule, the credentials, the distribution list and the failure handling all work, and none of that can be demonstrated on the day of delivery.

The dashboard row needs the written script or it degrades into a walkthrough. A script is a numbered list: open this tile, compare it to this figure, record the difference. Handing that to the client turns acceptance into a task with a completion state instead of a meeting with a vibe.

For dashboards intended for a wide internal audience, a conformance line is worth stating alongside the reconciliation one, because it is otherwise assumed. The W3C's WCAG 2.2 success criteria give a testable target for things like colour contrast and keyboard operation, and naming a level against a named set of views is far more useful than either promising accessibility or excluding it in the abstract.

A tolerance is not a concession. It is a statement about what two systems can be expected to agree on, and picking it is a technical judgement you should make before the client asks.

The method is to measure first. Before writing the criterion, run the comparison on a week of existing data and see what the natural variance is. If the two systems already differ by a consistent small amount for structural reasons, that is your floor and no amount of work will improve it.

Then write the tolerance above the measured floor with a margin, and write down why. "Within 2 percent daily, reflecting the measured difference in session definition between the two systems" survives a challenge. "Within 2 percent" invites one.

Some things get a tolerance of zero and should. Revenue reconciled to a finance system, row counts on a controlled load, the count of records in a migration. Where the correct answer is knowable exactly, accept exactly, and be explicit that this class is different from the estimated ones.

Where the deliverable feeds decisions across locations or business units, the acceptance has to cover the split as well as the total. A franchise operator with more than fifteen clinics reporting to both corporate and individual franchisees needs each location's figure to be right, not just the rollup, and a total that reconciles while the split does not is a failure that a total-level test will pass.

Name the approver in the document. Not a department, a person, with a deputy. On analytics work this is frequently the hardest clause to agree, because the person who commissioned the work is often not the person who can judge whether the numbers are right.

That gap is worth surfacing early rather than discovering at acceptance. If nobody on the client side can validate a metric, the acceptance test cannot depend on their judgement, and it has to be written against a source document instead. A definition everybody agreed to in writing is the only thing that works, which is the subject of metric definitions that survive a team change.

Attach a window and a deemed-acceptance clause. If written feedback does not arrive within the stated number of business days, the deliverable is accepted and the milestone is invoiceable. Without it a client can hold a project open indefinitely at no cost to themselves while your team holds capacity.

Tie payment to accepted deliverables rather than to calendar dates. A date-based milestone can arrive with nothing accepted, which is the standard route to a project running months past its invoicing schedule.

At programme scale the same discipline is what makes work fundable at all. A clinical research organisation automating its database change management got there by decomposing into 159 tracked work items with acceptance criteria, which is what let a VP sponsor approve phases rather than approve a direction.

Acceptance criteria written at delivery time are a summary of what you built. Written at scoping time they are a specification, and they change the build.

They also change the estimate, usually upward and correctly. The moment somebody writes "reconciles to the finance system within zero tolerance" it becomes obvious that reconciliation is a work stream rather than a checkbox, and it gets its own hours. That is the criterion doing its job before a single line of code exists.

Write them for the smallest deliverable you invoice against, not for the project. A project-level acceptance criterion is a single point of failure at the end. Per-deliverable criteria let you close and bill in sequence.

The upstream requirement is a definition of what is being built that both sides refer to, which Atlassian frames as a single agreed statement of what the thing must do. Acceptance criteria are that document turned into tests, so if the first does not exist the second is being invented from memory.

All of this belongs inside a scope document rather than in a separate note, and the structure it fits into is set out in scoping a fixed-fee statement of work. Where the same three criteria keep getting rewritten from scratch on every proposal, the fix is a template, which is what statement of work scoping delivers.