Where AI-assisted analysis goes wrong

Start with the honest boundary, because most of the confident writing in this area crosses it. What is confirmed is what a vendor documents about a specific product version, and what you have measured yourself on your own data. Everything else is inference.

That distinction is not pedantry. Claims about accuracy, productivity gain and reliability circulate as though they were properties of a category, when they are at best measurements of one system on one task at one point in time. Treat an unsourced percentage in this field the way you would treat an unsourced conversion rate in a pitch deck.

So this article does not tell you what any tool does. It tells you what goes wrong when analysis is produced with assistance, how each failure presents, and what check catches it. Those failure modes are stable even as the tools change, which is what makes them worth writing down.

The general discipline is not new. Microsoft's responsible AI guidance frames it as a set of principles and tooling for assessing and governing AI systems rather than a property you can assume. The word doing the work there is assessing.

The most common failure in marketing analysis is not a wrong calculation, it is a right calculation over the wrong definition. Conversions counted one way, sessions counted another, a lookback window nobody named.

Assisted analysis makes this worse rather than better, because the output reads as settled. A paragraph that says paid search delivered a 3.1 return on ad spend carries no visible marker of which conversion action, which attribution model, or which date basis produced it.

Detection: require every number in an output to be traceable to a query. Not to a source system, to a query, with the filters visible. If the analysis cannot produce the query, the number does not go in front of a client.

This is the same discipline that resolves the oldest reporting argument in the business. Google Ads and GA4 disagree for structural reasons before any defect is involved, and an assisted summary that reconciles them without saying how has not reconciled anything.

Analysis over data that stopped updating produces confident, internally consistent, wrong conclusions. Nothing errors. The shapes are plausible. The trend is simply three weeks old.

Google's machine learning guidance names this directly as a category that afflicts data-driven systems more than conventional ones, advising teams to know their freshness requirements and to watch for silent failures, because tables can go stale for months while performance degrades unnoticed. That was written about models, and it describes assisted analysis over a warehouse exactly.

Detection: assert freshness upstream of the analysis, not inside it. The query that feeds the summary should fail if the source partition does not cover the claimed period. That assertion belongs in the pipeline where it can block, not in a reviewer's checklist where it can be forgotten.

The general form of the rule: a system that cannot fail loudly will eventually fail quietly, and quiet failures in analysis reach clients.

Ask why campaign performance improved and you will get reasons it improved. The framing of a request is an input, and a confident answering system will honour the premise rather than challenge it.

This is a human failure mode too, which is why it is easy to miss. The same guidance warns that you are not a typical end user and that team judgement carries confirmation bias, so observed problems should be quantified rather than described. Assistance amplifies whatever bias the question carried in.

Detection: ask the inverse question and compare. If "why did this improve" and "why did this get worse" both produce fluent, plausible answers over the same period, neither answer is evidence of anything.

The structural fix is to specify the test before you look. State what result would change the decision, then run it. That is ordinary analytical hygiene and it is the single cheapest defence in this list.

Every detection step in that table is a human action taking under five minutes. That is deliberate. Controls that require tooling do not get applied under deadline, and deadline is when this work is produced.

The sixth row is the one that costs money in multi-location and multi-brand accounts. An aggregate that looks healthy while one location bleeds is not an AI problem, it is a dimensional problem that assistance makes easier to overlook because the summary is so readable.

The last row is worth applying to any recommendation regardless of how it was produced. An action item with no stated cost has not been thought about, it has been generated.

Scepticism about output is not an argument against use. The tasks where assistance is least risky share one property: the output is checkable in less time than it would take to produce by hand.

Drafting a query you then read and run. Restating a long thread as a list of decisions you then verify against the thread. Producing three alternative framings of a section you are stuck on. In each case the human is checking rather than trusting, and the check is fast.

The tasks where it is most risky share the opposite property: the output is expensive to verify and cheap to accept. A summary of a data set nobody will re-query. A conclusion in a document that goes straight to a client.

Project delivery guidance makes the same distinction in older language, noting that managing this kind of work means being explicit about how success is measured and what the system is actually for before anything is built. Applied here, it means deciding what a correct output looks like before you generate one.

None of these checks survive as personal discipline. They survive as a review gate: a named step, with named checks, that runs before anything reaches a client. A review standard with specific checks is what turns this catalogue into something that actually catches errors.

The cultural half matters as much as the procedural one. Blameless review of what went wrong, shared ownership of the output, and a working assumption that failures are systemic rather than personal are the conditions under which people report a mistake early. That is the core of what a functioning delivery culture is for, and it applies unchanged here.

Write down what your agency permits, on what data, with what disclosure. A short policy an agency of fifteen can actually enforce beats a long one nobody has read.

Where the underlying numbers are the thing in dispute, no review process substitutes for reconciling the definitions. That is what a conversion attribution audit produces: one authoritative source per decision, with the expected variance written down in advance. On the platform migration that moved 154 data sources, the same principle held: the count was defensible because every source had a named definition, not because the summary read well.