You Can't Re-Run an AI Control to Prove It Worked

A founder I work with automated her monthly reconciliation last year. An AI tool reads every bank and card feed, matches transactions, flags what doesn't tie, and posts the rest. Close dropped from eight days to three. Then she signed a term sheet to sell a majority stake, and the buyer's quality-of-earnings team asked something the tool was never built to answer. Show us this control worked. Every month. For the trailing year.

She could show it ran. She couldn't show it worked. Those turned out to be different problems.

Running and proving are not the same job

When an auditor tests a control a person runs, they re-perform it. Pull a sample, redo the match by hand, confirm the reviewer would have caught the exception. The re-performance is the evidence.

That move doesn't exist for an AI control. Re-run the reconciliation today and you get today's model version, reading today's data, against today's rules. It says nothing about what happened in March. By then the model may have been retrained. The prompt may have shifted. The vendor may have shipped four silent updates. The exact control she was being asked to prove no longer existed in a form anyone could execute.

This isn't a public-company problem that stops at the SOX line. The PCAOB's active standards still contain no AI-specific provisions, and the SEC's reporting manual added none in its June update. Auditors are asking anyway. So are lenders at renewal and buyers in diligence, which is where a $2M-$20M company actually meets the question first.

The evidence has to be caught in the moment

The most common finding in AI-assisted control programs this year is blunt: management can't produce the trail an auditor needs to judge whether the AI did its job. Not because the work was sloppy. Because nobody captured the right thing while it happened.

What proves an AI control worked is a record written as it ran. The inputs it read that day. The model and rule version in effect. The output it produced. The point a human looked and signed. Contemporaneous, all of it. You cannot assemble it in diligence, because the state that produced March's numbers is gone by June, and an approximation reads as exactly that to anyone who has seen a real one.

A human control leaves a trail you can walk again. An AI control leaves only what you thought to capture while it was working. Decide that before you switch it on, not when a buyer asks.

Ask the question before someone else does

Pick one closed month. Ask your tool to hand back everything it acted on: the feeds it read, the version it ran, the matches it made, the exceptions it raised, the human who cleared them. If it can rebuild that month cleanly, you own a control you can defend. If it hands you a stack of posted entries and a "completed" flag, you own automation, and a gap that only shows up under a stranger's questions.

That tool will keep closing your books on time right up until the day it has to prove it did. Build the record before the day arrives.