How AI is reshaping the audit

Headlines about how AI is changing different professions are everywhere, and auditing is no exception. But what does the hype boil down to for auditors in the real world, and are regulators on top of these new developments?

Three finance leaders with AI expertise—Will Bible, audit and assurance digital products leader at Deloitte; Shawn Panson, US assurance transformation leader at PwC; and Melissa Burgum, CAO and corporate controller at Accenture—addressed the PCAOB on the topic at the board’s investor advisory group meeting in late April.

The consensus? AI means “the mechanics of how audits get done are going to change,” as group member Jen Sisson, CEO of the International Corporate Governance Network, put it, and the PCAOB needs to think about what it needs to do differently to respond.

Resources needed. At large firms with the means to make heavy investments in AI, change is happening fast. “The models are getting markedly better and the case for using them stronger every couple of months,” Bible said. “This isn’t a one-year, two-year, three-year cycle.” Tools that could only read PDFs six months ago, Panson told the Wall Street Journal in April, now can process 30 types of data and match information between two sets of records.

PCAOB chair Jim Logothetis questioned whether firms without deep pockets might be left behind. “For the smaller firms, it’s almost impossible for them to develop some of these things because they just don’t have the funds,” he observed. A major reason they’re turning to private equity funding, he said, is to increase their technological capacity so they can remain competitive.

Human assistance. Human auditors aren’t going away anytime soon, both Deloitte’s Bible and PwC’s Panson said. AI is still more of an “accelerator” and an “overall quality control” in the auditing process “as opposed to something parallel or new or different,” Bible said. “It does speed up some of those initial steps of data gathering, research, etcetera,” he said. That means auditors can spend less time on “very rote tasks,” Panson said. And auditors still must review the AI’s output, Bible said.

Bible also doesn’t think AI tools are able to evaluate anomalies just yet. “The role of a person doing that and evaluating primary audit evidence is not being challenged at this moment,” he said, though he noted that could change as the technology develops.

AI also hasn’t changed Deloitte’s workforce much, Bible said. “The migration of data from a stack of invoices into a spreadsheet has gotten more efficient,” he said. “That doesn’t mean that the role level in the firm is obsolete.”

But Panson implied that more workforce disruption may be on the horizon, calling Bible’s remarks “today’s answer.” PwC has “tweaked down [its] new entry hiring just a couple of percentage points,” he said.

More work? Though AI might make auditors more efficient, it could also make their workload heavier and more complex. AI enables auditors to look at more data than they could before. “The volume coming through the funnel is only going to increase for everyone,” Bible said. 

AI has even made full population tests a possibility. Accenture’s Burgum said her auditors look at the entire population of revenue transactions and use algorithms to identify outliers. Though that means they have “slightly more findings,” she said, it gives a better audit. “We’re actually seeing efficiencies because our sample sizes are decreasing, but we’re focused on the right things.”

That said, not all clients yet have the type of consistent, standardized data necessary for full population tests to be performed. “There’s not many places where you can find full populations of good, reliable external evidence to source from,” Bible noted.

Quality control. The advent of AI has amplified certain types of risk within audits. AI tools can “hallucinate” incorrect answers, and they’re probabilistic rather than deterministic. “Deterministic means I write an explicit instruction, the machine then executes that instruction, and if I give it that instruction 1,000 times, it will 1,000 times give me the exact same answer,” Gina Sanchez, the CEO of Chantico Global, said at the PCAOB meeting. “AI is a probabilistic mechanism, which means that if the context changes, if the way that you ask the question changes, you can get different answers."

AI tools are improving with time, Panson said: “We definitely see less in the way of hallucinations,” he noted. PwC, he said, builds its own agents, trains them, and controls the data that goes into them, which he believes minimizes risk. Before the firm releases a tool, he said, it puts it through “an extensive governance process” and runs many pilots to ensure it’s working as it should.

AI tools aren’t like “old-school deterministic” software, Panson agreed, which means they need to be continually monitored. PwC uses a “model evaluation tool” that “runs metrics daily” and allows users to look at what the AI tool did at any point in time, he said.

Oversight of the tools, too, has had to change. “The addition of AI has made those audit plan discussions a lot more involved,” Burgum said. She has a yearly session of “almost a full day” where her team and her auditors discuss how AI was used in an audit and what to expect.

Regulator catch-up. There’s a real need for regulatory guidance in the area of AI, Bible said. Audit firms have questions about areas such as what quality control standards for AI on the issuer side should look like and what constitutes a deviation or a deficiency, he said, while Panson suggested regulators should develop frameworks that address such questions as “What does it mean for a tool to have auditability [or] observability?”

Logothetis expressed concerns that the PCAOB needs to do more to stay on top of AI. The organization has “just started thinking about” training inspectors on AI, he said. In fact, “when they go out and visit a specific firm or inspect a specific firm, they need to be experts in their platform—at least as good as the auditors themselves, if not better.” (Some firms are assisting in this area: Panson said that PwC discloses to the PCAOB when it uses AI tools in audits, and that it’s demoed the tools to PCAOB inspectors and leadership.)

But more needs to be done, Logothetis feels: “The reality is, the firms are ahead of us,” he said.