Data security concerns mount with blanket AI deployments

AI is officially everywhere, and maybe you’re tired of hearing about it. Or maybe you chat daily with your best friend, ChatGPT or Claude, and tell them all your deepest, darkest secrets.

You may think those secrets are safe, stored away in an LLM’s beautiful cloud of 1s and 0s. But you should maybe avoid telling it business secrets; AI use poses major data security risks for enterprises, according to Paul Martini, CEO of cloud security provider iboss.

A finance organization, or any division within a company, buys a purchased version of Microsoft Copilot or Claude and says, ‘Well, that’s what we’re using, so we’re secure,’” Martini told CFO Brew.

“What they forget is, any employee can sign up for any personal ChatGPT…I think that there’s a lot of, what I would say shadow AI, and it’s dangerous because it’s personal or financial records that are just leaking to large language models. And once it leaks to those models, anybody can ask the model to tell [them] more about some information. So data loss and leakage is going to happen in a completely different way.”

In a 2025 IBM report on data breach costs, 13% of the 600 organizations surveyed “across 16 countries and geographic regions and 17 industries” reported data breaches related to AI applications. And among those that did, “almost all (97%) lacked proper AI access controls.”

Messaging clarity. Companies that don’t communicate clearly on how to interact with the approved AI tools put confidential information at risk, subscription management platform Zuora CIO Karthik Chakkarapani said. It’s about not speaking in “technical jargon and security jargon” when talking about AI security with other teams.

“The finance teams, or the CFOs or the COOs, need to understand that AI risk is not abstract; it can involve forecast, pricing, contracts, customer data, board materials, and strategic decisions,” he said. “Or any information, as a matter of fact.

“The message should be very simple: Use approved tools, protect confidential data, verify outputs, and escalate questions early,” he advised. But he also said it involves creating an AI policy that employees can reference if they have any concerns or questions.

And even then, “Policy documents are important, but they’re not good enough. If you don’t enforce it, nobody’s going to use it. So teams need very clear information and dos and don’ts, practical examples, continuous training and enablement, [a] simple decision framework,” Chakkarapani said. “Can I upload this information into ChatGPT or not?”

Roll your own. OK, so ChatGPT and Claude both pose security risks. What about building a proprietary LLM, then? Derek Bush, CISO of enterprise cloud software maker Infor, doesn’t think that’s the solution.

“There’s a huge operational uplift on that. There is not a real savings, and a lot of times that costs you more because you can’t keep up with how fast the industry is going,” Bush said.

Chakkarapani said his thinking on this topic has evolved over the past six months. Some projects that are “low effort, low risk, low cost” can be vibe coded, he said, but not everything.

“Previously, until I would say fall of 2025, the focus was still on, build by partner, leverage AI more often as an additional tool on top of your enterprise application. But today, you could go build your own applications very quickly and rapidly. That doesn’t mean that you can go and vibe code your CRM or HCM or ERP application over the weekend,” he said.

Before you rip your hair out, let’s give an action item Chakkarapani and Bush agreed can help prevent data security breaches via corporate AI use: Get back to the basics and build a strong security foundation.

“It goes back to good foundational elements and governance, and if you do that from the leadership perspective and drive it down, that risk goes down. It’s a manageable risk. Overall, the concept of AI in that really experimental phase, from what I’m seeing, is starting to drop down, and it’s more directional,” Bush added.