Failure, then success for an AI project

It seems like a lot of people, including CFOs, are excited by AI. But what happens when this technology fails to deliver on its big promise? That’s the problem that Harri, an employee experience platform, faced last year.

Harri, founded in 2012, provides human capital management software to hospitality and service companies like McDonalds and Cava.

The privately held, 650-plus-employee company has been using AI, including machine learning and algorithmic capabilities, within its product for “quite a period of time,” CFO Jack Gordon told CFO Brew. The last two years it has “very aggressively moved” into using native AI functionality within its platform, he said. Then, last summer, Harri “made a very big push”Every CFO in the “scale up and start up ecosystem” is discussing AI and how teams are using it to make them smarter and more efficient, Harri CFO Jack Gordon said. “It's the only thing that CFOs are talking about right now.” Related resources: To buy AI tools, or to build them in-house? That is the question. to adopt AI, including Claude and Claude Code, across its back office.

The project. Harri planned to focus AI on the “customer cube,” a data model that multiple teams use to track revenue across different product lines. “We run all of our analytics and customer revenue reporting and data analysis off of this data set,” Gordon said.

“We wanted to automate the calculations, the aggregation of the data, and the reporting around it,” he said, aiming to “free up capacity of our team from doing these manual workflows.” However, he admitted that “there wasn’t a ton of architecting and thought process that went into” how the company would implement AI. “It was just, ‘We know these things are strong and powerful. Let’s drop them on and see if they can do what we want them to do.’”

Gordon’s team ran two processes in parallel: One was the automated AI experiment and the other was the normal manual version, where Gordon’s team calculated and aggregated the same data as the AI.

“Within a matter of weeks, [Gordon and his team] realized that it was failing.” The AI was “unable to reproduce the reporting and do the calculations that we were able to do on our own. It was just spitting out the wrong numbers,” Gordon said.

He was “disappointed” that this seemingly “very quick and easy solution” wasn’t working.

Asking for help. What was the problem? “It was partly a data issue.” The cube had “three different systems that fed into this process, and the data was not standardized, and it was not centralized,” Gordon said. Exceptions in the data also required manual intervention, which AI lacked the context to handle.

Gordon reached out to “a number of different partners,” including Google. “They helped us architect what a data structure should look like, that will allow you to take agentic actions on data,” he said. “The AI needed to learn. The AI needed to get context, which we provided to it through very detailed prompts, and we moved to centralize all the data into a data warehouse, centralize and standardize.”

Harri also “spun a team up in India” to help with the data migration. In late summer, Harri again tried to implement Claude and Claude Code into its customer cube, while running a parallel manual process. “It is within a very narrow band successful,” Gordon said. The automated version was “97% accurate, which we’re comfortable with at this point.”

This success means Harri plans to shut down its manual version and rely solely on the automated one. “We will go live fully, I think, [in October].”

Was it worth it? Gordon estimates the project took close to nine months. Surprisingly, the token costs for the AI project were “very little.” He credited Harri’s partnership with Google, which helped “architect the data pipelines to minimize the variable cost.”

Other costs included the time Gordon’s team spent on the project, which he called an opportunity cost; the cost of the team in India; and a three-month contract with a data architect. In all, Gordon estimated that the expense totaled $20,000 to $30,000.

A self-professed “data guy,” Gordon loves gaining access to data that was previously out of reach. When asked if the many months of data migration and AI implementation were worth it, he said: “Ultimately the answer, I think, will be yes. We still have a lot to understand about the capability sets of the AI, and it continues to get better and better. I think the answer is yes, but that’s caveated with, we still have a bunch of testing to do.”