The price is right

In a world of big data and artificial intelligence, we’re increasingly used to seeing customization everywhere we go. For some companies, that includes customizing the price that customers will pay, based on models that predict a customer’s ability, willingness, or desire to pay different price levels.

As an organization’s leading risk manager and as concerns about discrimination by AI continue to grow, finance needs to pay close attention to avoid unintended pricing discrimination.

Gray area. “There is a difference between price discrimination and price personalization,” Jose Mendoza, associate professor of practice at the University of Arizona, told CFO Brew, adding that while the former could be unethical or illegal, price personalization would be an effort “to adjust the price based on what you want.”

“Now that more and more retailers are jumping into doing this, they should be aware that this is something that you should be looking at,” Mendoza said.

Often, price personalization can mean offering a discount to buyers who are perceived to be more price-sensitive or less likely to use a good or service without a discount, and there are examples of this long before the era of big data and AI, according to Mendoza.

“Ladies’ Night for restaurants, early-bird specials as a time-based price discrimination, senior discounts, student discounts, military discounts—that’s not a problem,” Mendoza said. “The problem is when there are increases” in the price of goods or services.

These days, data points about the phone or browser one is using, one’s geography, purchasing history, or thousands of others, could be part of a personalization algorithm. Depending on which data points one uses, “you might be discriminating against protected categories such as, for example, gender, religion, age, gender preferences, and so forth,” Mendoza said.

Studies of AI have shown that algorithms can generate bias against skin types and gender. One investigation by The Markup found that AI bias in mortgage applicants led to as much as 80% of Black applicants being denied. That potential for discrimination, in turn, can lead to higher costs or rejections for such customers when borrowing on credit cards or taking out mortgages.

But a company selling goods and services instead of loans might wonder why it should care about the potential for discrimination here: After all, if an algorithm is going to suggest that people in protected groups have less ability to pay, won’t it inevitably charge them lower prices instead of higher ones?

Mendoza said that algorithms can create higher prices for these groups, too. He gave the example of a retailer charging a higher price when there are fewer competitors near a customer. Depending on who lives in that geography, “when you do an audit…you can find the difference is the person of color is going to have a higher price,” Mendoza said.

Another possibility for discrimination would be the move away from cash, with companies providing discounts or even the ability to purchase at all only to those who use certain electronic payments, or who have certain banking relationships or credit access, all of which can be a proxy for discrimination against protected categories, according to the NAACP.

Even something like offering a discount to those who enter a certain code online could create problems for certain consumers, such as senior citizens and people with disabilities, Mendoza said, adding, “We do all these promotional plans, when we need to be aware that there’s people who might have problems accessing it.”

Is there a fix? Ultimately, Mendoza said he thinks that companies need to focus on larger goals when considering AI-driven pricing: “Every time that we do pricing in retail, we have to be looking into developing a long lasting, profitable relationship that works for both the customer and for the retailer,” he said.

That might mean that the additional margins one could get with AI-driven pricing aren’t worth it for certain products or services: If you are “just optimizing pricing without even looking at how the customer feels after the purchase, you’re actually damaging that relationship with the customer,” Mendoza said.