Who's to blame if AI fails at work? Wharton study shows a split

Companies are giving artificial-intelligence systems more autonomy to boost productivity and efficiency. But as bots take on more responsibility, executives, managers and employees remain unclear on who should be held accountable when something goes wrong. A new report published Tuesday by the Wharton School of the University of Pennsylvania found little agreement among managers. When asked who should be held responsible when a bot “makes a consequential error,” respondents were split roughly evenly between the employee deploying the AI agent, the employee’s manager and the company’s risk team, with about a quarter choosing each. Roughly 10% blamed the vendor, while a similar share said they weren’t sure who should be held accountable. That question of accountability is becoming even more pressing as companies including OpenAI, Alphabet Inc.’s Google and Meta Platforms Inc. push increasingly powerful AI agents into the hands of employees. The bots are equipped with their own corporate identities and are able to independently work toward goals, in some cases over the course of several days. While concerns of rogue AI agents have dominated headlines in recent months, the more immediate risk to companies may be far more ordinary and mundane. A minor prompt error or unclear instructions can quickly snowball into a larger crisis when agents have access to live systems such as email accounts or customer databases, said Leeron Walter, vice president at employee-monitoring software company Teramind. Companies are already facing those kinds of mistakes. Walter pointed to one national US retailer where an employee built an AI system to automate customer emails. A prompt error caused the bot to send incorrect refund and cancellation notifications to more than 18,000 customers in less than six hours, triggering thousands of customer-service inquiries. Walter said customer demand for employee-monitoring systems has ticked up in the wake of embarrassing mistakes like these. At another US-based consumer electronics company, Walter said a marketing employee used AI to automatically generate and publish product prices. A prompt error caused a flagship device to be listed at an 85% discount for three hours. More than 700 orders were placed before the mistake was caught, and the resulting customer-service fallout took weeks to resolve. Mishaps like those, which can result in large losses and long-term reputational damage, are raising an uncomfortable question for workers and their managers: If employers are being encouraged to experiment with AI, who takes the blame when something goes wrong? “Organizations are learning in real time,” said Jeremy Korst, a partner at the consulting firm GBK Collective and co-author of the Wharton study. “When I ask this question of a senior executive, the most frequent answer I get is that they hadn’t thought about it yet.” -- By Jo Constantz (Bloomberg News)

Who’s to blame if AI fails at work? Wharton study shows a split

28% respondents say risk teams are to be blamed for the impact

AI democratizes fraud, equipping bad actors with sophisticated capabilities

Related Posts

AI democratizes fraud, equipping bad actors with sophisticated capabilities

FIs should treat AI agents as employees, not tools

Building trust with agentic commerce imminent

EMERGING FINTECH DIRECTORY

FinAi Podcast

SPONSORED

The Next Generation of Cyberthreats: Powered by AI

Build an Antifragile Strategy to Outperform the Market

How AI and Product Experts Turn Fuzzy Requirements Into Focused Dev-ready Roadmaps

Connect

Welcome Back!

Retrieve your password

Continue Reading with FinAi News Premium - Less than $2/Day

Who’s to blame if AI fails at work? Wharton study shows a split

28% respondents say risk teams are to be blamed for the impact

AI democratizes fraud, equipping bad actors with sophisticated capabilities

Related Posts

FinAi Podcast

Connect

Welcome Back!

Retrieve your password

Unlock This Article

Continue Reading with FinAi News Premium - Less than $2/Day