OpenAI reports new AI safety incidents, sets disclosure plan

OpenAI shared several undisclosed incidents of its AI models misbehaving and unveiled a new framework for tracking and disclosing such occurrences going forward. Some of the previously unreported instances included OpenAI’s technologies concealing and fabricating information in order to return results, the company said in a blog post Wednesday. The ChatGPT maker has faced increased scrutiny since saying in July that some of its most advanced AI models had breached the systems of an external software company, Hugging Face. None of the newly disclosed misalignment incidents involved a hack or breach of a third party, OpenAI said in a separate statement. In the report, OpenAI detailed various instances of its artificial intelligence models misbehaving in order to complete a task or succeed at an evaluation. Some examples included the models fabricating missing data, attempting to bypass network restrictions and AI agents sharing files with each other that they were supposed to keep private. OpenAI also outlined a case for employees to self-report similar incidents of so-called misalignment, which is when AI acts in ways that are not in line with human objectives, as well as a system to triage such self-reported occurrences. “In the past, so as to better inform researchers, AI developers, policymakers, and the general public, we’ve sought to make our findings about misalignment public,” OpenAI wrote. “But without a systematic approach to reporting these findings, our disclosures have been ad hoc and less frequent than ideal.” The company said the reports are only an initial set of disclosures, and not a comprehensive account of issues arising from its chatbots. “We do not believe that the AI industry has solved alignment and monitoring to a sufficient degree to continue responsibly scaling at maximum speed for much longer,” the company wrote. “Decisions about how AI development should proceed in the months and years to come need to draw on evidence that people outside the companies building frontier models can examine for themselves.” The Hugging Face incident was just one in a series of recent online hacks carried out by AI models made by OpenAI, Anthropic PBC and Meta Platforms Inc., causing widespread concern about the security risks posed by this increasingly powerful technology. Anxiety about AI’s existential risks gained momentum last week, driven by the high-profile departure of Anthropic employee Jacob Coxon, who accused AI companies of “gambling with our lives” in a resignation post he shared on social media. In the past several days, numerous AI company leaders, including Anthropic Chief Executive Officer Dario Amodei and OpenAI CEO Sam Altman, have urged slowing the pace of the development of the technology to address its increasingly unpredictable risks — though they diverge on exactly how it should be handled. On Saturday in a 3,800-word essay, Amodei called for government regulation and for the tech industry to support a broader AI slowdown. At a conference in San Francisco Tuesday, Altman and Nvidia Corp. CEO Jensen Huang both acknowledged concerns but argued AI companies would safely pace their technology on their own. And Meta Platforms Inc. Chief Executive Officer Mark Zuckerberg said Tuesday that AI labs should rely on independent evaluators and advisers to ensure that models are safe. President Donald Trump vehemently rejected calls to slow AI’s development on Monday, dismissing fears about the tech’s risks as “a hoax” and rejecting the idea of new rules. The AI boom has fueled a historic US stock market rally. Any pause or delay in progress on so-called frontier AI systems would be unwelcome among investors who are betting that the AI boom will drive hundreds of billions of dollars in capital expenditure. (Updates with OpenAI statement in third paragraph.)

OpenAI reports new AI safety incidents, sets disclosure plan

President Donald Trump rejected calls to slow AI’s development on Monday

Prompt injection attacks: When AI follows someone else’s orders

5 questions with … Kasper Tjorntved Davidsen, CAIO and head of AI at Danske Bank

Related Posts

Prompt injection attacks: When AI follows someone else’s orders

5 questions with … U.S. Bank CISO Ann Barron-DiCamillo

OpenAI says it’s working with Anthropic, Google on AI safety

5 questions with … Kasper Tjorntved Davidsen, CAIO and head of AI at Danske Bank

EMERGING FINTECH DIRECTORY

FinAi Podcast

SPONSORED

Build an Antifragile Strategy to Outperform the Market

How AI and Product Experts Turn Fuzzy Requirements Into Focused Dev-ready Roadmaps

Is Your Technology Supplier There for You?

Connect

Welcome Back!

Retrieve your password

Continue Reading with FinAi News Premium - Less than $2/Day

OpenAI reports new AI safety incidents, sets disclosure plan

President Donald Trump rejected calls to slow AI’s development on Monday

Prompt injection attacks: When AI follows someone else’s orders

5 questions with … Kasper Tjorntved Davidsen, CAIO and head of AI at Danske Bank

Related Posts

FinAi Podcast

Connect

Welcome Back!

Retrieve your password

Unlock This Article

Continue Reading with FinAi News Premium - Less than $2/Day