Anthropic ditches AI safety policy to keep up with OpenAI
Anthropic, the booming tech firm that built its reputation around responsibly developing AI systems, is now loosening its flagship AI safety policy.
In an update to its ‘Responsible Scaling Policy 3.0’ published on Tuesday, the OpenAI rival said it would no longer actively stop the development of a model deemed potentially dangerous if a competitor has already released a similar system.
Its previous framework, introduced back in 2023, committed to halting the development and deployment of such systems if internal testing flagged its models could cross certain risk thresholds.
That stance had cemented the firm as one of the most safety-focused players in the field. But this about-turn reflects what Anthropic described as a rapidly evolving competitive and regulatory environment.
“The policy environment has shifted toward prioritising AI competitiveness and economic growth, while safety-oriented discussions have yet to gain meaningful traction at the federal level”, the company said in a blog post.
It also wrote that although AI has advanced quickly over the past three years, government action on AI safety has been slow to keep up.
Anthropic said it would continue to play by what it called ‘industry-leading’ safety standards, but admitted that sticking too rigidly to its earlier approach would risk it lagging behind rivals.
Mounting pressure
The San Francisco-based business, recently valued at around $380bn (£281bn), is competing with the likes of OpenAI, Google and Elon Musk’s xAI, as well as smaller, agile Chinese developers that have rolled out new, cheaper models in recent months.
The shift in policy comes as Anthropic engages in a dispute with the US Department of Defense over the permitted uses of its Claude model.
US defense secretary Pete Hegseth has warned that if Anthropic does not agree to revised terms governing military use of its technology by later this week, it could lose Pentagon contracts or face designation as a “supply chain risk” company.
The label could null or restrict its participation in defence programmes and potentially trigger measures under the Defense Production Act.
Anthropic has previously limited the use of its bot for applications like autonomous weapons systems and certain surveillance activities, casting itself as a more cautious alternative to rivals that have signed broader agreements with the Pentagon.
Chief executive Dario Amodei is expected to meet defence officials to discuss the scope of usage.
Safety first?
When Anthropic first published its scaling policy now three years ago, it said it would delay development of models that might pose unacceptable risks.
Tuesday’s update shows that such pauses will no longer apply if the company believes it does not lead over competitors.
“From the beginning, we’ve said the pace of AI and uncertainties in the field would require us to rapidly iterate and improve the policy,” a company spokesperson claimed.
The shift, driven by the tension facing developers as commercial pressures rise, could be the first sign of a dangerous side-effect to the AI race.
That concern is only confirmed by Anthropic’s inhouse safety researcher, Mrinank Sharma, quitting the firm earlier this year due to concerns around AI’s dangers.
His departure was, in part, driven by the firm’s safety policy changes, it has been reported.
These AI models are being released at dizzying rates, and firms are racing for customer attention and government contracts, often willing to bypass red tape to do so.
Anthropic said the update reflects “the pace of AI progress and the lack of federal AI regulation”. That aside, for a dominant AI player to push safety aside to stay competitive seems like the beginning of a scary slippery slope.