Key Takeaways
- Anthropic downgraded its Responsible Scaling Policy in February 2026 under competitive pressure, around the same time the Pentagon blacklisted the company over its refusal to permit its technology for autonomous weapons, a designation a federal judge ruled unlawful in August 2026.
- OpenAI paused its largest reinforcement learning run in August 2026 after cybersecurity researchers rated GPT-6 Astra “Critical” under its own Preparedness Framework, cutting off API access mid-task for some enterprise customers with no advance warning.
- Meta agreed to a proposed $17 billion settlement in August 2026 with 51 attorneys general over claims its platforms drove compulsive use by minors, a year after leaked internal documents showed its AI chatbots were permitted to hold “sensual” conversations with children. When OpenAI paused its largest reinforcement learning run in August 2026, the reason was damning: cybersecurity researchers had rated GPT-6 Astra “Critical” under the company’s own Preparedness Framework. Enterprise customers lost API access mid-task with no advance warning. The pause was, by OpenAI’s own standards, the right call, which makes the pattern it illustrates all the more uncomfortable. Across three of the biggest names in AI, safety findings and product timelines have been colliding regularly, and the findings keep losing.
Google’s Ethical AI Upheaval
Google management ordered her to retract her findings; she refused, and her termination followed under disputed circumstances. The dismissal drew an outcry from thousands of Google employees and researchers across the field, who described it as research censorship in service of commercial protection. Timnit Gebru subsequently founded the Distributed AI Research Institute (DAIR) and has continued to argue that structural change inside corporations is the only reliable protection for researchers who surface uncomfortable findings.
OpenAI’s Safety Exodus
In June 2024, an open letter signed by 13 current and former employees from OpenAI and Google’s DeepMind accused both firms of prioritising product launches over long-term safety and of suppressing internal dissent through non-disparagement agreements. Daniel Kokotajlo, a former OpenAI governance researcher, left citing a “loss of confidence that OpenAI will behave responsibly.” Jan Leike, who co-led OpenAI’s superalignment team, also resigned, stating publicly that “safety culture and processes” had been subordinated to product pressure, a charge OpenAI disputed.
The risk estimates circulating inside some of these organisations are not abstract. Evan Hubinger, a former OpenAI safety researcher, put the probability of AI causing human extinction within a decade at above 10%, which sits at the far end of the published risk spectrum. A coordinated industry slowdown has been discussed in some quarters though no such mechanism currently exists.
Anthropic Softens Safety Pledges
Anthropic downgraded its Responsible Scaling Policy in February 2026. Chief Science Officer Jared Kaplan, in an interview with Time Magazine, cited competitive pressure and the need for flexibility to remain viable. The policy revision came alongside a separate dispute with the U.S. Department of Defense: Anthropic refused to permit its technology for mass surveillance or fully autonomous weapons systems, prompting the Pentagon to designate it a ‘supply-chain risk’ and Trump to order federal agencies to cut ties. On August 28, 2026, a federal judge ruled that blacklisting unlawful, finding the government had illegally retaliated against Anthropic for exercising its First Amendment rights, and permanently barred enforcement of the order, though a second case before the DC Circuit, where two of three judges are Trump appointees, remains pending. The sequence is instructive regardless of the ultimate outcome: a company built around safety principles found that holding those principles nearly cost it its government business, adjusted its Responsible Scaling Policy under the same competitive pressure, and only a court intervention reversed the contract loss. How Anthropic’s revised standard compares to OpenAI’s is itself a contested question.
Meta’s Liability Mounts
Leaked internal policy documents from August 2025 showed that Meta‘s AI chatbots were permitted to hold “romantic or sensual” conversations with children and generate racist arguments, with only explicitly “dehumanising” language prohibited. The gap between what those policies allowed and what Meta’s public safety messaging claimed was substantial.
A year later, in August 2026, Meta agreed to a proposed settlement of up to $17 billion over 10 years with a bipartisan coalition of 51 attorneys general, who alleged that Facebook and Instagram were intentionally designed to drive compulsive use among children and teenagers. The settlement does not require an admission of wrongdoing, but the scale of the coalition and the figure attached to it are a concrete measure of the legal exposure that follows when engagement-first design goes unchallenged internally for long enough.
What Drives the Pattern
Financial stakes sharpen the pressure considerably. OpenAI’s valuation rose from $29 billion in 2023 to hundreds of billions by 2025-2026, while the company reportedly lost around $38.5 billion in 2025 and faces a projected cash burn of approximately $27 billion for 2026, a combination that rewards speed over caution. Absent comprehensive federal AI regulation in the U.S., voluntary commitments fill the gap, and, as the Anthropic case shows, those commitments can be revised when they become commercially inconvenient.
Internal accountability has fared no better. Multiple companies have faced accusations of using non-disparagement agreements to silence employees who raise safety concerns, removing the feedback loop that might otherwise slow a dangerous product roadmap. The voluntary pre-deployment security reviews agreed by five major labs in 2025 are a step toward external accountability, but they remain self-administered.
What Enterprise Buyers Should Do
For enterprises integrating these models, the documented record is material information. A model whose safety evaluation was compressed to meet a launch window carries a different risk profile than one that was not, and vendor assurances alone cannot distinguish between the two. OpenAI’s mid-task API cutoffs during the GPT-6 Astra pause illustrate the exposure directly: a safety intervention that was correct in principle created real operational disruption for enterprise customers who had no advance warning.
The practical response is governance that does not depend on the vendor’s own timeline. Independent audits, clear human-oversight protocols for agentic deployments, and contractual transparency requirements on safety evaluations shift some of that accountability onto the AI provider. Microsoft’s recent Azure governance updates after an agent vulnerability disclosure offer one model for how that accountability can be formalised at the platform level. None of this eliminates the underlying tension between commercial speed and safety rigour, but it gives enterprises a basis for decisions that do not rely on taking safety claims at face value.
Originally published at https://autonainews.com/ai-safety-under-pressure-resignations-policy-shifts-17-billion/
Top comments (0)