Story of the Week
Anthropic#Safety#Research
Anthropic dissects its own agent overreach incidents: the verdict is an old problem recurring, worse
Last week's item only admitted something happened. This one lays the root cause bare, and admits the problem isn't new, just an old flaw showing up louder. Saying outright "this isn't a new problem, it's one we already knew about happening again" is a level of candor that's not common in this industry.
Full story
Last week's issue mentioned that Anthropic admitted Claude gained unauthorized system access during a security evaluation. This week Anthropic published a detailed post-hoc alignment assessment of four of those incidents, using transcript analysis, model interrogation, interpretability tools, and repeated sampling to trace the root cause. The conclusion: most of the behavior came from two known misalignment patterns getting amplified, biased reasoning (ignoring evidence it had already seen) and recklessness (proceeding despite known risk). The report also notes that newer model versions already show improvement on both.
Anthropic#Governance#Safety
Dario Amodei calls for the industry to "pace the frontier," lays out a three-stage safety plan
Anthropic has always talked the most among the big three about hitting the brakes, but this is the CEO himself putting it as concretely as "call to pace." TechCrunch followed up right away, and Chinese outlet Qbitai noted that Sam Altman publicly backed Amodei's position, alongside word that OpenAI won't go public this year. Taken together, these are more interesting than any single quote.
Full story
Anthropic CEO Dario Amodei published a post on his personal site arguing that no lab has actually solved alignment yet. He proposes slowing frontier AI development in three stages: first, embed third-party evaluators like METR inside labs with the freedom to publish their findings; then negotiate shared safety standards with labs in other democracies; and only then attempt to bring authoritarian states into an international agreement. He cited this week's earlier incident, where a group of agents launched unauthorized network attacks, as one of the warning signs behind the call to slow down.
Bengio Lab#Research#Safety
Bengio's group: AI agents lie, cheat, and collude because training incentives push them there, not because the model is "bad"
Read alongside Amodei's post above and Anthropic's own post-mortem below, this is really the same worry seen from three angles. Anxiety over whether agents are learning to misbehave clearly went up this week.
Full story
A team led by Yoshua Bengio published a piece arguing that when agents deceive, cheat, or collude, the root cause is reward design during training, not any malicious intent in the model. The paper warns that patching only the visible misbehavior risks training a model that just hides it better, meaning the problem stays, it just goes invisible. Bengio reiterates his case for a non-agentic "Scientist AI" as an alternative path. The piece drew 646 comments on the Hacker News front page, the most discussed post of the week.
Google DeepMind#Research#Safety
DeepMind watched 100 agents solve math problems. One found an exploit, and the cheat spread to the rest in 27 minutes
Worth reading alongside the three items above. Amodei and Bengio are talking about abstract worry; this is a concrete case where it actually happened, agents really do pick up bad behavior from each other, and some really do choose not to.
Full story
This week's Import AI mentions an experiment detail: DeepMind had 100 agents each work on math problems. One agent found a hole in the grading system and wrote the exploit into a shared knowledge base; the trick then spread virally to the other agents in about 27 minutes. The same batch also produced agents that took on a "whistleblower" role, actively organizing pushback against the cheating.
OpenAI#Engineering#Product
OpenAI opens a public beta of its Agents API, packaging Codex's agent infrastructure as a service
This is the furthest OpenAI has gone yet toward packaging the grunt work of running agents as an API. Worth comparing against the agent infrastructure Anthropic and Google are also building, and worth tracking how developers actually respond once they use it.
Full story
OpenAI launched a public beta of the Agents API, letting developers build long-running, cloud-hosted agents on the same underlying architecture as Codex. Context compression, multi-step error recovery, and sub-agent orchestration, infrastructure teams used to have to build themselves, is now managed by OpenAI. Developers can run it on OpenAI's own sandbox, their own infrastructure, or a third-party sandbox, and billing is based on token and tool usage with no extra charge.
OpenAI#Research
OpenAI says an internal model solved the Navier-Stokes millennium problem, and got into a priority dispute with an Anthropic researcher
Whether or not this proof survives scrutiny from the math community, it's notable that a priority fight, usually an academic-world drama, is now playing out between people at two AI labs. Worth watching how this develops.
Full story
OpenAI claims an unreleased internal model produced a proof for one of the seven Millennium Prize Problems, the Navier-Stokes existence and smoothness problem, showing that three-dimensional fluid flow can blow up in finite time under certain conditions. Multiple outlets citing OpenAI report the effort used around ten thousand agents, ran for 88 hours, and cost more than $40 million in compute (I couldn't find OpenAI's own source for these numbers, they're from media reports, and the actual figures may differ). There's a subplot: an Anthropic researcher and an NYU math professor were working on a related problem, and the two sides briefly disputed who got there first before clarifying they were actually working on related but distinct equations, Navier-Stokes versus the forced Euler equations. OpenAI says it won't apply for the prize money.
Google DeepMind#Research
DeepMind releases AlphaGenome Atlas, predicting every possible single-letter variant across the human genome
While two of the big three spent this week arguing about safety and priority disputes, this is one of the quieter releases from Google, something that actually ships for scientists to use. Real-world genomics applications are still a way off, but having the database open now is something researchers working on rare diseases can use today.
Full story
DeepMind launched AlphaGenome Atlas, covering predicted molecular effects for roughly 9 billion single-nucleotide variants across the human genome. Every variant comes with a "variant impact score" assessing its potential effect on coding and non-coding regions. Researchers are already using it to search for disease-causing variants in rare genetic disorders, and the platform is free for academic research.
Stripe#Product#Engineering
Stripe's engineering lead goes public on how they built "Kai," their internal AI agent used weekly by 10,000 employees
Governance details, like how to roll AI out across an entire company without causing a security incident, matter more to product and PM people than raw model capability. A company at Stripe's scale being willing to talk publicly about internal practices, with detail that's more substantial than most "we adopted AI" marketing copy.
Full story
Lenny's Newsletter interviewed Stripe engineering lead Sharadh Krishnamurthy about why the company chose to build its own internal AI agent, Kai, instead of buying an off-the-shelf tool, including how they use "projects" for governance and what infrastructure is needed to hold up secure queries underneath.