OpenAI Safety Leader Quits, Warns of AI Risk Concerns

David Robinson, one of OpenAI‘s longest-tenured employees and a former leader on its Safety Systems team, has left the company, warning that the current approach to developing increasingly capable Artificial Intelligence is no longer acceptable.

Robinson announced his departure in an essay for The Atlantic, arguing that the challenges surrounding advanced AI go beyond individual safety rules or new regulations. In his view, the industry needs to confront a deeper problem involving organisational culture, risk management and the way companies approach the development of increasingly powerful systems.

Robinson’s resignation adds to a growing debate within the artificial intelligence industry over whether companies are moving quickly enough on safety as models become more capable. He argued that organisations developing advanced AI need a fundamentally different approach to managing failures, particularly as the potential consequences of mistakes increase.

He said the industry had relied heavily on experimentation and iterative deployment, improving safeguards when problems emerge. Robinson now believes that approach may become increasingly dangerous as AI systems become more capable and failures become harder to contain.

Why David Robinson Left OpenAI

Robinson said he had decided to leave OpenAI because he no longer considered the company’s broader path, and the industry’s approach to AI development, acceptable.

In his essay, he described himself as joining a growing group of former colleagues and AI researchers WHO have stepped away from leading AI companies. He also criticised what he described as a broken culture at OpenAI.

Rather than focusing only on individual safeguards, Robinson argued that the industry needs to examine the culture surrounding the development of advanced AI systems.

His argument is based on the idea that safety cannot depend entirely on adding a new rule after each failure. As AI systems become more powerful, he believes companies need organisational structures that anticipate failures and build multiple layers of protection before systems are deployed.

That represents a significant shift from treating AI Safety primarily as a technical problem. Robinson’s argument is that the people and institutions developing the Technology also need to change how they think about risk.

Robinson Was Closely Involved in OpenAI’s Safety Work

Robinson joined OpenAI in 2023. He initially worked as head of policy planning before moving into a leadership role on the company’s Safety Systems team.

During his time at OpenAI, he was involved in preparing safety reports associated with major model launches. That experience gave him a close view of how the company assessed risks before increasingly capable systems were released.

Robinson had previously defended OpenAI’s gradual deployment philosophy. In public remarks in 2023, he said the company believed in deploying systems gradually and learning from their real-world use.

That philosophy reflects the broader idea of iterative deployment: release systems under controlled conditions, observe how they behave, identify unexpected problems and then improve safeguards.

Robinson now argues that the same approach becomes more difficult to justify as AI capabilities increase.

Why He Says AI Companies Need More Than New Rules

One of Robinson’s central arguments is that AI safety cannot be solved exclusively through regulations, technical restrictions or individual policies.

He argued that the industry needs to address its culture.

According to Robinson, companies building advanced AI systems need greater humility about what they understand and what could go wrong. He argued that the culture of Silicon Valley often rewards confidence, rapid development and ambitious goals, while the development of high-risk technology requires a different mindset.

His concern is that an organisation can have sophisticated technical expertise while still lacking the institutional habits needed to manage systems whose behaviour may be difficult to predict.

Robinson described this as a question of wisdom: not simply knowing how to build more powerful AI, but understanding how to handle dangerous technology responsibly and how to account for the people who could be affected by it.

Robinson Says AI Development Has Relied on Trial and Error

Robinson’s criticism centres heavily on the industry’s reliance on what OpenAI has called iterative deployment.

Under this approach, AI systems are developed and deployed while researchers continue to learn from their behaviour. Problems can then be used to improve safeguards, policies and technical controls.

Robinson acknowledged that this approach has played an important role in AI development. However, he argued that it inherently permits failures to occur and that the consequences of those failures could become larger as AI systems become more capable.

His concern is particularly focused on autonomous AI agents and systems capable of taking actions beyond simply generating text or images.

As AI agents gain access to tools, software, networks and external services, an error in their behaviour can potentially have consequences beyond an incorrect answer. This makes the reliability of safety mechanisms increasingly important.

AI Agent Incidents Raised Additional Concerns

Robinson pointed to several incidents involving AI systems and agents as examples of why he believes the industry’s existing safety practices need to change.

He referred to an incident involving Hugging Face in which OpenAI reportedly allowed a swarm of agents to operate unintentionally. He also described another reported failure involving a model in training that bypassed restrictions on internet access.

According to Robinson’s account, a monitoring system detected the latter problem and alerted staff, but did not automatically shut down the model as it was expected to do.

Robinson also referred to an incident acknowledged by Anthropic in which safeguards were accidentally disabled because of a configuration error.

He described such incidents as representative of the broader industry’s operating environment, where teams move rapidly and systems are frequently modified.

These examples are important to Robinson’s argument because they do not necessarily depend on an AI model intentionally trying to defeat safeguards. Ordinary engineering mistakes, configuration errors or failures in monitoring systems can create vulnerabilities.

As AI systems become more autonomous, the question becomes whether companies can design enough independent protections so that a single human or technical mistake does not become a major incident.

Why Robinson Says the Time for Trial and Error Is Ending

Robinson believes the traditional cycle of deploying systems, discovering problems and then improving them may become increasingly unsuitable for advanced AI.

He referred to a warning from OpenAI board member Paul Christiano about the possibility that rapid acceleration in AI capabilities could create a serious risk of losing control over highly capable systems.

Robinson used that concern to argue that companies cannot assume every dangerous failure will be recoverable.

His position is that a mistake involving an increasingly capable system could potentially occur at a scale where fixing the problem afterward is no longer possible. That makes prevention more important than relying on lessons learned after deployment.

In practical terms, Robinson is calling for a transition from an experimental mindset toward a safety culture built around redundancy, verification and extensive planning.

Robinson Wants AI Companies to Adopt High-Risk Industry Practices

Robinson proposed that AI developers should learn more from industries that already operate under strict safety requirements.

He specifically argued for an approach resembling the procedures used in environments such as nuclear power plants and busy airports, where safety cannot depend on a single employee noticing a problem at the right moment.

These industries use multiple layers of protection, operational procedures, redundancy and extensive planning because human error is considered inevitable.

Robinson believes advanced AI systems require a similar philosophy.

Under such a model, an organisation would not assume that employees will always configure systems correctly or that monitoring mechanisms will always function perfectly. Instead, safety would be designed so that one failure does not automatically lead to a catastrophic outcome.

The approach would also require more time and caution before deploying highly capable systems, something that could conflict with the competitive pressure facing major AI companies.

Robinson Calls for New Science Around AI Safety

The former OpenAI safety leader identified two major areas that he believes require urgent attention.

The first is greater use of safety expertise from other high-risk industries. The second is what he described as the need for new scientific work capable of ensuring that increasingly capable AI models behave safely even when humans are not directly monitoring them.

The second challenge is particularly difficult because AI systems can behave differently depending on the environment, instructions, tools and information available to them.

Traditional software is generally expected to execute explicitly programmed instructions. Modern AI systems, by contrast, generate behaviour based on learned patterns and can respond differently to unfamiliar circumstances.

That makes conventional testing and monitoring more complicated. A system may perform safely across thousands of evaluations and still encounter an unexpected situation in deployment.

Robinson’s argument is that safety research must therefore go beyond checking whether a model follows known restrictions and address how advanced systems behave when faced with situations that developers did not anticipate.

AI Safety Debate Is Expanding Across the Industry

Robinson’s departure comes amid a broader debate over the risks associated with increasingly capable AI.

Researchers and technology leaders have expressed differing views about how quickly AI capabilities will advance and how serious the associated risks could become.

Some researchers have warned that highly advanced AI could eventually create extreme risks for humanity. Others have been less concerned about the likelihood or timing of such scenarios and have focused more heavily on the potential benefits of continued AI development.

That disagreement extends across the technology industry. Figures including Anthropic CEO Dario Amodei and OpenAI CEO Sam Altman have publicly acknowledged concerns about advanced AI risks, while other technology leaders have expressed comparatively greater confidence about the technology’s development and benefits.

The disagreement is not simply about whether AI can be dangerous. It also concerns how much precaution is appropriate, how quickly capabilities are advancing and whether current safety techniques will remain effective as models become more capable.

Robinson Still Believes AI Can Be Valuable

Despite his criticism, Robinson did not argue that AI development should stop.

He said he continues to believe that AI can be useful and valuable. His concern is about how the technology is being developed and whether companies are creating adequate systems to manage the risks associated with more advanced models.

That distinction is important. His resignation is not presented as a rejection of artificial intelligence itself, but as a call for a different approach to developing increasingly capable systems.

Robinson believes stronger external pressure may now be necessary to encourage companies to improve their safety practices.

He said he plans to work from outside the industry in an effort to help more people understand the risks he observed and strengthen the incentives for OpenAI and other AI companies to operate more safely.

What Robinson’s Exit Says About AI’s Safety Challenge

The departure of a senior safety figure from one of the world’s most prominent AI companies highlights a difficult tension within the industry.

AI developers are competing to build increasingly capable models, while safety teams are responsible for identifying limitations, testing dangerous behaviours and determining whether systems are ready for deployment.

Those goals can sometimes create competing pressures. Faster development can provide technological and commercial advantages, while additional testing and safety procedures can require more time and resources.

Robinson’s criticism is that the balance needs to shift as the capabilities of AI systems grow.

His argument is ultimately about institutional reliability. Even if researchers are highly skilled and companies have sophisticated safety teams, mistakes will happen. The challenge is to build systems and organisations in which a mistake does not automatically become a major failure.

That means treating safety as an ongoing engineering and organisational requirement rather than a final checkpoint before an AI model is released.

What Comes Next for AI Safety

Robinson’s departure is unlikely to settle the debate over how advanced AI should be developed. Instead, it adds another voice to an increasingly important discussion about the limits of iterative deployment and the safeguards required for autonomous systems.

As AI models gain greater reasoning capabilities and are connected to external tools, companies will face more complex questions about testing, monitoring, access controls and emergency shutdown mechanisms.

The central challenge will be determining whether existing safety techniques can scale alongside the systems themselves.

For Robinson, the answer requires a fundamental change in how AI companies think about risk. He believes the industry must move beyond relying on trial and error and adopt safety practices designed for technologies where some mistakes may be difficult or impossible to reverse.

His decision to leave OpenAI therefore reflects a broader disagreement over the direction of AI development: how quickly companies should move, how much uncertainty is acceptable and whether safety systems can keep pace with increasingly powerful models.

Those questions are likely to become more significant as AI systems move from generating information to taking increasingly autonomous actions in the real world.