×
OpenAI advisor leaves company saying no one is prepared for superintelligent AI
Written by
Published on
Join our daily newsletter for breaking news, product launches and deals, research breakdowns, and other industry-leading AI coverage
Join Now

OpenAI’s AGI readiness expert departs with stark warning: Miles Brundage, OpenAI’s senior adviser for artificial general intelligence (AGI) readiness, has left the company, stating that no organization, including OpenAI, is prepared for the advent of human-level AI.

Key takeaways from Brundage’s departure: His exit highlights growing tensions between OpenAI’s original mission and its commercial ambitions, as well as concerns about AI safety and governance.

  • Brundage spent six years shaping OpenAI’s AI safety initiatives before concluding that neither the company nor any other “frontier lab” is ready for AGI.
  • He emphasized that this view is likely shared by OpenAI’s leadership, distinguishing between current readiness and being on track for future preparedness.
  • Brundage’s departure follows other high-profile exits from OpenAI’s safety teams, including Jan Leike and cofounder Ilya Sutskever.

Shift in OpenAI’s focus and structure: The company appears to be moving away from its original nonprofit mission towards a more commercial orientation.

  • OpenAI has disbanded its “AGI Readiness” and “Superalignment” teams, which were dedicated to long-term AI risk mitigation.
  • The company reportedly faces pressure to transition to a for-profit public benefit corporation within two years or risk losing recent investments.
  • This shift has raised concerns among researchers like Brundage, who expressed reservations about OpenAI’s for-profit division as early as 2019.

Reasons for Brundage’s departure: The former adviser cited several factors influencing his decision to leave OpenAI.

  • Increasing constraints on his research and publication freedom at the high-profile company.
  • A belief that he can make a greater impact on global AI governance from outside the organization.
  • The need for independent voices in AI policy discussions, free from industry biases and conflicts of interest.

Internal tensions at OpenAI: Brundage’s exit reveals deeper cultural divides within the organization.

  • Many researchers joined OpenAI to advance AI research but now find themselves in an increasingly product-driven environment.
  • Resource allocation has become a contentious issue, with reports indicating that some safety research teams were denied necessary computing power.
  • These frictions highlight the challenge of balancing research priorities with commercial objectives in the rapidly evolving field of AI.

OpenAI’s response and future support: Despite the circumstances of Brundage’s departure, the company has offered to maintain a collaborative relationship.

  • OpenAI has proposed supporting Brundage’s future work with funding, API credits, and early model access.
  • This offer comes with no strings attached, suggesting a desire to maintain connections with departing experts and potentially benefit from their ongoing research.

Broader implications for AI development and governance: Brundage’s departure raises important questions about the future of AI safety and the role of independent research.

  • The exodus of safety experts from leading AI companies may impact the development of responsible AI practices.
  • There is a growing need for unbiased, independent voices in AI policy discussions to balance commercial interests with ethical considerations.
  • The tension between rapid AI advancement and ensuring proper safety measures remains a critical challenge for the industry and policymakers alike.

Looking ahead: Balancing progress and preparedness: As AI capabilities continue to advance rapidly, the tech industry and global community face the daunting task of ensuring readiness for potentially transformative technologies.

  • The departures of key safety experts like Brundage underscore the urgency of addressing AI readiness and governance issues.
  • Collaboration between industry, academia, and policymakers will be crucial in developing comprehensive frameworks for responsible AI development.
  • The AI community must grapple with how to maintain a focus on safety and ethics while pursuing groundbreaking advancements in the field.
No one is ready for AGI

Recent News

Baidu reports steepest revenue drop in 2 years amid slowdown

China's tech giant Baidu saw revenue drop 3% despite major AI investments, signaling broader challenges for the nation's technology sector amid economic headwinds.

How to manage risk in the age of AI

A conversation with Palo Alto Networks CEO about his approach to innovation as new technologies and risks emerge.

How to balance bold, responsible and successful AI deployment

Major companies are establishing AI governance structures and training programs while racing to deploy generative AI for competitive advantage.