OpenAI advisor leaves company saying no one is prepared for superintelligent AI

OpenAI’s AGI readiness expert departs with stark warning: Miles Brundage, OpenAI’s senior adviser for artificial general intelligence (AGI) readiness, has left the company, stating that no organization, including OpenAI, is prepared for the advent of human-level AI.

Key takeaways from Brundage’s departure: His exit highlights growing tensions between OpenAI’s original mission and its commercial ambitions, as well as concerns about AI safety and governance.

• Brundage spent six years shaping OpenAI’s AI safety initiatives before concluding that neither the company nor any other “frontier lab” is ready for AGI.
• He emphasized that this view is likely shared by OpenAI’s leadership, distinguishing between current readiness and being on track for future preparedness.
• Brundage’s departure follows other high-profile exits from OpenAI’s safety teams, including Jan Leike and cofounder Ilya Sutskever.

Shift in OpenAI’s focus and structure: The company appears to be moving away from its original nonprofit mission towards a more commercial orientation.

• OpenAI has disbanded its “AGI Readiness” and “Superalignment” teams, which were dedicated to long-term AI risk mitigation.
• The company reportedly faces pressure to transition to a for-profit public benefit corporation within two years or risk losing recent investments.
• This shift has raised concerns among researchers like Brundage, who expressed reservations about OpenAI’s for-profit division as early as 2019.

Reasons for Brundage’s departure: The former adviser cited several factors influencing his decision to leave OpenAI.

• Increasing constraints on his research and publication freedom at the high-profile company.
• A belief that he can make a greater impact on global AI governance from outside the organization.
• The need for independent voices in AI policy discussions, free from industry biases and conflicts of interest.

Internal tensions at OpenAI: Brundage’s exit reveals deeper cultural divides within the organization.

• Many researchers joined OpenAI to advance AI research but now find themselves in an increasingly product-driven environment.
• Resource allocation has become a contentious issue, with reports indicating that some safety research teams were denied necessary computing power.
• These frictions highlight the challenge of balancing research priorities with commercial objectives in the rapidly evolving field of AI.

OpenAI’s response and future support: Despite the circumstances of Brundage’s departure, the company has offered to maintain a collaborative relationship.

• OpenAI has proposed supporting Brundage’s future work with funding, API credits, and early model access.
• This offer comes with no strings attached, suggesting a desire to maintain connections with departing experts and potentially benefit from their ongoing research.

Broader implications for AI development and governance: Brundage’s departure raises important questions about the future of AI safety and the role of independent research.

• The exodus of safety experts from leading AI companies may impact the development of responsible AI practices.
• There is a growing need for unbiased, independent voices in AI policy discussions to balance commercial interests with ethical considerations.
• The tension between rapid AI advancement and ensuring proper safety measures remains a critical challenge for the industry and policymakers alike.

Looking ahead: Balancing progress and preparedness: As AI capabilities continue to advance rapidly, the tech industry and global community face the daunting task of ensuring readiness for potentially transformative technologies.

• The departures of key safety experts like Brundage underscore the urgency of addressing AI readiness and governance issues.
• Collaboration between industry, academia, and policymakers will be crucial in developing comprehensive frameworks for responsible AI development.
• The AI community must grapple with how to maintain a focus on safety and ethics while pursuing groundbreaking advancements in the field.