Open-source AI models missing from near-future AI scenarios

The neglect of open source AI in near-future scenario modeling creates dangerous blind spots for safety planning and risk assessment. As powerful AI models become increasingly accessible outside traditional corporate safeguards, security experts must reckon with the proliferation of capabilities that cannot be easily contained or controlled. Addressing these gaps is essential for developing realistic safety frameworks that account for how AI technology actually spreads in practice.

The big picture: Security researcher Andrew Dickson argues that current AI scenario models fail to adequately account for open source AI development, creating unrealistic forecasts that underestimate potential risks.

• Dickson believes this oversight presents a significant gap in strategic planning, especially as open source models continue to demonstrate advanced capabilities.
• While not opposing open source AI in principle, Dickson advocates for proactive consideration of its unique challenges before risks become extreme.

Why this matters: Open source AI presents fundamentally different risk profiles than proprietary systems, with fewer points of control and broader access—including to malicious actors.

• The democratization of advanced AI capabilities through open models creates security vulnerabilities that challenge conventional risk mitigation approaches.
• When powerful models are widely distributed, safety measures relying on centralized control become increasingly ineffective.

Key vulnerabilities: Expert-level virology capabilities are already appearing in frontier AI models, highlighting concerns about AI-assisted CBRN (Chemical, Biological, Radiological, Nuclear) risks.

• Open models are already being deployed in combat scenarios, including drone operations in Ukraine.
• The combination of wide access and advanced capabilities creates opportunities for reckless experimentation and intentional misuse.

Potential threat multipliers: “Slowdown” scenarios in AI development may counterintuitively increase certain risks rather than mitigate them.

• Longer waiting periods between major capability advances could provide more time for proliferation of existing dangerous capabilities.
• The article warns about potential high-profile drone assassinations and other dangerous applications that don’t require superintelligent AI.

Between the lines: While Dickson values open source software as “one of humanity’s most important and valuable public goods,” his analysis suggests a tension between innovation benefits and security concerns that the AI safety community has not fully confronted.

• The call for more realistic scenario planning reflects growing concerns that current regulatory frameworks may be addressing yesterday’s problems rather than tomorrow’s threats.
• The article functions as part of a broader examination of difficult tradeoffs inherent in public access to increasingly powerful AI models.