New research evaluates AI risk by examining AI models’ cognitive capabilities

AI researchers propose a new framework for analyzing artificial intelligence risks by examining cognitive capabilities rather than specific tasks or behaviors.

Core concept and rationale: The approach breaks down AI systems into three fundamental components – knowledge, physical capabilities, and cognitive capabilities – with a particular focus on cognitive abilities as the key enabler of potential risks.

• Knowledge alone cannot create risk without processing capability
• Physical capabilities are relatively straightforward to monitor and control
• Cognitive capabilities serve as prerequisites for almost all potential risks
• Current methods struggle to identify which cognitive capabilities are necessary for dangerous tasks

Proposed methodology: The framework suggests creating a systematic mapping between cognitive capabilities and potential risks to enable proactive analysis.

• Develop comprehensive catalogs of both potential risks and cognitive capabilities
• Analyze how different combinations of capabilities could enable various risks
• Use either a risk-first or capabilities-first approach, with the latter preferred to reduce confirmation bias
• Framework allows for continuous updates as new capabilities or risks emerge

Implementation approaches: Several strategies are proposed to make the analysis manageable and effective.

• Dedicated research teams could be assembled to conduct systematic analysis
• AI-powered analysis pipelines could help process large amounts of data
• Crowdsourcing to the AI safety community could distribute the workload
• Even if analysis proves difficult, understanding the complexity would be valuable

Practical applications: The framework enables several concrete use cases for AI safety.

• Development of early warning systems based on capability combinations
• Optimization of AI training to minimize dangerous capabilities
• Creation of targeted evaluation methods for specific capability combinations
• Better understanding of AI scaling laws and future developments

Critical next steps: While comprehensive analysis would be ideal, initial focus areas have been identified.

• Prioritize high-impact risk categories
• Focus on core cognitive capabilities relevant to current AI systems
• Examine specific combinations most likely to enable critical risks
• Build structured approaches for ongoing monitoring and assessment

Looking ahead: This framework represents an important shift in AI risk assessment, moving from task-based analysis to a more fundamental understanding of how cognitive capabilities interact to create potential risks. However, successful implementation will require significant coordination across the AI research community and careful validation of the framework’s assumptions about capability combinations.