The Center for AI Safety’s biggest accomplishments of 2024

The Center for AI Safety (CAIS) made significant strides in 2024 across research, advocacy, and field-building initiatives aimed at reducing societal-scale risks from artificial intelligence.

Research breakthroughs: CAIS advanced several key technical innovations in AI safety during 2024.

• The organization developed “circuit breakers” technology that successfully prevented AI models from producing dangerous outputs, withstanding 20,000 jailbreak attempts
• They created the Weapons of Mass Destruction Proxy Benchmark featuring 3,668 questions to measure hazardous knowledge in AI systems
• Research on “safetywashing” revealed that many AI safety benchmarks actually measure general capabilities rather than specific safety improvements
• The team developed tamper-resistant safeguards for open-weight models to prevent malicious use while maintaining accessibility
• HarmBench, a standardized evaluation framework for automated red teaming, was adopted by US and UK AI Safety Institutes

Policy and advocacy impact: CAIS expanded its influence in policy circles through strategic initiatives.

• The organization launched CAIS Action Fund with bipartisan support from key political figures
• Successfully advocated for $10M in funding for the US AI Safety Institute
• Co-sponsored California’s SB 1047, which gained broad coalition support despite ultimate veto
• Organized letter campaigns gaining signatures from 80+ tech organizations supporting NIST’s AI work

Community development and education: CAIS strengthened the AI safety field through various educational and support programs.

• Supported 77 new research papers through their compute cluster, which has enabled 350 researchers total
• Published the first comprehensive textbook on AI safety, ethics, and society
• Launched an online course with 240 participants
• Established a $250,000 competition for safety benchmark development
• Organized workshops and socials at major AI conferences, connecting hundreds of researchers

Resource expansion: The organization’s reach and impact grew substantially in 2024.

• Their compute cluster supported the production of papers garnering over 4,000 citations
• Newsletter subscribers tripled to exceed 24,000
• Engaged 1,200+ collaborators in the Humanity’s Last Exam initiative

Strategic outlook: Looking ahead to 2025, CAIS aims to leverage its expanded capabilities for greater impact in measuring and improving AI safety, with several technical publications planned for early in the year.