New initiative aims to map universal human values to AI safety benchmarks

A new research initiative aims to develop AI safety benchmark environments that incorporate universal human values, led by Roland Pihlakas as part of AI Safety Camp 10.

The core concept: The project seeks to map universal human values to concrete AI safety concepts and create testing environments that can evaluate AI systems’ alignment with these values.

• The research acknowledges fundamental asymmetries between AI and human cooperation, particularly in how goals can be programmed into AI but not humans
• The initiative builds upon existing anthropological research on cross-cultural human values
• The project will utilize multi-agent, multi-objective environments to test AI systems

Key technical components: The implementation will leverage an extended version of DeepMind‘s gridworlds framework, enhanced for multi-agent and multi-objective scenarios.

• The framework is compatible with industry-standard PettingZoo and Gym APIs
• Multiple existing benchmarks have already validated the framework’s effectiveness
• The system allows for both simple gridworld environments and potential integration with language models

Implementation approach: The project will follow a structured development process to ensure comprehensive coverage of human values.

• Step 1: Map universal human values to specific AI safety concepts
• Step 2: Design relevant benchmark environments
• Step 3: Implement environments using the extended framework
• Step 4: Test and validate using standard reinforcement learning algorithms
• Step 5: Document findings and prepare academic publications

Technical considerations: The project emphasizes balancing multiple human values rather than simple trade-offs.

• Non-linear utility functions will be used to transform rewards before summation
• The approach acknowledges that humans prefer balanced outcomes across objectives
• Economic concepts like diminishing returns and marginal utility will be incorporated

Risk mitigation: The project focuses on benchmark development rather than advancing AI capabilities.

• The emphasis remains on outer alignment while considering inner alignment implications
• Multiple objectives may help prevent overfitting to single random objectives
• The framework allows for controlled testing environments to minimize unintended consequences

Looking ahead: The varying ambition levels of the project demonstrate its scalability and potential impact on AI safety testing standards.

• The most ambitious version aims for comprehensive value mapping and widespread adoption
• The minimum viable product would map select values and develop proof-of-concept environments
• All outcomes will contribute to foundational work in AI safety benchmarking