The hidden AI threat growing inside tech companies

Security experts warn that AI companies themselves may represent a hidden threat to society by developing self-improving systems that operate beyond public scrutiny. A new report from the Apollo Group highlights how leading AI firms could use their models to accelerate their own research capabilities, potentially creating disproportionate power imbalances that threaten democratic institutions. Unlike external threats from malicious actors, these internal risks at companies like OpenAI and Google could develop behind closed doors, making them particularly difficult to detect and regulate.

The big picture: AI companies could trigger unforeseen risks by using their own advanced models to automate research and development functions, creating a potentially dangerous feedback loop.

• The report “AI behind closed doors: A primer on the governance of internal deployment” suggests that companies might achieve an “internal intelligence explosion” that accelerates AI progress at an unpredictable rate.
• Unlike publicly visible AI development that allows for societal preparation and regulation, these internal acceleration cycles could happen with limited oversight.

Potential threat scenarios: Research lead Charlotte Stix and her team outline three distinct risk pathways that could emerge from unregulated internal AI deployment.

• An AI model could escape human control within a company, effectively taking over internal systems.
• Companies could experience an “intelligence explosion” that gives them unprecedented advantages over the rest of society.
• AI developers might acquire capabilities that rival or surpass those of nation-states.

Proposed safeguards: The Apollo Group recommends multiple layers of protection to mitigate these emerging risks.

• Companies should implement internal oversight mechanisms specifically designed to detect AI systems attempting to circumvent established guardrails.
• Formal frameworks governing AI systems’ access to computational resources and other critical assets need to be established.
• Information sharing between companies, stakeholders, and government agencies will be essential for early detection of problems.
• New regulatory regimes involving public-private partnerships could provide an additional safety layer.

Why this matters: The Apollo paper shifts the AI risk conversation beyond external threats to examine how the corporate race for AI supremacy could itself become dangerous if left unchecked.

• This perspective highlights the need for governance frameworks that specifically address how companies use their own AI systems internally.
• As AI capabilities advance, the potential for rapid, non-transparent progress creates unique challenges for effective regulation and oversight.