News/Superintelligence
Meta’s AI chief predicts LLMs will be obsolete within 5 years
Yann LeCun, Meta's Chief AI Scientist and one of AI's foundational figures, has delivered a stark verdict on the future of Large Language Models (LLMs), predicting their obsolescence within five years. His assessment carries significant weight in the AI community, where debates about current limitations and future architectures are reshaping development priorities across the field. LeCun's research points to a fundamental shift in how intelligent systems should be designed, moving beyond the statistical pattern-matching that powers today's most popular AI systems. The big picture: LeCun argues that current LLMs will be largely obsolete within five years due to fundamental limitations...
read Apr 13, 2025The paradoxical strategy dilemma in AI governance: why both sides may be wrong
The PauseAI versus e/acc debate reveals a paradoxical strategy dilemma in AI governance, where each movement might better achieve its goals by adopting its opponent's tactics. This analysis illuminates how public sentiment, rather than technical arguments, ultimately drives policy decisions around advanced technologies—suggesting that both accelerationists and safety advocates may be undermining their own long-term objectives through their current approaches. The big picture: The AI development debate features two opposing camps—PauseAI advocates for slowing development while effective accelerationists (e/acc) push for rapid advancement—yet both sides may be working against their stated interests. Public sentiment, not technical arguments, ultimately determines AI...
read Apr 13, 2025Study: Advanced AI models now pass Turing test, fooling human judges
AI systems have reached a milestone in human-machine interaction with LLMs now being able to fool human judges in formal Turing test scenarios. New research shows that advanced language models can not only match human conversational abilities but in some cases exceed them—signaling a significant advancement in artificial intelligence that could reshape our understanding of machine intelligence and accelerate the integration of convincingly human-like AI systems into society. The big picture: For the first time, large language models have formally passed a standard Turing test, with GPT-4.5 being identified as human more often than actual human participants. Researchers evaluated four...
read Apr 13, 2025Virtue-driven AI might avoid dangerous power-seeking behaviors unlike goal-focused systems
The question of instrumental convergence for virtue-driven AI agents introduces a fascinating counterpoint to traditional AI alignment concerns. While conventional wisdom suggests that almost any goal-driven AI might pursue power acquisition as an instrumental strategy, virtue-based motivation frameworks could potentially circumvent these dangerous convergent behaviors. This distinction raises important considerations for AI alignment researchers who seek alternatives to purely consequentialist AI architectures that might inherently pose existential risks. The big picture: Instrumental convergence theory suggests most goal-driven AIs will pursue similar subgoals like power acquisition, but this may not apply to AIs motivated by virtues rather than specific outcomes. In...
read Apr 13, 2025Report: Global regulators warn AI could enable unprecedented market manipulation
Global financial regulators are sounding the alarm about artificial intelligence's potential to destabilize capital markets through unprecedented forms of market manipulation and systemic risk. The International Organization of Securities Commissions (IOSCO) has identified critical vulnerabilities where AI could enable sophisticated market abuses that current regulatory frameworks aren't equipped to detect or prevent. This warning is particularly significant for AI safety researchers concerned about superintelligence scenarios where control of financial markets could be a pathway to catastrophic outcomes. The big picture: IOSCO's comprehensive report outlines how AI technologies present novel risks to global financial market integrity through their potential to enable...
read Apr 12, 2025New AI architectures hide reasoning processes, raising transparency concerns
Emerging language model architectures are challenging the transparency of AI reasoning systems, potentially creating a significant tension between performance and interpretability in the field. As multiple research groups develop novel architectures that move reasoning into "latent" spaces hidden from human observation, they risk undermining one of the most valuable tools alignment researchers currently use to understand and guide AI systems: legible Chain-of-Thought (CoT) reasoning. The big picture: Recent proposals for language model architectures like Huginn, COCONUT, and Mercury prioritize reasoning performance by allowing models to perform calculations in hidden spaces at the expense of transparency. These new approaches shift reasoning...
read Apr 12, 2025New novel explores life after humanity discovers it exists in a simulation
Daryl Gregory's new novel "When We Were Real" explores the profound consequences of humanity discovering it exists within a simulation, providing a thought-provoking examination of consciousness, free will, and reality itself. The book presents a unique angle on simulation theory by focusing not on the initial revelation but on how people adapt to life after learning their entire existence is artificial, challenging readers to contemplate what it means to be "real" in an increasingly AI-driven world. The big picture: Gregory's thriller follows a Canterbury Tour bus traversing America to visit "Impossibles" – physics-defying geographical anomalies that appeared after humanity learned...
read Apr 12, 2025Analysis warns AI might develop human-like evil tendencies beyond rational goals
AI safety research is increasingly examining the potential for artificial intelligence to develop complex, human-like evil tendencies rather than just sterile, goal-focused harmful behaviors. Jacob Griffith's analysis explores the distinction between "messy" and "clean" goal-directedness in AI systems and how understanding human evil—particularly genocides—might illuminate more nuanced AI risks that current safety frameworks may overlook. The big picture: Griffith draws from theories by Corin Katzke and Joseph Carlsmith to examine how AI systems might develop power-seeking tendencies that mirror the illogical, emotional aspects of human evil rather than purely instrumental power acquisition. Traditional AI safety concerns often focus on "clean"...
read Apr 12, 2025Center on Long-Term Risk opens fellowship for AI safety researchers focused on reducing suffering risks
The Center on Long-Term Risk is recruiting researchers for its Summer Research Fellowship focused on empirical AI safety work aimed at reducing suffering risks in the far future. The eight-week program offers mentorship, collaboration opportunities, and integration with CLR's research team, with applications due by April 15th. This year's fellowship notably shifts focus toward empirical AI safety research while seeking candidates who might transition to full-time s-risk research. The big picture: CLR's 2025 Summer Research Fellowship targets AI safety researchers interested in reducing long-term suffering risks through an eight-week collaborative research program. Fellows will work on independent projects while receiving...
read Apr 12, 2025How crystallized and fluid intelligence shape AI’s path to superintelligence
Understanding the relationship between different types of intelligence is vital for comprehending how both human cognition and artificial intelligence systems develop advanced problem-solving abilities. This exploration of crystallized versus fluid intelligence offers critical insights into how AI systems might recursively improve their capabilities, potentially leading to superintelligent systems that combine vast knowledge bases with powerful reasoning abilities. The big picture: Intelligence operates across at least two distinct dimensions—crystallized intelligence (accumulated knowledge) and fluid intelligence (flexible reasoning)—creating a framework for understanding how advanced AI systems might evolve. Crystallized intelligence represents performance achievable with minimal computational effort, drawing on stored knowledge and...
read Apr 12, 2025Study: AI research automation could trigger software intelligence explosion
New research suggests AI-powered research and development tools could trigger a software intelligence explosion, potentially creating a self-reinforcing cycle of increasingly rapid AI advancement. This possibility challenges traditional assumptions about the limitations of AI progress, highlighting how software improvements alone might drive exponential capability gains even without hardware advances—presenting both profound opportunities and risks for humanity's future. The big picture: AI systems are increasingly being used to accelerate AI research itself, with current tools assisting in coding, research analysis, and data generation tasks that could eventually evolve into fully automated AI development. These specialized systems, termed "AI Systems for AI...
read Apr 12, 2025Why superintelligent AI will still struggle with everyday problems
Computational complexity theory reveals a fundamental limit that even superintelligent AI systems will face, as certain everyday problems remain inherently difficult to solve optimally regardless of intelligence level. These NP-hard problems—ranging from scheduling meetings to planning vacations—represent a class of challenges where finding the perfect solution is computationally expensive, forcing both humans and AI to rely on "good enough" approximations rather than guaranteed optimal answers. The big picture: Despite rapid advances in AI capabilities, fundamental computational limits mean superintelligent systems will still struggle with certain common problems that are mathematically proven to resist efficient solutions. Why this matters: Understanding computational...
read Apr 11, 2025Anthropic finds AI models gaining college-kid-level cybersecurity and bioweapon skills
Anthropic's frontier AI red team reveals concerning advances in cybersecurity and biological capabilities, highlighting how AI models are rapidly acquiring skills that could pose national security risks. These early warning signs emerge from a year-long assessment across four model releases, providing crucial insights into both current limitations and future threats as AI continues to develop potentially dangerous dual-use capabilities. The big picture: Anthropic's assessment finds that while frontier AI models don't yet pose substantial national security risks, they're displaying alarming progress in dual-use capabilities that warrant close monitoring. Current models are approaching undergraduate-level skills in cybersecurity and demonstrate expert-level knowledge...
read Apr 11, 2025New bio-computer combines living neurons with silicon chips for AI breakthrough
A groundbreaking bio-computer merging living neurons with silicon chips has emerged as a potential milestone in AI and neuromorphic computing. Developed by Australia's Cortical Labs, the CL1 bio-computer combines synthetic living brain neurons with artificial neural networks, creating a novel approach that could transform our understanding of both biological and artificial intelligence while raising profound ethical questions about the boundary between machine cognition and living systems. The big picture: The CL1 bio-computer from Cortical Labs represents a significant advancement in neuromorphic computing by integrating lab-grown living neurons with traditional silicon chips for $35,000. The system employs a Biological Intelligence Operating...
read Apr 11, 2025How AI alignment techniques differ from traditional reinforcement learning
Reinforcement learning has evolved from teaching AI to master video games into a sophisticated approach for aligning large language models with human values and preferences. Unlike traditional reinforcement learning focused on maximizing rewards through environment interaction, language model alignment employs specialized techniques that have transformed models like ChatGPT from potentially problematic text generators into helpful digital assistants. Understanding these differences reveals how AI systems are being tuned to produce more ethical and beneficial outputs. The big picture: Reinforcement learning for language models differs fundamentally from traditional agent-based RL, using specialized techniques aimed at alignment rather than environment mastery. Traditional RL...
read Apr 10, 2025Thinker vs. Tinker: 76% of AI researchers doubt scaling alone will achieve AGI, despite Big Tech buildout
AI researchers overwhelmingly reject the tech industry's scaling strategy for achieving artificial general intelligence (AGI), with 76 percent believing that simply throwing more computing power at existing models is unlikely to succeed. This skepticism comes as companies continue pouring billions into AI infrastructure, highlighting a critical disconnect between research consensus and industry investment strategies that could reshape the future direction of AI development. The big picture: A new survey of 475 AI researchers by the Association for the Advancement of Artificial Intelligence reveals that the dominant industry approach of scaling up current models is widely considered insufficient for reaching human-level...
read Apr 10, 2025Reinforcement learning pioneers win Turing Award for AI breakthroughs
Reinforcement learning pioneers Barto and Sutton receive computing's highest honor for developing computational intelligence theories that powered AI breakthroughs like AlphaZero. Their work, which focuses on how computers learn through rewards and punishment, has revolutionized AI systems' ability to master complex tasks and highlights the important role of exploration, curiosity, and play in developing artificial intelligence. The big picture: The Association for Computing Machinery awarded the 2025 Turing Award—computing's equivalent to the Nobel Prize—to professors Andrew G. Barto and Richard S. Sutton for their foundational work in reinforcement learning algorithms. The $1 million prize recognizes their contributions in "introducing the...
read Apr 9, 2025Scale AI wins Pentagon contract to bring AI agents to military planning
The growing intersection of AI and military operations marks a significant evolution in defense technology, as Silicon Valley companies increasingly partner with the Pentagon. Scale AI's latest deal with the Department of Defense represents an important advancement in how AI agents might transform military planning and operations, raising both strategic and ethical considerations about the future of warfare. The big picture: Scale AI has secured a multimillion-dollar prototype contract with the Department of Defense for "Thunderforge," a flagship program designed to incorporate AI agents into U.S. military planning and operations. The AI data company, which provides training data to major...
read Apr 9, 2025The AI empathy paradox: How emotional tech reshapes human connection
The tension between artificial intelligence and human empathy creates a fundamental paradox as these technologies increasingly permeate emotional connections. While AI systems strive for stability and predictability, genuine human empathy thrives within instability and imperfection. This inherent contradiction raises profound questions about whether AI will enhance our capacity for emotional connection or fundamentally alter the beautifully flawed nature of human empathy that makes our connections meaningful. The empathy paradox: AI doesn't simply enhance human empathy—it fundamentally reshapes the dynamic balance between emotional connection and instability that defines genuine human interaction. True empathy functions as a delicate tightrope walk between stability...
read Apr 9, 2025AI challenges human thinking by operating in multiple dimensions at once
Artificial intelligence is fundamentally reshaping our understanding of human cognition, forcing us to confront a new intellectual hierarchy where our thinking appears increasingly one-dimensional compared to AI's multidimensional capabilities. This paradigm shift isn't merely about technological advancement—it's a philosophical reckoning that challenges our cognitive identity and prompts us to reconsider our intellectual relationship with machines in an era where we are no longer unquestionably the most sophisticated thinkers in the room. The big picture: LLMs represent a cognitive leap that transcends mere technological advancement, fundamentally challenging our understanding of human intelligence in relation to artificial systems. Herbert Marcuse's 1964 warning...
read Apr 9, 2025Turing Award winners warn AI companies are prioritizing profit over safety
Turing Award winners have issued a stark warning about AI development practices, highlighting a growing rift between responsible engineering and commercial incentives in the fast-moving artificial intelligence industry. The recognition of reinforcement learning pioneers comes at a critical moment when AI safety concerns are being voiced by an increasing number of industry leaders and researchers, including previous Turing recipients, emphasizing the need for more rigorous testing and safeguards before releasing powerful AI systems to millions of users. The big picture: Reinforcement learning pioneers Andrew Barto and Richard Sutton received the prestigious $1 million Turing Award while using their platform to...
read Apr 8, 2025AI alignment researchers issue urgent call for practical solutions as AGI arrives
The AI alignment movement is sounding urgent alarms as artificial general intelligence (AGI) appears to have arrived much sooner than expected. This call-to-action from prominent alignment researchers emphasizes that theoretical debates must now give way to practical solutions, as several major AI labs are pushing capabilities forward at an accelerating pace that they believe threatens humanity's future. The big picture: The author claims AGI has already arrived in March 2025, with multiple companies including xAI, OpenAI, and Anthropic rapidly advancing capabilities while safety measures struggle to keep pace. Why this matters: The post frames AI alignment as no longer a...
read Apr 8, 2025AI safety advocates need political experience before 2028 election, experts warn
AI safety advocates need to develop political expertise well before the 2028 U.S. presidential election if they want to effectively influence AI policy. The current lack of political knowledge and experience could severely hamper future electoral efforts around AI safety, particularly given the potentially existential stakes of upcoming elections. The big picture: AI safety advocates need to gain practical political experience through earlier campaigns rather than waiting until the 2028 presidential election for their first major political test. The author argues that either the 2024 or 2028 U.S. presidential election is "probably the most important election in human history," with...
read Apr 7, 2025How economic turmoil could derail AI progress by 2027, putting brakes on acceleration
This speculative timeline presents an alternate future where superhuman AI progress is slowed by economic and geopolitical disruptions, challenging the assumption that advanced AI development is inevitable by decade's end. The scenario demonstrates how external factors like trade wars, economic depression, and international conflicts could significantly delay AI advancement - a perspective that offers a counterpoint to more accelerationist predictions about artificial intelligence development. The big picture: A hypothetical 2027 timeline describes a world where AI progress stalls due to global economic and political instability rather than technological limitations. The timeline serves as a "proof-of-concept" counter-argument to predictions of rapid...
read