News/AI Safety
AI skeptic Gary Marcus calls for GenAI boycott to challenge big tech
AI regulation urgency: Gary Marcus, a prominent AI researcher and critic, calls for increased public pressure to regulate the rapidly advancing field of generative AI, highlighting concerns about its potential impact on democracy and creative professions. Marcus, a professor emeritus at New York University and serial entrepreneur, argues that Silicon Valley has strayed far from its "Don't be evil" ethos and is becoming increasingly powerful with minimal constraints. He draws parallels between the need for AI regulation and successful public health campaigns against smoking, suggesting that similar pressure is required to protect citizens from invasive and problematic AI technologies. Key...
read Oct 20, 2024Anthropic publishes new paper on mitigating risk of AI sabotage
AI Safety Evaluations Evolve to Address Potential Sabotage Risks: Anthropic's Alignment Science team has developed a new set of evaluations to test advanced AI models for their capacity to engage in various forms of sabotage, aiming to preemptively identify and mitigate potential risks as AI capabilities continue to improve. Key evaluation types and their purposes: Human decision sabotage: Tests an AI model's ability to influence humans towards incorrect decisions without arousing suspicion. Experiments involve human participants making fictional business decisions based on AI-provided information. Results showed that more aggressive models could sway decisions but also increased user suspicion. Code sabotage:...
read Oct 18, 20243 key AI policy decisions facing the next Congress
AI policy in limbo: Congress faces critical decisions: The 118th U.S. Congress has failed to pass any of the more than 100 AI-related bills introduced, leaving the rapidly evolving technology sector without clear regulatory guidance. Despite high public interest and concern about AI-related issues, including workplace discrimination, deepfake child sexual abuse material, and fraudulent robocalls, Congress has not taken concrete action. A 2020 Gallup survey found that 59% of Americans believe elected officials are paying "too little" attention to technology-related issues. While some states have introduced legislation to address specific AI harms, there has been no corresponding federal action. The...
read Oct 18, 2024An alarming number of people are asking AI to create child pornography
AI-generated child exploitation material: A disturbing trend emerges; The recent hack of Muah.AI, a platform allowing users to create AI chatbots and request images, has exposed a concerning surge in attempts to produce child sexual abuse material (CSAM) using artificial intelligence. Muah.AI, with nearly 2 million registered users, has become a focal point for discussions about the ethical implications of AI-generated content. The hacked data, reviewed by security consultant Troy Hunt, revealed tens of thousands of prompts related to CSAM, including searches for "13-year-old" and "prepubescent" alongside sexual content. While Muah.AI confirmed the hack, they disputed the scale of CSAM-related...
read Oct 15, 2024Anthropic publishes updated AI scaling policy for responsible growth
Anthropic's enhanced AI safety framework: Anthropic has unveiled a significant update to its Responsible Scaling Policy (RSP), introducing a more flexible and nuanced approach to mitigating potential catastrophic risks associated with frontier AI systems. The revised policy maintains Anthropic's commitment to not train or deploy models without adequate safeguards while adapting to the rapidly evolving AI landscape. Key improvements include new capability thresholds, refined processes for evaluating model capabilities and safeguard adequacy, and enhanced measures for internal governance and external input. AI Safety Level Standards: Anthropic's updated RSP incorporates graduated sets of safety and security measures that become more stringent...
read Oct 15, 2024AI chatbots can read and write invisible text — malicious actors are taking notice
Invisible AI-readable text: A new security concern: Researchers have uncovered a method to embed invisible Unicode characters into text that certain AI chatbots can interpret, but remain imperceptible to human readers, raising significant security implications for AI systems and beyond. The discovery of "ASCII smuggling": Johann Rehberger, a researcher, coined the term "ASCII smuggling" to describe this technique, which utilizes a deprecated block of 128 Unicode characters known as the Tags block. The Tags block was originally intended for language tags but has found a new, potentially malicious purpose in the realm of AI communication. This method creates a covert...
read Oct 13, 2024The most common ways to bypass restrictions on AI chatbots
AI bamboozlement techniques revealed: Recent discussions have brought to light various methods used to circumvent restrictions placed on generative AI systems, sparking a debate about the ethics and implications of such practices. The techniques, often referred to as "bamboozlement" or "jailbreaking," are designed to trick AI systems into providing information or responses that they are programmed to withhold. These methods range from being roundabout in questioning to framing requests in academic or hypothetical contexts, with the goal of bypassing built-in safeguards. The controversy surrounding AI restrictions: The revelation of these techniques has ignited a debate about the nature and necessity...
read Oct 13, 2024Nobel laureate joins ranks of past winners who warned of risks in their own fields
AI's Potential and Peril: A Nobel Laureate's Warning: Geoffrey Hinton, a Nobel Prize-winning computer scientist, has issued a stark warning about the potential dangers of artificial intelligence, joining a growing list of laureates who have cautioned about the risks associated with their own groundbreaking work. The AI Revolution: Capabilities and Concerns: Hinton draws a parallel between AI and the Industrial Revolution, emphasizing that AI's impact will be intellectual rather than physical, with potentially far-reaching consequences for humanity. Hinton predicts that AI systems will surpass human intellectual capabilities, raising concerns about their potential to get out of control and "take over."...
read Oct 13, 2024Meta AI chief Yann LeCun says existential risk of AI is ‘complete BS’
AI safety concerns challenged: Yann LeCun, Meta's AI Chief and renowned AI scientist, has dismissed predictions about artificial intelligence posing an existential threat to humanity as unfounded. LeCun, a decorated AI researcher and New York University professor, won the prestigious A.M. Turing award for his groundbreaking work in deep learning. In response to questions about AI becoming smart enough to endanger humanity, LeCun bluntly stated, "You're going to have to pardon my French, but that's complete B.S." This stance puts LeCun at odds with other prominent tech figures like OpenAI's Sam Altman and Elon Musk, who have expressed concerns about...
read Oct 11, 2024Anthropic CEO has lots and lots of things to say about AGI in new blog
AI's potential impact: Unprecedented promise and peril: Anthropic CEO Dario Amodei has shared his expansive vision for the future of artificial general intelligence (AGI), highlighting both its immense potential and significant risks. In a comprehensive blog post, Amodei challenges the perception that he is overly pessimistic about AI, instead emphasizing the technology's capacity for transformative positive impact. The CEO argues that the general public is underestimating both the radical upside and the severe downside potential of advanced AI systems. Amodei prefers the term "powerful AI" over AGI, suggesting a focus on capability rather than human-like intelligence. Optimistic outlook on AI's...
read Oct 10, 2024ChatGPT predicts its own global takeover timeline
The AI takeover debate: Separating fact from fiction: The concept of artificial intelligence (AI) "taking over the world" is a complex and nuanced topic that requires careful examination of current technological capabilities, potential future developments, and their societal implications. ChatGPT, when asked about AI takeover by Newsweek, provided a balanced perspective, emphasizing that current AI systems are far from achieving the level of intelligence required for such a scenario. The AI tool highlighted the distinction between narrow AI (specialized in specific tasks) and artificial general intelligence (AGI), which would be capable of performing any intellectual task a human can. Experts...
read Oct 9, 2024The 12 greatest dangers of AI, according to Gary Marcus
The AI revolution's dark side: Gary Marcus, an AI expert, outlines 12 immediate dangers of artificial intelligence in his new book "Taming Silicon Valley," highlighting the potential risks and societal impacts of this rapidly evolving technology. Marcus identifies automatically generated disinformation and deepfakes as the most pressing short-term concern, particularly in their potential to influence elections and manipulate public opinion. In the long term, Marcus expresses worry about the lack of knowledge on how to create safe and reliable AI systems, which could lead to unforeseen consequences. Economic implications and regulatory needs: The widespread adoption of AI technologies may necessitate...
read Oct 8, 2024Scott Wiener is open to working with Newsom on possible AI legislation next year
California's AI regulatory landscape evolves: Governor Gavin Newsom vetoed a sweeping AI regulation bill authored by Senator Scott Wiener, opting instead to establish a panel of experts to study future statewide AI safety rules. Newsom's office announced a panel of three experts to examine AI regulations, with the possibility of expanding the group in the future. The current panel members are Stanford Professor Fei-Fei Li, UC Berkeley Dean Jennifer Chayes, and Carnegie Endowment for Peace President Mariano-Florentino Cuéllar. Senator Wiener expressed openness to working with the panel, emphasizing the need for diverse viewpoints in the group. Expert panel composition raises...
read Oct 7, 2024Harvard undergrads use AI to reveal personal info from strangers’ photos
AI-powered facial recognition raises privacy concerns: Harvard undergraduates have developed a tool that can reveal personal information from a stranger's photo, highlighting the growing privacy risks associated with artificial intelligence and facial recognition technologies. AnhPhu Nguyen and Caine Ardayfio, juniors at Harvard, created I-XRAY, an AI tool that combines smart eyeglasses and online databases to obtain sensitive information about individuals. The system can potentially identify a person's name, address, and other personal details in about 90 seconds, using only a photograph of their face. I-XRAY demonstrates how existing privacy threats can be amplified by advancements in AI technology. How I-XRAY...
read Oct 7, 2024California’s AI legislation continues to advance despite AI safety bill veto
California's AI regulation leadership: California continues to lead the way in state-level artificial intelligence regulation, building on its history of consumer data protection with new laws addressing AI systems beyond personal data use. The California Consumer Privacy Act (CCPA) has already established the state as a leader in data protection, often mentioned alongside the European Union's General Data Protection Regulation (GDPR). Recent legislative efforts in California aim to address broader AI and machine learning system regulations, reflecting the state's role as the home of Silicon Valley and the largest state economy in the U.S. SB 1047 veto and controversy: Governor...
read Oct 7, 2024AI-generated nude images of classmates alarm parents and educators
Disturbing trend in AI-generated nudes among minors: A recent survey by anti-human trafficking nonprofit Thorn has uncovered a concerning phenomenon where adolescents are using artificial intelligence to create nude images of their peers. One in ten minors reported knowing peers who have used AI to generate nude images of other children, highlighting the prevalence of this issue. While the motivations may stem from adolescent behavior rather than intentional sexual abuse, the potential harm to victims is significant and should not be downplayed. Real-world consequences: The creation and distribution of AI-generated nude images of minors has already led to legal repercussions...
read Oct 3, 2024AI pioneer warns of catastrophic future without more regulation
AI pioneer's urgent call for regulation: Yoshua Bengio, a leading figure in artificial intelligence research, is sounding the alarm on potential catastrophic risks associated with unregulated AI development and deployment. Bengio, widely recognized as one of the "godfathers of AI" due to his groundbreaking work on artificial neural networks, has shifted his focus to advocating for stringent AI regulation and safety measures. His concerns span both short-term and long-term risks, ranging from the manipulation of elections and assistance to terrorist activities to the potential loss of human control over AI systems and the emergence of AI-enabled dictatorships. Current regulatory landscape:...
read Oct 1, 2024YouTuber claims AI company stole his voice for chatbot
AI-Generated Voice Controversy Sparks Ethical Debate: YouTuber Jeff Geerling's recent experience with unauthorized AI voice cloning by electronics company Elecrow has reignited discussions about the ethical implications of AI-generated content. Geerling, a software developer with approximately 700,000 YouTube subscribers, discovered that Elecrow had used an AI-generated version of his voice in dozens of promotional tutorials without his knowledge or consent. The company's CEO apologized and promptly removed the videos after Geerling raised the issue, highlighting the swift action taken in response to the controversy. This incident is not isolated, as voice actors and celebrities like Scarlett Johansson, Tom Hanks, and...
read Oct 1, 2024California AI safety bill veto may give smaller AI models a chance to flourish
California's AI bill veto: A win for innovation and open-source development: Governor Gavin Newsom's decision to veto SB 1047, a bill that would have imposed strict regulations on AI development in California, has sparked mixed reactions from industry leaders and policy experts. The vetoed bill would have required AI companies to implement "kill switches" for models, create written safety protocols, and undergo third-party safety audits before training models. It would have also granted California's attorney general access to auditors' reports and the right to sue AI developers. Critics of the bill argued that it could have a chilling effect on...
read Sep 30, 2024California cracks down on AI-generated child deepfakes
California takes bold steps to protect minors from AI-generated sexual imagery: Governor Gavin Newsom has signed two bills aimed at safeguarding children from the misuse of artificial intelligence to create explicit sexual content. The new laws close a legal loophole around AI-generated child sexual abuse imagery and clarify that such content is illegal, even if artificially created. District attorneys can now prosecute individuals who possess or distribute AI-generated child sexual abuse images as a felony offense, without needing to prove the materials depict a real person. These measures received strong bipartisan support in the California legislature. Broader context of AI...
read Sep 30, 2024A new AI safety initiative launches as Newsom vetoes California bill
Governor Newsom vetoes major AI regulation bill: California Governor Gavin Newsom has vetoed SB1047, a comprehensive artificial intelligence regulation bill authored by State Senator Scott Wiener, citing concerns about its broad scope and potential impact on AI innovation in the state. The bill aimed to establish safety and testing requirements for large-scale AI programs to prevent catastrophic risks. Newsom argued that the bill's stringent standards applied even to basic functions of large systems, potentially hindering beneficial AI development. The governor expressed concern that the bill's focus on large-scale models could create a false sense of security, overlooking potential dangers from...
read Sep 30, 2024Why Newsom vetoed AI safety bill SB 1047 and what comes next
California's AI regulation setback: Governor Gavin Newsom's veto of SB 1047, a pioneering artificial intelligence safety bill, marks a significant moment in the ongoing debate surrounding AI regulation and safety measures. The bill, introduced by Senator Scott Wiener, aimed to establish safety protocols for advanced AI models and hold developers accountable for potential harm or threats to public safety. Key provisions included requiring AI developers to submit safety plans to the state attorney general and implement mechanisms to shut down AI models in case of emergencies. The legislation garnered support from notable figures in the tech industry, including Elon Musk,...
read Sep 30, 2024AI safety bill vetoed by Newsom is a victory for tech giants
California Governor Vetoes Controversial AI Bill: Gavin Newsom has vetoed SB 1047, a high-profile artificial intelligence bill that faced significant opposition from Silicon Valley and tech industry leaders. The bill's key provisions: SB 1047 aimed to establish a new government agency to enforce compliance on developers of "covered models" - AI systems using a significant amount of computing power for training or fine-tuning. The bill would have imposed criminal penalties, including perjury charges, for non-compliance. It targeted AI models using 10^26 or 10^25 floating point operations (FLOPs) for training or fine-tuning, respectively. Opposition from tech industry: The bill faced widespread...
read Sep 30, 2024Gavin Newsom has rejected SB 1047 but the debate over AI safety is far from over
California Governor Vetoes Controversial AI Safety Bill: Governor Gavin Newsom has rejected a proposed legislation aimed at mitigating potential catastrophic risks associated with advanced artificial intelligence models, citing concerns over the bill's regulatory approach. SB 1047, the most contentious AI bill of the legislative session, sought to establish safeguards against the misuse of highly advanced AI systems for developing weapons of mass destruction. The bill garnered support from SAG-AFTRA and numerous Hollywood celebrities, who voiced concerns about AI's potential threats beyond the entertainment industry. Governor Newsom, while acknowledging the genuine issues addressed by the bill, expressed reservations about its regulatory...
read