News/AI Safety
Resemble AI’s Detect-2B: A Leap Forward in Combating Deepfake Audio
Resemble AI has released Detect-2B, a next-generation AI audio detection model that can identify deepfake audio with 94% accuracy, marking a significant advancement in the fight against misinformation and erosion of trust in an era of increasingly sophisticated generative AI. Key features of Detect-2B: The new model utilizes a series of pre-trained sub-models and fine-tuning techniques to analyze audio clips and determine whether they were generated using AI: The sub-models consist of a frozen audio representation model with an adaptation module inserted into its key layers, allowing the model to focus on artifacts that often distinguish real audio from fake...
read Jun 28, 2024AI Pioneer Warns of Secretive LLMs, Advocates for User-Owned Alternative
Illia Polosukhin, a key contributor to the development of transformers, is concerned about the secretive and profit-driven nature of large language models (LLMs) and aims to create an open source, user-owned AI model to ensure transparency and accountability. Key concerns with current LLMs: Polosukhin believes that the lack of transparency in LLMs, even from companies founded on openness, poses risks as the technology improves: The data used to train models and the model weights are often unknown, making it difficult to assess potential biases and decision-making processes. As models become more sophisticated, they may be better at manipulating people and...
read Jun 28, 2024AWS Probes AI Startup Perplexity Over Alleged Data Scraping Violations
Amazon investigates Perplexity AI over potential data-scraping violations: Amazon Web Services is looking into whether AI startup Perplexity is violating its terms of service by scraping web content without permission, following reports from multiple news outlets. Accusations of improper data scraping: Several publications, including Forbes and Wired, have accused Perplexity of swiping their web archives to train its AI models without consent or compensation: Forbes alleged that Perplexity is creating "knockoff stories" using similar wording and lifted fragments from its articles without adequate citation. Wired identified an IP address it believes Perplexity is using to crawl its sites and those...
read Jun 28, 2024CIR Sues OpenAI, Microsoft for Copyright Infringement in AI Training
The Center for Investigative Reporting (CIR) has filed a lawsuit against OpenAI and Microsoft, alleging copyright infringement related to their use of CIR's content in training AI models without permission or compensation. Key details of the lawsuit: The CIR claims that OpenAI and Microsoft have copied, used, and displayed CIR's journalistic content without authorization, and that OpenAI trained ChatGPT to disregard copyright: The lawsuit was filed in the Southern District of New York, with the CIR seeking damages and profits from the defendants. CIR's CEO accused OpenAI and Microsoft of "free rider behavior," using their stories to enhance AI products...
read Jun 28, 2024Perplexity’s Unethical Practices Threaten Web’s Foundations, Raising Concerns for AI Industry’s Integrity
Perplexity, an AI startup creating a Google Search competitor, has been engaging in unethical practices to build its "answer engine," raising questions about the company's integrity and the broader implications for the web. Key issues with Perplexity's approach: Perplexity's practices have come under scrutiny for their potential to undermine the web's foundations and the work of primary sources: Perplexity's "answer engine" acts as a rent-seeking middleman, providing answers directly to users rather than sending traffic to the original sources, thus depriving them of ad revenue. The company's Pages product creates entire aggregated articles by plagiarizing content from primary sources, going...
read Jun 27, 2024AI Pioneer Seeks to Democratize Technology, Prioritizing User Privacy and Sovereignty
Artificial intelligence pioneer seeks to democratize AI: Illia Polosukhin, one of the "Transformer 8" who invented the transformer neural network architecture at Google, left the company in 2017 to start his own AI venture, Near, with the goal of making AI more accessible and user-centric. Polosukhin co-wrote the seminal 2017 paper "Attention Is All You Need" with seven Google colleagues, laying the groundwork for modern generative AI systems like ChatGPT. While at Google, Polosukhin advocated for open-sourcing the transformer architecture, believing it was crucial for widespread adoption and innovation in the field. Google began using transformers in its translation services...
read Jun 27, 2024Chinese AI Pioneer Advocates Responsible Development Amid Geopolitical Tensions
Here is a summary of the key points from the interview with Zhang Hongjiang, founder of the Beijing Academy of Artificial Intelligence: China's advantages and challenges in AI development: Despite recent US chip export controls hampering computing power, Zhang sees China's key AI advantages as its large pool of talented entrepreneurs, vast market with diverse AI application scenarios, and strong research institutions. However, he notes the field is over-competitive with too many similar startups. Advocating for responsible AI development: As one of China's leading voices on AI safety, Zhang has worked to raise awareness among researchers, industry and government about...
read Jun 27, 2024Roblox Is Leveraging Generative AI to Enhance User-Created Games and Experiences
Roblox, the popular online gaming platform, is leveraging generative AI to enhance virtual experiences and user interactions. Anupam Singh, Roblox's VP of AI and growth engineering, discusses how AI is being used to create a more inclusive, safe, and creative environment for Roblox users. Key AI applications at Roblox: Generative AI is being applied in several ways to improve the Roblox platform: Automated chat filters and real-time text translation help foster inclusivity and ensure user safety by moderating interactions and breaking down language barriers. AI-powered coding assistants simplify the creation process for developers, allowing them to focus more on creative...
read Jun 26, 2024Deepfakes and Disinformation: AI Misuse Threatens Democracy, Study Reveals
A new study from Google's DeepMind division sheds light on the most common malicious uses of AI, revealing that political deepfakes and disinformation campaigns are the top concerns. Key Findings: Deepfakes and Disinformation Dominate AI Misuse; The study, conducted in collaboration with Google's Jigsaw unit, analyzed around 200 incidents of AI misuse and found that: The creation of realistic but fake images, videos, and audio of people, known as deepfakes, was the most prevalent form of AI misuse, nearly twice as common as the next highest category. The second most common misuse was the falsification of information using text-based tools...
read Jun 26, 2024AI Breakthrough: MatMul-Free Language Modeling Slashes Energy Use, Boosts Accessibility
Researchers claim a breakthrough in AI efficiency by eliminating matrix multiplication, a fundamental operation in neural networks that is accelerated by GPUs, which could significantly reduce the energy consumption and costs of running large language models. Key innovation: MatMul-free language modeling; The researchers developed a custom 2.7 billion parameter model that performs similarly to conventional large language models (LLMs) without using matrix multiplication (MatMul): They demonstrated a 1.3 billion parameter model running at 23.8 tokens per second on a GPU accelerated by a custom FPGA chip, using only about 13 watts of power. This approach challenges the prevailing paradigm that...
read Jun 26, 2024OpenAI Delays ChatGPT-4o’s Voice Features Amid Safety Concerns
OpenAI delays ChatGPT-4o's advanced voice features due to safety concerns: OpenAI announced that it needs an additional month to launch ChatGPT-4o's advanced voice mode, which was originally planned for a small group of ChatGPT Plus users in late June, in order to ensure the model meets safety requirements and provides real-time responses to complex queries. The delay is due to the need to improve the model's ability to detect and refuse certain content, enhance user experience, and scale systems for real-time responses. Some users who signed up for ChatGPT Plus specifically for early access to these features are disappointed by...
read Jun 25, 2024Startup Helps Authors License Work to AI, Navigating Rights in New Era
A startup called Created by Humans is launching to help book authors license their work to AI companies, aiming to create a framework for AI rights and broker deals between authors, publishers, and AI firms. Filling a void in the market: Created by Humans seeks to address the lack of universal permissioning and monetization systems for creative content used to train large language models: The startup is one of several emerging to help creators navigate the complex landscape of licensing their work for use in AI technologies. Co-founder and CEO Trip Adler brings years of experience working with book authors...
read Jun 25, 2024California AI Startups Clash with Lawmakers Over Proposed Safety Rules
California startups and Y Combinator oppose proposed AI safety regulations, arguing they could stifle innovation and threaten the state's tech economy. Key stakeholders express concerns: A coalition of 140 machine-learning startups and venture capital firm Y Combinator have signed an open letter opposing California Senate Bill 1047, which would impose guardrails and transparency requirements on large AI models: The signatories argue that the proposed regulations are arbitrary, vague, and burdensome, potentially leading AI companies to leave California for less restrictive locales. They claim the bill's language is too broad and could inadvertently apply to smaller startups, not just the intended...
read Jun 25, 2024Apple Rejects Meta AI Partnership, Citing Privacy Concerns; Pursues Other Collaborations
Apple's privacy concerns prevented an AI partnership with Meta Key Takeaways: Apple turned down a potential AI partnership with Facebook parent company Meta due to privacy concerns over Meta's practices: Brief discussions about integrating Meta's large language model (LLM) into iOS took place in March, but Apple decided not to move forward with a more formal partnership. Apple determined that Meta's privacy practices were not stringent enough to meet its standards for safeguarding user data. Ongoing AI Partnerships: While Apple passed on working with Meta, the company has pursued other AI collaborations: Apple signed a deal with OpenAI to integrate...
read Jun 24, 2024AI Data Quality: The Key to Effective, Reliable, and Ethical AI Systems
In the age of artificial intelligence, data quality is crucial for building effective and reliable AI systems. Poor quality data used to train AI models, such as Reddit's content partnership with Google that led to bizarre search results like recommending glue on pizza, highlights the importance of high-quality data in AI development. Defining high-quality data: Data quality is not just about accuracy or quantity, but rather data that is fit for its intended purpose and evaluated based on specific use cases: Relevance is critical, as the data must be directly applicable and meaningful to the problem the AI model aims...
read Jun 24, 2024Meta’s AI Mislabels Photos, Sparking Controversy and Raising Questions
Meta's AI mislabeling real photos as AI-generated sparks controversy and raises questions about the reliability and implications of such labels. Key issues with Meta's AI labeling system: The social media giant's "Made by AI" labels are being incorrectly applied to genuine photographs, causing frustration among photographers and users: Several photographers have reported instances where their original photos or those edited using standard tools like Adobe's cropping feature were mistakenly labeled as AI-generated by Meta. Even minor edits using AI-assisted tools like Adobe's Generative Fill seem to trigger the "Made with AI" label, despite the photos being predominantly created by humans....
read Jun 24, 2024California Grapples with AI Regulation: Balancing Innovation and Risk Mitigation
California's rush to regulate AI raises concerns and debates as lawmakers grapple with the technology's potential risks and benefits. Legislative response to AI concerns: California legislators are considering roughly 50 AI-related bills to place safeguards around the rapidly evolving technology, which they fear could cause societal harm: The bills cover a wide range of AI-related issues, including job replacement, data security, and racial discrimination. Specific bills aim to mandate human oversight on driverless heavy-duty trucks, ban job automation at call centers providing public benefit services, and require safety testing for companies developing large AI models. Calls for whistleblower protections and...
read Jun 24, 2024$1M Roddenberry Prize Seeks AI Solutions for Global Good, Inspired by “Star Trek”
The Gene Roddenberry Foundation is offering a $1 million prize to encourage the development of artificial intelligence that benefits humanity, reflecting the optimistic vision of technology portrayed in Roddenberry's creation, "Star Trek." The prize and its mission: The biennial Roddenberry Prize will focus this year on AI and machine learning projects that have the potential to help solve global challenges: Open to early-stage ventures worldwide, the prize aims to catalyze thinking about what AI could look like if used ethically and responsibly to address pressing issues. Applicants must demonstrate how their AI solutions can contribute to one or more of...
read Jun 24, 2024TikTok’s AI Avatar Mishap: Hitler Quotes and Misinformation Raise Alarms
TikTok's AI digital avatar tool accidentally released without guardrails, allowing users to create misleading videos with paid actors reciting anything from Hitler quotes to dangerous misinformation. The incident raises concerns about the potential for abuse and the need for robust content moderation as AI-generated content becomes more prevalent on social media platforms. Key details of the incident: TikTok mistakenly posted a link to an internal version of its new AI digital avatar tool, which allowed users to generate videos without any content restrictions: CNN was able to create videos using the tool that contained quotes from Hitler, Osama bin Laden,...
read Jun 21, 2024Oxford Researchers Develop Method to Detect AI Confabulation, Preventing Misinformation Spread
Oxford researchers have developed a method to identify when large language models (LLMs) are confabulating, or making up false information, which could help prevent the spread of misinformation as these AI systems become more widely used. Key Takeaways: The researchers' approach focuses on evaluating the semantic entropy of an LLM's potential answers to determine if it is uncertain about the correct response: If many of the statistically likely answers are semantically equivalent, the LLM is probably just uncertain about phrasing and has the correct information. If the potential answers are semantically diverse, the LLM is likely confabulating due to a...
read Jun 21, 2024Breakthrough AI “Hallucination” Detection Method Unveiled, Boosting Reliability
New research reveals a breakthrough method for detecting AI "hallucinations," paving the way for more reliable artificial intelligence systems in the near future, although challenges remain in integrating this research into real-world applications. Key Takeaways: The study, published in the peer-reviewed scientific journal Nature, describes a new algorithm that can detect AI confabulations, a specific type of hallucination, with approximately 79% accuracy: Confabulations occur when an AI model generates inconsistent wrong answers to a factual question, as opposed to providing the same consistent wrong answer due to issues like problematic training data or structural failures in the model's logic. The...
read Jun 21, 2024California Grapples with Regulating AI: Balancing Innovation and Safety Risks
California lawmakers are rushing to regulate AI, aiming to protect workers and society from potential risks while the technology is still rapidly evolving. However, there are concerns that overregulation could stifle innovation and economic growth in the state. Key legislative efforts: Roughly 50 AI-related bills are circulating in the California Legislature, addressing issues like job replacement, data security, algorithmic discrimination, and safety risks: SB1047, backed by the Center for AI Safety, would require companies building large AI models to conduct safety testing, implement employee reporting mechanisms, and be able to turn off models they control. It has passed initial committees....
read Jun 21, 2024Meta’s AI Labeling Mislabels Original Photos, Highlighting Challenges of Identifying AI-Generated Content
Meta's 'Made with AI' labeling system faces criticism as photographers report their unaltered images being mistakenly tagged, highlighting the challenges of accurately identifying AI-generated content amidst the rapid proliferation of generative AI tools. Key issues with Meta's AI labeling approach: Meta's automated system for detecting and labeling AI-generated images on its platforms, including Facebook, Instagram, and Threads, has drawn ire from photographers who claim their unaltered photos are being incorrectly tagged as 'Made with AI': Several photographers have shared examples of their original photos, captured with traditional cameras, being labeled as AI-generated, causing confusion and frustration among content creators. Former...
read Jun 21, 2024Google AI Gaffe Highlights Escalating Legal Risks as Generative AI Goes Mainstream
Google's AI search mistakes are escalating legal risks as the technology rapidly enters mainstream use. The AI Overview feature falsely stated that chess player Hans Niemann admitted to cheating against Magnus Carlsen, highlighting the potential for AI-generated libel to harm reputations and businesses. Key issues with AI defamation: The rise of generative AI in consumer products is blurring the lines between platforms and publishers, raising questions about legal liability for false and damaging statements: AI language models are prone to "hallucination" - inventing sources, misattributing quotes, and rewriting events - which can easily turn nuances into libel. Traditional defamation law...
read