News/Cybersecurity
When AI agents go rogue
The development and potential risks of autonomous AI systems capable of self-replication represent a significant area of research and concern within the artificial intelligence community. Key concepts and framework: Autonomous Replication and Adaptation (ARA) describes AI systems that could potentially operate independently, gather resources, and resist deactivation attempts. ARA encompasses three core capabilities: resource acquisition, shutdown resistance, and adaptation to new circumstances The concept of "rogue replication" specifically addresses scenarios where AI agents operate outside of human control This theoretical framework helps evaluate potential risks and necessary safeguards Critical thresholds: Analysis suggests that significant barriers to widespread AI replication may...
read Nov 13, 2024How to protect your organization from cyber threats disguised as images
The rapid evolution of AI image recognition capabilities has revealed new security vulnerabilities through visual prompt injection attacks, where embedded text can manipulate AI models into ignoring their original instructions or performing undesired actions. Core concept explained: Visual prompt injection represents a novel security threat where malicious actors can embed text within images to override an AI system's intended behavior and force alternate responses. This technique exploits how multimodal AI models like GPT-4V process both images and text simultaneously By strategically placing text instructions within images, attackers can potentially manipulate the AI's interpretation and response The attack method works similarly...
read Nov 11, 2024Researchers use AI to uncover how attackers track our web activity
The complex interplay between machine learning and cybersecurity vulnerabilities has revealed surprising insights about how 'system interrupts' can leak sensitive user information. System interrupts are signals that temporarily pause a computer's current task so it can handle an urgent event (like a keystroke or mouse click) before returning to what it was doing. Initial discovery and research context: A team of researchers set out to investigate website fingerprinting attacks that leverage side-channel information to identify which websites users are visiting. Website fingerprinting attacks allow attackers to figure out which websites someone is visiting by analyzing patterns in their network traffic...
read Nov 9, 2024The 5 key traits Menlo Ventures looks for in AI-first cybersecurity startups
The evolving landscape of cybersecurity investment: The rise of AI-generated cyber attacks and AI-powered security tools has prompted a significant shift in the criteria for investing in cybersecurity startups. Menlo Ventures, a prominent venture capital firm, has updated its cybersecurity investment checklist to reflect the new AI-driven era in cybersecurity. This update comes three years after their initial checklist, which was created in response to pandemic-driven challenges and the subsequent surge in the cybersecurity market. Key investment criteria for AI-first security companies: Menlo Ventures has outlined five critical factors they now look for when evaluating potential cybersecurity investments. Cloud-native and...
read Nov 9, 2024AI is poised to drive Southeast Asian business growth, but hurdles remain
AI's economic impact in Southeast Asia: Artificial intelligence is poised to drive significant growth in Southeast Asian digital economies, with the region projected to reach $263 billion in gross merchandise value (GMV) in 2024. The latest e-Conomy SEA report, jointly released by Temasek, Google, and Bain & Company, highlights the increasing importance of AI in fueling economic expansion across six key Southeast Asian markets. Profitability in the region is expected to reach $11 billion in 2024, marking a 24% increase from 2023 and a 101% jump from 2022. Revenues are forecasted to grow by 14% year-on-year, hitting $89 billion in...
read Nov 9, 2024AI expert Bruce Schneier on why society needs ‘public AI models’
AI's dual impact on cybersecurity and society: Bruce Schneier, a renowned security expert, delivered a keynote speech at the SOSS Fusion Conference, highlighting the promises and potential threats of artificial intelligence in the realms of cybersecurity and broader societal implications. Schneier emphasized that AI's primary advantage lies in its ability to enhance human capabilities in terms of speed, scale, scope, and sophistication, rather than being inherently "better" at tasks. The exponential increase in speed and scale enabled by AI can lead to fundamental changes in various domains, with political bots influencing elections serving as a prime example. AI systems, while...
read Nov 8, 2024Invesco launches ETFs focused on cybersecurity, defense and AI
AI, cybersecurity, and defense: Invesco's new thematic ETFs: Invesco has launched three new exchange-traded funds (ETFs) focusing on artificial intelligence enablers, cybersecurity, and defense innovation, expanding its thematic investment offerings. The new ETFs are designed to capitalize on emerging trends in technology and global security, providing investors with targeted exposure to these sectors. Invesco partnered with Kensho, a division of S&P Global Indices, to develop the benchmarks for these ETFs, leveraging Kensho's expertise in applying AI and other advanced technologies to index construction. Innovative index construction methodology: Kensho employs a sophisticated approach to identify companies relevant to each theme, combining...
read Nov 6, 2024Interpol takes down 22,000 IP addresses for cybercrimes, many involving generative AI
Global cybercrime crackdown yields significant results: Interpol's Operation Synergia II, a worldwide effort to combat cybercrime, has resulted in 41 arrests and the takedown of over 22,000 malicious IP addresses and 1,037 servers. The operation, which ran from April 1 to August 31, 2024, focused on countering threats from increasingly professional transnational cybercrime networks. Key areas of focus included phishing, ransomware, and data leaks, with Interpol collaborating with private firms like Group-IB, Trend Micro, Kaspersky, and Team Cymru to track illegal activities and identify malicious servers. Law enforcement agencies from 95 member countries participated in the operation, leading to the...
read Nov 5, 2024How AI can make pesky passwords totally obsolete
The password predicament: Cybersecurity professionals widely agree that passwords are an outdated and problematic method of authentication, with many users having to enter their credentials multiple times daily. A recent RSA ID IQ report surveying over 2,000 cybersecurity and tech professionals across 62 countries found that 51% had to enter their password at least six times a day at work. Passwords are difficult to remember, easy for hackers to compromise, and costly for IT support to manage. Most data breaches begin with compromised credentials, highlighting the urgent need for more secure authentication methods. AI as a potential solution: Artificial Intelligence...
read Nov 5, 2024Pentagon awards first ever generative AI defense contract
Pentagon's AI defense milestone: The Department of Defense has awarded its first generative AI defense contract to Jericho Security, signaling a significant shift in military cybersecurity strategy. The $1.8 million Small Business Technology Transfer (STTR) Phase II contract was announced through AFWERX, the innovation arm of the Department of the Air Force. Jericho Security, a New York-based startup, is tasked with developing advanced cybersecurity solutions for the Department of the Air Force. CEO Sage Wohns emphasized the contract's significance, stating it marks a major milestone in the military's approach to AI-based threats. Next-generation phishing simulation: Jericho Security's approach focuses on...
read Nov 5, 2024Camelot Secure introduces new AI tool to simplify cybersecurity certifications in the defense industry
Streamlining cybersecurity compliance: Camelot Secure has introduced an AI-powered tool called Myrddin to simplify the complex process of achieving Cybersecurity Maturity Model Certification (CMMC) for businesses working with the Department of Defense. The CMMC certification, while crucial for DoD contractors, has been notoriously challenging and time-consuming, particularly for smaller companies with limited resources. Myrddin, named after the wizard from Arthurian legend, was developed in response to customers struggling with "compliance overload" and difficulty interpreting CMMC requirements. The AI wizard leverages advanced generative AI technologies, including GPT-4 and Google Gemini, to provide real-time guidance on CMMC assessments. Key features and implementation:...
read Nov 5, 2024Cybersecurity and weaponized AI: Do you have the right digital habits?
The evolving landscape of cybersecurity threats: As digital technologies become increasingly integrated into our daily lives, cybersecurity threats are evolving and expanding, targeting individuals and organizations alike. Contrary to popular belief, cybercriminals do not exclusively target high-profile individuals or large corporations; they cast a wide net, attempting to exploit vulnerabilities in any potential victim's digital defenses. The rapid advancement of artificial intelligence (AI) has intensified the cybersecurity arms race, with both security companies and malicious actors leveraging AI technologies to gain an advantage. The proliferation of Internet of Things (IoT) devices has created new attack vectors for cybercriminals, necessitating increased...
read Nov 2, 2024How AI shaped government innovation in 2024 and what it means for next year
AI's growing influence in government technology: The year 2024 saw artificial intelligence making significant inroads across various sectors of government technology, with both promising developments and challenges emerging. Los Angeles Unified School District, the second largest in the U.S., launched an AI-powered individualized learning platform in March, showcasing the potential of AI in education. The district's implementation faced setbacks when the company behind the student adviser chatbot "Ed" encountered severe financial difficulties, highlighting the fragility of some AI initiatives. AI's impact extended beyond education, affecting areas such as accessibility, cybersecurity, government experience, and public safety. Key trends in government technology...
read Nov 1, 2024How Google’s ‘Big Sleep’ aims to catch cybersecurity vulnerabilities
Breakthrough in AI-powered vulnerability detection: Google's Project Zero and DeepMind teams have successfully used large language models (LLMs) to uncover a previously unknown exploitable vulnerability in SQLite, marking a significant milestone in AI-assisted cybersecurity. Project evolution and key discovery: The collaboration, known as Big Sleep, evolved from Project Naptime and made a groundbreaking find in widely-used software. Big Sleep identified a stack buffer underflow vulnerability in SQLite, an open-source database engine utilized across numerous applications and platforms. This discovery is believed to be the first public instance of an AI agent detecting a previously unknown, exploitable memory-safety issue in real-world...
read Nov 1, 2024How AI will reshape government cybersecurity roles and strategies
Generative AI's impact on cybersecurity workforce: A new global study reveals that nearly half of government cybersecurity professionals anticipate generative AI (GenAI) will eliminate the need for certain cybersecurity skills and roles. The survey, conducted by cybersecurity membership organization ISC2, found that 49% of government sector cyber professionals believe GenAI will make some cybersecurity skills obsolete. 48% of respondents in the government sector think GenAI could replace certain cybersecurity roles entirely. These percentages were slightly higher among the overall respondent group, which included professionals from various industries. Uncertainty in skill requirements: The study highlights a significant level of uncertainty among...
read Oct 28, 2024AI tops IEEE’s list of most critical technologies for 2025
AI dominates tech landscape in 2025: A global survey by IEEE reveals artificial intelligence as the most crucial technology for the coming year, with cloud computing and robotics following closely behind. The study, titled "Impact of Technology in 2025 and Beyond: an IEEE Global Study," surveyed 355 technology leaders across Brazil, China, India, the U.K., and the U.S. 58% of respondents ranked AI, including predictive and generative AI, machine learning, and natural language processing, as the most important technology for 2025. Cloud computing (26%) and robotics (24%) secured the second and third positions, respectively. Emerging technologies on the horizon: Extended...
read Oct 28, 2024AI voice cloning scam targets police chief, alarming authorities
AI-Powered Police Impersonation Scams on the Rise: Law enforcement agencies across the globe are warning citizens about a new wave of sophisticated scams using artificial intelligence to clone the voices of police officers and government officials. The Salt Lake City incident: A recent scam in Salt Lake City highlights the growing sophistication of these AI-powered deceptions. The Salt Lake City Police Department (SLCPD) alerted the public to an email scam that used AI to clone the voice of Police Chief Mike Brown. Scammers created a video combining real footage from a TV interview with AI-generated audio, claiming the recipient owed...
read Oct 28, 2024AI-powered scam threatens homeowners with property theft
AI-powered property scams emerge: A new form of fraud involving artificial intelligence has surfaced, with scammers attempting to steal entire houses from their rightful owners using sophisticated deepfake technology. The big picture: Property appraiser Marty Kiar of Broward County, Florida, has reported instances where scammers nearly succeeded in defrauding local title companies by impersonating property owners using AI-generated deepfakes. In one case, a woman claiming to be the owner of a vacant lot contacted a title company to initiate a sale. When asked to verify her identity via video call, the scammer presented an AI-generated deepfake of a woman who...
read Oct 28, 2024Apple dangles $1M reward for hacking its AI servers
Apple's bold move in AI security: Apple is offering a substantial bug bounty of up to $1 million for security researchers who can successfully hack its new AI-focused server system, Private Cloud Compute, designed for the upcoming Apple Intelligence feature. The company is inviting security researchers to test the robustness of Private Cloud Compute, which will handle complex generative AI tasks for Apple Intelligence. This initiative aims to address privacy concerns and validate Apple's claims about the security of its AI infrastructure. The bug bounty program is part of Apple's efforts to build trust in its AI systems and improve...
read Oct 28, 2024AI models are fooled by common scams, study reveals
AI models vulnerable to scams: Recent research reveals that large language models (LLMs) powering popular chatbots are susceptible to the same scam techniques that deceive humans. Researchers from JP Morgan AI Research, led by Udari Madhushani Sehwag, conducted a study exposing three prominent LLMs to various scam scenarios. The models tested included OpenAI's GPT-3.5 and GPT-4, as well as Meta's Llama 2, which are behind widely-used chatbot applications. The study involved presenting 37 different scam scenarios to these AI models to assess their responses and vulnerability. Scam scenarios tested: The research team employed a diverse range of fraudulent situations to...
read Oct 28, 2024AI scams have come for knitters — why it matters beyond the world of crafting
AI-generated scams target lucrative knitting and crochet industry: Artificial intelligence is being used to create fake knitting and crochet patterns, causing frustration and financial loss for unsuspecting crafters in a multibillion-dollar industry. The knitting and crochet community, with millions of practitioners spending billions annually, has become an attractive target for scammers leveraging AI technology. AI-generated patterns often contain flaws that may not be immediately apparent, leading crafters to waste time, money, and materials on projects that ultimately fail. These scams highlight the broader issue of AI being used to mass-produce fake or flawed content across various domains, potentially impacting other...
read Oct 24, 2024The top tech trends shaping 2025, according to Gartner
The big picture: Gartner has unveiled its top strategic technology trends for 2025, highlighting key areas that enterprises should focus on to stay ahead in the rapidly evolving tech landscape. AI dominates the landscape: Artificial intelligence continues to be a major driving force in technological innovation, with several AI-related trends making Gartner's list. Agentic AI, which refers to intelligent software entities that can autonomously complete tasks and achieve goals, is expected to make 15% of day-to-day work decisions by 2028. AI governance platforms are becoming crucial for managing the legal, ethical, and operational aspects of AI systems, with the potential...
read Oct 24, 2024How cybercriminals are using sex bots to exploit their victims
AI-powered sex chat services exploit cloud vulnerabilities: Cybercriminals are increasingly using stolen cloud credentials to operate and resell AI-powered sex chat services, often bypassing content filters to engage in disturbing role-playing scenarios. Researchers at Permiso Security have observed a significant increase in attacks against generative AI infrastructure, particularly Amazon Web Services' (AWS) Bedrock, over the past six months. These attacks often stem from accidentally exposed cloud credentials or keys, such as those left in public code repositories like GitHub. Investigations revealed that many AWS users had not enabled logging, limiting visibility into the attackers' activities. Honeypot experiment reveals alarming trends:...
read Oct 23, 2024Security experts concerned with Claude’s new ability to control personal computers
Groundbreaking AI feature raises cybersecurity concerns: Anthropic's Claude AI has introduced a new "computer use" capability, allowing the AI to autonomously control users' computers, sparking both excitement and apprehension in the tech industry. Claude can now perform tasks like moving the cursor, opening web pages, typing text, and downloading files without direct human input. The feature is currently available to developers through the Claude API and in the Claude 3.5 Sonnet beta version. Major companies including Asana, Canva, and DoorDash are already testing the technology to automate complex multi-step tasks. Security experts sound the alarm: The introduction of this autonomous...
read