Collaborate Disseminate
The self-assessment, one of three parts, is designed to guide businesses in using AI more responsibly. Continue reading U.K. Government Introduces Self-Assessment Tool to Help Businesses Manage AI Use
Interesting research: “Hacking Back the AI-Hacker: Prompt Injection as a Defense Against LLM-driven Cyberattacks“:
Large language models (LLMs) are increasingly being harnessed to automate cyberattacks, making sophisticated exploits more accessible and scalable. In response, we propose a new defense strategy tailored to counter LLM-driven cyberattacks. We introduce Mantis, a defensive framework that exploits LLMs’ susceptibility to adversarial inputs to undermine malicious operations. Upon detecting an automated cyberattack, Mantis plants carefully crafted inputs into system responses, leading the attacker’s LLM to disrupt their own operations (passive defense) or even compromise the attacker’s machine (active defense). By deploying purposefully vulnerable decoy services to attract the attacker and using dynamic prompt injections for the attacker’s LLM, Mantis can autonomously hack back the attacker. In our experiments, Mantis consistently achieved over 95% effectiveness against automated LLM-driven attacks. To foster further research and collaboration, Mantis is available as an open-source tool: …
Continue reading Prompt Injection Defenses Against LLM Cyberattacks
Really interesting research: “An LLM-Assisted Easy-to-Trigger Backdoor Attack on Code Completion Models: Injecting Disguised Vulnerabilities against Strong Detection“:
Abstract: Large Language Models (LLMs) have transformed code com-
pletion tasks, providing context-based suggestions to boost developer productivity in software engineering. As users often fine-tune these models for specific applications, poisoning and backdoor attacks can covertly alter the model outputs. To address this critical security challenge, we introduce CODEBREAKER, a pioneering LLM-assisted backdoor attack framework on code completion models. Unlike recent attacks that embed malicious payloads in detectable or irrelevant sections of the code (e.g., comments), CODEBREAKER leverages LLMs (e.g., GPT-4) for sophisticated payload transformation (without affecting functionalities), ensuring that both the poisoned data for fine-tuning and generated code can evade strong vulnerability detection. CODEBREAKER stands out with its comprehensive coverage of vulnerabilities, making it the first to provide such an extensive set for evaluation. Our extensive experimental evaluations and user studies underline the strong attack performance of CODEBREAKER across various settings, validating its superiority over existing approaches. By integrating malicious payloads directly into the source code with minimal transformation, CODEBREAKER challenges current security measures, underscoring the critical need for more robust defenses for code completion…
In this Help Net Security interview, Buzz Hillestad, CISO at Prismatic, discusses how AI’s advancement reshapes cybercriminal skillsets and lowers entry barriers for potential attackers. Hillestad highlights that, as AI tools become more accessib… Continue reading How AI will shape the next generation of cyber threats
AI-enhanced malicious attacks are a top concern for 80% of executives, and for good reason, as there is a lot of evidence that bad actors are exploiting the technology. Continue reading AI-Assisted Attacks Top Cyber Threat For Third Consecutive Quarter, Gartner Finds
About a year after Broadcom’s acquisition of VMware, the company released VMware Tanzu Data Services to make connections to some third-party data engines easier. Continue reading VMware Explore Barcelona 2024: Tanzu Platform 10 Enters General Availability
GPU supplier NVIDIA announced its plans to purchase compute management platform Run:ai in April. Continue reading EU Investigates NVIDIA Deal With Run:ai, iPad OS Compliance with Digital Markets Act
The advent of Generative AI and its application in real-life use cases has been on the cards for… Continue reading Enhance customer experiences with Generative AI
It was Supercon this weekend, and Hackaday staffers made their way to Pasadena for what was by all accounts an excellent event. Now they’re all on their way home on …read more Continue reading AI Not Needed For Hackaday Projects
I’ve been writing about the possibility of AIs automatically discovering code vulnerabilities since at least 2018. This is an ongoing area of research: AIs doing source code scanning, AIs finding zero-days in the wild, and everything in between. The AIs aren’t very good at it yet, but they’re getting better.
Here’s some anecdotal data from this summer:
Since July 2024, ZeroPath is taking a novel approach combining deep program analysis with adversarial AI agents for validation. Our methodology has uncovered numerous critical vulnerabilities in production systems, including several that traditional Static Application Security Testing (SAST) tools were ill-equipped to find. This post provides a technical deep-dive into our research methodology and a living summary of the bugs found in popular open-source tools…