Category Archives: LLM

ChatGPT 4 can exploit 87% of one-day vulnerabilities

Posted on by

Since the widespread and growing use of ChatGPT and other large language models (LLMs) in recent years, cybersecurity has been a top concern. Among the many questions, cybersecurity professionals wondered how effective these tools were in launching an attack. Cybersecurity researchers Richard Fang, Rohan Bindu, Akul Gupta and Daniel Kang recently performed a study to […]

The post ChatGPT 4 can exploit 87% of one-day vulnerabilities appeared first on Security Intelligence.

Continue reading ChatGPT 4 can exploit 87% of one-day vulnerabilities→

How generative AI Is expanding the insider threat attack surface

Posted on by

As the adoption of generative AI (GenAI) soars, so too does the risk of insider threats. This puts even more pressure on businesses to rethink security and confidentiality policies. In just a few years, artificial intelligence (AI) has radically changed the world of work. 61% of knowledge workers now use GenAI tools — particularly OpenAI’s […]

The post How generative AI Is expanding the insider threat attack surface appeared first on Security Intelligence.

Continue reading How generative AI Is expanding the insider threat attack surface→

Using LLMs to Exploit Vulnerabilities

Posted on by

Interesting research: “Teams of LLM Agents can Exploit Zero-Day Vulnerabilities.”

Abstract: LLM agents have become increasingly sophisticated, especially in the realm of cybersecurity. Researchers have shown that LLM agents can exploit real-world vulnerabilities when given a description of the vulnerability and toy capture-the-flag problems. However, these agents still perform poorly on real-world vulnerabilities that are unknown to the agent ahead of time (zero-day vulnerabilities).

In this work, we show that teams of LLM agents can exploit real-world, zero-day vulnerabilities. Prior agents struggle with exploring many different vulnerabilities and long-range planning when used alone. To resolve this, we introduce HPTSA, a system of agents with a planning agent that can launch subagents. The planning agent explores the system and determines which subagents to call, resolving long-term planning issues when trying different vulnerabilities. We construct a benchmark of 15 real-world vulnerabilities and show that our team of agents improve over prior work by up to 4.5×…

Continue reading Using LLMs to Exploit Vulnerabilities→

Using AI for Political Polling

Posted on by

Public polling is a critical function of modern political campaigns and movements, but it isn’t what it once was. Recent US election cycles have produced copious postmortems explaining both the successes and the flaws of public polling. There are two main reasons polling fails.

First, nonresponse has skyrocketed. It’s radically harder to reach people than it used to be. Few people fill out surveys that come in the mail anymore. Few people answer their phone when a stranger calls. Pew Research reported that 36% of the people they called in 1997 would talk to them, but only 6% by 2018. Pollsters worldwide have faced similar challenges…

Continue reading Using AI for Political Polling→

LLMs Acting Deceptively

Posted on by

New research: “Deception abilities emerged in large language models“:

Abstract: Large language models (LLMs) are currently at the forefront of intertwining AI systems with human communication and everyday life. Thus, aligning them with human values is of great importance. However, given the steady increase in reasoning abilities, future LLMs are under suspicion of becoming able to deceive human operators and utilizing this ability to bypass monitoring efforts. As a prerequisite to this, LLMs need to possess a conceptual understanding of deception strategies. This study reveals that such strategies emerged in state-of-the-art LLMs, but were nonexistent in earlier LLMs. We conduct a series of experiments showing that state-of-the-art LLMs are able to understand and induce false beliefs in other agents, that their performance in complex deception scenarios can be amplified utilizing chain-of-thought reasoning, and that eliciting Machiavellianism in LLMs can trigger misaligned deceptive behavior. GPT-4, for instance, exhibits deceptive behavior in simple test scenarios 99.16% of the time (P < 0.001). In complex second-order deception test scenarios where the aim is to mislead someone who expects to be deceived, GPT-4 resorts to deceptive behavior 71.46% of the time (P < 0.001) when augmented with chain-of-thought reasoning. In sum, revealing hitherto unknown machine behavior in LLMs, our study contributes to the nascent field of machine psychology…

Continue reading LLMs Acting Deceptively→

AI Will Increase the Quantity—and Quality—of Phishing Scams

Posted on by

A piece I coauthored with Fredrik Heiding and Arun Vishwanath in the Harvard Business Review:

Summary. Gen AI tools are rapidly making these emails more advanced, harder to spot, and significantly more dangerous. Recent research showed that 60% of participants fell victim to artificial intelligence (AI)-automated phishing, which is comparable to the success rates of non-AI-phishing messages created by human experts. Companies need to: 1) understand the asymmetrical capabilities of AI-enhanced phishing, 2) determine the company or division’s phishing threat severity level, and 3) confirm their current phishing awareness routines…

Continue reading AI Will Increase the Quantity—and Quality—of Phishing Scams→

How AI Will Change Democracy

Posted on by

I don’t think it’s an exaggeration to predict that artificial intelligence will affect every aspect of our society. Not by doing new things. But mostly by doing things that are already being done by humans, perfectly competently.

Replacing humans with AIs isn’t necessarily interesting. But when an AI takes over a human task, the task changes.

In particular, there are potential changes over four dimensions: Speed, scale, scope and sophistication. The problem with AIs trading stocks isn’t that they’re better than humans—it’s that they’re faster. But computers are better at chess and Go because they use more sophisticated strategies than humans. We’re worried about AI-controlled social media accounts because they operate on a superhuman scale…

Continue reading How AI Will Change Democracy→

Snowflake Arctic, a New AI LLM for Enterprise Tasks, is Coming to APAC

Posted on by

Data cloud company Snowflake’s Arctic is promising to provide APAC businesses with a true open source large language model they can use to train their own custom enterprise LLMs and inference more economically. Continue reading Snowflake Arctic, a New AI LLM for Enterprise Tasks, is Coming to APAC→