Category Archives: academic papers

Attacking the Performance of Machine Learning Systems

Posted on by

Interesting research: “Sponge Examples: Energy-Latency Attacks on Neural Networks“:

Abstract: The high energy costs of neural network training and inference led to the use of acceleration hardware such as GPUs and TPUs. While such devices enable us to train large-scale neural networks in datacenters and deploy them on edge devices, their designers’ focus so far is on average-case performance. In this work, we introduce a novel threat vector against neural networks whose energy consumption or decision latency are critical. We show how adversaries can exploit carefully-crafted sponge examples, which are inputs designed to maximise energy consumption and latency, to drive machine learning (ML) systems towards their worst-case performance. Sponge examples are, to our knowledge, the first denial-of-service attack against the ML components of such systems. We mount two variants of our sponge attack on a wide range of state-of-the-art neural network models, and find that language models are surprisingly vulnerable. Sponge examples frequently increase both latency and energy consumption of these models by a factor of 30×. Extensive experiments show that our new attack is effective across different hardware platforms (CPU, GPU and an ASIC simulator) on a wide range of different language tasks. On vision tasks, we show that sponge examples can be produced and a latency degradation observed, but the effect is less pronounced. To demonstrate the effectiveness of sponge examples in the real world, we mount an attack against Microsoft Azure’s translator and show an increase of response time from 1ms to 6s (6000×). We conclude by proposing a defense strategy: shifting the analysis of energy consumption in hardware from an average-case to a worst-case perspective…

Continue reading Attacking the Performance of Machine Learning Systems→

M1 Chip Vulnerability

Posted on by

This is a new vulnerability against Apple’s M1 chip. Researchers say that it is unpatchable.

Researchers from MIT’s Computer Science and Artificial Intelligence Laboratory, however, have created a novel hardware attack, which combines memory corruption and speculative execution attacks to sidestep the security feature. The attack shows that pointer authentication can be defeated without leaving a trace, and as it utilizes a hardware mechanism, no software patch can fix it.

The attack, appropriately called “Pacman,” works by “guessing” a pointer authentication code (PAC), a cryptographic signature that confirms that an app hasn’t been maliciously altered. This is done using speculative execution—a technique used by modern computer processors to speed up performance by speculatively guessing various lines of computation—to leak PAC verification results, while a hardware side-channel reveals whether or not the guess was correct…

Continue reading M1 Chip Vulnerability→

M1 Chip Vulnerability

Posted on by

This is a new vulnerability against Apple’s M1 chip. Researchers say that it is unpatchable.

Researchers from MIT’s Computer Science and Artificial Intelligence Laboratory, however, have created a novel hardware attack, which combines memory corruption and speculative execution attacks to sidestep the security feature. The attack shows that pointer authentication can be defeated without leaving a trace, and as it utilizes a hardware mechanism, no software patch can fix it.

The attack, appropriately called “Pacman,” works by “guessing” a pointer authentication code (PAC), a cryptographic signature that confirms that an app hasn’t been maliciously altered. This is done using speculative execution—a technique used by modern computer processors to speed up performance by speculatively guessing various lines of computation—to leak PAC verification results, while a hardware side-channel reveals whether or not the guess was correct…

Continue reading M1 Chip Vulnerability→

Remotely Controlling Touchscreens

Posted on by

Researchers have demonstrated controlling touchscreens at a distance, at least in a laboratory setting:

The core idea is to take advantage of the electromagnetic signals to execute basic touch events such as taps and swipes into targeted locations of the touchscreen with the goal of taking over remote control and manipulating the underlying device.

The attack, which works from a distance of up to 40mm, hinges on the fact that capacitive touchscreens are sensitive to EMI, leveraging it to inject electromagnetic signals into transparent electrodes that are built into the touchscreen so as to register them as touch events…

Continue reading Remotely Controlling Touchscreens→

The Limits of Cyber Operations in Wartime

Posted on by

Interesting paper by Lennart Maschmeyer: “The Subversive Trilemma: Why Cyber Operations Fall Short of Expectations“:

Abstract: Although cyber conflict has existed for thirty years, the strategic utility of cyber operations remains unclear. Many expect cyber operations to provide independent utility in both warfare and low-intensity competition. Underlying these expectations are broadly shared assumptions that information technology increases operational effectiveness. But a growing body of research shows how cyber operations tend to fall short of their promise. The reason for this shortfall is their subversive mechanism of action. In theory, subversion provides a way to exert influence at lower risks than force because it is secret and indirect, exploiting systems to use them against adversaries. The mismatch between promise and practice is the consequence of the subversive trilemma of cyber operations, whereby speed, intensity, and control are negatively correlated. These constraints pose a trilemma for actors because a gain in one variable tends to produce losses across the other two variables. A case study of the Russo-Ukrainian conflict provides empirical support for the argument. Qualitative analysis leverages original data from field interviews, leaked documents, forensic evidence, and local media. Findings show that the subversive trilemma limited the strategic utility of all five major disruptive cyber operations in this conflict…

Continue reading The Limits of Cyber Operations in Wartime→

Manipulating Machine-Learning Systems through the Order of the Training Data

Posted on by

Yet another adversarial ML attack:

Most deep neural networks are trained by stochastic gradient descent. Now “stochastic” is a fancy Greek word for “random”; it means that the training data are fed into the model in random order.

So what happens if the bad guys can cause the order to be not random? You guessed it—all bets are off. Suppose for example a company or a country wanted to have a credit-scoring system that’s secretly sexist, but still be able to pretend that its training was actually fair. Well, they could assemble a set of financial data that was representative of the whole population, but start the model’s training on ten rich men and ten poor women drawn from that set ­ then let initialisation bias do the rest of the work…

Continue reading Manipulating Machine-Learning Systems through the Order of the Training Data→

Websites that Collect Your Data as You Type

Posted on by

A surprising number of websites include JavaScript keyloggers that collect everything you type as you type it, not just when you submit a form.

Researchers from KU Leuven, Radboud University, and University of Lausanne crawled and analyzed the top 100,000 websites, looking at scenarios in which a user is visiting a site while in the European Union and visiting a site from the United States. They found that 1,844 websites gathered an EU user’s email address without their consent, and a staggering 2,950 logged a US user’s email in some form. Many of the sites seemingly do not intend to conduct the data-logging but incorporate third-party marketing and analytics services that cause the behavior…

Continue reading Websites that Collect Your Data as You Type→