This is Adversa’s monthly digest for July 2019. The power of Artificial Intelligence is in its unwavering attention and grand analytical capabilities. It makes sense to turn to AI when the stakes are as high as human lives, right? In this overview we dive into the attacks on autonomous cars and medical imaging.
At the end there we’ll also touch on the topic of adversarial text.
Successful attacks on image classification systems in real-world conditions are challenging. An adversarial example has to be robust to changes in lighting, viewing distance and angle. Attacking a traffic sign recognition system adds an extra layer of difficulty. Traffic signs have a simplistic design. If perturbations are to remain unnoticed by drivers, no new colors or patterns can be introduced. And finally, an attack directed at a commercial computer vision algorithm, such as the one in autonomous cars, has to be black-box. It means that an attacker works to fool a classifier without knowing its parameters or architecture.
Morgulis et al. manage to overcome all those challenges by expanding on the known DARTS pipeline. Their work has proven that it is possible to fool a commercial self-driving car and do so consistently. The proposed attack is targeted, which means perturbed signs are misread in a predetermined way, e.g. 70 miles per hour instead of 40. Their attack can also act as a Denial-of-Service (DoS) attack and freeze a vehicle’s system for ~1 minute.
Overall, the work of Morgulis et al. shatters the idea that adversarial examples are not transferable to real life.
Most attacks on self-driving cars target image-based classifiers. But commercial autonomous driving systems perceive their environment in 3D using laser beams. And these systems are significantly more complex as they consist of many indistinguishable steps. So Cao et al. developed LiDAR-Adv, an optimization-based black-box approach specific to 3D detection systems. It generates 3D adversarial objects that are either ignored or labeled as a specific adversarial target. Cao et al. tested their findings in real-world drive-by experiments. They achieved up to 70% rate of success depending on the size of the object and the nature of the attack.
This work makes us question whether even the most complex autonomous driving systems are secure enough.
In the medical field, AI can be used for diagnosis, e.g. in dermoscopy, lung and cell imaging. Medical images are different from natural ones: they are larger, more textured, less legible. These characteristics make deep neural network (DNN) models, more susceptible to adversarial attacks.
The study done by Ma et al. focused on the most common applications of AI in medicine. They study deep neural networks (DNN) used to analyze medical images. It turns out that such models are more susceptible to adversarial attacks than the ones designed for regular images. The researchers found the following reasons behind the disparity:
What is interesting is that these same characteristics make it easy to identify adversarial attacks in medical imaging. So much so that even a simple detector can achieve 98% success. This, the researchers say should be seen as the basis to build defenses.
Read more about the vulnerabilities of medical machine learning in our March digest.
A convincing adversarial text needs to only fool algorithms but also keep the meaning, the natural look, and the grammar of the original. This is a lot of benchmarks and most attacks had been falling short of at least one of them. That is until Jin et al. developed TextFooler, a baseline for creating natural adversarial text in a black-box setting. It forces text classifiers to assign a wrong label to a piece of writing. TextFooler can reduce the accuracy of BERT, the strongest language model so far, by 5–7 times on the classification task and over 20 times on the NLI (natural language inference) task. Here’s how it works:
In addition to developing the attack, Jin et al. completed an evaluation framework that includes automatic and human assessments. Both the attack and the framework provide insight into possible ways of robustness improvement for text classifiers and textual entailment models.
Check out more of our digests in Adversa’s blog. And tune in to our Twitter to keep up to date with new developments in AI Security.
Adversa is once again sharing the research that captured our interest. In June 2019 we marveled at the incredibly effective new adversarial model with a whopping 97% success rate, learned ...
