This is Adversa’s overview of the August 2019 research in the field of AI Security. As usual, we are covering adversarial attacks that allow us to build better defenses. This month we are blown away by adversarial commands to speech recognition systems that cannot be heard and successful adversarial intrusions into face recognition. In August significant progress was also made with regard to protection. We are breaking down 2 papers that detail how to better approach defenses in Human Pose Estimation and Intrusion Detection Systems.
Attacks on speech recognition systems (ASRs) have to get over several hurdles to be considered truly threatening. First of all, they have to be fed to the system over the air. Second, they have to be black-box, i.e. should be possible to perform without knowing anything about the target model. Third, they must be difficult to hear or imperceptible. And finally, they should function in any environment.
The researchers from the Ruhr-University Bochum created the first attack to fit all the above criteria. It is fittingly called Imperio. The approach is based on the Expectation Over Transformation (EOT) framework that models adversarial transformations within the optimization procedure. In the case of Imperio, transformations are room impulse responses, i.e. perturbations of sound caused by the shape and content of the room.
Imperio is an attack that can be fed to any ASR in any environment and cannot be heard by a human. A defense against such attacks is training ASRs to recognize them. Training ASRs to ignore all sounds imperceptible to humans is also an option. Otherwise, your pet Alexa or Siri may soon be carrying out somebody else’s commands.
To see how much progress has been made in this sphere of research, check out our April digest.
Face recognition systems (AFR) protect data on our phones, secure borders, and help law enforcement. Yet, when it comes to adversarial attacks, they can be the weak link in a security system. AFRs can be tricked into matching a person with a wrong profile, which is called an impersonation attack. They can also be forced to ignore the person altogether in case of an obfuscation attack.
Previous approaches to generating adversarial faces were impractical. They required knowledge of the model (white-box setting) and several photos of the same person. They took a long time and produced images that looked unnatural. Deb et al. managed to overcome all the previous limitations. They developed AdvFaces, “an automated adversarial face synthesis method that learns to generate minimal perturbations in the salient facial regions via Generative Adversarial Networks”. AdvFaces is trained in a white-box setting, and can then be used to attack any face recognition algorithm. It is successful 97% of the time in impersonation attacks and 24% — in obfuscation.
Human-pose estimation (HPE) uses regression and classification to find human joints and infer pose. Its exciting uses include augmented reality, gaming and animation, human-computer and computer-assisted living. All machine learning algorithms, including the ones that do HPE, are susceptible to adversarial attacks. What is interesting, though, is that such architectures are relatively robust, especially to single-step attacks, those generated by non-iterative methods. At the same time, attacks aimed at features deep within the model, rather than the output, are effective even if they are not model-specific. Shah et al. state that heatmap-based models are more robust than regression-based ones. And Stacked Hourglass HPE networks become more robust when the number of hourglasses is increased to 8.
The first line of defense against cyber threats are NIDS, Network Intrusion Detection Systems. They check incoming traffic for malicious activity. One way to improve these systems, train them to recognize new types of attacks, is to use machine learning. Generative Adversarial Networks (GANs) are a machine learning framework capable of creating mutated iterations of attacks that are fed into it.
Charlier et al. propose SynGAN, a framework that builds on Goodfelow’s GANs. It uses these generated attacks to train NIDS to recognize new approaches used by adversaries. The Generator part of it creates synthetic network attacks using the Gradient Penalty-Wasserstein Generative Adversarial Networks algorithm. The Discriminator then tries to tell apart the synthetic and the real attacks and gives feedback to the Generator. And the Evaluator attempts to differentiate between real and generated attacks. With Wasserstein distance and the gradient penalty, the generator and the discriminator improve at the same pace, which allows the optimization of the network’s weights.
At this point, the researchers are able to generate synthetic DDoS attacks. They are moving towards their final goal of being able to create enough varied adversarial traffic to test the effectiveness of commercial NIDS.
Check out more of our digests in Adversa’s blog. And tune in to our Twitter to keep up with new developments in AI Security.
This is Adversa’s monthly digest for July 2019. The power of Artificial Intelligence is in its unwavering attention and grand analytical capabilities. It makes sense to turn to AI when ...
