Vulnerability Detection and Analysis in Adversarial Deep Learning

Machine learning has been applied in various information systems, but its vulnerability has not been well understood yet. This chapter studies vulnerability to adversarial machine learning in information systems such as online services with interfaces that accept user data inputs and return machine learning results such as labels. Two types of attacks are considered: exploratory (or inference) attack and evasion attack. In an exploratory attack, the adversary collects labels of input data from an online classifier and applies deep learning to train a functionally equivalent classifier without knowing the inner working of the target classifier. The vulnerability includes the theft of intellectual property (quantified by the statistical similarity of the target and inferred classifiers) and the support of other attacks built upon the inference results. An example of follow-up attacks is the evasion attack, where the adversary deceives the classifier into misclassifying input data samples that are systematically selected based on the classification scores from the inferred classier. This attack is strengthened by generative adversarial networks (GANs) and adversarial perturbations producing synthetic data samples that are likely to be misclassified. The vulnerability is measured by the increase in misdetection rates. This quantitative understanding of the vulnerability in machine learning systems provides valuable insights into designing defence mechanisms against adversarial machine learning.