On the detection of adversarial attacks against deep neural networks

Weiyu Wang, Quanyan Zhu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Deep learning model has been widely studied and proven to achieve high accuracy in various pattern recognition tasks, especially in image recognition. However, due to its non-linear architecture and high-dimensional inputs, its ill-posedness [1] towards adversarial perturbations - small deliberately crafted perturbations on the input will lead to completely different outputs, has also attracted researchers' attention. 1 This work takes the traffic sign recognition system on the selfdriving car as an example, and aims at designing an additional mechanism to improve the robustness of the recognition system. It uses a machine learning model which learns the results of the deep learning model's predictions, with human feedback as labels and provides the credibility of current prediction. The mechanism makes use of both the input image and the recognition result as the sample space, querying a human user the True/False of current classification result the least number of times, and completing the task of detecting adversarial attacks.

Original languageEnglish (US)
Title of host publicationSafeConfig 2017 - Proceedings of the 2017 Workshop on Automated Decision Making for Active Cyber Defense, co-located with CCS 2017
PublisherAssociation for Computing Machinery, Inc
Pages27-30
Number of pages4
ISBN (Electronic)9781450352031
DOIs
StatePublished - Nov 3 2017
Event10th Workshop on Automated Decision Making for Active Cyber Defense, SafeConfig 2017 - Dallas, United States
Duration: Nov 3 2017 → …

Other

Other10th Workshop on Automated Decision Making for Active Cyber Defense, SafeConfig 2017
CountryUnited States
CityDallas
Period11/3/17 → …

Fingerprint

Traffic signs
Image recognition
Pattern recognition
Learning systems
Labels
Railroad cars
Feedback
Deep neural networks
Deep learning

Keywords

  • Active Learning
  • Adversarial Machine Learning
  • Deep Neural Network
  • Machine Learning Security
  • Support Vector Machine

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computational Theory and Mathematics
  • Computer Science Applications

Cite this

Wang, W., & Zhu, Q. (2017). On the detection of adversarial attacks against deep neural networks. In SafeConfig 2017 - Proceedings of the 2017 Workshop on Automated Decision Making for Active Cyber Defense, co-located with CCS 2017 (pp. 27-30). Association for Computing Machinery, Inc. https://doi.org/10.1145/3140368.3140373

On the detection of adversarial attacks against deep neural networks. / Wang, Weiyu; Zhu, Quanyan.

SafeConfig 2017 - Proceedings of the 2017 Workshop on Automated Decision Making for Active Cyber Defense, co-located with CCS 2017. Association for Computing Machinery, Inc, 2017. p. 27-30.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Wang, W & Zhu, Q 2017, On the detection of adversarial attacks against deep neural networks. in SafeConfig 2017 - Proceedings of the 2017 Workshop on Automated Decision Making for Active Cyber Defense, co-located with CCS 2017. Association for Computing Machinery, Inc, pp. 27-30, 10th Workshop on Automated Decision Making for Active Cyber Defense, SafeConfig 2017, Dallas, United States, 11/3/17. https://doi.org/10.1145/3140368.3140373
Wang W, Zhu Q. On the detection of adversarial attacks against deep neural networks. In SafeConfig 2017 - Proceedings of the 2017 Workshop on Automated Decision Making for Active Cyber Defense, co-located with CCS 2017. Association for Computing Machinery, Inc. 2017. p. 27-30 https://doi.org/10.1145/3140368.3140373
Wang, Weiyu ; Zhu, Quanyan. / On the detection of adversarial attacks against deep neural networks. SafeConfig 2017 - Proceedings of the 2017 Workshop on Automated Decision Making for Active Cyber Defense, co-located with CCS 2017. Association for Computing Machinery, Inc, 2017. pp. 27-30
@inproceedings{7316950f83fa4968b9d7da2e8751c7f7,
title = "On the detection of adversarial attacks against deep neural networks",
abstract = "Deep learning model has been widely studied and proven to achieve high accuracy in various pattern recognition tasks, especially in image recognition. However, due to its non-linear architecture and high-dimensional inputs, its ill-posedness [1] towards adversarial perturbations - small deliberately crafted perturbations on the input will lead to completely different outputs, has also attracted researchers' attention. 1 This work takes the traffic sign recognition system on the selfdriving car as an example, and aims at designing an additional mechanism to improve the robustness of the recognition system. It uses a machine learning model which learns the results of the deep learning model's predictions, with human feedback as labels and provides the credibility of current prediction. The mechanism makes use of both the input image and the recognition result as the sample space, querying a human user the True/False of current classification result the least number of times, and completing the task of detecting adversarial attacks.",
keywords = "Active Learning, Adversarial Machine Learning, Deep Neural Network, Machine Learning Security, Support Vector Machine",
author = "Weiyu Wang and Quanyan Zhu",
year = "2017",
month = "11",
day = "3",
doi = "10.1145/3140368.3140373",
language = "English (US)",
pages = "27--30",
booktitle = "SafeConfig 2017 - Proceedings of the 2017 Workshop on Automated Decision Making for Active Cyber Defense, co-located with CCS 2017",
publisher = "Association for Computing Machinery, Inc",

}

TY - GEN

T1 - On the detection of adversarial attacks against deep neural networks

AU - Wang, Weiyu

AU - Zhu, Quanyan

PY - 2017/11/3

Y1 - 2017/11/3

N2 - Deep learning model has been widely studied and proven to achieve high accuracy in various pattern recognition tasks, especially in image recognition. However, due to its non-linear architecture and high-dimensional inputs, its ill-posedness [1] towards adversarial perturbations - small deliberately crafted perturbations on the input will lead to completely different outputs, has also attracted researchers' attention. 1 This work takes the traffic sign recognition system on the selfdriving car as an example, and aims at designing an additional mechanism to improve the robustness of the recognition system. It uses a machine learning model which learns the results of the deep learning model's predictions, with human feedback as labels and provides the credibility of current prediction. The mechanism makes use of both the input image and the recognition result as the sample space, querying a human user the True/False of current classification result the least number of times, and completing the task of detecting adversarial attacks.

AB - Deep learning model has been widely studied and proven to achieve high accuracy in various pattern recognition tasks, especially in image recognition. However, due to its non-linear architecture and high-dimensional inputs, its ill-posedness [1] towards adversarial perturbations - small deliberately crafted perturbations on the input will lead to completely different outputs, has also attracted researchers' attention. 1 This work takes the traffic sign recognition system on the selfdriving car as an example, and aims at designing an additional mechanism to improve the robustness of the recognition system. It uses a machine learning model which learns the results of the deep learning model's predictions, with human feedback as labels and provides the credibility of current prediction. The mechanism makes use of both the input image and the recognition result as the sample space, querying a human user the True/False of current classification result the least number of times, and completing the task of detecting adversarial attacks.

KW - Active Learning

KW - Adversarial Machine Learning

KW - Deep Neural Network

KW - Machine Learning Security

KW - Support Vector Machine

UR - http://www.scopus.com/inward/record.url?scp=85037084091&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85037084091&partnerID=8YFLogxK

U2 - 10.1145/3140368.3140373

DO - 10.1145/3140368.3140373

M3 - Conference contribution

AN - SCOPUS:85037084091

SP - 27

EP - 30

BT - SafeConfig 2017 - Proceedings of the 2017 Workshop on Automated Decision Making for Active Cyber Defense, co-located with CCS 2017

PB - Association for Computing Machinery, Inc

ER -