A dual perturbation approach for differential private admm-based distributed empirical risk minimization

Tao Zhang, Quanyan Zhu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

The rapid growth of data has raised the importance of privacy-preserving techniques in distributed machine learning. In this paper, we develop a privacy-preserving method to a class of regularized empirical risk minimization (ERM) machine learning problems. We first decentralize the learning algorithm using the alternating direction method of multipliers (ADMM), and propose the method of dual variable perturbation to provide dynamic differential privacy. The mechanism leads to a privacy-preserving algorithm under mild conditions of the convexity and differentiability of the loss function and the regularizer. We study the performance of the algorithm measured by the number of data points required to achieve a bounded error. To design an optimal privacy mechanism, we analyze the fundamental tradeoff between privacy and accuracy, and provide guidelines to choose privacy parameters. Numerical experiments using the realworld database are performed to corroborate the results on the privacy and utility tradeoffs and design.

Original languageEnglish (US)
Title of host publicationAISec 2016 - Proceedings of the 2016 ACM Workshop on Artificial Intelligence and Security, co-located with CCS 2016
PublisherAssociation for Computing Machinery, Inc
Pages129-137
Number of pages9
ISBN (Electronic)9781450345736
DOIs
StatePublished - Oct 28 2016
Event9th ACM Workshop on Artificial Intelligence and Security, AISec 2016 - Vienna, Austria
Duration: Oct 28 2016 → …

Other

Other9th ACM Workshop on Artificial Intelligence and Security, AISec 2016
CountryAustria
CityVienna
Period10/28/16 → …

Fingerprint

Learning systems
Learning algorithms
Experiments
Optimal design

Keywords

  • ADMM
  • Differential privacy
  • Distributed optimization
  • Machine learning
  • Privacy tradeoffs

ASJC Scopus subject areas

  • Artificial Intelligence

Cite this

Zhang, T., & Zhu, Q. (2016). A dual perturbation approach for differential private admm-based distributed empirical risk minimization. In AISec 2016 - Proceedings of the 2016 ACM Workshop on Artificial Intelligence and Security, co-located with CCS 2016 (pp. 129-137). Association for Computing Machinery, Inc. https://doi.org/10.1145/2996758.2996762

A dual perturbation approach for differential private admm-based distributed empirical risk minimization. / Zhang, Tao; Zhu, Quanyan.

AISec 2016 - Proceedings of the 2016 ACM Workshop on Artificial Intelligence and Security, co-located with CCS 2016. Association for Computing Machinery, Inc, 2016. p. 129-137.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Zhang, T & Zhu, Q 2016, A dual perturbation approach for differential private admm-based distributed empirical risk minimization. in AISec 2016 - Proceedings of the 2016 ACM Workshop on Artificial Intelligence and Security, co-located with CCS 2016. Association for Computing Machinery, Inc, pp. 129-137, 9th ACM Workshop on Artificial Intelligence and Security, AISec 2016, Vienna, Austria, 10/28/16. https://doi.org/10.1145/2996758.2996762
Zhang T, Zhu Q. A dual perturbation approach for differential private admm-based distributed empirical risk minimization. In AISec 2016 - Proceedings of the 2016 ACM Workshop on Artificial Intelligence and Security, co-located with CCS 2016. Association for Computing Machinery, Inc. 2016. p. 129-137 https://doi.org/10.1145/2996758.2996762
Zhang, Tao ; Zhu, Quanyan. / A dual perturbation approach for differential private admm-based distributed empirical risk minimization. AISec 2016 - Proceedings of the 2016 ACM Workshop on Artificial Intelligence and Security, co-located with CCS 2016. Association for Computing Machinery, Inc, 2016. pp. 129-137
@inproceedings{128a8e1f088243d1ac6f5451a04cadad,
title = "A dual perturbation approach for differential private admm-based distributed empirical risk minimization",
abstract = "The rapid growth of data has raised the importance of privacy-preserving techniques in distributed machine learning. In this paper, we develop a privacy-preserving method to a class of regularized empirical risk minimization (ERM) machine learning problems. We first decentralize the learning algorithm using the alternating direction method of multipliers (ADMM), and propose the method of dual variable perturbation to provide dynamic differential privacy. The mechanism leads to a privacy-preserving algorithm under mild conditions of the convexity and differentiability of the loss function and the regularizer. We study the performance of the algorithm measured by the number of data points required to achieve a bounded error. To design an optimal privacy mechanism, we analyze the fundamental tradeoff between privacy and accuracy, and provide guidelines to choose privacy parameters. Numerical experiments using the realworld database are performed to corroborate the results on the privacy and utility tradeoffs and design.",
keywords = "ADMM, Differential privacy, Distributed optimization, Machine learning, Privacy tradeoffs",
author = "Tao Zhang and Quanyan Zhu",
year = "2016",
month = "10",
day = "28",
doi = "10.1145/2996758.2996762",
language = "English (US)",
pages = "129--137",
booktitle = "AISec 2016 - Proceedings of the 2016 ACM Workshop on Artificial Intelligence and Security, co-located with CCS 2016",
publisher = "Association for Computing Machinery, Inc",

}

TY - GEN

T1 - A dual perturbation approach for differential private admm-based distributed empirical risk minimization

AU - Zhang, Tao

AU - Zhu, Quanyan

PY - 2016/10/28

Y1 - 2016/10/28

N2 - The rapid growth of data has raised the importance of privacy-preserving techniques in distributed machine learning. In this paper, we develop a privacy-preserving method to a class of regularized empirical risk minimization (ERM) machine learning problems. We first decentralize the learning algorithm using the alternating direction method of multipliers (ADMM), and propose the method of dual variable perturbation to provide dynamic differential privacy. The mechanism leads to a privacy-preserving algorithm under mild conditions of the convexity and differentiability of the loss function and the regularizer. We study the performance of the algorithm measured by the number of data points required to achieve a bounded error. To design an optimal privacy mechanism, we analyze the fundamental tradeoff between privacy and accuracy, and provide guidelines to choose privacy parameters. Numerical experiments using the realworld database are performed to corroborate the results on the privacy and utility tradeoffs and design.

AB - The rapid growth of data has raised the importance of privacy-preserving techniques in distributed machine learning. In this paper, we develop a privacy-preserving method to a class of regularized empirical risk minimization (ERM) machine learning problems. We first decentralize the learning algorithm using the alternating direction method of multipliers (ADMM), and propose the method of dual variable perturbation to provide dynamic differential privacy. The mechanism leads to a privacy-preserving algorithm under mild conditions of the convexity and differentiability of the loss function and the regularizer. We study the performance of the algorithm measured by the number of data points required to achieve a bounded error. To design an optimal privacy mechanism, we analyze the fundamental tradeoff between privacy and accuracy, and provide guidelines to choose privacy parameters. Numerical experiments using the realworld database are performed to corroborate the results on the privacy and utility tradeoffs and design.

KW - ADMM

KW - Differential privacy

KW - Distributed optimization

KW - Machine learning

KW - Privacy tradeoffs

UR - http://www.scopus.com/inward/record.url?scp=85002424699&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85002424699&partnerID=8YFLogxK

U2 - 10.1145/2996758.2996762

DO - 10.1145/2996758.2996762

M3 - Conference contribution

AN - SCOPUS:85002424699

SP - 129

EP - 137

BT - AISec 2016 - Proceedings of the 2016 ACM Workshop on Artificial Intelligence and Security, co-located with CCS 2016

PB - Association for Computing Machinery, Inc

ER -