Dynamic Differential Privacy for ADMM-Based Distributed Classification Learning

Tao Zhang, Quanyan Zhu

Research output: Contribution to journalArticle

Abstract

Privacy-preserving distributed machine learning becomes increasingly important due to the recent rapid growth of data. This paper focuses on a class of regularized empirical risk minimization machine learning problems, and develops two methods to provide differential privacy to distributed learning algorithms over a network. We first decentralize the learning algorithm using the alternating direction method of multipliers, and propose the methods of dual variable perturbation and primal variable perturbation to provide dynamic differential privacy. The two mechanisms lead to algorithms that can provide privacy guarantees under mild conditions of the convexity and differentiability of the loss function and the regularizer. We study the performance of the algorithms, and show that the dual variable perturbation outperforms its primal counterpart. To design an optimal privacy mechanism, we analyze the fundamental tradeoff between privacy and accuracy, and provide guidelines to choose privacy parameters. Numerical experiments using customer information database are performed to corroborate the results on privacy and utility tradeoffs and design.

Original languageEnglish (US)
Pages (from-to)172-187
Number of pages16
JournalIEEE Transactions on Information Forensics and Security
Volume12
Issue number1
DOIs
StatePublished - Jan 1 2017

Fingerprint

Learning algorithms
Learning systems
Parallel algorithms
Experiments
Optimal design

Keywords

  • ADMM
  • differential privacy
  • distributed computing
  • dynamic programming
  • Machine learning
  • privacy

ASJC Scopus subject areas

  • Safety, Risk, Reliability and Quality
  • Computer Networks and Communications

Cite this

Dynamic Differential Privacy for ADMM-Based Distributed Classification Learning. / Zhang, Tao; Zhu, Quanyan.

In: IEEE Transactions on Information Forensics and Security, Vol. 12, No. 1, 01.01.2017, p. 172-187.

Research output: Contribution to journalArticle

@article{7adb7bd239fa40f0943775c3c33e0272,
title = "Dynamic Differential Privacy for ADMM-Based Distributed Classification Learning",
abstract = "Privacy-preserving distributed machine learning becomes increasingly important due to the recent rapid growth of data. This paper focuses on a class of regularized empirical risk minimization machine learning problems, and develops two methods to provide differential privacy to distributed learning algorithms over a network. We first decentralize the learning algorithm using the alternating direction method of multipliers, and propose the methods of dual variable perturbation and primal variable perturbation to provide dynamic differential privacy. The two mechanisms lead to algorithms that can provide privacy guarantees under mild conditions of the convexity and differentiability of the loss function and the regularizer. We study the performance of the algorithms, and show that the dual variable perturbation outperforms its primal counterpart. To design an optimal privacy mechanism, we analyze the fundamental tradeoff between privacy and accuracy, and provide guidelines to choose privacy parameters. Numerical experiments using customer information database are performed to corroborate the results on privacy and utility tradeoffs and design.",
keywords = "ADMM, differential privacy, distributed computing, dynamic programming, Machine learning, privacy",
author = "Tao Zhang and Quanyan Zhu",
year = "2017",
month = "1",
day = "1",
doi = "10.1109/TIFS.2016.2607691",
language = "English (US)",
volume = "12",
pages = "172--187",
journal = "IEEE Transactions on Information Forensics and Security",
issn = "1556-6013",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
number = "1",

}

TY - JOUR

T1 - Dynamic Differential Privacy for ADMM-Based Distributed Classification Learning

AU - Zhang, Tao

AU - Zhu, Quanyan

PY - 2017/1/1

Y1 - 2017/1/1

N2 - Privacy-preserving distributed machine learning becomes increasingly important due to the recent rapid growth of data. This paper focuses on a class of regularized empirical risk minimization machine learning problems, and develops two methods to provide differential privacy to distributed learning algorithms over a network. We first decentralize the learning algorithm using the alternating direction method of multipliers, and propose the methods of dual variable perturbation and primal variable perturbation to provide dynamic differential privacy. The two mechanisms lead to algorithms that can provide privacy guarantees under mild conditions of the convexity and differentiability of the loss function and the regularizer. We study the performance of the algorithms, and show that the dual variable perturbation outperforms its primal counterpart. To design an optimal privacy mechanism, we analyze the fundamental tradeoff between privacy and accuracy, and provide guidelines to choose privacy parameters. Numerical experiments using customer information database are performed to corroborate the results on privacy and utility tradeoffs and design.

AB - Privacy-preserving distributed machine learning becomes increasingly important due to the recent rapid growth of data. This paper focuses on a class of regularized empirical risk minimization machine learning problems, and develops two methods to provide differential privacy to distributed learning algorithms over a network. We first decentralize the learning algorithm using the alternating direction method of multipliers, and propose the methods of dual variable perturbation and primal variable perturbation to provide dynamic differential privacy. The two mechanisms lead to algorithms that can provide privacy guarantees under mild conditions of the convexity and differentiability of the loss function and the regularizer. We study the performance of the algorithms, and show that the dual variable perturbation outperforms its primal counterpart. To design an optimal privacy mechanism, we analyze the fundamental tradeoff between privacy and accuracy, and provide guidelines to choose privacy parameters. Numerical experiments using customer information database are performed to corroborate the results on privacy and utility tradeoffs and design.

KW - ADMM

KW - differential privacy

KW - distributed computing

KW - dynamic programming

KW - Machine learning

KW - privacy

UR - http://www.scopus.com/inward/record.url?scp=85020567479&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85020567479&partnerID=8YFLogxK

U2 - 10.1109/TIFS.2016.2607691

DO - 10.1109/TIFS.2016.2607691

M3 - Article

VL - 12

SP - 172

EP - 187

JO - IEEE Transactions on Information Forensics and Security

JF - IEEE Transactions on Information Forensics and Security

SN - 1556-6013

IS - 1

ER -