626_privacy/tensorflow_privacy
2
0
Fork 1
Code Issues Pull requests 1 Projects Releases Packages Wiki Activity Actions

Add an announcement on the public README about the new fast implementation of DP-SGD.

Browse source 
PiperOrigin-RevId: 512930920
This commit is contained in:
Andres Munoz MEdina 2023-02-28 07:45:04 -08:00 committed by A. Unique TensorFlower
parent a3e8a45559
commit d7cd3f8af1
1 changed files with 27 additions and 27 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

48
README.md
View file

@ -11,13 +11,14 @@ issues currently open.
## Latest Updates ## Latest Updates
2020-12-21: A new 2023-02-21: A new implementation of efficient per-example gradient clipping is
[vectorized version of the TF 2 optimizer](https://github.com/tensorflow/privacy/blob/master/tensorflow_privacy/privacy/optimizers/dp_optimizer_keras_vectorized.py) now available for
is available, which can deliver much faster performance. We recommend trying it [DP keras models](https://github.com/tensorflow/privacy/tree/master/tensorflow_privacy/privacy/keras_models)
first, and to fall back to using the original non-vectorized version only if consisting only of Dense and Embedding layers. The models use the fast gradient
this fails. We are thankful to the calculation results of [this paper](https://arxiv.org/abs/1510.01799). The
[authors of this paper](https://arxiv.org/abs/2010.09063) for spurring this implementation should allow for doing DP training without any meaningful memory
change. or runtime overhead. It also removes the need for tuning the number of
microbatches as it clips the gradient with respect to each example.
## Setting up TensorFlow Privacy ## Setting up TensorFlow Privacy
@ -32,11 +33,11 @@ installation documentation).
In addition to TensorFlow and its dependencies, other prerequisites are: In addition to TensorFlow and its dependencies, other prerequisites are:
* `scipy` >= 0.17 * `scipy` >= 0.17
* `mpmath` (for testing) * `mpmath` (for testing)
* `tensorflow_datasets` (for the RNN tutorial `lm_dpsgd_tutorial.py` only) * `tensorflow_datasets` (for the RNN tutorial `lm_dpsgd_tutorial.py` only)
### Installing TensorFlow Privacy ### Installing TensorFlow Privacy
@ -84,18 +85,16 @@ GitHub pull requests. To speed the code review process, we ask that:
## Tutorials directory ## Tutorials directory
To help you get started with the functionalities provided by this library, we To help you get started with the functionalities provided by this library, we
provide a detailed walkthrough [here](tutorials/walkthrough/README.md) that provide a detailed walkthrough [here](tutorials/walkthrough/README.md) that will
will teach you how to wrap existing optimizers teach you how to wrap existing optimizers (e.g., SGD, Adam, ...) into their
(e.g., SGD, Adam, ...) into their differentially private counterparts using differentially private counterparts using TensorFlow (TF) Privacy. You will also
TensorFlow (TF) Privacy. You will also learn how to tune the parameters learn how to tune the parameters introduced by differentially private
introduced by differentially private optimization and how to optimization and how to measure the privacy guarantees provided using analysis
measure the privacy guarantees provided using analysis tools included in TF tools included in TF Privacy.
Privacy.
In addition, the In addition, the `tutorials/` folder comes with scripts demonstrating how to use
`tutorials/` folder comes with scripts demonstrating how to use the library the library features. The list of tutorials is described in the README included
features. The list of tutorials is described in the README included in the in the tutorials directory.
tutorials directory.
NOTE: the tutorials are maintained carefully. However, they are not considered NOTE: the tutorials are maintained carefully. However, they are not considered
part of the API and they can change at any time without warning. You should not part of the API and they can change at any time without warning. You should not
@ -110,8 +109,8 @@ directory, but rather intended as a convenient archive.
## TensorFlow 2.x ## TensorFlow 2.x
TensorFlow Privacy now works with TensorFlow 2! You can use the new TensorFlow Privacy now works with TensorFlow 2! You can use the new Keras-based
Keras-based estimators found in estimators found in
`privacy/tensorflow_privacy/privacy/optimizers/dp_optimizer_keras.py`. `privacy/tensorflow_privacy/privacy/optimizers/dp_optimizer_keras.py`.
For this to work with `tf.keras.Model` and `tf.estimator.Estimator`, however, For this to work with `tf.keras.Model` and `tf.estimator.Estimator`, however,
@ -120,7 +119,8 @@ you need to install TensorFlow 2.4 or later.
## Remarks ## Remarks
The content of this repository supersedes the following existing folder in the The content of this repository supersedes the following existing folder in the
tensorflow/models [repository](https://github.com/tensorflow/models/tree/master/research/differential_privacy) tensorflow/models
[repository](https://github.com/tensorflow/models/tree/master/research/differential_privacy)
## Contacts ## Contacts