626_privacy/tensorflow_privacy
2
0
Fork 1
Code Issues Pull requests 1 Projects Releases Packages Wiki Activity Actions
876 commits 2 branches 0 tags 3.1 MiB
Commit graph

876 commits

This branch
This branch
All branches
Author SHA1 Message Date
A. Unique TensorFlower
fc6f1dc5d1 Sparsity Preserving DP-SGD in TF Privacy [4 of 5] 
Add contribution count function for embedding layer.

See https://research.google/blog/sparsity-preserving-differentially-private-training/ for more details on the algorithm.

PiperOrigin-RevId: 656091009
 2024-07-25 14:24:15 -07:00
Peter Hawkins
80802c248d [numpy] Fix users of NumPy APIs that are removed in NumPy 2.0. 
This change migrates users of APIs removed in NumPy 2.0 to their recommended replacements (https://numpy.org/devdocs/numpy_2_0_migration_guide.html).

PiperOrigin-RevId: 655999944
 2024-07-25 10:24:18 -07:00
A. Unique TensorFlower
a56f33c4c5 Sparsity Preserving DP-SGD in TF Privacy [3 of 5] 
Adds sparse noise utilities to privately select sparse indices from and add sparse noise to gradients.

See https://research.google/blog/sparsity-preserving-differentially-private-training/ for more details on the algorithm.

PiperOrigin-RevId: 654902527
 2024-07-22 14:46:56 -07:00
A. Unique TensorFlower
8747858b5b Sparsity Preserving DP-SGD in TF Privacy [2 of 5] 
Adds sparse noise utilities to privately select sparse indices from contribution counts.

See https://research.google/blog/sparsity-preserving-differentially-private-training/ for more details on the algorithm.

PiperOrigin-RevId: 654782588
 2024-07-22 09:25:44 -07:00
A. Unique TensorFlower
348895a7a3 Sparsity Preserving DP-SGD in TF Privacy [1 of 4] 
Adds layer registry and type aliases.

See https://research.google/blog/sparsity-preserving-differentially-private-training/ for more details on the algorithm.

PiperOrigin-RevId: 648747866
 2024-07-02 09:43:11 -07:00
William Kong
00384db109 Expose noise_multiplier and l2_norm_clip as public read-only attributes. 
PiperOrigin-RevId: 642298339
 2024-06-11 09:39:27 -07:00
Arun Ganesh
99c25e3be5 Improves user/group-level accounting in compute_dp_sgd_privacy_lib 
PiperOrigin-RevId: 633346332
 2024-05-13 15:13:34 -07:00
Arun Ganesh
3e42ce318f Increments dp-accounting version in requirements 
PiperOrigin-RevId: 633341323
 2024-05-13 14:57:48 -07:00
Galen Andrew
8b3d8e132f Improve numerical stability of RDP computation. 
PiperOrigin-RevId: 628482307
 2024-04-26 12:41:00 -07:00
William Kong
3deaae30a1 Add compatability for Einsum layers with dynamic shapes. 
PiperOrigin-RevId: 628111219
 2024-04-25 10:09:07 -07:00
William Kong
3fa0a2d362 Add support for multi-headed models that use fast gradient clipping. 
PiperOrigin-RevId: 627942683
 2024-04-24 20:46:43 -07:00
Michael Reneer
fefad2190e Remove duplicate requirement. 
PiperOrigin-RevId: 627166650
 2024-04-22 15:02:27 -07:00
William Kong
44dfac3770 Implement fast gradient clipping for loss functions that use inputs that are fed into shared weights. 
PiperOrigin-RevId: 625395017
 2024-04-16 11:19:18 -07:00
Steve Chien
0582cfdd1a Set TensorFlow version to at most 2.15.0 to avoid problems with using Keras 3. 
PiperOrigin-RevId: 625104162
 2024-04-15 15:34:17 -07:00
Steve Chien
e28c673b74 Update README.md to reflect the split in PyPI packages. 
PiperOrigin-RevId: 607516568
 2024-02-15 18:06:53 -08:00
Steve Chien
1b1c0b65d9 In setup_empirical.py, increase version of required tensorflow-privacy package to 0.9.0. 
PiperOrigin-RevId: 607060540
 2024-02-14 11:57:23 -08:00
Steve Chien
e3b4662d3e Fix small error in setup_empirical.py. 
PiperOrigin-RevId: 606290383
 2024-02-12 10:20:46 -08:00
Steve Chien
d1290796b3 Increase version to 0.9.0 after splitting off TF empirical privacy package. 
PiperOrigin-RevId: 605392906
 2024-02-08 12:22:59 -08:00
Steve Chien
308cbda4db Remove scripts from deprecated directory. 
PiperOrigin-RevId: 605370687
 2024-02-08 11:07:11 -08:00
Steve Chien
95b87270d9 Split TensorFlow Privacy pypi package into two parts-- one for DP training, and one for privacy tests. 
PiperOrigin-RevId: 605137291
 2024-02-07 16:48:44 -08:00
Steve Chien
bbb1b487c1 Fix statsmodels version to 0.14.0, since 0.14.1 causes a breakage. 
PiperOrigin-RevId: 591040422
 2023-12-14 13:45:38 -08:00
Fiona Lang
5e8a2688ee Fix packaging import. 
PiperOrigin-RevId: 591002147
 2023-12-14 11:27:17 -08:00
Zheng Xu
a4deb12ee0 Add a metric for TA-DP-FTRL, 
PiperOrigin-RevId: 590791663
 2023-12-13 20:03:31 -08:00
Steve Chien
81a4fd82f7 Temporarily disable two unit tests while fixing a problem with multiprocessor setup. 
PiperOrigin-RevId: 590782049
 2023-12-13 19:05:10 -08:00
William Kong
fbe5879023 Add utility functions for unwrapping BERT encoder layers into individual Keras layers. 
PiperOrigin-RevId: 588419989
 2023-12-06 07:29:57 -08:00
Pritish Kamath
93376c9d6a Add support for PLD Accountant in computing DP-SGD privacy statement [TF Privacy] 
PiperOrigin-RevId: 587854134
 2023-12-04 15:08:58 -08:00
William Kong
f51b637dda Add support for fast clipping of dense layer gradients where the dimension of the input is larger than 1. 
This change specifically wraps the fast clipping logic used in EinsumDense layers, which is a generalization of the Gramian-based that was used for dense layer clipping.

PiperOrigin-RevId: 585809850
 2023-11-27 17:58:39 -08:00
William Kong
b19088f048 Implement and test a registry function for tf.keras.layers.MultiHeadAttention. 
PiperOrigin-RevId: 584620638
 2023-11-22 07:17:47 -08:00
William Kong
03db50ba94 Add a parameter to the noise function that explicitly specifies the loss reduction type. 
PiperOrigin-RevId: 583507445
 2023-11-17 15:54:59 -08:00
William Kong
39c8a8c1af Implement and test a registry function for tfm.nlp.layers.EinsumDense + small formatting fixes. 
PiperOrigin-RevId: 576215816
 2023-10-24 11:54:45 -07:00
Michael Reneer
8b52ba246c Update the Python package scripts to use Python 3.11 explicitly. 
PiperOrigin-RevId: 572426991
 2023-10-10 18:43:52 -07:00
Michael Reneer
e1eef3c251 Add a description to the TFP Python package. 
PiperOrigin-RevId: 572384355
 2023-10-10 15:30:58 -07:00
Michael Reneer
1ecf9e18cc Update the documentation in the TFP requirements.txt. 
PiperOrigin-RevId: 572353174
 2023-10-10 13:40:55 -07:00
Michael Reneer
fd7cc7607f Update the Python package scripts to use Python 3.10 explicitly. 
Failure:

1. From the build scripts:

```
+ ./tools/build_pip_package.sh
Python 3.8.10
```

2. But `tensorflow-probability` does not support Python 3.8, https://pypi.org/project/tensorflow-probability/

The fix is to update TFP build scripts to use Python 3.10 explicitly because it appears there are unsupported version of Python on the build machines.

PiperOrigin-RevId: 572339377
 2023-10-10 12:50:56 -07:00
Michael Reneer
96b099f5c7 Bump version for github/pypi release with new tensorflow-probability dependency. 
PiperOrigin-RevId: 572323479
 2023-10-10 11:53:17 -07:00
Michael Reneer
5821848fdc Update TFP setup.py to document the required versions of Python. 
PiperOrigin-RevId: 572308763
 2023-10-10 11:05:59 -07:00
Michael Reneer
a9ea9ba4a0 Remove unneeded tensorflow-privacy dependencies. 
Please note:
1. I have manually tested the TFP release for python 3.10.
2. TFP does not define the set of python version that it is compatible with.

This change should be followed up with the following changes:
* Define the set of python version that TFP is compatible with.
* Setup a Python package building test.
* Release TFP so that projects depending on TFP can support Python 3.11

PiperOrigin-RevId: 572297305
 2023-10-10 10:34:54 -07:00
Michael Reneer
0eea97f220 Update to the latest version of tensorflow-probablity. 
PiperOrigin-RevId: 572283525
 2023-10-10 09:59:01 -07:00
Michael Reneer
43bfbc0af1 Cleanup and add build, test, and publish scripts for the TFP Python package. 
PiperOrigin-RevId: 571419982
 2023-10-06 13:26:05 -07:00
Steve Chien
0d1bd9db3e Add __init.py__ to registry_functions.py 
PiperOrigin-RevId: 570724438
 2023-10-04 09:53:39 -07:00
A. Unique TensorFlower
62a2d43d1c Add the second set of EinsumDense utility functions for implementing fast gradient norm computation. 
PiperOrigin-RevId: 568063831
 2023-09-24 16:16:40 -07:00
Galen Andrew
1be6e026e7 Bump version for github/pypi release with new attrs dependency. 
PiperOrigin-RevId: 567696437
 2023-09-22 13:00:14 -07:00
A. Unique TensorFlower
c037070e50 Improve the readability of the fast gradient clipping library. 
PiperOrigin-RevId: 566961891
 2023-09-20 07:47:43 -07:00
Michael Reneer
97aaf302eb Update the version of some dependencies in tensorflow_privacy. 
* `attrs` to `>=21.4`.
* `dp-accounting` to `>=0.4.3`.

PiperOrigin-RevId: 566758983
 2023-09-19 15:22:40 -07:00
A. Unique TensorFlower
e20c92243a Implement and test a registry function for tfm.nlp.layers.PositionEmbedding. 
PiperOrigin-RevId: 565450719
 2023-09-14 12:55:09 -07:00
A. Unique TensorFlower
c7db4fa8cb Add support for microbatching in the tf.keras.layers.LayerNormalization fast square norm function. 
PiperOrigin-RevId: 565050132
 2023-09-13 07:56:42 -07:00
A. Unique TensorFlower
bcc0d4927e Implement and test a registry function for tfm.nlp.layers.OnDeviceEmbedding. 
This CL also moves the common embedding `sqr_norm_fn` logic between
`tf.keras.layers.Embedding` and `tfm.nlp.layers.OnDeviceEmbedding` into a
new registry function utility file.

PiperOrigin-RevId: 564481407
 2023-09-11 13:18:08 -07:00
A. Unique TensorFlower
113b27be43 Add the first set of EinsumDense utility functions for implementing fast gradient norm computation. 
PiperOrigin-RevId: 564460945
 2023-09-11 12:06:12 -07:00
A. Unique TensorFlower
a23cccde8b Ensures types remain consistent. 
PiperOrigin-RevId: 563244784
 2023-09-06 16:13:14 -07:00
A. Unique TensorFlower
c92610e37a Implement and test a registry function for tf.keras.layers.LayerNormalization. 
PiperOrigin-RevId: 561423397
 2023-08-30 12:54:08 -07:00