ruby/tensorflow_privacy
1
0
Fork 0
forked from 626_privacy/tensorflow_privacy
Code Pull requests Activity
890 commits 2 branches 0 tags 3.1 MiB
Commit graph

539 commits

Author SHA1 Message Date
Shuang Song
ba903145df Update MIA README to include epsilon lower bound. 
PiperOrigin-RevId: 689814531
 2024-10-25 09:28:40 -07:00
A. Unique TensorFlower
d965556ebb Disable MLIR bridge for the test points that MLIR bridge silently fails 
PiperOrigin-RevId: 676660290
 2024-09-19 19:51:40 -07:00
A. Unique TensorFlower
e3fd3afdf8 Clarify documentation of labels_train/test usage wrt loss_train/test. 
PiperOrigin-RevId: 670641843
 2024-09-03 11:39:27 -07:00
William Kong
66d05a22a3 Fix a gradient clipping bug for layer normalization layers with microbatch axes. 
The previous code passed the unstacked gradients (a list) instead of the stacked gradients (a tensor) to the microbatcher, which led to unexpected behavior. This change passes the right argument and changes the original unit test to catch this bug.

PiperOrigin-RevId: 669413064
 2024-08-30 12:41:11 -07:00
A. Unique TensorFlower
b3963971e3 Sparsity Preserving DP-SGD in TF Privacy [5 of 5] 
Integrate sparsity preserving noise into DP Keras Model with fast gradient clipping.

See https://research.google/blog/sparsity-preserving-differentially-private-training/ for more details on the algorithm.

PiperOrigin-RevId: 666849100
 2024-08-23 10:46:12 -07:00
A. Unique TensorFlower
93c7e54327 Sparsity Preserving DP-SGD in TF Privacy 
Add function to merge varname_to_contribution_count_fn maps from different layers.

See https://research.google/blog/sparsity-preserving-differentially-private-training/ for more details on the algorithm.

PiperOrigin-RevId: 664906202
 2024-08-19 11:44:43 -07:00
A. Unique TensorFlower
38d80cae92 Automated Code Change 
PiperOrigin-RevId: 662904771
 2024-08-14 07:03:41 -07:00
A. Unique TensorFlower
bf6cf4dec9 Sparsity Preserving DP-SGD in TF Privacy 
Add support for calculating contribution counts to registry function for sparsity preserving noise.

See https://research.google/blog/sparsity-preserving-differentially-private-training/ for more details on the algorithm.

PiperOrigin-RevId: 662162597
 2024-08-12 11:22:06 -07:00
A. Unique TensorFlower
e42b574465 Sparsity Preserving DP-SGD in TF Privacy 
Add support for adding sparsity preserving noise in add_aggregate_noise

See https://research.google/blog/sparsity-preserving-differentially-private-training/ for more details on the algorithm.

PiperOrigin-RevId: 662148309
 2024-08-12 10:45:28 -07:00
A. Unique TensorFlower
09c68750d7 Sparsity Preserving DP-SGD in TF Privacy 
Refactor model_forward_backward_pass out of compute_gradients to allow for other optimizations such as sparsity preserving noise to integrate with it.

See https://research.google/blog/sparsity-preserving-differentially-private-training/ for more details on the algorithm.

PiperOrigin-RevId: 660924829
 2024-08-08 11:52:02 -07:00
A. Unique TensorFlower
8294cec132 Sparsity Preserving DP-SGD in TF Privacy 
Move get_registry_generator_fn from clip_grads.py to gradient_clipping_utils.py and change return type to dataclass.

See https://research.google/blog/sparsity-preserving-differentially-private-training/ for more details on the algorithm.

PiperOrigin-RevId: 660548431
 2024-08-07 14:54:27 -07:00
A. Unique TensorFlower
d3f527e775 Sparsity Preserving DP-SGD in TF Privacy 
Refactor utilities for adding noise into separate utility file.

See https://research.google/blog/sparsity-preserving-differentially-private-training/ for more details on the algorithm.

PiperOrigin-RevId: 660527638
 2024-08-07 13:58:31 -07:00
A. Unique TensorFlower
fc6f1dc5d1 Sparsity Preserving DP-SGD in TF Privacy [4 of 5] 
Add contribution count function for embedding layer.

See https://research.google/blog/sparsity-preserving-differentially-private-training/ for more details on the algorithm.

PiperOrigin-RevId: 656091009
 2024-07-25 14:24:15 -07:00
Peter Hawkins
80802c248d [numpy] Fix users of NumPy APIs that are removed in NumPy 2.0. 
This change migrates users of APIs removed in NumPy 2.0 to their recommended replacements (https://numpy.org/devdocs/numpy_2_0_migration_guide.html).

PiperOrigin-RevId: 655999944
 2024-07-25 10:24:18 -07:00
A. Unique TensorFlower
a56f33c4c5 Sparsity Preserving DP-SGD in TF Privacy [3 of 5] 
Adds sparse noise utilities to privately select sparse indices from and add sparse noise to gradients.

See https://research.google/blog/sparsity-preserving-differentially-private-training/ for more details on the algorithm.

PiperOrigin-RevId: 654902527
 2024-07-22 14:46:56 -07:00
A. Unique TensorFlower
8747858b5b Sparsity Preserving DP-SGD in TF Privacy [2 of 5] 
Adds sparse noise utilities to privately select sparse indices from contribution counts.

See https://research.google/blog/sparsity-preserving-differentially-private-training/ for more details on the algorithm.

PiperOrigin-RevId: 654782588
 2024-07-22 09:25:44 -07:00
A. Unique TensorFlower
348895a7a3 Sparsity Preserving DP-SGD in TF Privacy [1 of 4] 
Adds layer registry and type aliases.

See https://research.google/blog/sparsity-preserving-differentially-private-training/ for more details on the algorithm.

PiperOrigin-RevId: 648747866
 2024-07-02 09:43:11 -07:00
William Kong
00384db109 Expose noise_multiplier and l2_norm_clip as public read-only attributes. 
PiperOrigin-RevId: 642298339
 2024-06-11 09:39:27 -07:00
Arun Ganesh
99c25e3be5 Improves user/group-level accounting in compute_dp_sgd_privacy_lib 
PiperOrigin-RevId: 633346332
 2024-05-13 15:13:34 -07:00
Galen Andrew
8b3d8e132f Improve numerical stability of RDP computation. 
PiperOrigin-RevId: 628482307
 2024-04-26 12:41:00 -07:00
William Kong
3deaae30a1 Add compatability for Einsum layers with dynamic shapes. 
PiperOrigin-RevId: 628111219
 2024-04-25 10:09:07 -07:00
William Kong
3fa0a2d362 Add support for multi-headed models that use fast gradient clipping. 
PiperOrigin-RevId: 627942683
 2024-04-24 20:46:43 -07:00
William Kong
44dfac3770 Implement fast gradient clipping for loss functions that use inputs that are fed into shared weights. 
PiperOrigin-RevId: 625395017
 2024-04-16 11:19:18 -07:00
Steve Chien
d1290796b3 Increase version to 0.9.0 after splitting off TF empirical privacy package. 
PiperOrigin-RevId: 605392906
 2024-02-08 12:22:59 -08:00
Steve Chien
95b87270d9 Split TensorFlow Privacy pypi package into two parts-- one for DP training, and one for privacy tests. 
PiperOrigin-RevId: 605137291
 2024-02-07 16:48:44 -08:00
Fiona Lang
5e8a2688ee Fix packaging import. 
PiperOrigin-RevId: 591002147
 2023-12-14 11:27:17 -08:00
Zheng Xu
a4deb12ee0 Add a metric for TA-DP-FTRL, 
PiperOrigin-RevId: 590791663
 2023-12-13 20:03:31 -08:00
Steve Chien
81a4fd82f7 Temporarily disable two unit tests while fixing a problem with multiprocessor setup. 
PiperOrigin-RevId: 590782049
 2023-12-13 19:05:10 -08:00
William Kong
fbe5879023 Add utility functions for unwrapping BERT encoder layers into individual Keras layers. 
PiperOrigin-RevId: 588419989
 2023-12-06 07:29:57 -08:00
Pritish Kamath
93376c9d6a Add support for PLD Accountant in computing DP-SGD privacy statement [TF Privacy] 
PiperOrigin-RevId: 587854134
 2023-12-04 15:08:58 -08:00
William Kong
f51b637dda Add support for fast clipping of dense layer gradients where the dimension of the input is larger than 1. 
This change specifically wraps the fast clipping logic used in EinsumDense layers, which is a generalization of the Gramian-based that was used for dense layer clipping.

PiperOrigin-RevId: 585809850
 2023-11-27 17:58:39 -08:00
William Kong
b19088f048 Implement and test a registry function for tf.keras.layers.MultiHeadAttention. 
PiperOrigin-RevId: 584620638
 2023-11-22 07:17:47 -08:00
William Kong
03db50ba94 Add a parameter to the noise function that explicitly specifies the loss reduction type. 
PiperOrigin-RevId: 583507445
 2023-11-17 15:54:59 -08:00
William Kong
39c8a8c1af Implement and test a registry function for tfm.nlp.layers.EinsumDense + small formatting fixes. 
PiperOrigin-RevId: 576215816
 2023-10-24 11:54:45 -07:00
Michael Reneer
96b099f5c7 Bump version for github/pypi release with new tensorflow-probability dependency. 
PiperOrigin-RevId: 572323479
 2023-10-10 11:53:17 -07:00
Steve Chien
0d1bd9db3e Add __init.py__ to registry_functions.py 
PiperOrigin-RevId: 570724438
 2023-10-04 09:53:39 -07:00
A. Unique TensorFlower
62a2d43d1c Add the second set of EinsumDense utility functions for implementing fast gradient norm computation. 
PiperOrigin-RevId: 568063831
 2023-09-24 16:16:40 -07:00
Galen Andrew
1be6e026e7 Bump version for github/pypi release with new attrs dependency. 
PiperOrigin-RevId: 567696437
 2023-09-22 13:00:14 -07:00
A. Unique TensorFlower
c037070e50 Improve the readability of the fast gradient clipping library. 
PiperOrigin-RevId: 566961891
 2023-09-20 07:47:43 -07:00
A. Unique TensorFlower
e20c92243a Implement and test a registry function for tfm.nlp.layers.PositionEmbedding. 
PiperOrigin-RevId: 565450719
 2023-09-14 12:55:09 -07:00
A. Unique TensorFlower
c7db4fa8cb Add support for microbatching in the tf.keras.layers.LayerNormalization fast square norm function. 
PiperOrigin-RevId: 565050132
 2023-09-13 07:56:42 -07:00
A. Unique TensorFlower
bcc0d4927e Implement and test a registry function for tfm.nlp.layers.OnDeviceEmbedding. 
This CL also moves the common embedding `sqr_norm_fn` logic between
`tf.keras.layers.Embedding` and `tfm.nlp.layers.OnDeviceEmbedding` into a
new registry function utility file.

PiperOrigin-RevId: 564481407
 2023-09-11 13:18:08 -07:00
A. Unique TensorFlower
113b27be43 Add the first set of EinsumDense utility functions for implementing fast gradient norm computation. 
PiperOrigin-RevId: 564460945
 2023-09-11 12:06:12 -07:00
A. Unique TensorFlower
a23cccde8b Ensures types remain consistent. 
PiperOrigin-RevId: 563244784
 2023-09-06 16:13:14 -07:00
A. Unique TensorFlower
c92610e37a Implement and test a registry function for tf.keras.layers.LayerNormalization. 
PiperOrigin-RevId: 561423397
 2023-08-30 12:54:08 -07:00
Steve Chien
372c934d14 Fix bug in v1 estimators that was preventing use of microbatches. 
PiperOrigin-RevId: 560765153
 2023-08-28 11:14:38 -07:00
A. Unique TensorFlower
b4b47b1403 Generalize the testing API to support input Tensors of dimension >1, 
excluding the batch dimension.

This is a forward-looking change for testing more general layers such as
`tf.keras.layers.LayerNormalization` and `tf.keras.layers.EinsumDense`.

PiperOrigin-RevId: 560709678
 2023-08-28 07:50:52 -07:00
Richard Levasseur
6248be8290 Internal Code Change 
PiperOrigin-RevId: 557198768
 2023-08-15 11:42:02 -07:00
Shuang Song
27069d347d Fixes comments and membership scores for thresholds attack. 
PiperOrigin-RevId: 555579896
 2023-08-10 11:31:29 -07:00
Steve Chien
a32e6ae5d0 Add DP versions of v1 FTRL optimizer. 
PiperOrigin-RevId: 553186886
 2023-08-02 10:30:35 -07:00