626_privacy/tensorflow_privacy
2
0
Fork 1
Code Issues Pull requests 1 Projects Releases Packages Wiki Activity Actions
445 commits 2 branches 0 tags 3.1 MiB
Commit graph

199 commits

Author SHA1 Message Date
amad-person
eb215072bc Compute and populate PrivacyReportMetadata fields 2020-11-25 16:06:37 +08:00
Vadym Doroshenko
15515cb0f4 Fix for threshold attacks when logits are not provided. 
Don't try to compute number of classes when it's not needed.

PiperOrigin-RevId: 344060285
 2020-11-24 08:06:11 -08:00
amad-person
46bee91cda Refactor seq2seq logic and tests into separate files 2020-11-24 14:52:12 +08:00
Matthew Jagielski
3bf78f46fe add helper for computing noise_multiplier from epsilon 2020-11-19 21:22:02 -05:00
A. Unique TensorFlower
35a8096173 Merge pull request #137 from amad-person:add_seq2seq_mia_attacks 
PiperOrigin-RevId: 343047622
 2020-11-18 03:26:24 -08:00
amad-person
b25808cfbe Remove call to _get_slicing_spec in run_seq2seq_attack 2020-11-14 02:13:11 +08:00
Aadyaa Maddi
641c4dd98c
Add comment to explain support for LR 2020-11-13 12:20:30 +08:00
David Marn
caf71c11bc Bugfix for logits_or_probs with an accompanying test. 
PiperOrigin-RevId: 341604420
 2020-11-10 06:08:21 -08:00
amad-person
bfc5ef333a Fix type annotation 2020-11-08 22:02:26 +08:00
amad-person
afe3944b1d Fix nits 2020-11-08 21:59:05 +08:00
amad-person
ed2bdcadfa Add codelab for membership inference on seq2seq models 2020-11-06 16:53:29 +08:00
amad-person
16c36e4819 Add tests for membership inference attacks on seq2seq models 2020-11-06 16:49:54 +08:00
amad-person
2355e13f44 Add tests for rank generation 2020-11-06 16:48:44 +08:00
amad-person
4db54d9485 Add tests for Seq2SeqAttackInputData 2020-11-06 16:46:57 +08:00
amad-person
d1c1746cdb Add membership inference attack for seq2seq models 2020-11-06 16:44:52 +08:00
amad-person
cd57910e5c Add rank generation code 2020-11-06 16:43:46 +08:00
amad-person
9f07f2a871 Add Seq2SeqAttackInputData data structure 2020-11-06 16:42:31 +08:00
A. Unique TensorFlower
67f7f35383 Merge pull request #131 from lwsong:master 
PiperOrigin-RevId: 339012372
 2020-10-26 04:24:55 -07:00
Liwei Song
6e929da966 add test case for entropy attack 2020-10-23 09:30:09 -04:00
Liwei Song
893b615d72 use logical AND 2020-10-23 08:58:49 -04:00
Liwei Song
0fa87d200c update slicing test 2020-10-21 17:07:53 -04:00
Liwei Song
a41d6aace7 add threshold-entropy attack 2020-10-21 16:41:20 -04:00
David Marn
1981ebe2f2 Adds the TF Privacy Report codelab. 
PiperOrigin-RevId: 338222024
 2020-10-21 01:52:53 -07:00
Vadym Doroshenko
4143957701 Fixed train/test_size calculation. 
PiperOrigin-RevId: 337886488
 2020-10-19 10:38:11 -07:00
Yurii Sushko
19ae5c9669 Fix broken codelab link 
PiperOrigin-RevId: 337095513
 2020-10-14 08:28:17 -07:00
David Marn
d1a8a6cfda Multiple small changes to the TF Privacy Report: 
- Fix the legend to the bottom right
     - Manually set the size of the plot figure.
     - Fix a typo in the subplot title.

PiperOrigin-RevId: 337064528
 2020-10-14 04:41:35 -07:00
David Marn
1281d0c63e Adds an option to balance train and test AttackInputData and stratifies the train-test split. 
PiperOrigin-RevId: 336609893
 2020-10-12 00:43:16 -07:00
Steve Chien
d703168de2 Add TF1-compatible version of DP canned estimators, and some small cleanup.. 
PiperOrigin-RevId: 335954269
 2020-10-07 14:31:53 -07:00
David Marn
703cd413c6 Introduces an AttackResultsCollection class for the ML Privacy report. 
PiperOrigin-RevId: 335858822
 2020-10-07 06:59:26 -07:00
Peter Kairouz
e19c53a78c Add a function to compute RDP under heterogeneous applications of the subsampled Gaussian mechanism. 
PiperOrigin-RevId: 335706732
 2020-10-06 13:20:34 -07:00
David Marn
ab1090717c Internal change. 
PiperOrigin-RevId: 335385162
 2020-10-05 03:54:20 -07:00
Vadym Doroshenko
9a56402c0d Removing leftovers from the old API. 
PiperOrigin-RevId: 334792006
 2020-10-01 05:13:33 -07:00
A. Unique TensorFlower
a579cc4afc BUILD cleanups 
PiperOrigin-RevId: 334666492
 2020-09-30 13:27:37 -07:00
Shuang Song
20d0b884ba Move to new API. 
PiperOrigin-RevId: 334434385
 2020-09-29 12:16:06 -07:00
Shuang Song
bca2baae8d Remove old API. 
PiperOrigin-RevId: 334406920
 2020-09-29 10:19:37 -07:00
David Marn
78d30a0424 Refactors the pd_dataframe calculation to avoid hard-coded strings. 
PiperOrigin-RevId: 334334080
 2020-09-29 02:15:34 -07:00
David Marn
c30c3fcb7a Adds plots for multiple model labels to the ML Privacy Report. 
PiperOrigin-RevId: 334179759
 2020-09-28 09:59:37 -07:00
Steve Chien
837e014107 For DP Keras optimizers, add assertion that one of the DP-modified gradients methods has been called before apply_gradients(). In particular, this helps catch cases where the user has not yet upgraded to TF 2.4. 
PiperOrigin-RevId: 333620379
 2020-09-24 16:14:49 -07:00
Shuang Song
7c53757250 Option for plotting attack results in the same figure. 
PiperOrigin-RevId: 333225502
 2020-09-22 22:12:00 -07:00
Yurii Sushko
677b3d9e9a Quick docs fix. 
PiperOrigin-RevId: 332061086
 2020-09-16 12:21:44 -07:00
Yurii Sushko
78f76fac31 Add a note about API update to the docs. 
PiperOrigin-RevId: 332057406
 2020-09-16 12:04:27 -07:00
David Marn
942ad89da7 Adds the feedback from to the readme. 
PiperOrigin-RevId: 331956759
 2020-09-16 01:42:25 -07:00
David Marn
70f9585a24 Adds Privacy vs Utility charts to the Privacy Report for a single model. 
PiperOrigin-RevId: 331720083
 2020-09-15 01:30:21 -07:00
David Marn
fc38e3f733 Modifies Privacy Report metadata and adds an epoch chart. 
PiperOrigin-RevId: 331326000
 2020-09-12 09:11:45 -07:00
Vadym Doroshenko
f44b63eb78 Add probabilities to AttackInputData. 
PiperOrigin-RevId: 330723370
 2020-09-09 08:05:52 -07:00
A. Unique TensorFlower
6312a853d8 Merge pull request #119 from lwsong:master 
PiperOrigin-RevId: 330658958
 2020-09-08 22:44:06 -07:00
Vadym Doroshenko
8f3a61b50d Fixing calculating loss on logits. 
PiperOrigin-RevId: 329966058
 2020-09-03 12:06:29 -07:00
Vadym Doroshenko
f4fc9b2623 Updating comments to run_attack() and making non-API functions to be private. 
PiperOrigin-RevId: 329951618
 2020-09-03 10:56:31 -07:00
A. Unique TensorFlower
2f0a078dd9 Adds Privacy Report metadata to AttackResults. 
PiperOrigin-RevId: 329871255
 2020-09-03 01:05:12 -07:00
Steve Chien
8d89ef0a4b Update version to 0.5.1, and add dependence on tensorflow-estimator. 
PiperOrigin-RevId: 329822203
 2020-09-02 17:31:32 -07:00
Liwei Song
0e1c1eeef3 add entropy tests 2020-09-02 11:37:12 -04:00
Liwei Song
9b2e6a55b6 add entropy feature 2020-08-31 16:17:19 -04:00
Liwei Song
e547a10eec fix softmax issue 2020-08-31 15:24:46 -04:00
A. Unique TensorFlower
f677c9c440 Adding contribution guidelines to TF Privacy 
PiperOrigin-RevId: 328740117
 2020-08-27 08:21:03 -07:00
Shuang Song
f90c78bd54 Update tf_estimator_evaluation and keras_evaluation to new API. 
PiperOrigin-RevId: 328195220
 2020-08-24 13:03:28 -07:00
A. Unique TensorFlower
7a77d5d92c Modify Colab to use the new membership inference API. 
PiperOrigin-RevId: 327805944
 2020-08-21 07:21:38 -07:00
Shuang Song
d23772e163 Returns attack results as a Pandas data frame. 
PiperOrigin-RevId: 327675978
 2020-08-20 12:27:43 -07:00
A. Unique TensorFlower
52c1f8fdfe Overrides default __str__ methods. 
PiperOrigin-RevId: 327423772
 2020-08-19 06:49:14 -07:00
Steve Chien
6dccd9b537 Fix bug in keras optimizers where tape was not being used with tensor (as opposed to callable) losses. 
PiperOrigin-RevId: 327276721
 2020-08-18 12:00:20 -07:00
A. Unique TensorFlower
193ac3b1c8 Adds more validation checks for AttackInputData. 
PiperOrigin-RevId: 327191245
 2020-08-18 02:34:58 -07:00
Steve Chien
a69b013390 Add head for multi-label estimators in TF estimator framework. 
PiperOrigin-RevId: 327048185
 2020-08-17 10:28:45 -07:00
Galen Andrew
d939b22463 Bump version number. 
PiperOrigin-RevId: 326927560
 2020-08-16 15:12:13 -07:00
Steve Chien
d72e3400b7 Add DP-enabled version of DNNClassifier. 
PiperOrigin-RevId: 326482309
 2020-08-13 11:03:46 -07:00
A. Unique TensorFlower
f8515dfd71 Replaces predict with predict_proba. 
PiperOrigin-RevId: 326227257
 2020-08-12 07:24:29 -07:00
A. Unique TensorFlower
59192e6f5c Make validation that labels are integers. 
PiperOrigin-RevId: 326216555
 2020-08-12 06:00:31 -07:00
A. Unique TensorFlower
0fd06493cc Make sklearn classifiers in parallel. 
It's done only for those classifiers that run a significant amount of time.

PiperOrigin-RevId: 326215987
 2020-08-12 05:53:46 -07:00
Galen Andrew
37ff5d502e Add derive_metrics function to DPQuery. 
derive_metrics is a new function in the public API so customers can query aspects of the global state that change, such as the clip when using adaptive clipping.

PiperOrigin-RevId: 326174158
 2020-08-11 22:58:59 -07:00
A. Unique TensorFlower
06bb047525 Consistent string formatting. 
PiperOrigin-RevId: 326007570
 2020-08-11 06:17:36 -07:00
A. Unique TensorFlower
680aaa4499 Simple ROC curve plotting for membership inference attack results. 
PiperOrigin-RevId: 325982344
 2020-08-11 02:29:15 -07:00
Steve Chien
99afaed68e Refactor of common functions in binary and multiclass heads. 
PiperOrigin-RevId: 325957037
 2020-08-10 22:30:24 -07:00
Steve Chien
3a641e077e Add DP-enabled binary-class head and multi-class heads for Estimator. 
PiperOrigin-RevId: 325921076
 2020-08-10 17:19:54 -07:00
A. Unique TensorFlower
43a0e4be8a The new API for the membership inference attack. 
1. Colab and Keras/TF estimator integration still use the old API and will be updated in the subsequent CLs.
2. After dropping the old API in membership_inference_attack.py, membership_inference_attack_new.py will be renamed in membership_inference_attack.py.

PiperOrigin-RevId: 325823046
 2020-08-10 09:44:21 -07:00
A. Unique TensorFlower
68651eeddc Internal change. 
PiperOrigin-RevId: 325768017
 2020-08-10 02:51:11 -07:00
A. Unique TensorFlower
40419b56a3 Internal change. 
PiperOrigin-RevId: 325423652
 2020-08-07 06:12:18 -07:00
Steve Chien
5ad8676d38 Automated rollback of commit 5433436b86 
PiperOrigin-RevId: 325308999
 2020-08-06 14:19:46 -07:00
Steve Chien
5433436b86 Change GaussianSumQuery to not convert hyperparameters from Python numbers to Tensors. 
PiperOrigin-RevId: 325251302
 2020-08-06 09:56:40 -07:00
A. Unique TensorFlower
efca03b593 Internal change. 
PiperOrigin-RevId: 325249654
 2020-08-06 09:48:25 -07:00
A. Unique TensorFlower
08f960a1af Internal change. 
PiperOrigin-RevId: 325210305
 2020-08-06 05:25:58 -07:00
Steve Chien
e91c820b2a Adds a Keras optimizer version of DP-SGD. New optimizers are subclasses of tf.keras.optimizers.Optimizer and override both _compute_gradients and get_gradients. 
PiperOrigin-RevId: 325124698
 2020-08-05 21:20:59 -07:00
A. Unique TensorFlower
29651216cd Internal change. 
PiperOrigin-RevId: 324591262
 2020-08-03 12:08:41 -07:00
A. Unique TensorFlower
0a1cbb5b7b Internal change. 
PiperOrigin-RevId: 324574332
 2020-08-03 12:08:33 -07:00
Thomas O'Malley
d5e34b77c8 Allow loss to be passed as a Tensor to Optimizer.minimize 
PiperOrigin-RevId: 323895500
 2020-07-29 22:24:02 -07:00
Shuang Song
cea9e01670 A callback and a function to be called in the end of training for keras to perform membership inference attack. 
PiperOrigin-RevId: 323805663
 2020-07-29 22:23:53 -07:00
A. Unique TensorFlower
dcbfaa3f5e Internal change. 
PiperOrigin-RevId: 323529635
 2020-07-29 22:23:38 -07:00
A. Unique TensorFlower
f318fbb140 Internal change. 
PiperOrigin-RevId: 322996754
 2020-07-24 08:03:51 -07:00
Shuang Song
267ea7f90d A separate tensorboard function. 
PiperOrigin-RevId: 322820408
 2020-07-23 10:55:49 -07:00
A. Unique TensorFlower
2ec0f36d1e Added support for Keras optimizers and serialization. 
PiperOrigin-RevId: 322603030
 2020-07-22 10:29:03 -07:00
Steve Chien
87c01eb2f5 Enable optimizers to handle variables whose gradients are None. 
PiperOrigin-RevId: 322193798
 2020-07-20 11:59:20 -07:00
A. Unique TensorFlower
8ec709e3d7 Internal change. 
PiperOrigin-RevId: 321768596
 2020-07-17 06:37:46 -07:00
A. Unique TensorFlower
3549d23da3 Internal change. 
PiperOrigin-RevId: 321765113
 2020-07-17 06:07:53 -07:00
A. Unique TensorFlower
510dd207d5 Internal change. 
PiperOrigin-RevId: 321742857
 2020-07-17 02:43:53 -07:00
Shuang Song
a0e1b72838 A training hook and a function to be called in the end of training for tf estimator to perform membership inference attack. 
PiperOrigin-RevId: 321648371
 2020-07-16 14:39:09 -07:00
Galen Andrew
51eb7c3712 Add SumAggregationDPQuery to top-level symbols. 
PiperOrigin-RevId: 321185486
 2020-07-14 10:31:25 -07:00
Galen Andrew
78f29a8990 Add a version number to TF Privacy package. 
PiperOrigin-RevId: 321023649
 2020-07-13 13:37:28 -07:00
A. Unique TensorFlower
cad41f8f52 Automated rollback of commit 9408e0fca5 
PiperOrigin-RevId: 320726123
 2020-07-10 21:42:08 -07:00
Galen Andrew
9408e0fca5 Add a version number to TF Privacy package. 
PiperOrigin-RevId: 320713287
 2020-07-10 18:40:11 -07:00
Galen Andrew
4e5b2ae3e5 Add NoPrivacyQuantileEstimatorQuery to top level symbols. 
PiperOrigin-RevId: 320675305
 2020-07-10 14:21:59 -07:00
Galen Andrew
2f51adac89 Allow exact denominator for below estimate fraction used by quantile estimator. 
Also:
1) Check that records for quantile estimator query are scalars.
2) Add tests of quantile estimator with noise.
3) Add privacy ledger to no-privacy queries.
PiperOrigin-RevId: 320633937
 2020-07-10 10:56:06 -07:00
Galen Andrew
d1e2cc1930 Add NestedSumQuery for nested queries with sum aggregation. 
PiperOrigin-RevId: 320303703
 2020-07-08 18:05:04 -07:00
Timon Van Overveldt
c948e2fe7c Automated rollback of commit 3c2fbb2f63 
PiperOrigin-RevId: 318110878
 2020-06-24 12:00:46 -07:00