-- moving to Bolton Optimizer
Model is now just a convenient wrapper and example for users.
Optimizer holds ALL Bolton privacy requirements.
Optimizer is used as a context manager, and must be passed the model's layers.
Unit tests incomplete, committing for visibility into the design.
-- update to include pull request changes
changes include:
parameter renaming,
changing to mixin,
moving model to compile,
additional tests,
fixing huber loss
set_denominator was added so that the batch size doesn't need to be specified before constructing the optimizer, but it breaks the DPQuery abstraction. Now the optimizer uses a GaussianSumQuery instead of GaussianAverageQuery, and normalization by batch size is done inside the optimizer.
Also instead of creating all DPQueries with a PrivacyLedger and then wrapping with QueryWithLedger, it is now sufficient to create the queries with no ledger and QueryWithLedger will construct the ledger and pass it to all inner queries.
PiperOrigin-RevId: 251462353
1. Split DPQuery.accumulate_record function into preprocess_record and accumulate_preprocessed_record.
2. Add merge_sample_state function.
3. Add default implementations for some functions in DPQuery, and add base class SumAggregationDPQuery that implements some more. Only get_noised_result is still abstract.
4. Enforce that all states and parameters used as inputs and outputs to DPQuery functions are nested structures of tensors by replacing numbers with constants and Nones with empty tuples.
PiperOrigin-RevId: 247975791
At present, the script has no heavy dependencies except for the rdp_accountant, which is by itself pretty light-weight. However, importing rdp_accountant triggers __init__.py in third_party/py/tensorflow_privacy/privacy, which loads TF and all of tf.privacy. The CL adds a check to the __init__.py, which controls this behavior.
PiperOrigin-RevId: 243172355
Prior to this change the PrivacyLedger is running to keep a log of private queries, but the ledger is not actually used to compute the (epsilon, delta) guarantees. This CL adds a function to compute the RDP directly from the ledger.
Note I did verify that the tutorial builds and runs with the changes and for the first few iterations prints the same epsilon values as before the change.
PiperOrigin-RevId: 241063532
Moved query classes from dir optimizers into new dir dp_query. Added NormalizedQuery class for queries that divide the output of another query by a constant like GaussianAverageQuery.
PiperOrigin-RevId: 240167115
