ruby/tensorflow_privacy
1
0
Fork 0
forked from 626_privacy/tensorflow_privacy
Code Pull requests Activity

Specify TF 1.x in Colab.

Browse source 
PiperOrigin-RevId: 277385438
This commit is contained in:
Steve Chien 2019-10-29 16:07:06 -07:00 committed by A. Unique TensorFlower
parent cad86b925e
commit 853ec20b52
1 changed files with 115 additions and 110 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

225
tutorials/Classification_Privacy.ipynb
View file

@ -1,35 +1,20 @@
{ {
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"colab": {
"name": "Classification_Privacy.ipynb",
"provenance": [],
"collapsed_sections": [],
"toc_visible": true,
"include_colab_link": true
},
"kernelspec": {
"name": "python3",
"display_name": "Python 3"
}
},
"cells": [ "cells": [
{ {
"cell_type": "markdown", "cell_type": "markdown",
"metadata": { "metadata": {
"id": "view-in-github", "colab_type": "text",
"colab_type": "text" "id": "view-in-github"
}, },
"source": [ "source": [
"<a href=\"https://colab.research.google.com/github/tensorflow/privacy/blob/master/tutorials/Classification_Privacy.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>" "\u003ca href=\"https://colab.research.google.com/github/tensorflow/privacy/blob/master/tutorials/Classification_Privacy.ipynb\" target=\"_parent\"\u003e\u003cimg src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/\u003e\u003c/a\u003e"
] ]
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"metadata": { "metadata": {
"id": "XAVN6c8prKOL", "colab_type": "text",
"colab_type": "text" "id": "XAVN6c8prKOL"
}, },
"source": [ "source": [
"##### Copyright 2019 The TensorFlow Authors.\n", "##### Copyright 2019 The TensorFlow Authors.\n",
@ -39,12 +24,14 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 0,
"metadata": { "metadata": {
"id": "SassPC7WQAUO",
"colab_type": "code",
"cellView": "both", "cellView": "both",
"colab": {} "colab": {},
"colab_type": "code",
"id": "SassPC7WQAUO"
}, },
"outputs": [],
"source": [ "source": [
"#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n", "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
"# you may not use this file except in compliance with the License.\n", "# you may not use this file except in compliance with the License.\n",
@ -57,15 +44,13 @@
"# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n", "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
"# See the License for the specific language governing permissions and\n", "# See the License for the specific language governing permissions and\n",
"# limitations under the License." "# limitations under the License."
], ]
"execution_count": 0,
"outputs": []
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"metadata": { "metadata": {
"id": "KwDK47gfLsYf", "colab_type": "text",
"colab_type": "text" "id": "KwDK47gfLsYf"
}, },
"source": [ "source": [
"# Implement Differential Privacy with TensorFlow Privacy" "# Implement Differential Privacy with TensorFlow Privacy"
@ -78,21 +63,21 @@
"id": "MfBg1C5NB3X0" "id": "MfBg1C5NB3X0"
}, },
"source": [ "source": [
"<table class=\"tfo-notebook-buttons\" align=\"left\">\n", "\u003ctable class=\"tfo-notebook-buttons\" align=\"left\"\u003e\n",
" <td>\n", " \u003ctd\u003e\n",
" <a target=\"_blank\" href=\"https://colab.research.google.com/github/tensorflow/privacy/blob/master/tutorials/Classification_Privacy.ipynb\"><img src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" />Run in Google Colab</a>\n", " \u003ca target=\"_blank\" href=\"https://colab.research.google.com/github/tensorflow/privacy/blob/master/tutorials/Classification_Privacy.ipynb\"\u003e\u003cimg src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" /\u003eRun in Google Colab\u003c/a\u003e\n",
" </td>\n", " \u003c/td\u003e\n",
" <td>\n", " \u003ctd\u003e\n",
" <a target=\"_blank\" href=\"https://github.com/tensorflow/privacy/blob/master/tutorials/Classification_Privacy.ipynb\"><img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" />View source on GitHub</a>\n", " \u003ca target=\"_blank\" href=\"https://github.com/tensorflow/privacy/blob/master/tutorials/Classification_Privacy.ipynb\"\u003e\u003cimg src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" /\u003eView source on GitHub\u003c/a\u003e\n",
" </td>\n", " \u003c/td\u003e\n",
"</table>" "\u003c/table\u003e"
] ]
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"metadata": { "metadata": {
"id": "00fQV7e0Unz3", "colab_type": "text",
"colab_type": "text" "id": "00fQV7e0Unz3"
}, },
"source": [ "source": [
"## Overview" "## Overview"
@ -101,8 +86,8 @@
{ {
"cell_type": "markdown", "cell_type": "markdown",
"metadata": { "metadata": {
"id": "TUphKzYu01O9", "colab_type": "text",
"colab_type": "text" "id": "TUphKzYu01O9"
}, },
"source": [ "source": [
"[Differential privacy](https://en.wikipedia.org/wiki/Differential_privacy) (DP) is a framework that allows for measuring the privacy guarantees provided by a Machine Learning (ML) algorithm with respect to its input data. Recent advances allow the training of ML models with DP, greatly mitigating the risk of exposing sensitive training data in ML. Intuitively, a model trained with DP should not be affected by any single training example (or small set of training examples) in its dataset. \n", "[Differential privacy](https://en.wikipedia.org/wiki/Differential_privacy) (DP) is a framework that allows for measuring the privacy guarantees provided by a Machine Learning (ML) algorithm with respect to its input data. Recent advances allow the training of ML models with DP, greatly mitigating the risk of exposing sensitive training data in ML. Intuitively, a model trained with DP should not be affected by any single training example (or small set of training examples) in its dataset. \n",
@ -119,8 +104,8 @@
{ {
"cell_type": "markdown", "cell_type": "markdown",
"metadata": { "metadata": {
"id": "ijJYKVc05DYX", "colab_type": "text",
"colab_type": "text" "id": "ijJYKVc05DYX"
}, },
"source": [ "source": [
"## Setup" "## Setup"
@ -128,28 +113,34 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 0,
"metadata": { "metadata": {
"id": "ef56gCUqrdVn", "colab": {},
"colab_type": "code", "colab_type": "code",
"colab": {} "id": "ef56gCUqrdVn"
}, },
"outputs": [],
"source": [ "source": [
"from __future__ import absolute_import\n", "from __future__ import absolute_import\n",
"from __future__ import division\n", "from __future__ import division\n",
"from __future__ import print_function\n", "from __future__ import print_function\n",
"\n", "\n",
"try:\n",
" # %tensorflow_version only exists in Colab.\n",
" %tensorflow_version 1.x\n",
"except Exception:\n",
" pass\n",
"\n",
"import tensorflow as tf\n", "import tensorflow as tf\n",
"\n", "\n",
"import numpy as np" "import numpy as np"
], ]
"execution_count": 0,
"outputs": []
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"metadata": { "metadata": {
"id": "r_fVhfUyeI3d", "colab_type": "text",
"colab_type": "text" "id": "r_fVhfUyeI3d"
}, },
"source": [ "source": [
"Install TensorFlow Privacy." "Install TensorFlow Privacy."
@ -157,25 +148,25 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 0,
"metadata": { "metadata": {
"id": "RseeuA7veIHU", "colab": {},
"colab_type": "code", "colab_type": "code",
"colab": {} "id": "RseeuA7veIHU"
}, },
"outputs": [],
"source": [ "source": [
"!pip install tensorflow_privacy\n", "!pip install tensorflow_privacy\n",
"\n", "\n",
"from tensorflow_privacy.privacy.analysis import compute_dp_sgd_privacy\n", "from tensorflow_privacy.privacy.analysis import compute_dp_sgd_privacy\n",
"from tensorflow_privacy.privacy.optimizers.dp_optimizer import DPGradientDescentGaussianOptimizer" "from tensorflow_privacy.privacy.optimizers.dp_optimizer import DPGradientDescentGaussianOptimizer"
], ]
"execution_count": 0,
"outputs": []
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"metadata": { "metadata": {
"id": "mU1p8N7M5Mmn", "colab_type": "text",
"colab_type": "text" "id": "mU1p8N7M5Mmn"
}, },
"source": [ "source": [
"## Load and pre-process the dataset\n", "## Load and pre-process the dataset\n",
@ -185,11 +176,13 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 0,
"metadata": { "metadata": {
"id": "_1ML23FlueTr", "colab": {},
"colab_type": "code", "colab_type": "code",
"colab": {} "id": "_1ML23FlueTr"
}, },
"outputs": [],
"source": [ "source": [
"train, test = tf.keras.datasets.mnist.load_data()\n", "train, test = tf.keras.datasets.mnist.load_data()\n",
"train_data, train_labels = train\n", "train_data, train_labels = train\n",
@ -211,15 +204,13 @@
"assert train_data.max() == 1.\n", "assert train_data.max() == 1.\n",
"assert test_data.min() == 0.\n", "assert test_data.min() == 0.\n",
"assert test_data.max() == 1." "assert test_data.max() == 1."
], ]
"execution_count": 0,
"outputs": []
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"metadata": { "metadata": {
"id": "xVDcswOCtlr3", "colab_type": "text",
"colab_type": "text" "id": "xVDcswOCtlr3"
}, },
"source": [ "source": [
"## Define and tune learning model hyperparameters\n", "## Define and tune learning model hyperparameters\n",
@ -229,23 +220,23 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 0,
"metadata": { "metadata": {
"id": "E14tL1vUuTRV", "colab": {},
"colab_type": "code", "colab_type": "code",
"colab": {} "id": "E14tL1vUuTRV"
}, },
"outputs": [],
"source": [ "source": [
"epochs = 15\n", "epochs = 15\n",
"batch_size = 250" "batch_size = 250"
], ]
"execution_count": 0,
"outputs": []
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"metadata": { "metadata": {
"id": "qXNp_25y7JP2", "colab_type": "text",
"colab_type": "text" "id": "qXNp_25y7JP2"
}, },
"source": [ "source": [
"DP-SGD has three privacy-specific hyperparameters and one existing hyperamater that you must tune:\n", "DP-SGD has three privacy-specific hyperparameters and one existing hyperamater that you must tune:\n",
@ -260,11 +251,13 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 0,
"metadata": { "metadata": {
"id": "pVw_r2Mq7ntd", "colab": {},
"colab_type": "code", "colab_type": "code",
"colab": {} "id": "pVw_r2Mq7ntd"
}, },
"outputs": [],
"source": [ "source": [
"l2_norm_clip = 1.5\n", "l2_norm_clip = 1.5\n",
"noise_multiplier = 1.3\n", "noise_multiplier = 1.3\n",
@ -273,15 +266,13 @@
"\n", "\n",
"if batch_size % num_microbatches != 0:\n", "if batch_size % num_microbatches != 0:\n",
" raise ValueError('Batch size should be an integer multiple of the number of microbatches')" " raise ValueError('Batch size should be an integer multiple of the number of microbatches')"
], ]
"execution_count": 0,
"outputs": []
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"metadata": { "metadata": {
"id": "wXAmHcNOmHc5", "colab_type": "text",
"colab_type": "text" "id": "wXAmHcNOmHc5"
}, },
"source": [ "source": [
"## Build the learning model\n", "## Build the learning model\n",
@ -291,11 +282,13 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 0,
"metadata": { "metadata": {
"id": "oCOo8aOLmFta", "colab": {},
"colab_type": "code", "colab_type": "code",
"colab": {} "id": "oCOo8aOLmFta"
}, },
"outputs": [],
"source": [ "source": [
"model = tf.keras.Sequential([\n", "model = tf.keras.Sequential([\n",
" tf.keras.layers.Conv2D(16, 8,\n", " tf.keras.layers.Conv2D(16, 8,\n",
@ -313,15 +306,13 @@
" tf.keras.layers.Dense(32, activation='relu'),\n", " tf.keras.layers.Dense(32, activation='relu'),\n",
" tf.keras.layers.Dense(10, activation='softmax')\n", " tf.keras.layers.Dense(10, activation='softmax')\n",
"])" "])"
], ]
"execution_count": 0,
"outputs": []
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"metadata": { "metadata": {
"id": "FT4lByFg-I_r", "colab_type": "text",
"colab_type": "text" "id": "FT4lByFg-I_r"
}, },
"source": [ "source": [
"Define the optimizer and loss function for the learning model. Compute the loss as a vector of losses per-example rather than as the mean over a minibatch to support gradient manipulation over each training point. " "Define the optimizer and loss function for the learning model. Compute the loss as a vector of losses per-example rather than as the mean over a minibatch to support gradient manipulation over each training point. "
@ -329,11 +320,13 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 0,
"metadata": { "metadata": {
"id": "bqBvjCf5-ZXy", "colab": {},
"colab_type": "code", "colab_type": "code",
"colab": {} "id": "bqBvjCf5-ZXy"
}, },
"outputs": [],
"source": [ "source": [
"optimizer = DPGradientDescentGaussianOptimizer(\n", "optimizer = DPGradientDescentGaussianOptimizer(\n",
" l2_norm_clip=l2_norm_clip,\n", " l2_norm_clip=l2_norm_clip,\n",
@ -343,15 +336,13 @@
"\n", "\n",
"loss = tf.keras.losses.CategoricalCrossentropy(\n", "loss = tf.keras.losses.CategoricalCrossentropy(\n",
" from_logits=True, reduction=tf.losses.Reduction.NONE)" " from_logits=True, reduction=tf.losses.Reduction.NONE)"
], ]
"execution_count": 0,
"outputs": []
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"metadata": { "metadata": {
"id": "LI_3nXzEGmrP", "colab_type": "text",
"colab_type": "text" "id": "LI_3nXzEGmrP"
}, },
"source": [ "source": [
"## Compile and train the learning model\n" "## Compile and train the learning model\n"
@ -359,11 +350,13 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 0,
"metadata": { "metadata": {
"id": "z4iV03VqG1Bo", "colab": {},
"colab_type": "code", "colab_type": "code",
"colab": {} "id": "z4iV03VqG1Bo"
}, },
"outputs": [],
"source": [ "source": [
"model.compile(optimizer=optimizer, loss=loss, metrics=['accuracy'])\n", "model.compile(optimizer=optimizer, loss=loss, metrics=['accuracy'])\n",
"\n", "\n",
@ -371,15 +364,13 @@
" epochs=epochs,\n", " epochs=epochs,\n",
" validation_data=(test_data, test_labels),\n", " validation_data=(test_data, test_labels),\n",
" batch_size=batch_size)" " batch_size=batch_size)"
], ]
"execution_count": 0,
"outputs": []
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"metadata": { "metadata": {
"id": "TL7_lX5sHCTI", "colab_type": "text",
"colab_type": "text" "id": "TL7_lX5sHCTI"
}, },
"source": [ "source": [
"## Measure the differential privacy guarantee\n", "## Measure the differential privacy guarantee\n",
@ -397,8 +388,8 @@
{ {
"cell_type": "markdown", "cell_type": "markdown",
"metadata": { "metadata": {
"id": "wUEk25pgmnm-", "colab_type": "text",
"colab_type": "text" "id": "wUEk25pgmnm-"
}, },
"source": [ "source": [
"Two metrics are used to express the DP guarantee of an ML algorithm:\n", "Two metrics are used to express the DP guarantee of an ML algorithm:\n",
@ -417,22 +408,22 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 0,
"metadata": { "metadata": {
"id": "ws8-nVuVDgtJ", "colab": {},
"colab_type": "code", "colab_type": "code",
"colab": {} "id": "ws8-nVuVDgtJ"
}, },
"outputs": [],
"source": [ "source": [
"compute_dp_sgd_privacy.compute_dp_sgd_privacy(n=60000, batch_size=250, noise_multiplier=1.3, epochs=15, delta=1e-5)" "compute_dp_sgd_privacy.compute_dp_sgd_privacy(n=60000, batch_size=250, noise_multiplier=1.3, epochs=15, delta=1e-5)"
], ]
"execution_count": 0,
"outputs": []
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"metadata": { "metadata": {
"id": "c-KyttEWFRDc", "colab_type": "text",
"colab_type": "text" "id": "c-KyttEWFRDc"
}, },
"source": [ "source": [
"The tool reports that for the hyperparameters chosen above, the trained model has an $\\epsilon$ value of 1.18." "The tool reports that for the hyperparameters chosen above, the trained model has an $\\epsilon$ value of 1.18."
@ -441,8 +432,8 @@
{ {
"cell_type": "markdown", "cell_type": "markdown",
"metadata": { "metadata": {
"id": "SA_9HMGBWFM3", "colab_type": "text",
"colab_type": "text" "id": "SA_9HMGBWFM3"
}, },
"source": [ "source": [
"## Summary\n", "## Summary\n",
@ -452,5 +443,19 @@
"* Measure the privacy guarantee provided using analysis tools included in TensorFlow Privacy" "* Measure the privacy guarantee provided using analysis tools included in TensorFlow Privacy"
] ]
} }
] ],
"metadata": {
"colab": {
"collapsed_sections": [],
"name": "Classification_Privacy.ipynb",
"provenance": [],
"toc_visible": true
},
"kernelspec": {
"display_name": "Python 3",
"name": "python3"
}
},
"nbformat": 4,
"nbformat_minor": 0
} }