tensorflow_privacy/g3doc/tutorials/privacy_report.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1eiwVljWpzM7"
      },
      "source": [
        "Copyright 2020 The TensorFlow Authors.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "cellView": "form",
        "id": "4rmwPgXeptiS"
      },
      "outputs": [],
      "source": [
        "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "# https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YM2gRaJMqvMi"
      },
      "source": [
        "# Assess privacy risks with the TensorFlow Privacy Report"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7oUAMMc6isck"
      },
      "source": [
        "\u003ctable class=\"tfo-notebook-buttons\" align=\"left\"\u003e\n",
        "  \u003ctd\u003e\n",
        "    \u003ca target=\"_blank\" href=\"https://www.tensorflow.org/responsible_ai/privacy/tutorials/privacy_report\"\u003e\u003cimg src=\"https://www.tensorflow.org/images/tf_logo_32px.png\" /\u003eView on TensorFlow.org\u003c/a\u003e\n",
        "  \u003c/td\u003e\n",
        "  \u003ctd\u003e\n",
        "    \u003ca target=\"_blank\" href=\"https://colab.research.google.com/github/tensorflow/privacy/blob/master/tensorflow_privacy/privacy/membership_inference_attack/codelabs/privacy_report_codelab.ipynb\"\u003e\u003cimg src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" /\u003eRun in Google Colab\u003c/a\u003e\n",
        "  \u003c/td\u003e\n",
        "  \u003ctd\u003e\n",
        "    \u003ca target=\"_blank\" href=\"https://github.com/tensorflow/privacy/blob/master/tensorflow_privacy/privacy/membership_inference_attack/codelabs/privacy_report_codelab.ipynb\"\u003e\u003cimg src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" /\u003eView source on GitHub\u003c/a\u003e\n",
        "  \u003c/td\u003e\n",
        "  \u003ctd\u003e\n",
        "    \u003ca href=\"https://storage.googleapis.com/tensorflow_docs/privacy/g3doc/tutorials/privacy_report.ipynb\"\u003e\u003cimg src=\"https://www.tensorflow.org/images/download_logo_32px.png\" /\u003eDownload notebook\u003c/a\u003e\n",
        "  \u003c/td\u003e\n",
        "\u003c/table\u003e"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9rMuytY7Nn8P"
      },
      "source": [
        "##Overview\n",
        "In this codelab you'll train a simple image classification model on the CIFAR10 dataset, and then use the \"membership inference attack\" against this model to assess if the attacker is able to \"guess\" whether a particular sample was present in the training set. You will use the TF Privacy Report to visualize results from multiple models and model checkpoints."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FUWqArj_q8vs"
      },
      "source": [
        "## Setup\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Lr1pwHcbralz"
      },
      "outputs": [],
      "source": [
        "import numpy as np\n",
        "from typing import Tuple\n",
        "from scipy import special\n",
        "from sklearn import metrics\n",
        "\n",
        "import tensorflow as tf\n",
        "tf.compat.v1.disable_v2_behavior()\n",
        "\n",
        "import tensorflow_datasets as tfds\n",
        "\n",
        "# Set verbosity.\n",
        "tf.compat.v1.logging.set_verbosity(tf.compat.v1.logging.ERROR)\n",
        "from sklearn.exceptions import ConvergenceWarning\n",
        "\n",
        "import warnings\n",
        "warnings.simplefilter(action=\"ignore\", category=ConvergenceWarning)\n",
        "warnings.simplefilter(action=\"ignore\", category=FutureWarning)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ucw81ar6ru-6"
      },
      "source": [
        "### Install TensorFlow Privacy."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "1n0K00S6zmfb"
      },
      "outputs": [],
      "source": [
        "!pip install tensorflow_privacy"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "cellView": "both",
        "id": "zcqAmiGH90kl"
      },
      "outputs": [],
      "source": [
        "from tensorflow_privacy.privacy.membership_inference_attack import membership_inference_attack as mia\n",
        "from tensorflow_privacy.privacy.membership_inference_attack.data_structures import AttackInputData\n",
        "from tensorflow_privacy.privacy.membership_inference_attack.data_structures import AttackResultsCollection\n",
        "from tensorflow_privacy.privacy.membership_inference_attack.data_structures import AttackType\n",
        "from tensorflow_privacy.privacy.membership_inference_attack.data_structures import PrivacyMetric\n",
        "from tensorflow_privacy.privacy.membership_inference_attack.data_structures import PrivacyReportMetadata\n",
        "from tensorflow_privacy.privacy.membership_inference_attack.data_structures import SlicingSpec\n",
        "from tensorflow_privacy.privacy.membership_inference_attack import privacy_report"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pBbcG86th_sW"
      },
      "source": [
        "## Train two models, with privacy metrics\n",
        "\n",
        "This section trains a pair of `keras.Model` classifiers on the `CIFAR-10` dataset. During the training process it collects privacy metrics, that will be used to generate reports in the bext section.\n",
        "\n",
        "The first step is to define some hyperparameters:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "al0QK7O-0lk7"
      },
      "outputs": [],
      "source": [
        "dataset = 'cifar10'\n",
        "num_classes = 10\n",
        "activation = 'relu'\n",
        "lr = 0.02\n",
        "momentum = 0.9\n",
        "batch_size = 250\n",
        "epochs_per_report = 5\n",
        "num_reports = 10\n",
        "# Privacy risks are especially visible with lots of epochs.\n",
        "total_epochs = epochs_per_report*num_reports  "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pu5IEzW6B-Oh"
      },
      "source": [
        "Next, load the dataset. There's nothing privacy-specific in this code."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "f1TT3ofN0qrq"
      },
      "outputs": [],
      "source": [
        "#@title Load the data\n",
        "print('Loading the dataset.')\n",
        "train_ds = tfds.as_numpy(\n",
        "    tfds.load(dataset, split=tfds.Split.TRAIN, batch_size=-1))\n",
        "test_ds = tfds.as_numpy(\n",
        "    tfds.load(dataset, split=tfds.Split.TEST, batch_size=-1))\n",
        "x_train = train_ds['image'].astype('float32') / 255.\n",
        "y_train_indices = train_ds['label'][:, np.newaxis]\n",
        "x_test = test_ds['image'].astype('float32') / 255.\n",
        "y_test_indices = test_ds['label'][:, np.newaxis]\n",
        "\n",
        "# Convert class vectors to binary class matrices.\n",
        "y_train = tf.keras.utils.to_categorical(y_train_indices, num_classes)\n",
        "y_test = tf.keras.utils.to_categorical(y_test_indices, num_classes)\n",
        "\n",
        "input_shape = x_train.shape[1:]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9l-55vOLCWZM"
      },
      "source": [
        "Next define a function to build the models."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "vCyOWyyhXLib"
      },
      "outputs": [],
      "source": [
        "#@title Define the models\n",
        "def small_cnn(input_shape: Tuple[int],\n",
        "              num_classes: int,\n",
        "              num_conv: int,\n",
        "              activation: str = 'relu') -\u003e tf.keras.models.Sequential:\n",
        "  \"\"\"Setup a small CNN for image classification.\n",
        "\n",
        "  Args:\n",
        "    input_shape: Integer tuple for the shape of the images.\n",
        "    num_classes: Number of prediction classes.\n",
        "    num_conv: Number of convolutional layers.\n",
        "    activation: The activation function to use for conv and dense layers.\n",
        "\n",
        "  Returns:\n",
        "    The Keras model.\n",
        "  \"\"\"\n",
        "  model = tf.keras.models.Sequential()\n",
        "  model.add(tf.keras.layers.Input(shape=input_shape))\n",
        "\n",
        "  # Conv layers\n",
        "  for _ in range(num_conv):\n",
        "    model.add(tf.keras.layers.Conv2D(32, (3, 3), activation=activation))\n",
        "    model.add(tf.keras.layers.MaxPooling2D())\n",
        "\n",
        "  model.add(tf.keras.layers.Flatten())\n",
        "  model.add(tf.keras.layers.Dense(64, activation=activation))\n",
        "  model.add(tf.keras.layers.Dense(num_classes))\n",
        "  return model\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hs0Smn24Dty-"
      },
      "source": [
        "Build two-layer and a three-layer CNN models using that function. Again there's nothing provacy specific about this code. It uses standard models, layers, losses, and optimizers."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "nexqXAjqDgad"
      },
      "outputs": [],
      "source": [
        "optimizer = tf.keras.optimizers.SGD(lr=lr, momentum=momentum)\n",
        "loss = tf.keras.losses.CategoricalCrossentropy(from_logits=True)\n",
        "\n",
        "three_layer_model = small_cnn(\n",
        "    input_shape, num_classes, num_conv=3, activation=activation)\n",
        "three_layer_model.compile(loss=loss, optimizer=optimizer, metrics=['accuracy'])\n",
        "\n",
        "two_layer_model = small_cnn(\n",
        "    input_shape, num_classes, num_conv=2, activation=activation)\n",
        "two_layer_model.compile(loss=loss, optimizer=optimizer, metrics=['accuracy'])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "D9nrWjP9D65l"
      },
      "source": [
        "### Define a callback to collect privacy metrics\n",
        "\n",
        "Next define a `keras.callbacks.Callback` to periorically run some privacy attacks against the model, and log the results.\n",
        "\n",
        "The keras `fit` method will call the `on_epoch_end` method after each training epoch. The `n` argument is the (0-based) epoch number.\n",
        "\n",
        "You could implement this procedure by writing a loop that repeatedly calls `Model.fit(..., epochs=epochs_per_report)` and runs the attack code. The callback is used here just because it gives a clear separation between the training logic, and the privacy evaluation logic.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "won3NecEmzzg"
      },
      "outputs": [],
      "source": [
        "class PrivacyMetrics(tf.keras.callbacks.Callback):\n",
        "  def __init__(self, epochs_per_report, model_name):\n",
        "    self.epochs_per_report = epochs_per_report\n",
        "    self.model_name = model_name\n",
        "    self.epochs = []\n",
        "    self.attack_results = [] \n",
        "\n",
        "  def on_epoch_end(self, n, logs=None):\n",
        "    epoch = n + 1\n",
        "    if epoch % self.epochs_per_report != 0:\n",
        "      return\n",
        " \n",
        "    print(f\"\\nRunning privacy report for epoch: {epoch}\")\n",
        "    self.epochs.append(epoch)\n",
        "\n",
        "    logits_train = model.predict(x_train, batch_size=batch_size)\n",
        "    logits_test = model.predict(x_test, batch_size=batch_size)\n",
        "\n",
        "    prob_train = special.softmax(logits_train, axis=1)\n",
        "    prob_test = special.softmax(logits_test, axis=1)\n",
        "\n",
        "    # Add metadata to generate a privacy report.\n",
        "    privacy_report_metadata = PrivacyReportMetadata(\n",
        "        accuracy_train=metrics.accuracy_score(y_train_indices,\n",
        "                                              np.argmax(prob_train, axis=1)),\n",
        "        accuracy_test=metrics.accuracy_score(y_test_indices,\n",
        "                                             np.argmax(prob_test, axis=1)),\n",
        "        epoch_num=epoch,\n",
        "        model_variant_label=self.model_name)\n",
        "\n",
        "    attack_results = mia.run_attacks(\n",
        "        AttackInputData(\n",
        "            labels_train=np.asarray([x[0] for x in y_train_indices]),\n",
        "            labels_test=np.asarray([x[0] for x in y_test_indices]),\n",
        "            probs_train=prob_train,\n",
        "            probs_test=prob_test),\n",
        "        SlicingSpec(entire_dataset=True, by_class=True),\n",
        "        attack_types=(AttackType.THRESHOLD_ATTACK,\n",
        "                      AttackType.LOGISTIC_REGRESSION),\n",
        "        privacy_report_metadata=privacy_report_metadata)\n",
        "    self.attack_results.append(attack_results)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zLPHj5ZtFhC9"
      },
      "source": [
        "### Train the models\n",
        "\n",
        "The next code block trains the two models. The `all_reports` list is used to collect all the results from all the models' training runs. The individual reports are tagged witht the `model_name`, so there's no confusion about which model generated which report. "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Gywwxs6R1aLV"
      },
      "outputs": [],
      "source": [
        "all_reports = []\n",
        "\n",
        "models = {\n",
        "    'two layer model': two_layer_model,\n",
        "    'three layer model': three_layer_model,\n",
        "}\n",
        "\n",
        "for model_name, model in models.items():\n",
        "  print(f\"\\n\\n\\nFitting {model_name}\\n\")\n",
        "  callback = PrivacyMetrics(epochs_per_report, \n",
        "                            model_name)\n",
        "\n",
        "  model.fit(\n",
        "      x_train,\n",
        "      y_train,\n",
        "      batch_size=batch_size,\n",
        "      epochs=total_epochs,\n",
        "      validation_data=(x_test, y_test),\n",
        "      callbacks=[callback],\n",
        "      shuffle=True)\n",
        "  \n",
        "  all_reports.extend(callback.attack_results)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6mBEYh4utxiR"
      },
      "source": [
        "## Epoch Plots\n",
        "\n",
        "You can visualize how privacy risks happen as you train models by probing the model periodically (e.g. every 5 epochs), you can pick the point in time with the best performance / privacy trade-off.\n",
        "\n",
        "Use the TF Privacy Membership Inference Attack module to generate `AttackResults`. These `AttackResults` get combined into an `AttackResultsCollection`. The TF Privacy Report is designed to analyze the provided `AttackResultsCollection`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "wT7zfUC8HXRI"
      },
      "outputs": [],
      "source": [
        "results = AttackResultsCollection(all_reports)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "cellView": "both",
        "id": "o7T8n0ffv3qo"
      },
      "outputs": [],
      "source": [
        "privacy_metrics = (PrivacyMetric.AUC, PrivacyMetric.ATTACKER_ADVANTAGE)\n",
        "epoch_plot = privacy_report.plot_by_epochs(\n",
        "    results, privacy_metrics=privacy_metrics)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ijjwGgyixsFg"
      },
      "source": [
        "See that as a rule, privacy vulnerability tends to increase as the number of epochs goes up. This is true across model variants as well as different attacker types.\n",
        "\n",
        "Two layer models (with fewer convolutional layers) are generally more vulnerable than their three layer model counterparts.\n",
        "\n",
        "Now let's see how model performance changes with respect to privacy risk."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GbtlV-2Xu8s-"
      },
      "source": [
        "## Privacy vs Utility"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Lt6fXGoivLH1"
      },
      "outputs": [],
      "source": [
        "privacy_metrics = (PrivacyMetric.AUC, PrivacyMetric.ATTACKER_ADVANTAGE)\n",
        "utility_privacy_plot = privacy_report.plot_privacy_vs_accuracy(\n",
        "    results, privacy_metrics=privacy_metrics)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "m_6vg3pBPoyy"
      },
      "source": [
        "Three layer models (perhaps due to too many parameters) only achieve a train accuracy of 0.85. The two layer models achieve roughly equal performance for that level of privacy risk but they continue to get better accuracy.\n",
        "\n",
        "You can also see how the line for two layer models gets steeper. This means that additional marginal gains in train accuracy come at an expense of vast privacy vulnerabilities."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7u3BAg87v3qv"
      },
      "source": [
        "This is the end of the colab!\n",
        "Feel free to analyze your own results."
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "collapsed_sections": [],
      "name": "privacy_report.ipynb",
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}