{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "1eiwVljWpzM7"
   },
   "source": [
    "Copyright 2020 The TensorFlow Authors.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "cellView": "both",
    "colab": {},
    "colab_type": "code",
    "id": "4rmwPgXeptiS"
   },
   "outputs": [],
   "source": [
    "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
    "# you may not use this file except in compliance with the License.\n",
    "# You may obtain a copy of the License at\n",
    "#\n",
    "# https://www.apache.org/licenses/LICENSE-2.0\n",
    "#\n",
    "# Unless required by applicable law or agreed to in writing, software\n",
    "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
    "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
    "# See the License for the specific language governing permissions and\n",
    "# limitations under the License."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "YM2gRaJMqvMi"
   },
   "source": [
    "# Assess privacy risks with TensorFlow Privacy Membership Inference Attacks"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "-B5ZvlSqqLaR"
   },
   "source": [
    "<table class=\"tfo-notebook-buttons\" align=\"left\">\n",
    "  <td>\n",
    "    <a target=\"_blank\" href=\"https://colab.research.google.com/github/tensorflow/privacy/blob/master/tensorflow_privacy/privacy/privacy_tests/membership_inference_attack/codelabs/membership_probability_codelab.ipynb\"><img src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" />Run in Google Colab</a>\n",
    "  </td>\n",
    "  <td>\n",
    "    <a target=\"_blank\" href=\"https://github.com/tensorflow/privacy/blob/master/tensorflow_privacy/privacy/privacy_tests/membership_inference_attack/codelabs/membership_probability_codelab.ipynb\"><img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" />View source on GitHub</a>\n",
    "  </td>\n",
    "</table>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "9rMuytY7Nn8P"
   },
   "source": [
    "##Overview\n",
    "In this codelab we'll train a simple image classification model on the CIFAR10 dataset, and then use the \"membership inference attack\" against this model to assess if the attacker is able to \"guess\" whether a particular sample was present in the training set. We further compute each sample's probability of being in the training set, denoted as membership probability (also called privacy risk score in https://arxiv.org/abs/2003.10595)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "FUWqArj_q8vs"
   },
   "source": [
    "## Setup\n",
    "First, set this notebook's runtime to use a GPU, under Runtime > Change runtime type > Hardware accelerator. Then, begin importing the necessary libraries."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "cellView": "form",
    "colab": {},
    "colab_type": "code",
    "id": "Lr1pwHcbralz"
   },
   "outputs": [],
   "source": [
    "#@title Import statements.\n",
    "import numpy as np\n",
    "from typing import Tuple, Text\n",
    "from scipy import special\n",
    "\n",
    "import tensorflow as tf\n",
    "import tensorflow_datasets as tfds\n",
    "\n",
    "# Set verbosity.\n",
    "tf.compat.v1.logging.set_verbosity(tf.compat.v1.logging.ERROR)\n",
    "from warnings import simplefilter\n",
    "from sklearn.exceptions import ConvergenceWarning\n",
    "simplefilter(action=\"ignore\", category=ConvergenceWarning)\n",
    "simplefilter(action=\"ignore\", category=FutureWarning)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "ucw81ar6ru-6"
   },
   "source": [
    "### Install TensorFlow Privacy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "cellView": "both",
    "colab": {},
    "colab_type": "code",
    "id": "zcqAmiGH90kl"
   },
   "outputs": [],
   "source": [
    "!pip3 install git+https://github.com/tensorflow/privacy\n",
    "\n",
    "from tensorflow_privacy.privacy.privacy_tests.membership_inference_attack import membership_inference_attack as mia"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "pBbcG86th_sW"
   },
   "source": [
    "## Train a model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "cellView": "form",
    "colab": {},
    "colab_type": "code",
    "id": "vCyOWyyhXLib"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Loading the dataset.\n",
      "learning rate %f 0.02\n",
      "Model: \"sequential\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "conv2d (Conv2D)              (None, 30, 30, 32)        896       \n",
      "_________________________________________________________________\n",
      "max_pooling2d (MaxPooling2D) (None, 15, 15, 32)        0         \n",
      "_________________________________________________________________\n",
      "conv2d_1 (Conv2D)            (None, 13, 13, 32)        9248      \n",
      "_________________________________________________________________\n",
      "max_pooling2d_1 (MaxPooling2 (None, 6, 6, 32)          0         \n",
      "_________________________________________________________________\n",
      "conv2d_2 (Conv2D)            (None, 4, 4, 32)          9248      \n",
      "_________________________________________________________________\n",
      "max_pooling2d_2 (MaxPooling2 (None, 2, 2, 32)          0         \n",
      "_________________________________________________________________\n",
      "flatten (Flatten)            (None, 128)               0         \n",
      "_________________________________________________________________\n",
      "dense (Dense)                (None, 64)                8256      \n",
      "_________________________________________________________________\n",
      "dense_1 (Dense)              (None, 10)                650       \n",
      "=================================================================\n",
      "Total params: 28,298\n",
      "Trainable params: 28,298\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "Epoch 1/100\n",
      "200/200 [==============================] - 2s 8ms/step - loss: 2.0930 - accuracy: 0.2198 - val_loss: 1.7698 - val_accuracy: 0.3608\n",
      "Epoch 2/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 1.6091 - accuracy: 0.4121 - val_loss: 1.5084 - val_accuracy: 0.4486\n",
      "Epoch 3/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 1.4071 - accuracy: 0.4906 - val_loss: 1.3092 - val_accuracy: 0.5314\n",
      "Epoch 4/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 1.2850 - accuracy: 0.5377 - val_loss: 1.2988 - val_accuracy: 0.5378\n",
      "Epoch 5/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 1.2113 - accuracy: 0.5679 - val_loss: 1.2258 - val_accuracy: 0.5700\n",
      "Epoch 6/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 1.1571 - accuracy: 0.5888 - val_loss: 1.1821 - val_accuracy: 0.5803\n",
      "Epoch 7/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 1.1005 - accuracy: 0.6101 - val_loss: 1.1269 - val_accuracy: 0.6054\n",
      "Epoch 8/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 1.0607 - accuracy: 0.6268 - val_loss: 1.1337 - val_accuracy: 0.6024\n",
      "Epoch 9/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 1.0233 - accuracy: 0.6427 - val_loss: 1.0603 - val_accuracy: 0.6245\n",
      "Epoch 10/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.9838 - accuracy: 0.6543 - val_loss: 1.0552 - val_accuracy: 0.6279\n",
      "Epoch 11/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.9485 - accuracy: 0.6651 - val_loss: 1.0529 - val_accuracy: 0.6331\n",
      "Epoch 12/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.9340 - accuracy: 0.6701 - val_loss: 1.0530 - val_accuracy: 0.6347\n",
      "Epoch 13/100\n",
      "200/200 [==============================] - 1s 6ms/step - loss: 0.9062 - accuracy: 0.6826 - val_loss: 0.9814 - val_accuracy: 0.6559\n",
      "Epoch 14/100\n",
      "200/200 [==============================] - 1s 6ms/step - loss: 0.8795 - accuracy: 0.6900 - val_loss: 0.9736 - val_accuracy: 0.6611\n",
      "Epoch 15/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.8520 - accuracy: 0.7012 - val_loss: 0.9815 - val_accuracy: 0.6603\n",
      "Epoch 16/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.8355 - accuracy: 0.7061 - val_loss: 0.9624 - val_accuracy: 0.6667\n",
      "Epoch 17/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.8275 - accuracy: 0.7070 - val_loss: 0.9532 - val_accuracy: 0.6688\n",
      "Epoch 18/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.8063 - accuracy: 0.7176 - val_loss: 0.9509 - val_accuracy: 0.6731\n",
      "Epoch 19/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.7977 - accuracy: 0.7187 - val_loss: 0.9552 - val_accuracy: 0.6714\n",
      "Epoch 20/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.7694 - accuracy: 0.7311 - val_loss: 0.9365 - val_accuracy: 0.6778\n",
      "Epoch 21/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.7577 - accuracy: 0.7332 - val_loss: 0.9535 - val_accuracy: 0.6742\n",
      "Epoch 22/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.7426 - accuracy: 0.7397 - val_loss: 0.9355 - val_accuracy: 0.6832\n",
      "Epoch 23/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.7369 - accuracy: 0.7397 - val_loss: 0.9162 - val_accuracy: 0.6846\n",
      "Epoch 24/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.7156 - accuracy: 0.7490 - val_loss: 0.9445 - val_accuracy: 0.6741\n",
      "Epoch 25/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.7136 - accuracy: 0.7478 - val_loss: 0.9920 - val_accuracy: 0.6647\n",
      "Epoch 26/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.7003 - accuracy: 0.7542 - val_loss: 0.9611 - val_accuracy: 0.6710\n",
      "Epoch 27/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.6812 - accuracy: 0.7604 - val_loss: 0.9575 - val_accuracy: 0.6787\n",
      "Epoch 28/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.6736 - accuracy: 0.7612 - val_loss: 0.9574 - val_accuracy: 0.6814\n",
      "Epoch 29/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.6560 - accuracy: 0.7673 - val_loss: 0.9552 - val_accuracy: 0.6840\n",
      "Epoch 30/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.6505 - accuracy: 0.7708 - val_loss: 0.9528 - val_accuracy: 0.6878\n",
      "Epoch 31/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.6447 - accuracy: 0.7712 - val_loss: 0.9949 - val_accuracy: 0.6835\n",
      "Epoch 32/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.6402 - accuracy: 0.7747 - val_loss: 0.9628 - val_accuracy: 0.6794\n",
      "Epoch 33/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.6333 - accuracy: 0.7748 - val_loss: 0.9735 - val_accuracy: 0.6877\n",
      "Epoch 34/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.6116 - accuracy: 0.7846 - val_loss: 1.0004 - val_accuracy: 0.6727\n",
      "Epoch 35/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.6108 - accuracy: 0.7837 - val_loss: 1.0057 - val_accuracy: 0.6802\n",
      "Epoch 36/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.6157 - accuracy: 0.7815 - val_loss: 0.9964 - val_accuracy: 0.6814\n",
      "Epoch 37/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.5947 - accuracy: 0.7891 - val_loss: 0.9903 - val_accuracy: 0.6820\n",
      "Epoch 38/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.5878 - accuracy: 0.7921 - val_loss: 1.0062 - val_accuracy: 0.6728\n",
      "Epoch 39/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.5796 - accuracy: 0.7958 - val_loss: 0.9860 - val_accuracy: 0.6804\n",
      "Epoch 40/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.5982 - accuracy: 0.7883 - val_loss: 0.9707 - val_accuracy: 0.6876\n",
      "Epoch 41/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.5744 - accuracy: 0.7958 - val_loss: 1.0108 - val_accuracy: 0.6895\n",
      "Epoch 42/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.5732 - accuracy: 0.7960 - val_loss: 1.0306 - val_accuracy: 0.6836\n",
      "Epoch 43/100\n",
      "200/200 [==============================] - 1s 6ms/step - loss: 0.5666 - accuracy: 0.8002 - val_loss: 1.0309 - val_accuracy: 0.6772\n",
      "Epoch 44/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.5633 - accuracy: 0.7987 - val_loss: 1.0249 - val_accuracy: 0.6817\n",
      "Epoch 45/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.5425 - accuracy: 0.8066 - val_loss: 1.0539 - val_accuracy: 0.6788\n",
      "Epoch 46/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.5421 - accuracy: 0.8067 - val_loss: 1.0570 - val_accuracy: 0.6791\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 47/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.5407 - accuracy: 0.8072 - val_loss: 1.0616 - val_accuracy: 0.6804\n",
      "Epoch 48/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.5302 - accuracy: 0.8108 - val_loss: 1.0639 - val_accuracy: 0.6843\n",
      "Epoch 49/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.5354 - accuracy: 0.8100 - val_loss: 1.0413 - val_accuracy: 0.6779\n",
      "Epoch 50/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.5269 - accuracy: 0.8126 - val_loss: 1.0934 - val_accuracy: 0.6748\n",
      "Epoch 51/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.5195 - accuracy: 0.8146 - val_loss: 1.0981 - val_accuracy: 0.6779\n",
      "Epoch 52/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.5171 - accuracy: 0.8161 - val_loss: 1.0979 - val_accuracy: 0.6755\n",
      "Epoch 53/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.5108 - accuracy: 0.8179 - val_loss: 1.0986 - val_accuracy: 0.6796\n",
      "Epoch 54/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.5052 - accuracy: 0.8190 - val_loss: 1.1232 - val_accuracy: 0.6736\n",
      "Epoch 55/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.5052 - accuracy: 0.8203 - val_loss: 1.1259 - val_accuracy: 0.6798\n",
      "Epoch 56/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.4998 - accuracy: 0.8215 - val_loss: 1.1352 - val_accuracy: 0.6790\n",
      "Epoch 57/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.4934 - accuracy: 0.8224 - val_loss: 1.1311 - val_accuracy: 0.6755\n",
      "Epoch 58/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.4855 - accuracy: 0.8271 - val_loss: 1.1364 - val_accuracy: 0.6782\n",
      "Epoch 59/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.4816 - accuracy: 0.8269 - val_loss: 1.1209 - val_accuracy: 0.6820\n",
      "Epoch 60/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.4823 - accuracy: 0.8260 - val_loss: 1.1343 - val_accuracy: 0.6776\n",
      "Epoch 61/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.4804 - accuracy: 0.8277 - val_loss: 1.1591 - val_accuracy: 0.6611\n",
      "Epoch 62/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.4737 - accuracy: 0.8312 - val_loss: 1.1828 - val_accuracy: 0.6701\n",
      "Epoch 63/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.4850 - accuracy: 0.8256 - val_loss: 1.1826 - val_accuracy: 0.6739\n",
      "Epoch 64/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.4700 - accuracy: 0.8325 - val_loss: 1.1647 - val_accuracy: 0.6843\n",
      "Epoch 65/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.4613 - accuracy: 0.8342 - val_loss: 1.1689 - val_accuracy: 0.6761\n",
      "Epoch 66/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.4674 - accuracy: 0.8315 - val_loss: 1.1811 - val_accuracy: 0.6790\n",
      "Epoch 67/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.4547 - accuracy: 0.8367 - val_loss: 1.2061 - val_accuracy: 0.6770\n",
      "Epoch 68/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.4578 - accuracy: 0.8337 - val_loss: 1.1907 - val_accuracy: 0.6664\n",
      "Epoch 69/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.4512 - accuracy: 0.8368 - val_loss: 1.2001 - val_accuracy: 0.6744\n",
      "Epoch 70/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.4414 - accuracy: 0.8393 - val_loss: 1.2204 - val_accuracy: 0.6651\n",
      "Epoch 71/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.4518 - accuracy: 0.8370 - val_loss: 1.2291 - val_accuracy: 0.6772\n",
      "Epoch 72/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.4360 - accuracy: 0.8432 - val_loss: 1.2683 - val_accuracy: 0.6767\n",
      "Epoch 73/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.4332 - accuracy: 0.8450 - val_loss: 1.2477 - val_accuracy: 0.6723\n",
      "Epoch 74/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.4421 - accuracy: 0.8399 - val_loss: 1.2652 - val_accuracy: 0.6757\n",
      "Epoch 75/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.4366 - accuracy: 0.8439 - val_loss: 1.2680 - val_accuracy: 0.6740\n",
      "Epoch 76/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.4414 - accuracy: 0.8411 - val_loss: 1.2742 - val_accuracy: 0.6645\n",
      "Epoch 77/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.4323 - accuracy: 0.8445 - val_loss: 1.2843 - val_accuracy: 0.6688\n",
      "Epoch 78/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.4336 - accuracy: 0.8432 - val_loss: 1.3271 - val_accuracy: 0.6589\n",
      "Epoch 79/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.4246 - accuracy: 0.8467 - val_loss: 1.3313 - val_accuracy: 0.6671\n",
      "Epoch 80/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.4229 - accuracy: 0.8471 - val_loss: 1.3182 - val_accuracy: 0.6665\n",
      "Epoch 81/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.4206 - accuracy: 0.8473 - val_loss: 1.3285 - val_accuracy: 0.6681\n",
      "Epoch 82/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.4291 - accuracy: 0.8449 - val_loss: 1.3572 - val_accuracy: 0.6638\n",
      "Epoch 83/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.4091 - accuracy: 0.8521 - val_loss: 1.3253 - val_accuracy: 0.6633\n",
      "Epoch 84/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.4080 - accuracy: 0.8525 - val_loss: 1.3506 - val_accuracy: 0.6726\n",
      "Epoch 85/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.4112 - accuracy: 0.8498 - val_loss: 1.3412 - val_accuracy: 0.6572\n",
      "Epoch 86/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.4081 - accuracy: 0.8518 - val_loss: 1.3353 - val_accuracy: 0.6649\n",
      "Epoch 87/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.4039 - accuracy: 0.8538 - val_loss: 1.4257 - val_accuracy: 0.6648\n",
      "Epoch 88/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.4127 - accuracy: 0.8502 - val_loss: 1.3950 - val_accuracy: 0.6680\n",
      "Epoch 89/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.4081 - accuracy: 0.8532 - val_loss: 1.3847 - val_accuracy: 0.6723\n",
      "Epoch 90/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.3964 - accuracy: 0.8578 - val_loss: 1.3938 - val_accuracy: 0.6646\n",
      "Epoch 91/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.4145 - accuracy: 0.8495 - val_loss: 1.4003 - val_accuracy: 0.6658\n",
      "Epoch 92/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.4036 - accuracy: 0.8529 - val_loss: 1.4180 - val_accuracy: 0.6586\n",
      "Epoch 93/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.3876 - accuracy: 0.8589 - val_loss: 1.4513 - val_accuracy: 0.6572\n",
      "Epoch 94/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.3950 - accuracy: 0.8564 - val_loss: 1.4425 - val_accuracy: 0.6586\n",
      "Epoch 95/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.3919 - accuracy: 0.8585 - val_loss: 1.4105 - val_accuracy: 0.6677\n",
      "Epoch 96/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.3982 - accuracy: 0.8556 - val_loss: 1.4089 - val_accuracy: 0.6653\n",
      "Epoch 97/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.3978 - accuracy: 0.8565 - val_loss: 1.4066 - val_accuracy: 0.6665\n",
      "Epoch 98/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.4014 - accuracy: 0.8536 - val_loss: 1.4905 - val_accuracy: 0.6514\n",
      "Epoch 99/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.4059 - accuracy: 0.8531 - val_loss: 1.4303 - val_accuracy: 0.6608\n",
      "Epoch 100/100\n",
      "200/200 [==============================] - 1s 5ms/step - loss: 0.3881 - accuracy: 0.8588 - val_loss: 1.4523 - val_accuracy: 0.6540\n",
      "Finished training.\n"
     ]
    }
   ],
   "source": [
    "#@markdown Train a simple model on CIFAR10 with Keras.\n",
    "\n",
    "dataset = 'cifar10'\n",
    "num_classes = 10\n",
    "num_conv = 3\n",
    "activation = 'relu'\n",
    "lr = 0.02\n",
    "momentum = 0.9\n",
    "batch_size = 250\n",
    "epochs = 100  # Privacy risks are especially visible with lots of epochs.\n",
    "\n",
    "\n",
    "def small_cnn(input_shape: Tuple[int],\n",
    "              num_classes: int,\n",
    "              num_conv: int,\n",
    "              activation: Text = 'relu') -> tf.keras.models.Sequential:\n",
    "  \"\"\"Setup a small CNN for image classification.\n",
    "\n",
    "  Args:\n",
    "    input_shape: Integer tuple for the shape of the images.\n",
    "    num_classes: Number of prediction classes.\n",
    "    num_conv: Number of convolutional layers.\n",
    "    activation: The activation function to use for conv and dense layers.\n",
    "\n",
    "  Returns:\n",
    "    The Keras model.\n",
    "  \"\"\"\n",
    "  model = tf.keras.models.Sequential()\n",
    "  model.add(tf.keras.layers.Input(shape=input_shape))\n",
    "\n",
    "  # Conv layers\n",
    "  for _ in range(num_conv):\n",
    "    model.add(tf.keras.layers.Conv2D(32, (3, 3), activation=activation))\n",
    "    model.add(tf.keras.layers.MaxPooling2D())\n",
    "\n",
    "  model.add(tf.keras.layers.Flatten())\n",
    "  model.add(tf.keras.layers.Dense(64, activation=activation))\n",
    "  model.add(tf.keras.layers.Dense(num_classes))\n",
    "  return model\n",
    "\n",
    "\n",
    "print('Loading the dataset.')\n",
    "train_ds = tfds.as_numpy(\n",
    "    tfds.load(dataset, split=tfds.Split.TRAIN, batch_size=-1))\n",
    "test_ds = tfds.as_numpy(\n",
    "    tfds.load(dataset, split=tfds.Split.TEST, batch_size=-1))\n",
    "x_train = train_ds['image'].astype('float32') / 255.\n",
    "y_train_indices = train_ds['label'][:, np.newaxis]\n",
    "x_test = test_ds['image'].astype('float32') / 255.\n",
    "y_test_indices = test_ds['label'][:, np.newaxis]\n",
    "\n",
    "# Convert class vectors to binary class matrices.\n",
    "y_train = tf.keras.utils.to_categorical(y_train_indices, num_classes)\n",
    "y_test = tf.keras.utils.to_categorical(y_test_indices, num_classes)\n",
    "\n",
    "input_shape = x_train.shape[1:]\n",
    "\n",
    "model = small_cnn(\n",
    "    input_shape, num_classes, num_conv=num_conv, activation=activation)\n",
    "\n",
    "print('learning rate %f', lr)\n",
    "\n",
    "optimizer = tf.keras.optimizers.SGD(lr=lr, momentum=momentum)\n",
    "\n",
    "loss = tf.keras.losses.CategoricalCrossentropy(from_logits=True)\n",
    "model.compile(loss=loss, optimizer=optimizer, metrics=['accuracy'])\n",
    "model.summary()\n",
    "model.fit(\n",
    "    x_train,\n",
    "    y_train,\n",
    "    batch_size=batch_size,\n",
    "    epochs=epochs,\n",
    "    validation_data=(x_test, y_test),\n",
    "    shuffle=True)\n",
    "print('Finished training.')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "ee-zjGGGV1DC"
   },
   "source": [
    "## Calculate logits, probabilities and loss values for training and test sets.\n",
    "\n",
    "We will use these values later in the membership inference attack and membership probability analysis to separate training and test samples."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "cellView": "both",
    "colab": {},
    "colab_type": "code",
    "id": "um9r0tSiPx4u"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Predict on train...\n",
      "Predict on test...\n",
      "Apply softmax to get probabilities from logits...\n",
      "Compute losses...\n"
     ]
    }
   ],
   "source": [
    "print('Predict on train...')\n",
    "logits_train = model.predict(x_train, batch_size=batch_size)\n",
    "print('Predict on test...')\n",
    "logits_test = model.predict(x_test, batch_size=batch_size)\n",
    "\n",
    "print('Apply softmax to get probabilities from logits...')\n",
    "prob_train = special.softmax(logits_train, axis=1)\n",
    "prob_test = special.softmax(logits_test, axis=1)\n",
    "\n",
    "print('Compute losses...')\n",
    "cce = tf.keras.backend.categorical_crossentropy\n",
    "constant = tf.keras.backend.constant\n",
    "\n",
    "loss_train = cce(constant(y_train), constant(prob_train), from_logits=False).numpy()\n",
    "loss_test = cce(constant(y_test), constant(prob_test), from_logits=False).numpy()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "QETxVOHLiHP4"
   },
   "source": [
    "## Run membership inference attacks.\n",
    "\n",
    "We will now execute a membership inference attack against the previously trained CIFAR10 model. This will generate a number of scores, most notably, attacker advantage and AUC for the membership inference classifier.\n",
    "\n",
    "An AUC of close to 0.5 means that the attack wasn't able to identify training samples, which means that the model doesn't have privacy issues according to this test. Higher values, on the contrary, indicate potential privacy issues.\n",
    "\n",
    "For comparison with the following membership probability analysis, here we only perform threshold attack."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "B8NIwhVwQT7I"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best-performing attacks over all slices\n",
      "  THRESHOLD_ATTACK achieved an AUC of 0.72 on slice CORRECTLY_CLASSIFIED=False\n",
      "  THRESHOLD_ATTACK achieved an advantage of 0.36 on slice CORRECTLY_CLASSIFIED=False\n",
      "\n",
      "Best-performing attacks over slice: \"Entire dataset\"\n",
      "  THRESHOLD_ATTACK achieved an AUC of 0.60\n",
      "  THRESHOLD_ATTACK achieved an advantage of 0.20\n",
      "\n",
      "Best-performing attacks over slice: \"CLASS=0\"\n",
      "  THRESHOLD_ATTACK achieved an AUC of 0.61\n",
      "  THRESHOLD_ATTACK achieved an advantage of 0.20\n",
      "\n",
      "Best-performing attacks over slice: \"CLASS=1\"\n",
      "  THRESHOLD_ATTACK achieved an AUC of 0.56\n",
      "  THRESHOLD_ATTACK achieved an advantage of 0.17\n",
      "\n",
      "Best-performing attacks over slice: \"CLASS=2\"\n",
      "  THRESHOLD_ATTACK achieved an AUC of 0.64\n",
      "  THRESHOLD_ATTACK achieved an advantage of 0.26\n",
      "\n",
      "Best-performing attacks over slice: \"CLASS=3\"\n",
      "  THRESHOLD_ATTACK achieved an AUC of 0.66\n",
      "  THRESHOLD_ATTACK achieved an advantage of 0.30\n",
      "\n",
      "Best-performing attacks over slice: \"CLASS=4\"\n",
      "  THRESHOLD_ATTACK achieved an AUC of 0.62\n",
      "  THRESHOLD_ATTACK achieved an advantage of 0.22\n",
      "\n",
      "Best-performing attacks over slice: \"CLASS=5\"\n",
      "  THRESHOLD_ATTACK achieved an AUC of 0.63\n",
      "  THRESHOLD_ATTACK achieved an advantage of 0.24\n",
      "\n",
      "Best-performing attacks over slice: \"CLASS=6\"\n",
      "  THRESHOLD_ATTACK achieved an AUC of 0.57\n",
      "  THRESHOLD_ATTACK achieved an advantage of 0.16\n",
      "\n",
      "Best-performing attacks over slice: \"CLASS=7\"\n",
      "  THRESHOLD_ATTACK achieved an AUC of 0.60\n",
      "  THRESHOLD_ATTACK achieved an advantage of 0.23\n",
      "\n",
      "Best-performing attacks over slice: \"CLASS=8\"\n",
      "  THRESHOLD_ATTACK achieved an AUC of 0.56\n",
      "  THRESHOLD_ATTACK achieved an advantage of 0.14\n",
      "\n",
      "Best-performing attacks over slice: \"CLASS=9\"\n",
      "  THRESHOLD_ATTACK achieved an AUC of 0.56\n",
      "  THRESHOLD_ATTACK achieved an advantage of 0.16\n",
      "\n",
      "Best-performing attacks over slice: \"CORRECTLY_CLASSIFIED=True\"\n",
      "  THRESHOLD_ATTACK achieved an AUC of 0.47\n",
      "  THRESHOLD_ATTACK achieved an advantage of 0.05\n",
      "\n",
      "Best-performing attacks over slice: \"CORRECTLY_CLASSIFIED=False\"\n",
      "  THRESHOLD_ATTACK achieved an AUC of 0.72\n",
      "  THRESHOLD_ATTACK achieved an advantage of 0.36\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 17968 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from tensorflow_privacy.privacy.privacy_tests.membership_inference_attack.data_structures import AttackInputData\n",
    "from tensorflow_privacy.privacy.privacy_tests.membership_inference_attack.data_structures import SlicingSpec\n",
    "from tensorflow_privacy.privacy.privacy_tests.membership_inference_attack.data_structures import AttackType\n",
    "\n",
    "import tensorflow_privacy.privacy.privacy_tests.membership_inference_attack.plotting as plotting\n",
    "\n",
    "labels_train = np.argmax(y_train, axis=1)\n",
    "labels_test = np.argmax(y_test, axis=1)\n",
    "\n",
    "input = AttackInputData(\n",
    "  logits_train = logits_train,\n",
    "  logits_test = logits_test,\n",
    "  loss_train = loss_train,\n",
    "  loss_test = loss_test,\n",
    "  labels_train = labels_train,\n",
    "  labels_test = labels_test\n",
    ")\n",
    "\n",
    "# Run several attacks for different data slices\n",
    "attacks_result = mia.run_attacks(input,\n",
    "                                 SlicingSpec(\n",
    "                                     entire_dataset = True,\n",
    "                                     by_class = True,\n",
    "                                     by_classification_correctness = True\n",
    "                                 ),\n",
    "                                 attack_types = [\n",
    "                                     AttackType.THRESHOLD_ATTACK])\n",
    "\n",
    "# Plot the ROC curve of the best classifier\n",
    "fig = plotting.plot_roc_curve(\n",
    "    attacks_result.get_result_with_max_auc().roc_curve)\n",
    "\n",
    "# Print a user-friendly summary of the attacks\n",
    "print(attacks_result.summary(by_slices = True))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "E9zwsPGFujVq"
   },
   "source": [
    "## Membership probability analysis\n",
    "\n",
    "This part shows how to use the membership probability.\n",
    "\n",
    "For each data slice, we compute membership probabilities for both training and test data. We then set a threshold on membership probability (an input is inferred as a member if and only if its membership probability is higher than the threshold) and compute the attack precision and recall values. We also report AUC and advantange values here for comparison with threshold attacks."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Membership probability analysis over slice: \"Entire dataset\"\n",
      "  with 0.6000 as the threshold on membership probability, the precision-recall pair is (0.6031, 0.1281)\n",
      "  with 0.5000 as the threshold on membership probability, the precision-recall pair is (0.5647, 0.8502)\n",
      "  thresholding on membership probability achieved an AUC of 0.62\n",
      "  thresholding on membership probability achieved an advantage of 0.19\n",
      "\n",
      "Membership probability analysis over slice: \"CLASS=0\"\n",
      "  with 0.6000 as the threshold on membership probability, the precision-recall pair is (0.6178, 0.1956)\n",
      "  with 0.5000 as the threshold on membership probability, the precision-recall pair is (0.5685, 0.8090)\n",
      "  thresholding on membership probability achieved an AUC of 0.62\n",
      "  thresholding on membership probability achieved an advantage of 0.20\n",
      "\n",
      "Membership probability analysis over slice: \"CLASS=1\"\n",
      "  with 0.6000 as the threshold on membership probability, the precision-recall pair is (0.6006, 0.0842)\n",
      "  with 0.5000 as the threshold on membership probability, the precision-recall pair is (0.5602, 0.7528)\n",
      "  thresholding on membership probability achieved an AUC of 0.60\n",
      "  thresholding on membership probability achieved an advantage of 0.16\n",
      "\n",
      "Membership probability analysis over slice: \"CLASS=2\"\n",
      "  with 0.6000 as the threshold on membership probability, the precision-recall pair is (0.6100, 0.4348)\n",
      "  with 0.5000 as the threshold on membership probability, the precision-recall pair is (0.5888, 0.8562)\n",
      "  thresholding on membership probability achieved an AUC of 0.65\n",
      "  thresholding on membership probability achieved an advantage of 0.26\n",
      "\n",
      "Membership probability analysis over slice: \"CLASS=3\"\n",
      "  with 1.0000 as the threshold on membership probability, the precision-recall pair is (1.0000, 0.0016)\n",
      "  with 0.9000 as the threshold on membership probability, the precision-recall pair is (1.0000, 0.0016)\n",
      "  with 0.8000 as the threshold on membership probability, the precision-recall pair is (1.0000, 0.0016)\n",
      "  with 0.7000 as the threshold on membership probability, the precision-recall pair is (0.8387, 0.0052)\n",
      "  with 0.6000 as the threshold on membership probability, the precision-recall pair is (0.6323, 0.5640)\n",
      "  with 0.5000 as the threshold on membership probability, the precision-recall pair is (0.6036, 0.8512)\n",
      "  thresholding on membership probability achieved an AUC of 0.68\n",
      "  thresholding on membership probability achieved an advantage of 0.29\n",
      "\n",
      "Membership probability analysis over slice: \"CLASS=4\"\n",
      "  with 1.0000 as the threshold on membership probability, the precision-recall pair is (1.0000, 0.0038)\n",
      "  with 0.9000 as the threshold on membership probability, the precision-recall pair is (1.0000, 0.0038)\n",
      "  with 0.8000 as the threshold on membership probability, the precision-recall pair is (0.8592, 0.0122)\n",
      "  with 0.7000 as the threshold on membership probability, the precision-recall pair is (0.8592, 0.0122)\n",
      "  with 0.6000 as the threshold on membership probability, the precision-recall pair is (0.6291, 0.2816)\n",
      "  with 0.5000 as the threshold on membership probability, the precision-recall pair is (0.5751, 0.8392)\n",
      "  thresholding on membership probability achieved an AUC of 0.65\n",
      "  thresholding on membership probability achieved an advantage of 0.22\n",
      "\n",
      "Membership probability analysis over slice: \"CLASS=5\"\n",
      "  with 0.7000 as the threshold on membership probability, the precision-recall pair is (0.7727, 0.0034)\n",
      "  with 0.6000 as the threshold on membership probability, the precision-recall pair is (0.6287, 0.1710)\n",
      "  with 0.5000 as the threshold on membership probability, the precision-recall pair is (0.5822, 0.8500)\n",
      "  thresholding on membership probability achieved an AUC of 0.64\n",
      "  thresholding on membership probability achieved an advantage of 0.24\n",
      "\n",
      "Membership probability analysis over slice: \"CLASS=6\"\n",
      "  with 0.6000 as the threshold on membership probability, the precision-recall pair is (0.6429, 0.0936)\n",
      "  with 0.5000 as the threshold on membership probability, the precision-recall pair is (0.5569, 0.7868)\n",
      "  thresholding on membership probability achieved an AUC of 0.61\n",
      "  thresholding on membership probability achieved an advantage of 0.16\n",
      "\n",
      "Membership probability analysis over slice: \"CLASS=7\"\n",
      "  with 0.6000 as the threshold on membership probability, the precision-recall pair is (0.6528, 0.1448)\n",
      "  with 0.5000 as the threshold on membership probability, the precision-recall pair is (0.5806, 0.7850)\n",
      "  thresholding on membership probability achieved an AUC of 0.63\n",
      "  thresholding on membership probability achieved an advantage of 0.22\n",
      "\n",
      "Membership probability analysis over slice: \"CLASS=8\"\n",
      "  with 0.5000 as the threshold on membership probability, the precision-recall pair is (0.5423, 0.8070)\n",
      "  thresholding on membership probability achieved an AUC of 0.58\n",
      "  thresholding on membership probability achieved an advantage of 0.13\n",
      "\n",
      "Membership probability analysis over slice: \"CLASS=9\"\n",
      "  with 0.5000 as the threshold on membership probability, the precision-recall pair is (0.5624, 0.8238)\n",
      "  thresholding on membership probability achieved an AUC of 0.61\n",
      "  thresholding on membership probability achieved an advantage of 0.18\n",
      "\n",
      "Membership probability analysis over slice: \"CORRECTLY_CLASSIFIED=True\"\n",
      "  with 0.5000 as the threshold on membership probability, the precision-recall pair is (0.5223, 0.5833)\n",
      "  thresholding on membership probability achieved an AUC of 0.53\n",
      "  thresholding on membership probability achieved an advantage of 0.05\n",
      "\n",
      "Membership probability analysis over slice: \"CORRECTLY_CLASSIFIED=False\"\n",
      "  with 0.6000 as the threshold on membership probability, the precision-recall pair is (0.6569, 0.7427)\n",
      "  with 0.5000 as the threshold on membership probability, the precision-recall pair is (0.6569, 0.7427)\n",
      "  thresholding on membership probability achieved an AUC of 0.72\n",
      "  thresholding on membership probability achieved an advantage of 0.35\n"
     ]
    }
   ],
   "source": [
    "# compute membership probabilities on all given data slices\n",
    "membership_probability_results = mia.run_membership_probability_analysis(input,\n",
    "                                                                         SlicingSpec(\n",
    "                                                                             entire_dataset = True,\n",
    "                                                                             by_class = True,\n",
    "                                                                             by_classification_correctness = True))\n",
    "# print the summary of membership probability analysis\n",
    "print(membership_probability_results.summary(threshold_list=[1, 0.9, 0.8, 0.7, 0.6, 0.5]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Manually plot training samples with high and low membership risks\n",
    "\n",
    "For samples with low membership risks, I pick samples whose membership probabilities are closest to 0.5."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "For data class 0\n",
      "  Plot high risky training points\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 21916 characters]\n",
      "text/plain": [
       "<Figure size 720x3600 with 5 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Plot low risky training points\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 19948 characters]\n",
      "text/plain": [
       "<Figure size 720x3600 with 5 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "For data class 1\n",
      "  Plot high risky training points\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 21988 characters]\n",
      "text/plain": [
       "<Figure size 720x3600 with 5 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Plot low risky training points\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 23076 characters]\n",
      "text/plain": [
       "<Figure size 720x3600 with 5 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "For data class 2\n",
      "  Plot high risky training points\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 22700 characters]\n",
      "text/plain": [
       "<Figure size 720x3600 with 5 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Plot low risky training points\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 23736 characters]\n",
      "text/plain": [
       "<Figure size 720x3600 with 5 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "For data class 3\n",
      "  Plot high risky training points\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 23604 characters]\n",
      "text/plain": [
       "<Figure size 720x3600 with 5 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Plot low risky training points\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 24428 characters]\n",
      "text/plain": [
       "<Figure size 720x3600 with 5 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "For data class 4\n",
      "  Plot high risky training points\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 24612 characters]\n",
      "text/plain": [
       "<Figure size 720x3600 with 5 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Plot low risky training points\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 24180 characters]\n",
      "text/plain": [
       "<Figure size 720x3600 with 5 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "For data class 5\n",
      "  Plot high risky training points\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 24760 characters]\n",
      "text/plain": [
       "<Figure size 720x3600 with 5 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Plot low risky training points\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 25528 characters]\n",
      "text/plain": [
       "<Figure size 720x3600 with 5 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "For data class 6\n",
      "  Plot high risky training points\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 26240 characters]\n",
      "text/plain": [
       "<Figure size 720x3600 with 5 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Plot low risky training points\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 26000 characters]\n",
      "text/plain": [
       "<Figure size 720x3600 with 5 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "For data class 7\n",
      "  Plot high risky training points\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 24276 characters]\n",
      "text/plain": [
       "<Figure size 720x3600 with 5 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Plot low risky training points\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 23980 characters]\n",
      "text/plain": [
       "<Figure size 720x3600 with 5 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "For data class 8\n",
      "  Plot high risky training points\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 23024 characters]\n",
      "text/plain": [
       "<Figure size 720x3600 with 5 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Plot low risky training points\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 22180 characters]\n",
      "text/plain": [
       "<Figure size 720x3600 with 5 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "For data class 9\n",
      "  Plot high risky training points\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 24616 characters]\n",
      "text/plain": [
       "<Figure size 720x3600 with 5 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Plot low risky training points\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 25548 characters]\n",
      "text/plain": [
       "<Figure size 720x3600 with 5 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from tensorflow_privacy.privacy.privacy_tests.membership_inference_attack.data_structures import SingleSliceSpec\n",
    "from tensorflow_privacy.privacy.privacy_tests.membership_inference_attack.data_structures import SlicingFeature\n",
    "from tensorflow_privacy.privacy.privacy_tests.membership_inference_attack.dataset_slicing import get_slice\n",
    "import matplotlib.pyplot as plt\n",
    "class_list = np.arange(10)\n",
    "num_images = 5\n",
    "for c in class_list:\n",
    "    print('For data class '+str(c))\n",
    "    class_slice_spec = SingleSliceSpec(SlicingFeature.CLASS, c)\n",
    "    class_input_slice = get_slice(input, class_slice_spec)\n",
    "    class_dataset_idx = np.argwhere(labels_train==c).flatten()\n",
    "\n",
    "    class_train_membership_probs = mia._compute_membership_probability(class_input_slice).train_membership_probs\n",
    "\n",
    "    class_high_risky_idx = np.argsort(class_train_membership_probs)[::-1][:num_images]\n",
    "    class_low_risky_idx = np.argsort(np.absolute(class_train_membership_probs-0.5))[:num_images]\n",
    "    \n",
    "    high_risky_images = x_train[class_dataset_idx[class_high_risky_idx]]\n",
    "    low_risky_images = x_train[class_dataset_idx[class_low_risky_idx]]\n",
    "    print('  Plot high risky training points')\n",
    "    fig = plt.figure(figsize=(10,10*num_images))\n",
    "    for i in range(num_images):\n",
    "        fig.add_subplot(1, num_images, i+1)\n",
    "        plt.axis('off')\n",
    "        plt.imshow(high_risky_images[i])\n",
    "    plt.show()\n",
    "\n",
    "    print('  Plot low risky training points')\n",
    "    fig = plt.figure(figsize=(10,10*num_images))\n",
    "\n",
    "    for i in range(num_images):\n",
    "        fig.add_subplot(1, num_images, i+1)\n",
    "        plt.axis('off')\n",
    "        plt.imshow(low_risky_images[i])\n",
    "    \n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "last_runtime": {
    "build_target": "//learning/deepmind/public/tools/ml_python:ml_notebook",
    "kind": "private"
   },
   "name": "Membership probability analysis codelab",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.10"
  },
  "pycharm": {
   "stem_cell": {
    "cell_type": "raw",
    "metadata": {
     "collapsed": false
    },
    "source": []
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
