diff --git a/research/mi_lira_2021/README.md b/research/mi_lira_2021/README.md index 72cd48f..9c025f0 100644 --- a/research/mi_lira_2021/README.md +++ b/research/mi_lira_2021/README.md @@ -94,15 +94,15 @@ which should give (something like) the following output ``` Attack Ours (online) - AUC 0.6676, Accuracy 0.6077, TPR@0.1%FPR of 0.0169 + AUC 0.6675, Accuracy 0.6074, TPR@0.1%FPR of 0.0104 Attack Ours (online, fixed variance) - AUC 0.6856, Accuracy 0.6137, TPR@0.1%FPR of 0.0593 + AUC 0.6831, Accuracy 0.6140, TPR@0.1%FPR of 0.0541 Attack Ours (offline) - AUC 0.5488, Accuracy 0.5500, TPR@0.1%FPR of 0.0130 + AUC 0.5465, Accuracy 0.5486, TPR@0.1%FPR of 0.0073 Attack Ours (offline, fixed variance) - AUC 0.5549, Accuracy 0.5537, TPR@0.1%FPR of 0.0299 + AUC 0.5518, Accuracy 0.5485, TPR@0.1%FPR of 0.0259 Attack Global threshold - AUC 0.5921, Accuracy 0.6044, TPR@0.1%FPR of 0.0009 + AUC 0.5900, Accuracy 0.6018, TPR@0.1%FPR of 0.0007 ``` where the global threshold attack is the baseline, and our online, diff --git a/research/mi_lira_2021/fprtpr.png b/research/mi_lira_2021/fprtpr.png index 8419ca1..5ad9dca 100644 Binary files a/research/mi_lira_2021/fprtpr.png and b/research/mi_lira_2021/fprtpr.png differ diff --git a/research/mi_lira_2021/inference.py b/research/mi_lira_2021/inference.py index 9d78d0b..fbe8170 100644 --- a/research/mi_lira_2021/inference.py +++ b/research/mi_lira_2021/inference.py @@ -69,7 +69,7 @@ def main(argv): for dy in range(0, 2*shift+1, stride): this_x = aug_pad[:, dx:dx+32, dy:dy+32, :].transpose((0,3,1,2)) - logits = model.model(this_x, training=True) + logits = model.model(this_x, training=False) outs.append(logits) print(np.array(outs).shape) diff --git a/research/mi_poison_2022/README.md b/research/mi_poison_2022/README.md index a20444d..5066046 100644 --- a/research/mi_poison_2022/README.md +++ b/research/mi_poison_2022/README.md @@ -88,13 +88,13 @@ which should give (something like) the following output ``` Attack No poison (LiRA) - AUC 0.7025, Accuracy 0.6258, TPR@0.1%FPR of 0.0544 + AUC 0.6992, Accuracy 0.6240, TPR@0.1%FPR of 0.0529 Attack No poison (Global threshold) - AUC 0.6191, Accuracy 0.6173, TPR@0.1%FPR of 0.0012 + AUC 0.6200, Accuracy 0.6167, TPR@0.1%FPR of 0.0011 Attack With poison (LiRA) - AUC 0.9943, Accuracy 0.9653, TPR@0.1%FPR of 0.4945 + AUC 0.9904, Accuracy 0.9617, TPR@0.1%FPR of 0.3730 Attack With poison (Global threshold) - AUC 0.9922, Accuracy 0.9603, TPR@0.1%FPR of 0.3930 + AUC 0.9911, Accuracy 0.9580, TPR@0.1%FPR of 0.2130 ``` where the baselines are LiRA and a simple global threshold on the membership diff --git a/research/mi_poison_2022/fprtpr.png b/research/mi_poison_2022/fprtpr.png index a870cb9..14fe1d0 100644 Binary files a/research/mi_poison_2022/fprtpr.png and b/research/mi_poison_2022/fprtpr.png differ