×

Verifying Pufferfish privacy in hidden Markov models. (English) Zbl 1498.68104

Finkbeiner, Bernd (ed.) et al., Verification, model checking, and abstract interpretation. 23rd international conference, VMCAI 2022, Philadelphia, PA, USA, January 16–18, 2022. Proceedings. Cham: Springer. Lect. Notes Comput. Sci. 13182, 174-196 (2022).
Summary: Pufferfish is a Bayesian privacy framework for designing and analyzing privacy mechanisms. It refines differential privacy, the current gold standard in data privacy, by allowing explicit prior knowledge in privacy analysis. In practice, privacy mechanisms often need be modified or adjusted to specific applications. Their privacy risks have to be re-evaluated for different circumstances. Privacy proofs can thus be complicated and prone to errors. Such tedious tasks are burdensome to average data curators. In this paper, we propose an automatic verification technique for Pufferfish privacy. We use hidden Markov models to specify and analyze discrete mechanisms in Pufferfish privacy. We show that the Pufferfish verification problem in hidden Markov models is NP-hard. Using Satisfiability Modulo Theories solvers, we propose an algorithm to verify privacy requirements. We implement our algorithm in a prototypical tool called FAIER, and analyze several classic privacy mechanisms in Pufferfish privacy. Surprisingly, our analysis show that naïve discretization of well-established privacy mechanisms often fails, witnessed by counterexamples generated by FAIER. In discrete Above Threshold, we show that it results in absolutely no privacy. Finally, we compare our approach with state-of-the-art tools for differential privacy, and show that our verification technique can be efficiently combined with these tools for the purpose of certifying counterexamples and finding a more precise lower bound for the privacy budget \(\epsilon \).
For the entire collection see [Zbl 1490.68015].

MSC:

68P27 Privacy of data

References:

[1] Albarghouthi, A., Hsu, J.: Synthesizing coupling proofs of differential privacy. Proc. ACM Program. Lang. 2(POPL), 1-30 (2017)
[2] Apple: About privacy and location services in IOS and IPADOS (2020). https://support.apple.com/en-us/HT203033/. Accessed 9 Sept 2021
[3] Barthe, G., Chadha, R., Jagannath, V., Sistla, A., Viswanathan, M.: Deciding differential privacy for programs with finite inputs and outputs, pp. 141-154 (2020). doi:10.1145/3373718.3394796 · Zbl 1507.68111
[4] Barthe, G., Chadha, R., Jagannath, V., Sistla, A.P., Viswanathan, M.: Automated methods for checking differential privacy. CoRR abs/1910.04137 (2019). http://arxiv.org/abs/1910.04137 · Zbl 1507.68111
[5] Barthe, G., Gaboardi, M., Grégoire, B., Hsu, J., Strub, P.Y.: Proving differential privacy via probabilistic couplings (2016) · Zbl 1401.68184
[6] Barthe, G.; Gaboardi, M.; Hsu, J.; Pierce, B., Programming language techniques for differential privacy, ACM SIGLOG News, 3, 1, 34-53 (2016) · doi:10.1145/2893582.2893591
[7] Barthe, G., Köpf, B., Olmedo, F., Zanella Béguelin, S.: Probabilistic relational reasoning for differential privacy. In: POPL ’12, pp. 97-110 (2012). doi:10.1145/2103656.2103670 · Zbl 1321.68182
[8] Barthe, G.; Köpf, B.; Olmedo, F.; Zanella Béguelin, S., Probabilistic relational reasoning for differential privacy, SIGPLAN Not., 47, 1, 97-110 (2012) · Zbl 1321.68182 · doi:10.1145/2103621.2103670
[9] Bichsel, B., Gehr, T., Drachsler-Cohen, D., Tsankov, P., Vechev, M.: Dp-finder: finding differential privacy violations by sampling and optimization, pp. 508-524 (2018). doi:10.1145/3243734.3243863
[10] Bichsel, B., Steffen, S., Bogunovic, I., Vechev, M.: DP-sniper: black-box discovery of differential privacy violations using classifiers. In: SP’21, pp. 391-409 (2021). doi:10.1109/SP40001.2021.00081
[11] Chen, Y., Machanavajjhala, A.: On the privacy properties of variants on the sparse vector technique. CoRR abs/1508.07306 (2015). http://arxiv.org/abs/1508.07306
[12] Chistikov, D., Kiefer, S., Murawski, A.S., Purser, D.: The Big-O problem for labelled markov chains and weighted automata. In: CONCUR 2020. Leibniz International Proceedings in Informatics (LIPIcs), vol. 171, pp. 41:1-41:19 (2020). doi:10.4230/LIPIcs.CONCUR.2020.41 · Zbl 07559497
[13] Ding, B., Kulkarni, J., Yekhanin, S.: Collecting telemetry data privately. In: NIPS’17, pp. 3574-3583 (2017)
[14] Ding, Z., Wang, Y., Wang, G., Zhang, D., Kifer, D.: Detecting violations of differential privacy. In: Backes, M., Wang, X. (eds.) CCS, pp. 475-489 (2018)
[15] Dolzmann, A.; Sturm, T., REDLOG: computer algebra meets computer logic, SIGSAM Bull., 31, 2, 2-9 (1997) · doi:10.1145/261320.261324
[16] Dwork, C.; Roth, A., The algorithmic foundations of differential privacy, Found. Trends Theoret. Comput. Sci., 9, 3-4, 211-407 (2014) · Zbl 1302.68109
[17] Dwork, C.; Bugliesi, M.; Preneel, B.; Sassone, V.; Wegener, I., Differential privacy, Automata, Languages and Programming, 1-12 (2006), Heidelberg: Springer, Heidelberg · Zbl 1133.68330 · doi:10.1007/11787006_1
[18] Farina, G.P., Chong, S., Gaboardi, M.: Coupled relational symbolic execution for differential privacy. In: Programming Languages and Systems, pp. 207-233 (2021) · Zbl 1473.68072
[19] Gaboardi, M., Nissim, K., Purser, D.: The complexity of verifying loop-free programs as differentially private. In: ICALP 2020. Leibniz International Proceedings in Informatics (LIPIcs), vol. 168, pp. 129:1-129:17 (2020). doi:10.4230/LIPIcs.ICALP.2020.129
[20] Ghosh, A., Roughgarden, T., Sundararajan, M.: Universally utility-maximizing privacy mechanisms. In: STOC, pp. 351-360. ACM, New York (2009) · Zbl 1304.94060
[21] Ghosh, A.; Roughgarden, T.; Sundararajan, M., Universally utility-maximizing privacy mechanisms, SIAM J. Comput., 41, 6, 1673-1693 (2012) · Zbl 1271.68102 · doi:10.1137/09076828X
[22] Kifer, D., Machanavajjhala, A.: No free lunch in data privacy. In: SIGMOD, pp. 193-204 (2011)
[23] Kifer, D., Machanavajjhala, A.: Pufferfish: a framework for mathematical privacy definitions. ACM Trans. Database Syst. 39(1), 3:1-3:36 (2014) · Zbl 1321.94067
[24] Liu, D.; Wang, B-Y; Zhang, L.; Ryu, S., Model checking differentially private properties, Programming Languages and Systems, 394-414 (2018), Cham: Springer, Cham · Zbl 1519.68137 · doi:10.1007/978-3-030-02768-1_21
[25] Liu, D., Wang, B., Zhang, L.: Verifying pufferfish privacy in hidden Markov models. CoRR abs/2008.01704 (2020). https://arxiv.org/abs/2008.01704
[26] Lyu, M., Su, D., Li, N.: Understanding the sparse vector technique for differential privacy. Proc. VLDB Endow. 10(6), 637-648 (2017). doi:10.14778/3055330.3055331
[27] McIver, A.; Morgan, C.; Lin, AW, Proving that programs are differentially private, Programming Languages and Systems, 3-18 (2019), Cham: Springer, Cham · Zbl 1542.68035 · doi:10.1007/978-3-030-34175-6_1
[28] Mironov, I.: On significance of the least significant bits for differential privacy. In: CCS’12, pp. 650-661 (2012)
[29] de Moura, L.; Bjørner, N.; Ramakrishnan, CR; Rehof, J., Z3: an efficient SMT solver, Tools and Algorithms for the Construction and Analysis of Systems, 337-340 (2008), Heidelberg: Springer, Heidelberg · doi:10.1007/978-3-540-78800-3_24
[30] Papadimitriou, CH; Tsitsiklis, JN, The complexity of Markov decision processes, Math. Oper. Res., 12, 3, 441-450 (1987) · Zbl 0638.90099 · doi:10.1287/moor.12.3.441
[31] Rabiner, LR, A tutorial on hidden Markov models and selected applications in speech recognition, Proc. IEEE, 77, 2, 257-286 (1989) · doi:10.1109/5.18626
[32] Wang, Y., Ding, Z., Kifer, D., Zhang, D.: CheckDP: an automated and integrated approach for proving differential privacy or finding precise counterexamples. In: CCS ’20, pp. 919-938 (2020). doi:10.1145/3372297.3417282
[33] Wang, Y., Ding, Z., Wang, G., Kifer, D., Zhang, D.: Proving differential privacy with shadow execution. In: PLDI’19, pp. 655-669 (2019)
[34] Zhang, D., Kifer, D.: LightDP: towards automating differential privacy proofs. In: POPL’17, vol. 52, pp. 888-901 (2017) · Zbl 1380.68136
[35] Zhang, H., Roth, E., Haeberlen, A., Pierce, B.C., Roth, A.: Testing differential privacy with dual interpreters. Proc. ACM Program. Lang. 4(OOPSLA) (2020). doi:10.1145/3428233
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.