Noise sensitivity and stability of deep neural networks for binary classification. (English) Zbl 1524.68318
Summary: A first step is taken towards understanding often observed non-robustness phenomena of deep neural net (DNN) classifiers. This is done from the perspective of Boolean functions by asking if certain sequences of Boolean functions represented by common DNN models are noise sensitive or noise stable, concepts defined in the Boolean function literature. Due to the natural randomness in DNN models, these concepts are extended to annealed and quenched versions. Here we sort out the relation between these definitions and investigate the properties of two standard DNN architectures, the fully connected and convolutional models, when initiated with Gaussian weights.
MSC:
| 68T07 | Artificial neural networks and deep learning |
| 06E30 | Boolean functions |
| 60C05 | Combinatorial probability |
| 60K35 | Interacting random processes; statistical mechanics type models; percolation theory |
| 62H30 | Classification and discrimination; cluster analysis (statistical aspects) |
Keywords:
Boolean functions; noise stability; noise sensitivity; deep neural networks; feed-forward neural networksReferences:
| [1] | Ahlberg, D., Partially observed Boolean sequences and noise sensitivity, Combin. Probab. Comput., 23, 3, 317-330 (2014) · Zbl 1326.60012 |
| [2] | Ahlberg, D.; Griffiths, S.; Morris, R.; Tassion, V., Quenched voronoi percolation, Adv. Math., 286, 889-911 (2016), URL https://www.sciencedirect.com/science/article/pii/S0001870815003436 · Zbl 1335.60178 |
| [3] | Ahlberg, D.; de la Riva, D.; Griffiths, S., On the rate of convergence in quenched voronoi percolation, Electron. J. Probab., 26, none, 1-26 (2021) · Zbl 1491.60168 |
| [4] | Benjamini, I.; Kalai, G.; Schramm, O., Noise sensitivity of boolean functions and applications to percolation, Publ. Math. Inst. Hautes Études Sci., 90, 1, 5-43 (1999) · Zbl 0986.60002 |
| [5] | Cybenko, G., Approximation by superpositions of a sigmoidal function, Math. Control Signals Systems, 2, 4, 303-314 (1989) · Zbl 0679.94019 |
| [6] | Garban, C.; Steif, J. E., Noise Sensitivity of Boolean Functions and Percolation, Vol. 5 (2014), Cambridge University Press |
| [7] | Goodfellow, I. J.; Shlens, J.; Szegedy, C., Explaining and harnessing adversarial examples (2014), arXiv preprint arXiv:1412.6572 |
| [8] | Gripon, V.; Hacene, G. B.; Löwe, M.; Vermet, F., Improving accuracy of nonparametric transfer learning via vector segmentation, (2018 IEEE International Conference on Acoustics, Speech and Signal Processing. 2018 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP (2018), IEEE), 2966-2970 |
| [9] | Koh, P. W.; Liang, P., Understanding black-box predictions via influence functions, (International Conference on Machine Learning (2017), PMLR), 1885-1894 |
| [10] | O’Donnell, R., Analysis of Boolean Functions (2014), Cambridge University Press · Zbl 1336.94096 |
| [11] | Peixoto, T. P.; Drossel, B., Noise in random Boolean networks, Phys. Rev. E, 79, 3, Article 036108 pp. (2009) |
| [12] | Peres, Y., Noise stability of weighted majority, (In and Out of Equilibrium 3: Celebrating Vladas Sidoravicius (2021), Springer), 677-682 · Zbl 1469.60341 |
| [13] | Vanneuville, H., Quantitative quenched voronoi percolation and applications (2018), arXiv preprint arXiv:1806.08448 |
| [14] | Xiao, Y.; Dougherty, E. R., The impact of function perturbations in Boolean networks, Bioinformatics, 23, 10, 1265-1273 (2007) |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
