Acknowledgement
Hi Jun Choe and Hayeong Koh were supported by Basic Science Research Program through the National Research Foundation of Korea(NRF) funded by the Korea government (No. 2015R1A5A1009350, No. 20181A2A3074566). Jimin Lee was supported by Basic Science Research Program through the National Research Foundation of Korea(NRF) funded by the Korea government (No. 2020R1I1A1A01071731).
References
- S. Arora, R. Ge, B. Neyshabur, and Y. Zhang, Stronger generalization bounds for deep nets via a compression approach, arXiv preprint arXiv:1802.05296, 2018.
- P. L. Bartlett, D. J. Foster, and M. J. Telgarsky, Spectrally-normalized margin bounds for neural networks, In Advances in Neural Information Processing Systems 30, pages 6240-6249, 2017.
- G. Elsayed, D. Krishnan, H. Mobahi, K. Regan, and S. Bengio, Large margin deep networks for classification, In Advances in neural information processing systems, pages 842-852, 2018.
- D. Haussler, Probably approximately correct learning, University of California, Santa Cruz, Computer Research Laboratory, 1990.
- Y. Jiang, D. Krishnan, H. Mobahi, and S. Bengio, Predicting the generalization gap in deep networks with margin distributions, arXiv preprint arXiv:1810.00113, 2018.
- J. Langford and J. Shawe-Taylor, Pac-bayes & margins, In Advances in neural information processing systems, pages 439-446, 2003.
- D. A. McAllester, PAC-Bayesian model averaging, in Proceedings of the Twelfth Annual Conference on Computational Learning Theory (Santa Cruz, CA, 1999), 164-170, ACM, New York, 1999. https://doi.org/10.1145/307400.307435
- D. A. McAllester, Pac-bayesian stochastic model selection, Machine Learning 51 (2003), no. 1, 5-21. https://doi.org/10.1023/A:1021840411064
- D. McAllester, Simplified pac-bayesian margin bounds, In Learning theory and Kernel machines, pages 203-215. Springer, 2003.
- B. Neyshabur, S. Bhojanapalli, and N. Srebro, A pac-bayesian approach to spectrally-normalized margin bounds for neural networks, International Conference on Learning Representations, 2018.
- L. Schmidt, S. Santurkar, D. Tsipras, K. Talwar, and A. Madry, Adversarially robust generalization requires more data, In Advances in Neural Information Processing Systems, pages 5014-5026, 2018.
- L. Shen-Huan, W. Lu, and Z. Zhi-Hua, Optimal margin distribution network, CoRR, abs/1812.10761, 2018.
- C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. Goodfellow, and R. Fergus, Intriguing properties of neural networks, arXiv preprint arXiv:1312.6199, 2013.
- J. A. Tropp, User-friendly tail bounds for sums of random matrices, Found. Comput. Math. 12 (2012), no. 4, 389-434. https://doi.org/10.1007/s10208-011-9099-z
- D. Yin, K. Ramchandran, and P. Bartlett, Rademacher complexity for adversarially robust generalization, International Conference on Machine Learning, 2019.
- C. Zhang, S. Bengio, M. Hardt, B. Recht, and O. Vinyals, Understanding deep learning requires rethinking generalization, arXiv preprint arXiv:1611.03530, 2016.