Perspective functions: properties, constructions, and examples. (English) Zbl 1396.26021
Summary: Many functions encountered in applied mathematics and in statistical data analysis can be expressed in terms of perspective functions. One of the earliest examples is the Fisher information, which appeared in statistics in the 1920s. We analyze various algebraic and convex-analytical properties of perspective functions and provide general schemes to construct lower semicontinuous convex functions from them. Several new examples are presented and existing instances are featured as special cases.
MSC:
| 26B25 | Convexity of real functions of several variables, generalizations |
| 26B35 | Special properties of functions of several variables, Hölder conditions, etc. |
| 90C25 | Convex programming |
Keywords:
Berhu function; convex function; convex optimization; Huber function; integral functional; perspective function; statistical divergenceReferences:
| [1] | Akian, M., Gaubert, S., Hochart, A.: Minimax representation of nonexpansive functions and application to zero-sum recursive games. J. Convex Anal., to appear · Zbl 1386.49009 |
| [2] | Ali, SM; Silvey, SD, A general class of coefficients of divergence of one distribution from another, J. Roy. Statist. Soc., B28, 131-142, (1966) · Zbl 0203.19902 |
| [3] | Alibert, JJ; Bouchitté, G; Fragalà, I; Lucardesi, I, A nonstandard free boundary problem arising in the shape optimization of thin torsion rods, Interfaces Free Bound., 15, 95-119, (2013) · Zbl 1270.49044 · doi:10.4171/IFB/296 |
| [4] | Basseville, M, Distance measures for signal processing and pattern recognition, Signal Process., 18, 349-369, (1989) · doi:10.1016/0165-1684(89)90079-0 |
| [5] | Bauschke, HH; Borwein, JM; Combettes, PL, Essential smoothness, essential strict convexity, and Legendre functions in Banach spaces, Comm. Contemp. Math., 3, 615-647, (2001) · Zbl 1032.49025 · doi:10.1142/S0219199701000524 |
| [6] | Bauschke, H.H., Combettes, P.L.: Convex Analysis and Monotone Operator Theory in Hilbert Spaces. Springer, New York (2011) · Zbl 1218.47001 · doi:10.1007/978-1-4419-9467-7 |
| [7] | Ben-Tal, A; Ben-Israel, A; Teboulle, M, Certainty equivalents and information measures: duality and extremal principles, J. Math. Anal. Appl., 157, 211-236, (1991) · Zbl 0736.94004 · doi:10.1016/0022-247X(91)90145-P |
| [8] | Benamou, J-D; Brenier, Y, A computational fluid mechanics solution to the Monge-Kantorovich mass transfer problem, Numer. Math., 84, 375-393, (2000) · Zbl 0968.76069 · doi:10.1007/s002110050002 |
| [9] | Bercher, J-F, Some properties of generalized Fisher information in the context of nonextensive thermostatistics, Physica A, 392, 3140-3154, (2013) · Zbl 1395.94206 · doi:10.1016/j.physa.2013.03.062 |
| [10] | Berlinet, A; Vajda, I, Selection rules based on divergences, Statistics, 45, 479-495, (2011) · Zbl 1229.62035 · doi:10.1080/02331880903573385 |
| [11] | Bien, J., Gaynanova, I., Lederer, J., Müller, C.L.: Non-convex global minimization and false discovery rate control for the TREX. J. Comp. Graph. Stat., to appear · Zbl 1295.90047 |
| [12] | Boekee, DE; Csiszár, I (ed.); Elias, P (ed.), An extension of the Fisher information measure, 113-123, (1977), Keszthely · Zbl 0398.94010 |
| [13] | Borwein, JM; Lewis, AS; Limber, MN; Noll, D, Maximum entropy reconstruction using derivative information, part 2: computational results, Numer. Math., 69, 243-256, (1995) · Zbl 0840.65147 · doi:10.1007/s002110050090 |
| [14] | Borwein, JM; Lewis, AS; Noll, D, Maximum entropy reconstruction using derivative information, part 1: Fisher information and convex duality, Math. Oper. Res., 21, 442-468, (1996) · Zbl 0884.90121 · doi:10.1287/moor.21.2.442 |
| [15] | Bouchitté, G; Fragalà, I; Lucardesi, I; Seppecher, P, Optimal thin torsion rods and Cheeger sets, SIAM J. Math. Anal., 44, 483-512, (2012) · Zbl 1403.74069 · doi:10.1137/110828538 |
| [16] | Bougeard, M.L.: Connection between some statistical estimation criteria, lower-C2 functions and Moreau-Yosida approximates. In: Bulletin International Statistical Institute, 47th session, contributed papers, vol. 1, 159-160 (1989) |
| [17] | Bougeard, ML; Caquineau, CD, Parallel proximal decomposition algorithms for robust estimation, Ann. Oper. Res., 90, 247-270, (1999) · Zbl 1045.62511 · doi:10.1023/A:1018916816192 |
| [18] | Brasco, L; Buttazzo, G; Santambrogio, F, A benamou-brenier approach to branched transport, SIAM J. Math. Anal., 43, 1023-1040, (2011) · Zbl 1223.49047 · doi:10.1137/10079286X |
| [19] | Briceño-Arias, L.M., Kalise, D., Silva, F.J.: Proximal methods for stationary mean field games with local couplings, arXiv:1608.07701v1.pdf(2016) · Zbl 1410.91090 |
| [20] | Ceria, S; Soares, J, Convex programming for disjunctive convex optimization, Math. Program., A86, 595-614, (1999) · Zbl 0954.90049 · doi:10.1007/s101070050106 |
| [21] | Chaux, C; Combettes, PL; Pesquet, J-C; Wajs, VR, A variational formulation for frame-based inverse problems, Inverse Problems, 23, 1495-1518, (2007) · Zbl 1141.65366 · doi:10.1088/0266-5611/23/4/008 |
| [22] | Chen, J-S, The semismooth-related properties of a merit function and a descent method for the nonlinear complementarity problem, J. Global Optim., 36, 565-580, (2006) · Zbl 1144.90493 · doi:10.1007/s10898-006-9027-y |
| [23] | Choquet, G.: Topologie. Masson, Paris (1964). (English translation: Topology. Academic Press, New York, 1966) · Zbl 0116.31401 |
| [24] | Cohen, ML, The Fisher information and convexity, IEEE Trans. Inform. Theory, 14, 591-592, (1968) · doi:10.1109/TIT.1968.1054175 |
| [25] | Combettes, P.L., Müller, C. L.: Perspective functions: Proximal calculus and applications in high-dimensional statistics. J. Math. Anal. Appl., published online 2016-12-15 |
| [26] | Csiszár, I, Information-type measures of difference of probability distributions and indirect observations, Studia Sci. Math. Hungar., 2, 299-318, (1967) · Zbl 0157.25802 |
| [27] | Dacorogna, B; Maréchal, P, The role of perspective functions in convexity, polyconvexity, rank-one convexity and separate convexity, J. Convex Anal., 15, 271-284, (2008) · Zbl 1152.26012 |
| [28] | Elad, M; Matalon, B; Zibulevsky, M, Coordinate and subspace optimization methods for linear least squares with non-quadratic regularization, Appl. Comput. Harmon. Anal., 23, 346-367, (2007) · Zbl 1133.65022 · doi:10.1016/j.acha.2007.02.002 |
| [29] | Fisher, RA, Theory of statistical estimation, Proc. Cambridge. Philos. Soc., 22, 700-725, (1925) · JFM 51.0385.01 · doi:10.1017/S0305004100009580 |
| [30] | Fitschen, JH; Laus, F; Steidl, G, Transport between RGB images motivated by dynamic optimal transport, J. Math. Imaging Vis., 56, 409-429, (2016) · Zbl 1353.94009 · doi:10.1007/s10851-016-0644-x |
| [31] | Frieden, B.R., Gatenby, R.A. (eds.): Exploratory Data Analysis Using Fisher Information. Springer, New York (2007) · Zbl 1222.62004 |
| [32] | Hartley, R.I., Zisserman, A.: Multiple view geometry in computer vision, 2nd ed. Cambridge University Press (2003) · Zbl 0956.68149 |
| [33] | Hijazi, H; Bonami, P; Cornuéjols, G; Ouorou, A, Mixed-integer nonlinear programs featuring “on/off” constraints, Comput. Optim. Appl., 52, 537-558, (2012) · Zbl 1250.90058 · doi:10.1007/s10589-011-9424-0 |
| [34] | Hiriart-Urruty, J.-B., Lemaréchal, C.: Convex Analysis and Minimization Algorithms. Springer, New York (1993) · Zbl 0795.49001 |
| [35] | Hiriart-Urruty, J-B; Martínez-Legaz, J-E, Convex solutions of a functional equation arising in information theory, J. Math. Anal Appl., 328, 1309-1320, (2007) · Zbl 1154.39025 · doi:10.1016/j.jmaa.2006.06.035 |
| [36] | Huber, PJ, Robust estimation of a location parameter, Ann. Stat., 35, 73-101, (1964) · Zbl 0136.39805 · doi:10.1214/aoms/1177703732 |
| [37] | Huber, P.J., Ronchetti, E.M.: Robust Statistics, 2nd ed. Wiley, New York (2009) · Zbl 1276.62022 · doi:10.1002/9780470434697 |
| [38] | Jung, M.N., Kirches, C., Sager, S.: On perspective functions and vanishing constraints in mixed-integer nonlinear optimal control. In: Facets of Combinatorial Optimization, pp 387-417. Springer, Heidelberg (2013) · Zbl 1320.49011 |
| [39] | Lambert-Lacroix, S; Zwald, L, Robust regression through the huber’s criterion and adaptive lasso penalty, Electron. J. Stat., 5, 1015-1053, (2011) · Zbl 1274.62467 · doi:10.1214/11-EJS635 |
| [40] | Lambert-Lacroix, S; Zwald, L, The adaptive berhu penalty in robust regression, J. Nonparametr. Stat., 28, 487-514, (2016) · Zbl 1348.62197 · doi:10.1080/10485252.2016.1190359 |
| [41] | Laurent, P.-J.: Approximation et Optimisation. Hermann, Paris (1972) · Zbl 0238.90058 |
| [42] | Lederer, J., Mu̇ller, C. L.: Don’t fall for tuning parameters: Tuning-free variable selection in high dimensions with the TREX, Proc. Twenty-Ninth AAAI Conf. Artif. Intell., pp. 2729-2735. AAAI Press, Austin (2015) |
| [43] | Lemaréchal, C.: Personnal communication |
| [44] | Liese, F; Vajda, I, On divergences and informations in statistics and information theory, IEEE Trans. Inform. Theory, 52, 4394-4412, (2006) · Zbl 1287.94025 · doi:10.1109/TIT.2006.881731 |
| [45] | Lions, P-L; Toscani, G, A strengthened central limit theorem for smooth densities, J. Funct. Anal., 129, 148-167, (1995) · Zbl 0822.60018 · doi:10.1006/jfan.1995.1046 |
| [46] | Micchelli, CA; Morales, JM; Pontil, M, Regularizers for structured sparsity, Adv. Comput. Math., 38, 455-489, (2013) · Zbl 1297.62153 · doi:10.1007/s10444-011-9245-9 |
| [47] | Moehle, N; Boyd, S, A perspective-based convex relaxation for switched-affine optimal control, Systems Control Lett., 86, 34-40, (2015) · Zbl 1325.93026 · doi:10.1016/j.sysconle.2015.09.002 |
| [48] | Moreau, JJ, Fonctions convexes duales et points proximaux dans un espace hilbertien, C. R. Acad. Sci. Paris Sér. A Math., 255, 2897-2899, (1962) · Zbl 0118.10502 |
| [49] | Ndiaye, E., Fercoq, O., Gramfort, A., Leclère, V., Salmon, J.: Efficient smoothed concomitant lasso estimation for high dimensional regression, arXiv:1606.02702v1.pdf (2016) · Zbl 0840.65147 |
| [50] | Nesterov, Yu., Nemirovskii, A.: Interior-Point Polynomial Algorithms in Convex Programming. SIAM Philadelphia (1994) · Zbl 0824.90112 |
| [51] | Nikolova, M; Ng, MK, Analysis of half-quadratic minimization methods for signal and image recovery, SIAM J. Sci. Comput., 27, 937-966, (2005) · Zbl 1141.49318 · doi:10.1137/030600862 |
| [52] | Noll, D, Reconstruction with noisy data: an approach via eigenvalue optimization, SIAM J. Optim., 8, 82-104, (1998) · Zbl 0917.49032 · doi:10.1137/S105262349629856X |
| [53] | Owen, AB, A robust hybrid of lasso and ridge regression, Contemp. Math., 443, 59-71, (2007) · Zbl 1134.62047 · doi:10.1090/conm/443/08555 |
| [54] | Papadakis, N; Peyré, G; Oudet, E, Optimal transport with proximal splitting, SIAM J. Imaging Sci., 7, 212-238, (2014) · Zbl 1295.90047 · doi:10.1137/130920058 |
| [55] | Pardo, L.: Statistical Inference Based on Divergence Measures. Chapman and Hall/CRC, Boca Raton (2006) · Zbl 1118.62008 |
| [56] | Rey, W.J.J.: Introduction to Robust and Quasi-Robust Statistical Methods. Springer, Berlin (1983) · Zbl 0525.62040 · doi:10.1007/978-3-642-69389-2 |
| [57] | Rockafellar, R.T.: Convex Analysis. Princeton University Press, Princeton (1970) · Zbl 0193.18401 · doi:10.1515/9781400873173 |
| [58] | Toscani, G, A strengthened entropy power inequality for log-concave densities, IEEE Trans. Inform. Theory, 61, 6550-6559, (2015) · Zbl 1359.94371 · doi:10.1109/TIT.2015.2495302 |
| [59] | Vapnik, V.N.: The Nature of Statistical Learning Theory, 2nd ed. Springer, New York (2000) · Zbl 0934.62009 · doi:10.1007/978-1-4757-3264-1 |
| [60] | Villani, C, Fisher information estimates for boltzmann’s collision operator, J. Math. Pures. Appl., 77, 821-837, (1998) · Zbl 0918.60093 · doi:10.1016/S0021-7824(98)80010-X |
| [61] | Zach, C; Pollefeys, M, Practical methods for convex multi-view reconstruction, Lecture Notes Comput. Sci., 6314, 354-367, (2010) · doi:10.1007/978-3-642-15561-1_26 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
