×

On the application of iterative methods of nondifferentiable optimization to some problems of approximation theory. (English) Zbl 1407.65022

Summary: We consider the data fitting problem, that is, the problem of approximating a function of several variables, given by tabulated data, and the corresponding problem for inconsistent (overdetermined) systems of linear algebraic equations. Such problems, connected with measurement of physical quantities, arise, for example, in physics, engineering, and so forth. A traditional approach for solving these two problems is the discrete least squares data fitting method, which is based on discrete \(\ell_2\)-norm. In this paper, an alternative approach is proposed: with each of these problems, we associate a nondifferentiable (nonsmooth) unconstrained minimization problem with an objective function, based on discrete \(\ell_1\)- and/or \(\ell_{\infty}\)-norm, respectively; that is, these two norms are used as proximity criteria. In other words, the problems under consideration are solved by minimizing the residual using these two norms. Respective subgradients are calculated, and a subgradient method is used for solving these two problems. The emphasis is on implementation of the proposed approach. Some computational results, obtained by an appropriate iterative method, are given at the end of the paper. These results are compared with the results, obtained by the iterative gradient method for the corresponding “differentiable” discrete least squares problems, that is, approximation problems based on discrete \(\ell_2\)-norm.

MSC:

65D10 Numerical smoothing, curve fitting
Full Text: DOI

References:

[1] Andersen, K. D., An efficient Newton barrier method for minimizing a sum of Euclidean norms, SIAM Journal on Optimization, 6, 1, 74-95 (1996) · Zbl 0842.90105 · doi:10.1137/0806006
[2] Barrodale, I.; Roberts, F., An improved algorithm for discrete \(l_1\) linear approximation, SIAM Journal on Numerical Analysis, 10, 5, 839-848 (1973) · Zbl 0266.65016 · doi:10.1137/0710069
[3] Barrodale, I.; Roberts, F., An efficient algorithm for discrete \(l_1\) linear approximation with constraints, SIAM Journal on Numerical Analysis, 15, 3, 603-611 (1978) · Zbl 0387.65027 · doi:10.1137/0715040
[4] Bartels, R. H.; Conn, A. R.; Sinclair, J. W., Minimization techniques for piecewise differentiable functions: the \(l_1\) solution to an overdetermined linear system, SIAM Journal on Numerical Analysis, 15, 2, 224-241 (1978) · Zbl 0376.65018 · doi:10.1137/0715015
[5] Bazaraa, M. S.; Sherali, H. D.; Shetty, C. M., Nonlinear Programming. Theory and Algorithms (1993), New York, NY, USA: John Wiley & Sons, New York, NY, USA · Zbl 0774.90075
[6] Bertsekas, D. P., A new class of incremental gradient methods for least squares problems, SIAM Journal on Optimization, 7, 4, 913-926 (1997) · Zbl 0887.49025 · doi:10.1137/S1052623495287022
[7] Björck, Å., Numerical Methods for Least Squares Problems (1996), Philadelphia, Pa, USA: Society for Industrial and Applied Mathematics (SIAM), Philadelphia, Pa, USA · Zbl 0734.65031 · doi:10.1137/1.9781611971484
[8] Calamai, P. H.; Conn, A. R., A stable algorithm for solving the multifacility location problem involving Euclidean distances, Society for Industrial and Applied Mathematics, 1, 4, 512-526 (1980) · Zbl 0469.65040 · doi:10.1137/0901037
[9] Calamai, P. H.; Conn, A. R., A projected Newton method for \(l_p\) norm location problems, Mathematical Programming, 38, 1, 75-109 (1987) · Zbl 0642.90035 · doi:10.1007/BF02591853
[10] Chatelon, J. A.; Hearn, D. W.; Lowe, T. J., A subgradient algorithm for certain minimax and minisum problems, Mathematical Programming, 15, 2, 130-145 (1978) · Zbl 0392.90065 · doi:10.1007/BF01609012
[11] Clarke, F., Optimization and Nonsmooth Analysis. Optimization and Nonsmooth Analysis, Classics in Applied Mathematics, 5 (1990), Philadelphia, Pa, USA: SIAM, Philadelphia, Pa, USA · Zbl 0696.49002 · doi:10.1137/1.9781611971309
[12] Coleman, T. F.; Li, Y., A global and quadratically convergent method for linear \(l_\infty\) problems, SIAM Journal on Numerical Analysis, 29, 4, 1166-1186 (1992) · Zbl 0756.65087 · doi:10.1137/0729071
[13] Coleman, T. F.; Li, Y., A globally and quadratically convergent affine scaling method for linear \(l_1\) problems, Mathematical Programming, 56, 1-3, 189-222 (1992) · Zbl 0760.90069 · doi:10.1007/BF01580899
[14] Demyanov, V. F.; Vasiliev, L. V., Nondifferentiable Optimization (1985), Berlin, Germany: Springer, Berlin, Germany · Zbl 0593.49001
[15] Fischer, J., An algorithm for discrete linear \(l_p\) approximation, Numerische Mathematik, 38, 1, 129-139 (1981) · Zbl 0463.65041 · doi:10.1007/BF01395812
[16] Kantorovich, L.; Akilov, G., Functional Analysis (1983), Moscow, Russia: Nauka, Moscow, Russia
[17] Korneichuk, N. P., Extremum Problems of Approximation Theory, Moscow, Russia: Nauka, Moscow, Russia · Zbl 0940.41008
[18] Lawson, C. L.; Hanson, R. J., Solving Least Squares Problems (1995), Philadelphia, Pa, USA: SIAM, Philadelphia, Pa, USA · Zbl 0860.65029 · doi:10.1137/1.9781611971217
[19] Li, Y., A globally convergent method for \(l_p\) problems, SIAM Journal on Optimization, 3, 3, 609-629 (1993) · Zbl 0780.65033 · doi:10.1137/0803030
[20] Mangasarian, O. L., Nonlinear Programming. Nonlinear Programming, Classics in Applied Mathematics, 10 (1994), Philadelphia, PA, USA: SIAM, Philadelphia, PA, USA · Zbl 0833.90108
[21] Merle, G.; Späth, H., Computational experiences with discrete \(l_p\)-approximation, Computing, 12, 4, 315-321 (1974) · Zbl 0282.65014 · doi:10.1007/BF02253335
[22] Overton, M. L., A quadratically convergent method for minimizing a sum of Euclidean norms, Mathematical Programming, 27, 1, 34-63 (1983) · Zbl 0536.65053 · doi:10.1007/BF02591963
[23] Remez, E., Fundamentals of Numerical Methods for Chebyshev Approximation, Kiev, Ukraine: Naukova Dumka, Kiev, Ukraine · Zbl 0233.65007
[24] Rockafellar, R. T., Convex Analysis (1997), Princeton, NJ, USA: Princeton University Press, Princeton, NJ, USA · Zbl 0932.90001
[25] Stefanov, S. M., Polynomial algorithms for projecting a point onto a region defined by a linear constraint and box constraints in \(R^n\), Journal of Applied Mathematics, 2004, 5, 409-431 (2004) · Zbl 1083.90041 · doi:10.1155/S1110757X04309071
[26] Stefanov, S. M., Well-posedness and primal-dual analysis of some convex separable optimization problems, Advances in Operations Research, 2013 (2013) · Zbl 1264.90142 · doi:10.1155/2013/279030
[27] Watson, G. A., On two methods for discrete \(l_p\) approximation, Computing, 18, 3, 263-266 (1977) · Zbl 0365.65008 · doi:10.1007/BF02253212
[28] Wolfe, J. M., On the convergence of an algorithm for discrete \(l_p\) approximation, Numerische Mathematik, 32, 4, 439-459 (1979) · Zbl 0427.65010 · doi:10.1007/BF01401047
[29] Xue, G.; Ye, Y., An efficient algorithm for minimizing a sum of Euclidean norms with applications, SIAM Journal on Optimization, 7, 4, 1017-1036 (1997) · Zbl 0885.68074 · doi:10.1137/S1052623495288362
[30] Yosida, K., Functional Analysis. Functional Analysis, Classics in Mathematics (1995), Berlin, Germany: Springer, Berlin, Germany · Zbl 0152.32102
[31] Yu, J.; Vishwanathan, S. V. N.; Günter, S.; Schraudolph, N. N., A quasi-Newton approach to nonsmooth convex optimization problems in machine learning, The Journal of Machine Learning Research, 11, 1145-1200 (2010) · Zbl 1242.90296
[32] Zhuravlev, Y. I.; Laptin, Y.; Vinogradov, A.; Zhurbenko, N.; Likhovid, A., Non smooth optimization methods in the problems of constructing a linear classifier, International Journal Information Models and Analyses, 1, 103-111 (2012)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.