×

Adaptive constraint reduction for convex quadratic programming. (English) Zbl 1245.90082

Summary: We propose an adaptive, constraint-reduced, primal-dual interior-point algorithm for convex quadratic programming with many more inequality constraints than variables. We reduce the computational effort by assembling, instead of the exact normal-equation matrix, an approximate matrix from a well chosen index set which includes indices of constraints that seem to be most critical. Starting with a large portion of the constraints, our proposed scheme excludes more unnecessary constraints at later iterations. We provide proofs for the global convergence and the quadratic local convergence rate of an affine-scaling variant. Numerical experiments on random problems, on a data-fitting problem, and on a problem in array pattern synthesis show the effectiveness of the constraint reduction in decreasing the time per iteration without significantly affecting the number of iterations. We note that a similar constraint-reduction approach can be applied to algorithms of Mehrotra’s predictor-corrector type, although no convergence theory is supplied.

MSC:

90C20 Quadratic programming
90C25 Convex programming
90C51 Interior-point methods
Full Text: DOI

References:

[1] Absil, P.-A., Tits, A.L.: Newton-KKT interior-point methods for indefinite quadratic programming. Comput. Optim. Appl. 36(1), 5–41 (2007) · Zbl 1278.90288 · doi:10.1007/s10589-006-8717-1
[2] Atkinson, K.E.: An Introduction to Numerical Analysis. Wiley, Washington (2005)
[3] Burges, C.: A tutorial on support vector machines for pattern recognition. Data Min. Knowl. Discov. 2(2), 121–167 (1998) · doi:10.1023/A:1009715923555
[4] Cortes, C., Vapnik, V.: Support-vector networks. Mach. Learn. 20(3), 273–297 (1995) · Zbl 0831.68098
[5] Dantzig, G.B.: Linear Programming and Extensions. Princeton University Press, Princeton (1963) · Zbl 0108.33103
[6] Dantzig, G.B., Ye, Y.: A build–up interior method for linear programming: Affine scaling form. Technical report, University of Iowa, Iowa City, IA 52242, USA (July 1991)
[7] Ferris, M.C., Munson, T.S.: Interior-point methods for massive support vector machines. SIAM J. Optim. 13(3), 783–804 (2002) · Zbl 1039.90092 · doi:10.1137/S1052623400374379
[8] Griva, I., Nash, S.G., Sofer, A.: Linear and Nonlinear Optimization, 2nd edn. SIAM, Philadelphia (2009) · Zbl 1159.90002
[9] Hertog, D., Roos, C., Terlaky, T.: A build-up variant of the path-following method for LP. Technical Report DUT-TWI-91-47, Delft University of Technology, Delft, The Netherlands (1991) · Zbl 0734.65050
[10] Hertog, D., Roos, C., Terlaky, T.: Adding and deleting constraints in the path–following method for linear programming. In: Advances in Optimization and Approximation (Nonconvex Optimization and Its Applications), vol. 1, pp. 166–185. Kluwer Academic, Dordrecht (1994) · Zbl 0828.90084
[11] Higham, N.J.: Analysis of the Cholesky decomposition of a semi-definite matrix. In: Cox, M.G., Hammarling, S.J. (eds.) Reliable Numerical Computation, Walton Street, Oxford OX2 6DP, UK, 1990, pp. 161–185. Oxford University Press, Oxford (1990)
[12] Householder, A.S.: The Theory of Matrices in Numerical Analysis. Blaisdell, New York (1964). Reprinted by Dover, New York (1975) · Zbl 0161.12101
[13] Jung, J.H.: Adaptive constraint reduction for convex quadratic programming and training support vector machines. PhD thesis, University of Maryland (2008). Available at http://hdl.handle.net/1903/8020
[14] Jung, J.H., O’Leary, D.P., Tits, A.L.: Adaptive constraint reduction for training support vector machines. Electron. Trans. Numer. Anal. 31, 156–177 (2008) · Zbl 1177.90308
[15] Karmarkar, N.: A new polynomial-time algorithm for linear programming. Combinatorica 4(4), 373–395 (1984) · Zbl 0557.90065 · doi:10.1007/BF02579150
[16] Luo, Z.-Q., Sun, J.: An analytic center based column generation algorithm for convex quadratic feasibility problems. SIAM J. Optim. 9(1), 217–235 (1998) · Zbl 1032.90526 · doi:10.1137/S1052623495294943
[17] Mehrotra, S.: On the implementation of a primal-dual interior point method. SIAM J. Optim. 2(4), 575–601 (1992) · Zbl 0773.90047 · doi:10.1137/0802028
[18] Nocedal, J., Wright, S.J.: Numerical Optimization. Springer, Berlin (2000) · Zbl 0930.65067
[19] Nordebo, S., Zang, Z., Claesson, I.: A semi-infinite quadratic programming algorithm with applications to array pattern synthesis. IEEE Trans. Circuits Syst. II, Analog Digit. Signal Process. 48(3), 225–232 (2001) · doi:10.1109/82.924065
[20] Saad, Y.: Iterative Methods for Sparse Linear Systems, 2nd edn. SIAM, Philadelphia (2003). Chap. 9 · Zbl 1031.65046
[21] Schölkopf, B., Smola, A.J.: Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. MIT Press, Cambridge (2001) · Zbl 1008.62507
[22] Tikhonov, A.N., Arsenin, V.Y.: Solutions of Ill-Posed Problems. Wiley, Washington (2005) · Zbl 0499.65030
[23] Tits, A.L., Zhou, J.L.: A simple, quadratically convergent interior point algorithm for linear programming and convex quadratic programming. In: Hager, W.W., Hearn, D.W., Pardalos, P.M. (eds.) Large Scale Optimization: State of the Art, pp. 411–427. Kluwer Academic, Dordrecht (1994) · Zbl 0811.90069
[24] Tits, A.L., Absil, P.-A., Woessner, W.P.: Constraint reduction for linear programs with many inequality constraints. SIAM J. Optim. 17(1), 119–146 (2006) · Zbl 1112.90049 · doi:10.1137/050633421
[25] Tone, K.: An active-set strategy in an interior point method for linear programming. Math. Program. 59(3), 345–360 (1993) · Zbl 0804.90093 · doi:10.1007/BF01581252
[26] Wang, W., O’Leary, D.P.: Adaptive use of iterative methods in predictor-corrector interior point methods for linear programming. Numer. Algorithms 25(1–4), 387–406 (2000) · Zbl 0977.65054 · doi:10.1023/A:1016614603137
[27] Watson, G.: Choice of norms for data fitting and function approximation. Acta Numer. 7, 337–377 (1998) · Zbl 0909.65003 · doi:10.1017/S0962492900002853
[28] Winternitz, L., Nicholls, S.O., Tits, A., O’Leary, D.: A constraint reduced variant of Mehrotra’s Predictor-Corrector Algorithm, 2007. Submitted for publication. http://www.optimization-online.org/DB_FILE/2007/07/1734.pdf · Zbl 1245.90056
[29] Wright, S.J.: Primal-Dual Interior-Point Methods. SIAM, Philadelphia (1997) · Zbl 0863.65031
[30] Ye, Y.: A ”build-down” scheme for linear programming. Math. Program. 46(1), 61–72 (1990) · Zbl 0698.90054 · doi:10.1007/BF01585727
[31] Ye, Y.: An $O(n\^{\(\backslash\)mbox{3}}L)$ potential reduction algorithm for linear programming. Math. Program. 50, 239–258 (1991) · Zbl 0734.90057 · doi:10.1007/BF01594937
[32] Ye, Y.: A potential reduction algorithm allowing column generation. SIAM J. Optim. 2(1), 7–20 (1992) · Zbl 0767.90049 · doi:10.1137/0802002
[33] Zhang, Y.: Solving large–scale linear programs by interior–point methods under the MATLAB environment. Technical Report 96–01, Dept. of Mathematics and Statistics, Univ. of Maryland Baltimore County (1996)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.