×

Optimal designs in sparse linear models. (English) Zbl 1441.62198

From the authors’ abstract: “The Lasso approach is widely adopted for screening and estimating active effects in sparse linear models with quantitative factors. Many design schemes have been proposed based on different criteria to make the Lasso estimator more accurate. This article applies \(\Phi_l\)-optimality to the asymptotic covariance matrix of the Lasso estimator.”
The \(\Phi_l\) optimality considered in this paper is defined through the eigenvalues of the asymptotic covariance matrix \(M\) of the Lasso estimator. Thus, \(\Phi_l(M)\) is defined as \(\Phi_l (M) = ({\mathrm{tr}} M^l )^{1/l}\). When the parameter \(l\) ranges over \([0, +\infty]\), one obtains different notions of optimality, namely: \begin{align*} \Phi_0 (M) = \det(M) \qquad & \text{\(D\)-optimality} \\ \Phi_1 (M) = \operatorname{tr}(M) \qquad &\text{\(A\)-optimality} \\ \Phi_\infty(M) = \text{max eigenvalue of \(M\)} \qquad & \text{\(E\)-optimality}. \end{align*} A simulation study with several scenarios and an example based on real-world data are described in order to show the performance of the estimation procedure under the proposed design scheme.

MSC:

62K05 Optimal statistical designs
62J07 Ridge regression; shrinkage estimators (Lasso)

Software:

glmnet; glasso
Full Text: DOI

References:

[1] Belloni, A.; Wang, L., Square-root lasso: pivotal recovery of sparse signals via conic programming, Biometrika, 98, 4, 791-806 (2010) · Zbl 1228.62083 · doi:10.1093/biomet/asr043
[2] Cai, T.; Liu, W.; Luo, X., A constrained \(l_1\)-minimization approach to sparse precision matrix estimation, J Am Stat Assoc, 106, 494, 594-607 (2011) · Zbl 1232.62087 · doi:10.1198/jasa.2011.tm10155
[3] Chow, S.; Yang, T.; Zhou, H.; Adamatzky, A.; Chen, G., Global optimizations by intermittent diffusion, Chaos, CNN, memristors and beyond, 466-479 (2013), Singapore: World Scientific Press, Singapore
[4] Cook, Rd; Nachtsheim, Cj, A comparison of algorithms for constructing exact d-optimal designs, Technometrics, 22, 3, 315-324 (1980) · Zbl 0459.62061 · doi:10.1080/00401706.1980.10486162
[5] Deng X, Lin CD, Qian PZG (2013) The lasso with nearly orthogonal Latin hypercube designs. https://uq.wisc.edu/papers/Lasso_Design.pdf
[6] Dette, H.; Melas, Vb; Guchenko, R., Bayesian t-optimal discriminating designs, Annal Stat, 43, 5, 1959-1985 (2014) · Zbl 1331.62382 · doi:10.1214/15-AOS1333
[7] Dette, H.; Titoff, S., Optimal discrimination designs, Annal Stat, 37, 4, 2056-2082 (2009) · Zbl 1168.62066 · doi:10.1214/08-AOS635
[8] Friedman, J.; Hastie, T.; Tibshirani, R., Sparse inverse covariance estimation with the graphical lasso, Biostatistics, 9, 3, 432-441 (2008) · Zbl 1143.62076 · doi:10.1093/biostatistics/kxm045
[9] Friedman, J.; Hastie, T.; Tibshirani, R., Regularization paths for generalized linear models via coordinate descent, J Stat Softw, 33, 1, 1-22 (2010) · doi:10.18637/jss.v033.i01
[10] Gilmour, Sg; Trinca, La, Optimum design of experiments for statistical inference, J Royal Stat Soc Ser C, 61, 3, 345-401 (2012) · doi:10.1111/j.1467-9876.2011.01000.x
[11] Javanmard, A.; Montanari, A., Confidence intervals and hypothesis testing for high-dimensional regression, J Mach Learn Res, 15, 2869-2909 (2014) · Zbl 1319.62145
[12] Jones, B.; Lin, Dkj; Nachtsheim, Cj, Bayesian d-optimal supersaturated designs, J Stat Plan Inference, 138, 1, 86-92 (2008) · Zbl 1144.62058 · doi:10.1016/j.jspi.2007.05.021
[13] Kaymal T (2013) Assessing the operational effectiveness of a small surface combat ship in an anti-surfacewarfare environment. Masters thesis, Naval Postgraduate School, Monterey, California. https://calhoun.nps.edu/handle/10945/34685
[14] Kiefer, Jc, General equivalence theory for optimum designs, Annal Stat, 2, 5, 849-879 (1974) · Zbl 0291.62093 · doi:10.1214/aos/1176342810
[15] Li, Ww; Wu, Cfj, Columnwise-pairwise algorithms with applications to the construction of supersaturated designs, Technometrics, 39, 2, 171-179 (1997) · Zbl 0889.62066 · doi:10.1080/00401706.1997.10485082
[16] Nguyen, Nk, An algorithmic approach to constructing supersaturated designs, Technometrics, 38, 1, 69-73 (1996) · Zbl 0900.62416 · doi:10.1080/00401706.1996.10484417
[17] Pukelsheim, Fj, Optimal design of experiments (1993), New York: Wiley, New York · Zbl 0834.62068
[18] Ravi SN, Ithapu VK, Johnson SC, Singh V (2016) Experimental design on a budget for sparse linear models and applications. In: Proceedings of the 33rd international conference on machine learning 48, 583-592
[19] Satterthwaite, Fe, Random balance experimentation, Technometrics, 1, 2, 111-137 (1959) · doi:10.1080/00401706.1959.10489853
[20] Silvey, Sd, Optimal design (1980), New York: Springer, New York · Zbl 0468.62070
[21] Sun, T.; Zhang, Ch, Scaled sparse linear regression, Biometrika, 99, 4, 879-898 (2012) · Zbl 1452.62515 · doi:10.1093/biomet/ass043
[22] Tibshirani, R., Regression shrinkage and selection via the lasso, J Royal Stat Soc Ser B, 58, 1, 267-288 (1996) · Zbl 0850.62538
[23] Van De Geer, S.; Bühlmann, P.; Ritov, Y.; Dezeure, R., On asymptotically optimal confidence regions and tests for high-dimensional models, Annal Stat, 42, 3, 1166-1202 (2014) · Zbl 1305.62259 · doi:10.1214/14-AOS1221
[24] Wu, Cfj; Hamada, M., Experiments: planning, analysis and parameter design optimization (2009), New York: Wiley, New York · Zbl 1229.62100
[25] Xing D, Wan H, Zhu MY, Sanchez SM, Kaymal T (2013) Simulation screening experiments using lasso-optimal supersatured design and analysis: a maritime operations application. In: Proceedings of the 2013 winter simulation conference: simulation: making decisions in a complex world, 497-508
[26] Zhang, Ch; Zhang, Ss, Confidence intervals for low dimensional parameters in high dimensional linear models, J Royal Stat Soc Ser B, 76, 1, 217-242 (2014) · Zbl 1411.62196 · doi:10.1111/rssb.12026
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.