Determination of singular value truncation threshold for regularization in ill-posed problems. (English) Zbl 1469.65080
Summary: Appropriate regularization parameter specification is the linchpin for solving ill-posed inverse problems when regularization method is applied. This paper presents a novel technique to determine cut off singular values in the truncated singular value decomposition (TSVD) methods. Simple formulae are presented to calculate the index number of the singular value, beyond which all the smaller singular values and the corresponding vectors are truncated. The determination method of optimal truncation threshold is firstly theoretically inferred. Two-dimensional inverse problems processing Radon transform are then exemplified. Formulae to solve the problem with insufficient image resolution and projection angle number are derived by the currently proposed method. The results show that accuracy of the current method is similar to that of TSVD but with much superior efficiency. On the other hand, insufficiency in input data affects the output accuracy of the inverse solution, a least square method can be engaged to establish formulae calculating the truncation threshold. For an insufficient set of input data, the percentage difference between inversely reconstructed signal and TSVD reconstructed signal is about 3%. The current formulae offer reliable and more efficient approach to calculate the truncation threshold when TSVD is applied to solve inverse problems with known system characteristics.
MSC:
| 65F22 | Ill-posedness and regularization problems in numerical linear algebra |
| 65F15 | Numerical computation of eigenvalues and eigenvectors of matrices |
| 44A12 | Radon transform |
Keywords:
inverse problem; ill-posed problems; TSVD regularization method; truncation threshold; noise level; Radon transformReferences:
| [1] | Jin KH, McCann MT, Froustey E, et al. Deep convolutional neural network for inverse problems in imaging. IEEE Trans Image Process. 2017;26(9):4509-4522. · Zbl 1409.94275 · doi:10.1109/TIP.2017.2713099 |
| [2] | Koner P, Dash P. Maximizing the information content of Ill-posed space-based measurements using deterministic inverse method. Remote Sens (Basel). 2018;10(7):994-1017. · doi:10.3390/rs10070994 |
| [3] | Koner PK, Harris A, Maturi E. A physical deterministic inverse method for operational satellite remote sensing: an application for sea surface temperature retrievals. IEEE Trans Geosci Remote Sens. 2015;53(11):5872-5888. · doi:10.1109/TGRS.2015.2424219 |
| [4] | Khan SU, Yang S, Wang L, et al. A modified particle swarm optimization algorithm for global optimizations of inverse problems. IEEE Trans Magn. 2016;52(3):1-4. · doi:10.1109/TMAG.2015.2487678 |
| [5] | Liu G. FEA-AI and AI-AI: two-way deepnets for real-time computations for both forward and inverse mechanics problems. Int J Comput Methods. 2019;16(08):1950045. · Zbl 1489.74059 · doi:10.1142/S0219876219500452 |
| [6] | Liu G, Ma W, Han X. An inverse procedure for identification of loads on composite laminates. Compos Part B: Eng. 2002;33(6):425-432. · doi:10.1016/S1359-8368(02)00027-6 |
| [7] | Liu G, Zaw K, Wang Y. Rapid inverse parameter estimation using reduced-basis approximation with asymptotic error estimation. Comput Methods Appl Mech Eng. 2008;197(45-48):3898-3910. · Zbl 1194.74102 · doi:10.1016/j.cma.2008.03.012 |
| [8] | Liu G, Duan S, Zhang Z, et al. Tubenet: a special trumpetnet for explicit solutions to inverse problems. Int J Comput Methods. 2020;17(7):1-31. 2050030. · Zbl 07342029 |
| [9] | Han X, Liu J. Numerical simulation-based design. SINGAPOR: Springer Verlag; 2017. |
| [10] | Liu G, Han X. Computational inverse techniques in nondestructive evaluation. Florida: CRC Press; 2003. · Zbl 1067.74002 · doi:10.1201/9780203494486 |
| [11] | Zhang H, Ma J, Moore W, et al. Characterization of the previous normal-dose CT scan induced nonlocal means regularization method for low-dose CT image reconstruction, arXiv preprint arXiv:1702.06668, 2017. |
| [12] | Zhang L, Zeng L, Wang C, et al. A non-smooth and non-convex regularization method for limited-angle CT image reconstruction. J Inverse ILL-Posed Probl. 2018;26(6):799-820. · Zbl 1404.94011 · doi:10.1515/jiip-2017-0042 |
| [13] | Harkat H, Bennani SD, Mansouri A, et al. Inversion of GPR data: 2D forms reconstruction for cavities based on born approximation, TSVD regularization and image processing algorithms. 2016 International Conference on Information Technology for Organizations Development (IT4OD); 2016, IEEE. p. 1-6. |
| [14] | Miyato T, Maeda S-I, Koyama M, et al. Virtual adversarial training: a regularization method for supervised and semi-supervised learning. IEEE Trans Pattern Anal Mach Intell. 2018;41(8):1979-1993. · doi:10.1109/TPAMI.2018.2858821 |
| [15] | Gavrilyuk A, Osinkin D, Bronin D. The use of Tikhonov regularization method for calculating the distribution function of relaxation times in impedance spectroscopy. Russ J Electrochem. 2017;53(6):575-588. · doi:10.1134/S1023193517060040 |
| [16] | Tikhonov AN, Goncharsky A, Stepanov V, et al. Numerical methods for the solution of ill-posed problems. Moscow, Russia: Springer Science & Business Media; 2013. |
| [17] | Landweber L. An iteration formula for Fredholm integral equations of the first kind. Am J Math. 1951;73(3):615-624. · Zbl 0043.10602 · doi:10.2307/2372313 |
| [18] | Backus G, Gilbert F. Uniqueness in the inversion of inaccurate gross earth data. Philos Trans Royal Soc London Series A Math Phys Sci. 1970;266(1173):123-192. · doi:10.1098/rsta.1970.0005 |
| [19] | Awrejcewicz J, Krysko VA. Nonclassical thermoelastic problems in nonlinear dynamics of shells: applications of the Bubnov-Galerkin and finite difference numerical methods. New York: Springer Science & Business Media; 2012. |
| [20] | Hansen PC. Truncated singular value decomposition solutions to discrete ill-posed problems with ill-determined numerical rank. SIAM J Sci Stat Comput. 1990;11(3):503-518. · Zbl 0699.65029 · doi:10.1137/0911028 |
| [21] | Buss SR. Introduction to inverse kinematics with Jacobian transpose, pseudoinverse and damped least squares methods. IEEE J Robot Autom. 2004;17(1-19):16-35. |
| [22] | Hoerl AE, Kennard RW. Ridge regression: Biased estimation for nonorthogonal problems. Technometrics. 1970;12(1):55-67. · Zbl 0202.17205 |
| [23] | Lanza A, Morigi S, Sgallari F. Convex image denoising via non-convex regularization with parameter selection. J Math Imaging Vis. 2016;56(2):195-220. · Zbl 1391.94088 · doi:10.1007/s10851-016-0655-7 |
| [24] | Shen H, Peng L, Yue L, et al. Adaptive norm selection for regularized image restoration and super-resolution. IEEE Trans Cybern. 2015;46(6):1388-1399. · doi:10.1109/TCYB.2015.2446755 |
| [25] | Wang Z. Multi-parameter Tikhonov regularization and model function approach to the damped Morozov principle for choosing regularization parameters. J Comput Appl Math. 2012;236(7):1815-1832. · Zbl 1247.65072 · doi:10.1016/j.cam.2011.10.014 |
| [26] | Morozov VA. Methods for solving incorrectly posed problems. New York (NY): Springer; 1984. · Zbl 0549.65031 · doi:10.1007/978-1-4612-5280-1 |
| [27] | Dauben Jr HJ, Wilson JD, Laity JL. Diamagnetic susceptibility exaltation as a criterion of aromaticity. J Am Chem Soc. 1968;90(3):811-813. · doi:10.1021/ja01005a059 |
| [28] | Engl HW, Hanke M, Neubauer A. Regularization of inverse problems. Dordrecht: Springer Science & Business Media; 1996. · Zbl 0859.65054 · doi:10.1007/978-94-009-1740-8 |
| [29] | Cosgrove RB, Milanfar P, Kositsky J. Trained detection of buried mines in SAR images via the deflection-optimal criterion. IEEE Trans Geosci Remote Sens. 2004;42(11):2569-2575. · doi:10.1109/TGRS.2004.834591 |
| [30] | Xu P. Truncated SVD methods for discrete linear ill-posed problems. Geophys J Int. 1998;135(2):505-514. · doi:10.1046/j.1365-246X.1998.00652.x |
| [31] | Xu P, Shen Y, Fukuda Y, et al. Variance component estimation in linear inverse ill-posed models. J Geod. 2006;80(2):69-81. · doi:10.1007/s00190-006-0032-1 |
| [32] | Xu P. Iterative generalized cross-validation for fusing heteroscedastic data of inverse ill-posed problems. Geophys J Int. 2009;179(1):182-200. · doi:10.1111/j.1365-246X.2009.04280.x |
| [33] | Golub G, Heath M, Wahba G. Generalized cross-validation as a method for choosing a good ridge parameter. Technometrics. 1979;21(2):215-223. · Zbl 0461.62059 |
| [34] | Hansen PC. Analysis of discrete Ill-posed problems by means of the L-curve. SIAM Rev. 1992;34(4):561-580. · Zbl 0770.65026 · doi:10.1137/1034115 |
| [35] | O’Leary DP, Hansen PC. The use of the L-curve in the regularization of discrete Ill-posed problems. Siam J Sci Comput. 1993;14(6):1487-1480. · Zbl 0789.65030 · doi:10.1137/0914086 |
| [36] | Hansen PC. Rank-deficient and discrete ill-posed problems: numerical aspects of linear inversion. Philadelphia: SIAM; 1998. · Zbl 0890.65037 · doi:10.1137/1.9780898719697 |
| [37] | Fan Q, Fang X-H, Fan J. Comparison of direct regularization methods of Ill-conditioned problems solution. J Guizhou Univer (Nat Sci). 2011;033(4):29-32. |
| [38] | Wu Z, Bian S, Xiang C, et al. A new method for TSVD regularization truncated parameter selection. Math Probl Eng. 2013 Nov;2013(1):1-9. · Zbl 1296.65063 |
| [39] | Wang CL, Yue SH. New selection methods of regularization parameter for electrical resistance tomography image reconstruction. 2016 IEEE International Instrumentation and Measurement Technology Conference (I2MTC); 2016. |
| [40] | Xiao Y, Song Q, Li S, et al. Comparisons of regularization methods and regularization parameter selection methods in sound source identification using inverse boundary element method. Noise Control Eng J. 2019;67(3):219-227. · doi:10.3397/1/376720 |
| [41] | Hou R, Xia Y, Bao Y, et al. Selection of regularization parameter for l1-regularized damage detection. J Sound Vib. 2018;423:141-160. · doi:10.1016/j.jsv.2018.02.064 |
| [42] | Mueller JL, Siltanen S. Linear and nonlinear inverse problems with practical applications. Philadelphia: SIAM; 2012. · Zbl 1262.65124 · doi:10.1137/1.9781611972344 |
| [43] | Mukundan R, Ramakrishnan K. Moment functions in image analysis-theory and applications. River Edge (NJ): World Scientific; 1998. · Zbl 0998.94506 · doi:10.1142/3838 |
| [44] | Wang J, Chen J, Wang X, et al. Research on noise characteristics of industrial CT detecting system. Nucl Electron Detect Technol. 2010;30(7):929-934. |
| [45] | Hämäläinen K, Harhanen L, Kallonen A, et al. Tomographic X-ray data of a walnut. arXiv preprint arXiv:1502.04064, 2015. |
| [46] | Hansen PC. Regularization tools: a Matlab package for analysis and solution of discrete ill-posed problems. Numer Algorithms. 1994;6(1):1-35. · Zbl 0789.65029 · doi:10.1007/BF02149761 |
| [47] | Hansen PC, Jensen TK, Rodriguez G. An adaptive pruning algorithm for the discrete L-curve criterion. J Comput Appl Math. 2015;198(2):483-492. · Zbl 1101.65044 · doi:10.1016/j.cam.2005.09.026 |
| [48] | Yufang C, Fanping F, Jue W, et al. Optimization reconstruction of biregular term from limited-angle projectionsIOP conference series: earth and environmental science Vol. 332. Singapore: IOP Publishing; 2019. p. 042002. · doi:10.1088/1755-1315/332/4/042002 |
| [49] | Hämäläinen K, Harhanen L, Hauptmann A, et al. Total variation regularization for large-scale X-ray tomography. Int J Tomogr Simul. 2014;25(1):1-25. |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
