Online learning in optical tomography: a stochastic approach. (English) Zbl 1507.65304
Summary: We study the inverse problem of radiative transfer equation (RTE) using stochastic gradient descent method (SGD) in this paper. Mathematically, optical tomography amounts to recovering the optical parameters in RTE using the incoming-outgoing pair of light intensity. We formulate it as a PDE-constraint optimization problem, where the mismatch of computed and measured outgoing data is minimized with same initial data and RTE constraint. The memory and computation cost it requires, however, is typically prohibitive, especially in high dimensional space. Smart iterative solvers that only use partial information in each step is called for thereafter. Stochastic gradient descent method is an online learning algorithm that randomly selects data for minimizing the mismatch. It requires minimum memory and computation, and advances fast, therefore perfectly serves the purpose. In this paper we formulate the problem, in both nonlinear and its linearized setting, apply SGD algorithm and analyze the convergence performance.
MSC:
| 65R32 | Numerical methods for inverse problems for integral equations |
| 45K05 | Integro-partial differential equations |
| 60H35 | Computational methods for stochastic equations (aspects of stochastic analysis) |
Keywords:
stochastic gradient descent; radiative transfer equation; optical tomography; online learningSoftware:
HOGWILDReferences:
| [1] | Arridge, S., Optical tomography in medical imaging, Inverse Problems, 15, R41-R93, (1999) · Zbl 0926.35155 · doi:10.1088/0266-5611/15/2/022 |
| [2] | Bal, G., Inverse transport theory and applications, Inverse Problems, 25, (2009) · Zbl 1178.35377 · doi:10.1088/0266-5611/25/5/053001 |
| [3] | Bal, G.; Jollivet, A., Time-dependent angularly averaged inverse transport, Inverse Problems, 25, (2009) · Zbl 1177.35246 · doi:10.1088/0266-5611/25/7/075010 |
| [4] | Bal, G.; Langmore, I.; Monard, F., Inverse transport with isotropic sources and angularly averaged measurement, Inverse Problems Imaging, 2, 23-42, (2008) · Zbl 1171.65104 · doi:10.3934/ipi.2008.2.23 |
| [5] | Bal, G.; Monard, F., Inverse transport with isotropic time-harmonic sources, SIAM J. Math. Anal., 44, 134-161, (2012) · Zbl 1244.35163 · doi:10.1137/11083397X |
| [6] | Bottou, L., Large-Scale Machine Learning with Stochastic Gradient Descent, 177-186, (2010), Heidelberg: Physica-Verlag HD, Heidelberg · Zbl 1436.68293 |
| [7] | Bottou, L., Stochastic Gradient Descent Tricks, 421-436, (2012), Berlin: Springer, Berlin |
| [8] | Chen, K.; Li, Q.; Wang, L., Stability of stationary inverse transport equation in diffusion scaling, Inverse Problems, 34, (2018) · Zbl 1474.35690 · doi:10.1088/1361-6420/aa990c |
| [9] | Cheng, Y.; Gamba, I. M.; Ren, K., Recovering doping profiles in semiconductor devices with the Boltzmann–Poisson model, J. Comput. Phys., 230, 3391-3412, (2011) · Zbl 1220.82099 · doi:10.1016/j.jcp.2011.01.034 |
| [10] | Choulli, M.; Stefanov, P., An inverse boundary value problem for the stationary transport equation, Osaka J. Math., 36, 87-104, (1998) · Zbl 0998.35064 |
| [11] | Dout, S.; Schmitt, B.; Lopes-Gautier, R.; Carlson, R.; Soderblom, L.; Shirley, J., Mapping So_{2} frost on Io by the modeling of nims hyperspectral images, Icarus, 149, 107-132, (2001) · doi:10.1006/icar.2000.6513 |
| [12] | Egger, H.; Schlottbom, M., An lp theory for stationary radiative transfer, Appl. Anal., 93, 1283-1296, (2014) · Zbl 1292.35091 · doi:10.1080/00036811.2013.826798 |
| [13] | Egger, H.; Schlottbom, M., Numerical methods for parameter identification in stationary radiative transfer, Comput. Optim. Appl., 62, 67-83, (2015) · Zbl 1333.49050 · doi:10.1007/s10589-014-9657-9 |
| [14] | Epstein, C., Introduction to the Mathematics of Medical Imaging, (2007), Philadelphia, PA: SIAM, Philadelphia, PA |
| [15] | Feng, Y.; Li, L.; Liu, J. G., Semi-groups of stochastic gradient descent and online principal component analysis: properties and diffusion approximations, Commun. Math. Sci., (2018) · Zbl 1417.60065 |
| [16] | Haltmeier, M.; Leitao, A.; Scherzer, O., Kaczmarz methods for regularizing nonlinear ill-posed equations I: convergence analysis, Inverse Problems Imaging, 1, 289, (2007) · Zbl 1123.65051 · doi:10.3934/ipi.2007.1.289 |
| [17] | Konecny, J.; Qu, Z.; Richtarik, P., Semi-stochastic coordinate descent, Optim. Methods Softw., 32, 993-1005, (2017) · Zbl 1386.90080 · doi:10.1080/10556788.2017.1298596 |
| [18] | Lehtikangas, O.; Tarvainen, T.; Kim, A.; Arridge, S., Finite element approximation of the radiative transport equation in a medium with piece-wise constant refractive index, J. Comput. Phys., 282, 345-359, (2015) · Zbl 1352.65524 · doi:10.1016/j.jcp.2014.11.025 |
| [19] | Leitao, A.; Svaiter, B. F., On projective landweberkaczmarz methods for solving systems of nonlinear ill-posed equations, Inverse Problems, 32, (2016) · Zbl 1344.65052 · doi:10.1088/0266-5611/32/2/025004 |
| [20] | Li, Q.; Tai, C.; Precup, D.; Teh, Y. W., Stochastic modified equations and adaptive stochastic gradient algorithms, 2101-2110, (2017), Cham: Springer, Cham |
| [21] | Li, Q.; Wang, L., Implicit asymptotic preserving method for linear transport equation, Commun. Comput. Phys., 22, 157-181, (2017) · Zbl 1488.65262 · doi:10.4208/cicp.OA-2016-0105 |
| [22] | Mandt, S.; Hoffman, M. D.; Blei, D. M., A variational analysis of stochastic gradient algorithms, vol 48, (2016) |
| [23] | Montejo, L. D.; Jia, J.; Kim, H. K.; Netz, U. J.; Blaschke, S.; Muller, G. A.; Hielscher, A. H., Computer-aided diagnosis of rheumatoid arthritis with optical tomography, part 1: feature extraction, J. Biomed. Opt., 18, (2013) · doi:10.1117/1.JBO.18.7.076001 |
| [24] | Montejo, L. D.; Jia, J.; Kim, H. K.; Netz, U. J.; Blaschke, S.; Muller, G. A.; Hielscher, A. H., Computer-aided diagnosis of rheumatoid arthritis with optical tomography, part 2: image classification, J. Biomed. Opt., 18, (2013) · doi:10.1117/1.JBO.18.7.076002 |
| [25] | Moulines, E.; Bach, F. R.; Shawe-Taylor, J., Non-asymptotic analysis of stochastic approximation algorithms for machine learning, Advances in Neural Information Processing Systems 24, 451-459, (2011), Red Hook, NY: Curran Associates and Inc., Red Hook, NY |
| [26] | Natterer, F., The Mathematics of Computerized Tomography, (2001), Philadelphia, PA: SIAM, Philadelphia, PA · Zbl 0973.92020 |
| [27] | Needell, D.; Ward, R.; Srebro, N.; Ghahramani, Z., Stochastic gradient descent, weighted sampling, and the randomized kaczmarz algorithm, Advances in Neural Information Processing Systems 27, 1017-1025, (2014), Red Hook, NY: Curran Associates and Inc., Red Hook, NY |
| [28] | Ozen, H. C.; Bal, G., Dynamical polynomial chaos expansions and long time evolution of differential equations with random forcing, SIAM/ASA J. Uncertain. Quantification, 4, 609-635, (2016) · Zbl 1346.60108 · doi:10.1137/15M1019167 |
| [29] | Prieto, K.; Dorn, O., Sparsity and level set regularization for diffuse optical tomography using a transport model in 2d, Inverse Problems, 33, (2017) · Zbl 1361.65104 · doi:10.1088/0266-5611/33/1/014001 |
| [30] | Recht, B.; Re, C.; Wright, S.; Niu, F.; Shawe-Taylor, J., Hogwild: a lock-free approach to parallelizing stochastic gradient descent, Advances in Neural Information Processing Systems 24, 693-701, (2011), Red Hook, NY: Curran Associates and Inc., Red Hook, NY |
| [31] | Ren, K., Recent developments in numerical techniques for transport-based medical imaging methods, Commun. Comput. Phys., 8, 1-50, (2010) · Zbl 1364.94100 · doi:10.4208/cicp.220509.200110a |
| [32] | Ren, K.; Zhang, R.; Zhong, Y., Inverse transport problems in quantitative pat for molecular imaging, Inverse Problems, 31, (2015) · Zbl 1329.92068 · doi:10.1088/0266-5611/31/12/125012 |
| [33] | Roux, N. L.; Schmidt, M.; Bach, F. R.; Pereira, F., A stochastic gradient method with an exponential convergence rate for finite training sets, Advances in Neural Information Processing Systems 25, 2663-2671, (2012), Red Hook, NY: Curran Associates and Inc., Red Hook, NY |
| [34] | Saratoon, T.; Tarvainen, T.; Cox, B.; Arridge, S., A gradient-based method for quantitative photoacoustic tomography using the radiative transfer equation, Inverse Problems, 29, (2013) · Zbl 1295.92019 · doi:10.1088/0266-5611/29/7/075006 |
| [35] | Stefanov, P.; Tamasan, A., Uniqueness and non-uniqueness in inverse radiative transfer, Proc. Am. Math. Soc., 137, 2335-2344, (2009) · Zbl 1173.35122 |
| [36] | Strohmer, T.; Vershynin, R., A randomized kaczmarz algorithm with exponential convergence, J. Fourier Anal. Appl., 15, 262, (2008) · Zbl 1169.68052 · doi:10.1007/s00041-008-9030-4 |
| [37] | Tang, J.; Han, W.; Han, B., A theoretical study for rte-based parameter identification problems, Inverse Problems, 29, (2013) · Zbl 1287.45006 · doi:10.1088/0266-5611/29/9/095002 |
| [38] | Tarvainen, T.; Kolehmainen, V.; Arridge, S. R.; Kaipio, J. P., Image reconstruction in diffuse optical tomography using the coupled radiative transportdiffusion model, J. Quant. Spectrosc. Radiat. Transfer, 112, 2600-2608, (2011) · doi:10.1016/j.jqsrt.2011.07.008 |
| [39] | Wang, J., Stability estimates of an inverse problem for the stationary transport equation, Ann. Inst. Henri Poincare, 70, 473-495, (1999) · Zbl 0963.35204 |
| [40] | Zhang, L.; Mahdavi, M.; Jin, R.; Burges, C. J C., Linear convergence with condition number independent access of full gradients, Advances in Neural Information Processing Systems 26, 980-988, (2013), Red Hook, NY: Curran Associates and Inc., Red Hook, NY |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
