×

Importance sampling for the empirical measure of weakly interacting diffusions. (English) Zbl 1530.60027

In the setting of a weakly interacting particle system, the authors detail a control-based importance sampling scheme for estimation of statistics of the form \[ \mathbb{E}\left[\exp\left\{-NG\left(\mu_T\right)\right\}\right]\,, \] where \(\mu_T\) is the empirical measure of the particles at the terminal time \(T\), there are \(N\) particles in total, and \(G\) is an appropriate function. This involves defining a suitably controlled version of the underlying system, and using statistics from this controlled system to estimate quantities of interest for the original system with reduced variance for large \(N\). Suitable controls are given using a zero-viscosity Hamilton-Jacobi-Bellman (HJB) equation on a Wasserstein space.
One of the main results of the present paper is that, under certain assumptions (including boundedness and continuity of coefficients, and existence and uniqueness of solutions) for a bounded and continuous function \(G\), this procedure requires only a subexponential number of samples to achieve a given relative error as \(N\to\infty\). Under stronger assumptions, the number of samples required vanishes in the limit, so that a single sample suffices for large \(N\) in some situations.
These results are complemented by explicit consideration of some linear-quadratic examples in which the HJB equation can be solved analytically, and by numerical results.

MSC:

60F10 Large deviations
60F05 Central limit and other weak theorems
65C05 Monte Carlo methods
35F21 Hamilton-Jacobi equations

References:

[1] Adams, S.; Dirr, N.; Peletier, MA; Zimmer, J., From a large-deviations principle to the Wasserstein gradient flow: a new micro-macro passage, Commun. Math. Phys., 307, 3, 791-815 (2011) · Zbl 1267.60106 · doi:10.1007/s00220-011-1328-4
[2] Adams, S.; Dirr, N.; Peletier, MA; Zimmer, J., Large deviations and gradient flows, Philos. Trans. R. Soc. A, 371, 2005, 20120341 (2013) · Zbl 1292.82023 · doi:10.1098/rsta.2012.0341
[3] Babin, V.; Roland, C.; Sagui, C., Adaptively biased molecular dynamics for free energy calculations, J. Chem. Phys., 128, 13, 128-135 (2008) · doi:10.1063/1.2844595
[4] Backhoff, J.; Conforti, G.; Gentil, I.; Léonard, C., The mean field Schrödinger problem: ergodic behavior, entropy estimates and functional inequalities, Probab. Theory Relat. Fields, 178, 2, 475-530 (2020) · Zbl 1453.91021 · doi:10.1007/s00440-020-00977-8
[5] Basei, M.; Pham, H., A weak martingale approach to linear-quadratic McKean-Vlasov stochastic control problems, J. Optim. Theory Appl., 181, 2, 347-382 (2019) · Zbl 1416.49040 · doi:10.1007/s10957-018-01453-z
[6] Bashiri, K.; Menz, G., Metastability in a continuous mean-field model at low temperature and strong interaction, Stoch. Process. Appl., 134, 132-173 (2021) · Zbl 1469.60304 · doi:10.1016/j.spa.2020.12.007
[7] Ben Rached, N., Haji-Ali, A.-L., Pillai, S.M.S., Tempone, R.: Multilevel importance sampling for McKean-Vlasov stochastic differential equation. arXiv:2208.03225 [math.NA] (2022)
[8] Ben Rached, N., Haji-Ali, A.-L., Pillai, S.M.S., Tempone, R.: Single level importance sampling for McKean-Vlasov stochastic differential equations. arXiv: 2207.06926 [math.NA] (2022)
[9] Bezemek, ZW; Spiliopoulos, K., Large deviations for interacting multiscale particle systems, Stoch. Process. Appl., 155, 27-108 (2023) · Zbl 1508.60036 · doi:10.1016/j.spa.2022.09.010
[10] Bezemek, Z.W., Spiliopoulos, K.: Moderate deviations for fully coupled multiscale weakly interacting particle systems. Stochastics and Partial Differential Equations: Analysis and Computations (2023). doi:10.1007/s40072-023-00301-0 · Zbl 1508.60036
[11] Borrell, ER; Quer, J.; Richter, L.; Schütte, C., Improving control based importance sampling strategies for metastable diffusions via adapted metadynamics, SIAM J. Sci. Comput. (2023) · Zbl 1537.65005 · doi:10.1137/22M1503464
[12] Bouchet, F.; Gawȩdzki, K.; Nardini, C., Perturbative calculation of quasi-potential in non-equilibrium diffusions: a mean-field example, J. Stat. Phys., 163, 1157-1210 (2016) · Zbl 1341.82056 · doi:10.1007/s10955-016-1503-2
[13] Boué, M.; Dupuis, P., A variational representation for certain functionals of Brownian motion, Ann. Probab., 26, 4, 1641-1659 (1998) · Zbl 0936.60059 · doi:10.1214/aop/1022855876
[14] Budhiraja, A.; Conroy, M., Asymptotic behavior of stochastic currents under large deviation scaling with mean field interaction and vanishing noise, Annali della Scuola Normale di Pisa - Classe di Scienze, 23, 5, 1749-1805 (2022) · Zbl 1506.60037
[15] Budhiraja, A.; Conroy, M., Empirical measure and small noise asymptotics under large deviation scaling for interacting diffusions, J. Theor. Probab., 35, 295-349 (2022) · Zbl 1484.60028 · doi:10.1007/s10959-020-01071-4
[16] Budhiraja, A.; Dupuis, P.; Fischer, M., Large deviation properties of weakly interacting particles via weak convergence methods, Ann. Probab., 40, 1, 74-100 (2012) · Zbl 1242.60026 · doi:10.1214/10-AOP616
[17] Budhiraja, A.; Wu, R., Moderate deviation principles for weakly interacting particle systems, Probab. Theory Relat. Fields, 168, 721-771 (2017) · Zbl 1369.60012 · doi:10.1007/s00440-016-0723-3
[18] Burzoni, M.; Ignazio, V.; Reppen, AM; Soner, HM, Viscosity solutions for controlled McKean-Vlasov jump-diffusions, SIAM J. Control Optim., 58, 3, 1676-1699 (2020) · Zbl 1472.49054 · doi:10.1137/19M1290061
[19] Carmona, R.; Delarue, F., Probabilistic Theory of Mean Field Games with Applications I (2018), New York: Springer, New York · Zbl 1422.91014 · doi:10.1007/978-3-319-56436-4
[20] Carmona, R.; Laurière, M., Convergence analysis of machine learning algorithms for the numerical solution of mean field control and games: II—the finite horizon case, Ann. Appl. Probab., 32, 6, 4065-4105 (2022) · Zbl 1505.65243 · doi:10.1214/21-AAP1715
[21] Chassagneux, J.F., Crisan, D., Delarue, F.: A probabilistic approach to classical solutions of the master equation for large population equilibria. Memoirs of the American Mathematical Society (2022) 280, 1379. doi:10.1090/memo/1379 · Zbl 1520.91003
[22] Chassagneux, JF; Crisan, D.; Delarue, F., Numerical method for FBSDEs of McKean-Vlasov type, Ann. Appl. Probab., 29, 3, 1640-1684 (2019) · Zbl 1466.65012 · doi:10.1214/18-AAP1429
[23] Chaudru de Raynal, PE; Frikha, N., Well-posedness for some non-linear diffusion processes and related PDE on the Wasserstein space, Journal de Mathematiques Pures et Appliquees, 159, 1-167 (2022) · Zbl 1494.60063 · doi:10.1016/j.matpur.2021.12.001
[24] Chaintron, LP; Diez, A., Propagation of chaos: a review of models, methods and applications. I. Models and methods, Kinet. Relat. Models, 15, 6, 895-1015 (2022) · Zbl 1496.82016 · doi:10.3934/krm.2022017
[25] Comer, J.; Gumbart, JC; Hénin, J.; Lelièvre, T.; Pohorille, A.; Chipot, C., The adaptive biasing force method: everything you always wanted to know but were afraid to ask, J. Phys. Chem. B, 118, 3, 1129-1151 (2015) · doi:10.1021/jp506633n
[26] Cosso, A.; Gozzi, F.; Kharroubi, I.; Pham, H.; Rosestolato, M., Master Bellman equation in the Wasserstein space: uniqueness of viscosity solutions, Trans. Am. Math. Soc. (2023) · Zbl 1520.93608 · doi:10.1090/tran/8986
[27] Cosso, A.; Gozzi, F.; Kharroubi, I.; Pham, H.; Rosestolato, M., Optimal control of path-dependent McKean-Vlasov SDEs in infinite dimension, Ann. Appl. Probab., 33, 4, 2863-2918 (2023) · Zbl 1520.93608 · doi:10.1214/22-AAP1880
[28] Dawson, DA, Critical dynamics and fluctuations for a mean-field model of cooperative behavior, J. Stat. Phys., 31, 29-85 (1983) · doi:10.1007/BF01010922
[29] Dawson, DA; Gärtner, J., Large deviations from the Mckean-Vlasov limit for weakly interacting diffusions, Stochastics, 20, 4, 247-308 (1987) · Zbl 0613.60021 · doi:10.1080/17442508708833446
[30] Del Moral, P.; Guionnet, A., Large deviations for interacting particle systems: applications to non-linear filtering, Stoch. Process. Appl., 78, 1, 69-95 (1998) · Zbl 0934.60026 · doi:10.1016/S0304-4149(98)00057-X
[31] Delarue, F.; Lacker, D.; Ramanan, K., From the master equation to mean field game limit theory: large deviations and concentration of measure, Ann. Probab., 48, 1, 211-263 (2020) · Zbl 1445.60025 · doi:10.1214/19-AOP1359
[32] dos Reis, G.; Salkeld, W.; Tugaut, J., Freidlin-Wentzell LDP in path space for McKean-Vlasov equations and the functional iterated logarithm law, Ann. Probab., 29, 3, 1487-1540 (2019) · Zbl 1466.60056
[33] dos Reis, G.; Smith, G.; Tankov, P., Importance sampling for McKean-Vlasov SDEs, Appl. Math. Comput. (2023) · Zbl 07702363 · doi:10.1016/j.amc.2023.128078
[34] Dupuis, P.; Ellis, RS, A Weak Convergence Approach to the Theory of Large Deviations (1997), New York: Wiley, New York · Zbl 0904.60001 · doi:10.1002/9781118165904
[35] Dupuis, P.; Spiliopoulos, K.; Zhou, X., Escaping from an attractor: importance sampling and rest points I, Ann. Appl. Probab., 25, 5, 2909-2958 (2015) · Zbl 1334.65007 · doi:10.1214/14-AAP1064
[36] Dupuis, P.; Spiliopoulos, K.; Wang, H., Importance sampling for multiscale diffusions, Multiscale Model. Simul., 10, 1, 1-27 (2012) · Zbl 1250.60031 · doi:10.1137/110842545
[37] Dupuis, P.; Wang, H., Importance sampling, large deviations, and differential games, Stoch. Stoch. Rep., 76, 6, 481-508 (2004) · Zbl 1076.65003 · doi:10.1080/10451120410001733845
[38] Dupuis, P.; Wang, H., Subsolutions of an Isaacs equation and efficient schemes for importance sampling, Math. Oper. Res., 32, 3, 723-757 (2007) · Zbl 1341.62042 · doi:10.1287/moor.1070.0266
[39] E, W.; Ren, W.; Vanden-Eijnden, E., Minimum action method for the study of rare events, Commun. Pure Appl. Math., 57, 5, 637-656 (2004) · Zbl 1050.60068 · doi:10.1002/cpa.20005
[40] Ebener, L.; Margazoglou, G.; Friedrich, J.; Biferale, L.; Grauer, R., Instanton based importance sampling for rare events in stochastic PDEs, Chaos, 29, 6, 063102 (2019) · Zbl 1476.60005 · doi:10.1063/1.5085119
[41] Feng, J.; Katsoulakis, M., A comparison principle for Hamilton-Jacobi equations related to controlled gradient flows in infinite dimensions, Arch. Ration. Mech. Anal., 192, 275-310 (2009) · Zbl 1159.49036 · doi:10.1007/s00205-008-0133-5
[42] Feng, J.; Kurtz, G., Large Deviations for Stochastic Processes (2006), Providence: American Mathematical Society, Providence · Zbl 1113.60002 · doi:10.1090/surv/131
[43] Feng, J.; Mikami, T.; Zimmer, J., A Hamilton-Jacobi PDE associated with hydrodynamic fluctuations from a nonlinear diffusion equation, Commun. Math. Phys., 385, 1-54 (2021) · Zbl 1467.35102 · doi:10.1007/s00220-021-04110-1
[44] Feng, J.; Nguyen, T., Hamilton-Jacobi equations in space of measures associated with a system of conservation laws, Journal de Mathématiques Pures et Appliquées, 97, 4, 318-390 (2012) · Zbl 1242.49059 · doi:10.1016/j.matpur.2011.11.004
[45] Fischer, M., On the form of the large deviation rate function for the empirical measures of weakly interacting systems, Bernoulli, 20, 4, 1765-1801 (2014) · Zbl 1327.60070 · doi:10.3150/13-BEJ540
[46] Fleming, WH; James, MR, Asymptotic series and exit time probabilities, Ann. Probab., 20, 3, 1369-1384 (1992) · Zbl 0771.60055 · doi:10.1214/aop/1176989695
[47] Fouque, JP; Zhang, Z., Deep learning methods for mean field control problems with delay, Front. Appl. Math. Stat., 12, 1369-1384 (2020)
[48] Freidlin, MI; Wentzell, AD, Random Perturbations of Dynamical Systems (2012), Heidelberg: Springer, Heidelberg · Zbl 1267.60004 · doi:10.1007/978-3-642-25847-3
[49] Garnier, J.; Papanicolaou, G.; Yang, TW, Large deviations for a mean field model of systemic risk, SIAM J. Financ. Math., 4, 1, 151-184 (2013) · Zbl 1283.60044 · doi:10.1137/12087387X
[50] Garnier, J.; Papanicolaou, G.; Yang, TW, Consensus convergence with stochastic effects, Vietnam J. Math., 45, 51-75 (2017) · Zbl 1372.92118 · doi:10.1007/s10013-016-0190-2
[51] Gasteratos, I., Salins, M., Spiliopoulos, K.: Importance sampling for stochastic reaction-diffusion equations in the moderate deviation regime. arXiv:2206.00646 [math.PR] (2022) · Zbl 1518.60037
[52] Gavish, N.; Nyquist, P.; Peletier, M., Large deviations and gradient flows for the Brownian one-dimensional hard-rod system, Potential Anal., 58, 71-121 (2021) · Zbl 1506.60041
[53] Geha, M., Jacquier, A., Žurič, Ž.: Large and moderate deviations for importance sampling in the Heston model. SSRN Electronic Journal (2021). doi:10.2139/ssrn.3952672
[54] Germain, M.; Laurière, M.; Pham, H.; Warin, X., DeepSets and their derivative networks for solving symmetric PDEs, J. Sci. Comput., 91, 63 (2022) · Zbl 07545424 · doi:10.1007/s10915-022-01796-w
[55] Germain, M.; Mikael, J.; Warin, X., Numerical resolution of McKean-Vlasov FBSDEs using neural networks, Methodol. Comput. Appl. Probab., 24, 2557-2586 (2022) · Zbl 1505.65010 · doi:10.1007/s11009-022-09946-1
[56] Germain, M.; Pham, H.; Warin, X., Rate of convergence for particle approximation of PDEs in Wasserstein space, J. Appl. Probab., 59, 4, 992-1008 (2022) · Zbl 1501.35449 · doi:10.1017/jpr.2021.102
[57] Grafke, T., Schäfer, T., Vanden-Eijnden, E.: Sharp asymptotic estimates for expectations, probabilities, and mean first passage times in stochastic systems with small noise. Commun. Pure Appl. Math. (2023). doi:10.1002/cpa.22177 · Zbl 1532.60053
[58] Grünbaum, FA, Propagation of chaos for the Boltzmann equation, Arch. Ration. Mech. Anal., 42, 323-345 (1971) · Zbl 0236.45011 · doi:10.1007/BF00250440
[59] Gvalani, RS; Schlichting, A., Barriers of the McKean-Vlasov energy via a mountain pass theorem in the space of probability measures, J. Funct. Anal., 279, 11, 437-461 (2020) · Zbl 1473.60061 · doi:10.1016/j.jfa.2020.108720
[60] Han, J., Hu, R., Long, J.: Learning high-dimensional McKean-Vlasov forward-backward stochastic differential equations with general distribution dependence. arXiv:2204.11924 [math.OC] (2022)
[61] Hartmann, C.; Banisch, R.; Sarich, M.; Badowski, T.; Schütte, C., Characterization of rare events in molecular dynamics, Entropy, 16, 1, 350-376 (2014) · Zbl 1338.82038 · doi:10.3390/e16010350
[62] Hartmann, C., Richter, L.: Nonasymptotic bounds for suboptimal importance sampling. arXiv: 2102.09606 [math.ST] (2021)
[63] Kac, M., Probability and Related Topics in Physical Sciences (1957), Providence: American Mathematical Society, Providence
[64] Karatzas, I.; Shreve, S., Brownian Motion and Stochastic Calculus (1998), New York: Springer, New York · Zbl 0638.60065 · doi:10.1007/978-1-4612-0949-2
[65] Lacker, D., Limit theory for controlled McKean-Vlasov dynamics, SIAM J. Control Optim., 55, 3, 1641-1672 (2017) · Zbl 1362.93167 · doi:10.1137/16M1095895
[66] Laurière, M., Numerical methods for mean field games and mean field type control, Proc. Symp. Appl. Math., 78, 221-282 (2021) · Zbl 1536.91049 · doi:10.1090/psapm/078/06
[67] Laurièr, M.; Pironneau, O., Dynamic programming for mean-field type control, J. Optim. Theory Appl., 169, 902-924 (2016) · Zbl 1342.35420 · doi:10.1007/s10957-015-0785-x
[68] Liu, W.; Song, Y.; Zhai, J.; Zhang, T., Large and moderate deviation principles for McKean-Vlasov SDEs with jumps, Potential Anal., 59, 3, 1141-1190 (2023) · Zbl 1523.60056 · doi:10.1007/s11118-022-10005-0
[69] Morse, MR; Spiliopoulos, K., Importance sampling for slow-fast diffusions based on moderate deviations, SIAM J. Multiscale Model. Simul., 18, 1, 315-350 (2020) · Zbl 1440.60024 · doi:10.1137/18M1192962
[70] Nilsson, V., Nyquist, P.: A note on large deviations for interacting particle dynamics for finding mixed equilibria in zero-sum games. arXiv:2206.15177 [stat.ML] (2022)
[71] Nüsken, N.; Richter, L., Solving high-dimensional Hamilton-Jacobi-Bellman PDEs using neural networks: perspectives from the theory of controlled diffusions and measures on path space, Part. Differ. Equ. Appl., 2, 48 (2021) · Zbl 1480.35101 · doi:10.1007/s42985-021-00102-x
[72] Orrieri, C., Large deviations for interacting particle systems: joint mean-field and small-noise limit, Electron. J. Probab., 25, 1-44 (2020) · Zbl 1459.60069 · doi:10.1214/20-EJP516
[73] Pham, H.; Wei, X., Bellman equation and viscosity solutions for mean-field stochastic control problem, ESAIM: Control Optim. Calc. Var., 24, 1, 437-461 (2018) · Zbl 1396.93134
[74] Salins, M.; Spiliopoulos, K., Rare event simulation via importance sampling for linear SPDE’s, Stoch. Part. Differ. Equ.: Anal. Comput., 5, 4, 652-690 (2017) · Zbl 1391.60161
[75] Spiliopoulos, K., Nonasymptotic performance analysis of importance sampling schemes for small noise diffusions, J. Appl. Probab., 53, 3, 797-810 (2015) · Zbl 1334.60166 · doi:10.1239/jap/1445543847
[76] Talbi, M.; Touzi, N.; Zhang, J., Dynamic programming equation for the mean field optimal stopping problem, SIAM J. Control Optim., 61, 4, 2140-2164 (2023) · Zbl 1515.60113 · doi:10.1137/21M1404259
[77] Tong, S.; Stadler, G., Large deviation theory-based adaptive importance sampling for rare events in high dimensions, SIAM/ASA J. Uncertain. Quant., 11, 3, 788-813 (2023) · Zbl 1518.65009 · doi:10.1137/22M1524758
[78] Valsson, O.; Tiwary, P.; Parrinello, M., Enhancing important fluctuations: rare events and metadynamics from a conceptual viewpoint, Annu. Rev. Phys. Chem., 67, 159-184 (2016) · doi:10.1146/annurev-physchem-040215-112229
[79] Vanden-Eijnden, E.; Weare, J., Rare event simulation of small noise diffusions, Commun. Pure Appl. Math., 65, 12, 1770-1803 (2012) · Zbl 1268.65015 · doi:10.1002/cpa.21428
[80] Yong, J., Linear-quadratic optimal control problems for mean-field stochastic differential equations, SIAM J. Control Optim., 51, 4, 2809-2838 (2013) · Zbl 1275.49060 · doi:10.1137/120892477
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.