×

Online prediction with history-dependent experts: the general case. (English) Zbl 1522.91012

Summary: We study the problem of prediction of binary sequences with expert advice in the online setting, which is a classic example of online machine learning. We interpret the binary sequence as the price history of a stock, and view the predictor as an investor, which converts the problem into a stock prediction problem. In this framework, an investor, who predicts the daily movements of a stock, and an adversarial market, who controls the stock, play against each other over turns. The investor combines the predictions of \(n \geq 2\) experts in order to make a decision about how much to invest at each turn, and aims to minimize their regret with respect to the best-performing expert at the end of the game. We consider the problem with history-dependent experts, in which each expert uses the previous days of history of the market in making their predictions. We prove that the value function for this game, rescaled appropriately, converges as \(N \to \infty\) at a rate of \(O \left(N^{- 1 / 6}\right)\) to the viscosity solution of a nonlinear degenerate elliptic PDE, which can be understood as the Hamilton-Jacobi-Isaacs equation for the two-person game. As a result, we are able to deduce asymptotically optimal strategies for the investor. Our results extend those established by the first author and R.V. Kohn [14] for \(n = 2\) experts and \(d \leq 4\) days of history.
{© 2022 The Authors. Communications on Pure and Applied Mathematics published by Wiley Periodicals LLC.}

MSC:

91A05 2-person games
90C39 Dynamic programming
49L25 Viscosity solutions to Hamilton-Jacobi equations in optimal control and differential games

References:

[1] Amin, K.; Kale, S.; Tesauro, G.; Turaga, D. Budgeted prediction with expert advice. Twenty‐Ninth AAAI Conference on Artificial Intelligence, 2015.
[2] Andoni, A.; Panigrahy, R.A differential equations approach to optimizing regret trade‐offs. Preprint, 2013. 1305.1359 [cs.LG]
[3] Antunovic, T.; Peres, Y.; Sheffield, S.; Somersille, S.Tug‐of‐war and infinity Laplace equation with vanishing Neumann boundary condition. Comm. Partial Differential Equations37 (2012), no. 10, 1839-1869. doi: https://doi.org/10.1080/03605302.2011.642450 · Zbl 1268.35065 · doi:10.1080/03605302.2011.642450
[4] Armstrong, S. N.; Smart, C. K.A finite difference approach to the infinity Laplace equation and tug‐of‐war games. Trans. Amer. Math. Soc.364 (2012), no. 2, 595-636. doi: https://doi.org/10.1090/S0002-9947-2011-05289-X. · Zbl 1239.91011 · doi:10.1090/S0002-9947-2011-05289-X
[5] Bayraktar, E.; Ekren, I.; Zhang, Y.On the asymptotic optimality of the comb strategy for prediction with expert advice. Ann. Appl. Probab.30 (2020), no. 6, 2517-2546. doi: https://doi.org/10.1214/20-AAP1565 · Zbl 1529.68253 · doi:10.1214/20-AAP1565
[6] Calder, J. Lecture notes on viscosity solutions (2018). Available at: http://www-users.math.umn.edu/ jwcalder/viscosity_solutions.pdf.
[7] Calder, J.; Drenska, N. Asymptotically optimal strategies for online prediction with history‐dependent experts. J. Fourier Anal. Appl. 27 (2021), no. 2, Paper No. 20, 20 pp. doi: 10.1007/s00041‐021‐09815‐4 · Zbl 1459.91171
[8] Calder, J.; Smart, C. K.The limit shape of convex hull peeling. Duke Math. J.169 (2020), no. 11, 2079-2124. doi: https://doi.org/10.1215/00127094-2020-0013 · Zbl 1455.35047 · doi:10.1215/00127094-2020-0013
[9] Cesa‐Bianchi, N.; Freund, Y.; Haussler, D.; Helmbold, D. P.; Schapire, R. E.; Warmuth, M. K.How to use expert advice. J. ACM44 (1997), no. 3, 427-485. doi: https://doi.org/10.1145/258128.258179 · Zbl 0890.68066 · doi:10.1145/258128.258179
[10] Cesa‐Bianchi, N.; Lugosi, G. Prediction,learning, and games. Cambridge University Press, Cambridge, 2006. doi: https://doi.org/10.1017/CBO9780511546921 · Zbl 1114.91001 · doi:10.1017/CBO9780511546921
[11] Cover, T. M.Behavior of sequential predictors of binary sequences. Technical report. Stanford University California Stanford Electronics Labs, 1966.
[12] Crandall, M. G.; Ishii, H.; Lions, P.‐L. User’s guide to viscosity solutions of second order partial differential equations. Bull. Amer. Math. Soc. (N.S.)27 (1992), no. 1, 1-67. doi: 10.1090/S0273‐0979‐1992‐00266‐5 · Zbl 0755.35015
[13] Drenska, N.A PDE approach to a prediction problem involving randomized strategies. Ph.D. thesis, New York University, 2017.
[14] Drenska, N.; Kohn, R. V.A PDE approach to the prediction of a binary sequence with advice from two history‐dependent experts. Preprint, 2020. 2007.12732 [math.OC]
[15] Drenska, N.; Kohn, R. V.Prediction with expert advice: a PDE perspective. J. Nonlinear Sci.30 (2020), no. 1, 137-173. doi: https://doi.org/10.1007/s00332-019-09570-3 · Zbl 1439.35484 · doi:10.1007/s00332-019-09570-3
[16] Evans, L.Partial differential equations. Second edition. Graduate Studies in Mathematics, 19. American Mathematical Society, Providence, R.I., 2010. doi: https://doi.org/10.1090/gsm/019 · Zbl 1194.35001 · doi:10.1090/gsm/019
[17] Freund, Y.; Schapire, R. E. A decision‐theoretic generalization of on‐line learning and an application to boosting. J. Comput. System Sci. 55 (1997), no. 1, part 2, 119-139. doi: 10.1006/jcss.1997.1504 · Zbl 0880.68103
[18] Giga, Y.; Goto, S.; Ishii, H.; Sato, M.‐H.Comparison principle and convexity preserving properties for singular degenerate parabolic equations on unbounded domains. Indiana Univ. Math. J.40 (1991), no. 2, 443-470. doi: https://doi.org/10.1512/iumj.1991.40.40023 · Zbl 0836.35009 · doi:10.1512/iumj.1991.40.40023
[19] Gravin, N.; Peres, Y.; Sivan, B.Towards optimal algorithms for prediction with expert advice. Proceedings of the Twenty‐Seventh Annual ACM‐SIAM Symposium on Discrete Algorithms, 528-547. ACM, New York, 2016. doi: https://doi.org/10.1137/1.9781611974331.ch39 · Zbl 1411.68103 · doi:10.1137/1.9781611974331.ch39
[20] Hannan, J. Approximation to Bayes risk in repeated play. Contributions to the theory of games, vol. 3, 97-139. Annals of Mathematics Studies, 39. Princeton University Press, Princeton, N.J., 1957. · Zbl 0078.32804
[21] Haussler, D.; Kivinen, J.; Warmuth, M. K. Tight worst‐case loss bounds for predicting with expert advice. Computational learning theory (Barcelona, 1995), 69-83. Lecture Notes in Comput. Sci., 904. Lecture Notes in Artificial Intelligence. Springer, Berlin, 1995. doi: 10.1007/3‐540‐59119‐2_169
[22] Kohn, R. V.; Serfaty, S.A deterministic‐control‐based approach motion by curvature. Comm. Pure Appl. Math.59 (2006), no. 3, 344-407. doi: https://doi.org/10.1002/cpa.20101 · Zbl 1206.53072 · doi:10.1002/cpa.20101
[23] Kohn, R. V.; Serfaty, S.A deterministic‐control‐based approach to fully nonlinear parabolic and elliptic equations. Comm. Pure Appl. Math.63 (2010), no. 10, 1298-1350. doi: https://doi.org/10.1002/cpa.20336 · Zbl 1204.35070 · doi:10.1002/cpa.20336
[24] Lewicka, M.; Manfredi, J. J. The obstacle problem for the p‐laplacian via optimal stopping of tug‐of‐war games. Probab. Theory Related Fields167 (2017), no. 1‐2, 349‐378. doi: https://doi.org/10.1007/s00440-015-0684-y · Zbl 1358.60058 · doi:10.1007/s00440-015-0684-y
[25] Littlestone, N.; Warmuth, M. K.The weighted majority algorithm. Inform. and Comput.108 (1994), no. 2, 212-261. doi: https://doi.org/10.1006/inco.1994.1009 · Zbl 0804.68121 · doi:10.1006/inco.1994.1009
[26] Michael Kapralov, R. P.Prediction strategies without loss. Neural Information Processing Systems Foundation, 2012. https://www.microsoft.com/en-us/research/publication/prediction-strategies-without-loss/.
[27] Naor, A.; Sheffield, S.Absolutely minimal Lipschitz extension of tree‐valued mappings. Math. Ann.354 (2012), no. 3, 1049-1078. doi: https://doi.org/10.1007/s00208-011-0753-1 · Zbl 1276.46062 · doi:10.1007/s00208-011-0753-1
[28] Peres, Y.; Schramm, O.; Sheffield, S.; Wilson, D. B.Tug‐of‐war and the infinity Laplacian. J. Amer. Math. Soc.22 (2009), no. 1, 167-210. doi: https://doi.org/10.1090/S0894-0347-08-00606-1 · Zbl 1206.91002 · doi:10.1090/S0894-0347-08-00606-1
[29] Peres, Y.; Sheffield, S. Tug‐of‐war with noise: A game‐theoretic view of the p‐Laplacian. Duke Math. J.145 (2008), no. 1, 91-120. doi: https://doi.org/10.1215/00127094-2008-048 · Zbl 1206.35112 · doi:10.1215/00127094-2008-048
[30] Rokhlin, D.PDE approach to the problem of online prediction with expert advice: A construction of potential‐based strategies. International Journal of Pure and Applied Mathematics114 (2017), no. 4, 907-915. doi: https://doi.org/10.12732/ijpam.v114i4.20 · doi:10.12732/ijpam.v114i4.20
[31] Yadkori, Y. A.; Bartlett, P. L.; Gabillon, V.Near minimax optimal players for the finite‐time 3‐expert prediction problem. Advances in Neural Information Processing Systems30, 2017, 3033-3045.
[32] Zhu, K.Two problems in applications of PDE. Ph.D. thesis, New York University, 2014.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.