Ambiguity aversion in multi-armed bandit problems. (English) Zbl 1274.91110
Summary: In multi-armed bandit problems, information acquired from experimentation is valuable because it tells the agent whether to select a particular option again in the future. This article tests whether people undervalue this information because they are ambiguity averse, or have a distaste for uncertainty about the average quality of each alternative. It is shown that ambiguity averse agents have lower than optimal Gittins indexes, appearing to undervalue information from experimentation, but are willing to pay more than ambiguity neutral agents to learn the true mean of the payoff distribution, appearing to overvalue objectively given information. This prediction is tested with a laboratory experiment that elicits a Gittins index and a willingness to pay on six two-armed bandits. Consistent with the predictions of ambiguity aversion, the Gittins indexes are significantly lower than optimal and willingnesses to pay are significantly higher than optimal.
MSC:
| 91A60 | Probabilistic games; gambling |
| 91A90 | Experimental studies |
| 91B82 | Statistical methods; economic indices and measures |
| 62C10 | Bayesian problems; characterization of Bayes procedures |
| 90C40 | Markov and semi-Markov decision processes |
References:
| [1] | Acuna, D., & Schrater, P. (2008). Bayesian modeling of human sequential decision-making on the multi-armed bandit problem. In V. Sloutsky, B. Love, K. McRae (Eds.), Proceedings of the 30th annual conference of the cognitive science society. Washington, DC: Cognitive Science Society. |
| [2] | Anderson, C. (2001). Behavioral models of strategies in multiarmed bandit problems. California Institute of Technology Ph.D. Dissertation. |
| [3] | Anscombe F., Aumann R. (1963) A definition of subjective probability. Annals of Mathematical Statistics 34: 199–205 · Zbl 0114.07204 · doi:10.1214/aoms/1177704255 |
| [4] | Banks J., Olson M., Porter D. (1997) An experimental analysis ofthe bandit problem. Econ Theory 10: 55–77 · Zbl 0881.90001 · doi:10.1007/s001990050146 |
| [5] | Berry D., Fristedt B. (1985) Bandit problems. Chapman and Hall, New York · Zbl 0659.62086 |
| [6] | Boyce, J., Bruner, D., & McKee, M. (2009). Be my guinea pig: Information spillovers in a one-arm bandit problem. Appalachian State University Department of Economics working paper. |
| [7] | Braunstein Y., Schotter A. (1982) Labor market search: An experimental study. Economic Inquiry 20: 134–144 · doi:10.1111/j.1465-7295.1982.tb01146.x |
| [8] | Camerer C. (1995) Individual decision making. In: Roth A., Kagel J. (eds) Handbook of experimental economics. Princeton University Press, Princeton |
| [9] | Camerer C., Weber M. (1992) Recent developments in modeling preferences: Uncertainty and ambiguity. Journal of Risk and Uncertainty 5: 325–370 · Zbl 0775.90102 · doi:10.1007/BF00122575 |
| [10] | Cox J., Oaxaca R. (1989) Laboratory experiments with a finite-horizon job-search model. Journal of Risk and Uncertainty 2: 301–329 · doi:10.1007/BF00209391 |
| [11] | Cox, J., & Oaxaca, R. (1990). Unemployment insurance and job search. In Research in labor economics (Vol. 11, pp. 223–240). Greenwich, CT: JAI Press. |
| [12] | Cox J., Oaxaca R. (1992) Direct tests of the reservation wage property. The Economic Journal 102: 1423–1432 · doi:10.2307/2234798 |
| [13] | Cox, J., & Oaxaca, R. (1996). Testing job search models: The laboratory approach. In S. W. Polachek (Ed.), Research in labor economics (Vol. 15, pp. 171–207). Greenwich, CT: JAI Press |
| [14] | Cox J., Oaxaca R. (2000) Good news and bad news: Search from unknown wage offer distributions. Experimental Economics 2: 197–226 · Zbl 0966.91048 |
| [15] | de Finetti B. (1977) Probabilities of Probabilities. In: Aykac A., Brumat C. (eds) New directions in the application of Bayesian methods. North Holland, Amsterdam, pp 1–10 |
| [16] | Einhorn H., Hogarth R. (1985) Ambiguity and uncertainty in probabilistic inference. Psychology Review 92: 433–461 · doi:10.1037/0033-295X.92.4.433 |
| [17] | Ellsberg D. (1961) Risk ambiguity and the savage axioms. Quarterly Journal of Economics 75: 643–669 · Zbl 1280.91045 · doi:10.2307/1884324 |
| [18] | Frisch D., Baron J. (1988) Ambiguity and rationality. Journal of Behavioral Decision Making 1: 149–157 · doi:10.1002/bdm.3960010303 |
| [19] | Gans N., Knox G., Croson R. (2007) Simple models of discrete choice and their performance in bandit experiments. Manufacturing and Service Operations Management 9: 383–408 · doi:10.1287/msom.1060.0130 |
| [20] | Gilboa I., Schmeidler D. (1989) Maxmin expected utility with a non-unique prior. Journal of Mathematical Economics 18: 141–153 · Zbl 0675.90012 · doi:10.1016/0304-4068(89)90018-9 |
| [21] | Gittins J. (1989) Multi-arm bandit allocation indicies. Wiley, New York · Zbl 0699.90068 |
| [22] | Gittins J., Jones D. et al (1974) A dynamic allocation index for the sequential design of experiments. In: Gani J. (eds) Progress in statistics.. North Holland, Amsterdam, pp 241–266 · Zbl 0303.62064 |
| [23] | Halevy Y. (2007) Ellsberg revisited: An experimental study. Econometrica 75: 503–536 · Zbl 1132.91423 · doi:10.1111/j.1468-0262.2006.00755.x |
| [24] | Hey J. (1987) Still searching. Journal of Economic Behaviorand Organization 8: 137–144 · doi:10.1016/0167-2681(87)90026-6 |
| [25] | Hogarth R., Einhorn H. (1990) Venture theory: A model of decision weights. Management Science 36: 780–803 · doi:10.1287/mnsc.36.7.780 |
| [26] | Kahn B., Sarin R. (1988) Modeling ambiguity in decisions under uncertainty. Journal of Consumer Research 15: 265–272 · doi:10.1086/209163 |
| [27] | Klabinoff P., Marinacci S., Mukerji M. (2009) Recurseive smooth ambiguity preferences. Journal of Economic Theory 144: 930–976 · Zbl 1160.91324 · doi:10.1016/j.jet.2008.10.007 |
| [28] | Meyer R., Shi Y. (1995) Sequential choice under ambiguity: Intuitive solutions to the armed-bandit problem. Management Science 41: 817–834 · Zbl 0843.90005 · doi:10.1287/mnsc.41.5.817 |
| [29] | Norton, D., & Isaac R. (2010). Experts with a conflict of interest: A source of ambiguity?. University of Florida, Department of Economics working paper. |
| [30] | Pratt J., Wise D., Zeckhauser R. (1979) Price differences in almost competitive markets. Quarterly Journal of Economics 94: 189–211 · Zbl 0414.90012 · doi:10.2307/1883191 |
| [31] | Roth A., Malouf M. (1979) Game theoretic models and the role ofinformation in bargaining: An experimental study. Psychological Review 86: 574–594 · doi:10.1037/0033-295X.86.6.574 |
| [32] | Schmeidler D. (1989) Subjective probability and expected utility without additivity. Econometrica 57: 571–587 · Zbl 0672.90011 · doi:10.2307/1911053 |
| [33] | Schotter A., Braunstein Y. (1981) Economic search: An experimental study. Economic Inquiry 19: 1–25 · doi:10.1111/j.1465-7295.1981.tb00600.x |
| [34] | Segal U. (1987) The Ellsberg paradox and risk aversion: An anticipated utility approach. International Economic Review 28: 175–202 · Zbl 0659.90010 · doi:10.2307/2526866 |
| [35] | Steyvers M., Lee M., Wagenmakers E.-L. (2009) A Bayesian analysis of human decision-making on bandit problems. Journal of Mathematical Psychology 53: 168–179 · Zbl 1176.90319 · doi:10.1016/j.jmp.2008.11.002 |
| [36] | Traeger, C. (2010). Subjective risk, confidence and ambiguity. University of California, Berekely Department of Agricultural and Resource Economics Working Paper 1103. |
| [37] | Yates F., Zukowski L. (1976) Characterization of ambiguity in decision making. Behavioral Science 21: 19–25 · doi:10.1002/bs.3830210104 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
