Incentives, search engines, and the elicitation of subjective beliefs: evidence from representative online survey experiments. (English) Zbl 07605543
Summary: A large literature studies subjective beliefs about economic facts using unincentivized survey questions. We devise randomized experiments in a representative online survey to investigate whether incentivizing belief accuracy affects stated beliefs about average earnings by professional degree and average public school spending. Incentive provision does not impact earnings beliefs, but improves school-spending beliefs. Response spikes suggest that the latter effect likely reflects increased online-search activity. Consistently, an experiment that just encourages search-engine usage produces very similar results. Another experiment provides no evidence of experimenter-demand effects. Overall, results suggest a trade-off between increased respondent effort and the risk of inducing online-search activity when incentivizing beliefs in online surveys.
MSC:
| 62-XX | Statistics |
| 91-XX | Game theory, economics, finance, and other social and behavioral sciences |
References:
| [1] | Alesina, Alberto; La Ferrara, Eliana, Preferences for redistribution in the land of opportunities, J. Public Econ., 89, 897-931 (2005) |
| [2] | Alesina, Alberto, Miano, Armando, Stantcheva, Stefanie, 2018a. Immigration and Redistribution. NBER Working Paper 24733. |
| [3] | Alesina, Alberto; Stantcheva, Stefanie; Teso, Edoardo, Intergenerational mobility and preferences for redistribution, Amer. Econ. Rev., 108, 521-554 (2018) |
| [4] | Ansolabehere, Stephen; Meredith, Marc; Snowberg, Erik, Asking about numbers: Why and how, Political Anal., 21, 48-69 (2013) |
| [5] | Arcidiacono, Peter, Ability sorting and the returns to college major, J. Econometrics, 121, 343-375 (2004) · Zbl 1282.62252 |
| [6] | Arcidiacono, Peter; Joseph Hotz, V.; Kang, Songman, Modeling college major choices using elicited measures of expectations and counterfactuals, J. Econometrics, 166, 3-16 (2012) · Zbl 1441.62590 |
| [7] | Arechar, Antonio A.; Gächter, Simon; Molleman, Lucas, Conducting interactive experiments online, Exp. Econ., 21, 99-131 (2018) |
| [8] | Armantier, Olivier; d. Bruin, Wändi B.; Potter, Simon; Topa, Giorgio; van der Klaauw, Wilbert; Zafar, Basit, Measuring inflation expectations, Annu. Rev. Econ., 5, 273-301 (2013) |
| [9] | Armantier, Olivier; Nelson, Scott; Topa, Giorgio; van der Klaauw, Wilbert; Zafar, Basit, The price is right: Updating inflation expectations in a randomized price information experiment, Rev. Econ. Stat., 98, 503-523 (2016) |
| [10] | Armona, Luis, Fuster, Andreas, Zafar, Basit, 2016. Home Price Expectations and Behavior: Evidence from a Randomized Information Experiment. Federal Reserve Bank of New York Staff Report 798. · Zbl 1508.91188 |
| [11] | Attanasio, Orazia, Boneva, Teodora, Rauh, Christopher, 2018. Parental Beliefs About Returns to Different Types of Investments in School Children. HCEO Working Paper 2018-032. |
| [12] | Belfield, Chris; Boneva, Teodora; Rauh, Christopher; Shaw, Jonathan, What drives enrollment gaps in further education? The role of beliefs in sequential schooling decisions, Economica, 87, 490-529 (2020) |
| [13] | Benítez-Silva, Hugo; Buchinsky, Moshe; Chan, Hiu Man; Cheidvasser, Sofia; Rust, John, How large is the bias in self-reported disability?, J. Appl. Econometrics, 19, 649-670 (2004) |
| [14] | Bertrand, Marianne; Mullainathan, Sendhil, Do people mean what they say? Implications for subjective survey data, Amer. Econ. Rev., 91, 67-72 (2001) · Zbl 0989.91057 |
| [15] | Blanco, Mariana; Engelmann, Dirk; Koch, Alexander K.; Normann, Hans-Theo, Belief elicitation in experiments: Is there a hedging problem?, Exp. Econ., 13, 412-438 (2010) · Zbl 1198.91054 |
| [16] | Bleemer, Zachary; Zafar, Basit, Intended college attendance: Evidence from an experiment on college returns and costs, J. Public Econ., 157, 184-211 (2018) |
| [17] | Boneva, Teodora, Rauh, Christopher, 2017. Socio-Economic Gaps in University Enrollment: The Role of Perceived Pecuniary and Non-Pecuniary Returns. HCEO Working Paper 2017-080. |
| [18] | Boneva, Teodora; Rauh, Christopher, Parental beliefs about returns to educational investments - The later the better?, J. Eur. Econom. Assoc., 16, 1669-1711 (2018) |
| [19] | Bullock, John G.; Gerber, Alan S.; Hill, Seth J.; Huber, Gregory A., Partisan bias in factual beliefs about politics, Q. J. Political Sci., 10, 519-578 (2015) |
| [20] | Cavallo, Alberto; Cruces, Guillermo; Perez-Truglia, Ricardo, Inflation expectations learning and supermarket prices: Evidence from survey experiments, Am. Econ. J.: Macroecon., 9, 1-35 (2017) |
| [21] | Clifford, Scott; Jerit, Jennifer, Is there a cost to convenience? An experimental comparison of data quality in laboratory and online studies, J. Exp. Political Sci., 1, 120-131 (2014) |
| [22] | Coibion, Olivier; Gorodnichenko, Yuriy, What can survey forecasts tell us about information rigidities?, J. Political Econ., 120, 116-159 (2012) |
| [23] | Coibion, Olivier; Gorodnichenko, Yuriy; Kumar, Saten, How do firms form their expectations? New survey evidence, Amer. Econ. Rev., 108, 2671-2713 (2018) |
| [24] | Delavande, Adeline; Giné, Xavier; McKenzie, David, Eliciting probabilistic expectations with visual aids in developing countries: How sensitive are answers to variations in elicitation design?, J. Appl. Econometrics, 26, 479-497 (2011) |
| [25] | Delavande, Adeline; Zafar, Basit, University choice: The role of expected earnings, non-pecuniary outcomes, and financial constraints, J. Political Econ., 127, 2343-2393 (2019) |
| [26] | Fuster, Andreas, Perez-Truglia, Ricardo, Zafar, Basit, 2018. Expectations with Endogenous Information Acquisition: An Experimental Investigation. NBER Working Paper 24767. |
| [27] | Gächter, Simon; Renner, Elke, The effects of (incentivized) belief elicitation in public goods experiments, Exp. Econ., 13, 364-377 (2010) |
| [28] | Grewenig, Elisabeth; Lergetporer, Philipp; Simon, Lisa; Werner, Katharina; Woessmann, Ludger, Can online surveys represent the entire population?, CESifo Working Paper, 7222 (2018) |
| [29] | Grigorieff, Alexis, Roth, Christopher, Ubfal, Diego, 2016. Does Information Change Attitudes Towards Immigrants? Representative Evidence from Survey Experiments. IZA Discussion Paper 10419. |
| [30] | Haaland, Ingar, Roth, Christopher, 2017). Beliefs About Racial Discrimination: Representative Evidence. Working Paper. |
| [31] | Horton, John J.; Rand, David G.; Zeckhauser, Richard J., The online laboratory: Conducting experiments in a real labor market, Exp. Econ., 14, 399-425 (2011) |
| [32] | Hoxby, Caroline, Turner, Sarah, 2013. Expanding College Opportunities for High-Achieving, Low Income Students. Stanford Institute for Economic Policy Research Discussion Paper 12-014. |
| [33] | Hurley, Terrance M.; Shogren, Jason F., An experimental comparison of induced and elicited beliefs, J. Risk Uncertain., 30, 169-188 (2005) · Zbl 1123.91337 |
| [34] | Karadja, Mounir; Mollerstrom, Johanna; Seim, David, Richer (and holier) than thou? The effect of relative income improvements on demand for redistribution, Rev. Econ. Stat., 99, 201-212 (2017) |
| [35] | Karagözoğlu, Emin; Kocher, Martin G., Bargaining under time pressure from deadlines, Exp. Econ., 22, 419-440 (2019) |
| [36] | Kaufmann, Katja M., Understanding the income gradient in college attendance in Mexico: The role of heterogeneity in expected returns, Quant. Econ., 5, 583-630 (2014) |
| [37] | Keusch, Florian; Zhang, Chan, A review of issues in gamified surveys, Soc. Sci. Comput. Rev., 35, 147-166 (2017) |
| [38] | Kocher, Martin G.; Schindler, David; Trautmann, Stefan T.; Xu, Yilong, Risk, time pressure, and selection effects, Exp. Econ., 22, 216-246 (2019) |
| [39] | Kuziemko, Ilyana; Norton, Michael I.; Saez, Emmanuel; Stantcheva, Stefanie, How elastic are preferences for redistribution? Evidence from randomized survey experiments, Amer. Econ. Rev., 105, 1478-1508 (2015) |
| [40] | Lergetporer, Philipp; Schwerdt, Guido; Werner, Katharina; West, Martin R.; Woessmann, Ludger, How information affects support for education spending: Evidence from survey experiments in Germany and the United States, J. Public Econ., 167, 138-157 (2018) |
| [41] | Lergetporer, Philipp, Werner, Katharina, Woessmann, Ludger, 2018b. Does Ignorance of Economic Returns and Costs Explain the Educational Aspiration Gap? Evidence from Representative Survey Experiments. CESifo Working Paper 7192. |
| [42] | Lergetporer, Philipp, Werner, Katharina, Woessmann, Ludger, 2018c. Educational Inequality and Public Policy Preferences: Evidence from Representative Survey Experiments. CESifo Working Paper 7192. |
| [43] | Mankiw, N. Gregory; Reis, Ricardo; Wolfers, Justin, Disagreement about inflation expectations, NBER Macroecon. Annu., 18, 209-248 (2003) |
| [44] | Manski, Charles F., Measuring expectations, Econometrica, 72, 1329-1376 (2004) · Zbl 1137.91361 |
| [45] | Manski, Charles F., Survey measurement of probabilistic macroeconomic expectations: Progress and promise, NBER Macroecon. Annu., 32, 411-471 (2018) |
| [46] | Mummolo, Jonathan; Peterson, Erik, Demand effects in survey experiments: An empirical assessment, Am. Political Sci. Rev., 113, 517-529 (2019) |
| [47] | Prior, Markus; Sood, Gaurav; Khanna, Kabir, You cannot be serious: The impact of accuracy incentives on partisan bias in reports of economic perceptions, Q. J. Political Sci., 10, 489-518 (2015) |
| [48] | Puleston, Jon, Improving online surveys, Int. J. Mark. Res., 53, 557-560 (2011) |
| [49] | de Quidt, Jonathan; Haushofer, Johannes; Roth, Christopher, Measuring and bounding experimenter demand, Amer. Econ. Rev., 108, 3266-3302 (2018) |
| [50] | Roth, Christopher, Settele, Sonja, Wohlfart, Johannes, 2019. Public Debt and the Demand for Government Spending and Taxation. Working Paper. |
| [51] | Roth, Christopher; Wohlfart, Johannes, How do expectations about the macroeconomy affect personal expectations and behavior?, Rev. Econ. Stat. (2019), (forthcoming) |
| [52] | Schlag, Karl H.; Tremewan, James; Van der Weele, Joël J., A penny for your thoughts: A survey of methods for eliciting beliefs, Exp. Econ., 18, 457-490 (2015) |
| [53] | Schotter, Andrew; Trevino, Isabel, Belief elicitation in the laboratory, Annu. Rev. Econ., 6, 103-128 (2014) |
| [54] | Schueler, Beth E.; West, Martin R., Sticker shock: How information affects citizen support for public school funding, Public Opin. Q., 80, 90-113 (2016) |
| [55] | Statistisches Bundesamt, Beth E., Bildungsausgaben: Ausgaben je Schülerin und Schüler 2013 (2016), Statistisches Bundesamt: Statistisches Bundesamt Wiesbaden |
| [56] | Statistisches Bundesamt, Beth E., Bildungsausgaben: Ausgaben je Schülerin und Schüler 2014 (2017), Statistisches Bundesamt: Statistisches Bundesamt Wiesbaden |
| [57] | Statistisches Bundesamt, Beth E., Bildungsausgaben: Ausgaben je Schülerin und Schüler 2015 (2018), Statistisches Bundesamt: Statistisches Bundesamt Wiesbaden |
| [58] | Stinebrickner, Ralph; Stinebrickner, Todd R., A major in science? Initial beliefs and final outcomes for college major and dropout, Rev. Econom. Stud., 81, 426-472 (2014) · Zbl 1405.91129 |
| [59] | Trautmann, Stefan T.; van de Kuilen, Gijs, Belief elicitation: A horse race among truth serums, Econom. J., 125, 2116-2135 (2015) |
| [60] | Wang, Stephanie W., Incentive effects: The case of belief elicitation from individuals in groups, Econom. Lett., 111, 30-33 (2011) |
| [61] | Wiswall, Matthew; Zafar, Basit, Determinants of college major choice: Identification using an information experiment, Rev. Econom. Stud., 82, 791-824 (2015) · Zbl 1405.91132 |
| [62] | Zafar, Basit, How do college students form expectations?, J. Labor Econ., 29, 301-348 (2011) |
| [63] | Zimmermann, Florian, The dynamics of motivated beliefs, Amer. Econ. Rev., 110, 337-361 (2020) |
| [64] | Zizzo, Daniel J., Experimenter demand effects in economic experiments, Exp. Econ., 13, 75-98 (2010) · Zbl 1201.91027 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
