Predicting Field Experiment Results in a Lab
| Název práce v češtině: | Predikce výsledků field experimentu v laboratoři |
|---|---|
| Název v anglickém jazyce: | Predicting Field Experiment Results in a Lab |
| Klíčová slova: | laboratory experiments, prediction accuracy, field experiments, behavioral economics |
| Klíčová slova anglicky: | laboratory experiments, prediction accuracy, field experiments, behaviorální ekonomie |
| Akademický rok vypsání: | 2016/2017 |
| Typ práce: | diplomová práce |
| Jazyk práce: | angličtina |
| Ústav: | Institut ekonomických studií (23-IES) |
| Vedoucí / školitel: | doc. PhDr. Lubomír Cingl, Ph.D. |
| Řešitel: | skrytý - zadáno vedoucím/školitelem |
| Datum přihlášení: | 09.11.2016 |
| Datum zadání: | 09.11.2016 |
| Datum a čas obhajoby: | 21.06.2017 08:30 |
| Místo konání obhajoby: | Opletalova - Opletalova 26, O206, Opletalova - místn. č. 206 |
| Datum odevzdání elektronické podoby: | 19.05.2017 |
| Datum proběhlé obhajoby: | 21.06.2017 |
| Oponenti: | prof. PhDr. Ladislav Krištoufek, Ph.D. |
| Kontrola URKUND: | |
| Seznam odborné literatury |
| Ben-David, I., Graham, J. R., & Harvey, C. R. (2013). Managerial miscalibration. The Quarterly Journal of Economics, 128(4), pp. 1547-1584.
Bott, K., Cappelen, A. W., Sørensen, E. Ø., & Tungodden, B. (2014). You’ve got mail: A randomised field experiment on tax evasion. NHH Norwegian School of Economics. Coffman, L., & Niehaus, P. (2014). Pathways of persuasion. Working paper. DellaVigna, S., & Pope, D. (2016). Predicting Experimental Results: Who Knows What? (No. w22566). National Bureau of Economic Research. Fellner, G., Sausgruber, R., & Traxler, C. (2013). Testing enforcement strategies in the field: Threat, moral appeal and social information. Journal of the European Economic Association, 11(3), 634-660. Fischbacher, U. (2007). z-Tree: Zurich toolbox for ready-made economic experiments. Experimental economics, 10(2), 171-178. Sanders, M., Mitchell, F., & Chonaire, A. N. (2015). Just Common Sense? How well do experts and lay-people do at predicting the findings of Behavioural Science Experiments. Working paper. Schram, A. (2005). Artificiality: The tension between internal and external validity in economic experiments. Journal of Economic Methodology, 12(2), 225-237. |
| Předběžná náplň práce |
| MOTIVATION
Can be results of a field experiment predicted in a laboratory experiment? With growing interest of general public in economic experiments it could be beneficial to forecast results of large field studies in a simple lab environment, obtaining first estimates of effectiveness of individual treatments and thus minimize potential unnecessary expenses. This thesis has therefore two related dimensions. First, it challenges an external validity of laboratory experiments and second, it follows and expands the literature on prediction accuracy in economics. With a recent upsurge of field experiments, the topic of external validity of laboratory experiments has been widely discussed. Indeed, laboratory experiments are often criticized for excessive artificiality that leads to results different from real world behavior (see e.g. discussion in Schram, (2005)). Therefore, in this thesis we would like to scrutinize the extent to which predictions made in our lab experiment could be related to the actual effectiveness of field experiment treatments. Economics has also a long tradition in studying accuracy of predictions. One example can be found in the literature on prediction accuracy in macroeconomics and finance, (e.g. Ben-David et al. (2013)). Other example constitute prediction markets, i.e. markets where participants trade in contracts whose payoff depend on unknown future events. Often, the information collected in these markets is more accurate than the forecasts generated by more traditional methods such as opinion polls or expert judgment. Newly, the attention has been drawn to forecasting of experimental results. DellaVigna & Pope (2016) analyze to which extent academic experts can anticipate the impact of different treatments and compare their forecasts to the sample of non-experts including students and online sample. Other examples can be found in Coffman & Niehaus (2014) who survey 7 experts on persuasion or in Sanders et al. (2015) who ask 25 faculty and students from two universities to predict results of 15 experiments organized by the UK Nudge Unit. HYPOTHESES Based on the paper by DellaVigna & Pope (2016) we state following research hypotheses: H1: The average forecast predicts the experimental results quite well. H2: There is some wisdom-of-crowds effect, i.e. the average forecast outperforms the individual ones. H3: Accuracy defined as rank ordering treatments outperforms accuracy defined in absolute numbers. METHODS We will run a controlled laboratory experiment with a student subject pool. We will analyze the data using standard forecast accuracy measures, i.e. mean absolute (squared) errors and their cumulative distribution functions. CONTRIBUTION We would like to contribute to the growing body of literature focused on forecasting of experimental results. This is a relevant up-to-date topic since we can easily collect valuable information if it turns out that lab predictions dispose of a certain predictive value. Moreover, this thesis examines the extent to which laboratory results reflect field behavior, adding another important argument into a discussion of external validity. The fact that the topic of forecasting experimental results is rather new and unexplored also increases its relevance. |
| Předběžná náplň práce v anglickém jazyce |
| MOTIVATION
Can be results of a field experiment predicted in a laboratory experiment? With growing interest of general public in economic experiments it could be beneficial to forecast results of large field studies in a simple lab environment, obtaining first estimates of effectiveness of individual treatments and thus minimize potential unnecessary expenses. This thesis has therefore two related dimensions. First, it challenges an external validity of laboratory experiments and second, it follows and expands the literature on prediction accuracy in economics. With a recent upsurge of field experiments, the topic of external validity of laboratory experiments has been widely discussed. Indeed, laboratory experiments are often criticized for excessive artificiality that leads to results different from real world behavior (see e.g. discussion in Schram, (2005)). Therefore, in this thesis we would like to scrutinize the extent to which predictions made in our lab experiment could be related to the actual effectiveness of field experiment treatments. Economics has also a long tradition in studying accuracy of predictions. One example can be found in the literature on prediction accuracy in macroeconomics and finance, (e.g. Ben-David et al. (2013)). Other example constitute prediction markets, i.e. markets where participants trade in contracts whose payoff depend on unknown future events. Often, the information collected in these markets is more accurate than the forecasts generated by more traditional methods such as opinion polls or expert judgment. Newly, the attention has been drawn to forecasting of experimental results. DellaVigna & Pope (2016) analyze to which extent academic experts can anticipate the impact of different treatments and compare their forecasts to the sample of non-experts including students and online sample. Other examples can be found in Coffman & Niehaus (2014) who survey 7 experts on persuasion or in Sanders et al. (2015) who ask 25 faculty and students from two universities to predict results of 15 experiments organized by the UK Nudge Unit. HYPOTHESES Based on the paper by DellaVigna & Pope (2016) we state following research hypotheses: H1: The average forecast predicts the experimental results quite well. H2: There is some wisdom-of-crowds effect, i.e. the average forecast outperforms the individual ones. H3: Accuracy defined as rank ordering treatments outperforms accuracy defined in absolute numbers. METHODS We will run a controlled laboratory experiment with a student subject pool. We will analyze the data using standard forecast accuracy measures, i.e. mean absolute (squared) errors and their cumulative distribution functions. CONTRIBUTION We would like to contribute to the growing body of literature focused on forecasting of experimental results. This is a relevant up-to-date topic since we can easily collect valuable information if it turns out that lab predictions dispose of a certain predictive value. Moreover, this thesis examines the extent to which laboratory results reflect field behavior, adding another important argument into a discussion of external validity. The fact that the topic of forecasting experimental results is rather new and unexplored also increases its relevance. |