External validity in economic experiments: can students be used to approximate the behavior of farmers?

• Applicant: Dr. Sven Grüner
• Subject Area: Agricultural Economics, Agricultural Policy, Agricultural Sociology
• Term: from 2018 to 2022
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 388911356

External validity is one of the most crucial aspects in the meaningful interpretation of eco-nomic experiments. The critical question is which predictive power an experimentally observed behavior has for a particular real-world context and social group of interest. This is a poignant question because, for the sake of convenience, it has become common practice to use students as substitute experimental subjects. The robustness and meaningfulness of experimental studies, especially when conducted with students, can be impaired by a number of problems. The first one concerns the replicability of the behavioral effects that are observed in a sample of experimental subjects contingent on experimentally varied conditions. The second concern is whether the experimental design itself is suited for making experimental subjects reveal their true preferences, or whether distortions, such as a social desirability bias caused by insufficient incentives, linger. Given the widespread use of students as experimental subjects, a further concern is related to the transferability of results: what can we scientifically infer from the experimental behavior of the student population for the behavior of the social group of interest?Comprehensive methodological studies that address the generalizability of findings from economic experiments in the light of the entirety of these problems are scarce. With a view to the use of experiments in agricultural economics, no systematic studies are known that investigate the robustness and especially the transferability issue. Against this background, this project is aimed at finding out which contribution "robustness-promoting" approaches can make to a more informed interpretation of experimental findings. For one thing, we look at new empirical and statistical strategies for improving statistical inference, such as preregistering study designs, accounting for multiple testing problems, and carrying out power analyses and replications. With a view to experimental design, we furthermore study whether and how systematically varied incentives affect behavior. This is an especially interesting issue because experimental economists, contrary to experimental psychologists, contend that monetary incentives are needed to make experimental subjects reveal their true preferences.We use multi-period business simulation games, in which students and farmers alike run virtual farms and make production program decisions in two contexts, as experimental device to study the following questions: (1) Is there a gap, or how big is the gap, between students' and farmers' behavior contingent on systematically varied incentives? (2) Which factors can explain eventually observed behavioral differences between students and farmers? (3) Which contribution to external validity can be provided by new strategies such as the preregistration of studies, the consideration of multiple testing, and power analysis and replication?

• DFG Programme: Research Grants