Selective reporting and the evolving research landscape in economics
Final Report Abstract
This project contributes to meta-research in economics with the aim of improving the reliability of published research. The main accomplishment of this project is the development of DORIS (Diagnosis of reporting errors in scraped tables), a software tool that automatically gathers statistical tests from economics articles. By automating the manual coding process, this tool can benefit future meta-research and the peer review process. Applying DORIS to the top 50 economics journals resulted in a data set comprising more than 600,000 statistical tests. We use this unique data and other data to (1) assess the extent of statistical reporting errors in economics at a large scale, (2) analyze the extent of selective reporting in economics, its sources and the effectiveness of data and code availability policies implemented by journals, (3) explore the evolving research landscape in economics, and (4) analyze the relationship between statistical findings and the language of journal articles, such as readability and the use of superlatives.
Publications
-
THE (NON‐)SIGNIFICANCE OF REPORTING ERRORS IN ECONOMICS: EVIDENCE FROM THREE TOP JOURNALS. Journal of Economic Surveys, 35(1), 348-373.
Pütz, Peter & Bruns, Stephan B.
-
Statistical reporting errors in economics. Center for Open Science.
Bruns, Stephan; Herwartz, Helmut; Ioannidis, John P.A.; Islam, Chris-Gabriel & Raters, Fabian H. C.
-
Estimating the extent of selective reporting: An application to economics. Research Synthesis Methods, 15(4), 590-602.
Bruns, Stephan B.; Deressa, Teshome K.; Stanley, T. D.; Doucouliagos, Chris & Ioannidis, John P. A.
-
Footprint of publication selection bias on meta‐analyses in medicine, environmental sciences, psychology, and economics. Research Synthesis Methods, 15(3), 500-511.
Bartoš, František; Maier, Maximilian; Wagenmakers, Eric‐Jan; Nippold, Franziska; Doucouliagos, Hristos; Ioannidis, John P. A.; Otte, Willem M.; Sladekova, Martina; Deresssa, Teshome K.; Bruns, Stephan B.; Fanelli, Daniele & Stanley, T. D.
