SPP 2317:  META-REP: A Meta-scientific Programme to Analyse and Optimise Replicability in the Behavioural, Social, and Cognitive Sciences

• Subject Area: Social and Behavioural Sciences ; Biology ; Humanities ; Computer Science, Systems and Electrical Engineering ; Medicine
• Term: since 2021
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 441890184

Meta-scientific research on the replicability, the reproducibility, and the generalizability of empirical findings has been – and continues to be – a “hot topic” in the social, cognitive, and behavioral sciences. New journals on meta-science have been launched; established journals are devoting “special issues” to replication projects; new scientific communities, institutes, and initiatives have been established; and many journals and funding agencies have changed their policies in order to increase the visibility and the relevance of replicability, reproducibility, and generalizability. The DFG-funded Priority Program “META-REP” (SPP 2317) is contributing visibly to these developments by addressing three overarching questions using (meta-)scientific methods and approaches: (1) What do we mean when we talk about “replicability”? (the “WHAT” question); (2) Why are replication rates so low and variable? (the “WHY” question); and (3) What can we do to increase replication rates? (the “HOW” question). In the first funding phase of META-REP (2021-2024), the 15 funded projects have contributed significantly to answering these questions, and the coordination project has done its best to coordinate, support, and incentivize these efforts, for instance, by (1) installing four work groups (“task forces”) dealing with overarching meta-scientific questions, (2) funding cross-project research endeavors (“treasure box projects”), (3) organizing events within the program (retreats, lecture series, program-specific workshops), (4) organizing an international meta-science conference in 2024, (5) providing IT services to projects and PIs, (6) collaborating with meta-scientific initiatives beyond META-REP, and (7) conducting meta-scientific research itself. The coordination project aims to continue and intensify these activities in the second funding phase. More specifically, we aim to make META-REP even more visible in the international (meta-)scientific community, strengthen collaborative ties with partners around the world, and providing platforms for scientific exchange about meta-science in the social, cognitive, and behavioral sciences.

• DFG Programme: Priority Programmes
• International Connection: Australia, Austria, Netherlands, Sweden, Switzerland, United Kingdom, USA

Projects

• A large-scale collaborative assessment of the replicability and robustness of EEG research (Applicants Busch, Niko ;  Pavlov, Ph.D., Yuri )
• A principled Bayesian workflow to study robust individual differences in cognitive science (Applicant Haaf, Ph.D., Julia M. )
• Addressing the generalizability crisis with large language models (Applicant Wulff, Dirk )
• Applying Principles from the Validity-Based Framework of Replication in Metastudies to Test Causal Relations (Applicant Jekel, Marc )
• Cognitive modelling: Blessing or curse for replicability in Psychology? (Applicants Rausch, Manuel ;  Zehetleitner, Michael )
• Conceptual Replications – Guidelines for implementation and factors influencing replicability across different fields in psychology (Applicants Gast, Anne ;  Pohl, Steffi ;  Sengewald, Marie-Ann ;  Twardawski, Mathias )
• Coordination Funds (Applicant Gollwitzer, Mario )
• Enhancing Reproducibility and Robustness of Observational Social Science Research (Applicants Auspurg, Katrin ;  Brüderl, Josef )
• Facing the replication crisis in machine learning modeling (Applicants Jankowsky, Kristin ;  Schroeders, Ulrich )
• Fostering Proactive Replicability in Computational Communication Science via Frontloading Effort and Automating Protocols (Applicants Breuer, Johannes ;  Haim, Mario )
• From “Academic works” to “Working in academia”: Analyzing reform proposals in an agent-based modelling approach (Applicant Schönbrodt, Felix )
• METEOR 2.0 – MastEring community knowledge-driven robusTnEss analysis in cOgnitive neuRoscience (Applicants Debener, Stefan ;  Gießing, Carsten ;  Hildebrandt, Andrea ;  Thiel, Christiane M. )
• Models, Measures, Moderators - Exploring and Explaining Heterogeneity in Psychological Replications (Applicant Heene, Moritz )
• Replicability and Robustness of Instrumental Variables and Randomized Controlled Trials in Economics (Applicant Ankel-Peters, Jörg )
• Standardization of Behavior Research Methods as a Determinant of Replicability (Applicant Lennartz, Ruben )
• SYNTH: Evaluating Transformer Models to Generate Synthetic Replications, Nomological Nets, and Peer Reviews (Applicants Hommel, Björn ;  Lennartz, Ruben )
• Systematising Theory Building (Applicant Schmalz, Xenia )
• The reproducibility and robustness of secondary analyses in educational research: The role of publication bias and researcher degrees of freedom (Applicants Jansen, Malte ;  Kocaj, Aleksander )
• The role of measurement in the replicability of empirical findings (Applicants Frick, Susanne ;  Wetzel, Eunike )
• The Role of Theory in Understanding and Resolving the Reliability Crisis (Applicant Breznau, Ph.D., Nate )
• Theory Specification and Replicability (Applicant Glöckner, Andreas )

• Spokesperson: Professor Dr. Mario Gollwitzer