Project Details
Projekt
Theory Specification and Replicability
Applicant
Professor Dr. Andreas Glöckner
Subject Area
Social Psychology, Industrial and Organisational Psychology
Term
since 2024
Project identifier
Deutsche Forschungsgemeinschaft (DFG) - Project number 546418746
Studies in the 2010s have shown that a high proportion of empirical results published in high-ranking journals at the time could not be replicated. This finding has led to debates about possible reasons for the low replication rates and about methodological standards and practices in research in psychology and other sciences. Initial discussions and initiatives focused on improving standards of transparency and openness on the one hand and on the potential context dependence of effects on the other. The developed standards of preregistration require researchers to define more precise hypotheses and operationalizations a priori. This revealed a more fundamental problem. In some areas of psychology, theories are sometimes rather vaguely specified. This underspecification of theories and the resulting weak logical link between theories and empirical tests can contribute to low replication rates and an increased false positive rate. This could be due to a high degree of leeway in the derivation of hypotheses, the selection of operationalizations and the performance of statistical tests. In the planned project, this relationship between replicability and the degree of theoretical specification of theories will be investigated empirically. For this purpose, a methodology for formal theory specification will be developed that is particularly adapted to the replication context. This will be used to specify 50 prominent theories in social psychology and to analyze the relationship between the properties of theories - especially their degree of specification - and the replicability of empirical findings that test these theories. We will estimate replicability using a z-curve analysis and also use other methods to test the robustness of this estimate. A methodology based on web-scraping and large language models will be developed to automatically identify relevant articles that test aspects of the relevant theories and to code further potential control and moderator variables. Finally, based on the theory specifications, an Open Theory Database will be developed and made available online to the research community. The database and the methodology for theory specification and selection of relevant articles will provide the technical and conceptual basis for more efficient cumulative theory development and simplify systematic analyses of theories and evidence.
DFG Programme
Priority Programmes
International Connection
Austria
Cooperation Partner
Professorin Dr. Susann Fiedler
