Hierarchische MPT Modellierung - Methodische Vergleiche und Anwendungsrichtlinien
Zusammenfassung der Projektergebnisse
In this scientific network, we systematically explored the impact of different estimation methods on parameter estimation in multinomial processing tree (MPT) models, which are suited for the analysis of categorical response data from a variety of psychological paradigms. To this end, the network brought together psychologists working primarily on the methodological development of MPT modeling with those primarily focusing on empirical applications of these models to conduct a large-scale reanalysis. The network members defined a multiverse of nine MPT estimation approaches resulting from different combinations of the statistical framework (frequentist or Bayesian) and the level of data aggregation (complete, partial or no pooling), with a subgroup developing the R package MPTmultiverse allowing to conduct such a multiverse analysis on a given data set with one function call. After a large-scale literature search, documented in accordance with the PRISM guidelines, the network members gathered and reanalyzed a total of 164 independent data sets that were available on the individual level. The core result of the integration and comparison of these estimates across the MPT multiverse is that although there is overall large agreement between the methods, nontrivial deviations occurred and could only in part be explained by a set of theoretically and empirically motivated moderators. Therefore, a multiverse approach is advised to check if the present data set is sensitive to modeling choices. The second objective of the network, the derivation of application guidelines for research including MPT modeling, could not be fully achieved because—somewhat surprisingly, but underscoring the importance of the conducted empirical reanalysis—the moderators identified in previous simulation studies did not prove very useful in explaining deviations across estimation approaches in these empirical data sets. Some deviations appeared to be rather idiosyncratic to the specific MPT model. Unfortunately, the Covid-19 pandemic did not allow for the final two network meetings to take place that would have been devoted to developing such modelspecific application guidelines. Nonetheless, the resulting core article from the reanalysis (Singmann et al., 2023) offers some application guidelines based on the model-independent moderator identified to have at least some impact in the empirical reanalysis. Furthermore, the discussions within the network inspired a simulation study and a more formal treatment and exploration of different types of aggregation variance of MPT models. The R package MPTmultiverse, developed for the network’s reanalysis, is publicly available on CRAN and has already been used in one published empirical study. Overall, the network’s outputs will contribute to increasing the transparency of “researcher degrees of freedom” in cognitive modeling results.
Projektbezogene Publikationen (Auswahl)
-
A Bayesian and Frequentist Multiverse Pipeline for MPT models – Applications to Recognition Memory. Paper presented at MathPsych 2018, Madison, Wisconsin, USA
Singmann, H.; Heck, D. W.; Kapetaniou, G. E.; Groß, J. & Kuhlmann, B. G.
-
MPTmultiverse: Multiverse Analysis of Multinomial Processing Tree Models. CRAN: Contributed Packages.
Singmann, Henrik; Heck, Daniel W. & Barth, Marius
-
A Bayesian and Frequentist Multiverse Pipeline for MPT models – Applications to Recognition Memory. Paper presented at TeaP 2019, London, UK
Singmann, H.; Heck, D. W.; Barth, M.; Groß, J. & Kuhlmann, B. G.
-
New Results from the Bayesian and Frequentist MPT Multiverse. Paper presented at MathPsych 2019, Montreal, Canada
Singmann, H.; Heck, D. W.; Barth, M.; Groß, J. & Kuhlmann, B. G.
-
Parameter estimation approaches for multinomial processing tree models: A comparison for models of memory and judgment. Journal of Mathematical Psychology, 98, 102402.
Groß, Julia & Pachur, Thorsten
-
Parameter agreement and sources of disagreement across the Bayesian and frequentist MPT multiverse. Paper presented at virtual MathPsych 2021
Singmann, H.; Groß, J. & Kuhlmann, B. G.
-
Parameter Agreement and Sources of Disagreement Across the Bayesian and Frequentist MPT Multiverse. Talk presented at the 15th Conference of the Section Methods & Evaluation, Mannheim, Germany
Groß, J.; Kuhlmann, B. G. & Singmann, H.
-
Multinomial Processing Tree Models of Cognition: Aggregation Invariance Properties. Talk presented at the 63rd Annual Meeting of the Psychonomic Society, 2022, Boston, MA, USA
Erdfelder, E.; Quevedo Pütter, J. & Schnuerch, M.
-
On Aggregation Invariance of Multinomial Processing Tree Models.
Erdfelder, Edgar; Quevedo, Pütter Julian & Schnuerch, Martin
-
Evaluating the robustness of parameter estimates in cognitive models: A meta-analytic review of multinomial processing tree models across the multiverse of estimation methods. Psychological Bulletin, 150(8), 965-1003.
Singmann, Henrik; Heck, Daniel W.; Barth, Marius; Erdfelder, Edgar; Arnold, Nina R.; Aust, Frederik; Calanchini, Jimmy; Gümüsdagli, Fabian E.; Horn, Sebastian S.; Kellen, David; Klauer, Karl C.; Matzke, Dora; Meissner, Franziska; Michalkiewicz, Martha; Schaper, Marie Luisa; Stahl, Christoph; Kuhlmann, Beatrice G. & Groß, Julia
