Assessing and accounting for between-sample variation of psychometric measurement models

• Applicant: Dr. Felix Fischer
• Subject Area: Personality Psychology, Clinical and Medical Psychology, Methodology
• Term: Funded in 2019
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 426668949

Current psychometric models of patient-reported outcomes (PROs) typically assume that model parameters are not random but fixed. In analyses of PROs, where such models are used in independent samples such as differential item functioning, factor analysis or the development of construct-based scales, it has been frequently observed that model parameters usually vary across samples. As these differences are often statistically significant, but practically irrelevant, it is promising to explicitly model them as random effects in order to improve PRO measurement and related inference.The objective of this project is to investigate this phenomena across 56 distinct studies with 17.357 participants, collected within an individual participant data meta-analysis of the PHQ-9. Between-sample variance of model parameters will be assessed and Bayesian models will be used to incorporate such information in analysis. More specifically, the aims of the project are (1) to investigate between-samples variability of model parameter estimates both in confirmatory factor analyis and item-response theory, (2) to identify variables on study level such as sample size, country of origin, disease groups, age and gender, which are associated with systematic and relevant variation of model parameters, (3) to estimate depression severity within a Bayesian item-response theory model which incorporates information about random parameter variation across samples and to (4) to investigate impact of such a model on predictive validity by comparing the diagnostic accuracy of estimates of depression severity derived by these models with PHQ-9 sum scores commonly used.This project will provide insights into the nature of systematic and unsystematic between-sample variation of psychometric models for a frequently used and highly relevant PRO, the PHQ-9. Furthermore, it will provide means to account for minor, practically irrelevant model differences in statistical analysis of PROs and therefore inform development of future measurement models, e.g. in the development of instrument-independent, construct-based scales for PROs.

• DFG Programme: Research Fellowships
• International Connection: Canada
• Hosts: Professor Dr. Carl Falk; Professor Dr. Brett Thombs