Average and Conditional Treatment Effects on Binary Outcomes

• Applicant: Dr. Christoph Kiefer
• Subject Area: Personality Psychology, Clinical and Medical Psychology, Methodology
• Term: since 2023
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 522862825

This project proposal is concerned with the computation of differential effects of psychological interventions on binary outcomes – more precisely, we want to develop a statistical framework to investigate average and conditional effects of a treatment or intervention based on binary (e.g., logistic) regression models. In recent years, examining differential effects of psychological interventions received growing attention, which is strongly related to the research fields of “personalised medicine” or “differential indication”. Binary outcomes are common in this research area and include, for example, remission, treatment success, reaching a cut-off for clinically significant change, or occurrence of side effects; all of which can depend on individuals' social, biological, and health preconditions and their interactions with the treatment. The proposed approach answers three major questions regarding the shortcomings of earlier approaches evaluating treatment effectiveness on binary outcomes: 1. Why are odds ratios problematic? In binary regression models it is common to inspect treatment effects in form of risk ratios or odds ratios. While recent research suggests that these ratio effects are often misinterpreted in applied research, it is often cautioned that these ratio effects might not even have a causal interpretation, even in randomized experiments. Thus, our proposed approach is based on modern causality theories, which provide unambiguous definitions of causal effects as differences between conditional expectations under treatment and under control, but also illustrates how causally interpretable ratio effects can be defined. 2. Why should we account for sampling variability? Traditional approaches examining treatment effects on binary outcomes, treat the size of treatment groups and values of observed covariates as fixed-by-design, that is, predetermined by the experimenter. If in fact group sizes and covariates are not predetermined – as is the case in most psychological studies – this assumption can lead to too liberal statistical inference, that is, an inflation of significant results when in fact no effect is present. The proposed approach offers an alternative by treating both group sizes and covariates as randomly sampled variables, which can substantially improve statistical inferences. 3. Why is it important to control for measurement error? Many psychological constructs-of-interest (e.g., depression, intelligence) are measured with error. Ignoring measurement error in potential moderators might lead to biased estimation of both the regression coefficients and the corresponding treatment effects and, additionally, decreases the power of detecting potential moderators. Our proposed approach allows to account for measurement error in covariates and thus more accurate estimates. Thus, the overarching goal of this project is to foster reliable evaluation of differential effects on binary outcomes in psychological settings.

• DFG Programme: Research Grants
• International Connection: Belgium
• Cooperation Partner: Professor Dr. Yves Rosseel
• Co-Investigator: Professor Dr. Axel Mayer