Servicenavigation

Project Details

Back

Projekt Print View

Enrichment of the knowledge and methodology to evaluate the transitivity and consistency assumptions in complex networks of interventions

Subject Area Epidemiology and Medical Biometry/Statistics
Term from 2021 to 2025
Project identifier Deutsche Forschungsgemeinschaft (DFG) - Project number 462260733
 
Final Report Year 2025

Final Report Abstract

Transitivity is the cornerstone of network meta-analysis (NMA) for ensuring valid effects between treatments that are not directly compared in any study. There is a rich literature on the importance and notion of transitivity; however, there is currently a shortage of methods to evaluate the validity of transitivity transparently. To this aim, a systematic literature survey of 721 published systematic reviews with NMA was initially performed to provide empirical evidence on the reporting and evaluation quality of transitivity. Although the reporting quality of transitivity has slightly improved, there is a low awareness of this assumption in general. Most systematic reviews evaluated the plausibility of transitivity using methods for consistency assessment rather than methods targeting the PICO features of the included studies that underlie the definition of transitivity. This limitation inspired the introduction of established unsupervised methods to investigate transitivity. Specifically, the inter-study dissimilarities were calculated using the weighted Gower's dissimilarity coefficient based on study-level aggregate participant and methodological characteristics reported across studies. Then, the inter-study dissimilarities were used to (i) develop an index of inter-comparison dissimilarities, measured in the interval 0% (complete similarity) to 100%, and (ii) apply hierarchical agglomerative clustering to detect comparisons, whose studies were arranged into multiple clusters, raising concerns for potential intransitivity. The index of inter-comparison dissimilarities was empirically evaluated using a database of studylevel aggregate characteristics extracted from 209 published systematic reviews. The evaluation revealed a wide range of inter-comparison dissimilarities, pointing to potential intransitivity, which should be expected. This research study also revealed several demerits in using statistical tests to evaluate transitivity since fewer datasets could be used, with fewer characteristics due to testing-related issues. This empirical study inspired the creation of the tracenma R package, a database of 217 published systematic reviews with NMA, including 4 to 35 study-level aggregate characteristics. Methodological characteristics dominated the database, followed by clinical, which were subject to most missing data. Some characteristics required transformations for being reported inconsistently across the studies. The tracenma database is aimed at further developing and appraising the methodology for transitivity assumption. Occasioned by the low power of local inconsistency tests and misinterpreting large p-values as evidence of consistency, an interpretation index of inconsistency was proposed based on the Kullback-Leibler divergence measure. A threshold of acceptably low inconsistency was also developed to aid in detecting nodes with material inconsistency when statistical tests are inconclusive. The proposed index was applied to 57 networks with a binary outcome using the Bayesian node-splitting approach, and conclusions were compared with those based on whether the 95% credible interval of the inconsistency included zero. The empirical study revealed more split nodes with potential material inconsistency using the proposed index and raised caution that true inconsistency may be concealed in sparse networks with substantial statistical heterogeneity.

Publications

 
 

Additional Information

© 2026 DFG Disclaimer / Copyright / Privacy Policy

Textvergrößerung und Kontrastanpassung