Resampling-based statistical inference methods for the evaluation of complex models in biometrics - Part II
Final Report Abstract
Resampling methods (including bootstrap and permutation tests) are well established inference methods in (bio)-medical sciences. They are typically used in small sample size situations or when general asymptotic results are not even applicable. However, how and when do these methods work? Are they always applicable —no matter about the general distributional setup? Permutation methods in particular are said to work only if data is exchangeable. First and foremost, most real trials (especially observational studies and lab experiments) are complex experimental designs involving several groups, repeated measures, missing values and clustered data. Existing statistical methodology was restricted to specific distributional assumptions, dependency patterns and large samples. They could not handle missing values and clustered data appropriately; indeed, most methods were restricted to complete case analysis only. Furthermore, solutions for small samples in complex designs did only exist to a certain extent. Therefore, this project aimed to develop robust and flexible statistical inference methods for various biomedical designs with small sample sizes, without imposing restrictive assumptions on data distribution models. We investigated the use of different resampling methods as well as other small sample size approximations in particular to develop statistical inference methods for various complex designs. Throughout the project, we developed new parametric (mean-based) and purely nonparametric statistical methods for the analysis of general factorial repeated measures and multivariate designs involving independent groups observed at multiple time points. They include novel point estimators of Wilcoxon-Mann-Whitney effects in factorial clustered data designs with missing values. We did not restrict the missing value mechanism to missing completely at random data, but could relax the assumptions to missing at random models. Furthermore, we focused on modern multiple contrast test methodologies (so-called maximum tests) for testing arbitrary but research specific null hypotheses. The methods can further be used to test standard null hypotheses of no main group effect, no main time effect and no interaction effect between group and time, and differences from baseline, in factorial designs, respectively. The validity of all methods developed is underpinned in rigorous mathematical investigations herewith answering the aforementioned questions raised in which situations the resampling methods work. We investigated the quality of the methods in extensive simulation studies finding the minimal needed sample size that guarantees control of the type 1 error rate. It turns out that mostly all of the newly developed wild bootstrap tests control the nominal level even in (very) small samples of n=10 (depending on the complexity of the design). All in all, we reached all research goals summarized in the work packages of the initial proposal and believe that the project has made several significant contributions to statistical sciences and especially biostatistics. The project majorly contributed to education and workforce development in biostatistical science, involving graduate research assistants, undergraduate students, and visiting doctoral students and researchers. Within this project, we published several joint publications in international high-quality peer-reviewed journals. All results are implemented in freely available R software packages GFD, MANOVA.RM, nparLD and rankFD. The research results are a solid foundation for future joint research. Ongoing discussions have generated additional research questions related to other biostatistical problems, which are being addressed in a subsequent joint renewal proposal.
Publications
-
The Behrens–Fisher problem with covariates and baseline adjustments. Metrika, 83(2), 197-215.
Cao, Cong; Pauly, Markus & Konietschke, Frank
-
The nonparametric Behrens‐Fisher problem with dependent replicates. Statistics in Medicine, 38(25), 4939-4962.
Roy, Akash; Harrar, Solomon W. & Konietschke, Frank
-
Ranks and Pseudo‐ranks—Surprising Results of Certain Rank Tests in Unbalanced Designs. International Statistical Review, 89(2), 349-366.
Brunner, Edgar; Konietschke, Frank; Bathke, Arne C. & Pauly, Markus
-
Small sample sizes: A big data problem in high-dimensional data analysis. Statistical Methods in Medical Research, 30(3), 687-701.
Konietschke, Frank; Schwab, Karima & Pauly, Markus
-
A comprehensive treatment of quadratic-form-based inference in repeated measures designs under diverse asymptotics. Electronic Journal of Statistics, 15(1).
Sattler, Paavo
-
Asymptotic‐based bootstrap approach for matched pairs with missingness in a single arm. Biometrical Journal, 63(7), 1389-1405.
Amro, Lubna; Pauly, Markus & Ramosaj, Burim
-
Permutation tests are robust and powerful at 0.5% and 5% significance levels. Behavior Research Methods, 53(6), 2712-2724.
Noguchi, Kimihiro; Konietschke, Frank; Marmolejo-Ramos, Fernando & Pauly, Markus
-
Ranking procedures for repeated measures designs with missing data: Estimation, testing and asymptotic theory. Statistical Methods in Medical Research, 31(1), 105-118.
Rubarth, Kerstin; Pauly, Markus & Konietschke, Frank
-
Estimation and Testing of Wilcoxon–Mann–Whitney Effects in Factorial Clustered Data Designs. Symmetry, 14(2), 244.
Rubarth, Kerstin; Sattler, Paavo; Zimmermann, Hanna Gwendolyn & Konietschke, Frank
-
On the Relation between Prediction and Imputation Accuracy under Missing Covariates. Entropy, 24(3), 386.
Ramosaj, Burim; Tulowietzki, Justus & Pauly, Markus
