Time to fill the gaps: a mixed-methods study into the role of the time variable in the construct of computer-administered C-Tests in three languages

• Applicant: Professorin Dr. Anastasia Drackert
• Subject Area: Applied Linguistics, Computational Linguistics
• Term: since 2021
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 462766474

C-Tests are highly-efficient tools for measuring global language proficiency used for placement and screening purposes, in second language acquisition research (SLA) and educational monitoring (Grotjahn, 2019). In a C-Test, testees are required to fill the gaps in partially deleted words. The new time-reduced speeded C-Test (S-C-Test), appears to be even more promising in terms of efficiency, but has not yet been sufficiently investigated. Grothjahn (2010) argues that, in contrast to a canonical C-Test with a generous time limit that primarily measures the amount of learners’ declarative and procedural knowledge, a S-C-Test additionally gauges the level of automaticity of their skills and the efficiency of information processing. Accordingly, a S-C-Test is expected to correlate higher with listening and speaking skills than a canonical C-Test but lower with writing and reading. Consequently, a partially different construct is hypothesized to underlie a S-C-Test. Although initial findings on the role of time in the construct are available (Fadaeipour & Zohoorian, 2017; Zimmermann, 2019), many questions remain unanswered, especially for computerized C-tests. In particular, we do not know how the time variable influences test reliability and learners’ scores depending on their proficiency level as well as what role different components (e.g., declarative knowledge) play in the construct of differently timed C-Tests. Importantly, no systematic evidence is available on the impact of time on the cognitive processes elicited by the test and realized in the strategies used by learners. Given the crucial role the time variable is hypothesized to play in the construct, the proposed multilingual study aims to specifically investigate this role in a comprehensive way to allow for a higher degree of generalizability of the results for 1) different levels of proficiency, 2) multiple languages (English, German, Russian) and 3) computer-administered C-Tests by employing a variety of methods and collecting a substantial data set building on several relevant types of evidence. Using a sample (n=540) of university language learners, the study aims to provide evidence on what differently timed C-Tests measure, building upon and furthering our knowledge about the fluidity of the construct (Drackert & Timukova, 2020 a&b; Sigott, 2006). Moreover, this will be the first study to look into the componential structure of the C-Test construct using structural equation modeling and process-oriented video-based analysis of strategies. The improved understanding of the influence of time on the C-Test construct and of the construct as such will be beneficial for the practical use of C-Tests. It will allow stakeholders to tailor the instrument to specific purposes and groups and will lead to better informed decisions about learners’ L2 proficiency. Consequently, it will make the tool more precise, more efficient, and more cost-effective.

• DFG Programme: Research Grants