Modeling educational effectiveness and equity at the school level

• Applicant: Professor Dr. Johannes Hartig
• Subject Area: Education Systems and Educational Institutions ; General and Domain-Specific Teaching and Learning
• Term: from 2016 to 2021
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 316303269

• Final Report Year: 2023

Final Report Abstract

The project „Modeling Educational Effectiveness and Equity at the School Level“ had three aims: (1) Examining the consistency of different operationalizations of school effectiveness. (2) Analyzing associations between the effectiveness of schools and structural as well as process variables. (3) Testing the stability of school effectiveness across time and examining predictors for change in school effectiveness across time. Schools were considered effective when they showed a comparatively high gain in average student achievement or a comparatively positive development of learning-related motivational and affective student characteristics, controlling for background variables („value added“). In addition to the quality of these outcomes, also the development of associations between social difference categories and the educational outcomes within the schools was taken into account (equality of outcomes). Structural variables that were correlated with the quality and equality of the outcomes were the school composition and the educational track. Secondary analysis of data from the two longitudinal studies LAU and KESS was used to answer the research questions. A first result of the project was that only part of the instruments used within the two studies were adequate for to operationalize the outcomes. The consistency of operationalization – across those that were found to be adequate – was further small. Thus, school effectiveness does not appear to be one coherent construct. Rather, the effectiveness appears to be complex and incoherent – an evaluation of the effectiveness of a school can vary considerably depending on the criterion chosen. Regarding the second question, only few school composition effects on the development of student outcomes over time were found. Whereas gains in student achievement were higher in academic tracks, changes in non-cognitive outcomes were often more positive in the nonacademic tracks. Analysis of the development of the school average achievement gains between grade 4 and 10 show some stability of school effectiveness over a period of 10 years. The stability of between school differences in effectiveness is to some extent explained by their allocation to academic vs. nonacademic tracks, but not completely.

Publications

• Die Entwicklung und die Konsistenz non-kognitiver Schuleffektivitätskriterien. In Möller, J. (Chair), Chancen und Herausforderungen von Sekundäranalysen. Symposium im Rahmen der Gesellschaft für Empirische Bildungsforschung (GEBF), Basel, Schweiz
  Grützmacher, L.; Hartig, J. & Vieluf, S.
  
• Effects of classroom composition on the development of students' learning motivation. In K. Adl-Amini & S. Vieluf (Chair), Effects of Group Heterogeneity on Teaching and Learning. Symposium at the German Psychological Society (DGPs), Frankfurt am Main, Germany
  Grützmacher, L.; Vieluf, S. & Hartig, J.
  
• Measurement Invariance Analysis of Reading Attitudes. In J. Fischer (Chair), Data comparability as prerequisite for effective evaluations of learning and teaching. Symposium at the European Association for Research on Learning and Instruction (EARLI SIG 18+ 23), Groningen, Netherlands
  Grützmacher, L.; Vieluf, S., Köhler, C. & Hartig, J.
  
• Messinvarianzanalysen des Leseselbstkonzepts über die Zeit, über Ebenen und über Schulformen. In Hübner, N. (Chair), Chancen und Herausforderungen von Sekundärdatenanalysen. Symposium im Rahmen der Gesellschaft für Empirische Bildungsforschung (GEBF), Köln, Deutschland
  Grützmacher, L.; Hartig, J. & Vieluf, S.
  
• The Development and Consistency of Cognitive and Non-Cognitive School Effectiveness Criteria. In Gillies, R. (Chair), Single Paper: Learning, Teaching and Instruction (EARLI), Aachen, Germany
  Grützmacher, L.; Vieluf, S. & Hartig, J.
  
• Are questionnaire scales which measure non-cognitive constructs suitable as school effectiveness criteria? A measurement invariance analysis. School Effectiveness and School Improvement, 32(3), 430-447.
  Grützmacher, Luisa; Vieluf, Svenja & Hartig, Johannes
  
• Are some schools less unjust than others? In A. Naumann & D. Kolovou (Chair/Organisers), Students’ cognitive and non-cognitive learning gains: The role of teaching and teacher competencies. Symposium at the European Association for Research on Learning and Instruction (EARLI), Online
  Grützmacher, L.; Vieluf, S. & Hartig, J.
  
• Ergebnisqualität und Ergebnisgleichheit von Schulen - Operationalisierung und Konsistenz von Schuleffekten. Münster: Waxmann
  Grützmacher, L.