Qualitative Grading in Science — Rationality, Aggregation, and Applications

• Applicants: Professorin Dr. Dunja Seselja; Dr. Minkyung Wang
• Subject Area: Theoretical Philosophy
• Term: since 2026
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 567404969

Modern science is often equated with quantitative methods—measurement, statistical testing, and probability-based reasoning. Yet, qualitative judgments are pervasive across scientific disciplines. From evaluating research proposals and experimental designs to providing expert policy advice, scientists routinely rely on qualitative assessments where probabilities may be unavailable or impractical. For instance, the IPCC uses standardized qualitative scales to assess evidence, agreement, and confidence. Despite widespread use, qualitative assessments have received little philosophical and formal scrutiny. Traditional frameworks focus on dichotomous (true/false) or probabilistic accounts of belief, leaving qualitative scales (e.g., very unlikely, unlikely, likely) underdeveloped. Notable exceptions, like the Majority Judgment method by Balinski and Laraki, show the potential of aggregating graded judgments in contexts like elections or wine tasting. However, such approaches remain unadapted to scientific belief and decision-making, where aggregating complex qualitative evidence poses distinct challenges. As a result, we lack an understanding of how to adequately aggregate expert judgments based on qualitative grades in various scientific contexts. Bridging this gap is crucial for improving scientific deliberation and expert decision-making. This project aims to bridge this gap by developing and applying a novel framework for qualitative judgments in science. Our research is organized around four interconnected objectives. First, we will construct an epistemology of qualitative judgments tailored to scientific items—such as hypotheses, research questions, and projects—across diverse contexts, including basic inquiry, applied research, and policy guidance. Second, we will develop a new formal qualitative aggregation framework that extends ideas from Majority Judgment to the scientific domain while interfacing with established binary and probabilistic methods. Third, we will apply these theoretical innovations to different scientific contexts—ranging from deliberation under scientific disagreement to "fast science" environments requiring rapid, reliable decisions—thereby providing improved guidelines for collective scientific judgment. Fourth, we will reach out to the relevant stakeholders, such as expert agencies, to discuss implementing our results in concrete scientific contexts. The project draws on insights from epistemology, social epistemology, and the philosophy of science, combining formal and computational tools to deepen our understanding of qualitative grading in science and to provide robust methodologies for enhancing the epistemic quality of group decisions. The project results will be of direct relevance to expert deliberation and scientific practice.

• DFG Programme: Research Grants
• International Connection: France
• Partner Organisation: Agence Nationale de la Recherche / The French National Research Agency
• Cooperation Partner: Dr. Thomas Boyer-Kassem