Exploring the effectiveness of transparency and truncation for mitigating gender and nationalistic biases in judging panels

• Applicant: Professor Dr. Tim Pawlowski
• Subject Area: Economic Policy, Applied Economics ; Statistics and Econometrics
• Term: since 2023
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 525061914

Subjective performance evaluations (SPEs) are commonly used by organizations to assess, select, and rank candidates in different competitive situations; for instance, in education for assessing and selecting students, in research for assessing and selecting grant applicants, in the labor market for assessing and selecting job applicants, or in sports for judging the performance of athletes and determining champions. Since many of such SPEs have serious economic consequences for both candidates and organizations, individual or panels of experts are usually appointed to ensure the quality of evaluation. There is, however, substantial evidence for the existence of different biases in SPEs leading to favoritism or discrimination against candidates. While the existence of nationalistic bias in SPEs is well established in the literature, evidence on the existence of gender bias is mixed. Since nationalistic bias might even crowd out gender bias, gaining further insights on this interplay is highly relevant. Moreover, the most prominent forms of regulation, i.e., transparency and truncation, may theoretically affect evaluations of individual judges as well as final panel evaluations, potentially mitigating such biases. Empirical evidence, however, on whether and how transparency and / or truncation might indeed mitigate any of this bias ex ante is scarce and inconclusive. In this project, we intend to exploit rich panel data from real competitive environments with high stakes, i.e., professional sports, in order to test whether (i) nationalistic bias might be crowding out gender bias. Moreover, we intend to explore whether and how much (ii) transparency, i.e., the disclosure of judges’ identity, affects these biases by exerting external pressure and control. Likewise, we intend to explore whether and how much (iii) truncation of extreme scores affects these biases by changing the incentive structure for judges. Given the (monetary) relevance of performances and their evaluations in combination with the (increasing) interest of athletes, coaches, and federations to track all performance data over time, the data we intend to use are of large quantity and high quality. Moreover, using data from professional sports is generally promising, since reliable and precise performance measures, candidate and judge characteristics, and SPEs are readily available, as opposed to other settings. As such, with this project we intend to inform decision-makers in sports policy and management. Given the comparability of settings, we expect valuable information also for more general human resource management, education policy, and beyond. Overall, we intend to contribute to a better understanding of how judging panels in general and SPEs in particular should be designed in order to achieve less biased outcomes.

• DFG Programme: Research Grants
• International Connection: Finland
• Cooperation Partner: Professor Dr. Alex Krumer