Investigating the validity of surveys based on indirect questions

• Applicant: Dr. Adrian Hoffmann
• Subject Area: General, Cognitive and Mathematical Psychology
• Term: from 2020 to 2022
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 439602023

The validity of self-reports is at risk if questions about sensitive characteristics are answered dishonestly. To increase the validity of survey results and the willingness to provide honest answers, indirect questioning techniques use randomization procedures to guarantee the confidentiality of answers to sensitive questions. However, recent findings with experimentally induced external criteria suggest that when indirect questions are used, false positives may occur and may lead to an overestimation of the prevalence of socially undesirable attributes. While the false-negative rate was usually lower than when using direct questions, it was also repeatedly found to exceed 0%. Therefore, underestimates of the prevalence of sensitive attributes in indirect surveys cannot be ruled out either. Potential causes for deficiencies in the specificity and sensitivity of indirect questioning techniques have not yet been sufficiently understood. The central aim of the current project therefore is to study these causes experimentally. To this end, we want to investigate empirically whether and to what extent the validity of indirect surveys is influenced by non-compliant or random response behavior (Exp. 1) and by the formulation of the sensitive statement (Exp. 2). To test whether a necessary condition for valid prevalence estimates is met in the first place, we also want to conduct the first examination of the retest stability of estimates obtained via indirect questions (Exp. 3). Against the background of recent doubts regarding the success of the text-based instructions that have as yet been used exclusively for indirect questions, we want to develop and validate a new, video-based instruction format to improve respondents' attention, understanding and motivation (Exp. 4). In all studies, we want to include the respondents’ level of education as a potential moderator to critically examine the applicability of indirect questioning techniques to different target populations. The results of the planned studies will be relevant for many fields of research in which surveys on the prevalence of socially (un-)desirable attributes are conducted. Our aim is to investigate the dangers that threaten the validity and thus the applicability of indirect questioning techniques, and to test whether and how these dangers can be controlled.

• DFG Programme: Research Grants
• Co-Investigator: Professor Dr. Jochen Musch