Forced-choice vs. rating scale in personality assessment: A comparison of two response formats

• Applicant: Professorin Dr. Eunike Wetzel
• Subject Area: Personality Psychology, Clinical and Medical Psychology, Methodology
• Term: since 2014
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 268188171

The forced-choice format has been suggested as an alternative to rating scales (RS) that might be less susceptible to response biases such as faking. In the forced-choice format, multiple items are presented to participants simultaneously and participants either rank the items according to how well the items describe them, or select the one that describes them most and the one that describes them least. In the first part of this project, we developed the Big Five Triplets, a multidimensional forced-choice (MFC) instrument assessing the Big Five with triplets. In four studies we found that MFC triplets seemed to be more demanding to respond to than RS items, but this did not appear to have a detrimental effect on test motivation. We found that similar criterion-related validities were achieved in the MFC and RS format. Furthermore, we found that the MFC format was effective in reducing faking, though the degree to which faking was reduced depended on how well the items that were presented simultaneously were matched regarding their desirability. While these results are promising for the MFC format, there are also costs to applying MFC including the complex test construction, the necessity of analyzing data with item response models to obtain normative scores, and the need to administer longer instruments to achieve similar reliability as in rating scale instruments. The goal of the second part of the project is to assess the viability and feasibility of the MFC format as an alternative to RS. Three overarching research questions that are relevant to the cost-benefit trade-off between MFC and RS will be investigated: 1) How can the best matching of items with respect to their desirability be achieved? 2) How can careless responding be detected in MFC data? 3) Does the MFC format yield higher criterion-related validities when the RS format is affected by response biases related to the preference of certain categories (response styles)? The goal of Study 1 is to compare different methods of matching the items within triplets and to investigate which method is best in terms of reducing faking. The goal of Study 2 is to develop and validate methods for detecting careless responding in MFC data and to compare the proportion of careless responders between MFC and RS. Study 3 will compare criterion-related validities between MFC and RS in heterogeneous samples in which the MFC format is free from response styles, while the RS format is influenced by response styles. Study 3a will address cultural heterogeneity and investigate MFC versus RS in five countries: China, Germany, Poland, Spain, and the United States. Study 3b will address heterogeneity with respect to education level and socioeconomic status using a population-representative sample from Germany. The research in this project will inform test constructors and test users considering applying MFC instruments and provide recommendations regarding the MFC format as an alternative to RS.

• DFG Programme: Research Grants