Proceedings of the 7th World Conference on Research in Teaching and Education
Year: 2024
DOI:
[PDF]
The Vertical Arrangement of Response Options in Multiple-Choice Test Items Is Not Always Better
Consuelo Soto, Séverin Lions, Gabriel Ortega, Melissa Arjona, María Paz Blanco, Pablo Dartnell
ABSTRACT:
Multiple-choice tests are widely used to measure learning outcomes. Accordingly, constructing high-quality test items is critical, and many authors have advanced item-writing guidelines. One frequently mentioned guideline is to arrange the response options vertically. However, empirical evidence to support this recommendation is scarce and has only been obtained for items with text-like options. This study aimed to understand if the arrangement of options affects performance at solving items with large-sized options, such as graphs and pictures. Fifty-seven high-school students completed a multiple-choice science and mathematics test with 24 four-choice items whose options were graphs or pictures and presented in one of four experimental conditions (vertical without page break, vertical with page break, clockwise squared 2*2, counterclockwise squared 2*2). Then, students took part in a post-resolution cognitive interview about their experience, practices, perceptions, and beliefs regarding the arrangement of options. Results showed that the arrangement of options hardly affected performance, the only effect being that the vertical condition with page break resulted in longer response time. Students negatively perceived the vertical condition with page break, most frequently preferred the vertical arrangement they considered most common, and reported that the squared arrangements facilitate visual exploration and comparison between options. These results suggested that the vertical arrangement does not offer clear advantages over squared arrangements for items with large-sized options.
keywords: Assessment, Item-writing, Multiple-choice, Response Options, Spatial Arrangement