Community posts
And if in doubt.... guess 'D'
by Simon Bates
University of British Columbia
By now, hopefully you're enjoying a well-deserved summer break (at least those in the Northern Hemisphere.....) In the summer spirit, here's an interesting question that we were asked this week.
This week, we gave a virtual workshop on PeerWise, as part of the Western Conference on Science Education, held in London, Ontario. (Slides here, if you're interested) One of the participants asked a seemingly innocent question that got us thinking.
What is the most common choice of correct answer chosen by authors of an MCQ?
Whilst not knowing the answer there and then, we realized that we were sitting on a goldmine of data! The PeerWise system now contains over 600,000 student-authored questions. Granted, not all of these are 5 item MCQs, but a substantial fraction are. Could we mine this data to see if there really was a preferred answer choice and, if so, which option was it?
It turns out that there are nearly a quarter of a million 5-item 'active' MCQs in the PeerWise system, and that the most commonly chosen answer by authors of questions is 'D'. The percentage of 'D' correct answers (24.98%) may not look all that much more than the 20% that might be expected if the choice of answers was totally random, but the sheer number of questions analyzed here (223,435) makes this a highly significant result (in the sense of 'statistical significance', possibly less so in the sense of 'change-the-world-significant'.....)
It's interesting to note that the extremities of the answer range are both below the 'random choice' value of 20%. There's a certain logic in thinking that authors may want to conceal the right answer somewhere other than the last, or perhaps even more so, the first answer choice.
Is this just a peculiarity of student-generated questions, and what about questions with fewer than 5 answer choices, you might be thinking? The collective wisdom of the internet is not a great deal of help here. Various sites (Yahoo answers included) include commentary that lacks a definitive answer, but is not short of 'definite' proclamations that it is answer B. Or C. Or, if you're not sure, pick the longest answer. Clearly, some people would be better served by adopting a different strategy when preparing to answer MCQs, such as actually learning the material. Learning solutions magazine claims the most common answer on a 4-item question is C or D. When I got down the Google search page to a link to 'Wild guessing strategies on tests', I stopped reading. Feel free to supplement this with your own research....
(Just for the record, from our analysis of another 220,000 4-item MCQs, the most popular answer chosen by authors is C, by a nose from B, both of which are well above the 25% expected value if truly random.)