1. The stimulus material to which the item refers will contain material which is relevant to the objectives of the test. Considering and responding to the material will be a worthwhile educational experience for the test-taker.
2. The stem and the keyed answer together will represent a meaningful and worthwhile response to a key or central issue in the stimulus material, not a merely peripheral one. Doing the item will not be a merely trivial experience for the test-taker1.
3. Each of the distractors will be plausible: that is, they will represent a possibly relevant view of the matters raised in the stimulus and the stem.
4. The item will be independent. Finding the keyed answer will not depend on successful answering to any other item in the test. No clues as to the keyed answer will be given anywhere else in the test.
5. A successful response to the item-stem will depend on the test-taker understanding a key issue in the stimulus, not eliminating distractors to find a ‘best answer’ or merely recognising a stated fact.
6. The question stated or implied in the item will be positively worded. Where an important issue in the material unavoidably requires negative wording if it is to be tested at all, this will be in the stem, printed in bold capitals (‘NOT’; ’EXCEPT’). No additional negatives will be used in any of the options.
7. The item will contain four or five options for answering, and be laid out in standard form.
8. Each of the options will be roughly the same length. If this is impossible, then two groups of options will be of similar length (e.g. two short and three longer).
9. The item will have been trial-tested and found to have a facility between 20 and 80 percent.
10. In trial-testing, the keyed answer to the item will have been found to discriminate positively, and distractors to discriminate negatively.
• reduces the guessing factors
• versatile – can be used to measure a wide range of cognitive processes
• reduces problem of subjective scoring
• analysis of results can provide much diagnostic information
• easy to mark
Multiple-choice item writing is a difficult art at the best of times but how often we make it more difficult than it need be! We start off thinking it will be easy and go at it very quickly before all the matters we need to keep in mind have been considered. Rushing the process won’t help – items do not spring fully-formed on to the page. We need to ponder, review, get to know what it is we want to ask and what the material might let us ask. Quickly prepared items are often a waste of time – we find that out when we show them to another person, and that person merely says: ‘trivial’ or ‘faulty’.
Guidelines for Constructing Multiple-Choice Items
1. Construct each item to assess a single written objective.
2. Base each item on a specific problem stated clearly in the stem.
3. Include as much of the item as possible in the stem, but do not include irrelevant material.
4. State the stem in positive form (in general).
5. Word the alternatives clearly and concisely.
6. Keep the alternatives mutually exclusive.
7. Keep the alternatives homogeneous in content.
8. Keep the alternatives free from clues as to which response is correct.
8.1. Keep the grammar of each alternative consistent with the stem.
8.2. Keep the alternatives parallel in form.
8.3. Keep the alternatives similar in length.
8.4. Avoid textbook, verbatim phrasing.
8.5. Avoid the use of specific determiners.
8.6. Avoid including keywords in the alternatives.
8.7. Use plausible distractors.
9. Avoid the alternatives “all of the above” and “none of the above” (in general).
10. Use as many functional distractors as are feasible.
11. Include one and only one correct or clearly best answer in each item.
12. Present the answer in each of the alternative positions approximately an equal number of times, in a random order.
13. Lay out the items in a clear and consistent manner.
14. Use proper grammar, punctuation, and spelling.
15. Avoid using unnecessarily difficult vocabulary.
16. Analyze the effectiveness of each item after each administration of the test.
No comments: