Scoring test questions

Possibilities for scoring questions

The first decision to make is whether the test question (or item) can be scored automatically, or whether it needs to be manually scored. Not all test questions in a specific test need to be scored in the same way; even parts of the same test question (if there are two interactions in the same item, for example) can be marked in different ways, or use different scoring methods.

First, we'll focus on what the options are for scoring test questions automatically, and then have a brief look at the manual scoring platform, which is available as an add-on to TAO.

Automatic scoring

Once you've established that an item can be scored automatically (this will depend on the interaction types used in it), the main decision you need to make when scoring the response is whether to award partial credit if only part of the answer is correct. This issue only arises when a test question has multiple parts, of course.

Deciding not to award partial credit

If the scoring method is configured to award NO partial credit (which, as mentioned above, is the default method applied to test items in TAO), this means there are only two possibilities: either the response is wholly correct, or it is not. 

In this case, even if part of the response is correct, the whole response is marked as incorrect, and no marks are awarded to the test-taker for this question.

In the example question below, test-takers are asked to say which countries in the list (Belgium, Germany, Austria and Denmark) have no common border with Luxembourg. There are two correct responses: Austria and Denmark. The test-taker, however, chose only one of these.   

Awarding no partial credit

The black bar at the bottom shows the responses chosen by the test-taker, and the scores given. MAXSCORE shows that there was only one mark available in this question, and SCORE shows that the test-taker scored zero. In other words, although they got one of the two correct responses, no partial credit has been awarded for it.

Note: Details on how to configure this option are given in the chapter 'Summary and configuration of scoring methods' below.

Awarding partial credit for correct parts of an answer

As an alternative to scoring a test question as completely correct or incorrect, you can decide that you would like the test-taker to receive partial credit for correct parts of the response – i.e. for responses which are partially, but not wholly, correct. This can be useful for questions where the answer is given in multiple parts.

In the example question below, test-takers are asked to identify what each musician did in the rock band The Beatles. Of the responses given, two were correct, but the other two were not.  

Awarding partial credit

The black bar at the bottom shows the responses chosen by the test-taker, and the scores given. MAXSCORE shows that there was a maximum of four marks in this question (one for putting each of the musician's names in the correct place), and SCORE shows that the test-taker scored two marks (out of the possible four). In other words, partial credit is awarded for the two which were correct.   

Using this scoring method, you also have the possibility of awarding more marks for some parts of the answer than others, if you consider them to be of greater importance. The marks to be given for each part of the response is configured when creating the item.

Manual scoring

TAO works in conjunction with a manual scoring environment called TAO Grader. Using this platform, items in tests that have been delivered on the TAO platform can be manually scored, making it possible to score open-ended questions. 

An example of a test item containing an open-ended question is the following, which asks test-takers to describe a place:

A question which needs manual scoring

The manual scoring platform is fully integrated in the TAO platform: when the test-taker has finished a test, the responses to items which have been identified as needing manual scoring are sent to TAO Grader, where scoring can begin immediately. Once all the scores have been submitted by the manual scorers for the responses of a particular test session (i.e. a delivery execution), they are returned to TAO for further processing.

See the chapter Scoring items manually in the User Guide for more information on manual scoring.