Using Metadata to Define Outcomes

Metadata can be used as a filter to look at test results too. So, in a test that covered different difficulty levels, for example, you can choose to view the score of the results of only the low difficulty level items.

To do this, when you define the difficulty property of an item on the Manage item class schema screen, you will see four symbols to the right of it, starting with a magnifying glass.  By clicking on the flag – i.e. by flagging this property for this item - you are saying that when you assemble a test using this item, you want the information about the difficulty level of the item to be flagged in the QTI XML file. In other words, you want to use this property at the level of the test.

You can check that this property is flagged on the level of the test in the property settings - just click on the cogs and you will see that low is defined as one of the categories of the test.

You can then use this category to filter the scores: in the test settings, there are different options for Outcome processing. This is where the criteria are defined which are used to evaluate the test-taker's responses.

One of these options is Cut score. If you choose Cut score (which will then appear as cut score pass ratio), you can define a variable that defines whether the test-taker has passed or failed the test.

If you then check the Category score box, this leverages the metadata. When you do this, there isn’t just one single outcome (pass or fail) that is generated by TAO for the whole test - a pass/fail for each category will also be generated. In other words, as well as the total score the test-taker has for the whole test, you also get a list of partial scores. So, for example, a student could get a pass for items that are "medium" in terms of their difficulty level, but not for items that are "high" in difficulty. Or, he/she could get a pass in items testing reading skills but a fail in items testing writing skills if this information has been stored as a property in the metadata. All these scores can be viewed in Outcome declarations, which appears at the bottom of the properties panel.