QTIWorks conformance and implementation notes

This document summarises QTIWorks' support for (and conformance of) the QTI 2.1 specification. It is currently written against Draft 13 (July 2012).

Assessment Test, Section and Item Information Model

The discussion in this section follows the chapters of the QTI 2.1 "Assessment Test, Section and Item Information Model" document, starting with chapter 4.

4. Items

The assessmentItem class is fairly well implemented in QTIWorks right up to the level of rendering and delivery.

The itemSessionControl class is partly implemented in the base qtiworks-jqtiplus module, but needs refactored. This is only really used for delivering items within tests.

QTIWorks supports the delivery of single assessmentItems, using the notion of an item delivery to configure aspects of the process, which features some concepts from the itemSessionControl class (normally only used in test delivery) as well as a number of other options.

5. Item Variables

• QTIWorks supports all three types of Item Variables.
• All combinations of baseType and cardinality are implemented.
  • The base qtiworks-jqtiplus module implements the file type quite sparsely and fills in the details within the qtiworks-engine code module. If you are using JQTI+ for your own projects then you will need to do a bit of work to fill in these details to suit yourself.
• record variables assume additional characteristics if they resemble the special "Maths Content" variables used by the MathAssess extensions and if the MathAssess extension package has been registered.

5.1. Response Variables

This is all fully implemented. Note that correctResponse may not be useful in some types of questions.

5.1.1. Built-in Response Variables

These are all supported.

However note that the management of duration is currently not fully implemented in the QTIWorks engine.

5.2. Outcome Variables

Fully implemented with the following caveats and clarifications:

• The interpretation and longInterpretation attributes are implemented but never used.
• The word positive in the context of normalMaximum is assumed to be strictly greater than zero, including the legal float value +0.

6. Content Model

This is fully implemented, subject to the following notes and clarifications:

• The base attribute on the flow abstract class is supported but not used. This property is renamed as baseUri in the qtiworks-jqtiplus code module in order to avoid a clash with a property of the same name in the stringInteraction abstract class. Reading and writing of this property from XML prefixes this element to put it in the XML namespace.
• The expansion of template variables in param is not currently implemented.

6.3. MathML

MathML is supported subject to the following notes:

• The only validation performed on MathML is schema validation.
• Rendering of content MathML is not currently supported. We will most likely use the simplest implementation, which is to perform a Content MathML to Presentation MathML transform. This works OK but provides no control over notations used. The results should be acceptable in UK contexts.

6.4. Variable Content

This is supported subject to the following notes:

• the view attribute is not currently used in item delivery.
• the use attribute on rubricBlock is not presently supported.
• the current implementation of printedVariable is deficient and needs updated to support all format types and the new attributes added since draft 2.

6.5. Formatting Items With Stylesheets

The stylesheet class is implemented but only supported at item level as far as rendering goes. There are currently no plans to support it within individual blocks (e.g. rubricBlock)

7. Interactions

All interactions are supported, though a few are not implemented (as far as delivery and rendering goes) and many of the graphic ones have rather dated implementations. Further details:

• The orientation attribute is not implemented anywhere yet.
• The placeholderText attribute of stringInteraction is never used in rendering.
• The textFormat atribute of extendedTextInteraction is never used in rendering.
• Various min attributes are fully supported but never form part of the candidate's interactions with the resulting entry widget while interacting: associationInteraction, hotspotInteraction, matchInteraction, selectPointInteraction.
• The new minAssociates attribute of graphicAssocationInteraction is not currently implemented at all.
• Neither mediaInteraction and drawingInteraction are supported at the rendering & deliver layer (i.e. within the qtiworks-engine code module) and there are currently no plans to implement these. Volunteers for adding functionality for these are welcome!
• The following interactions are rendered using rather old-fashioned Java applets, which could all do with replacement:
• associateInteraction
• graphicAssociateInteraction
• graphicGapMatchInteraction
• graphicOrderInteraction
• hostspotInteraction
• positionObjectInteraction
• selectPointInteraction
• The following interactions have rather dated rendering and could do with updating:
• matchInteraction
• gapMatchInteraction

8. Response Processing

QTIWorks supports all aspects of response processing.

All of the standard QTI 2.1 response processing templates listed in the spec are supported, and the original QTI 2.0 templates are also included for completeness. The information model is not amazingly clear on the use of .xml file extensions for templates, which has lead to confusion amongst authors, so we support both here. Note that the standard RP templates are always loaded from an internal cache.

9. Modal Feedback

This is supported and rendered acceptably.

10. Item Templates

Template processing is supported in full, subject to the following notes:

• templateConstraint currently runs up to 100 times before failing. There is no way to specify the number of times to run within the question, which some people felt would be useful.

11. Tests

Development of tests is due to start in Q4 2012.

The Java information model in qtiworks-jqtiplus supports most of the classes in the information model, with the excepption of assessmentSectionRef.

12. Outcome Processing

This will be refactored completed during test development.

13. Test-level Feedback

As above.

14. Pre-conditions and Branching

As above.

15. Expressions

All expressions (with the exception of those used in tests only) have been fully implemented, subject to the following general notes and caveats:

• Expressions supporting integerOrVariableRef and friends need a bit more work to clarify what should happen if the resulting value does not satisfy the constraints stated. The specification is silent on this issue. See below for further notes on how QTIWorks has implemented each of these.
• The "double-dotted" variable references described in the variable class were added to QTI 2.1 very late and are not yet supported.
• The definition attribute of customOperator is never used. Registration of a new customOperator is done via the JqtiExtensionPackage Java class.
• There are a number of corner-cases that the spec is not explicit on, especially in the case of arithmetic operators. The results in these situations are usually reasonable but perhaps not expected.

Further notes on specific expressions:

• anyN: If either min or max is a variable reference that evaluates to NULL, then any NULLs are replaced with the integer 0 and a runtime warning will be recorded.
• randomInteger: If min is a variable reference that evaluates to NULL then it will be replaced with 0 and a runtime warning will be recorded. Likewise, if max evaluates to NULL then it is replaced with the resulting value of min plus 1; if step evaluates to NULL then it is replaced with 1.
• index: If n is a variable reference and evaluates to NULL or an integer outside the bounds of the container, then NULL is returned.
• repeat: If numberRepeats is a variable reference that evaluates to NULL or an integer less than 1, then NULL is returned.
• equalRounded & roundTo: If figures is a variable reference that evaluates to NULL or an integer outside the constraints of the roundingMode, then NULL is returned.

16. Item and Test Fragments

The include class has never been implemented and we currently have no plans to support this.

17. Basic Data Types

All data types are supported, with the exception of string256 which is silently treated as a string. Care should be taken by clients using the qtiworks-jqtiplus module to ensure that string256 is suitably limited if the results are going to be serialized to XML in order for it to be valid.

18. Accessible Portable Item Protocol

This is not supported and we have no plans to add support for this for the time being. The model in qtiworks-jqtiplus should be rich enough to support this (at least in the sense of reflecting the spec and reading/writing XML), so a third party would be welcome to add support for this.

Results Reporting

The model for this is largely complete but needs some further work.

Meta-data and Usage Data

This is not supported and there are no plans to add support for this.

Extensions

QTIWorks includes a full implementation of the MathAssess extensions, implemented by the qtiworks-mathassess code module and the helper module qtiworks-mathassess-glue.

The base qtiworks-jqtiplus code module includes an API for third parties to write and register their own exceptions. This API is currently subject to change but is reasonably simple.