Skip to content

QTIWorks conformance and implementation notes

Jump to bottom Edit New page
Dave McKain edited this page Feb 19, 2016 · 55 revisions

This document summarises QTIWorks' support for (and conformance of) the QTI 2.1 specification.

Assessment Test, Section and Item Information Model

The discussion in this section follows the chapters of the QTI 2.1 "Assessment Test, Section and Item Information Model" document, starting with chapter 4.

4. Items

The assessmentItem class is fairly well implemented in QTIWorks right up to the level of rendering and delivery.

The itemSessionControl class is also implemented fully.

QTIWorks supports the delivery of a standalone assessmentItem, using the notion of an item delivery to configure aspects of the process, which features most concepts from the itemSessionControl class (normally only used in test delivery) as well as a number of other options. Low level "running" of items is via the ItemSessionController class in the qtiworks-jqtiplus module, which is a state machine for manipulating an instance of an ItemSessionState, which represents the current candidate state of a single item.

5. Item Variables

  • QTIWorks supports all three types of Item Variables.
  • All combinations of baseType and cardinality are implemented.
    • The base qtiworks-jqtiplus module implements the file type quite sparsely and fills in the details within the qtiworks-engine code module. If you are using JQTI+ for your own projects then you will need to do a bit of work to fill in these details to suit yourself.
  • record variables assume additional characteristics if they resemble the special "Maths Content" variables used by the MathAssess extensions and if the MathAssess extension package has been registered.

5.1. Response Variables

This is all fully implemented. Note that correctResponse may not be useful in some types of questions.

5.1.1. Built-in Response Variables

These are all supported.

However note that the management of duration is currently not fully implemented in the QTIWorks engine.

5.2. Outcome Variables

Fully implemented with the following caveats and clarifications:

  • The interpretation and longInterpretation attributes are implemented but never used.
  • The word positive in the context of normalMaximum is assumed to be strictly greater than zero, including the legal float value +0.

6. Content Model

This is fully implemented, subject to the following notes and clarifications:

  • The base attribute on the flow abstract class is supported but not used. This property is renamed as baseUri in the qtiworks-jqtiplus code module in order to avoid a clash with a property of the same name in the stringInteraction abstract class. Reading and writing of this property from XML prefixes this element to put it in the XML namespace.
  • The expansion of template variables in param is not currently implemented.

6.3. MathML

MathML is supported subject to the following notes:

  • The only validation performed on MathML is schema validation.
  • QTIWorks implements a more relaxed version of 6.3.1, allowing outcome and response variables to be substituted within MathML as well.

6.4. Variable Content

This is supported subject to the following notes:

  • the view attribute is not currently used in item delivery.
  • the use attribute on rubricBlock is not presently supported.
  • the current implementation of printedVariable is deficient and needs updated to support all format types and the new attributes added since draft 2.

6.5. Formatting Items With Stylesheets

The stylesheet class is implemented but only the immediate stylesheet child of an assessmentItem is used during rendering. There are currently no plans to support styles declared in stylesheets within individual blocks (e.g. rubricBlock)

7. Interactions

All interactions are supported, though a few are not implemented (as far as delivery and rendering goes) and many of the graphic ones have rather dated implementations. Further details:

  • The orientation attribute is not implemented anywhere yet.
  • The placeholderText attribute of stringInteraction is never used in rendering.
  • The textFormat atribute of extendedTextInteraction is never used in rendering.
  • Various min attributes are fully supported but never form part of the candidate's interactions with the resulting entry widget while interacting: associationInteraction, hotspotInteraction, matchInteraction, selectPointInteraction.
  • The new minAssociates attribute of graphicAssocationInteraction is not currently implemented at all.
  • Neither mediaInteraction and drawingInteraction are supported at the rendering & delivery layer (i.e. within the qtiworks-engine code module) and there are currently no plans to implement these. Volunteers for adding functionality for these are welcome!
  • The following interactions are rendered using rather old-fashioned Java applets, which could all do with replacement:
  • associateInteraction
  • graphicAssociateInteraction
  • graphicGapMatchInteraction
  • graphicOrderInteraction
  • hostspotInteraction
  • positionObjectInteraction
  • selectPointInteraction
  • The following interactions have rather dated rendering and could do with updating:
  • matchInteraction
  • gapMatchInteraction

8. Response Processing

QTIWorks supports all aspects of response processing.

All of the standard QTI 2.1 response processing templates listed in the spec are supported, and the original QTI 2.0 templates are also included for completeness. The information model is not amazingly clear on the use of .xml file extensions for templates, which has lead to confusion amongst authors, so we support both here. Note that the standard RP templates are always loaded from an internal cache.

9. Modal Feedback

This is supported and rendered acceptably.

10. Item Templates

Template processing is supported in full, subject to the following notes:

  • By default, templateConstraint currently runs up to 100 times before failing. It is possible to change (but not remove) this limit via the JQTI+ API, or by the "delivery settings" concept within the QTIWorks engine itself. However, it is currently not possible to change this limit within the assessmentItem itself, which some people thought might be useful.

11. Tests

QTIWork supports most of the concepts in this part of the information model, subject to the following notes:

  • The timeLimit concept is supported in JQTI+ but not used (or enforced) anywhere in the QTIWorks delivery engine. There are no current plans to support this as it would require some refactoring and redesign to the delivery engine.
  • The assessmentSectionRef class is not implemented, and there are no current plans to support this.
  • There is also no support for the (vaguely-defined) extension attributes on the ordering and selection classes.
  • While we have implemented branchRule and preCondition, we may issue some additional warnings about corner cases that are not documented in the information model.

Low level "running" of tests is done via the TestSessionController class in the qtiworks-jqtiplus module, which is effectively a state machine for mutating an instance of a TestSessionState object.

In terms of full delivery of a test via the QTIWorks Engine, we adopt a few sensible navigational conventions when handling tests containing multiple testParts. When delivering tests containing one testPart, the sole testPart is entered immediately when the test is started, and the overall test feedback is combined with the testPart feedback and shown immediately when the testPart is ended. For tests with multiple testParts, we provide an explicit test entry page, and show the overall test feedback separately once the final testPart has been exited.

12. Outcome Processing

The model and logic for this is fully implemented.

13. Test-level Feedback

See section 11 above.

14. Pre-conditions and Branching

See section 11 above.

15. Expressions

All expressions (with the exception of those used in tests only) have been fully implemented, subject to the following general notes and caveats:

  • Expressions supporting integerOrVariableRef and friends need a bit more work to clarify what should happen if the resulting value does not satisfy the constraints stated. The specification is silent on this issue. See below for further notes on how QTIWorks has implemented each of these.
  • The variable class currently does not support the special case of its "dotted" notation for accessing the duration value of a particular item or section within a test. (The other forms of "dotted" notation are however implemeneted fully.)
  • The definition attribute of customOperator is never used. Registration of a new customOperator is done via the JqtiExtensionPackage Java class.
  • There are a number of corner-cases that the spec is not explicit on, especially in the case of arithmetic operators. The results in these situations are usually reasonable but perhaps not expected.

Further notes on specific expressions:

  • anyN: If either min or max is a variable reference that evaluates to NULL, then any NULLs are replaced with the integer 0 and a runtime warning will be recorded.
  • randomInteger & randomFloat: If min is a variable reference that evaluates to NULL then it will be replaced with 0 and a runtime warning will be recorded. Likewise, if max evaluates to NULL then it is replaced with the resulting value of min plus 1; if step evaluates to NULL then it is replaced with 1. Runtime warnings will be recorded in all of these cases.
  • randomFloat: If min is a variable reference that evaluates to NULL then it will be replaced with 0 and a runtime warning will be recorded. Likewise, if max evaluates to NULL then it is replaced with the resulting value of min plus 1. If the resulting value of min is greater than max then NULL is returned. Runtime warnings will be recorded in all of these cases.
  • index: If n is a variable reference and evaluates to NULL or an integer outside the bounds of the container, then NULL is returned and a runtime warning will be recorded.
  • repeat: If numberRepeats is a variable reference that evaluates to NULL or an integer less than 1, then NULL is returned and a runtime warning will be recorded.
  • equal: If no tolerance value is given, then both are treated as 0.0. (The spec is not very precise here.) If either tolerance value is a variable reference that evaluates to NULL or a non-positive number, then NULL is returned and a runtime warning will be recorded.
  • equalRounded & roundTo: If figures is a variable reference that evaluates to NULL or an integer outside the constraints of the roundingMode, then NULL is returned and a runtime warning will be recorded.
  • patternMatch: If pattern is a variable reference that evaluates to NULL then NULL is returned and a runtime warning will be recorded.

16. Item and Test Fragments

The include class has never been implemented and we currently have no plans to support this.

17. Basic Data Types

All data types are supported, with the exception of string256 which is silently treated as a string. Care should be therefore taken by software using the qtiworks-jqtiplus module to ensure that string256 is suitably limited if the results are going to be serialized to XML in order to ensure that this XML is schema valid.

18. Accessible Portable Item Protocol

This is not supported and we have no plans to add support for this for the time being. The model in qtiworks-jqtiplus should be rich enough to support this (at least in the sense of reflecting the spec and reading/writing XML), so a third party would be welcome to add support for this.

Results Reporting

This is implemented acceptably.

Meta-data and Usage Data

This is not supported and there are no plans to add support for this.

Extensions

QTIWorks includes a full implementation of the MathAssess extensions, implemented by the qtiworks-mathassess code module and the helper module qtiworks-mathassess-glue.

The base qtiworks-jqtiplus code module includes an API for third parties to write and register their own extensions. This API is now fairly stable and reasonably simple. Documentation about writing extensions shall be added soon.

Add a custom footer
Add a custom sidebar
Clone this wiki locally