Skip to content

External Cache

Bill Majurski edited this page Jun 7, 2017 · 11 revisions

Toolkit is made up of two components: the WAR file which is installed under Tomcat and the External Cache. The WAR is the tool and its static data. The External Cache is the data store for toolkit where all your test results, simulators, and configurations live. The External Cache persists across installations/updated to the WAR file.

Technically the External Cache is a readable/writable directory on the same system that runs the Tomcat supporting Toolkit. The linkage between Toolkit and the External Cache is an entry in the toolkit.properties file inside Toolkit. If you are running multiple copies or versions of Toolkit they must each have a different directory configured as their External Cache. The path to the External Cache (/home/bill/tmp/ec on my development machine) must not contain white space or special characters.

Creation

Toolkit will not run without an External Cache. The internal file toolkit.properties contains a pointer to this directory. If the directory configured does not exist or is not writable at toolkit startup an error message will be displayed on the user interface. To use toolkit you must edit toolkit.properties using the tool named Toolkit Configuration.

If toolkit detects that the External Cache is fresh (readable, writable, but empty) then it will automatically initialize it. A key part of this initialization is the creation of the default environment which is named default. This environment will be initialized with the file codes.xml which contains the XDS Affinity Domain code configuration. As delivered for IHE Connectathons, this will be the required code configuration for the event.

Structure

These are the main sections of the external cache (defined by the directory name within the cache)

actors/
environment/
simdb/
TestLogCache/

actors

actors/
    blue.xml
    red.xml

Collection of XML files that define the configured systems known by toolkit. Earlier versions of Toolkit referred to these as Sites. We are gradually moving to the name Gazelle uses - system. A system has a name and a collection of IDs and endpoints that define the configuration of a system. A system can hold the configuration for a single actor of each type. So, if you have two Document Registries you need to define two systems. But if you have a Document Registry and a Document Repository they can be configured in a single system.

Although a site file can be edited by hand it is usually edited with the Site/Actor Configuration tool (recently renamed to Systems Configuration tool).

Cannot display Site/Actor Configuration tool graphic

Each system is defined by an XML file. The XML (and not the filename) defines the system name. For example (see below), this file blue.xml which holds a system definition. The system name is OTHER_NIST_BLUE_2016 and not blue.

<site name="OTHER_NIST_BLUE_2016">
   <home>urn:oid:1.3.6.1.4.1.21367.2011.2.6.152</home>
   <transaction name="ret.b" secure="1">https://nist26:9085/tf6/services/xdsrepositoryb</transaction>
   <transaction name="xcr" secure="1">https://nist26:9085/tf6/services/rg</transaction>
   <transaction name="pr.b" secure="1">https://nist26:9085/tf6/services/xdsrepositoryb</transaction>
   <transaction name="pr.b" secure="0">http://nist26:9080/tf6/services/xdsrepositoryb</transaction>
   <transaction name="sq.b" secure="0">http://nist26:9080/tf6/services/xdsregistryb</transaction>
   <transaction name="ret.b" secure="0">http://nist26:9080/tf6/services/xdsrepositoryb</transaction>
   <transaction name="r.b" secure="0">http://nist26:9080/tf6/services/xdsregistryb</transaction>
   <transaction name="sq.b" secure="1">https://nist26:9085/tf6/services/xdsregistryb</transaction>
   <transaction name="xcq" secure="0">http://nist26:9080/tf6/services/rg</transaction>
   <transaction name="igq" secure="1">https://nist26:9085/tf6/services/ig</transaction>
   <transaction name="r.b" secure="1">https://nist26:9085/tf6/services/xdsregistryb</transaction>
   <transaction name="igr" secure="0">http://nist26:9080/tf6/services/ig</transaction>
   <transaction name="igq" secure="0">http://nist26:9080/tf6/services/ig</transaction>
   <transaction name="igr" secure="1">https://nist26:9085/tf6/services/ig</transaction>
   <transaction name="xcq" secure="1">https://nist26:9085/tf6/services/rg</transaction>
   <transaction name="xcr" secure="0">http://nist26:9080/tf6/services/rg</transaction>
   <repository uid="1.3.6.1.4.1.21367.2011.2.3.232"
         secure="1">https://nist26:9085/tf6/services/xdsrepositoryb</repository>
   <repository uid="1.3.6.1.4.1.21367.2011.2.3.232"
         secure="0">http://nist26:9080/tf6/services/xdsrepositoryb</repository>
   <patientIdFeed host="null" port="null"/>
</site>

environment

An environment defines the code set used by the Affinity Domain and the certificate used by toolkit to initiate TLS transactions. When toolkit receives transactions, in simulators, the certificate configuration is done in Tomcat.

The default environment, installed in the external cache when it is initialized by toolkit, looks like:

environment/
           default/
                  codes.xml

When we add the configuation for the North American Connectathon in 2016 the environment configuration looks like:

environment/
           default/
                  codes.xml
           NA2016/
                  codes.xml
                  keystore/
                           keystore
                           keystore.properties

The new directory, keystore, holds the Java keystore and the properties describing it. The contents of keystore.password are:

keyStorePassword=changeit

The directory names under the environment directory name the usable environments. They are chosen at the top of the tool pages where they are used.

Starting with Toolkit V4.0.0 an environment can define a separate Test Kit. This test kit can be a replacement or extension of the internal test kit.

Not available

simdb

This is the database of simulator configurations and data. Simulators are organized by Test Session and based on an Environment. A simulator name is the concatenation of the Test Session name, two underscore characters, and the simulator ID. Simulators are literally owned by a Test Session and so have the Test Session name embedded in their name.

Test Session

To understand simulator naming you must first understand the use of Test Session. A Test Session is a separate collection of data to support a testing situation. Some think of Test Session as being the same as User. But toolkit does not perform user authentication and does not maintain information about users. Test Session is tied to what you are testing and not who is doing the testing. For example, if you are testing two Document Registry implementations and you want to keep their test results separate you would use two Test Sessions.

A test session holds two types of data: conformance test results and simulators. Each of these data types are maintained in different parts of the external cache.

simdb continued

The structure of simdb looks like:

simdb/
      bill__rr/
              reg/
              rep/
              rr/
              sim_type.txt
              simctl.ser
              simId.txt

bill__rr is the name of a simulator. bill is the name of the Test Session that owns the simulator. rr is the the local name (within the test session) of the simulator. The full name (bill__rr) is always used as the identifer.

sim_type.txt - holds the simulator type

simId.txt - holds the simulator ID (bill__rr in this example)

simctl.ser - holds the simulator configuration (binary file)

This particular simulator is a composite simulator. It is constructed from multiple simpler simulators.
This is a combined Document Repository and Document Registry. The reg directory holds data for the Registry. The rep directory holds data for the Repository. The rr directory (simulator local name) holds data for managing the composition.

So the reg directory holds the content of the Registry simulator and the rep directory holds the content of the Repository simulator. The next two sections cover the details.

Registry state

Inside the reg directory is

reg/
    pif/
    rb/
    sq/

where pif (Patient Identify Feed), rb (Register), and sq (Stored Query) are the transactions accepted by the Registry. Each time the Registry receives a, for example, Register transaction, something is added to the rb directory.

Each of these directories have the same structure. I'll show the rb (Register) as an example

reg/
    pif/
    rb/
        DATE_TIME1/
        DATE_TIME2/
        DATE_TIME3/
        DATE_TIME4/
        DATE_TIME5/
    sq/

Each DATE_TIME directory is the recording of an event. For the Register (rb) directory this is the receipt of a Register transaction. (Note: the pif directory has a slightly different structure.)

The event directory is almost the same for every transaction. The parts that are common are:

date.ser   - date/time as a Java serialize object
ip.txt     - IP address of sender
log.txt    - log of the transaction
request_body.bin    - HTTP body of the request message 
request_body.txt    - HTTP body of the request message assuming Ascii encoding
request_hdr.txt     - HTTP header of the request message
response_body.txt   - HTTP response message body
response_hdr.txt    - HTTP response message header

For the Register transaction there is also

Registry/

which holds the registry database contents. Each file in this directory looks like

0bdf43c6_6ac7_44cb_83f5_a715c5988a27.xml

where the file name is a UUID without the urn:uuid: prefix. Each file represents a single Registry Object and the filename is the UUID of that Registry Object. The formatting of the XML is such that when a response message to a Stored Query is to be constructed these file can be concatenated together to form much of the body of the message.

Registry Index

With the content of a Registry distributed across event (transaction) specific directories it is necessary to maintain an index for easy retrieval. That index is stored in

simdb/ bill__rr/ reg/ rep/ rr/reg_db.ser - the index for the registry rr/rep_db.ser - the index for the repository

The index is a serialized Java object (binary) and cannot easily be read using command line utilities.

Repository State

The Repository State is slightly different. There is a prb (Provide and Register) directory for the content of that transaction. Inside are the same base set of files and a directory named Repository which contains one file per Document submitted in this event (transaction). This directory listing would look like

0bdf43c6_6ac7_44cb_83f5_a715c5988a27.bin

where this is the binary (byte array in Java) form of the document.

Repository Index

See Registry Index above

TestLogCache

This directory holds the results of conformance tests. The structure of this directory looks like:

TestLogCache/
     bill/
          11966/
                11966
                patientFeed/
                            log.xml
                submit/
                            log.xml
                eval/
                            log.xml
     GazelleSts/
     orchestration/
     site.txt

The directory bill names the Test Session used to run the tests.

The directory 11966 names the test that was run.

The file 11966 is a binary form of the log files used by the Inspector.

patientFeed, submit, eval are test sections - the individual parts of the test. Each section has its own log file, log.xml.

The directory GazelleSts holds details of the connection to the Gazelle Secure Token Service for getting and validating SAML assertions.

The directory orchestration hold test context for the Conformance Test Tool.

The file site.txt holds the name of the system being tested in this Test Session by the Conformance Test Tool. More specifically this is managed by the Test Context.

Clone this wiki locally