Skip to content

Implementing a ResourceRS

Jump to bottom
Jason Stedman edited this page Aug 30, 2019 · 19 revisions

Each type of resource needs its own ResourceRS. These must implement IResourceRS.

All requests are POSTs that take in a QueryRequest object.

README

Please include a README with a ResourceRS that includes:

  • Keys in the resourceCredentials map
  • path to the ResourceRS
  • example queries

Creating a new ResourceRS

New ResourceRSs should be included as a separate module under the pic-sure-resources module. ResourceRSs should have a unique @Path for PICSURE-2.0 to direct resource calls to.

Calling Your Resource

Each QueryRequest has a targetURL that should point to an instance of your resource. This will have been configured on the PICSURE-2.0. Use this as the baseURL and add the necessary path following it to reach a particular endpoint.

Using Resource Credentials

Each QueryRequest has a resourceCredentials map that should contain any credentials required to access the resource. The keys in this map that your ResourceRS will be looking for should be identified in a README so that system admins can give instructions when installing that resource on a PICSURE-2.0 instance.

Endpoints

/info

If your resource has no info endpoint, you may construct an info response. (See HSAPIResourceRS for an example). The ResourceInfo object contains a name (of the resource), an ID, and a list of QueryFormats. QueryFormat has a name, description, specification, and examples, so the info from your resource or that you construct must fit this format. The QueryFormatDeserializer will label anything that is not name, description, or examples as specification. Specification is a map and can contain any relevant information specifying the resource. Examples is an array of maps. Each map should be an example query to send.

/search

The SearchResults object that this endpoint returns can return results in any form - the results field is simply an Object. Additionally, there is a String field searchQuery where the original query submitted should be stored.

Queries

All queries are stored in the PICSURE-2.0 database with the original query request. The UUID of these queries can be used to request query status, result, or metadata, and contains the resource result ID as well as the status (as of the last time it was checked).

/query

This endpoint is for asynchronous queries. It returns a QueryStatus with a resource result id so that the result can be fetched later through the /query/{queryId}/result endpoint. In addition to the resourceResultId, the resource status should be mapped to a PicsureStatus. The resultMetadata field can be used for any additional data that needs to be saved with this query.

/query/{resourceQueryId}/status

The resourceQueryId should match the id of a saved query on your resource. This endpoint returns the same object (QueryStatus) as the /query endpoint.

/query/{resourceQueryId}/result

This endpoint returns the response from the resource as-is.

/query/sync

This endpoint is for synchronous queries which immediately return a result. It returns the response as-is.

Throwing Errors

When throwing an error, if possible, please use one of PICSURE-2.0's errors located in the pic-sure-util module. This will ensure that the error message is correctly passed to the user. Common error messages are included as static Strings in the relevant Exception class.

  • ApplicationException should be thrown whenever the error is with the code or configuration. Some possible cases include:
    • MISSING_TARGET_URL : The targetURL should have been added to the request by the service layer
    • MISSING_RESOURCE_PATH : The resourceRS path should also have been added by the service layer
    • MISSING_RESOURCE : A query stored in the database should have its corresponding resource included
  • NotAuthorizedException should be thrown whenever there is a problem with credentials, i.e. the resource throws a 401.
    • MISSING_CREDENTIALS : use when the credentials have not been included (or perhaps have been mislabeled in the resourceCredentials map)
  • ProtocolException should be thrown when there is a problem with the request submitted.
    • MISSING_RESOURCE_ID : Requests should have a value in the resourceUUID field
    • RESOURCE_NOT_FOUND : The submitted resourceUUID does not match any installed resource
    • MISSING_DATA : Some part or all of the data required for this request is missing
    • MISSING_QUERY_ID : The query id for a query/status or query/result request is missing
    • QUERY_NOT_FOUND : The submitted query id does not match any in the database
    • INCORRECTLY_FORMATTED_REQUEST : The query request submitted is not in the expected format
  • ResourceInterfaceException should be thrown when there is a problem with the resource

If your resource has no equivalent for a particular method, please throw an UnsupportedOperationException.

Utility methods

There are various utility methods in HttpClientUtil

Advice for implementers

There are two common strategies for PIC-SURE Resource implementation, resources that are created from scratch to be PIC-SURE Resources(native) and those that are shimmed to be PIC-SURE compatible(shimmed).

The only requirement to be PIC-SURE compatible is that an API has some subset of the operations defined by the https://github.com/hms-dbmi/pic-sure/blob/master/pic-sure-resources/pic-sure-resource-api/src/main/java/edu/harvard/dbmi/avillach/service/IResourceRS.java interface. Keep in mind that we encourage you using whatever language or toolset you prefer to implement your PIC-SURE resource and to manage its source code separately from the central PIC-SURE repository. This prevents us from becoming your bottleneck for releases and promotes innovation.

Also please read the documentation for implementing resources https://github.com/hms-dbmi/pic-sure/wiki/Implementing-a-ResourceRS

If you decide you are creating a PIC-SURE shimmed resource, the best strategy is to create an API that maps the operations defined in the ResourceRS to the operations supported by your backing resource. How you do this depends on your backing resource and how it behaves. You should design it from the perspective of the existing resource and consider how the functionality it provides can be exposed through the operations of the ResourceRS.

If you decide you are creating a PIC-SURE native resource, you should start with the operations of the ResourceRS and project your use-cases onto them.

Either way, PIC-SURE does not specify how you structure your queries or how you respond to them. It only requires some ceremonial wrapping of JSON around your queries for compatibility.

This is what PIC-SURE requires clients to include in a query:

{ "resourceUUID": "<the resource UUID as registered in the PIC-SURE instance being queried>", "query": <any valid JSON element> }

If your resource has specific authentication requirements it optionally also supports a resourceCredentials element:

{ "resourceUUID": "<the resource UUID as registered in the PIC-SURE instance being queried>", "query": <any valid JSON element>, "resourceCredentials":<a single level hash of string to string mappings> }

The resourceCredentials are excluded from auditing and assertions about them cannot be made in the PIC-SURE Auth Micro-app's rules based authorization. They are supported for resources that do not delegate authX to the PIC-SURE ecosystem.

Below there is an example query from a PIC-SURE HPDS environment. Note that everything inside the query element is specific to the PIC-SURE HPDS resource, not the PIC-SURE API itself. PIC-SURE API does audit the entire query except any resourceCredentials entries.

{ "resourceUUID": "66343762-6666-6637-3532-346531316539", "query": { "categoryFilters": { "\\Diagnosis\\ICD-10-CM Diagnoses (2015) with ICD9\\Diseases of the respiratory system (J00-J99)\\Chronic lower respiratory diseases (J40-J47)\\Asthma\\Other and unspecified asthma\\Other asthma\\Exercise induced bronchospasm\\": [ "Billing Diagnosis", "Final" ] }, "numericFilters": { "\\Demographics\\Age\\": { "min": "10", "max": "15" } }, "requiredFields": [ "\\Bio Specimens\\NucleicAcid\\" ], "variantInfoFilters": [ { "categoryVariantInfoFilters": { "Consequence": [ "inframe_insertion" ] }, "numericVariantInfoFilters": { "AF": { "min": ".95", "max": "1" } } } ], "expectedResultType": "COUNT" } }

This query is not an example of how you should structure queries for other resources, it is just to illustrate the flexibility that is provided through PIC-SURE.

When considering the use-cases of your resource, it is important to consider scalability implications of the semantics of each of the ResourceRS specified endpoints. A /search call is synchronous, as is a /query/sync call. If you are making lots of calls across other resources or if your query processing time may exceed the timeout of intermediate proxies, you might want to implement your search instead using a /query endpoint because this is asynchronous.

A client would recieve the QueryStatus object back from the /query endpoint and poll the /query//status endpoint, which would result in calls to your resource at the /query//status endpoint. Once your resource returned a QueryStatus with a PicSureStatus of AVAILABLE, the client would retrieve the results from /query//result which would stream the response from your ResourceRS /query//result, so if you have large responses you might want to consider returning a link to S3 for instance to avoid streaming all that data through PIC-SURE itself.

Home

Clone this wiki locally