Merge branch 'release/drs-1.1.0'

README.md

@@ -30,11 +30,12 @@ For more information see our HTML documentation links in the table below.
 
 |  **Branch** | **Reference Documentation** | **[OpenAPI YAML description](openapi/data_repository_service.swagger.yaml)** |
 | --- | --- | --- |
-| **master**: the current release | [HTML](https://ga4gh.github.io/data-repository-service-schemas/docs/) | [Swagger](https://ga4gh.github.io/data-repository-service-schemas/swagger-ui/#/DataRepositoryService/) |
+| **master**: The current release | [HTML](https://ga4gh.github.io/data-repository-service-schemas/docs/) | [Swagger](https://ga4gh.github.io/data-repository-service-schemas/swagger-ui/#/DataRepositoryService/) |
 | **develop**: the stable development branch, into which feature branches are merged | [HTML](https://ga4gh.github.io/data-repository-service-schemas/preview/develop/docs/) | [Swagger](https://ga4gh.github.io/data-repository-service-schemas/preview/develop/swagger-ui/#/DataRepositoryService/) |
-| **release 1.0.0**: the 1.0.0 release of DRS that we are submitting to GA4GH for standards approval | [HTML](https://ga4gh.github.io/data-repository-service-schemas/preview/release/drs-1.0.0/docs/) | [Swagger](https://ga4gh.github.io/data-repository-service-schemas/preview/release/drs-1.0.0/swagger-ui/#/DataRepositoryService/) |
-| **release 0.1**: simplifying DRS to core functionality | [HTML](https://ga4gh.github.io/data-repository-service-schemas/preview/release/drs-0.1.0/docs/) | [Swagger](https://ga4gh.github.io/data-repository-service-schemas/preview/release/drs-0.1.0/swagger-ui/#/DataRepositoryService/) |
-| **release 0.0.1**: the initial DRS after the rename from DOS | [HTML](https://ga4gh.github.io/data-repository-service-schemas/preview/release/0.0.1/docs/) | [Swagger](https://ga4gh.github.io/data-repository-service-schemas/preview/release/0.0.1/swagger-ui/#/DataRepositoryService/) |
+| **release 1.1.0**: The 1.1.0 release of DRS that includes *no* API changes only documentation changes. This introduces a new URI convention using compact identifiers along with clear directions on how to use identifiers.org/n2t.net to resolve them. | [HTML](https://ga4gh.github.io/data-repository-service-schemas/preview/release/drs-1.1.0/docs/) | [Swagger](https://ga4gh.github.io/data-repository-service-schemas/preview/release/drs-1.1.0/swagger-ui/#/DataRepositoryService/) |
+| **release 1.0.0**: The 1.0.0 release of DRS that is now an approved GA4GH standard | [HTML](https://ga4gh.github.io/data-repository-service-schemas/preview/release/drs-1.0.0/docs/) | [Swagger](https://ga4gh.github.io/data-repository-service-schemas/preview/release/drs-1.0.0/swagger-ui/#/DataRepositoryService/) |
+| **release 0.1**: Simplifying DRS to core functionality | [HTML](https://ga4gh.github.io/data-repository-service-schemas/preview/release/drs-0.1.0/docs/) | [Swagger](https://ga4gh.github.io/data-repository-service-schemas/preview/release/drs-0.1.0/swagger-ui/#/DataRepositoryService/) |
+| **release 0.0.1**: The initial DRS after the rename from DOS | [HTML](https://ga4gh.github.io/data-repository-service-schemas/preview/release/0.0.1/docs/) | [Swagger](https://ga4gh.github.io/data-repository-service-schemas/preview/release/0.0.1/swagger-ui/#/DataRepositoryService/) |
 
 To monitor development work on various branches, add 'preview/\<branch-name\>' to the master URLs above (e.g., 'https://ga4gh.github.io/data-repository-service-schemas/preview/\<branch-name\>/docs').
 

build.gradle

@@ -61,7 +61,7 @@ asciidoctor {
     sourceDir asciiDocDir
     outputDir file("docs")
     sources {
-        include 'index.adoc'
+        include 'index.adoc', 'more_background_on_compact_identifiers.adoc'
     }
     backends = ['html5', 'pdf']
     attributes = [

docs/asciidoc/back_matter.adoc

@@ -1,5 +1,3 @@
-
-
 == Appendix: Motivation
 
 [cols="40a,60a"]
@@ -39,3 +37,88 @@ This spec defines a standard **Data Repository Service (DRS) API** (“the yello
 The world's biomedical data is controlled by groups with very different policies and restrictions on where their data lives and how it can be accessed. A primary purpose of DRS is to support unified access to disparate and distributed data. (As opposed to the alternative centralized model of "let's just bring all the data into one single data repository”, which would be technically easier but is no more realistic than “let’s just bring all the websites into one single web host”.)
 
 In a DRS-enabled world, tool builders don’t have to worry about where the data their tools operate on lives -- they can count on DRS to give them access. And tool users only need to know which DRS server is managing the data they need, and whether they have permission to access it; they don’t have to worry about how to physically get access to, or (worse) make a copy of the data. For example, if I have appropriate permissions, I can run a pooled analysis where I run a single tool across data managed by different DRS servers, potentially in different locations.
+
+== Appendix: Background Notes on DRS URIs
+
+=== Design Motivation
+
+DRS URIs are aligned with the https://www.nature.com/articles/sdata201618[FAIR data principles] and the https://doi.org/10.1038/sdata.2018.2[Joint Declaration of Data Citation Principles] -- both hostname-based and compact identifier-based URIs provide globally unique, machine-resolvable, persistent identifiers for data.
+
+* We require all URIs to begin with `drs://` as a signal to humans and  systems consuming these URIs that the response they will ultimately receive, after transforming the URI to a fetchable URL, will be a DRS JSON packet. This signal differentiates DRS URIs from the wide variety of other entities (HTML documents, PDFs, ontology notes, etc.) that can be represented by compact identifiers.
+* We support hostname-based URIs because of their simplicity and efficiency for server and client implementers.
+* We support compact identifier-based URIs, and the meta-resolver services of identifiers.org and n2t.net (Name-to-Thing), because of the wide adoption of compact identifiers in the research community. as detailed by https://doi.org/10.1038/sdata.2018.29[Wimalaratne et al (2018)] in "Uniform resolution of compact identifiers for biomedical data."
+
+== Appendix: Compact Identifier-Based URIs
+
+.Note: Identifiers.org/n2t.net API Changes
+****
+The examples below show the current API interactions with https://n2t.net/e/compact_ids.html[n2t.net] and https://docs.identifiers.org/[identifiers.org] which may change over time.  Please refer to the documentation from each site for the most up-to-date information.  We will make best efforts to keep the DRS specification current but DRS clients MUST maintain their ability to use either the identifiers.org or n2t.net APIs to resolve compact identifier-based DRS URIs.
+****
+
+=== Registering a DRS Server on a Meta-Resolver
+
+See the documentation on the https://n2t.net/e/compact_ids.html[n2t.net] and https://docs.identifiers.org/[identifiers.org] meta-resolvers for adding your own compact identifier type and registering your DRS server as a resolver. You can register new prefixes (or mirrors by adding resource provider codes) for free using a simple online form.  For more information see link:more_background_on_compact_identifiers[More Background on Compact Identifiers].
+
+=== Calling Meta-Resolver APIs for Compact Identifier-Based DRS URIs
+
+Clients resolving Compact Identifier-based URIs need to convert a prefix (e.g. “drs.42”) into an URL pattern. They can do so by calling either the identifiers.org or the n2t.net API, since the two meta-resolvers keep their mapping databases in sync.
+
+==== Calling the identifiers.org API as a Client
+
+It takes two API calls to get the URL pattern.
+
+(i) The client makes a GET request to identifiers.org to find information about the prefix:
+
+  GET https://registry.api.identifiers.org/restApi/namespaces/search/findByPrefix?prefix=drs.42
+
+This request returns a JSON structure including various URLs containing an embedded namespace id, such as:
+
+  "namespace" : {
+   "href":"https://registry.api.identifiers.org/restApi/namespaces/1234"
+  }
+
+(ii) The client extracts the namespace id (in this example 1234), and uses it to make a second GET request to identifiers.org to find information about the namespace:
+
+  GET https://registry.api.identifiers.org/restApi/resources/search/findAllByNamespaceId?id=1234
+
+This request returns a JSON structure including an urlPattern field, whose value is an URL pattern containing a `${id}` parameter, such as:
+
+  "urlPattern" : "https://drs.myexample.org/ga4gh/drs/v1/objects/{$id}"
+
+==== Calling the n2t.net API as a Client
+
+It takes one API call to get the URL pattern.
+
+The client makes a GET request to n2t.net to find information about the namespace. (Note the trailing colon.)
+
+  GET https://n2t.net/drs.42:
+
+This request returns a text structure including a redirect field, whose value is an URL pattern containing a `$id` parameter, such as:
+
+   redirect: https://drs.myexample.org/ga4gh/drs/v1/objects/$id
+
+=== Caching with Compact Identifiers
+
+Identifiers.org/n2t.net compact identifier resolver records do not change frequently.  This reality is  useful for caching resolver records and their URL patterns for performance reasons.  Builders of systems that use compact identifier-based DRS URIs should cache prefix resolver records from identifiers.org/n2t.net and occasionally refresh the records (such as every 24 hours).  This approach will reduce the burden on these community services since we anticipate many DRS URIs will be regularly resolved in workflow systems. Alternatively, system builders may decide to directly mirror the registries themselves, instructions are provided on the identifiers.org/n2t.net websites.
+
+=== Security with Compact Identifiers
+
+As mentioned earlier, identifiers.org/n2t.net performs some basic verification of new prefixes and provider code mirror registrations on their sites.  However, builders of systems that consume and resolve DRS URIs may have certain security compliance requirements and regulations that prohibit relying on an external site for resolving compact identifiers.  In this case, systems under these security and compliance constraints may wish to whitelist certain compact identifier resolvers and/or vet records from identifiers.org/n2t.net before enabling in their systems.
+
+=== Accession Encoding to Valid DRS IDs
+
+The compact identifier format used by identifiers.org/n2t.net does not percent-encode reserved URI characters but, instead, relies on the first ":" character to separate prefix from accession. Since these accessions can contain any characters, and characters like "/" will interfere with DRS API calls, you _must_ percent encode the accessions extracted from DRS compact identifier-based URIs when using as DRS IDs in subsequent DRS GET requests.  An easy way for a DRS client to handle this is to get the initial DRS object JSON response from whatever redirects the compact identifier resolves to, then look for the `self_uri` in the JSON, which will give you the correctly percent-encoded DRS ID for subsequent DRS API calls such as the `access` method.
+
+=== Additional Examples
+
+For additional examples, see the document link:more_background_on_compact_identifiers[More Background on Compact Identifiers].
+
+== Appendix: Hostname-Based URIs
+
+=== Encoding DRS IDs
+
+In hostname-based DRS URIs, the ID is always percent-encoded to ensure special characters do not interfere with subsequent DRS endpoint calls.  As such, ":" is not allowed in the URI and is a convenient way of differentiating from a compact identifier-based DRS URI.  Also, if a given DRS service implementation uses compact identifier accessions as their DRS IDs, they must be percent encoded before using them as DRS IDs in hostname-based DRS URIs and subsequent GET requests to a DRS service endpoint.
+
+=== Future DRS Versions and Service Registry/Info
+
+In the future, as new major versions of DRS are released, a DRS server might support multiple API versions on different URL paths. At that point we expect to add support for https://github.com/ga4gh-discovery/ga4gh-service-registry[service-registry] and https://github.com/ga4gh-discovery/ga4gh-service-info[service-info] endpoints to the API, and to update the URI resolution logic to describe how to use those endpoints when translating hostname-based DRS URIs to URLs.