From cace65808718b08a6bc01ec9310b308e982f14df Mon Sep 17 00:00:00 2001 From: Bill Riehl Date: Thu, 19 Dec 2024 16:40:11 -0500 Subject: [PATCH] update docs --- README.md | 37 +++++++++++++++++++++++---------- import_specifications/readme.md | 6 +++++- 2 files changed, 31 insertions(+), 12 deletions(-) diff --git a/README.md b/README.md index e79e1ba8..fee3089d 100644 --- a/README.md +++ b/README.md @@ -819,21 +819,27 @@ Error Connecting to auth service ... ### Parse bulk specifications This endpoint parses one or more bulk specification files in the staging area -into a data -structure (close to) ready for insertion into the Narrative bulk import or -analysis cell. +into a data structure (close to) ready for insertion into the Narrative bulk +import or analysis cell. -It can parse `.tsv`, `.csv`, and Excel (`.xls` and `.xlsx`) files. Templates for -the currently -supported data types are available in +By default, it can parse `.tsv`, `.csv`, and Excel (`.xls` and `.xlsx`) files. +Templates for the currently supported data types are available in the [templates](./import_specifications/templates) directory of this repo. See the [README.md](./import_specifications/templates/README.md) file for instructions on template usage. +When given the `dts` flag in the URL, this endpoint can also parse manifest +files that come from the KBase [Data Transfer Service](https://github.com/kbase/dts) (DTS) +See [here](https://kbase.github.io/dts) for further documentation on the service. +This service copies data files from external sources into a KBase user's staging +area, accompanied by a `manifest.json` file. These also contain information on how to +load files into KBase importer apps. However, since they are a very specific format, +this means adding a `dts` flag to the URL. When this flag is present, all files +are expected to be `.json` files and conform to the [DTS schema](./import_specifications/schema/dts_manifest_schema.json). + See the [import specification ADR document](./docs/import_specifications.ADR.md) -for design -details. +for design details. **URL** : `ci.kbase.us/services/staging_service/bulk_specification` @@ -847,7 +853,7 @@ details. **Code** : `200 OK` -**Content example** +**Content examples** ``` GET bulk_specification/?files=file1.[,file2.,...] @@ -855,7 +861,16 @@ GET bulk_specification/?files=file1.[,file2.,...] `` is one of `csv`, `tsv`, `xls`, or `xlsx`. -Reponse: + +``` +GET bulk_specification/?files=file1.json[,file2.json,...]&dts +``` + +When using the `dts` flag, all files must be JSON and have the `.json` extension. + +Both versions of the endpoint respond in the same format. + +Response: ``` { @@ -1041,7 +1056,7 @@ POST write_bulk_specification/ - `data` contains any data to be written to the file as example data, and is analogous to the data structure returned from the parse endpoint. To specify that no data should be written to the template provide an empty list. - `` is the value for the input for a given `spec.json` ID and import or analysis instance, where an import/analysis instance is effectively a row in the data file. Each data file row is provided in order for each type. Each row is provided in a mapping of `spec.json` ID to the data for the row. Lines > 3 in the templates are user-provided data, and each line corresponds to a single import or analysis. -Reponse: +Response: ``` { diff --git a/import_specifications/readme.md b/import_specifications/readme.md index 22f8b717..45bcbda5 100644 --- a/import_specifications/readme.md +++ b/import_specifications/readme.md @@ -1,2 +1,6 @@ This directory contains templates for bulk import specifications to be parsed by the -import specfication endpoint, and example code for how to generate them automatically. \ No newline at end of file +import specfication endpoint, and example code for how to generate them automatically. + +This also contains a subdirectory for storing JSON Schemas that describe JSON-based +import specifications. These should be referenced by deployment and testing configuration +as necessary.