docs: update docs to explain path style/virtual-hosted style addressi…

…ng and clarify local-storage README.md
umccr · Mar 21, 2024 · 233f1de · 233f1de
1 parent 173c433
commit 233f1de
Show file tree

Hide file tree

Showing 3 changed files with 35 additions and 0 deletions.
diff --git a/deploy/examples/local_storage/README.md b/deploy/examples/local_storage/README.md
@@ -10,6 +10,35 @@ docker compose up
 
 This launches a `LocalStorage` htsget-actix server serving data from the [`data`][data] directory.
 
+The htsget-rs server can then be queried:
+
+```sh
+curl http://127.0.0.1:8080/reads/data/bam/htsnexus_test_NA12878
+```
+
+Which outputs:
+```sh
+{
+  "htsget": {
+    "format": "BAM",
+    "urls": [
+      {
+        "url": "http://0.0.0.0:8081/data/bam/htsnexus_test_NA12878.bam",
+        "headers": {
+          "Range": "bytes=0-2596770"
+        }
+      },
+      {
+        "url": "data:;base64,H4sIBAAAAAAA/wYAQkMCABsAAwAAAAAAAAAAAA=="
+      }
+    ]
+  }
+}
+```
+
+The volumes of the [`compose.yml`][compose] can be changed to any directory to serve data from it using
+default settings, and `curl http://127.0.0.1:8080/reads/data/<id>`, noting the extra `data` prefix.
+
 [local]: ../../../htsget-config/README.md#resolvers
 [compose]: compose.yml
 [data]: ../../../data
diff --git a/deploy/examples/local_storage/compose.yml b/deploy/examples/local_storage/compose.yml
@@ -7,4 +7,5 @@ services:
       - "8080:8080"
       - "8081:8081"
     volumes:
+      # Change this to any data location to serve files using default settings.
       - ./../../../data:/data
diff --git a/deploy/examples/minio/README.md b/deploy/examples/minio/README.md
@@ -11,6 +11,10 @@ used by MinIO.
 set on the MinIO server and DNS resolution must allow accessing the MinIO server using `bucket.<MINIO_DOMAIN>`.
     * Path style addressing can be used instead by setting `path_style = true` under the htsget-rs resolvers storage type.
 
+The caveats around the addressing style occur because there are two different addressing styles for S3 buckets, path style, e.g.
+`http://minio:9000/bucket`, and virtual-hosted style, e.g. `http://bucket.minio:9000`. AWS has declared path style addressing
+as [deprecated][path-style-deprecated], so this example sets up virtual-hosted style addressing as the default. 
+
 ## Deployment using Docker
 
 The above configuration can be applied using docker-compose to set the relevant environment variables. Additionally, if using
@@ -57,6 +61,7 @@ The url tickets can then be fetched within the compose network context:
 docker exec -it minio curl -H "Range: bytes=0-2596770" "http://data.minio:9000/bam/htsnexus_test_NA12878.bam?x-id=GetObject&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=user%2F20240320%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20240320T014007Z&X-Amz-Expires=1000&X-Amz-SignedHeaders=host%3Brange&X-Amz-Signature=33a75bd6363ccbfd5ce8edf7e102a5edff8ca7cee17e3c654db01a880e98072d"
 ```
 
+[path-style-deprecated]: https://aws.amazon.com/blogs/aws/amazon-s3-path-deprecation-plan-the-rest-of-the-story/
 [storage]: ../../../htsget-config/README.md#resolvers
 [minio]: https://min.io/
 [env-variables]: https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-envvars.html