Merge remote-tracking branch 'refs/remotes/origin/bridgeScore' into b…

…ridgeScore

README.md

@@ -1,6 +1,6 @@
 # Score
 
-Score is a file transfer service designed for cloud-based projects, providing a robust API for secure file transfer and storage operations. It serves as an intermediary between object storage systems and user authorization mechanisms, using pre-signed URLs for efficient and protected data access.
+Score is a file transfer service designed to enable large-file upload and download, providing a robust API for secure file transfer and storage operations. It serves as an intermediary between object storage systems and user authorization mechanisms, using pre-signed URLs for efficient and protected data access.
 
 </br>
 
@@ -11,7 +11,20 @@ Score is a file transfer service designed for cloud-based projects, providing a
 > 
 > *Score is part of [Overture](https://www.overture.bio/), a collection of open-source software microservices used to create platforms for researchers to organize and share genomics data.*
 > 
-> 
+
+## Key Features
+
+- **Multi-cloud Support**: Compatible with AWS S3, Azure Storage, and any object storage with an S3 compliant API (Minio, Ceph, etc.)
+- **High-performance Transfers**: Implements multipart uploads and downloads for optimal throughput
+- **Genomic Data Handling (SamTools)**: Supports BAM/CRAM file slicing by genomic region and provides built-in samtools operations for BAM file handling
+- **Data Integrity**: Ensures file integrity through MD5 checksum validation on uploads and downloads
+- **Security**: Implements ACL-based security using OAuth2 with study code-scoped access
+- **Metadata Integration**: Integrates with the Song metadata management system for comprehensive data tracking
+- **File Bundling**: Enables efficient transfer of multiple files in a single bundle
+- **Resumable Downloads**: Supports resuming downloads after network interruptions
+- **FUSE Support**: Offers file system in Userspace (FUSE) support for enhanced file operations
+- **Interactive API Documentation:** Built-in Swagger UI for easy API interaction and exploration
+
 ## Repository Structure
 The repository is organized with the following directory structure:
 ```

docs/overview.md

@@ -1,27 +1,27 @@
 # Overview
 
-Score is a file transfer service designed for cloud-based projects, providing a robust API for secure file transfer and storage operations. It serves as an intermediary between object storage systems and user authorization mechanisms, using pre-signed URLs for efficient and protected data access.
+Score is a file transfer service designed to enable large-file upload and download, providign a robust API for secure file transfer and storage operations. It serves as an intermediary between object storage systems and user authorization mechanisms, using pre-signed URLs for efficient and protected data access.
 
 ## System Architecture
 
 Score's primary function is to broker authenticated access to your object storage provider. It achieves this by:
 
-1. Validating user access rights against an authorization system (Keycloak)
+1. Validating user access rights against an authorization system (OAuth)
 2. Generating time-limited pre-signed URLs for object access
 3. Facilitating secure data transfer between clients and object storage
 
 ![Score Arch](./assets/scoreArch.svg 'Score Architecture Diagram')
 
 As part of the larger Overture.bio software suite, Score is typically used with multiple other services including:
 
-- **Song:** A metadata management service made to manage file metadata independently from object storage concerns. 
+- **Song:** A metadata management service made to manage file metadata independently from object storage concerns
 - **Score Client:** A command line tool to streamline interactions with Scores REST API endpoints
 - **Keycloak:** The authorization and authentication service used to provided OAuth2 authentication for Score
 
 
 ## Key Features
 
-- **Multi-cloud Support**: Compatible with AWS S3, Azure Storage, and Google Cloud Storage
+- **Multi-cloud Support**: Compatible with AWS S3, Azure Storage, and any object storage with an S3 compliant API (Minio, Ceph, etc.)
 - **High-performance Transfers**: Implements multipart uploads and downloads for optimal throughput
 - **Genomic Data Handling (SamTools)**: Supports BAM/CRAM file slicing by genomic region and provides built-in samtools operations for BAM file handling
 - **Data Integrity**: Ensures file integrity through MD5 checksum validation on uploads and downloads
@@ -30,11 +30,11 @@ As part of the larger Overture.bio software suite, Score is typically used with
 - **File Bundling**: Enables efficient transfer of multiple files in a single bundle
 - **Resumable Downloads**: Supports resuming downloads after network interruptions
 - **FUSE Support**: Offers file system in Userspace (FUSE) support for enhanced file operations
-- **Interactive API Documentation:** Built-in Swagger UI for easy API interaction and exploration.
+- **Interactive API Documentation:** Built-in Swagger UI for easy API interaction and exploration
 
 
 ## Repository Structure
-
+The repository is organized with the following directory structure:
 ```
 .
 ├── /score-client
@@ -44,24 +44,8 @@ As part of the larger Overture.bio software suite, Score is typically used with
 └── /score-test
 ```
 
-[Click here to view the Score respository on GitHub ](https://github.com/overture-stack/score)
-
-#### Score-client
-
-[Explaination]
-
-#### Score-core
-
-[Explaination]
-
-#### Score-fs
-
-[Explaination]
-
-#### Score-server
-
-[Explaination]
-
-#### Score-test
-
-[Explaination]
+- **score-client:** Command line app for uploading and downloading files, published as a [docker container](https://github.com/overture-stack/score/pkgs/container/score) and availabe as an executable jar from [github releases](https://github.com/overture-stack/score/releases)
+- **score-core:** Core library containing shared utilities and data models used by all other packages
+- **score-fs:** File system operations module for managing local files
+- **score-server:** Main server application that handles object storage and transfers, published as a [docker container](https://github.com/overture-stack/score/pkgs/container/score-server)
+- **score-test:** Integration and end-to-end test suite for all packages

docs/setup.md

@@ -23,9 +23,9 @@ We'll use our Conductor service, a flexible Docker Compose setup, to spin up Sco
 
 2. Run the appropriate start command for your operating system:
 
-    | Operating System | Command |
-    |------------------|---------|
-    | Unix/macOS       | `make scoreDev` |
+    | Operating System | Command               |
+    | ---------------- | --------------------- |
+    | Unix/macOS       | `make scoreDev`       |
     | Windows          | `./make.bat scoreDev` |
 
     <details>
@@ -35,16 +35,16 @@ We'll use our Conductor service, a flexible Docker Compose setup, to spin up Sco
 
     ![ScoreDev](./assets/scoreDev.svg 'Score Dev Environment')
 
-    | Service | Port | Description | Purpose in Score Development |
-    |---------|------|-------------|------------------------------|
-    | Conductor | `9204` | Orchestrates deployments and environment setups | Manages the overall development environment |
-    | Keycloak-db | - | Database for Keycloak (no exposed port) | Stores Keycloak data for authentication |
-    | Keycloak | `8180` | Authorization and authentication service | Provides OAuth2 authentication for Score |
-    | Song-db | `5433` | Database for Song | Stores metadata managed by Song |
-    | Song | `8080` | Metadata management service | Manages metadata for files stored by Score |
-    | Minio | `9000` | Object storage provider | Simulates S3-compatible storage for Score |
+    | Service     | Port   | Description                                     | Purpose in Score Development                |
+    | ----------- | ------ | ----------------------------------------------- | ------------------------------------------- |
+    | Conductor   | `9204` | Orchestrates deployments and environment setups | Manages the overall development environment |
+    | Keycloak-db | -      | Database for Keycloak (no exposed port)         | Stores Keycloak data for authentication     |
+    | Keycloak    | `8180` | Authorization and authentication service        | Provides OAuth2 authentication for Score    |
+    | Song-db     | `5433` | Database for Song                               | Stores metadata managed by Song             |
+    | Song        | `8080` | Metadata management service                     | Manages metadata for files stored by Score  |
+    | Minio       | `9000` | Object storage provider                         | Simulates S3-compatible storage for Score   |
 
-    - Ensure all ports are free on your system before starting the environment.
+    - Ensure these ports are free on your system before starting the environment.
     - You may need to adjust the ports in the `docker-compose.yml` file if you have conflicts with existing services.
 
     For more information, see our [Conductor documentation linked here](/docs/other-software/Conductor)
@@ -99,16 +99,16 @@ We'll use our Conductor service, a flexible Docker Compose setup, to spin up Sco
         <summary>**Click here for a summary of the Score-server spring profiles**</summary>
 
         **Score Profiles**
-        | Profile | Description |
-        |---------|-------------|
-        | `default` | Common settings for all environments. Includes server, S3, bucket, object, upload, and authentication configurations. |
-        | `ssl` | Enables SSL configuration for using a self-signed certificate in production deployments. |
-        | `azure` | Configuration for Azure blob storage. Includes Azure-specific settings and bucket policies. |
-        | `s3` | Configuration for Amazon S3 or S3-compatible storage. Includes endpoint, access key, and secret key settings. |
-        | `prod` | Production environment configuration. Enables secure S3 connections and sets the metadata URL. |
-        | `secure` | Security configuration for OAuth2 and JWT. Includes settings for resource server, authentication server, and scope definitions. |
-        | `dev` | Development environment configuration. Uses non-secure S3 connections, local endpoints, and disables upload cleaning. |
-        | `benchmark` | Configuration for benchmarking purposes. Includes SSL settings and a non-secure S3 endpoint. |
+        | Profile     | Description                                                                                                                     |
+        | ----------- | ------------------------------------------------------------------------------------------------------------------------------- |
+        | `default`   | Common settings for all environments. Includes server, S3, bucket, object, upload, and authentication configurations.           |
+        | `ssl`       | Enables SSL configuration for using a self-signed certificate in production deployments.                                        |
+        | `azure`     | Configuration for Azure blob storage. Includes Azure-specific settings and bucket policies.                                     |
+        | `s3`        | Configuration for Amazon S3 or S3-compatible storage. Includes endpoint, access key, and secret key settings.                   |
+        | `prod`      | Production environment configuration. Enables secure S3 connections and sets the metadata URL.                                  |
+        | `secure`    | Security configuration for OAuth2 and JWT. Includes settings for resource server, authentication server, and scope definitions. |
+        | `dev`       | Development environment configuration. Uses non-secure S3 connections, local endpoints, and disables upload cleaning.           |
+        | `benchmark` | Configuration for benchmarking purposes. Includes SSL settings and a non-secure S3 endpoint.                                    |
 
         </details>
 

pom.xml

@@ -20,7 +20,7 @@ ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF S
 
   <groupId>bio.overture</groupId>
   <artifactId>score</artifactId>
-  <version>5.10.1-SNAPSHOT</version>
+  <version>5.12.0-SNAPSHOT</version>
   <packaging>pom</packaging>
   <name>${project.artifactId}</name>
   <description>${project.name}</description>

score-client/pom.xml

@@ -21,7 +21,7 @@ ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF S
   <parent>
     <groupId>bio.overture</groupId>
     <artifactId>score</artifactId>
-    <version>5.10.1-SNAPSHOT</version>
+    <version>5.12.0-SNAPSHOT</version>
     <relativePath>../pom.xml</relativePath>
   </parent>
 

score-client/src/main/resources/application.yml

@@ -91,7 +91,3 @@ logging:
     springframework.web: DEBUG
     com.amazonaws.services: DEBUG
 
----
-###############################################################################
-# Profile - "debug"
-###############################################################################

score-core/pom.xml

@@ -21,7 +21,7 @@ ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF S
   <parent>
     <groupId>bio.overture</groupId>
     <artifactId>score</artifactId>
-    <version>5.10.1-SNAPSHOT</version>
+    <version>5.12.0-SNAPSHOT</version>
     <relativePath>../pom.xml</relativePath>
   </parent>
 

score-fs/pom.xml

@@ -21,7 +21,7 @@ ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF S
   <parent>
     <groupId>bio.overture</groupId>
     <artifactId>score</artifactId>
-    <version>5.10.1-SNAPSHOT</version>
+    <version>5.12.0-SNAPSHOT</version>
     <relativePath>../pom.xml</relativePath>
   </parent>
 

score-server/pom.xml

@@ -21,7 +21,7 @@ ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF S
   <parent>
     <groupId>bio.overture</groupId>
     <artifactId>score</artifactId>
-    <version>5.10.1-SNAPSHOT</version>
+    <version>5.12.0-SNAPSHOT</version>
     <relativePath>../pom.xml</relativePath>
   </parent>
 

score-server/src/main/resources/application.yml

@@ -19,7 +19,8 @@
 ###############################################################################
 # Application Configurations for the Score Server
 #
-# This file contains various configuration profiles for the Score Server application.
+# This file contains placeholder configuration values for the various
+# application profiles used by Score.
 # Please update the values according to your specific environment.
 ###############################################################################
 
@@ -35,7 +36,7 @@ server:
     mime-types: application/json
   # Allowed origins for CORS (Cross-Origin Resource Sharing), multiple values can be added, separated by commas
   cors: 
-    allowedOrigins: http://localhost:8081 # Update this with your frontend application URL(s)
+    allowedOrigins: http://localhost:8081 # If fetching files from a web application, update this with your frontend application domain
 s3:
   # S3 connection settings
   secured: true                 # Use HTTPS for S3 connections
@@ -55,8 +56,8 @@ metadata:
   useLegacyMode: false          # Set to true to use legacy metadata handling
 bucket:
   name:
-    object: oicr.icgc           # Bucket name for object storage (update as needed)
-    state: oicr.icgc            # Bucket name for state storage (update as needed)
+    object: score.data           # Bucket name for object storage (update as needed)
+    state: score.data            # Bucket name for state storage (update as needed)
   size:
     pool: 0                     # Size of the connection pool
     key: 2                      # Size of the key used for encryption
@@ -83,10 +84,6 @@ management:
     web:
       cors:
         allowedOrigins: http://localhost:8081 # Update this with your management console URL(s)
-spring:
-  mvc:
-    pathmatch:
-      matching-strategy: ant_path_matcher  # Use Ant-style path matching
 logging:
   level:
     root: INFO
@@ -117,6 +114,9 @@ spring:
       # Connection Timeouts (Optional)
       # connection-timeout: 5000
       # read-timeout: 15000
+  mvc:
+    pathmatch:
+      matching-strategy: ant_path_matcher  # Use Ant-style path matching
 ---
 
 ###############################################################################
@@ -146,7 +146,7 @@ spring:
       on-profile: azure
 azure:
   endpointProtocol: https
-  accountName: oicricgc  # Replace with your Azure account name
+  accountName:  # Add your Azure account name
   accountKey:  # Add your Azure account key here
 bucket:
   name:
@@ -191,7 +191,7 @@ spring:
 s3:
   secured: true
 metadata:
-  url: https://meta.icgc.org  # Update with your actual metadata server URL
+  url: https://song.example.com  # Update with your actual metadata server URL
 
 ---
 

score-test/pom.xml

@@ -21,7 +21,7 @@ ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF S
   <parent>
     <groupId>bio.overture</groupId>
     <artifactId>score</artifactId>
-    <version>5.10.1-SNAPSHOT</version>
+    <version>5.12.0-SNAPSHOT</version>
     <relativePath>../pom.xml</relativePath>
   </parent>