From ea2756332a1974ab7f006ff708ef8b8422574ac6 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?J=C3=A9r=C3=B4me=20Gurhem?= Date: Tue, 3 Sep 2024 14:18:35 +0200 Subject: [PATCH 1/5] docs: Preview features description --- README.md | 36 ++++++++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+) diff --git a/README.md b/README.md index ddc904ae6..f59d017bf 100644 --- a/README.md +++ b/README.md @@ -18,6 +18,42 @@ ArmoniK.Core provides services for submitting computational tasks, keeping track More detailed information on the inner working of ArmoniK.Core is available [here](https://aneoconsulting.github.io/ArmoniK.Core/). +## Features availability + +As this repository is open-source and we follow trunk based development, all features will be integrated into main, in multiple small steps. +Breaking changes will be limited as much as possible or gated behind feature flags. +If breaking changes should happen, they will be documented in the releases. +Therefore, features in development or testing that cannot fit within a branch and need to be integrated into main will be marked as preview so that users know that these features are still in development and subject to changes. + +### General availability + +This section will list the features and plugins that are generally available, highly curated and for which support is available. + +- Queue plugins + - RabbitMQ + - AMQP + - PubSub +- Storage plugins + - Local + - Redis + - S3 +- Database plugins + - MongoDB + +### Preview + +This section will list features and plugins that are available on main but still in development. Their API should be stable enough for advanced users who want to test them before GA. Breaking changes will be notified in the release notes. Any feedback or issue encountered with these features are welcome ! + +- Queue plugins + - SQS + +### Beta (Internal preview) + +This section will list features and plugins in active development. They are not stable enough and breaking changes may occur without any notice. Breaking changes will not always be documented. Before they are moved to the "Preview" stage, issues might be closed with the minimal messages "Under Active Development". + +- Partitionning + - Partition metrics exporter + ## Installation ArmoniK.Core can be installed only on Linux machines. For Windows users, it is possible to do it on [WSL2](https://learn.microsoft.com/en-us/windows/wsl/about). From dea71bbb9b72b4182493637619cc4556c4cc7159 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?J=C3=A9r=C3=B4me=20Gurhem?= <88328270+aneojgurhem@users.noreply.github.com> Date: Tue, 3 Sep 2024 14:37:11 +0200 Subject: [PATCH 2/5] Add suggestion on tests by community Co-authored-by: ngruelaneo <100275739+ngruelaneo@users.noreply.github.com> --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index f59d017bf..c58f13489 100644 --- a/README.md +++ b/README.md @@ -49,7 +49,7 @@ This section will list features and plugins that are available on main but still ### Beta (Internal preview) -This section will list features and plugins in active development. They are not stable enough and breaking changes may occur without any notice. Breaking changes will not always be documented. Before they are moved to the "Preview" stage, issues might be closed with the minimal messages "Under Active Development". +This section will list features and plugins in active development. They are not stable enough and breaking changes may occur without any notice. Breaking changes will not always be documented. They can be used by the community for testing purposes. Before to be moved to the "Preview" stage, issues might be closed with the minimal messages "Under Active Development". - Partitionning - Partition metrics exporter From 29cf37f132504af2a8772f2c1e71c5ba96225b6e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?J=C3=A9r=C3=B4me=20Gurhem?= Date: Thu, 5 Sep 2024 14:48:02 +0200 Subject: [PATCH 3/5] add deprecated section --- README.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/README.md b/README.md index c58f13489..69d59b7e5 100644 --- a/README.md +++ b/README.md @@ -54,6 +54,12 @@ This section will list features and plugins in active development. They are not - Partitionning - Partition metrics exporter +### Deprecated + +This section will list the deprecated features and plugins. Features are flagged as deprecated then they will be removed in the future. They will not be modified anymore and kept as-is until deletion. + +- gRPC service called `Submitter`. + ## Installation ArmoniK.Core can be installed only on Linux machines. For Windows users, it is possible to do it on [WSL2](https://learn.microsoft.com/en-us/windows/wsl/about). From 2b7472279f1cdbf8434cd77579d2b5a4e42298bb Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?J=C3=A9r=C3=B4me=20Gurhem?= Date: Tue, 10 Sep 2024 09:14:20 +0200 Subject: [PATCH 4/5] refactor: use table for features, API and plugin status --- README.md | 86 ++++++++++++++++++++++++++++++++----------------------- 1 file changed, 50 insertions(+), 36 deletions(-) diff --git a/README.md b/README.md index 69d59b7e5..32401d6ba 100644 --- a/README.md +++ b/README.md @@ -18,47 +18,61 @@ ArmoniK.Core provides services for submitting computational tasks, keeping track More detailed information on the inner working of ArmoniK.Core is available [here](https://aneoconsulting.github.io/ArmoniK.Core/). -## Features availability +## Development process -As this repository is open-source and we follow trunk based development, all features will be integrated into main, in multiple small steps. +This repository follows trunk based development so all features will be integrated into main, in multiple small steps. Breaking changes will be limited as much as possible or gated behind feature flags. If breaking changes should happen, they will be documented in the releases. Therefore, features in development or testing that cannot fit within a branch and need to be integrated into main will be marked as preview so that users know that these features are still in development and subject to changes. -### General availability - -This section will list the features and plugins that are generally available, highly curated and for which support is available. - -- Queue plugins - - RabbitMQ - - AMQP - - PubSub -- Storage plugins - - Local - - Redis - - S3 -- Database plugins - - MongoDB - -### Preview - -This section will list features and plugins that are available on main but still in development. Their API should be stable enough for advanced users who want to test them before GA. Breaking changes will be notified in the release notes. Any feedback or issue encountered with these features are welcome ! - -- Queue plugins - - SQS - -### Beta (Internal preview) - -This section will list features and plugins in active development. They are not stable enough and breaking changes may occur without any notice. Breaking changes will not always be documented. They can be used by the community for testing purposes. Before to be moved to the "Preview" stage, issues might be closed with the minimal messages "Under Active Development". - -- Partitionning - - Partition metrics exporter - -### Deprecated - -This section will list the deprecated features and plugins. Features are flagged as deprecated then they will be removed in the future. They will not be modified anymore and kept as-is until deletion. - -- gRPC service called `Submitter`. +**General availability:** The APIs, plugins and features that are generally available are highly curated and for which support is available. +**Preview:** The APIs, plugins and features that are available on main but still in development. Their API should be stable enough for advanced users who want to test them before GA. Breaking changes should be minimal and will be notified in the release notes. Any feedback or issue encountered with these features are welcome ! +**Beta (Internal preview):** The APIs, plugins and features in active development. They are not stable enough and breaking changes may occur without any notice. Breaking changes will not always be documented. They can be used by the community for testing purposes. Before to be moved to the "Preview" stage, issues might be closed with the minimal messages "Under Active Development". +**Deprecated:** The deprecated APIs, plugins and features. Features are flagged for future removal. Only production critical bug fixes will be considered. + +| Plugins | Type | Description | Status | +|----------|----------------|-----------------------------------------------|---------| +| RabbitMQ | Queue | RabbitMQ client using AMQP protocol 0.9.2 | GA | +| AMQP | Queue | AMQP client using AMQP protocol 1.0.0 | GA | +| PubSub | Queue | Google PubSub client | GA | +| SQS | Queue | AWS SQS client | Preview | +| Redis | Object Storage | Redis client used to store binary data | GA | +| Local | Object Storage | File system used to store binary data | GA | +| S3 | Object Storage | AWS S3 client used to store binary data | GA | +| MongoDB | Database | MongoDB client to store ArmoniK internal data | GA | + +| APIs | Description | Status | +|-------------------------------|-------------------------------------------------------------------------------------|------------| +| Agent.CreateTask | Streamed RPC to submit payloads and tasks from the worker | Deprecated | +| Agent.Get*Data | RPC to get special Results | Beta | +| Agent.* | Service to submit created results, create new results and new tasks from the worker | GA | +| Applications.ListApplications | RPC to list applications | GA | +| Authentication.GetCurrentUser | RPC to retrieve the current logged-in user | GA | +| Events.GetEvents | Streamed RPC to subscribe to events happening on tasks and results during execution | GA | +| HealthChecks.CheckHealth | RPC to get an overview of the cluster health | GA | +| Partitions.ListPartitions | RPC to list partitions | GA | +| Partitions.GetPartition | RPC to get partition information | GA | +| Partition metrics exporter | Metrics exporter to expose metrics on partition instance sharing | Beta | +| Results.WatchResults | Bi-directionnal RPC to subscribe to events on Results | Beta | +| Results.* | Service to create, download, list and upload results | GA | +| Sessions.* | Service to create, list, cancel, pause, resume, close etc sessions | GA | +| Submitter.* | Old streamed service to create tasks and payloads altogether | Deprecated | +| Tasks.* | Service to submit, list and cancel tasks | GA | +| Versions.* | Service to retrieve | GA | +| Worker.* | Service implemeted by users that consumes the tasks scheduled by ArmoniK | GA | + +| Docker Image Platform | Status | +|-----------------------|---------| +| linux/amd64 | GA | +| linux/arm64 | Preview | +| windows/amd64 | Beta | + + ## Installation From c7fd38a8826ab5f6f7c5c2f5ea371fdf470ce3bf Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?J=C3=A9r=C3=B4me=20Gurhem?= Date: Tue, 10 Sep 2024 16:05:21 +0200 Subject: [PATCH 5/5] docs: take suggestions into account --- README.md | 19 ++++++++++--------- 1 file changed, 10 insertions(+), 9 deletions(-) diff --git a/README.md b/README.md index 32401d6ba..c91ec8895 100644 --- a/README.md +++ b/README.md @@ -25,10 +25,11 @@ Breaking changes will be limited as much as possible or gated behind feature fla If breaking changes should happen, they will be documented in the releases. Therefore, features in development or testing that cannot fit within a branch and need to be integrated into main will be marked as preview so that users know that these features are still in development and subject to changes. -**General availability:** The APIs, plugins and features that are generally available are highly curated and for which support is available. +**General availability (GA):** The APIs, plugins and features are thoroughly tested and support is available. **Preview:** The APIs, plugins and features that are available on main but still in development. Their API should be stable enough for advanced users who want to test them before GA. Breaking changes should be minimal and will be notified in the release notes. Any feedback or issue encountered with these features are welcome ! -**Beta (Internal preview):** The APIs, plugins and features in active development. They are not stable enough and breaking changes may occur without any notice. Breaking changes will not always be documented. They can be used by the community for testing purposes. Before to be moved to the "Preview" stage, issues might be closed with the minimal messages "Under Active Development". +**Beta (Internal preview):** The APIs, plugins and features in active development. They are not stable enough and breaking changes may occur without any notice. Breaking changes will not always be documented. They can be used by the community for testing purposes. Before being moved to the "Preview" stage, issues might be closed with the minimal messages "Under Active Development". **Deprecated:** The deprecated APIs, plugins and features. Features are flagged for future removal. Only production critical bug fixes will be considered. +**Work in progress (WIP):** The APIs, plugins and features are not fully implemented and may have some missing parts. | Plugins | Type | Description | Status | |----------|----------------|-----------------------------------------------|---------| @@ -44,22 +45,22 @@ Therefore, features in development or testing that cannot fit within a branch an | APIs | Description | Status | |-------------------------------|-------------------------------------------------------------------------------------|------------| | Agent.CreateTask | Streamed RPC to submit payloads and tasks from the worker | Deprecated | -| Agent.Get*Data | RPC to get special Results | Beta | +| Agent.Get*Data | RPC to get special Results | WIP | | Agent.* | Service to submit created results, create new results and new tasks from the worker | GA | | Applications.ListApplications | RPC to list applications | GA | | Authentication.GetCurrentUser | RPC to retrieve the current logged-in user | GA | -| Events.GetEvents | Streamed RPC to subscribe to events happening on tasks and results during execution | GA | +| Events.GetEvents | Streamed RPC to subscribe to task and result events during execution | GA | | HealthChecks.CheckHealth | RPC to get an overview of the cluster health | GA | | Partitions.ListPartitions | RPC to list partitions | GA | | Partitions.GetPartition | RPC to get partition information | GA | -| Partition metrics exporter | Metrics exporter to expose metrics on partition instance sharing | Beta | -| Results.WatchResults | Bi-directionnal RPC to subscribe to events on Results | Beta | +| Partition metrics exporter | Metrics exporter to expose partition metrics on shared resources | Beta | +| Results.WatchResults | Bi-directionnal RPC to subscribe to events on Results | WIP | | Results.* | Service to create, download, list and upload results | GA | | Sessions.* | Service to create, list, cancel, pause, resume, close etc sessions | GA | -| Submitter.* | Old streamed service to create tasks and payloads altogether | Deprecated | +| Submitter.* | Deprecated streamed service to create tasks and payloads altogether | Deprecated | | Tasks.* | Service to submit, list and cancel tasks | GA | -| Versions.* | Service to retrieve | GA | -| Worker.* | Service implemeted by users that consumes the tasks scheduled by ArmoniK | GA | +| Versions.* | Service to retrieve versions | GA | +| Worker.* | Service implemented by users that consumes the tasks scheduled by ArmoniK | GA | | Docker Image Platform | Status | |-----------------------|---------|