-
Notifications
You must be signed in to change notification settings - Fork 59
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[KOGITO-9940] Add documentation for Jobs Service support in the Opera…
…tor (#530)
- Loading branch information
Showing
3 changed files
with
177 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
168 changes: 168 additions & 0 deletions
168
serverlessworkflow/modules/ROOT/pages/cloud/operator/enabling-jobs-service.adoc
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,168 @@ | ||
= Managing Jobs Service with the Operator | ||
:compat-mode!: | ||
// Metadata: | ||
:description: Configure Jobs Service using the `SonataFlowPlatform` CR. | ||
:keywords: sonataflow, serverless, operator, kubernetes, jobs service | ||
|
||
|
||
This document describes how to configure the Jobs Service instance using the SonataFlowPlarform CR. | ||
|
||
== Automate the Jobs Service instance management with the `SonataFlow` Operator | ||
|
||
It is possible to deploy the Jobs Service manually, leveraging the Operator offers a more seamless integration by | ||
combining it with namespace configuration through the SonataFlowPlatform Custom Resource (CR). When the Operator oversees | ||
the lifecycle of the jobs service, it automatically injects necessary properties during creation into the SonataFlow workflows. | ||
This integration eliminates the need for including these properties within the SonataFlow workflow CR instance, simplifying workflow management. | ||
== Configuring Jobs Service in the SonataFlowPlatform CR | ||
|
||
To enable the deployment of a Jobs Service instance, the `SonataFlowPlatform` CRD exposes a set of fields that allow the user to | ||
configure the running instance. | ||
|
||
=== Ephemeral persistence | ||
The basic runtime is to deploy the Jobs Service with an ephemeral backend running in the same container | ||
as the Jobs Service runtime. | ||
|
||
[source,yaml,subs="attributes+"] | ||
--- | ||
apiVersion: sonataflow.org/v1alpha08 | ||
kind: SonataFlowPlatform | ||
metadata: | ||
name: sonataflow-platform | ||
spec: | ||
services: | ||
jobService: {} | ||
--- | ||
|
||
When executing this manifest, the operator will reconcile generating a pod hosting the Jobs Service: | ||
|
||
[source,shell,subs="attributes+"] | ||
--- | ||
$>kubectl get pod -n sonataflow | ||
NAME READY STATUS RESTARTS AGE | ||
sonataflow-platform-jobs-service-cdf85d969-sbwkj 1/1 Running 0 108s | ||
--- | ||
|
||
Keep in mind that this setup is not recommended for production environments, especially because the data does not persist when the pod restarts. | ||
|
||
=== Using an existing PostgreSQL service | ||
For robust environments it is recommened to use an dedicated database service and configure Jobs Service to make use of it. Currently, the Jobs Service | ||
only supports PostgreSQL database. | ||
|
||
Configuring Jobs Service to communicate with an existing PostgreSQL instance is supported in two ways. In both cases it requires providing the persistence | ||
configuration, one by using the Jobs Service's persistence field and the other one using the persistence field defined in the `SonataFlowPlatform` CR | ||
deployed in the same namespace. | ||
|
||
By default, the persistence specification defined in the `SonataFlow` workflow's CR takes priority over the one in the `SonataFlowPlatform` persistence specification. | ||
|
||
==== Using the persistence field defined in the `SonataFlowPlatform` CR | ||
Using the persistence configuration in the `SonataFlowPlatform` CR located in the same namespace requires to have the `SonataFlow` CR persistence field configured | ||
to have an empty `{}` value, signaling the Operator to derive the persistence from the active `SonataFlowPlatform`, when available. If no persistence is defined | ||
the operator will fallback to the ephemeral persistence previously described. | ||
|
||
[source,yaml,subs="attributes+"] | ||
--- | ||
apiVersion: sonataflow.org/v1alpha08 | ||
kind: SonataFlowPlatform | ||
metadata: | ||
name: sonataflow-platform | ||
spec: | ||
persistence: | ||
postgresql: | ||
secretRef: | ||
name: postgres-secrets | ||
userKey: POSTGRES_USER | ||
passwordKey: POSTGRES_PASSWORD | ||
serviceRef: | ||
name: postgres | ||
port: 5432 | ||
databaseName: sonataflow | ||
--- | ||
|
||
And the `SonataFlow` CR looks like this: | ||
|
||
[source,yaml,subs="attributes+"] | ||
--- | ||
apiVersion: sonataflow.org/v1alpha08 | ||
kind: SonataFlow | ||
metadata: | ||
name: callbackstatetimeouts | ||
annotations: | ||
sonataflow.org/description: Callback State Timeouts Example k8s | ||
sonataflow.org/version: 0.0.1 | ||
spec: | ||
persistence: {} | ||
... | ||
--- | ||
|
||
When using the `jdbcUrl` field instead of `serviceRef`, the user is responsible for providing the correct JDBC URL connection value that does not contain a `database schema` | ||
because the operator will use verbatim the contents of this field as the JDBC connection in the Jobs Service pod, and if it provides a schema that has been used or formatted | ||
by a different client, the pod will fail to run. | ||
|
||
===== Using the persistence field inside the service specification | ||
You can specify define the persistence configuration directly in the Jobs Service specification. The structure is the same as in the `SonataFlowPlatform` CR and also | ||
consist on the credentials to access the PostgreSQL instance, and the kubernetes service reference to generate the connectivity. | ||
|
||
[source,yaml,subs="attributes+"] | ||
--- | ||
apiVersion: sonataflow.org/v1alpha08 | ||
kind: SonataFlowPlatform | ||
metadata: | ||
name: callbackstatetimeouts | ||
spec: | ||
persistence: | ||
postgresql: | ||
secretRef: | ||
name: postgres-secrets | ||
userKey: POSTGRES_USER | ||
passwordKey: POSTGRES_PASSWORD | ||
serviceRef: | ||
name: postgres | ||
port: 5432 | ||
databaseName: sonataflow | ||
databaseSchema: jobs-service | ||
--- | ||
|
||
In this example we're using a PostgreSQL service located in the same namespace where the Jobs Service is deployed, pointing to the default PostgreSQL port of 5432 | ||
and referencing the database `sonataflow` and schema `jobs-service`. By default, when the Jobs Service pod starts it will recreate the schema in the given location | ||
if it doesn't exist. | ||
|
||
The values of `POSTGRES_USER` and `POSTGRES_PASSWORD` point to the keys in the secret that contains the PostgreSQL credentials. | ||
|
||
[source,shell,subs="attributes+"] | ||
--- | ||
$>kubectl describe secrets postgres-secrets | ||
Name: postgres-secrets | ||
Namespace: default | ||
Labels: <none> | ||
Annotations: <none> | ||
|
||
Type: Opaque | ||
|
||
Data | ||
\\\\==== | ||
PGDATA: 28 bytes | ||
POSTGRES_DATABASE: 10 bytes | ||
POSTGRES_PASSWORD: 10 bytes | ||
POSTGRES_USER: 10 bytes | ||
--- | ||
|
||
Putting everything together will render an output similar to this one below: | ||
|
||
[source,shell,subs="attributes+"] | ||
--- | ||
$>kubectl get pod -w -n sonataflow | ||
NAME READY STATUS RESTARTS AGE | ||
postgres-64c9ddb6bc-v96qb 1/1 Running 0 14m | ||
sonataflow-platform-jobs-service-79bb6bc67f-gvcg4 1/1 Running 0 1m | ||
--- | ||
|
||
== Conclusion | ||
The Operator extends its scope to manage the lifecycle of the Jobs Service instance, thus removing the burden on the users and allowing them to focus on the | ||
implementation of the `Sonataworkflows. It takes care of managing the Jobs Service deployment, dynamically configures its application properties | ||
when the Data Index service is also available and managed by the Operator, and finally inject application properties in `SonataFlow` workflows in the target | ||
namespace when the Jobs Service is available. | ||
|
||
== Additional resources | ||
* xref:cloud/operator/install-serverless-operator.adoc[] | ||
|
||
include::../../../pages/_common-content/report-issue.adoc[] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters