Camunda 8 Saas connection

camunda-community-hub · Oct 19, 2023 · 732fdbe · 732fdbe
1 parent 6200cbe
commit 732fdbe
Show file tree

Hide file tree

Showing 38 changed files with 1,371 additions and 371 deletions.
diff --git a/.gitignore b/.gitignore
@@ -1 +1,2 @@
 /.idea/
+/src/main/resources/application-mycloud.txt
diff --git a/README.md b/README.md
@@ -81,7 +81,7 @@ for the development.
 
 Visit [Unit Scenario](doc/unitscenario/README.md)
 
-### Load test (flowscenario)
+### Load test (flow-scenario)
 
 The flow scenario describes an environment and sends a requirement like "generate 500 PI every 40 seconds".
 The flow scenario has a duration and objective to verify.
@@ -103,11 +103,73 @@ Process-Automator does not contain any Camunda server. It connects to an existin
 communication interfaces exist, one for Camunda 7 and one for Camunda 8. A scenario can then pilot a
 Camunda 7 or a Camunda 8 server.
 
+The scenario does not contain any server information. It contains only the server.
+
+Process-automator reference a list of server in the configuration, under multiple ways:
+
+`````yaml
+automator:
+    serversConnection: Camunda7Diamond,CAMUNDA_7,http://localhost:8080/engine-rest; \
+  Camunda8Safir,CAMUNDA_8,127.0.0.1:26500,demo,demo,http://localhost:8081
+
+    serversList:
+      - type: "camunda7"
+        name: "camunda7Emeraud"
+        url: "http://localhost:8080/engine-rest"
+        workerMaxJobsActive: 20
+
+      - type: "camunda8"
+        name: "Camunda8Ruby"
+        zeebeGatewayAddress: "127.0.0.1:26500"
+        operateUserName: "demo"
+        operateUserPassword: "demo"
+        operateUrl: "http://localhost:8081"
+        taskListUrl: "http://localhost:8082"
+        workerExecutionThreads: 10
+        workerMaxJobsActive: 10
+
+      - type: "camunda8saas"
+        name: "Camunda8Grena"        
+        workerExecutionThreads: 10
+        workerMaxJobsActive: 10
+        operateUrl: "https://ont-1.operate.camunda.io/25fdd1e6-e4a1-4362-b49c-5eced08cb893"
+        taskListUrl: "https://ont-1.tasklist.camunda.io/25fdd1e6-e4a1-4362-b49c-5eced08cb893"
+
+        operateUserName: "demo"
+        operateUserPassword: "demo"
+
+        region: "ont-1"
+        clusterId: "25xxxx93"
+        clientId: "ekxxxx9L"
+        oAuthUrl: "https://login.cloud.camunda.io/oauth/token"
+        audience: ""
+        secret: "4BPxxxxxN"
+
+`````
+The `serverConnection` is ease to manipulate in a Docker or a Kubernetes environment.
+
+At the execution, two parameters are mandatory:
+* the scenario to run
+* the server to connect to run the scenario
+
+By this way, it's possible to use the same scenario on different environment.
+
+Example to run the CLI command
+
+````
+`java -jar target/process-execution-automator.jar \
+  -s Camunda8Ruby \ 
+  -v  \
+  -l MAIN \ 
+  -x run doc/unittestscenario/resources/C8LoanManagementScn.json
+````
 ## Use in Docker
 A docker image is created. The image can be used in a docker-compose.
 
 Visit [Docker documentation](doc/docker/README.md)
 
+Scenario and Server configuration can be setup at startup. 
+
 ## Use in Kubernetes
 
 The project can be used in a docker environment to create, for example,

diff --git a/doc/applicationreference/README.md b/doc/applicationreference/README.md
@@ -79,7 +79,12 @@ This section declared how to connect the server "Diamond": is that a Camunda7 se
 There ed is two ways to declare a list of server:
 a ServerString Connection.
 
-This is a list of <name>,CAMUNDA_7|CAMUNDA_8,(<property>)*
+### URL
+
+List of server. Easy to use in the CLI on the command line.
+
+
+This is a list of <name>,CAMUNDA_7|CAMUNDA_8|CAMUNDA_8_SAAS,(<property>)*
 
 For Camunda 7:
 ```
@@ -93,17 +98,46 @@ For Camunda 8 to Self-manage
 
 For Camunda 8 Saas:
 ```
-<name>,CAMUNDA_8_SAAS,zeebeCloudRegister,zeebeCloudRegion,zeebeCloudClusterId,zeebeCloudClientId,clientSecret,OperateUserName,OperateUserPassword,OperateUrl,ExecutionThreads,MaxJobActive
+<name>,CAMUNDA_8_SAAS,zeebeCloudRegister,zeebeCloudRegion,zeebeCloudClusterId,zeebeCloudClientId,zeebeCloudOAuthUrl,zeebeCloudAudience,clientSecret,OperateUserName,OperateUserPassword,OperateUrl,,ExecutionThreads,MaxJobActive
 ```
 
+**Example**
 
+````yaml
 
 
-# Another way to provide the list of server connection
-In the YAML, the list of servers can be done by giving
+automator.serversConnection: \ 
+   Camunda7Diamond,CAMUNDA_7,http://localhost:8080/engine-rest; \
+   Camunda8Safir,CAMUNDA_8,127.0.0.1:26500,demo,demo,http://localhost:8081
 
-```
-automator.servers:
+````
+
+### List of servers
+In the YAML, the list of servers can be done by giving a list:
+
+| Parameter              | Explanation                                                  | Type    | Default |
+|------------------------|--------------------------------------------------------------|---------|---------|
+| name                   | Name of server (to be use to define the server to connect)   | String  | ""      |
+| type                   | "CAMUNDA_7", "CAMUNDA_8", "CAMUNDA_8_SAAS"                   | String  | ""      |
+| workerExecutionThreads | (CAMUNDA_8, CAMUNDA_8_SAAS) Number of threads to use         | Integer | 20      |           
+| workerMaxJobsActive    | (all) Number of jobs to fetch                                | Integer | 10      |               
+| url                    | (CAMUNDA_7) Url to connect to Camunda 7                      | String  | ""      |
+| zeebeGatewayAddress    | (CAMUNDA_8) Address to connect Camunda 8                     | String  | ""      |
+| operateUrl             | (CAMUNDA_8, CAMUNDA_8_SAAS) Url to connect Operate           | String  | ""      |  
+| taskListUrl            | (CAMUNDA_8, CAMUNDA_8_SAAS) Url to connect Tasklist          | String  | ""      | 
+| operateUserName        | (CAMUNDA_8, CAMUNDA_8_SAAS) User name to connect to Operate  | String  | "demo"  |
+| operateUserPassword    | (CAMUNDA_8, CAMUNDA_8_SAAS) Password to connect to Operate   | String  | "demo"  |                    
+| region                 | (CAMUNDA_8_SAAS) Saas region                                 | String  | ""      |
+| clusterId              | (CAMUNDA_8_SAAS) Cluster ID                                  | String  | ""      |
+| clientId               | (CAMUNDA_8_SAAS) Client ID                                   | String  | ""      |
+| oAuthUrl               | (CAMUNDA_8_SAAS) Authorization URL                           | String  | ""      |
+| audience               | (CAMUNDA_8_SAAS) Audience                                    | String  | ""      |
+| secret                 | (CAMUNDA_8_SAAS) Secret                                      | String  | ""      |
+
+***Example***
+
+```yaml
+automator.serversList:
     - name: "Gold"
       type: "CAMUNDA_7"
       url: "http://localhost:8080/engine-rest"
@@ -121,10 +155,120 @@ automator.servers:
 
     - name: "Diamond"
       type: "CAMUNDA_8_SAAS"
-      zeebeCloudRegister:
-      zeebeCloudRegion:
-      clientSecret:
-      zeebeCloudClusterId:
-      zeebeCloudClientId:
+      region:
+      clusterId:
+      clientId:
+      secret:
+      oAuthUrl:
+      audience:
+      workerExecutionThreads: 10
+      workerMaxJobsActive: 10
+      
 ```
 
+### Explicit values
+
+This method is simplest to use to configure a Docker/Kubernetes connection
+
+
+This definition is very simple to use in the K8 definition, because one variable can be override
+
+For example, to set up a Camunda 8 server, use in the environment:
+````yaml
+env:
+- name: JAVA_TOOL_OPTIONS
+  value: >-
+    -Dautomator.startup.serverName=zeebeCloud
+    -Dautomator.servers.camunda8.name=zeebeCloud
+    -Dautomator.servers.camunda8.zeebeGatewayAddress=camunda-zeebe-gateway:26500
+    -Dautomator.servers.camunda8.operateUserName=demo
+    -Dautomator.servers.camunda8.operateUserPassword=demo
+    -Dautomator.servers.camunda8.operateUrl=http://camunda-operate:80
+    -Dautomator.servers.camunda8.taskListUrl=http://camunda-tasklist:80
+    -Dautomator.servers.camunda8.workerExecutionThreads=1
+````
+
+Variables are:
+
+Prefix **automator.servers.camunda7**
+
+| Parameter                  | Explanation                                                 | Type     | Default |
+|----------------------------|-------------------------------------------------------------|----------|---------|
+| name                       | Name of server (to be use to define the server to connect)  | String   | ""      |
+| url                        | Url to connect to Camunda 7                                 | String   | ""      |
+| workerMaxJobsActive        | Number of jobs to fetch                                     | Integer  | 20      |
+
+
+Prefix **automator.servers.camunda8**
+
+| Parameter              | Explanation                                                | Type    | Default |
+|------------------------|------------------------------------------------------------|---------|---------|
+| name                   | Name of server (to be use to define the server to connect) | String  | ""      |
+| zeebeGatewayAddress    | Address to connect Camunda 8                               | String  | ""      |
+| operateUrl             | Url to connect Operate                                     | String  | ""      |  
+| taskListUrl            | Url to connect Tasklist                                    | String  | ""      | 
+| operateUserName        | User name to connect to Operate                            | String  | "demo"  |
+| operateUserPassword    | Password to connect to Operate                             | String  | "demo"  |                    
+| workerExecutionThreads | Number of threads to use                                   | Integer | 20      |           
+| workerMaxJobsActive    | Number of jobs to fetch                                    | Integer | 10      |               
+
+
+Prefix **automator.servers.camunda8Saas**
+
+| Parameter              | Explanation                                                | Type    | Default |
+|------------------------|------------------------------------------------------------|---------|---------|
+| name                   | Name of server (to be use to define the server to connect) | String  | ""      |
+| operateUrl             | Url to connect Operate                                     | String  | ""      |  
+| taskListUrl            | Url to connect Tasklist                                    | String  | ""      | 
+| operateUserName        | User name to connect to Operate                            | String  | "demo"  |
+| operateUserPassword    | Password to connect to Operate                             | String  | "demo"  |                    
+| region                 | Saas regions                                               | String  | ""      |
+| clusterId              | Cluster ID                                                 | String  | ""      |
+| clientId               | Client ID                                                  | String  | ""      |
+| oAuthUrl               | Authorization URL                                          | String  | ""      |
+| audience               | Audience                                                   | String  | ""      |
+| secret                 | Secret                                                     | String  | ""      |
+| workerExecutionThreads | Number of threads to use                                   | Integer | 20      |           
+| workerMaxJobsActive    | Number of jobs to fetch                                    | Integer | 10      |
+
+
+***Example***
+
+
+````yaml
+automator.servers:
+
+    camunda7:
+      name: "Camunda7Granit"
+      url: "http://localhost:8080/engine-rest"
+      workerMaxJobsActive: 20
+
+
+    camunda8:
+      name: "Camunda8Calcair"
+      zeebeGatewayAddress: "127.0.0.1:26500"
+      operateUserName: "demo"
+      operateUserPassword: "demo"
+      operateUrl: "http://localhost:8081"
+      taskListUrl: "http://localhost:8082"
+      workerExecutionThreads: 10
+      workerMaxJobsActive: 10
+
+    camunda8Saas:
+      name: "Camunda8Marbble"
+      clusterId: "25fdd1e6-e4a1-4362-b49c-5eced08cb893"
+      clientId: "eknOoiO5GYDdFf4ZjDSh8yaLG-BVCw9L"
+      oAuthUrl: "https://login.cloud.camunda.io/oauth/token"
+      audience: ""
+      secret: "4BPUva1U4lDtoG2-torvAtx6w5RbHULUFhGZ-bBXOMWwZJG3d3VDlfPHjVO3Kz-N"
+      operateUrl: "https://ont-1.operate.camunda.io/25fdd1e6-e4a1-4362-b49c-5eced08cb893"
+      taskListUrl: "https://ont-1.tasklist.camunda.io/25fdd1e6-e4a1-4362-b49c-5eced08cb893"
+
+      operateUserName: "demo"
+      operateUserPassword: "demo"
+
+      region: "ont-1"
+      workerExecutionThreads: 10
+      workerMaxJobsActive: 10
+````
+
diff --git a/doc/loadtestscenario/Tutorial.md b/doc/loadtestscenario/Tutorial.md
@@ -24,8 +24,6 @@ zeebe:
   env:
     - name: ZEEBE_BROKER_EXECUTION_METRICS_EXPORTER_ENABLED
       value: "true"
-    - name: ZEEBE_BROKER_PROCESSING_MAXCOMMANDSINBATCH
-      value: "5000"
   resources:
     requests:
       cpu: "1"
@@ -221,13 +219,13 @@ Specify in the application parameter what you want to run.
 
 `````yaml
 Automator.startup:
-scenarioPath: ./doc/loadtestsscenario/resources
+  scenarioPath: ./doc/loadtestsscenario/resources
 # List of scenarios separated by ;
-scenarioAtStartup: C8CrawlUrlScn.json;
+  scenarioAtStartup: C8CrawlUrlScn.json;
 # DEBUG, INFO, MONITORING, MAIN, NOTHING
-logLevel: MAIN
+  logLevel: MAIN
 # string composed with DEPLOYPROCESS, WARMINGUP, CREATION, SERVICETASK (ex: "CREATION", "DEPLOYPROCESS|CREATION|SERVICETASK")
-policyExecution: DEPLOYPROCESS|WARMINGUP|CREATION|SERVICETASK|USERTASK
+  policyExecution: DEPLOYPROCESS|WARMINGUP|CREATION|SERVICETASK|USERTASK
 `````
 
 
@@ -249,22 +247,23 @@ To be close to the final platform, let's run the process-automator not locally b
 
 The main point is to provide the scenario to the pod.
 
-1. Create a config map for the scenario
+Create a config map for the scenario
 ````
 cd doc/loadtestscenario/
 kubectl create configmap crawurlscnmap --from-file=resources/C8CrawlUrlScn.json 
 ````
 
+How this scenario is accessible in the pod? Check the `ku-c8CrawUrl.yaml` file
 
-2. Create a volume and mount the configMap in that volume
+1. Create a volume and mount the configMap in that volume
 ````yaml
       volumes:
         - name: scenario
           configMap:
             name: crawurlscnmap
 ````
 
-3. Mount the volume in the container
+2. Mount the volume in the container
 
 `````yaml
 volumeMounts:
@@ -274,17 +273,30 @@ volumeMounts:
     readOnly: true
 `````
 
-4. Reference the file in parameters
+3. Reference the file in parameters
 
 `````
          -Dautomator.startup.scenarioResourceAtStartup=file:/C8CrawlUrlScn.json
 `````
 
-Then, deploy and start the docker image with
+
+Then, deploy and start the docker image.
+
+All this configuration is available in the file `ku-c8CrawlUrl.yaml`, so run it with
 ````
 kubectl create -f ku-c8CrawlUrl.yaml
 ````
 
+Follow the advance
+````
+kubectl get pods
+````
+Identify the correct pods, and access the log
+````
+kubectl logs -f ku-processautomator-xxxxxx
+````
+
+
 
 ### Generate the Docker image again
 An alternative consists of placing the scenario under `src/resources/` and building a new image.

diff --git a/doc/loadtestscenario/ku-c8CrawlUrl.yaml b/doc/loadtestscenario/ku-c8CrawlUrl.yaml
@@ -20,16 +20,18 @@ spec:
     spec:
       containers:
         - name: ku-processautomator
-          image: ghcr.io/camunda-community-hub/process-execution-automator:latest
+          image: ghcr.io/camunda-community-hub/process-execution-automator:1.3.0
           imagePullPolicy: Always
           env:
             - name: JAVA_TOOL_OPTIONS
               value: >-
+                -Dautomator.startup.serverName=zeebeCloud
+                -Dautomator.servers.camunda8.name=zeebeCloud
                 -Dautomator.servers.camunda8.zeebeGatewayAddress=camunda-zeebe-gateway:26500
                 -Dautomator.servers.camunda8.operateUserName=demo
                 -Dautomator.servers.camunda8.operateUserPassword=demo
                 -Dautomator.servers.camunda8.operateUrl=http://camunda-operate:80
-                -Dautomator.servers.camunda8.taskListUrl=
+                -Dautomator.servers.camunda8.taskListUrl=http://camunda-tasklist:80
                 -Dautomator.servers.camunda8.workerExecutionThreads=1
                 -Dautomator.startup.scenarioResourceAtStartup=file:/C8CrawlUrlScn.json
                 -Dautomator.startup.policyExecution=WARMINGUP|CREATION|SERVICETASK|USERTASK

diff --git a/doc/loadtestscenario/resources/C8CrawlUrlScn.json b/doc/loadtestscenario/resources/C8CrawlUrlScn.json
@@ -1,8 +1,8 @@
 {
   "name": "C8CrawlUrl",
   "processId": "CrawlUrl",
-  "type": "FLOW",
   "serverType": "Camunda_8",
+  "typeScenario" : "FLOW",
   "deployments": [
     {
       "serverType": "CAMUNDA_8",
Original file line number	Diff line number	Diff line change
		@@ -1 +1,2 @@
		/.idea/
		/src/main/resources/application-mycloud.txt