Skip to content

Commit

Permalink
Add ContentManager to upload test
Browse files Browse the repository at this point in the history
  • Loading branch information
Pierre-yves-monnet committed Dec 4, 2024
1 parent 581a902 commit a217c3b
Show file tree
Hide file tree
Showing 25 changed files with 571 additions and 267 deletions.
29 changes: 11 additions & 18 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -449,43 +449,36 @@ automator.servers:
Rebuilt the image via
````
mvn clean install
mvn springboot:build-image
````
# Push the docker image
The docker image is build using the Dockerfile present on the root level.
Push the image to
```
ghcr.io/camunda-community-hub/process-execution-automator:
```
## Detail
Run command
````
mvn clean install
````
Now, create a docker image
````
docker build -t pierre-yves-monnet/processautomator:1.7.1 .
docker build -t pierre-yves-monnet/process-execution-automator:1.8.0 .
docker build pycamunda/camunda-community-hub/process-execution-automator:1.8.0
````
Push the image to the Camunda hub (you must be login first to the docker registry)
````
docker tag pierre-yves-monnet/processautomator:1.7.1 ghcr.io/camunda-community-hub/process-execution-automator:1.7.1
docker push ghcr.io/camunda-community-hub/process-execution-automator:1.7.1
docker tag pierre-yves-monnet/process-execution-automator:1.8.0 ghcr.io/camunda-community-hub/process-execution-automator:1.8.0
docker push ghcr.io/camunda-community-hub/process-execution-automator:1.8.0
````
docker tag pierre-yves-monnet/process-execution-automator:1.8.0 pycamunda/camunda-hub:process-execution-automator-1.8.0
docker push pycamunda/camunda-hub:process-execution-automator-1.8.0
Tag as the latest:
````
docker tag pierre-yves-monnet/processautomator:1.7.1 ghcr.io/camunda-community-hub/process-execution-automator:latest
docker tag pierre-yves-monnet/process-execution-automator:1.8.0 ghcr.io/camunda-community-hub/process-execution-automator:latest
docker push ghcr.io/camunda-community-hub/process-execution-automator:latest
````
Check on
https://github.com/camunda-community-hub/process-execution-automator/pkgs/container/process-execution-automator
https://github.com/camunda-community-hub/zeebe-cherry-runtime/pkgs/container/process-execution-automator
74 changes: 47 additions & 27 deletions doc/unittestscenario/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,26 +16,18 @@ There is multiple use case:

### Verification (path and performance)

![Process](../explanationProcess.png)
![ScoreAcceptance.png](resources/ScoreAcceptance.png)

in a CD/CI, you want to verify that a process follows the same behavior in the same performance
time. Running every day (or hours) or asking via an API call to replay a scenario is useful to
verify there is no difference. If the customer is 4555, do we still move the process instance to
Review Level 1"? The second verification is the performance. The scenario can record an expected
duration target (for example, 4 seconds to execute the Get Context service task. Does the execution
still at this time?
verify there is no difference. If the score is 200, do we still move the process instance to
Send Acceptation?

### Coverage report

Execute multiple scenarios to be sure that all the process is covered correctly. An "Execution
round" is a set of scenarios executed at the same time. At the end of the execution, a coverage test
can be performed. A CD/CI verification may be to check the scenario execution, the target time, and
the coverage.

### Advance process instances to a step for development

During the development, you verify the task "Notify applicant". To test it in the situation, you
must have a process instance in the process and pass four user tasks. Each test takes time: when you
During the development, you debug the task "Send rejection". To test it in the situation, you
must have a process instance in the process and pass all user tasks. Each test takes time: when you
deploy a new process or want a new process instance, you need to execute again the different user
task. Using Automator with the correct scenario solves the issue. Deploy a new process, but instead
of starting from the beginning of a new process instance, start it via Automator. The scenario will
Expand All @@ -50,27 +42,55 @@ In the unit scenario, you should place some Event (for example, the end event):

This verification implies to give an Operate access.

The scenario will contains:

The name, the process ID
The scenario contain:

A list of flow to execute under the attribut `executions`
* The name, the process ID,

* A list of flow to execute under attribut `executions`
* A list of verification under attribut `verifications`

* a STARTEVENT, to start one process instance
* the list of all SERVICETASK

## Scenario definition

## Generate from a real execution
Automator can generate a scenario from a real execution. The user creates a process instance and
executes it. It executes user tasks until the end of the process instance or at a certain point. Via
the UI (or the API), the user gives the process instance. Automator queries Camunda Engine to
collect the history of the process and, for each user task, which variable was provided. A new
scenario is created from this example.
Check the scenario:

Note: this function is yet available
[ScoreAcceptanceScn.json](resources/ScoreAcceptanceScn.json)

## execute

In progress
1. First, upload the scenario file in a config map

```
kubectl create configmap scoreacceptancescn --from-file=doc/unittestscenario/resources/scoreacceptancescn.json -n camunda
```

2. Deploy the scenario on the cluster, via the Modeler

3. Create the pod process-execution-automator

```
kubectl create -f doc/unittestscenario/resources/UnittestAutomator.yaml -n camunda
```
This configuration will upload the scenario


4. Port forward

```
kubectl port-forward svc/process-execution-automator 8381:8381 -n camunda
```

6. Check the scenario is uploaded

```
curl -X GET "http://localhost:8381/api/content/list" -H "Content-Type: application/json"
```


7. upload the scenario
```
curl -X POST -F "file=@/path/to/your/file.txt" http://localhost:8080/api/files/upload
curl -X GET "http://localhost:8381/api/unittest/get?id=1732767184446" -H "Content-Type: application/json"
```

21 changes: 20 additions & 1 deletion doc/unittestscenario/SendUnitTestCommand.rest
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
### POST Request
POST http://localhost:8381/api/unittest/run?name=ScoreAcceptanceScn&server=Camunda8Ruby&wait=false
POST http://localhost:8381/api/unittest/run?name=ScoreAcceptanceScn&server=Camunda8Ruby&wait=true
Content-Type: application/json

{
Expand All @@ -19,3 +19,22 @@ Content-Type: application/json

{
}

### Content Manager
GET http://localhost:8381/api/content/list
Content-Type: application/json

{
}

### Upload file
POST http://localhost:8381/api/content/add
Content-Type: multipart/form-data

--boundary
Content-Disposition: form-data; name="File"; filename="file1.txt"
Content-Type: text/plain

< ./resources/ScoreAcceptanceScn.json

--boundary--
4 changes: 2 additions & 2 deletions src/main/java/org/camunda/automator/AutomatorAPI.java
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,8 @@
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;

import java.io.File;
import java.io.InputStream;
import java.nio.file.Path;

@Component
public class AutomatorAPI {
Expand All @@ -48,7 +48,7 @@ public Scenario createScenario() {
* @return the scenario
* @throws AutomatorException if scenario can't be read
*/
public Scenario loadFromFile(File scenarioFile) throws AutomatorException {
public Scenario loadFromFile(Path scenarioFile) throws AutomatorException {
return Scenario.createFromFile(scenarioFile);
}

Expand Down
39 changes: 25 additions & 14 deletions src/main/java/org/camunda/automator/AutomatorCLI.java
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,13 @@
import org.springframework.stereotype.Component;

import java.io.File;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.ArrayList;
import java.util.List;
import java.util.stream.Stream;

@SpringBootApplication

Expand Down Expand Up @@ -98,14 +103,20 @@ private static BpmnEngineList decodeConfiguration(String propertiesFileName) thr
throw new Exception("Not yet implemented");
}

private static List<File> detectRecursiveScenario(File folderRecursive) {
List<File> listFiles = new ArrayList<>();
for (File file : folderRecursive.listFiles()) {
if (file.isDirectory()) {
listFiles.addAll(detectRecursiveScenario(file));
} else if (file.getName().endsWith(".json")) {
listFiles.add(file);
}
private static List<Path> detectRecursiveScenario(Path folderRecursive) {
List<Path> listFiles = new ArrayList<>();
try (Stream<Path> files = Files.list(folderRecursive)) {
// Iterate over all files in the directory
files.forEach(file -> {
if (Files.isRegularFile(file)) {
listFiles.add(file);
}
if (Files.isDirectory(file)) {
listFiles.addAll(detectRecursiveScenario(file));
}
});
} catch (IOException e) {
logger.error("During detection scenario file: {}", e.getMessage());
}
return listFiles;
}
Expand All @@ -122,8 +133,8 @@ private static void logOutLn(String message) {
public void run(String[] args) {
if (!isRunningCLI)
return;
File scenarioFile = null;
File folderRecursive = null;
Path scenarioFile = null;
Path folderRecursive = null;

RunParameters runParameters = new RunParameters();
runParameters.setExecution(true)
Expand Down Expand Up @@ -184,13 +195,13 @@ public void run(String[] args) {
if (args.length < i + 1)
throw new AutomatorException("Bad usage : run <scenarioFile>");
action = ACTION.RUN;
scenarioFile = new File(args[i + 1]);
scenarioFile = Paths.get(args[i + 1]);
i++;
} else if ("recursive".equals(args[i])) {
if (args.length < i + 1)
throw new AutomatorException("Bad usage : recursive <folder>");
action = ACTION.RECURSIVE;
folderRecursive = new File(args[i + 1]);
folderRecursive = Paths.get(args[i + 1]);
i++;
} else {
printUsage();
Expand Down Expand Up @@ -230,8 +241,8 @@ public void run(String[] args) {
logger.info(scenarioExecutionResult.getSynthesis(runParameters.isFullDetailsSynthesis()));
}
case RECURSIVE -> {
List<File> listScenario = detectRecursiveScenario(folderRecursive);
for (File scenarioFileIndex : listScenario) {
List<Path> listScenario = detectRecursiveScenario(folderRecursive);
for (Path scenarioFileIndex : listScenario) {
Scenario scenario = automatorAPI.loadFromFile(scenarioFileIndex);
BpmnEngine bpmnEngineScenario = automatorAPI.getBpmnEngine(serverDefinition, true);
RunResult scenarioExecutionResult = automatorAPI.executeScenario(
Expand Down
10 changes: 5 additions & 5 deletions src/main/java/org/camunda/automator/AutomatorRest.java
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;

import java.io.File;
import java.nio.file.Path;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
Expand Down Expand Up @@ -116,11 +116,11 @@ private void startTest(String scenarioName, String serverName, String unitTestId
// now proceed the scenario
try {
Scenario scenario = null;
File scenarioFile = contentManager.getFromName(scenarioName);
Path scenarioFile = contentManager.getFromName(scenarioName);
try {
scenario = automatorAPI.loadFromFile(scenarioFile);
} catch (Exception e) {
logger.error("Error during accessing InputStream from File [{}]: {}", scenarioFile.getAbsolutePath(),
logger.error("Error during accessing InputStream from File [{}]: {}", scenarioFile.toAbsolutePath().toString(),
e.getMessage());
}
if (scenario == null) {
Expand All @@ -140,13 +140,13 @@ private void startTest(String scenarioName, String serverName, String unitTestId
return;
}

bpmnEngine.turnHighFlowMode(true);
bpmnEngine.turnHighFlowMode(false);
logger.info("Scenario [{}] file[{}] use BpmnEngine {}", scenario.getName(), scenario.getName(),
bpmnEngine.getSignature());
RunResult scenarioExecutionResult = automatorAPI.executeScenario(bpmnEngine, runParameters, scenario);
logger.info("AutomatorRest: end scenario [{}] in {} ms", scenario.getName(),
scenarioExecutionResult.getTimeExecution());
bpmnEngine.turnHighFlowMode(false);

resultMap.put(JSON_STATUS, "EXECUTED");
resultMap.putAll(resultToJson(scenarioExecutionResult));

Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
package org.camunda.automator.bpmnengine;

import io.camunda.operate.search.DateFilter;
// import io.camunda.operate.search.DateFilter;

import io.camunda.zeebe.client.api.worker.JobWorker;
import org.camunda.automator.configuration.BpmnEngineList;
import org.camunda.automator.definition.ScenarioDeployment;
Expand Down Expand Up @@ -186,10 +187,10 @@ List<ProcessDescription> searchProcessInstanceByVariable(String processId,
/* CountInformation */
/* */
/* ******************************************************************** */
long countNumberOfProcessInstancesCreated(String processId, DateFilter startDate, DateFilter endDate)
long countNumberOfProcessInstancesCreated(String processId, Date startDate, Date endDate)
throws AutomatorException;

long countNumberOfProcessInstancesEnded(String processId, DateFilter startDate, DateFilter endDate)
long countNumberOfProcessInstancesEnded(String processId, Date startDate, Date endDate)
throws AutomatorException;

long countNumberOfTasks(String processId, String taskId) throws AutomatorException;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -31,12 +31,14 @@ public static BpmnEngineList getCamunda7(String serverUrl) {
return bpmEngineConfiguration;
}

public static BpmnEngineList getCamunda8(String zeebeGatewayAddress) {
public static BpmnEngineList getCamunda8(String zeebeGatewayAddress, String zeebeGrpcAddress, String zeebeRestAddress) {
BpmnEngineList bpmEngineConfiguration = new BpmnEngineList();

BpmnEngineList.BpmnServerDefinition serverDefinition = new BpmnEngineList.BpmnServerDefinition();
serverDefinition.serverType = BpmnEngineList.CamundaEngine.CAMUNDA_8;
serverDefinition.zeebeGatewayAddress = zeebeGatewayAddress;
serverDefinition.zeebeGrpcAddress = zeebeGrpcAddress;
serverDefinition.zeebeRestAddress = zeebeRestAddress;

bpmEngineConfiguration.addExplicitServer(serverDefinition);

Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
package org.camunda.automator.bpmnengine.camunda7;

import io.camunda.operate.search.DateFilter;

import org.camunda.automator.bpmnengine.BpmnEngine;
import org.camunda.automator.configuration.BpmnEngineList;
import org.camunda.automator.definition.ScenarioDeployment;
Expand Down Expand Up @@ -422,7 +422,7 @@ public Map<String, Object> getVariables(String processInstanceId) throws Automat
/* ******************************************************************** */

@Override
public long countNumberOfProcessInstancesCreated(String processName, DateFilter startDate, DateFilter endDate)
public long countNumberOfProcessInstancesCreated(String processName, Date startDate, Date endDate)
throws AutomatorException {

try {
Expand All @@ -445,7 +445,7 @@ public long countNumberOfProcessInstancesCreated(String processName, DateFilter
Date datePI = stringToDate(t.getBusinessKey());
if (datePI == null)
return false;
return datePI.after(startDate.getDate());
return datePI.after(startDate);
}).count();

} while (processInstanceDtos.size() >= SEARCH_MAX_SIZE && maxLoop < 1000);
Expand All @@ -458,7 +458,7 @@ public long countNumberOfProcessInstancesCreated(String processName, DateFilter
}

@Override
public long countNumberOfProcessInstancesEnded(String processName, DateFilter startDate, DateFilter endDate)
public long countNumberOfProcessInstancesEnded(String processName, Date startDate, Date endDate)
throws AutomatorException {
throw new AutomatorException("Not yet implemented");
}
Expand Down
Loading

0 comments on commit a217c3b

Please sign in to comment.