[incubator-kie-issues-576] Data Audit apps for kogito (#1905)

* [incubator-kie-issues-576] Data Audit app for kogito * [incubator-kie-issues-576] add tests for data audit * added data audit context to local * make proxy independent of runtime * tweak events * fix deletion of spring boot subsystem * fix logic in data audit * fix tests * fix json objects in variable values * added pagination * fix roles in process state query * add mechanism extension * fix for assignments * fix deadlines notifications * fix spring boot test * add queries for processes * repacle anonymous classe by reusable classes * job log changed to match data index * add microprofile to subsystem * make context factory independent * fix names * readme.md * fix compilation problems * fix artifact id and added new query * fix master pom adding data audit module * fix names in the projects to fit naming conventions * formatting pom.xml * setting configuration * fix service * fix jdbc driver dependencies * fix jdbc driver dependencies * added jpa container service * Fix naming conventions for tables * fix sla * fix configuration. simplification * fix service * fix for orm * fix classloader * clean unnecessary fields * fix readme.md * fix transaction error in springboot * fix try for transactions wthout transaction manager provider * fix * fix query * fix plural queries * Quarkus 3 upgrade fixes: - `999-SNAPSHOT` version bump - javax to jakarta migration - test fixes * - fix log --------- Co-authored-by: Pere Fernández <[email protected]>
apache · Dec 27, 2023 · 44ca768 · 44ca768
1 parent 3d94e16
commit 44ca768
Show file tree

Hide file tree

Showing 124 changed files with 9,839 additions and 105 deletions.
diff --git a/data-audit/.gitignore b/data-audit/.gitignore
@@ -0,0 +1,167 @@
+/target/
+!.mvn/wrapper/maven-wrapper.jar
+
+### STS ###
+.apt_generated
+.classpath
+.factorypath
+.project
+.settings
+.springBeans
+.sts4-cache
+
+
+ObjectStore/
+PutObjectStoreDirHere/
+
+# Created by https://www.gitignore.io/api/git,java,maven,eclipse,windows
+
+### Eclipse ###
+
+.metadata
+bin/
+tmp/
+*.tmp
+*.bak
+*.swp
+*~.nib
+local.properties
+.settings/
+.loadpath
+.recommenders
+
+# External tool builders
+.externalToolBuilders/
+
+# Locally stored "Eclipse launch configurations"
+*.launch
+
+# PyDev specific (Python IDE for Eclipse)
+*.pydevproject
+
+# CDT-specific (C/C++ Development Tooling)
+.cproject
+
+# CDT- autotools
+.autotools
+
+# Java annotation processor (APT)
+.factorypath
+
+# PDT-specific (PHP Development Tools)
+.buildpath
+
+# sbteclipse plugin
+.target
+
+# Tern plugin
+.tern-project
+
+# TeXlipse plugin
+.texlipse
+
+# STS (Spring Tool Suite)
+.springBeans
+
+# Code Recommenders
+.recommenders/
+
+# Annotation Processing
+.apt_generated/
+
+# Scala IDE specific (Scala & Java development for Eclipse)
+.cache-main
+.scala_dependencies
+.worksheet
+
+### Eclipse Patch ###
+# Eclipse Core
+.project
+
+# JDT-specific (Eclipse Java Development Tools)
+.classpath
+
+# Annotation Processing
+.apt_generated
+
+.sts4-cache/
+
+### Git ###
+# Created by git for backups. To disable backups in Git:
+# $ git config --global mergetool.keepBackup false
+*.orig
+
+# Created by git when using merge tools for conflicts
+*.BACKUP.*
+*.BASE.*
+*.LOCAL.*
+*.REMOTE.*
+*_BACKUP_*.txt
+*_BASE_*.txt
+*_LOCAL_*.txt
+*_REMOTE_*.txt
+
+### Java ###
+# Compiled class file
+*.class
+
+# Log file
+*.log
+
+# BlueJ files
+*.ctxt
+
+# Mobile Tools for Java (J2ME)
+.mtj.tmp/
+
+# Package Files #
+*.jar
+*.war
+*.nar
+*.ear
+*.zip
+*.tar.gz
+*.rar
+
+# virtual machine crash logs, see http://www.java.com/en/download/help/error_hotspot.xml
+hs_err_pid*
+
+### Maven ###
+target/
+pom.xml.tag
+pom.xml.releaseBackup
+pom.xml.versionsBackup
+pom.xml.next
+release.properties
+dependency-reduced-pom.xml
+buildNumber.properties
+.mvn/timing.properties
+.mvn/wrapper/maven-wrapper.jar
+
+### Windows ###
+# Windows thumbnail cache files
+Thumbs.db
+ehthumbs.db
+ehthumbs_vista.db
+
+# Dump file
+*.stackdump
+
+# Folder config file
+[Dd]esktop.ini
+
+# Recycle Bin used on file shares
+$RECYCLE.BIN/
+
+# Windows Installer files
+*.cab
+*.msi
+*.msix
+*.msm
+*.msp
+
+# Windows shortcuts
+*.lnk
+
+
+# End of https://www.gitignore.io/api/git,java,maven,eclipse,windows
diff --git a/data-audit/README.md b/data-audit/README.md
@@ -0,0 +1,84 @@
+### Data Audit subsystem
+
+This is the implementation of the Kogito Audit subystem. The subsystem allows to store events from issued by
+- process engine
+- user task
+- job service
+
+Main features are:
+
+- runs as colocated services with quarkus and springboot
+- module to write your own subsystem
+- graphql modules to query data
+- extension points to develop new storage easily
+- extension points to develop new queries extending graphql definitions
+
+## Design of the Data Audit
+
+The system contains several modules and common modules to redistribute responsabilities.
+
+![Data Audit Architecture](img/design.png "Data Audit Architecture")
+
+Data Audit Common: Provides the common framework to create implementations.
+Data Audit «Quarkus»: Provides the wiring to use Data Audit with Quarkus as colocated service (deployment)
+Data Audit «SpringBoot»: Provides the wiring to use Data Audit with SpringBoot as colocated service (deployment)
+
+Now we have and implementation examples
+
+
+Data Audit JPA Common: Provides the common exension not depending on the runtime
+Data Audit JPA «Quarkus»: Provides the wiring between the specific implementation and Quarkus System
+Data Audit JPA «SpringBoot»: Provides the wiring between the specific implementation and Springboot colocated system
+
+
+## Queries
+
+The way to retrieve information from the data audit is using GraphQL. This way we can abstract how the information is retrieved and allow different needs depending on the user.
+
+## JPA implementation
+
+The jpa implementation allows you to store those events to be stored in a database.
+
+## Extension Points
+
+There are two different extensions. Those who belong to the runtime to wire things and those which does not depends on the runtime
+
+Extension points depending on the runtime is:
+org.kie.kogito.app.audit.spi.DataAuditContextFactory: this allows creating the context needed by a particular implementation.
+
+Extension points not depending on the runtime:
+org.kie.kogito.app.audit.spi.DataAuditStore: This is responsible to store the data in certain way
+org.kie.kogito.app.audit.spi.GraphQLSchemaQuery<T>: This is responsible to execute a GraphQL query
+org.kie.kogito.app.audit.spi.GraphQLSchemaQueryProvider: this allow the subsystem to identify additional queries provided by the end user
+
+
+## How to use in with Quarkus/Springboot
+
+You need to add two different dependencies to your project.
+
+	<dependency>
+	    <groupId>org.kie.kogito</groupId>
+	    <artifactId>kogito-addons-data-audit-<runtime></artifactId>
+	    <version>${version}</version>
+	</dependency>
+	<dependency>
+	    <groupId>org.kie.kogito</groupId>
+	    <artifactId>kogito-addons-data-audit-jpa-<runtime></artifactId>
+	    <version>${version}</version>
+	</dependency>
+
+
+
+
+The first dependency is related how to you want to deploy it. In this case as collocated/embedded service
+The second dependency is which implementation you want to use.
+
+Each implementation might require additional dependencies. In our case for using JPA implementation we might require driver
+
+	<dependency>
+	    <groupId>io.quarkus</groupId>
+	    <artifactId>quarkus-jdbc-h2</artifactId>
+	</dependency>
+
+No additional configuration is requires besides the default datasource use by the application already.
+
diff --git a/data-audit/data-audit-common-service/pom.xml b/data-audit/data-audit-common-service/pom.xml
@@ -0,0 +1,40 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+	<modelVersion>4.0.0</modelVersion>
+	<parent>
+		<artifactId>data-audit</artifactId>
+		<groupId>org.kie.kogito</groupId>
+		<version>999-SNAPSHOT</version>
+	</parent>
+
+	<artifactId>data-audit-common-service</artifactId>
+
+	<name>Kogito Apps :: Data Audit :: Service Common</name>
+
+	<properties>
+		<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
+	</properties>
+
+	<dependencies>
+		<dependency>
+			<groupId>org.kie.kogito</groupId>
+			<artifactId>kogito-events-core</artifactId>
+		</dependency>
+		<dependency>
+			<groupId>org.kie.kogito</groupId>
+			<artifactId>jobs-service-api</artifactId>
+		</dependency>
+		<dependency>
+			<groupId>org.slf4j</groupId>
+			<artifactId>slf4j-api</artifactId>
+		</dependency>
+		<dependency>
+			<groupId>org.apache.kafka</groupId>
+			<artifactId>kafka-clients</artifactId>
+		</dependency>
+	</dependencies>
+
+
+</project>
diff --git a/...mon-service/src/main/java/org/kie/kogito/app/audit/json/JsonJobDataEventDeserializer.java b/...mon-service/src/main/java/org/kie/kogito/app/audit/json/JsonJobDataEventDeserializer.java
@@ -0,0 +1,70 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.kie.kogito.app.audit.json;
+
+import java.io.IOException;
+import java.time.OffsetDateTime;
+
+import org.kie.kogito.event.job.JobInstanceDataEvent;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.fasterxml.jackson.core.JsonParser;
+import com.fasterxml.jackson.core.JsonProcessingException;
+import com.fasterxml.jackson.databind.DeserializationContext;
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.deser.std.StdDeserializer;
+
+public class JsonJobDataEventDeserializer extends StdDeserializer<JobInstanceDataEvent> {
+
+    private static final Logger LOGGER = LoggerFactory.getLogger(JsonJobDataEventDeserializer.class);
+
+    private static final long serialVersionUID = 6152014726577574241L;
+
+    public JsonJobDataEventDeserializer() {
+        this(null);
+    }
+
+    public JsonJobDataEventDeserializer(Class<?> vc) {
+        super(vc);
+    }
+
+    @Override
+    public JobInstanceDataEvent deserialize(JsonParser jp, DeserializationContext ctxt)
+            throws IOException, JsonProcessingException {
+        JsonNode node = jp.getCodec().readTree(jp);
+        LOGGER.debug("Deserialize process instance data event: {}", node);
+
+        JobInstanceDataEvent event = new JobInstanceDataEvent(
+                node.has("type") ? node.get("type").asText() : null,
+                node.has("source") ? node.get("source").asText() : null,
+                node.has("data") ? node.get("data").binaryValue() : null,
+                node.has("kogitoprocinstanceid") ? node.get("kogitoprocinstanceid").asText() : null,
+                node.has("kogitorootprociid") ? node.get("kogitorootprociid").asText() : null,
+                node.has("kogitoprocid") ? node.get("kogitoprocid").asText() : null,
+                node.has("kogitorootprocid") ? node.get("kogitorootprocid").asText() : null,
+                node.has("kogitoidentity") ? node.get("kogitoidentity").asText() : null);
+
+        event.setId(node.has("id") ? node.get("id").asText() : null);
+        event.setKogitoIdentity(node.has("kogitoidentity") ? node.get("kogitoidentity").asText() : null);
+        event.setTime(node.has("time") ? jp.getCodec().treeToValue(node.get("time"), OffsetDateTime.class) : null);
+
+        return event;
+    }
+}