[incubator-kie-issues#1395] Write developer documentation for the DMN…

… implementation (#6029) * [DBACLD-143918] Write developer documentation for the DMN implementation - WIP * [DBACLD-143918] Write developer documentation for the DMN implementation - WIP * [DBACLD-143918] Write developer documentation for the DMN implementation - WIP * [DBACLD-143918] Write developer documentation for the DMN implementation - WIP * [DBACLD-143918] Write developer documentation for the DMN implementation * [incubator-kie-issues#1395] Add puml explanation to main README.md * [incubator-kie-issues#1395] Expand guide as per PR suggestion --------- Co-authored-by: Gabriele-Cardosi <[email protected]>
apache · Jul 23, 2024 · b00ef1d · b00ef1d
1 parent 6ed1281
commit b00ef1d
Show file tree

Hide file tree

Showing 6 changed files with 298 additions and 0 deletions.
diff --git a/README.md b/README.md
@@ -83,5 +83,15 @@ The following two commands will execute tests on machine with locale different t
 1. `make test -Ptest-en`
 2. `make test -DTestEn`
 
+Documenting tips
+================
+
+UML diagrams have been used for architectural and design documentation. Those diagrams are in ".puml" format and have been created using the [PlantUML](https://plantuml.com/https://plantuml.com/) tool.
+Plugins exists to use it in different IDE:
+* [IDEA](https://plugins.jetbrains.com/plugin/7017-plantuml-integration)
+* [Eclipse](https://marketplace.eclipse.org/content/plantuml-plugin)
+* [VisualStudio](https://marketplace.visualstudio.com/items?itemName=jebbs.plantuml)
+
+
 
 
diff --git a/kie-dmn/Developer_Guide.md b/kie-dmn/Developer_Guide.md
@@ -0,0 +1,188 @@
+# kie-dmn - Developer Guide
+
+This module (and all its submodules) contains the code that make up the DMN engine.
+
+The engine is responsible, in very general terms, of
+1. compile the dmn model
+2. validate it
+3. evaluate it, against a given input; please note that execution could be statically-interpreted or dynamically-code-generated (see later on [Interpreted vs Codegen](#interpreted-vs-codegen))
+
+## Structure
+### DTOs
+#### * [Definitions](kie-dmn-model%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fdmn%2Fmodel%2Fapi%2FDefinitions.java): the direct representation of the original XML
+#### * [DMNResource](kie-dmn-core%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fdmn%2Fcore%2Fassembler%2FDMNResource.java): a wrapper used to forward `Definitions` with additional informations (e.g. [ResourceWithConfiguration](..%2Fkie-api%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fapi%2Fio%2FResourceWithConfiguration.java))
+#### * [DMNNode](kie-dmn-api%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fdmn%2Fapi%2Fcore%2Fast%2FDMNNode.java): the _ast_ representation of high-level nodes inside the model; it map to **_BusinessKnowledge_**, **_Decision_**, **_DecisionService_**, **_ItemDef_**, or **_InputData_**
+#### * [BaseNode](kie-dmn-feel%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fdmn%2Ffeel%2Flang%2Fast%2FBaseNode.java): the _ast_ representation of low-level nodes inside the model (e.g. **_Boolean_**, **_AtLiteral_**, **_UnaryTest_**, **_String_**, ecc.)
+#### * [CompiledFEELExpression](kie-dmn-feel%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fdmn%2Ffeel%2Fcodegen%2Ffeel11%2FCompiledFEELExpression.java): the object that represents and actually executes a specific _FEEL-expression_; it is also a `FunctionalInterface`, since it extends `Function<EvaluationContext, Object>`
+#### * [InterpretedExecutableExpression](kie-dmn-feel%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fdmn%2Ffeel%2Flang%2Fimpl%2FInterpretedExecutableExpression.java): the `CompiledFEELExpression` instantiated for statically-interpreted execution of the specific _FEEL-expression_
+#### * [CompiledExecutableExpression](kie-dmn-feel%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fdmn%2Ffeel%2Flang%2Fimpl%2FCompiledExecutableExpression.java): the `CompiledFEELExpression` instantiated for code-generated execution of the specific _FEEL-expression_
+#### * [ProcessedExpression](kie-dmn-feel%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fdmn%2Ffeel%2Fcodegen%2Ffeel11%2FProcessedExpression.java): the result of the compilation of a specific _FEEL-expression_; in turns, it wraps the actual `CompiledFEELExpression` to be executed; please note that a _FEEL-expression_ could represent an aggregation of other sub-expressions; so, in turns, a `ProcessedExpression` may contain nested ones
+#### * [DMNExpressionEvaluator](kie-dmn-core%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fdmn%2Fcore%2Fapi%2FDMNExpressionEvaluator.java): a `FunctionalInterface` that it is invoked by evaluation objects (e.g. `Decision`) to actually invoke the generated `ProcessedExpression`;
+#### * [DMNModel](kie-dmn-api%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fdmn%2Fapi%2Fcore%2FDMNModel.java): the high-level result of the compilation, to be used to invoke the overall evaluation; it contains `Definitions` and all the `DMNNode`s obtained by compilation
+#### * [DMNResult](kie-dmn-api%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fdmn%2Fapi%2Fcore%2FDMNResult.java): an instance that encapsulates all the information resulting from a DMN service invocation;
+#### * [DMNDecisionResult](kie-dmn-api%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fdmn%2Fapi%2Fcore%2FDMNDecisionResult.java): an instance that stores the result of the evaluation of a decision
+
+### Code usage
+
+#### Compilation
+The simplest way to compile a model is to instantiate a [DMNRuntime](kie-dmn-api%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fdmn%2Fapi%2Fcore%2FDMNRuntime.java) from its file, e.g.
+ ```java
+      DMNRuntime runtime = DMNRuntimeUtil.createRuntime("simple-item-def.dmn", this.getClass() );
+ ``` 
+[DMNRuntimeUtil](kie-dmn-core%2Fsrc%2Ftest%2Fjava%2Forg%2Fkie%2Fdmn%2Fcore%2Futil%2FDMNRuntimeUtil.java) has different overrides of the `createRuntime` method
+
+#### Validation
+To validate a dmn model, it is necessary to:
+1. get an instance of [DMNValidator](kie-dmn-validation%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fdmn%2Fvalidation%2FDMNValidator.java), built with some [DMNProfile](kie-dmn-core%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fdmn%2Fcore%2Fcompiler%2FDMNProfile.java); e.g.
+    ```java
+      List<DMNProfile> defaultDMNProfiles = DMNAssemblerService.getDefaultDMNProfiles(ChainedProperties.getChainedProperties(ClassLoaderUtil.findDefaultClassLoader()));
+      DMNValidator validator = DMNValidatorFactory.newValidator(defaultDMNProfiles);
+   ``` 
+2. invoke one of the `DMNValidator` methods; e.g.
+   ```java
+      List<DMNMessage> validate = validator.validate(
+                getFile("dmn14simple.dmn"),
+                VALIDATE_SCHEMA, VALIDATE_MODEL, VALIDATE_COMPILATION);
+   ```
+
+### Execution
+To execute a dmn model, it is necessary to:
+1. get an instance of `DMNRuntime`, built around the given model; e.g.
+    ```java
+      DMNRuntime runtime = DMNRuntimeUtil.createRuntime("simple-item-def.dmn", this.getClass() );
+   ``` 
+2. get the instance of `DMNModel` out of the former; e.g.
+    ```java
+      DMNModel dmnModel = runtime.getModel("https://github.com/kiegroup/kie-dmn/itemdef", "simple-item-def" );
+   ```
+3. instantiate a [DMNContext](kie-dmn-api%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fdmn%2Fapi%2Fcore%2FDMNContext.java) with the required input; e.g.
+    ```java
+      DMNContext context = DMNFactory.newContext();
+      context.set( "Monthly Salary", 1000 );
+   ```
+4. get the `DMNResult`
+   ```java
+      DMNResult dmnResult = runtime.evaluateAll(dmnModel, context );
+   ```
+5. depending on the model, the actual result could be stored in the given context, or inside a `DMNDecisionResult`
+
+Please note that `DMNRuntime` has different `evaluate*` methods
+
+### Execution flows
+
+#### Compilation
+
+The original xml is first parsed to `Definitions` by the [DMNMarshaller](kie-dmn-api%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fdmn%2Fapi%2Fmarshalling%2FDMNMarshaller.java), that is a simple xml-marshaller([DMNAssemblerService#addResourcesAfterRules](kie-dmn-core%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fdmn%2Fcore%2Fassembler%2FDMNAssemblerService.java)).
+
+Then, each `Definitions` is wrapped inside a `DMNResource` and _compiled_ by [DMNCompilerImpl](kie-dmn-core%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fdmn%2Fcore%2Fcompiler%2FDMNCompilerImpl.java).
+
+Each element of the `Definitions` is further translated to `DMNNode`.
+
+Then, the text representing each `DMNNode` is compiled by the [DecisionCompiler](kie-dmn-core%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fdmn%2Fcore%2Fcompiler%2FDecisionCompiler.java) and, in turn, by [FEELImpl](kie-dmn-feel%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fdmn%2Ffeel%2Flang%2Fimpl%2FFEELImpl.java), that would return a `ProcessedExpression`. 
+
+Inside `ProcessedExpression`, the expression is parsed to `org.antlr.v4.runtime.tree.ParseTree` by `org.kie.dmn.feel.parser.feel11.FEEL_1_1Parser` (auto-generated), and that `ParsedTree` is visited by [ASTBuilderVisitor](kie-dmn-feel%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fdmn%2Ffeel%2Fparser%2Ffeel11%2FASTBuilderVisitor.java) to get the `BaseNode` **_ast_**, that recursively contains all the nested `BaseNode`s.
+
+Finally, the `CompiledFEELExpression` to be executed is returned (`ProcessedExpression#asCompiledFEELExpression`).
+
+![Sequence Diagram](uml/Compilation.png)
+
+##### Interpreted vs Codegen
+The retrieved `CompiledFEELExpression` could be a _statically-interpreted_ `InterpretedExecutableExpression` (that wraps the original `BaseNode` **_ast_**) or could be a _dynamically-code-generated_ `CompiledExecutableExpression`.
+In the first case, evaluation is executed by the DMN code as it is statically defined.
+In the latter case, code is generated out of the given model. In that code, some variable will be directly written in the generated, speeding up its execution.
+Beside that, generated code will invoke the same functions as the interpreted one.
+Codegen execution is enabled in two ways:
+1. adding the [DoCompileFEELProfile](kie-dmn-feel%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fdmn%2Ffeel%2Fparser%2Ffeel11%2Fprofiles%2FDoCompileFEELProfile.java) to the FEEL instantiation
+2. setting the `doCompile` boolean in the `CompilerContext` (`CompilerContext.setDoCompile(true)`)
+
+When codegen is enabled, first the model is read and parsed as in the interpreted way; then:
+1. source code is generated out of the given  `BaseNode` **_ast_** (by `ASTCompilerVisitor`)
+2. code is compiled in-memory to a `CompiledExecutableExpression` (by [CompilerBytecodeLoader](kie-dmn-feel%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fdmn%2Ffeel%2Fcodegen%2Ffeel11%2FCompilerBytecodeLoader.java))
+3. the above `CompiledFEELExpression` is wrapped and returned inside a `CompiledExecutableExpression`
+
+#### Validation
+Depending on a series of flags ( `VALIDATE_SCHEMA`, `VALIDATE_MODEL`, `VALIDATE_COMPILATION`, `ANALYZE_DECISION_TABLE`), [DMNValidatorImpl](kie-dmn-validation%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fdmn%2Fvalidation%2FDMNValidatorImpl.java) executes the validation of the given model. 
+
+Behind the scenes, the validation also uses the _rule engine_ and the _rules_ defined in the different `kie-dmn/kie-dmn-validation/src/main/resources/org/kie/dmn/validation/DMNv1()/*.drl` files.
+This validation is fired inside `private List<DMNMessage> validateModel(DMNResource mainModel, List<DMNResource> otherModels)`
+
+#### Execution
+
+When `DMNRuntime#evaluate*` is invoked, a `DMNResult` is instantiated, containing a map with `DMNDecisionResult`s, initially marked as `NOT_EVALUATED`.
+
+Then, for each [DecisionNode](kie-dmn-api%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fdmn%2Fapi%2Fcore%2Fast%2FDecisionNode.java): 
+1. the mapped `DMNDecisionResult` is marked as `EVALUATING`
+2. the associated [DMNExpressionEvaluator](kie-dmn-core%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fdmn%2Fcore%2Fapi%2FDMNExpressionEvaluator.java) is retrieved (`DecisionNodeImpl#getEvaluator()`)
+3. the `DMNExpressionEvaluator#evaluate(DMNRuntimeEventManager, DMNResult)` method is invoked (see [DMNExpressionEvaluator](#--dmnexpressionevaluatorjava-a-functionalinterface-that-it-is-invoked-by-evaluation-objects-eg-decision-to-actually-invoke-the-generated-processedexpression))
+4. inside the concrete implementation, a new `EvaluationContextImpl` is created from the `FEELImpl` instance
+5. the `FEELimpl#evaluate(CompiledExpression expr, EvaluationContext ctx)` method is invoked, passing the `CompiledExpression` _**expression**_ wrapped by the current `DMNExpressionEvaluator`
+6. in turns, the `ProcessedExpression#apply(EvaluationContext)` method is invoked (`ProcessedExpression` is the actual type of `CompiledExpression`)
+7. this result in invocation of the wrapped `CompiledFEELExpression` **_executableFEELExpression_** (that could be an `InterpretedExecutableExpression` or a `CompiledExecutableExpression`)
+8. this, recursively, iterate over all the nested expressions, invoking the [FEELFunction](kie-dmn-feel%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fdmn%2Ffeel%2Fruntime%2FFEELFunction.java)s mapped to any expression, until a given final result is provided, as `Object`
+9. then, the returned object is coerced to the type requested by the decision, the result is stored in the mapped `DMNDecisionResult`, and that latter is marked as `SUCCEEDED`
+
+![Sequence Diagram](uml/Execution.png)
+
+### [BaseFEELFunction](kie-dmn-feel%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fdmn%2Ffeel%2Fruntime%2Ffunctions%2FBaseFEELFunction.java)
+
+Every _FEEL function_ is mapped to a concrete class extending `BaseFEELFunction`.
+
+During execution, this `FEELFunction` is looked-up by name, then the actual method to be invoked is looked-for, **_reflectively_**, based on the given input parameters.
+
+The critical point where this happen is the `BaseFEELFunction#getCandidateMethod(EvaluationContext ctx, Object[] originalInput, boolean isNamedParams)` method.
+
+Based on a algorithm defined in the [ScoreHelper](kie-dmn-feel%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fdmn%2Ffeel%2Fruntime%2Ffunctions%2FScoreHelper.java), each `invoke` method is tested and provided a score. The one with the highest score will be used for actual function evaluation.
+
+### Coercion
+`Coercion` is the feature for which a given object is transformed to an _equivalent_ object of a different type.
+One example of that coercion is applied whenever a number, or a string representing a number, is received, in which case it is translated to `BigDecimal`. Another example is when a method expects a list, and a single object is provided: in that case, the object is _coerced_ to a singleton list. The rules for coercion are the ones provided by the DMN specification.
+
+During `invoke` method discovery, inside `BaseFEELFunction`, the given input parameters are _coerced_ to potentially match the ones expected by the current `invoke` method, and the _coerced_ values are stored to be used for the execution of that specific method.
+
+
+## Development guidelines
+
+The main goal for the DMN engine are performance and maintainability. 
+
+About the former, whenever important refactoring are done, it would be important to also execute [DMN Benchmarks](https://github.com/apache/incubator-kie-benchmarks/) to verify for modification of them, fixing eventual regressions. 
+
+About the latter, it would be important to strive for simplicity of reading, instead of too-hard-to-read conciseness.
+It would also be important to follow goo'ol' rules, like:
+* self-explanatory method names
+* self-resilient methods, i.e. a method should work by itself, and eventually manage failures, without depending on some status/assumption provided by invoking code
+* meaningfully property/variable names
+* focused classes, dedicated to a clear and specific task
+* unit-test methods as much as possible, especially newly created ones
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/kie-dmn/uml/Compilation.png b/kie-dmn/uml/Compilation.png
diff --git a/kie-dmn/uml/Compilation.puml b/kie-dmn/uml/Compilation.puml
@@ -0,0 +1,45 @@
+/'
+  Licensed to the Apache Software Foundation (ASF) under one
+  or more contributor license agreements.  See the NOTICE file
+  distributed with this work for additional information
+  regarding copyright ownership.  The ASF licenses this file
+  to you under the Apache License, Version 2.0 (the
+  "License"); you may not use this file except in compliance
+  with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+  Unless required by applicable law or agreed to in writing,
+  software distributed under the License is distributed on an
+  "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+  KIND, either express or implied.  See the License for the
+  specific language governing permissions and limitations
+  under the License.
+'/
+@startuml
+participant DMNAssemblerService
+DMNAssemblerService -> DMNMarshaller : unmarshal(Reader)
+DMNMarshaller -> DMNAssemblerService: Definitions
+DMNAssemblerService -> DMNCompilerImpl: compile(Definitions, Resource, Collection<DMNModel> || Resource, Collection<DMNModel>)
+DMNCompilerImpl -> DMNMarshaller : unmarshal(Resource.getReader())
+DMNMarshaller -> DMNCompilerImpl: Definitions
+DMNCompilerImpl -> DMNCompilerImpl: new DMNModelImpl
+loop every ItemDefinition in Definitions.getItemDefinition
+DMNCompilerImpl -> DMNCompilerImpl: new ItemDefNodeImpl
+DMNCompilerImpl -> DMNModelImpl: addItemDefinition(ItemDefNodeImpl)
+end
+loop every DRGElement in Definitions.getDrgElement
+DMNCompilerImpl -> DRGElementCompiler: compileNode(DRGElement, DMNCompilerImpl, DMNModel)
+DRGElementCompiler -> FEELImpl: compile(String, CompilerContext)
+FEELImpl -> ProcessedExpression: new
+ProcessedExpression -> FEEL_1_1Parser: compilation_unit
+FEEL_1_1Parser -> ProcessedExpression: ParseTree
+ProcessedExpression -> ParseTree: accept(ASTBuilderVisitor)
+ParseTree -> ProcessedExpression: BaseNode
+FEELImpl -> ProcessedExpression: asCompiledFEELExpression
+ProcessedExpression -> FEELImpl: InterpretedExecutableExpression || CompiledExecutableExpression
+FEELImpl -> DRGElementCompiler: DMNExpressionEvaluator
+DRGElementCompiler -> DRGElement: setEvaluator(DMNExpressionEvaluator)
+end
+DMNCompilerImpl -> DMNAssemblerService: DMNModel
+@enduml
diff --git a/kie-dmn/uml/Execution.png b/kie-dmn/uml/Execution.png
diff --git a/kie-dmn/uml/Execution.puml b/kie-dmn/uml/Execution.puml
@@ -0,0 +1,55 @@
+/'
+  Licensed to the Apache Software Foundation (ASF) under one
+  or more contributor license agreements.  See the NOTICE file
+  distributed with this work for additional information
+  regarding copyright ownership.  The ASF licenses this file
+  to you under the Apache License, Version 2.0 (the
+  "License"); you may not use this file except in compliance
+  with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+  Unless required by applicable law or agreed to in writing,
+  software distributed under the License is distributed on an
+  "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+  KIND, either express or implied.  See the License for the
+  specific language governing permissions and limitations
+  under the License.
+'/
+@startuml
+participant Actor
+Actor -> DMNRuntimeUtil : createRuntime
+DMNRuntimeUtil -> Actor: DMNRuntime
+Actor -> DMNRuntime: getModel
+DMNRuntime -> Actor : DMNModel
+Actor -> DMNFactory: newContext
+DMNFactory -> Actor: DMNContext
+Actor -> Actor: (populate dmn context)
+Actor -> DMNRuntime: evaluate*(DMNModel, DMNContext)
+DMNRuntime -> DMNResultImplFactory: createResult
+DMNResultImplFactory -> DMNRuntime: DMNResult
+loop every DecisionNode in DMNModel.decisions
+DMNRuntime -> DMNResult: getDecisionResultById
+DMNResult -> DMNRuntime: DMNDecisionResultImpl
+DMNRuntime -> DMNRuntime: DMNResult.setEvaluationStatus(EVALUATING)
+loop every DMNNode in DecisionNode.dependencies
+DMNRuntime -> DMNRuntime: DMNNode evaluate*
+end
+DMNRuntime -> DecisionNode: decision.getEvaluator().evaluate
+DecisionNode -> DMNExpressionEvaluator: evaluate(DMNRuntimeEventManager, DMNResult)
+DMNExpressionEvaluator -> FEELImpl: newEvaluationContext
+FEELImpl -> DMNExpressionEvaluator: EvaluationContextImpl
+DMNExpressionEvaluator -> FEELImpl: evaluate(CompiledExpression, EvaluationContextImpl)
+FEELImpl -> ProcessedExpression: apply(EvaluationContextImpl)
+ProcessedExpression -> CompiledFEELExpression: (InterpretedExecutableExpression || CompiledExecutableExpression) apply(EvaluationContextImpl)
+CompiledFEELExpression -> ProcessedExpression: Object
+ProcessedExpression -> FEELImpl: Object
+FEELImpl -> DMNExpressionEvaluator: Object
+DMNExpressionEvaluator -> DMNRuntime: EvaluatorResult(Object)
+DMNRuntime -> CoerceUtil: coerceValue(DMNType, Object)
+CoerceUtil -> DMNRuntime: Object
+DMNRuntime -> DMNResult: setResult(Object)
+DMNRuntime -> DMNResult: setEvaluationStatus(SUCCEEDED || FAILED)
+DMNRuntime -> Actor: DMNResult
+end
+@enduml