Skip to content

Commit

Permalink
[DBACLD-143918] Write developer documentation for the DMN implementation
Browse files Browse the repository at this point in the history
  • Loading branch information
Gabriele-Cardosi committed Jul 22, 2024
1 parent 72691f3 commit 6eddb73
Show file tree
Hide file tree
Showing 5 changed files with 185 additions and 24 deletions.
109 changes: 85 additions & 24 deletions kie-dmn/Developer_Guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,30 +7,31 @@ The engine is responsible, in very general terms, of
2. validate it
3. evaluate it, against a given input; please note that execution could be statically-interpreted or dynamically-code-generated (see later on [Interpreted vs Codegen](#interpreted-vs-codegen))

## DTOs
* [Definitions](kie-dmn-model%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fdmn%2Fmodel%2Fapi%2FDefinitions.java): the direct representation of the original XML
* [DMNResource](kie-dmn-core%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fdmn%2Fcore%2Fassembler%2FDMNResource.java): a wrapper used to forward `Definitions` with additional informations (e.g. [ResourceWithConfiguration.java](..%2Fkie-api%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fapi%2Fio%2FResourceWithConfiguration.java))
* [DMNNode](kie-dmn-api%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fdmn%2Fapi%2Fcore%2Fast%2FDMNNode.java): the _ast_ representation of high-level nodes inside the model; it map to **_BusinessKnowledge_**, **_Decision_**, **_DecisionService_**, **_ItemDef_**, or **_InputData_**
* [BaseNode](kie-dmn-feel%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fdmn%2Ffeel%2Flang%2Fast%2FBaseNode.java): the _ast_ representation of low-level nodes inside the model (e.g. **_Boolean_**, **_AtLiteral_**, **_UnaryTest_**, **_String_**, ecc.)
* [CompiledFEELExpression](kie-dmn-feel%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fdmn%2Ffeel%2Fcodegen%2Ffeel11%2FCompiledFEELExpression.java): the object that represents and actually executes a specific _FEEL-expression_
* [InterpretedExecutableExpression](kie-dmn-feel%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fdmn%2Ffeel%2Flang%2Fimpl%2FInterpretedExecutableExpression.java): the `CompiledFEELExpression` instantiated for statically-interpreted execution of the specific _FEEL-expression_
* [CompiledExecutableExpression](kie-dmn-feel%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fdmn%2Ffeel%2Flang%2Fimpl%2FCompiledExecutableExpression.java): the `CompiledFEELExpression` instantiated for code-generated execution of the specific _FEEL-expression_
* [ProcessedExpression](kie-dmn-feel%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fdmn%2Ffeel%2Fcodegen%2Ffeel11%2FProcessedExpression.java): the result of the compilation of a specific _FEEL-expression_; in turns, it wraps the actual `CompiledFEELExpression` to be executed; please note that a _FEEL-expression_ could represent an aggregation of other sub-expressions; so, in turns, a `ProcessedExpression` may contain nested ones
* [DMNExpressionEvaluator.java](kie-dmn-core%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fdmn%2Fcore%2Fapi%2FDMNExpressionEvaluator.java): a `FunctionalInterface` that it is invoked by evaluation objects (e.g. `Decision`) to actually invoke the generated `ProcessedExpression`;
* [DMNModel](kie-dmn-api%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fdmn%2Fapi%2Fcore%2FDMNModel.java): the high-level result of the compilation, to be used to invoke the overall evaluation; it contains `Definitions` and all the `DMNNode`s obtained by compilation
* [DMNResult](kie-dmn-api%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fdmn%2Fapi%2Fcore%2FDMNResult.java): an instance that encapsulates all the information resulting from a DMN service invocation;
* [DMNDecisionResult](kie-dmn-api%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fdmn%2Fapi%2Fcore%2FDMNDecisionResult.java): an instance that stores the result of the evaluation of a decision

## Code usage

### Compilation
## Structure
### DTOs
#### * [Definitions](kie-dmn-model%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fdmn%2Fmodel%2Fapi%2FDefinitions.java): the direct representation of the original XML
#### * [DMNResource](kie-dmn-core%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fdmn%2Fcore%2Fassembler%2FDMNResource.java): a wrapper used to forward `Definitions` with additional informations (e.g. [ResourceWithConfiguration](..%2Fkie-api%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fapi%2Fio%2FResourceWithConfiguration.java))
#### * [DMNNode](kie-dmn-api%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fdmn%2Fapi%2Fcore%2Fast%2FDMNNode.java): the _ast_ representation of high-level nodes inside the model; it map to **_BusinessKnowledge_**, **_Decision_**, **_DecisionService_**, **_ItemDef_**, or **_InputData_**
#### * [BaseNode](kie-dmn-feel%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fdmn%2Ffeel%2Flang%2Fast%2FBaseNode.java): the _ast_ representation of low-level nodes inside the model (e.g. **_Boolean_**, **_AtLiteral_**, **_UnaryTest_**, **_String_**, ecc.)
#### * [CompiledFEELExpression](kie-dmn-feel%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fdmn%2Ffeel%2Fcodegen%2Ffeel11%2FCompiledFEELExpression.java): the object that represents and actually executes a specific _FEEL-expression_; it is also a `FunctionalInterface`, since it extends `Function<EvaluationContext, Object>`
#### * [InterpretedExecutableExpression](kie-dmn-feel%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fdmn%2Ffeel%2Flang%2Fimpl%2FInterpretedExecutableExpression.java): the `CompiledFEELExpression` instantiated for statically-interpreted execution of the specific _FEEL-expression_
#### * [CompiledExecutableExpression](kie-dmn-feel%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fdmn%2Ffeel%2Flang%2Fimpl%2FCompiledExecutableExpression.java): the `CompiledFEELExpression` instantiated for code-generated execution of the specific _FEEL-expression_
#### * [ProcessedExpression](kie-dmn-feel%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fdmn%2Ffeel%2Fcodegen%2Ffeel11%2FProcessedExpression.java): the result of the compilation of a specific _FEEL-expression_; in turns, it wraps the actual `CompiledFEELExpression` to be executed; please note that a _FEEL-expression_ could represent an aggregation of other sub-expressions; so, in turns, a `ProcessedExpression` may contain nested ones
#### * [DMNExpressionEvaluator](kie-dmn-core%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fdmn%2Fcore%2Fapi%2FDMNExpressionEvaluator.java): a `FunctionalInterface` that it is invoked by evaluation objects (e.g. `Decision`) to actually invoke the generated `ProcessedExpression`;
#### * [DMNModel](kie-dmn-api%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fdmn%2Fapi%2Fcore%2FDMNModel.java): the high-level result of the compilation, to be used to invoke the overall evaluation; it contains `Definitions` and all the `DMNNode`s obtained by compilation
#### * [DMNResult](kie-dmn-api%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fdmn%2Fapi%2Fcore%2FDMNResult.java): an instance that encapsulates all the information resulting from a DMN service invocation;
#### * [DMNDecisionResult](kie-dmn-api%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fdmn%2Fapi%2Fcore%2FDMNDecisionResult.java): an instance that stores the result of the evaluation of a decision

### Code usage

#### Compilation
The simplest way to compile a model is to instantiate a [DMNRuntime](kie-dmn-api%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fdmn%2Fapi%2Fcore%2FDMNRuntime.java) from its file, e.g.
```java
DMNRuntime runtime = DMNRuntimeUtil.createRuntime("simple-item-def.dmn", this.getClass() );
```
[DMNRuntimeUtil](kie-dmn-core%2Fsrc%2Ftest%2Fjava%2Forg%2Fkie%2Fdmn%2Fcore%2Futil%2FDMNRuntimeUtil.java) has different overrides of the `createRuntime` method

### Validation
#### Validation
To validate a dmn model, it is necessary to:
1. get an instance of [DMNValidator](kie-dmn-validation%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fdmn%2Fvalidation%2FDMNValidator.java), built with some [DMNProfile](kie-dmn-core%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fdmn%2Fcore%2Fcompiler%2FDMNProfile.java); e.g.
```java
Expand All @@ -44,7 +45,7 @@ To validate a dmn model, it is necessary to:
VALIDATE_SCHEMA, VALIDATE_MODEL, VALIDATE_COMPILATION);
```

## Execution
### Execution
To execute a dmn model, it is necessary to:
1. get an instance of `DMNRuntime`, built around the given model; e.g.
```java
Expand All @@ -67,9 +68,9 @@ To execute a dmn model, it is necessary to:

Please note that `DMNRuntime` has different `evaluate*` methods

## Execution flows
### Execution flows

### Compilation
#### Compilation

The original xml is first parsed to `Definitions` by the [DMNMarshaller](kie-dmn-api%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fdmn%2Fapi%2Fmarshalling%2FDMNMarshaller.java), that is a simple xml-marshaller([DMNAssemblerService#addResourcesAfterRules](kie-dmn-core%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fdmn%2Fcore%2Fassembler%2FDMNAssemblerService.java)).

Expand All @@ -83,19 +84,79 @@ Inside `ProcessedExpression`, the expression is parsed to `org.antlr.v4.runtime.

Finally, the `CompiledFEELExpression` to be executed is returned (`ProcessedExpression#asCompiledFEELExpression`).

#### Interpreted vs Codegen
![Sequence Diagram](uml/Compilation.png)

##### Interpreted vs Codegen
The retrieved `CompiledFEELExpression` could be a _statically-interpreted_ `InterpretedExecutableExpression` (that wraps the original `BaseNode` **_ast_**) or could be a _dynamically-code-generated_ `CompiledExecutableExpression`, in which case:
1. source code is generated out of the given `BaseNode` **_ast_** (by `ASTCompilerVisitor`)
2. code is compiled in-memory to a `CompiledExecutableExpression (by [CompilerBytecodeLoader](kie-dmn-feel%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fdmn%2Ffeel%2Fcodegen%2Ffeel11%2FCompilerBytecodeLoader.java))
3. the above `CompiledFEELExpression` is wrapped and returned inside a `CompiledExecutableExpression`

### Validation
#### Validation
Depending on a series of flags ( `VALIDATE_SCHEMA`, `VALIDATE_MODEL`, `VALIDATE_COMPILATION`, `ANALYZE_DECISION_TABLE`), [DMNValidatorImpl](kie-dmn-validation%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fdmn%2Fvalidation%2FDMNValidatorImpl.java) executes the validation of the given model.

Behind the scenes, the validation also uses the _rule engine_ and the _rules_ defined in the different `kie-dmn/kie-dmn-validation/src/main/resources/org/kie/dmn/validation/DMNv1()/*.drl` files.
This validation is fired inside `private List<DMNMessage> validateModel(DMNResource mainModel, List<DMNResource> otherModels)`
### Execution
#### Execution
When `DMNRuntime#evaluate*` is invoked, a `DMNResult` is instantiated, containing a map with `DMNDecisionResult`s, initially marked as `NOT_EVALUATED`.
Then, for each [DecisionNode](kie-dmn-api%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fdmn%2Fapi%2Fcore%2Fast%2FDecisionNode.java):
1. the mapped `DMNDecisionResult` is marked as `EVALUATING`
2. the associated [DMNExpressionEvaluator](kie-dmn-core%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fdmn%2Fcore%2Fapi%2FDMNExpressionEvaluator.java) is retrieved (`DecisionNodeImpl#getEvaluator()`)
3. the `DMNExpressionEvaluator#evaluate(DMNRuntimeEventManager, DMNResult)` method is invoked (see [DMNExpressionEvaluator](#--dmnexpressionevaluatorjava-a-functionalinterface-that-it-is-invoked-by-evaluation-objects-eg-decision-to-actually-invoke-the-generated-processedexpression))
4. inside the concrete implementation, a new `EvaluationContextImpl` is created from the `FEELImpl` instance
5. the `FEELimpl#evaluate(CompiledExpression expr, EvaluationContext ctx)` method is invoked, passing the `CompiledExpression` _**expression**_ wrapped by the current `DMNExpressionEvaluator`
6. in turns, the `ProcessedExpression#apply(EvaluationContext)` method is invoked (`ProcessedExpression` is the actual type of `CompiledExpression`)
7. this result in invocation of the wrapped `CompiledFEELExpression` **_executableFEELExpression_** (that could be an `InterpretedExecutableExpression` or a `CompiledExecutableExpression`)
8. this, recursively, iterate over all the nested expressions, invoking the [FEELFunction](kie-dmn-feel%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fdmn%2Ffeel%2Fruntime%2FFEELFunction.java)s mapped to any expression, until a given final result is provided, as `Object`
9. then, the returned object is coerced to the type requested by the decision, the result is stored in the mapped `DMNDecisionResult`, and that latter is marked as `SUCCEEDED`
![Sequence Diagram](uml/Execution.png)
### [BaseFEELFunction](kie-dmn-feel%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fdmn%2Ffeel%2Fruntime%2Ffunctions%2FBaseFEELFunction.java)
Every _FEEL function_ is mapped to a concrete class extending `BaseFEELFunction`.
During execution, this `FEELFunction` is looked-up by name, then the actual method to be invoked is looked-for, **_reflectively_**, based on the given input parameters.
The critical point where this happen is the `BaseFEELFunction#getCandidateMethod(EvaluationContext ctx, Object[] originalInput, boolean isNamedParams)` method.
Based on a algorithm defined in the [ScoreHelper](kie-dmn-feel%2Fsrc%2Fmain%2Fjava%2Forg%2Fkie%2Fdmn%2Ffeel%2Fruntime%2Ffunctions%2FScoreHelper.java), each `invoke` method is tested and provided a score. The one with the highest score will be used for actual function evaluation.
### Coercion
During `invoke` method discovery, inside `BaseFEELFunction`, the given input parameters are _coerced_ to potentially match the ones expected by the current `invoke` method. The rules for this coercion are the ones from the DMN specification.
Another coercion is applied whenever a number, or a string representing a number, is received, in which case it is translated to `BigDecimal`
## Development guidelines
The main goal for the DMN engine are performance and maintainability.
About the former, whenever important refactoring are done, it would be important to also execute [DMN Benchmarks](https://github.com/apache/incubator-kie-benchmarks/) to verify for modification of them, fixing eventual regressions.
About the latter, it would be important to strive for simplicity of reading, instead of too-hard-to-read conciseness.
It would also be important to follow goo'ol' rules, like:
* self-explanatory method names
* self-resilient methods, i.e. a method should work by itself, and eventually manage failures, without depending on some status/assumption provided by invoking code
* meaningfully property/variable names
* focused classes, dedicated to a clear and specific task
* unit-test methods as much as possible, especially newly created ones
Expand Down
Binary file added kie-dmn/uml/Compilation.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
45 changes: 45 additions & 0 deletions kie-dmn/uml/Compilation.puml
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
/'
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
'/
@startuml
participant DMNAssemblerService
DMNAssemblerService -> DMNMarshaller : unmarshal(Reader)
DMNMarshaller -> DMNAssemblerService: Definitions
DMNAssemblerService -> DMNCompilerImpl: compile(Definitions, Resource, Collection<DMNModel> || Resource, Collection<DMNModel>)
DMNCompilerImpl -> DMNMarshaller : unmarshal(Resource.getReader())
DMNMarshaller -> DMNCompilerImpl: Definitions
DMNCompilerImpl -> DMNCompilerImpl: new DMNModelImpl
loop every ItemDefinition in Definitions.getItemDefinition
DMNCompilerImpl -> DMNCompilerImpl: new ItemDefNodeImpl
DMNCompilerImpl -> DMNModelImpl: addItemDefinition(ItemDefNodeImpl)
end
loop every DRGElement in Definitions.getDrgElement
DMNCompilerImpl -> DRGElementCompiler: compileNode(DRGElement, DMNCompilerImpl, DMNModel)
DRGElementCompiler -> FEELImpl: compile(String, CompilerContext)
FEELImpl -> ProcessedExpression: new
ProcessedExpression -> FEEL_1_1Parser: compilation_unit
FEEL_1_1Parser -> ProcessedExpression: ParseTree
ProcessedExpression -> ParseTree: accept(ASTBuilderVisitor)
ParseTree -> ProcessedExpression: BaseNode
FEELImpl -> ProcessedExpression: asCompiledFEELExpression
ProcessedExpression -> FEELImpl: InterpretedExecutableExpression || CompiledExecutableExpression
FEELImpl -> DRGElementCompiler: DMNExpressionEvaluator
DRGElementCompiler -> DRGElement: setEvaluator(DMNExpressionEvaluator)
end
DMNCompilerImpl -> DMNAssemblerService: DMNModel
@enduml
Binary file added kie-dmn/uml/Execution.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
55 changes: 55 additions & 0 deletions kie-dmn/uml/Execution.puml
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
/'
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
'/
@startuml
participant Actor
Actor -> DMNRuntimeUtil : createRuntime
DMNRuntimeUtil -> Actor: DMNRuntime
Actor -> DMNRuntime: getModel
DMNRuntime -> Actor : DMNModel
Actor -> DMNFactory: newContext
DMNFactory -> Actor: DMNContext
Actor -> Actor: (populate dmn context)
Actor -> DMNRuntime: evaluate*(DMNModel, DMNContext)
DMNRuntime -> DMNResultImplFactory: createResult
DMNResultImplFactory -> DMNRuntime: DMNResult
loop every DecisionNode in DMNModel.decisions
DMNRuntime -> DMNResult: getDecisionResultById
DMNResult -> DMNRuntime: DMNDecisionResultImpl
DMNRuntime -> DMNRuntime: DMNResult.setEvaluationStatus(EVALUATING)
loop every DMNNode in DecisionNode.dependencies
DMNRuntime -> DMNRuntime: DMNNode evaluate*
end
DMNRuntime -> DecisionNode: decision.getEvaluator().evaluate
DecisionNode -> DMNExpressionEvaluator: evaluate(DMNRuntimeEventManager, DMNResult)
DMNExpressionEvaluator -> FEELImpl: newEvaluationContext
FEELImpl -> DMNExpressionEvaluator: EvaluationContextImpl
DMNExpressionEvaluator -> FEELImpl: evaluate(CompiledExpression, EvaluationContextImpl)
FEELImpl -> ProcessedExpression: apply(EvaluationContextImpl)
ProcessedExpression -> CompiledFEELExpression: (InterpretedExecutableExpression || CompiledExecutableExpression) apply(EvaluationContextImpl)
CompiledFEELExpression -> ProcessedExpression: Object
ProcessedExpression -> FEELImpl: Object
FEELImpl -> DMNExpressionEvaluator: Object
DMNExpressionEvaluator -> DMNRuntime: EvaluatorResult(Object)
DMNRuntime -> CoerceUtil: coerceValue(DMNType, Object)
CoerceUtil -> DMNRuntime: Object
DMNRuntime -> DMNResult: setResult(Object)
DMNRuntime -> DMNResult: setEvaluationStatus(SUCCEEDED || FAILED)
DMNRuntime -> Actor: DMNResult
end
@enduml

0 comments on commit 6eddb73

Please sign in to comment.