Evaluation Framework

The Evaluation Framework is a set of open source software libraries and tools in C# , implemented in .NET, and Python for efficiently evaluating simulation models of mechatronic systems, in particular by using the game engine Unity as simulation environment, and visualizing the evaluation results in an interactive GUI.

This project consists of 3 tools to evaluate many variants of (Unity) simulations and visualize the results. The tools can be used independently, or in a sequence.

---

Tool 1: Define Variants and Targets in Unity

1. Include EvaluationFrameworkUnity into the Asset folder of your Unity project.
2. Define the parameters you want to vary and the target values you want to evaluate.
3. Generate the resulting .json files and compile your Unity project as a standalone .exe.

Go here for additional information on this specific tool.
Have a look at the developer documentation for this tool here.

Tool 2: Evaluate all Variants

Execute the C# console application EvaluationFramework to evaluate all variants.

Go here for additional information on this specific tool.
Have a look at the developer documentation for this tool here.

Tool 3: Visualize the Results

Visualize the evaluation results in a special UI using the python application VisualizationGUI.

Go here for additional information on this specific tool.
Have a look at the developer documentation for this tool here.

Contents

• Unity3D: Source code of the Evaluation Framework connected to the game engine Unity (Tool 1)
• EvaluationFramework: .NET solution of the Evaluation Framework (Tool 2)
• EvaluationFrameworkROS: .NET solution of the Evaluation Framework with the extension of communicating with a ROS system via ROS# (Tool 2)
• VisualizationGUI: Python code of the GUI visualizing the evaluation results (Tool 3)

Here is an application example illustrating what can be done with the Evaluation Framework.

Releases

Each release contains:

• EvaluationFrameworkUnity.zip: Zip file containing the Unity package EvaluationFrameworkUnity (Tool 1).
• EvaluationFrameworkROSUnity.zip: Zip file containing the Unity package EvaluationFrameworkROSUnity (Tool 1).
• EvaluationFramework.zip: Zipped binaries, i.e. the build of EvaluationFramework (Tool 2)
• EvaluationFrameworkROS.zip: Zipped binaries, i.e. the build of EvaluationFrameworkROS (Tool 2)
• VisualizationGUI.zip: Zip file containing the VisualizationGUI python application (Tool 3)

Please get the latest development version directly from the tip of this master branch.

Licensing

The Evaluation Framework is open source under the Apache 2.0 license and is free for commercial use.

Further Info

• Contribution and Feedback
• Platform Support
• Software Version Requirements
• External Dependencies
• Contributors and Acknowledgements

We are also curious to hear how you intend to use - or are already using - the Evaluation Framework. If you like, come tell us more about your project here.

For general information, please read the Wiki.
If you have any questions, feel free to contact the Author.

---

© Siemens AG, 2020

Author: Michael Dyck ([email protected])