Skip to content

Basic Idea

Jump to bottom
Michael Dyck edited this page Sep 17, 2020 · 3 revisions

Basic Idea

The Evaluation Framework is a set of C# and Python libraries enabling the easy and efficient evaluation of Unity simulation models.

Virtual Prototyping

The idea of the Evaluation Framework is based on the concept of Virtual Prototyping, see e.g. Wikipedia or this publication. In Virtual Prototyping, (usually 3D) simulation models of e.g. mechatronic systems are examined and evaluated with the goal of optimizing the production costs, time-to-market, and the product itself. Without the need for physical prototypes, different product designs from different engineering specialities (mechanics, electronics, control, software development) can be tested and evaluated early in the production process. In order to do so, the following parameters have to be specified:

  • Design alternatives of the product ot be tested, e.g. design of mechanical and electrical components, controller structures, etc.. In the Evaluation Framework these parameters are referred to as EvaluationParamters
  • Decision criteria the product is evaluated against, e.g. product weight for specific mechanical designs, electric energy consumption with different electrical components, stability of various controllers, etc.. In the Evaluation Framework these parameters are referred to as ObjectiveValues

Both the EvaluationParameters and ObjectiveValues together define the complete Evaluation Space.
Hence, in Virtual Prototyping, countless design alternatives, i.e. EvaluationParameters, of simulation models of mechatronic systems (or other products) are simulated and the specified decision criteria, i.e. ObjectiveValues, are used to evalute - and finally optimize - the performance of the respective product designs.

Evaluation Framework

The Evaluation Framework realizes this idea of performing Virtual Prototyping through three distinct software components:

  • Definition of Evaluation Space: The first component of the Evaluation Framework, i.e. EvaluationFrameworkUnity EvaluationFrameworkROSUnity, allows users to intuitively and easily define EvaluationParamters and ObjectiveValues from any Unity simulation model.
  • Execution and Evaluation: The second component of the Evaluation Framework is implemented as a .NET solution in C#, see EvaluationFramework and EvaluationFrameworkROS. This component efficiently simulates the Unity project for all combinations of the previously defined EvaluationParamters, retrieves the ObjectiveValues and stores them in a table-formatted .txt-file.
  • Point Cloud Visualization: The third component of the Evaluation Framework visualizes the evaluation results obtained from the second step in an interactive Python GUI. The corresponding source code can be found in the VisualizationGUI folder. The GUI has multiple interactive elements that allow the user to inspect the Evaluation Space from different perspectives, compare ObjectiveValues for different combinations of EvaluationParameters and identify conflicting design alternatives and product solutions.

For a full example on using the Evaluation Framework, please have a look at this application example.

© Siemens AG, 2020

Author: Michael Dyck ([email protected])

Clone this wiki locally