-
Notifications
You must be signed in to change notification settings - Fork 4
Home
#Welcome to Data Reconciliation and Gross Error Detection Benchmark Problems Repository
The aim of this project is to provide a collection of benchmark problems for Data Reconciliation and Gross Error Detection from literature review.
The collection of benckmark problems are organized systematically according to the dificulty and the problem characteristic that is supposed to be tested.
The idea is to incremment the benchmark repository adding new problems and writing the problems in other languages, such as matlab, GAMS, python, C, C++, java and others.
Initially, the benchmark problems are implemented in the free software Scilab (version 5.3 and 5.4, 5.5.2) using the sci-ipopt optimization toolbox (interface between Scilab and Ipopt optimization solver), since it is a free software and can be used by anyone without charge.
To install the benchmark problems, one should download the latest version of the .zip pack at the "download" tab.
If one is familiar with SVN version control system, it can be "checked out" at the "source" tab, that will have a link to the checkout command. It can also be used GIT version control system.
The zipped version of the benchmark set are always the latest one:
dr-ged-benchmarks-master.zip
Scilab can be downloaded at http://www.scilab.org/download/5.5.2 (32 or 64 bits version), it is encouraged to install the latest Scilab version. Scilab 6.0.2 does not work with SciIpopt, so it will be necessary to use version 5.5.2.
- In menu applications, ATOMS, search for Diffcode, or navigate by category in Optimization.
- Select install (it will take some time to download).
3.2.1 This instructions is for LINUX users.
With Scilab version 5.5.2 it is very easy to install Sci-Ipopt either for 32 bits or 64 bits version.
- In menu applications, ATOMS, search for Sci-IPOpt, or navigate by category in Optimization.
- Select install (it will take some time to download).
Install "libgfortran3". In linux terminal, run:
sudo apt-get install libgfortran3
It may be necessary to tell the system where are the libraries:
export LD_LIBRARY_PATH="path to lib without quotes"
Restart Scilab, the toolboxes will start automatically and you will see the following message at your Scilab prompt:
Startup execution: loading initial environment
Start Helptbx
Start Apifun Load macros Load help Type "help apifun_overview" for quick start.
Start Diffcode Load macros Load help Load demos Type "help diffcode_overview" for quick start. Type "demo_gui()" and search for Diffcode for Demonstrations.
Start SciIPOpt Load macros Load gateways Load help Load demos
3.2.2 Windows Users
Download Scilab 5.5.2
With Scilab version 5.5.2 it is very easy to install Sci-Ipopt either for 32 bits or 64 bits version.
- In menu applications, ATOMS, search for Sci-IPOpt, or navigate by category in Optimization.
- Select install (it will take some time to download).
Restart Scilab, the toolboxes will start automatically and you will see the following message at your Scilab prompt:
Startup execution: loading initial environment
Start Helptbx
Start Apifun Load macros Load help Type "help apifun_overview" for quick start.
Start Diffcode Load macros Load help Load demos Type "help diffcode_overview" for quick start. Type "demo_gui()" and search for Diffcode for Demonstrations.
Start SciIPOpt Load macros Load gateways Load help Load demos ``
The file structure can be inspected going to Source Tab, that will list a tree in the left. Go to "trunk" and one will find Data Reconciliation or GED (Gross Error Detection)
Data reconciliation have 4 sub folders
-
data_analysis
The aim of this problems is to test data filter (filter on the variables before DR is applied), data filter window selection and influence on the dynamics on the DR results. These problems were built using the xcos (simulink version of scilab).
-
error_in_variables
Error in Variables (EVM) is a problem where a DR is carried out together with parameter estimation, the aim of this problem is to test algorithms for this problem class.
-
linear
Although linear problems are easy to solve in DR, since it have analytical solution, the aim of this problem is to test different objetctive function types, called robust functions. These robust functions can provide results with gross errors removed, however, to do this, some parameters must be tunned in these function. The aim of this class problems is to test these parameters set up.
-
non-linear
The aim of these problems is to test DR in more complex and non-linear problems. Some features are tested such as:
-
How DR behaves with observable non-measured variables.
-
How DR behaves with non-observable non-measured variables.
-
How DR behaves with models which are approximated by model simplifications.
Each folder has 3 sobfolders:
- diagrams: a diagram of the process in the .dia and .png format.
- scilab: the scilab implementation.
- spreadsheet: a Excel spreadsheet of the data, results or some data handling necessary to run the model.
Data Reconciliation problems description can be found here
-
linear
- scilab:
- 16 Scilab linear test problems (P1-P16).
- "P7_test_sample_size" and "P8_test_sample_size" - Test the influence of the number of simulation runs.
- functions: The GED test functions and help funtions.
- jacobians: The jacobians of 16 problems.
- spreadsheet:
- The data for the problems (exact measurements, standard deviations, and results of the Global Test, Measurement Test and Nodal Test). The diagrams of these problemas are the same as "linear/diagrams" folder.
-
nonlinear
-
scilab: the scilab implementation.
-
8 nonlinear problems for GED methods testing.
-
functions: The GED test functions and help funtions.
The diagrams of these problems are the same as "nonlinear/diagrams" folder in the data reconciliation set.
-
-
data_analysis The aim of this problems is to test:
- Data filter (filter on the variables before DR is applied), data filter window selection and influence on the dynamics on the DR results.
- The GED techniques with different types of measurement bias, such as constant bias, drifting bias, miscallibration and also leakings.
- The GED techniques in single and multiple gross errors scenarios.
These problems were built using the xcos (simulink version of scilab).
Each folder has 3 subfolders:
- diagrams: a diagram of the process in the .dia and .png format.
- scilab: the scilab implementation.
- spreadsheet: a Excel spreadsheet of the data, results or some data handling necessary to run the model.
The implementation detailing can be found for data analysis, linear problems and nonLinear problemas .