Home

Roadmap TODO and Roadmap DONE

Roadmap

Basic tests for all the methods (PARTIALLY DONE)

This is mandatory to verify code correctness and ease within-versions and portability work (including Octave and Windows support).

• Method level: parameter(s) + dataset + expected performance
• Script level: scripts with CV parameter(s) + dataset + expected performance (TODO performance reference)

Portability (DONE)

Tested platforms:

• Matlab's versions (DONE)
• Windows support
  • Tested on Windows 10: Matlab 2017b and Octave 4.2
  • Windows 8.1: Matlab 2016b and Octave 4.2
• Linux support:
  • Tested on Linux Mint/Ubuntu 17.1: Matlab 2009a, 2013b, 2016b, 2017b and Octave 4.0
  • Tested on Debian 7: Matlab 2009a

Known errors and limitations (DONE)

We have some errors and limitations to improve:

• [ Handling folds execution failure #8 ] (https://github.com/ayrna/orca/issues/8) (DONE)
• Parameters processing can confuse parameters with similar name #7 (DONE)
• Errors with highly imbalanced datasets, see Issue #9 (DONE)

Datasets and experimental design functionallity

• Add datasets loading tools (TODO?)
• Add k-fold and hold-out experimental designs provided a dataset name (TODO?)
• Perform experiments in all the datasets of a folder (DONE)

Website (PARTIALLY DONE)

• Instalation and quick instalation (DONE)
• Tutorials:
  • scripts (DONE)
  • code example (DONE)
  • how to add a method or a metric (TODO)

API homogenization (DONE)

We need to standardize several aspects of methods (https://github.com/ayrna/orca/issues/6)

• Define common 'abstract' interface for all the methods, including functions that are not used in the experimenter but can be used externally (for instance train/test methods).
• Code documentation, indentation, variable naming convention...

Code quality (DONE)

• We have some methods that have to be divided in several methods to simplify the code (an example is experiments post-processing steps)
• Matlab has a coding guide that we may follow:
  • https://blogs.mathworks.com/loren/2011/02/10/book-review-the-elements-of-matlab-style/
  • https://es.mathworks.com/matlabcentral/newsreader/view_thread/15500
  • https://blogs.mathworks.com/pick/2011/01/14/good-matlab-coding-practices/

Tools

• Create a tool to generate data partitions for ORCA from a single file.