noisy-moo

A wrapper-based framework for pymoo problem modification and algorithm benchmarking. Initially developed to test KNN-averaging¹.

Installation

Simply run

pip install nmoo

Getting started

In a notebook

See example.ipynb for a quick example.

For larger benchmarks

For larger benchmarks, you may want to use nmoo's CLI. First, create a module, say example.py, containing your benchmark factory (a function that returns your benchrmark), say make_benchmark(). Then, run it using

python -m nmoo run --verbose 10 example:make_benchmark

Refer to

python -m nmoo --help

for more information.

Main submodules and classes

• nmoo.benchmark.Benchmark: A Benchmark object represents... a benchmark 🤔. At construction, you can specify problems and algorithms to run, how many times to run them, what performance indicators to compute, etc. Refer to nmoo.benchmark.Benchmark.__init__ for more details.
• nmoo.wrapped_problem.WrappedProblem: The main idea of nmoo is to wrap problems in layers. Each layer should redefine pymoo.Problem._evaluate to intercept calls to the wrapped problem. It is then possible to apply/remove noise, keep a call history, log, etc.
• nmoo.denoisers: Sublasses of nmoo.wrapped_problem.WrappedProblem that implement denoising algorithms. In a simple scenario, a synthetic problem would be wrapped in a noise layer, and further wrapped in a denoising layer to test the performance of the latter.
• nmoo.noises: Sublasses of nmoo.wrapped_problem.WrappedProblem that apply noise.

Contributing

Dependencies

• python3.8 or newer;
• requirements.txt for runtime dependencies;
• requirements.dev.txt for development dependencies (optional);
• make (optional).

Simply run

virtualenv venv -p python3.8
. ./venv/bin/activate
pip install -r requirements.txt
pip install -r requirements.dev.txt

Documentation

Simply run

make docs

This will generate the HTML doc of the project, and the index file should be at docs/index.html. To have it directly in your browser, run

make docs-browser

Code quality

Don't forget to run

make

to format the code following black, typecheck it using mypy, and check it against coding standards using pylint.

Footnotes

1. Klikovits, S., Arcaini, P. (2021). KNN-Averaging for Noisy Multi-objective Optimisation. In: Paiva, A.C.R., Cavalli, A.R., Ventura Martins, P., Pérez-Castillo, R. (eds) Quality of Information and Communications Technology. QUATIC 2021. Communications in Computer and Information Science, vol 1439. Springer, Cham. https://doi.org/10.1007/978-3-030-85347-1_36 ↩