First version of Engineer

First version of Engineer

I have been competing in Numer.ai with my own models for a while now, and half a year ago decided to clean up the current version of the model enough so that I can release it.

So here is Engineer. It is a genetic programming algorithm. Its precursors were considerably more complex, consisting of multiple DNA strings, and with a neural network bolted on top. But this bare-bone engineer seems to be doing fine as well, and has a decent speed in evolving a solution. Its public track record is located at numer.ai.

Some housekeeping notes:

1. There is one file that was too big to directly include in the repository - and that is the pairwise correlations between the Rain data features. So download and unzip v42correlations.zip, and place it in the /data directory.

2. If you want to make use of its ability to automatically poll, fetch, predict and submit predictions, you need to replace the two .dummy files in the /secret directory with versions that contain the right model-ids and upload keys, and rename them to model-ids.secret upload-key.secret. After that, engineer should be good to run.

3. set MKL_NUM_THREADS to 1 in your environment if you are going to train your own models with it. There is enough parallelism higher up in the execution chain. Using the default MKL_NUM_THREADS just slows things down and heats up your house.