astarpa2

A*PA2 evals

This directory contains the evals for the A*PA2 paper.

• experiments/ contains .yaml files with experiment configurations, roughly corresponding to the plots in the paper. (Some .yaml files are the source of multiple plots.)
• Use just download_results to download and extract the results.zip from the astarpa2-evals release on github.
• Alternatively, just release reruns all experiments locally, but be aware that this takes a long time (~8 hours). You could make things faster by running more in parallel (add e.g. -j 5 to the run rule in makefile), or by lowering the number of sequences tested (e.g. lower total_size in experiments/*.yaml).
• evals.ipynb contains all plots and other statistics presented in the paper from the results/*.json files generated by one of the two steps above.