curl -L https://nixos.org/nix/install | sh
Create an image
gcloud compute images create nixos-18091228-a4c4cbb613c-x86-64-linux --source-uri gs://nixos-cloud-images/nixos-image-18.09.1228.a4c4cbb613c-x86_64-linux.raw.tar.gz
Create instances with that image and at least a 2TB SSD and an external IP.
Update the included configuration.nix
and then for each instance:
gcloud compute scp configuration.nix root@<machine>:/etc/nixos/configuration.nix
gcloud compute ssh root@<machine> -- nixos-rebuild switch
nix ping-store --store ssh://<machine> # shouldn't print anything
Add these lines to ~/.config/nix/nix.conf
:
builders = ssh://<ip> - - <jobs> 1 benchmark; ...<for each ip> ; ...
system-features = local kvm # local machine cannot be used for benchmarks
builders-use-substitutes = true
mkdir out
nix build -f . tests.regression_tests.vtr_reg_strong.all -k
If you'd like to see all the output:
nix-build -A tests.regression_tests.vtr_reg_strong.all -k
Data will be in result/summary/data.feather
, which can be loaded in Python with:
import pandas as pd
df = pd.read_feather('result/summary/data.feather')
print(df)
Add a top level attribute to tests.nix
, following the existing examples.
You can use nix repl
to explore:
~/src/vtr-nix-test$ nix repl
Welcome to Nix version 2.4. Type :? for help.
nix-repl> :l
Added 20 variables.
nix-repl> tests.<TAB>
tests.default_vtr_rev tests.dusty_sa_sweep tests.inner_num_sweep_nightly_2 tests.make_inner_num_sweep tests.vtr_dusty_sa
tests.dot_to_us tests.flag_sweep tests.inner_num_sweep_nightly_3 tests.make_inner_num_sweep_comparison tests.vtr_node_reordering
tests.dusty_sa tests.flags_to_string tests.inner_num_sweep_weekly tests.node_reordering
tests.dusty_sa_new_inner_num_sweep tests.inner_num_sweep_nightly tests.inner_num_sweep_with_flag_high tests.regression_tests