Home

Welcome to the bramble wiki!

Mostly following this tute.

Creating the master image:

• Starting with the Raspbian Lite image

username: pi

password: raspberry

• raspi-config
• Network Options > Hostname > raspberrypi-master
• Network Options > Wi-fi > ...
• Interfacing Options > SSH > Enable
• Advanced Options > Expand Filesystem
• Advanced Options > Memory Split

HPL

• Followed instructions from tute on how to install mpich-3.2
• sudo apt-get install libatlas-base-dev gfortran
• Then download and install HPL:

cd hpl
wget https://netlib.org/benchmark/hpl/hpl-2.2.tar.gz
tar xf hpl-2.2.tar.gz
cd hpl-2.2/setup
sh make_generic
cd ..
cp setup/Make.UNKNOWN Make.rpi
nano Make.rpi

Modify the following lines:

TOPdir       = $(HOME)/hpl/hpl-2.2
MPdir        = /home/rpimpi/mpi-install/
MPinc        = -I $(MPdir)/include
MPlib        = $(MPdir)/lib/libmpich.so
LAdir        = /usr/lib/atlas-base/
LAlib        = $(LAdir)/libf77blas.a $(LAdir)/libatlas.a

Update bin/rpi/HPL.dat to:

Innovative Computing Laboratory, University of Tennessee
HPL.out      output file name (if any)
6            device out (6=stdout,7=stderr,file)
1            # of problems sizes (N)
5040         Ns
1            # of NBs
128          NBs
0            PMAP process mapping (0=Row-,1=Column-major)
1            # of process grids (P x Q)
1            Ps
1            Qs
16.0         threshold
1            # of panel fact
2            PFACTs (0=left, 1=Crout, 2=Right)
1            # of recursive stopping criterium
4            NBMINs (>= 1)
1            # of panels in recursion
2            NDIVs
1            # of recursive panel fact.
1            RFACTs (0=left, 1=Crout, 2=Right)
1            # of broadcast
1            BCASTs (0=1rg,1=1rM,2=2rg,3=2rM,4=Lng,5=LnM)
1            # of lookahead depth
1            DEPTHs (>=0)
2            SWAP (0=bin-exch,1=long,2=mix)
64           swapping threshold
0            L1 in (0=transposed,1=no-transposed) form
0            U  in (0=transposed,1=no-transposed) form
1            Equilibration (0=no,1=yes)
8            memory alignment in double (> 0)

Setting Up network:

• Clone the above image, flash onto new SD cards (this took about 20 mins per 16GB card for me) and insert into every pi in the network.
• I was using Etcher on MacOSX to flash the SD cards. And I used SD Clone on MacOSX to produce the clone master image.
• Plug pis into the switch and power on the switch (and pis).
• sudo raspi-config and change the hostname in each pi (i have been using raspberrypi-## for the slave nodes and raspberrypi-master for the master node).
• I also ran ifconfig on each pi and noted the eth0 inet (IP) address.
• Once we know the hostname and IP for every pi we can generate and distribute ssh-keys as described in the tutorial above.
• ssh back to raspberrypi-master and enter the ip addresses on seperate lines in a file called machinefile

169.254.186.206
169.254.12.216
169.254.56.18
169.254.24.170

• Update the HPL.dat file so that PXQ=num_nodes and increase Ns to increase computational load.
• Execute HPL across multiple nodes mpiexec -f ~/machinefile -n 5 ./xhpl

SLURM

I'm just following Simon's SLURM tutorial from the other day.

I had to add cgroup.conf to the /etc/slurm-lnll directory on all nodes running slurmd. My cgroup.conf is below:

CgroupMountpoint=/cgroup
ConstrainCores=yes
AllowedRAMSpace=98.5

Also had to change the permissions for the StateSaveLocation specified in slurm.conf so that the SlurmUser (also specified in slurm.conf) could read/write. In my case this looked like sudo chown slurm: /var/state/slurm

Found this nice tool for creating .conf files.

SLURM Top

Sweet command that monitor's useage across all nodes.

Voice Activation with Google Cloud's Voice API