Raspberry Pi Cluster

I was inspired by a Technology blogger Gary Sims to build my own (small) super computer using Raspberry Pi’s and a message passing interface library. Following his Prime Numbers example the key concepts are:

MPI library - Message passing interface, this is a standard way of sending blocks of data into a cluster from one node to another. For the examples below I used mpiexec which is free and open source.

Scatter - the MPI program will see each core in your cluster as something it can ask to do work. It will scatter the numbers accross the nodes to check if they are prime.

Gather - the MPI program will then gather the results. This is done on the master node that you started the command from.

Raspberry Pi Cluster

In this video Gary explains what a Raspberry Pi Supercomputer Cluster is and the key concepts of the message passing interface.

Setup Hardware / OS

Follow the OS Steps (For Cluster), then call them

  • node1
  • node2

The distro come with most applications already installed, some of the applications I used were git, nano and ssh. Additional installs:

1
sudo apt-get install git python-mpi4py keychain ansible

Setup Passwordless SSH Access

See the Keychain section on this passwordless SSH access post.

Hostfile

When running mpiexec python python_script.py it can be run with the switch -hostfile HOSTFILENAME, so the full command is then mpiexec -hostfile hostfile python python_script.py

The content of the hostfile is either the DNS name or IP address of all the nodes in the cluster.

1
2
node1
node3

Work Load

The PI’s need some problems to solve! Clone https://github.com/carlpaton/python-hoon to the same dir on both PI’s. Its important that the directorys are the same as the MASTER node (using MPI) will look in these directorys for the script to run.

Check mpi4py

The script prints hostnames and ranks (process id in MPI) from each MPI process in a sequential manner.

1
2
3
cd python-hoon/mpiexec
time mpiexec python check_mpi4py.py
time mpiexec -np 4 -hostfile hostfile python check_mpi4py.py          ~ limit to 4 cores

This script was copied from chainermn.readthedocs.io

Prime Numbers

Use parallel computing to find prime numbers in the range 0 to 10000. This is a Python 2 script cloned from github.com/garyexplains

1
2
cd python-hoon/mpiexec
time mpiexec -hostfile hostfile python primenumbers27.py              ~ no core limit, use all nodes in `hostfile`

The results below show the time taken to complete the script primenumbers27.py using 8 cores, 4 cores and finally just 1 core. The results should probably be averaged out or something but running just once like the below clearly shows speed improvements. Using Wifi and the speed of my SD cards would also come into play.

8 Core 6.42s

4 Core 8.02s

1 Core 28.06s

What Are Prime Numbers?

  • Greater than 1
  • Whole number with exactly two factors, itself and 1.

Some examples of prime numbers: 2,3,5,7,11,13,17,19

The number 4 is not a prime number because it can be divided evenly by 4, 2, and 1. The number 5 is a prime number because it cannot be divided evenly by any other numbers except for 5 and 1.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
2 / 1 = 2
2 / 2 = 1

3 / 1 = 3
3 / 2 = 1.x
3 / 3 = 1

4 / 1 = 4
4 / 2 = 2
4 / 3 = 1.x
4 / 4 = 1

5 / 1 = 5
5 / 2 = 2.x
5 / 3 = 1.x
5 / 4 = 1.x
5 / 5 = 1

6 / 1 = 6
6 / 2 = 3
6 / 3 = 2
6 / 4 = 1.x
6 / 5 = 1.x
6 / 6 = 1

7 / 1 = 7
7 / 2 = 3.x
7 / 3 = 2.x
7 / 4 = 1.x
7 / 5 = 1.x
7 / 6 = 1.x
7 / 7 = 1

References

Docker Swarm

This cluster can also be run with Docker in swarm mode.