How can I create an HPC cluster with two laptops?

 

How should you start learning programming?

Taking your first steps into programming might seem scary. Without a proper plan and the right approach to learning, just getting started might take so much time and effort that a(Continue reading in feed)

5 Answers

Nikos Kazazakis

, PhD Deterministic Global Optimization & High Performance Computing

Answered May 31, 2018

You can create a cluster in 5 easy steps:

1. Make sure that both computers have each other’s public key to ssh without a password.
2. Connect them with an ethernet cable, and make sure that the network between them is active. You can easily check this if you turn off wifi on one laptop and try to access the internet from it, using the inter-laptop network.
3. Configure your hosts file using the ip addresses in your LAN:

$ cat /etc/hosts 
127.0.0.1       localhost 
172.50.88.34    client 

1. Create an nfs drive in an unencrypted directory in both computers. One computer will be the nfs server, and the other one will be the client. You will know that this works properly once you are able to modify a file in that drive on one computer, and you can see that file automatically syncing in the other computer. Anything you run in that directory is synced in real time over the network (hard drive access is very slow).
2. Run something in the nfs directory while the computers are networked. Depending on what you are running it might just use all cores automatically, but most likely you will need to specify more info specific to the binary you are running. For instance, for MPI you need to provide a hostfile:

$ mpirun -np 4 --hostfile my_hostfile ./my_binary 

This will not quite be an HPC cluster however, in the sense that HPC typically uses more advanced stuff for scheduling jobs and controlling user environments (such as PBS and modules).

Keep in mind that sending data to your second laptop over the network to be processed in the additional cores is very slow (it takes milliseconds), which means that you need to send large, time consuming tasks to be processed at that remote computer if you want to observe any speedups in your application (otherwise the communication overhead is too high for the cluster to be meaningful).

 

https://www.quora.com/How-can-I-create-an-HPC-cluster-with-two-laptops