Setting Up Fast AI on RC's Community Cluster

January 13, 2019

RC's community cluster is a great resource to take advantage of! I had a rough time getting the Fast AI course set up, so I wrote down some instructions (hopefully) help others save some time.

RC's community cluster is a great resource to take advantage of! I had a rough time getting the Fast AI course set up, so I wrote down some instructions (hopefully) help others save some time.

  1. Make an account on recurse.com/settings/cluster - this requires setting up and copying over an SSH key instructions here.

  2. SSH onto one of the GPU-enabled machines - I chose to use mercer (both Mercer and Crosby have powerful graphics cards). You can find more info about the specs for each machine here

ssh [email protected]
  1. Once on the machine, you should be in your own home directory. The command prompt should look something like this:
[email protected]:~$

  1. Install Conda so that you can create virtual environments to install software that won't affect others using the machine.

  2. Install the fast.ai library (instructions from the course forum)

git clone https://github.com/fastai/fastai.git
cd fastai
conda env create -f environment.yml
  1. Activate the environment for fast.ai
source activate fastai

Now, your environment should look something like this:

(fastai)[email protected]:~$
  1. On your local machine, in a separate tab, set up port forwarding so that the port you're serving up the jupyter notebook to can be opened on a browser on your local machine.

For example, if I want to open up the notebooks on port 8888 on my local machine, and use port 6565 on the server:

ssh -N -f -L localhost:8888:localhost:6565 [email protected]

You can also forward the same ports (8888:8888) if you don't find that less confusing.

  1. Back on the cluster, inside the fast.ai directory:
jupyter notebook --no-browser --port 6565

The --no-browser flag prevents jupyter from trying to open a browser inside the remote server, and the port flag specifies the port that the server will use.

  1. In your browser, navigate to the link that the jupyter notebook gives you, but change the port that you used above. In my case, jupyter served up
 http://localhost:6565/?token=#####

so I would navigate to

http://localhost:8888/?token=#####

Notes

  • Sometimes your env won't be automatically populated by conda, so you might have find your env elsewhere to run it:
source activate anaconda/envs/fast-ai
  • Make sure you're using python3
  • To see if your port forwarding is running/working correctly:
ps aux | grep ssh

####Helpful resources: