January 13, 2019
RC's community cluster is a great resource to take advantage of! I had a rough time getting the Fast AI course set up, so I wrote this rough guide to (hopefully) help others save some time.
Make an account on recurse.com/settings/cluster - this requires setting up and copying over an SSH key (instructions here)[https://help.github.com/articles/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent/].
SSH onto one of the GPU-enabled machines - I chose to use mercer (both Mercer and Crosby have powerful graphics cards. You can find more info on the machine specifics here)
Once on the machine, you should be in your own home directory. The command prompt should look something like this:
Install Conda so that you can create virtual environments to install software that won't affect others using the machine.
Install the fast.ai library (instructions from the course forum)
git clone https://github.com/fastai/fastai.git cd fastai conda env create -f environment.yml
Activate the environment for fast.ai
source activate fastai
On your local machine, in a separate tab, set up port forwarding so that the port you're serving up the jupyter notebook to can be opened on a browser on your local machine.
For example, if I want to open up the notebooks on port 8888 on my local machine, and use port 6565 on the server:
ssh -N -f -L localhost:8888:localhost:6565 email@example.com
Back on the cluster, inside the fast.ai directory:
jupyter notebook --no-browser --port 6565
The --no-browser flag prevents jupyter from trying to open a browser inside the remote server, and the port flag specifies the port that the server will use.
In your browser, navigate to the link that the jupyter notebook gives you, but change the port that you used above. In my case, jupyter served up
so I would navigate to
Sometimes your env won't be automatically populated by conda, so you might have find your env elsewhere to run it:
source activate anaconda/envs/fast-ai
To see if your port forwarding is running/working correctly:
ps aux | grep ssh