RC's community cluster is a great resource to take advantage of! I had a rough time getting the Fast AI course set up, so I wrote down some instructions (hopefully) help others save some time.
-
Make an account on recurse.com/settings/cluster - this requires setting up and copying over an SSH key instructions here.
-
SSH onto one of the GPU-enabled machines - I chose to use mercer (both Mercer and Crosby have powerful graphics cards). You can find more info about the specs for each machine here
ssh [email protected]
- Once on the machine, you should be in your own home directory. The command prompt should look something like this:
[email protected]:~$
-
Install Conda so that you can create virtual environments to install software that won't affect others using the machine.
-
Install the fast.ai library (instructions from the course forum)
git clone https://github.com/fastai/fastai.git
cd fastai
conda env create -f environment.yml
- Activate the environment for fast.ai
source activate fastai
Now, your environment should look something like this:
(fastai)[email protected]:~$
- On your local machine, in a separate tab, set up port forwarding so that the port you're serving up the jupyter notebook to can be opened on a browser on your local machine.
For example, if I want to open up the notebooks on port 8888 on my local machine, and use port 6565 on the server:
ssh -N -f -L localhost:8888:localhost:6565 [email protected]
You can also forward the same ports (8888:8888) if you don't find that less confusing.
- Back on the cluster, inside the fast.ai directory:
jupyter notebook --no-browser --port 6565
The --no-browser flag prevents jupyter from trying to open a browser inside the remote server, and the port flag specifies the port that the server will use.
- In your browser, navigate to the link that the jupyter notebook gives you, but change the port that you used above. In my case, jupyter served up
http://localhost:6565/?token=#####
so I would navigate to
http://localhost:8888/?token=#####
Notes
- Sometimes your env won't be automatically populated by conda, so you might have find your env elsewhere to run it:
source activate anaconda/envs/fast-ai
- Make sure you're using python3
- To see if your port forwarding is running/working correctly:
ps aux | grep ssh
####Helpful resources: