How to use GPUs on the ML Department server

The ML Dept server hostname is akeso.fi.muni.cz.

Follow these actions:

1. ssh akeso
   If you are not currently inside the faculty network, you must connect to the FI MUNI VPN first.
2. cd /nlp/projekty/... or cd /cyb-trials/... to your project directory
3. setup HuggingFace cache as detailed below
4. find available GPUs: run gpustat or nvidia-smi
5. export CUDA_DEVICE_ORDER=PCI_BUS_ID
6. choose a GPU with no running processes
7. limit your process to only selected GPU: export CUDA_VISIBLE_DEVICES=GID where GID is the ID of the selected GPU (the first column in the gpustat or nvidia-smi listing).

Contact kat_server@nlp.fi.muni.cz if something does not work as expected.

Connecting to a GPU server

Connecting to FI MUNI VPN

• Download the VPN configuration for FI MUNI from here: ​https://www.fi.muni.cz/tech/unix/vpn.html
• Use your faculty login x..... with your secondary password as the credentials

Connect to the server

• Run command:

  ssh xsurname@akeso.fi.muni.cz
  

• Type in the secondary password again

Change the cache location of HuggingFace Data

Keep your home directory small, there are quotas active.

• find out your project directory PROJECT_DIR. It is located in one of the NFS shares, i.e. /nlp/projekty/... or /cyb-trials/.... In the following instructions, we assume an environment variable PROJECT_DIR is set to your actual project directory, e.g. via

  export PROJECT_DIR=/nlp/projekty/...  # or
  export PROJECT_DIR=/cyb-trials/...
  

• You need to change the download location of HuggingFace models, datasets, .... The default location is in your home and it can easily get full
• prepare a cache directory in your project

  mkdir $PROJECT_DIR/hf_cache
  

• To change the HF cache location run:

  export HF_HOME="$PROJECT_DIR/hf_cache"
  

• To make this change permanent so you don't need to type it every time you enter the server, add it to your .bashrc file in your home directory (including the setting for PROJECT_DIR).

Create virtual environment for your project

ssh akeso
cd $PROJECT_DIR
virtualenv myenv
source myenv/bin/activate

This myenv activation must be done before any package is installed via pip/pip3 or the python/python3 command is run with your program.

Alternatively, you may run these commands with full path to your myenv and its bin subdirectory:

$PROJECT_DIR/myenv/bin/pip install ...
$PROJECT_DIR/myenv/bin/python my_program.py

The created environment may be dependent on the exact server setup and possibly may not work well at another server.

Run jupyter notebook on a GPU server

Connect your virtual env with the Jupyter notebook

• after activating your myenv environment install ipykernel

  pip install ipykernel
  python -m ipykernel install --user --name=myenv
  

  You should get the output:

  Installed kernelspec myenv in /home/user/.local/share/jupyter/kernels/myenv
  

• To check all your connected venvs

  jupyter kernelspec list
  

Run Jupyter notebook on a personally selected port

jupyter notebook --no-browser --port 2222

The output should contain one line in the form

http://localhost:2222/?token=...

You will use it later, after setting up an SSH tunnel.

Create SSH Tunnelling

• Open new second terminal and insert the following command

  ssh xsurname@akeso.fi.muni.cz -NL 2222:localhost:2222
  

  The second port must match the port selected when starting the jupyter notebook

Open Jupyter notebook on the local machine

• Point your browser to a the previously obtained address of the notebook ​http://localhost:2222?token=...
  If you have used the same port numbers, as we do, you do not need to change anything in the URL. Otherwise, the port must match the first port in the ssh tunelling step.
• If you have correctly copied the token, then you are logged in. Otherwise, to get the token again run:

  ssh akeso
  jupyter notebook list
  

  Copy the jupyter notebook URL and open it in your local browser.

The notebook is opened with listing the directory of your project.

References used

1. SSH tunell: ​https://medium.com/@apbetahouse45/how-to-run-jupyter-notebooks-on-remote-server-part-1-ssh-a2be0232c533
2. FI MUNI VPN: ​https://www.fi.muni.cz/tech/unix/vpn.html
3. Jupyter notebook with virtualenv: ​https://janakiev.com/blog/jupyter-virtual-envs/
4. HuggingFace dataset location change: ​https://huggingface.co/docs/datasets/v1.12.0/cache.html