How to use GPUs on NLP Centre servers

The following servers have GPUs:

apollo - 11 GPUs, from 15 GB to 80 GB
akeso - 3 GPUs, from 15 GB to 45 GB
ephimetheus1, ephimetheus2, ephimetheus3 and ephimetheus4 - each 2 GPUs with 15 GB
turnus03 and turnus04 - each one GPU with 11 GB

Follow these actions:

ssh ... to one of the servers (e.g. ssh apollo). If you are not currently inside the faculty network, you must connect to the FI MUNI VPN first.
cd /nlp/projekty/... to your project directory
setup HuggingFace cache as detailed below
find available GPUs: run gpustat or nvidia-smi
export CUDA_DEVICE_ORDER=PCI_BUS_ID
choose a GPU with no running processes
limit your process to only selected GPU: export CUDA_VISIBLE_DEVICES=GID where GID is the ID of the selected GPU (the first column in the gpustat or nvidia-smi listing).

Contact nlpunix@aurora.fi.muni.cz if something does not work as expected.

Connecting to a GPU server

Download the VPN configuration for FI MUNI from here: https://www.fi.muni.cz/tech/unix/vpn.html
Use your faculty login x..... with your secondary password as the credentials

Run command (replace ... with the server name followed by a dot .):
```
ssh xsurname@....fi.muni.cz
```
Type in the secondary password again

Keep your home directory small, there are quotas active.

You need to change the download location of HuggingFace models, datasets, .... The default location is in your home and it can easily get full
prepare a cache directory in your project
```
mkdir /nlp/projekty/.../hf_cache
```

To change the HF cache location run:

export HF_HOME="/nlp/projekty/.../hf_cache"

To make this change permanent so you don't need to type it every time you enter the server, add it to your .bashrc file in your home directory

ssh ... # to the server
cd /nlp/projekty/...  # your project
virtualenv myenv
source myenv/bin/activate

This myenv activation must be done before any package is installed via pip/pip3 or the python/python3 command is run with your program.

Alternatively, you may run these commands with full path to your myenv and its bin subdirectory:

/nlp/projekty/.../myenv/bin/pip install ...
/nlp/projekty/.../myenv/bin/python my_program.py

The created environment may be dependent on the exact server setup and possibly may not work well at another server.

after activating your myenv environment install ipykernel

pip install ipykernel
python -m ipykernel install --user --name=myenv

You should get the output:

Installed kernelspec myenv in /home/user/.local/share/jupyter/kernels/myenv

jupyter notebook --no-browser --port 2222

The output should contain one line in the form

http://localhost:2222/?token=...

You will use it later, after setting up an SSH tunnel.

Open new second terminal and insert the following command, e.g. to apollo
```
ssh xsurname@apollo.fi.muni.cz -NL 2222:localhost:2222
```
The second port must match the port selected when starting the jupyter notebook

Point your browser to a the previously obtained address of the notebook http://localhost:2222?token=...
If you have used the same port numbers, as we do, you do not need to change anything in the URL. Otherwise, the port must match the first port in the ssh tunelling step.
If you have correctly copied the token, then you are logged in. Otherwise, to get the token again run:
```
ssh ... # the server
jupyter notebook list
```
Copy the jupyter notebook URL and open it in your local browser.

The notebook is opened with listing the directory of your project.