Running Python codes on RCC Midway 3 (with GPU)

[JunsolKim]

1. Access Midway 3 via Thinlinc (remote access tool; https://midway3.rcc.uchicago.edu/). Install ThinLinc Desktop client if you want a more stable connection to Midway 3. Here, you can open your terminal.

2. Move your codes and files to Midway 3. (See "Data Transfer" document in the user guide: https://mdw3.rcc.uchicago.edu/user-guide/).

3. In the ThinLinc terminal (Midway 3 terminal), run the following command. This allows you to run python codes on the Midway. Install the required packages (e.g., transformers) using pip.

module load python
source activate pytorch-gpu-1.2-cuda-10.0
pip install [package]

5. To use GPU (cuda), you need to start the sinteractive session using the following command. Change the RAM memory and the number of cpu cores as you need. After the sinteractive session is started, follow (3) to run the python code. (Note that you cannot use pip command in the sinteractive session. You need to install all required packages before you get into sinteractive session.)

sinteractive --account=soci40133 --partition=gpu --mem=100G --time=02:00:00 --gres=gpu:1 --cpus-per-task=4
module load python
module load cuda
source activate pytorch-gpu-1.2-cuda-10.0
python [.py code you want to run]