Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add guide for vscode with slurm #67

Merged
merged 31 commits into from
Nov 21, 2024
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
f180819
Add guide for vscode with slurm
lauraporta Nov 12, 2024
3800fa8
Include in toctree
lauraporta Nov 12, 2024
d2a644d
Linting
lauraporta Nov 12, 2024
1aa5d5a
Add link to linkcheck ignore
lauraporta Nov 12, 2024
e04f9eb
Apply suggestions from code review
lauraporta Nov 13, 2024
3b86f7b
Apply suggestions on code snippets
lauraporta Nov 13, 2024
49ad855
Add info on why we need autentication and remove section on extensions
lauraporta Nov 13, 2024
73928bf
Add note on UCL name
lauraporta Nov 18, 2024
25b8a94
Add suggestions from Niko and Sofia
lauraporta Nov 21, 2024
37c8511
Add options for local vscode usage
lauraporta Nov 21, 2024
d73a36a
Add reconnecting to job suggestion
lauraporta Nov 21, 2024
7ba2b82
Expand on additional servers
lauraporta Nov 21, 2024
a4fe6be
Fix aesteics bugs 🐛✨
lauraporta Nov 21, 2024
3a6abed
Linting
lauraporta Nov 21, 2024
b7256b0
Update docs/source/programming/vscode-with-slurm-job.md
lauraporta Nov 21, 2024
42c374e
Update docs/source/programming/vscode-with-slurm-job.md
lauraporta Nov 21, 2024
01504ee
Change example to dash plotly
lauraporta Nov 21, 2024
21ddaec
Change node name description
lauraporta Nov 21, 2024
825e7ef
Further wording fixes
lauraporta Nov 21, 2024
412b2bb
Update link
lauraporta Nov 21, 2024
fa7d9a0
Update docs/source/programming/vscode-with-slurm-job.md
lauraporta Nov 21, 2024
10b316b
Update docs/source/programming/vscode-with-slurm-job.md
lauraporta Nov 21, 2024
33a305d
Update docs/source/programming/vscode-with-slurm-job.md
lauraporta Nov 21, 2024
bd2e426
Update docs/source/programming/vscode-with-slurm-job.md
lauraporta Nov 21, 2024
ef64ffa
Update docs/source/programming/vscode-with-slurm-job.md
lauraporta Nov 21, 2024
37a71b0
Update docs/source/programming/vscode-with-slurm-job.md
lauraporta Nov 21, 2024
ba458c5
Update docs/source/programming/vscode-with-slurm-job.md
lauraporta Nov 21, 2024
d93e5fc
Update docs/source/programming/vscode-with-slurm-job.md
lauraporta Nov 21, 2024
f98307b
Update docs/source/programming/vscode-with-slurm-job.md
lauraporta Nov 21, 2024
2a939ed
Add further suggestion on title
lauraporta Nov 21, 2024
3a7c9f3
Add clarification on SLURM
lauraporta Nov 21, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions docs/source/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,8 @@
"https://support.zadarastorage.com",
"https://stackoverflow.com/questions/24447047/remove-git-annex-repository-from-file-tree",
"https://joinup.ec.europa.eu/collection/eupl/solution/joinup-licensing-assistant/jla-compatibility-checker",
"http://ccdb.ucsd.edu/home",
"https://opensource.org/licenses/BSD-3-Clause",
]
# The linkcheck builder will skip verifying that anchors exist when checking
# these URLs (e.g. because they are generated by JavaScript).
Expand Down
1 change: 1 addition & 0 deletions docs/source/programming/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ Small tips and tricks that do not warrant a long-form guide can be found in the
SLURM-arguments
SSH-SWC-cluster
SSH-vscode
vscode-with-slurm-job
Mount-ceph-ubuntu
Mount-ceph-ubuntu-temp
Cookiecutter-cruft
Expand Down
40 changes: 40 additions & 0 deletions docs/source/programming/vscode-with-slurm-job.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
# Using VSCode with SLURM-Managed Jobs on the SWC HPC Cluster

This guide explains how to set up and use VSCode within a SLURM-managed job on the SWC HPC cluster, offering a solution for users who require fast access to Ceph or WinStor and substantial computational resources, when a managed machine is not available.

This solution is proven to be easy to set up and constrained by the SLURM job's resource limits, ensuring that users can work efficiently without overloading the system.


## Instructions

First, connect to the bastion node by running:

```console
ssh [email protected]
```

Once connected, request an interactive job via SLURM to access a compute node:

```console
srun -p fast --pty bash -i
```

In this example, `-p fast` requests the fast partition, with default time settings, though you may adjust this according to your needs.

After connecting to a compute node, initiate VSCode Code Tunnel by typing:

```console
code tunnel
```

A URL will appear in the terminal, `https://github.com/login/device`. Follow this link, log in with your GitHub credentials, and enter the provided PIN to authorize access. After completing this step, a second link will appear in the terminal (e.g., `https://vscode.dev/tunnel/gpu-XXXX`), which you can follow to access your VSCode session running directly on the HPC compute node.

When you’re finished, simply exit the SLURM session to close the VSCode tunnel and release resources.

### Important Note on Extensions

When using VSCode in this setup, exercise caution with extensions. Some, like the Typescript and Javascript extension, are not recommended by IT due to potential performance issues. Install only essential extensions to maintain system efficiency.

## Additional Benefits of Code Tunnel

One advantage of using VSCode's code tunnel is that it forwards any HTTP servers launched from the same node, such as MLflow or Jupyter Notebook servers. When these are initiated, VSCode will notify you with a link that you can follow to access the server's UI directly.