Skip to content

SlurmManager: Add the custom_worker_flag option, but don't add it to the public API #74

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

DilumAluthge
Copy link
Member

@DilumAluthge DilumAluthge commented Mar 24, 2025

Description

I have a use case where, instead of starting each remote worker with julia --worker, I need to start the remote workers with julia -e 'my_custom_code()'.

This PR modifies SlurmManager to have a custom_worker_flag option. When custom_worker_flag is specified, it will replace --worker. When custom_worker_flag is not specified (which is the default), it falls back to the current behavior.

Example usage

using Distributed: addprocs
using SlurmClusterManager: SlurmManager

julia_code = """
    my_custom_code()
"""

addprocs(SlurmManager(); custom_worker_flag = `-e "$(julia_code)"`) 

This will start each remote worker with julia -e 'my_custom_code()'.

Public API vs private API

This PR does not add the new custom_worker_flag option to the public API. This allows us to change (or remove) it in the future.

@DilumAluthge DilumAluthge changed the title SSHManager: Add the custom_worker_flag option, but don't add it to the public API SlurmManager: Add the custom_worker_flag option, but don't add it to the public API Mar 24, 2025
@DilumAluthge DilumAluthge force-pushed the dpa/custom_worker_flag branch from 0ebf6a6 to a3027cc Compare March 24, 2025 01:01
Copy link

codecov bot commented Mar 24, 2025

Codecov Report

Attention: Patch coverage is 70.58824% with 5 lines in your changes missing coverage. Please review.

Project coverage is 87.03%. Comparing base (e49016d) to head (674f016).

Files with missing lines Patch % Lines
src/slurmmanager.jl 70.58% 5 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master      #74      +/-   ##
==========================================
- Coverage   90.42%   87.03%   -3.39%     
==========================================
  Files           2        2              
  Lines          94      108      +14     
==========================================
+ Hits           85       94       +9     
- Misses          9       14       +5     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@DilumAluthge
Copy link
Member Author

This is not ready to merge yet, but I figure it can't hurt to get an early round of reviews.

@DilumAluthge DilumAluthge force-pushed the dpa/custom_worker_flag branch from 6cd6fae to 36469c6 Compare April 29, 2025 08:45
@DilumAluthge DilumAluthge force-pushed the dpa/custom_worker_flag branch from 36469c6 to 674f016 Compare May 1, 2025 02:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant