Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Site specific drones, matching jobs requesting that site #344

Open
rodwalker opened this issue Apr 17, 2024 · 2 comments
Open

Site specific drones, matching jobs requesting that site #344

rodwalker opened this issue Apr 17, 2024 · 2 comments

Comments

@rodwalker
Copy link

As discussed in Freiburg, could we have jobs submitted to the CE with say
+site='LRZ-LMU'
only go to drones running on this site?
This would allow us to have multiple Panda queues, in the same overlayBS, each using local storage for stage-in/out.

@rodwalker
Copy link
Author

This would let us use the data-colocation capabilities of Panda. Maybe slightly more generic would be to accept some information about the input data location and size, then use this in the scheduling decision.
For example, we could have a topology saying DESY-HH is close to GoeGrid_NHR and the latter operates an XCache with some chance of the data being cached. Then build a Rank to prefer HH but it can run at NHR.

@mschnepf
Copy link
Member

mschnepf commented Aug 9, 2024

With scheduling decision, do you mean on the ATLAS level that they send pilots to a specific subsite, or do you mean on the Overlay Batch System (OBS) level?
Send jobs to a specific subsite needs to be handled by the OBS. TARDIS will just react to the utilization of the subsites which is a result of the decision made by OBS.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants