Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

_parse_slurm_node_list leads to incorrect node names if no brackets are in node name #495

Open
Leonngm opened this issue Jan 27, 2025 · 0 comments

Comments

@Leonngm
Copy link

Leonngm commented Jan 27, 2025

I realized that sometimes the number of nodes does not match the number of requested nodes. This is due to the function _parse_slurm_node_list failing, if the nodename does not contain any brackets.

Code to reproduce:

from dinov2.distributed import _parse_slurm_node_list

input = "node1g[0010,0011],node2g0005,node3g0007"
output = _parse_slurm_node_list(input)

# output: ['node1g0010', 'node1g0011', 'node2g0005,node3g007']
# len(output): 3
# expected output:  ['node1g0010', 'node1g0011', 'node2g0005', 'node3g007']
# len(expected output): 4

Can be fixed by adding:

    flattened_nodes = []
    for node in nodes:
        flattened_nodes.extend(node.split(","))
        
    return flattened_nodes

Fix using regular expressions thanks to @NikolasSchmitz:

def parse_slurm_node_list(s: str) -> List[str]:
    nodes = []
    p = re.compile(r"([^\[,]+)(?:\[([^\]]+)\])?,?")
    for m in p.finditer(s):
        prefix = m.group(1)
        suffixes = m.group(2)
        if suffixes:
            for suffix in suffixes.split(","):
                span = suffix.split("-")
                if len(span) == 1:
                    nodes.append(prefix + suffix)
                else:
                    width = len(span[0])
                    start, end = int(span[0]), int(span[1]) + 1
                    nodes.extend([prefix + f"{i:0{width}}" for i in range(start, end)])
        else:
            nodes.append(prefix)  # If no brackets, add the prefix as is
    return nodes
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant