A mini-DSL for specifying constraints and affinities

[abhi18av]

For now, a direct mapping is great since it works with mental model of Nomad users as per the job definitions https://developer.hashicorp.com/nomad/docs/job-specification/constraint

Though for future, maybe this is something that maybe we can parse in a more user-friendly manner using a small DSL, which can have the added benefit that we can add more Nextflow level checks for the constraints being assigned correctly. What do you tihnk @jagedn , something worth tracking ?

Otherwise, in failing cases, Nomad layer would give an error code/message to the NF layer and we'd have to parse that.

Originally posted by @abhi18av in #40 (comment)

[jagedn]

we can try to define here a first approach to the DSL (?)

[abhi18av]

Mmm, actually the first point that comes to my mind is regarding the use of $ in the nextflow config, which already has a specific meaning.

Therefore if we want to make use of a mini-DSL for Nomad config then we should rely upon a different symbol?

[jagedn]

so maybe something similar as:

job {
constraints {
   distinctHosts   // will be translated to attribute:"distinct_hosts" , value:"true"
   attribute my_attr eq 3  // will be translated to "attribute": "${attribute.my_attr}", operator: "=", value="3" 
   attribute my_attr lt 3  // will be translated to "attribute": "${attribute.my_attr}", operator: "<", value="3"    
   meta my_attr eq 3  // will be translated to "attribute": "${meta.my_attr}", operator: "=", value="3" 
}
}

[abhi18av]

Works, but doesn't feel like "Nextflow"

Ideally, we can just use the maps to work with a whitelisted set of keys like node-attributes

Something like

job {



   constraints {

       //NOTE: STATIC list of whitelisted attributes for nodes
        node {
             unique = [ id: "node-id", name: "node-name"]
             class = "linux-64bit" 
             pool  = "custom-pool"
        }

        //NOTE: These are common (STATIC list of) node properties
        attr {
             cpu = [arch: "amd64", numcores: 5 ...]
             consul = [],
             os = [name: , version: , build: ]
             platform = [],
             kernel = [],
             unique = [],
             driver = [ docker: [], ... ]

        }
      
        //NOTE: These properties can be setup DYNAMICALLY on the node
        meta {
            group = "tb-research"
            customAttr = "<4"
            ...
        }

    }


   // NOTE: I"m NOT sure how to use these exactly, need to do more reading for their usage. Maybe, they don't need to be exposed at all.
   vars {
      nomad : [ port: ...],
      consul: [clientKey: ...],
      vault: [token: ...]
   }

}

We can leave the parsing of <,>,=,<=,>= to the Nomad orchestrator itself?

At the moment, attributes like datacenter and region we can just put as the overall global default i.e. nomad.datacenter etc.

What are your thoughts @jagedn?

[jagedn]

Understood, I will implement one of them and we'll see how it looks

[jagedn]

this is a first iteration to implement Node constraints into our DSL (I would like to do a little refactor but can be used to validate the idea)

in this file you can see how we can now specify a "global" constraint and/or process constraint (using directives)

master...constraints#diff-a10ebc84928f2879f5dc4b264dcafe0d3953ed526ad7716ed3a8f265f48895deR15

[abhi18av]

Marking this as closed since we already have a prototype for the DSL, we'll update as and when needed to suit the needs of the project :)