SGE slots_per_host doesn't work

[kevinbache]

Setting slots_per_host=1 doesn't seem to work. sshing in the master node and running qconf -mattr exechost complex_values slots=1 <hostname> does, however. Would you be interested in a pull request that changed the sge plugin to use this mechanism?

[cancan101]

What do you mean it doesn't work? I use this setting and it works as I expect

[kevinbache]

I must be using some combination of settings that's invalidating it, then. With the config file below, the number of slots on each node remains at the default and they each suck up multiple jobs at once. Running qconf -se node001 on the master doesn't list any settings about slots either. I mentioned running qconf -mattr exechost complex_values slots=1 node001 from the master node because this added the slot setting to gconf and kept the worker nodes from consuming more than one job at once.

####################################
## StarCluster Configuration File ##
####################################
[global]
DEFAULT_TEMPLATE=kevin-template-west-1

[aws info]
AWS_ACCESS_KEY_ID = ######################
AWS_SECRET_ACCESS_KEY = ####################################
AWS_USER_ID= ####################
AWS_REGION_NAME = us-west-1
AWS_REGION_HOST = ec2.us-west-1.amazonaws.com

[key kevin-macbook]
KEY_LOCATION=~/.ssh/################

[vol kevin-vol-west-1]
volume_id = vol-#########
mount_path = /storage

[cluster kevin-template-west-1]
KEYNAME = kevin-macbook
CLUSTER_SIZE = 2
CLUSTER_USER = sgeadmin
CLUSTER_SHELL = bash
NODE_IMAGE_ID = ami-########
NODE_INSTANCE_TYPE = g2.2xlarge
DISABLE_QUEUE = True
PLUGINS = sge, git
MASTER_INSTANCE_TYPE = c3.large
SPOT_BID = 0.25
FORCE_SPOT_MASTER = True
VOLUMES = kevin-vol-west-1

[plugin sge]
setup_class = starcluster.plugins.sge.SGEPlugin
master_is_exec_host=False
slots_per_host=1

[plugin git]
setup_class = starcluster.plugins.git.GitCredentials
private_github_ssh_key_location = /Users/kevin/.ssh/########
git_user_name = Kevin Bache
git_user_email = [email protected]

[cancan101]

I have very similar settings and I do see that the number of slots is set to 1 on each of my hosts.
Why don't you add some debugging around:

StarCluster/starcluster/plugins/sge.py

Line 100 in 286dc61

num_slots = self.slots_per_host

also, I don't think you can change the config after the cluster has been started.

[kevinbache]

Somehow this is working for me now. With apologies, I'm going to close this issue as irreproducible.

[cancan101]

What is the git plugin? If it is what I think it is, take a look at #468

[kevinbache]

It's a little plugin I threw together to forward my git credentials to a cluster (ssh agent forwarding, set remote username/email) and pull some repos on cluster startup. The code is in my StarCluster fork. https://github.com/kevinbache/StarCluster/blob/develop/starcluster/plugins/git.py