How do you translate complex supervisor setups to gravity?

[pcm32]

Hi there,

we are trying to move our setup to 22.01 with gravity (the setup we use against the EBI cluster). Our setup depends partly on homebrew and the galaxy virtual env, so our previous supervisor jobs look like this:

  - name: "galaxy_handler-{{ galaxy_gxa_version }}"
    command: /usr/bin/bash -c "source ~/.bashrc && source {{ galaxy_venv_dir }}/bin/activate && python ./lib/galaxy/main.py -c {{ galaxy_cf_file_sv }} --server-name=handler_%(host_node_name)s_%(process_num)s --attach-to-pool job-handlers --attach-to-pool workflow-schedulers"
    state: present
    configuration: |
      directory: {{ galaxy_server_dir }}
      environment: TMPDIR="{{ shared_scratch | default(shared_base) }}/galaxy-{{ galaxy_gxa_version }}-wd/tmp"
      process_name: galaxy_handler%(process_num)s
      process_num: 1
      numprocs: {{ galaxy_handler_processes }}
      umask: 022
      autostart: true
      autorestart: true
      startsecs: 20
      ;user: {{ galaxy_user }}
      stopasgroup: true
      startretries: 1

is there a way with gravity to add all that configuration so that it ends up in the supervisor job? And also specific things that need to be sourced before the main galaxy python process is spinned (see command section)?

We also have other jobs being run by supervisor, like for instance nginx (which I guess gravity will configure), but we still do things like:

  - name: 'nginx'
    command: /usr/bin/bash -c "source ~/.bashrc && nginx -c {{ nginx_config_file }} -g 'daemon off;'"
    state: present
    configuration: |
      autostart: true
      autorestart: true
      stopasgroup: true
      stopsignal: QUIT

Should we avoid using gravity at all and configure web, handlers and auxiliary processes directly on supervisor? But then there is not much documentation on how to set up gunicorn for Galaxy without gravity.

Thanks!

[mvdbeek]

I'd start gravity from your supervisor config, so you only need to add a single unit there and you can use galaxyctl for regular maintenance, and that's where you can set your options as your doing now.

Other options are to include the units in gravity/supervisor/supervisord.conf.d/_default_.d/ in your main supervisor setup, like so

[include]
files = <path_to_gravity_state_dir_with_supervisor_configs>/*

Nothing's really changed wrt how you start galaxy, we've only replaced uwsgi with gunicorn, if you want to keep using the old way of specifying processes you can. Take a look at the generated supervisor configs (https://docs.galaxyproject.org/en/master/admin/scaling.html#gunicorn)