Adding new runners

Users usually don't want to run long-running or intensive tasks on their own computer. QA-Board will try to dispatch "batches" of runs to async task queues. Those will handle job management, fairness...

TODO

• Make it easy to add more runner types
• Add docs on how to create new runner types
• Enable tuning from the UI with runners (currently we assume we run from Samsung's infra......)

Available Runners

• Using the local computer with joblib
• LSF

Wishlist

• celery or python-rq since they are popular/easy/stable.
• ray since it's popular for ML.
• at least one cloud-based task runner from AWS/Azure/Google?
• kubernetes to be hype!

Adding a new runner

1. Think about the configuration parameters needed for your runner.

• Add CLI options for qa batch --myrunner-param in qaboard/qa.py, and merge them into job_options.
• Add the parameters to your project's qaboard.yaml:

runners:
  my-runner:
    parameter: value

2. Implement your runner in qaboard/runners/my_runner.py. It should implement a simple API:

from __future__ import annotations
from typing import Optional, List, Dict, Any
from .job import Job
from ..run import RunContext 

class MyRunner():
  type = "my-runner"

  def __init__(self, run_context : RunContext):
    self.run_context = run_context

  # only those 2 methods are called on Runners currently
  @staticmethod
  def start_jobs(jobs: List[Job], job_options: Optional[Dict[str, Any]] = None, blocking=True):
      raise NotImplementedError
  @staticmethod
  def stop_jobs(jobs: List[Job], job_options: Optional[Dict[str, Any]] = None):
      raise NotImplementedError

  def start(self, blocking=True, **kwargs):
    raise NotImplementedError

3. Register the runner like the existing ones in qaboard/runners/init.py
4. Happy debugging with qa batch --runner=my-runner!

Look at local.py and lsf.py for examples.

Tip: you can use environment variables like QA_RUNNER=my-runner, QA_MYRUNNER_PARAM...