Skip to content

Execution

Jump to bottom
Tim Stair edited this page Apr 23, 2020 · 1 revision

Step 3: Execution

Priority Operation Processing: API: ResourcePool

Overview

A Resource Pool is a Kubernetes cluster that pulls for full workflows called Agendas. A Resource Pool may have one or more queues it pulls from filtered by Insights.

images/download/attachments/226435076/Fission-ResourcePool-2.jpg

Features

Agenda Puller

  • The puller runs as a Kubernetes pod that pulls for work by calling Agenda Service getAgenda

  • Creates On-Demand Kubernetes Pod for Agenda Execution

Agenda Executor

  • Executes all operations in Agenda by spinning up resources dynamically

  • Metrics / Alerting

  • Progress reporting

  • Parallel processing

  • Node Graph / DAG model processing

Handler

  • Executor for a single Agenda Operation

  • Metrics / Alerting

  • API owner

  • Progress reporting

Technologies

  • Kubernetes

  • Kubernetes Annotations

  • Docker

  • Bananas

  • Graphite

Priority Operation Processing

  1. Submission
  2. Scheduling
  3. Execution
    the ResourcePool

Data Object API

Agenda
the workflow
Agenda Template
the workflow definition
Customer
Insight
the scheduling queue definition
Operation Progress
the state of the running Agenda operations
Progress
the state of the running Agendas
ResourcePool
the processing resources

Service Agenda

Agenda Service
the workflow submission
Progress Service
rolled up agenda progress summary
ResourcePool Service
getting work and updating progress

Timed Processes

AgendaReclaimer
restarting stuck Agendas
AgendaRetry
retrying failed Agendas
DataObjectReaper
reaping expired data objects
PodReaper
reaping stuck Kubernetes pods

Installation

Install

Development

DevKubernetesSetup
RunLocalExecution
- RunWithMiniKube

Demo / Examples

SampleAuthorizer
SampleHandler

Clone this wiki locally