Skip to content

Understanding Mitogen for Ansible

Jump to bottom
dw edited this page Feb 11, 2019 · 1 revision

Note: work in progress

This page provides a general overview of how Mitogen for Ansible 'actually works', without first having to know much background knowledge.

Classic Ansible

Ansible operates by running a 'strategy plug-in' from within its main process, after loading configuration data, inventory and playbooks.

The strategy plug-in is responsible for taking a playbook and turning it into actions somehow. In most circumstances this happens by the strategy forking the process (using the multiprocessing module), creating what is known as a WorkerProcess for each task, and waiting for those workers to crash or report results via a multiprocessing.Queue (which is part-implemented by a background thread running in the main process).

Within the WorkerProcess the task executor (lib/ansible/executor/) runs. It is responsible for constructing the 'action plug-in' corresponding to whatever the playbook contains, and a 'connection plug-in' over which the action will communicate with the target.

In the majority of cases when executing a regular Ansible module, the 'normal' action-plugin is used, possibly/probably in combination with the SSH plug-in.

Now the task executor runs the action plug-in. In the case of the 'normal' plug-in, it converts the module to be executed into a large ZIP file, and then (depending on configuration) uses some combination of ssh/scp commands to create temporary directories on the remote machine, copy or stream the ZIP file to the remote, execute it, and finally clean up.

In summary, given a playbook with a single task run on 2 hosts, using the linear strategy, something like this happens:

  1. Ansible starts up, parses arguments, loads inventory, strategy plug-in.
  2. Strategy runs, forks two WorkerProcesses, one for each host that will run the task.
  3. Strategy "sleeps" (actually it busy-loops) waiting for all the WorkerProcesses to complete.
  4. Each WorkerProcess loads the normal plug-in and the connection plug-in.
  5. The normal plug-in's run() turns the module to be run into a ZIP file, forks repeatedly to execute SSH/scp commands
  6. SSH is normally configured to use connection multiplexing, and so the first SSH command causes another fork and startup of a background daemon process to hold the connection open to the remote machine independent of any SSH invocation
Clone this wiki locally