architecture

Architecture

Contents

1. Basic Terms
   
2. Components
   2.1. Storage Driver
   2.2. Load Generator
   2.3. Load Step Context
   2.4. Metrics Manager
   2.5. After-test data aggregation
   
3. Concurrency
   3.1. Service Tasks
   3.2. Tuning
   

1. Basic Terms

• Storage

  A storage which performance is been measured in the test. Currently several types of storages are supported.

• Item

  The unit to account the performance rates. Maybe a file, object, directory, bucket, etc. An Item has the identifier property (name).

• Data Item

  An Item with a data payload. Any Data Item has the corresponding size property.

• Item Input

  The readable source of the items. This may be a CSV file, a binary stream, a collection or a bucket listing.

• Item Output

  The writable destination for the items. This may be a CSV file, a binary stream or a collection.

• Load Operation

  A load operation is a item linked with a particular operation type (write/read/delete/etc). Also, any operation has the state and the execution result as an extension of this state.

• Load Step

  Load step is an unit of test execution flow.

• Load Step Context

  Load step is an unit of metrics reporting. A Load Step may include several Load Step Contexts.

• Storage Driver

  Storage driver is a load operations executer. The storage driver is used by Mongoose to interact with the given storage. An implementation of Storage driver depends on the storage type and used as mongoose extention (plugin).

• Scenario

  A set of load steps combined and organized using flow elements provided by a scripting engine which supports JSR-223. Mongoose invokes the default scenario if no custom scenario is specified. The default scenario just runs the single linear load step.

• Entry Node

  In the distributed mode the node which is used to execute a scenario is a entry node

• Additional Node

  In the distributed mode, the additional nodes are used to execute a scenario steps together with entry node

2. Components

2.1. Storage Driver

Storage driver executes the load operations generated by Load Generators. It translates the Mongoose's abstract operations into the actual I/O requests and executes them. The basic property is the concurrency level and storage client configuration. The functionality includes:

• Low-level implementation of the operations execution functionality
• Rate limit related things
• Callbacks for the completed operations

The driver is a "layer" between the Load generator and the tested Storage. Since storages of different types have different interaction protocols, then the implementation of Storage driver depends on the storage type. Therefore, the base image of the mongoose only has the implementation of the Mock Storage driver which does nothing actually and useful only for demo/testing purposes.

Storage mock in a mongoose is called Dummy Mock. It is set by a parameter --storage-driver-type=dummy-mock.

To use mongoose with real Storage, use Storage driver extentions.

2.2. Load Generator

Load Generator is a component which generates the operations from the items got from the input. Many storage drivers may be associated with a load generator. The basic properties are:

• Origin Index (all operations generated share the same origin index)
• Operation type (create/read/etc)
• Shared rate throttle
• Shared weight throttle
• Storage drivers list
• Storage drivers balancer

2.3. Load Step Context

A load step context is an unit of test step control. Functionality:

• Execution control (timeouts handling, shutdown invocations, etc)

2.4. Metrics Manager

Metrics aggregation and representation. The component is a singleton which was differentiated from the Load Step Context component. Many load step contexts may be associated with the single metrics manager.

2.5. After-test data aggregation

This chapter describes how data is aggregated after Mongoose has finished a load step. In standalone mode there is nothing to be done as we synchronously write in the file specified by --output-fiie as we go. But with distributed mode we aggregate data from local temporary files from worker nodes as soon as workload is done. It happens when --item-output-file or --output-metrics-trace-persist are specified (or both). We read it by ~16Mb chunks (if there is enough data) and synchronously put it in the aggregated file on the entry node. But as we aggregate data in parallel from workers, each chunk can be a bit less than 16Mb so that chunk finishes at the end of line, so we don't mix lines from different workers.

Since version 4.3.0 some of the metrics are also only aggregated after the test step. To avoid a continuous stream of metrics passed over to entry node the timing metrics (latency and duration) are only aggregated after the test step. While the test is running only the mean value is calculated and passed over. After the step is done remote temporary files are aggregated into local entry node temporary files. Then they are deleted on the remote workers. After that on the entry node the files are first sorted and then merged. Mongoose takes the specified quantile values from the sorted values and then deletes files on the entry node.

3. Concurrency

Mongoose uses fixed count of threads to execute its tasks.

1. Calculate the asynchronously changing values defined by a pattern in the configuration
2. Generate the operations
3. Distribute the generated operations among the storage drivers uniformly
4. Storage drivers dispatch the incoming operations
5. Storage drivers perform the file/socket I/O
6. Load step context collects the actual concurrency measurements from the storage drivers
7. Load step context collects the completed operations results from the storage drivers
8. Refresh the registered metrics and their snapshots

The count of the tasks which are required to be executed concurrently may be large (for example in a case of distributed mode with many additional nodes). The traditional thread-per-task approach is inefficient if the count of the threads is much higher than the count of the available CPU cores. The fibers are used in Mongoose to execute the required work efficiently.

Mongoose tasks may be divided into two types: service tasks and load operations. So there are two different fixed thread pools for these types of tasks.

3.1. Service Tasks

Each Mongoose process shares the global service tasks executor (FibersExecutor instance). By default the count of the service tasks executor's threads is equal to the count of the available CPU cores. A user may set the different count of these threads using the configuration option load-service-threads.

3.2. Tuning

Using more threads for "calculations" causes higher throughput (operations count per unit of time), higher responsiveness (e.g. lower latency). However using more threads for I/O causes higher observable bandwidth (transferred bytes count per unit of time), which is especially obvious while performing I/O on the large data items.