[FEA] Multi-Segment Configuration

[ryanolson]

The default configuration hard coded into Executor is to create one copy of each segment on partition 0 for each segment in the pipeline definition.

Resolution of this issue should:

• provide a configuration definition to the pipeline definition on the number of each types of segments that should be created based on the local knowledge of the number of partitions available on the specific machine.

This means we might need to first register the pipeline definition with the executor and return to the user a pipeline manager object. The pipeline manager object will have both the knowledge of the pipeline definition as well as the resources visible to the executor.

auto pipeline = make_pipeline();
// define segments

// configure options
auto options = make_options().enable_performance_opts();

// construct the executor and runtime resources
Executor executor(std::move(options));

// register the pipeline and receive back the configuration object
auto manager = executor.register_pipeline(std::move(pipeline));

// set properties on manager
auto device_count = manager->device_count();
manager->make_segment_assignment("seg_1", 0);
for(int i=0; i<device_count; i++)
{
    // make two copies of "seg_2" per cuda device
    manager->make_segment_assignment("seg_2", i);
    manager->make_segment_assignment("seg_2", i);
}

// execute pipeline
manager->start();

// await pipeline
manager->join();

Another way to do this is to annotate the segment definition with scaling options. One could annotate "seg_2" with a SCALE_BY_DEVICE with a value of 2. This is however more of a global configuration, where as the method above lets you perform the configuration on a machine by machine basis.

For the multi-segment, single machine, the two forms are equivalent. But in the multi-segment, multi-machine scenario, the annotations on the segment might be too restrictive, where applying a per machine configuration either by code (above) or by a file, would allow you to specialize each machine individually.