As you've probably seen in the Beginner's Guide, streams start with a YAML template file describing how all of the pieces of a stream (or streams) fit together.
Larger template files may seem intimidating at first. This document is to help describe the basics of template files so you can start crafting your own.
We're referencing a few different types of stream classes here but we won't go into too much detail about how they work in this document. Please refer to the Beginner's Guide for some basics about different stream types.
Let's start with a very basic template file.
_type: Tumblr\StreamBuilder\Streams\NullStream
That's it! It doesn't do much, but that's an entire template file which uses the built-in StreamBuilder NullStream class (which never returns any stream elements). You could load this template into an application and it would 100% function.
In this case, all we've done is define the type (with the _type: property) of the stream. The type of a stream will always be a PHP class. It may be one of the built-in classes or it may be a custom class you've written yourself.
A more realistic example which uses a custom stream class might be:
_type: MyApplication\Streams\MyStream
YAML files handle hierarchy through indentation. That means, as we build on our template file, we will keep indenting our primary stream so it gets pushed further to the right.
Let's update our initial template to have a filter as an example:
_type: Tumblr\StreamBuilder\Streams\FilteredStream
stream:
_type: MyApplication\Streams\MyStream
stream_filter:
_type: MyApplication\StreamFilters\MyFilter
Notice how we've essentially wrapped our original NullStream so it's inside the FilteredStream.
The primary _type property of our file is now the FilteredStream and our NullStream is inside the filtered stream under the stream: property. The FilteredStream also has an additional stream_filter property which defines the class we're using to handle the actual filtering.
NOTE: It's important to see that any template file property that's defining a stream class must have a _type property.
Let's digress for just a moment to talk about template file properties that are available for different stream types.
Most stream classes support varying properties, some required, some optional. Most of the required properties are things like _type, stream, inner, etc.
At this point, the easiest way to see the options available is to look at the from_template method of the class for the stream type.
Let's look at the FilteredStream as an example:
public static function from_template(StreamContext $context): self {
// ...
return new self(
$stream,
$filter,
$context->get_current_identity(),
$context->get_optional_property('retry_count'),
$context->get_optional_property('overfetch_ratio'),
$skip_filters,
$context->get_optional_property('slice_result', true)
);
}
The basic rule of thumb is that constructor parameters that have default values (like retry_count, overfetch_ratio, skip_filters, slice_result) are all optional. The other parameters (like inner) are required.
Let's expand our template even more by adding an Injector stream.
_type: Tumblr\StreamBuilder\Streams\InjectedStream
injector:
_type: Tumblr\StreamBuilder\StreamInjectors\GeneralStreamInjector
allocator:
_type: Tumblr\StreamBuilder\InjectionAllocators\GlobalFixedInjectionAllocator
positions: [0, 5, 10]
inner:
_type: MyApplication\StreamInjectors\MyAdStream
stream:
_type: Tumblr\StreamBuilder\Streams\FilteredStream
stream:
_type: MyApplication\Streams\MyStream
stream_filter:
_type: MyApplication\StreamFilters\MyFilter
You can see that the primary _type of our template is an InjectedStream. We now have an injector property which defines the _type of the injector that we're using along with some required properties for the Injector.
First we have the injector allocator which determines where in the stream, we'll be injecting our data. In this case, we will always insert our data at positions 0, 5, and 10.
Last, we have the injector inner which defines the stream class that returns the actual data we're injecting. In this example, we have a pretend stream for injecting ads into our primary stream.
After the injector, we have the stream that we're injecting data into which is the same filtered stream we had before. Notice how that portion is unchanged except that the indentation has shifted to the right.
We could keep nesting and combining different types streams all day long, but hopefully one last layer will be a good enough example for learning purposes.
For the last bit, we'll add a Ranked stream to our existing setup.
_type: Tumblr\StreamBuilder\Streams\InjectedStream
injector:
_type: Tumblr\StreamBuilder\StreamInjectors\GeneralStreamInjector
allocator:
_type: Tumblr\StreamBuilder\InjectionAllocators\GlobalFixedInjectionAllocator
positions: [0, 5, 10]
inner:
_type: MyApplication\StreamInjectors\MyAdStream
stream:
_type: Tumblr\StreamBuilder\Streams\RankedStream
ranker:
_type: MyApplication\StreamRankers\MyRanker
inner:
_type: Tumblr\StreamBuilder\Streams\FilteredStream
stream:
_type: MyApplication\Streams\MyStream
stream_filter:
_type: MyApplication\StreamFilters\MyFilter
As you can see, the injector portion has stayed the same but we've changed the top-level type of the stream we're injecting into.
Now, instead of that top-level type being a FilteredStream, we now have a RankedStream (as defined by the _type). We've also added a ranker property which species the class that will be performing the actual ranking logic on our stream elements.
The Ranker stream also has an inner property which contains our filtered stream. Notice how the filtered stream (and our original NulLStream) are, once again, unchanged except for being indented further.
As you can see, StreamBuilder template files are extremely powerful. We can do very elaborate combinations of streams just using the various, built-in StreamBuilder classes.
Hopefully having some examples will make it easier to create your own templates.