[cudax] Make stream in async_buffer optional

[pciolkosz]

We store a stream in async_buffer to express dependencies the buffer should respect when its destructor is run. But sometimes it would be nice to just say a buffer no longer has any dependencies. For example:

      auto buf = cudax::make_async_buffer(stream, resource, ...);
      // use the buffer for some work in stream

      stream.sync();
      // if buffer is not used for other stream ordered work beyond this point
     //   there is no need to keep stream as its dependency

In the above case stream might get destroyed before buf or there might be some work inserted into the stream that is not related to it and it would unnecessarily delay the deallocation.
For these cases this PR changes the type stored in the buffer from stream_ref to cuda::std::optional<stream_ref> along with the getter and setter for it. This way in the above case user can call buf.set_stream(cuda::std::nullopt) and buffer destructor will call resource.deallocate_sync() instead of resource.deallocate() in that case.
The side effect of that change is that whenever user wants to query the stream from a buffer they will need to remember to unpack it from an optional, but I think it's fine.

Currently all ways to create an async_buffer take a stream that gets stored in the buffer. We could provide buffer construction API with no_init argument that does not take a stream and the buffer starts with the stream set to nullopt, but this need some thought first

[davebayer]

Suggested change

	_CCCL_HIDE_FROM_ABI async_buffer(const async_buffer& __other)
	: __buf_(__other.memory_resource(), __other.stream(), __other.size())
	_CCCL_HIDE_FROM_ABI explicit async_buffer(const async_buffer& __other)
	: __buf_(__other.memory_resource(), __other.stream(), __other.size())

I would like to make this constructor explicit, to avoid accidental coppies

[davebayer]

What's the point in taking the stream as optinal?

[jrhemstad]

I had the same question as @davebayer here. Do we even need to literally use an optional<T> here at all? Can't we just use a sentinel value of cudaStream_t{0} ?

[pciolkosz]

The problem is cudaStream_t{0} is a valid stream. We have ~0 casted to cudaStream_t that is recognized as invalid stream, but I think something like buffer.set_stream(cuda::invalid_stream) seems more convoluted than optional.
For me buffer.set_stream(cuda::std::nullopt) was the most intuitive way to say the buffer should have no stream dependency. Especially since we already return optional from buffer.stream(). Maybe there is some better way to express it, but I don't think it should be some special cudaStream_t value or a special stream.

[github-actions]

🥳 CI Workflow Results

🟩 Finished in 59m 07s: Pass: 100%/42 | Total: 4h 03m | Max: 27m 46s | Hits: 96%/20774

See results here.

[pciolkosz]

After some discussions I am closing this one. In the future if we still want similar functionality we will use invalid_stream instead