[RLlib] Composable (SingleAgent)RLModules

[0Pinky0]

Description

Is there any foreseeable plan on making (SingleAgent)RLModules composable?
For now I am trying to build up a training pipeline where a series of RLModules are composed into a MultiRLModules in order to serve as a single-agent policy network, but it seems default algorithm pipelines treat submodules in MultiRLModules as individual policies in multi-agent settings.
Similar idea was found in https://github.com/ray-project/ray/blob/master/rllib/examples/rl_modules/classes/vpg_using_shared_encoder_rlm.py, and such design may grant users more flexibility to perform model-specific operation like model surgery mentioned in OpenAI Five.

Use case

No response