Removing support for different messaging formats?

[dominikriemer]

Hi,
while looking at ways for further simplifying our data model, I was asking myself if we should remove support for different transport formats.
Currently, pipeline elements can define their own supported transport formats and StreamPipes currently supports the formats defined in the streampipes-dataformat-xmodules.
I don't think there is currently much value in letting users define the internally used data format. So I suggest to simplify this and only support a single data format. This would also ease things in the future if we decide to go for another format such as protobufs to better handle our various client implementations.

What do you think - are there any arguments to keep the existing approach?

[tenthe]

Hi @dominikriemer,

thanks a lot for starting the discussion.
I am in favor of this idea. Simplifying the model by supporting a single data format will reduce complexity without sacrificing any essential functionality.

Cheers,
Philipp

[bossenti]

Hi,

I'm also in favor of simplifying StreamPipes to that extent. I currently can't think of any real downside for end users by supporting only a single messaging format :)

Do you already have a preference for which messaging format to keep?

[dominikriemer]

I'd suggest to keep plain JSON for now since it's the default format we've used in the past. In the mid-term future I'd personally like to switch over to some binary representation.