OpenTelemetry Collector distributions

[atoulme]

The OpenTelemetry Collector is currently distributed by the OpenTelemetry project through two distributions, core and contrib.

In numerous discussions with the community, maintainers and a few SIG meetings, it has emerged that those distributions
emerged organically. We discuss them in detail below.

This OTEP aims to clarify what distributions the OpenTelemetry project should distribute. The OTEP will elicit all requirements
and dependencies attached to the current distributions, discuss the tooling in place, and propose a way forward.

This OTEP also aims to become the center of discussions for the requirements and needs of the community, rather than
seeing this issue being rehashed.

[jpkrohling]

Interestingly, I talked to @dyladan just yesterday about this, and we might want to come up with a broader definition of distribution that would also work for the SDKs. I can't speak for Daniel, but I believe this OTEP here would be good to be discussed after the OTEP I'm creating right now on the maturity levels and component stability indicators for OTel. One idea that we had was that SIGs/projects (like the Collector and SDKs) would have to produce a distribution containing their "tier-1" components (to be defined by the OTEP I'm drafting).

If you have an urgent need to make a definition of what a Collector distribution is, we can move forward with this OTEP here, but if it's not that urgent, I would request to wait for the other OTEP.

[atoulme]

I created this as an OTEP to carefully set our time and help us focus on the issue. There is definitely no urgency we need to apply here, and I welcome any conversation around this topic. Please bring forward your OTEP on stability levels and let's coordinate, I appreciate the heads up.

[carlosalberto]

Putting a P2 priority as we wait for the PR being prepared by @jpkrohling

[TylerHelmuth]

@jpkrohling have you had a chance to work on that otep?

broader definition of distribution that would also work for the SDKs

This is an interesting idea, but I don't think it needs to block this work. Any broad opentelemewtry-wide definition should be able to fit in nicely on top of any decisions we make about our own distributions of the collector.

@atoulme is this OTEP still a WIP or is it ready for review?

[Aneurysm9]

Suggested change

	The core distribution maps to well established technologies that predate OpenTelemetry, such as Jaeger, Zipkin, Prometheus and OpenCensus.
	The core distribution maps to well-established open source observability technologies that predate OpenTelemetry, such as Jaeger, Zipkin, Prometheus and OpenCensus.

I think there's slightly more nuance here than "before OTel".

Alternately, it could be prhased as such:

Suggested change

	The core distribution maps to well established technologies that predate OpenTelemetry, such as Jaeger, Zipkin, Prometheus and OpenCensus.
	The core distribution exists to support OpenTelemetry's original mission to supersede OpenTracing and OpenCensus. As such, it includes components supporting signals and technologies supported by those systems.

[jpkrohling]

@jpkrohling have you had a chance to work on that otep?

Yes, sorry, it's the maturity one: #232. The part where I think it connects with this one is this statement from @atoulme:

This OTEP aims to clarify what distributions the OpenTelemetry project should distribute

We can use the terminology from that, stating that core is a tier-1 distribution, while contrib isn't.

[TylerHelmuth]

@jpkrohling after reading through #232 I feel more confident in my feeling that this OTEP could proceed independently. I do not think this OTEP should cover what distributions the OpenTelemetry project should distribute for the entire project, only define what distributions the Collector SIG is willing to maintain.

I feel the Collector SIG is capable of creating a list of criteria for each the Collector distribution it supports. Once #232 is complete, the Collector SIG can attached the proposed Tier labels and Maturity labels to the distributions it has decided to maintain. It does not sound to me like Maturity or Tier label definitions will restrict what an OpenTelemetry SIG is allowed to maintain, only clarify expectations of those components for users and the GC/TC.

If the community agrees, I would like to continue moving forward with this OTEP scoped specifically to the OpenTelemetry Collector distributions.

[atoulme]

@atoulme is this OTEP still a WIP or is it ready for review?

It's very raw WIP, just listing as many issues and claims as possible and trying hard to avoid making recommendations yet.

[TylerHelmuth]

I am pretty unexperienced with the OTEP process, but the README is focused on OTEP changes for the Spec, which I don't feel is applicable to this OTEP. At what state would this OTEP be merged and what would be the implications of that merge? I don't feel whatever discussion we have about our own Collector-specific distributions has broad implications across the Spec/Languages like #232.

If this OTEP's goal is to foster discussion and build a list of requirements and dependencies attached to the current Collector distributions, would it be better as an issue in opentelemetry-collector-releases?

[jpkrohling]

If the community agrees, I would like to continue moving forward with this OTEP scoped specifically to the OpenTelemetry Collector distributions.

I completely agree -- I wanted to get that draft out before this one got completed as I saw a potential conflict on the stability expectations of each distribution, but now that it's out, this one should definitely proceed!

[atoulme]

Closing, let's continue in open-telemetry/opentelemetry-collector-releases#360