add document defining an OpenTelemetry Collector

[codeboten]

Changes

Adds a definition of an OpenTelemetry Collector

• Related issues Propose a definition of Collector to be included in the specification #4309

[cyrille-leclerc]

Should this specification define what an OpenTelemetry Collector plugin is? Is it any component of type receiver, processor, exporter, extension, or config map provider?

[yurishkuro]

Specification needs to define all the terms used in this definition, otherwise it does not remove ambiguity.

[codeboten]

Renamed plugin to component and added a section to define OpenTelemetry Collector component.

[jaronoff97]

Will another section contain the initial set of required plugins and features?

[reyang]

Do we want to emphasize "compiled"? I think the specification is trying to avoid being too specific to the actual implementation, maybe reword this to "MUST be able to be used with any additional ...".

[Aneurysm9]

Does this ultimately come down to requiring that an OpenTelemetry Collector must be Open Source? There is no binary plugin mechanism on the horizon so the only way for a user to bring new components is to build a new collector, which means they need the source and license to modify it to include new components.

[jpkrohling]

nit: link to the vanity link instead of a specific file branch in a specific repo? Or perhaps this? https://pkg.go.dev/go.opentelemetry.io/collector/component#Component

[jpkrohling]

Open points:

• Should we require a specific schema? We'd need to define a schema for this.
• Should we require a vaguely defined format, such as "OTel Collector YAML configuration file, as documented in the examples available on the website"?

[Aneurysm9]

I think a schema is required, at least for the shape of the configuration to the point where individual component configuration shapes become relevant. My understanding of the intent of this requirement is to enable interoperability between collectors, which requires clarity and precision about what must be accepted in a configuration file.

[mx-psi]

The Collector also accepts confmap.Providers and confmap.Converters, which do not accept this interface. Do we consider those out of scope?

[Aneurysm9]

I think they do need to be considered in scope. Interoperability of those components is important.

[jpkrohling]

This made sense as part of the short definition we had at the original issue, but here, it feels a bit strange: it might give the impression that a distribution doesn't have to enable users to include their own components, which is not in the spirit of the definition we discussed, right?

I'd propose to move the two requirements from the top of the doc here to this part:

- An OpenTelemetry Collector _MUST_ accept an [OpenTelemetry Collector configuration
  file](#opentelemetry-collector-configuration-file).
- An OpenTelemetry Collector _MUST_ be able to be compiled with any and all
  additional [Collector components](#opentelemetry-collector-components) that
  the user wishes to include.

[Aneurysm9]

I think there needs to be room for a compiled binary distribution that has a fixed set of components and features. The ADOT collector is only the ADOT collector when it has the set of components and features that are included in the binary that AWS produces and supports. Users can certainly fork the repo and modify the component factories to include any components they wish, but at that point it stops being the ADOT collector and is something else.

[Aneurysm9]

I think there needs to be room for a compiled binary distribution that has a fixed set of components and features. The ADOT collector is only the ADOT collector when it has the set of components and features that are included in the binary that AWS produces and supports. Users can certainly fork the repo and modify the component factories to include any components they wish, but at that point it stops being the ADOT collector and is something else.

[Aneurysm9]

Does this ultimately come down to requiring that an OpenTelemetry Collector must be Open Source? There is no binary plugin mechanism on the horizon so the only way for a user to bring new components is to build a new collector, which means they need the source and license to modify it to include new components.

[Aneurysm9]

I'm not sure I understand this goal. If a vendor produces a collector distribution that has a subset of available components because those are the components relevant to their service offerings and that they're willing to support, where do any other components (whether hosted in an OTel repo or not) fit into that picture? Do we mean that a distribution must offer end users the ability to modify its source and create their own build? We should be explicit about that if that is the case.

Given that the licensing of the collector's source code does not require that distribution of derivative works happen in source form I'm not sure that we have much ability here to enforce such a requirement. We can certainly try to use the "OpenTelemetry" mark as a cudgel, but I'm not sure it'll be as effective as may be desirable since the terms "collector" and "distribution" are very broad. It could perhaps be argued that "OpenTelemetry Collector" is a protectable mark and maybe even that "Collector" has acquired secondary meaning in this limited scope, but protecting such a mark against genericization is going to be a Sisyphean task.

[Aneurysm9]

Suggested change

	from: tmp/otel/specification/baggage/_index.md
	to: baggage/README.md
	from: tmp/otel/specification/collector/_index.md
	to: collector/README.md

[Aneurysm9]

I think a schema is required, at least for the shape of the configuration to the point where individual component configuration shapes become relevant. My understanding of the intent of this requirement is to enable interoperability between collectors, which requires clarity and precision about what must be accepted in a configuration file.

[Aneurysm9]

I think they do need to be considered in scope. Interoperability of those components is important.