[FEATURE] adding otlp endpoint #7996

nicolastakashi · 2024-12-16T21:45:30Z

I added CHANGELOG entry for this change.
Change is not relevant to the end user.

Changes

Verification

Signed-off-by: Nicolas Takashi <[email protected]>

pkg/receive/handler_otlp.go

+
+	tenant, err := tenancy.GetTenantFromHTTP(r, h.options.TenantHeader, h.options.DefaultTenantID, h.options.TenantField)
+	if err != nil {
+		level.Error(h.logger).Log("msg", "error getting tenant from HTTP", "err", err)


pkg/receive/handler_otlp.go

+
+	defer writeGate.Done()
+	if err != nil {
+		level.Error(tLogger).Log("err", err, "msg", "internal server error")


pkg/receive/handler_otlp.go

+		case errBadReplica:
+			responseStatusCode = http.StatusBadRequest
+		default:
+			level.Error(tLogger).Log("err", err, "msg", "internal server error")


pkg/receive/handler_otlp.go

Signed-off-by: Nicolas Takashi <[email protected]>

saswatamcode

Thanks for the efforts here! Looks good already. 🙂

I did recommend copying the translator struct over to use thanos protos natively instead of doing a full conversion, but it could be a maintenance burden.

Let's wait for other maintainers to have a look as well.

Couple of comments

pkg/receive/handler_otlp.go

pkg/receive/otlptranslator/metrics_to_prw.go

matej-g

Thanks for starting this work @nicolastakashi ❤️, I'm adding a couple more suggestions

pkg/receive/handler_otlp.go

matej-g · 2024-12-19T09:43:49Z

pkg/receive/otlptranslator/context.go

+// Copyright (c) The Thanos Authors.
+// Licensed under the Apache License 2.0.
+
+// Provenance-includes-location: https://github.com/prometheus/prometheus/blob/318d6bc4bfe2d81af88697940740a378b503335a/storage/remote/otlptranslator/prometheusremotewrite/context.go#L18


Just for my understanding, where are these provenance directives from? What does it mean? That the code is copied from Prometheus repo?

I copy the code from prometheus and prometheus copied from OTel 😅

test/e2e/e2ethanos/services.go

Signed-off-by: Nicolas Takashi <[email protected]>

saswatamcode

Thanks! ❤️
This already looks quite good. @nicolastakashi we can probably mark it ready for review, and get others to have a look as well?

cmd/thanos/receive.go

pkg/receive/handler.go

Signed-off-by: Nicolas Takashi <[email protected]>

Co-authored-by: Saswata Mukherjee <[email protected]> Signed-off-by: Nicolas Takashi <[email protected]>

Signed-off-by: Nicolas Takashi <[email protected]>

nicolastakashi · 2024-12-20T13:59:14Z

pkg/receive/handler.go

@@ -275,6 +277,18 @@ func NewHandler(logger log.Logger, o *Options) *Handler {
 		),
 	)

+	h.router.Post(


@saswatamcode just FYI use this endpoint naming will require users to use the metrics_endpoint instead of endpoint property, maybe worth document it or writing a blog post later about Thanos and OTLP wdyt?

Signed-off-by: Nicolas Takashi <[email protected]>

pkg/receive/otlptranslator/histograms.go

pedro-stanaka

LGTM - Thanks for working on this, this makes the project much more relevant in the whole Observability space.

That said, I am not a huge fan of the whole copying files over, but I am okay with it in name of optimization. The code for the handler looks good, I just wanted to see more core reuse in general (but not a problem to solve on this PR). We already have duplication because of Capn Proto ingestion and now also because of OTLP.

Once again, amazing work Nicolas! 💪

matej-g

Thanks again @nicolastakashi 🙌 , this is looking better with every pass.

I'm adding couple more suggestions to improve the parameters, I hope that's OK and doesn't add to much effort.

Two more things:

I think it would be good to have a separate documentation section to explain the OTLP endpoint and mainly to explain the behavior of resource attributes and how the parameters affect the ingested metrics / labels.
With regards to the translator code, I'm personally also leaning towards just importing instead of copying and all of the code, and then seeing if we need to copy the code over for further optimization. But it's not a strong position, if people see value in including this optimization from the get-go, I'm not opposed.

matej-g · 2024-12-23T09:42:18Z

cmd/thanos/receive.go

@@ -1052,6 +1056,9 @@ func (rc *receiveConfig) registerFlag(cmd extkingpin.FlagClause) {
 	cmd.Flag("receive.limits-config-reload-timer", "Minimum amount of time to pass for the limit configuration to be reloaded. Helps to avoid excessive reloads.").
 		Default("1s").Hidden().DurationVar(&rc.limitsConfigReloadTimer)

+	cmd.Flag("receive.otlp-disable-target-info", "Disable target information OTLP metrics ingested by Receive.").Default("false").BoolVar(&rc.otlpDisableTargetInfo)


Two suggestions:

Maybe it would feel a bit more natural to make it a positive setting, i.e. make receive.otlp-enable-target-info (since that is also the default)

Describe what exactly enabling / disabling this means, something like If enabled, converts the resource to the target info metric.

matej-g · 2024-12-23T09:53:10Z

cmd/thanos/receive.go

@@ -1052,6 +1056,9 @@ func (rc *receiveConfig) registerFlag(cmd extkingpin.FlagClause) {
 	cmd.Flag("receive.limits-config-reload-timer", "Minimum amount of time to pass for the limit configuration to be reloaded. Helps to avoid excessive reloads.").
 		Default("1s").Hidden().DurationVar(&rc.limitsConfigReloadTimer)

+	cmd.Flag("receive.otlp-disable-target-info", "Disable target information OTLP metrics ingested by Receive.").Default("false").BoolVar(&rc.otlpDisableTargetInfo)
+	cmd.Flag("receive.otlp-resource-attributes", "(Repeatable) Resource attributes to include in OTLP metrics ingested by Receive.").Default("").StringsVar(&rc.otlpResourceAttributes)


On second look, this might be a little confusing - this should indicate which resource attributes should be used as metric labels, correct? Maybe we could retain the original parameter name, something like receive.otlp-promote-resource-attributes?

[FEATURE] adding otlp endpoint

4415626

Signed-off-by: Nicolas Takashi <[email protected]>

pull-request-size bot added the size/L label Dec 16, 2024

nicolastakashi added 2 commits December 16, 2024 21:49

[FEATURE] adding otlp endpoint

58e4bc2

Signed-off-by: Nicolas Takashi <[email protected]>

[FEATURE] adding otlp endpoint

6b464fa

Signed-off-by: Nicolas Takashi <[email protected]>

github-advanced-security bot found potential problems Dec 16, 2024

View reviewed changes

Merge branch 'main' into chore/adding-otlp-endpoint

8a80246

saswatamcode reviewed Dec 18, 2024

View reviewed changes

pkg/receive/handler_otlp.go Outdated Show resolved Hide resolved

saswatamcode reviewed Dec 18, 2024

View reviewed changes

pkg/receive/handler_otlp.go Outdated Show resolved Hide resolved

nicolastakashi added 2 commits December 18, 2024 20:55

[FIX] e2e tests for otlp receiver

5f71e26

Signed-off-by: Nicolas Takashi <[email protected]>

Merge branch 'main' into chore/adding-otlp-endpoint

593aca7

nicolastakashi requested a review from saswatamcode December 18, 2024 21:14

nicolastakashi added 3 commits December 18, 2024 21:36

[CHORE] adding otlp flags

fdd473a

Signed-off-by: Nicolas Takashi <[email protected]>

[DOC] updating docs

18a48b6

Signed-off-by: Nicolas Takashi <[email protected]>

[CHORE] copying otlptranslator

13761e2

Signed-off-by: Nicolas Takashi <[email protected]>

pull-request-size bot added size/XXL and removed size/L labels Dec 18, 2024

nicolastakashi added 6 commits December 18, 2024 22:56

[CHORE] copying otlptranslator tests

6431b67

Signed-off-by: Nicolas Takashi <[email protected]>

[CHORE] copying otlptranslator tests

b163dab

Signed-off-by: Nicolas Takashi <[email protected]>

[CHORE] copying otlptranslator tests

543a28f

Signed-off-by: Nicolas Takashi <[email protected]>

[FIX] lint issues

5aa4063

Signed-off-by: Nicolas Takashi <[email protected]>

[FIX] lint issues

f9841ba

Signed-off-by: Nicolas Takashi <[email protected]>

[FIX] lint issues

991c8e5

Signed-off-by: Nicolas Takashi <[email protected]>

saswatamcode reviewed Dec 19, 2024

View reviewed changes

pkg/receive/handler_otlp.go Show resolved Hide resolved

pkg/receive/otlptranslator/metrics_to_prw.go Outdated Show resolved Hide resolved

matej-g reviewed Dec 19, 2024

View reviewed changes

nicolastakashi added 6 commits December 19, 2024 14:54

[CHORE] using multi errors

0dc5845

Signed-off-by: Nicolas Takashi <[email protected]>

[CHORE] using multi errors

65dc21b

Signed-off-by: Nicolas Takashi <[email protected]>

[FIX] span naming convention

ab46f3b

Signed-off-by: Nicolas Takashi <[email protected]>

[TEST] adding handler otlp unit test

8be010c

Signed-off-by: Nicolas Takashi <[email protected]>

[TEST] upgrade collector version

5b4600b

Signed-off-by: Nicolas Takashi <[email protected]>

[FIX] golang lint

706e41e

Signed-off-by: Nicolas Takashi <[email protected]>

Merge branch 'main' into chore/adding-otlp-endpoint

b57f248

nicolastakashi requested review from saswatamcode and matej-g December 20, 2024 10:22

saswatamcode reviewed Dec 20, 2024

View reviewed changes

cmd/thanos/receive.go Outdated Show resolved Hide resolved

pkg/receive/handler.go Outdated Show resolved Hide resolved

nicolastakashi and others added 4 commits December 20, 2024 13:28

[CHORE] adding allow size bytes limit gate

fc5e612

Signed-off-by: Nicolas Takashi <[email protected]>

[FIX] unit test otlp endpoint

12dbf04

Signed-off-by: Nicolas Takashi <[email protected]>

Apply suggestions from code review

63ce67b

Co-authored-by: Saswata Mukherjee <[email protected]> Signed-off-by: Nicolas Takashi <[email protected]>

[FIX] unit test otlp endpoint

d2e6b60

Signed-off-by: Nicolas Takashi <[email protected]>

nicolastakashi commented Dec 20, 2024

View reviewed changes

[DOC] updating docs

a2bef89

Signed-off-by: Nicolas Takashi <[email protected]>

nicolastakashi marked this pull request as ready for review December 20, 2024 14:06

pedro-stanaka reviewed Dec 20, 2024

View reviewed changes

pkg/receive/otlptranslator/histograms.go Show resolved Hide resolved

pedro-stanaka approved these changes Dec 20, 2024

View reviewed changes

Merge branch 'main' into chore/adding-otlp-endpoint

493fe90

matej-g reviewed Dec 23, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[FEATURE] adding otlp endpoint #7996

[FEATURE] adding otlp endpoint #7996

nicolastakashi commented Dec 16, 2024

saswatamcode left a comment

matej-g left a comment

matej-g Dec 19, 2024

nicolastakashi Dec 19, 2024

saswatamcode left a comment

nicolastakashi Dec 20, 2024

pedro-stanaka left a comment

matej-g left a comment

matej-g Dec 23, 2024

matej-g Dec 23, 2024

[FEATURE] adding otlp endpoint #7996

Are you sure you want to change the base?

[FEATURE] adding otlp endpoint #7996

Conversation

nicolastakashi commented Dec 16, 2024

Changes

Verification

saswatamcode left a comment

Choose a reason for hiding this comment

matej-g left a comment

Choose a reason for hiding this comment

matej-g Dec 19, 2024

Choose a reason for hiding this comment

nicolastakashi Dec 19, 2024

Choose a reason for hiding this comment

saswatamcode left a comment

Choose a reason for hiding this comment

nicolastakashi Dec 20, 2024

Choose a reason for hiding this comment

pedro-stanaka left a comment

Choose a reason for hiding this comment

matej-g left a comment

Choose a reason for hiding this comment

matej-g Dec 23, 2024

Choose a reason for hiding this comment

matej-g Dec 23, 2024

Choose a reason for hiding this comment