Auto instrumentation fails when the annotation set on both the namespace and the pod with different values

[hossamdash]

Component(s)

No response

What happened?

Description

I have a use case where most of the apps inside a namespace are of one language (python apps), with some being a different language (go applications) . I set the annotation for to auto inject python on the namespace level and the two annotations to inject go ( the inject and auto target )on the pod level.

Steps to Reproduce

put any annotation related to injection on the namespace equal to true
put a different annotation on the pod level (other than go, tried also with nodejs)

Expected Result

the annotation on the pod level to take precedence and instrumentation happening

Actual Result

the instrumentation doesn't happen it with the operator spewing out an error message saying I need to provide container names for all instrumentations.

Kubernetes Version

1.30.4+k3s1

Operator version

0.111.0

Collector version

NA

Environment information

Environment

OS: arch-linux

Log output

{"level":"ERROR","timestamp":"2024-10-29T11:05:03Z","message":"skipping instrumentation injection","namespace":"btech-apps","name":"trace-generator-pod","error":"incorrect instrumentation configuration - please provide container names for all instrumentations","stacktrace":"github.com/open-telemetry/opentelemetry-operator/pkg/instrumentation.(*instPodMutator).Mutate\n\t/home/runner/work/opentelemetry-operator/opentelemetry-operator/pkg/instrumentation/podmutator.go:408\ngithub.com/open-telemetry/opentelemetry-operator/internal/webhook/podmutation.(*podMutationWebhook).Handle\n\t/home/runner/work/opentelemetry-operator/opentelemetry-operator/internal/webhook/podmutation/webhookhandler.go:93\nsigs.k8s.io/controller-runtime/pkg/webhook/admission.(*Webhook).Handle\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/webhook/admission/webhook.go:181\nsigs.k8s.io/controller-runtime/pkg/webhook/admission.(*Webhook).ServeHTTP\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/webhook/admission/http.go:119\nsigs.k8s.io/controller-runtime/pkg/webhook/internal/metrics.InstrumentedHook.InstrumentHandlerInFlight.func1\n\t/home/runner/go/pkg/mod/github.com/prometheus/[email protected]/prometheus/promhttp/instrument_server.go:60\nnet/http.HandlerFunc.ServeHTTP\n\t/opt/hostedtoolcache/go/1.22.8/x64/src/net/http/server.go:2171\ngithub.com/prometheus/client_golang/prometheus/promhttp.InstrumentHandlerCounter.func1\n\t/home/runner/go/pkg/mod/github.com/prometheus/[email protected]/prometheus/promhttp/instrument_server.go:147\nnet/http.HandlerFunc.ServeHTTP\n\t/opt/hostedtoolcache/go/1.22.8/x64/src/net/http/server.go:2171\ngithub.com/prometheus/client_golang/prometheus/promhttp.InstrumentHandlerDuration.func2\n\t/home/runner/go/pkg/mod/github.com/prometheus/[email protected]/prometheus/promhttp/instrument_server.go:109\nnet/http.HandlerFunc.ServeHTTP\n\t/opt/hostedtoolcache/go/1.22.8/x64/src/net/http/server.go:2171\nnet/http.(*ServeMux).ServeHTTP\n\t/opt/hostedtoolcache/go/1.22.8/x64/src/net/http/server.go:2688\nnet/http.serverHandler.ServeHTTP\n\t/opt/hostedtoolcache/go/1.22.8/x64/src/net/http/server.go:3142\nnet/http.(*conn).serve\n\t/opt/hostedtoolcache/go/1.22.8/x64/src/net/http/server.go:2044"}

Additional context

If this is the default behavior and I have to put the same annotation from the namespace on the pod but set it to false in order for the instrumentation to take place, please consider having the new behavior I'm describing as the default especially since this is happening with single container pods

[hossamdash]

here my command line flags for the manager