-
Notifications
You must be signed in to change notification settings - Fork 411
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
add featuregate for k8s 1.28 native sidecar container #2801
base: main
Are you sure you want to change the base?
Conversation
pkg/sidecar/pod.go
Outdated
if featuregate.EnableNativeSidecarContainers.IsEnabled() { | ||
policy := corev1.ContainerRestartPolicyAlways | ||
container.RestartPolicy = &policy | ||
// TODO(frzifus): Add StartupProbe |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we pre-define a startup probe that checks the ${service.telemetry.metrics.address}/metrics
endpoint, expose it in the CRD or do something else?
wdyt @open-telemetry/operator-approvers ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there a reason to treat it any differently than we do readiness probes and liveness probes? We can even default to the readiness probe if not set.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nope, but reusing the readiness probe sounds good to me 👍
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there a reason we provide another probe type in our API?
opentelemetry-operator/apis/v1beta1/opentelemetrycollector_types.go
Lines 101 to 104 in d4b24f3
// Liveness config for the OpenTelemetry Collector except the probe handler which is auto generated from the health extension of the collector. | |
// It is only effective when healthcheckextension is configured in the OpenTelemetry Collector pipeline. | |
// +optional | |
LivenessProbe *Probe `json:"livenessProbe,omitempty"` |
Since that one does not contain a
ProbeHandler
:type Probe struct { |
Like here:
https://github.com/kubernetes/api/blob/5147c1a32f6a0b9b155bb84e59f933e0ff8a3792/core/v1/types.go#L2462-L2464
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think there's a good reason... is there much of a difference between these things? Maybe in v1beta1 we should just use the upstream definition?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Using the upstream definition would extend ours by the ProbeHandler.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this makes sense... is there a way to write an e2e for this?
Sure, you'd need to add a new test suite and run it with the gate enabled. Then checking if the sidecar is an initContainer should be straightforward. |
Signed-off-by: Benedikt Bongartz <[email protected]>
@swiatekm whats the best way to configure a k8s featuregate for a specific e2e test suite? Do we need to create a new e.g. opentelemetry-operator/.github/workflows/e2e.yaml Lines 22 to 24 in a515234
Do we need then to create a new step like
|
You should do the same thing we do for enabling other options in e2e tests. See here. You do need to add a new entry to the e2e test matrix, because you need a separate operator deployment with the flag enabled. Then you add a make target that adds the command-line option to the operator Deployment (same as we do for other operator flags), and use that make target for setup in the e2e tests. |
// It needs to be enabled with +featureGate=SidecarContainers. | ||
// See: | ||
// https://kubernetes.io/docs/reference/command-line-tools-reference/feature-gates/#feature-gates-for-alpha-or-beta-features | ||
EnableNativeSidecarContainers = featuregate.GlobalRegistry().MustRegister( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wonder if we could on k8s 1.28+ always use the native sidecar pattern and keep using the existing approach on older k8s versions.
This way no feature flag is needed and we would always use a recommended approach.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We can do that, but we would expect different results in our e2e tests. Which makes it hart to execute the same tests on both platform versions.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think I'm ok with that as well. I can't think of any issues it might cause.
Description:
Link to tracking Issue(s):
Testing:
Documentation: