Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a Kubernetes Distribution #507

Merged
Merged
Show file tree
Hide file tree
Changes from 16 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
Expand Up @@ -13,3 +13,5 @@
#

* @open-telemetry/collector-contrib-approvers

distributions/otelcol-k8s/ @open-telemetry/collector-contrib-approvers @open-telemetry/helm-approvers @open-telemetry/operator-approvers
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't here be as well the operator maintainers?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't @open-telemetry/operator-approvers include the maintainers?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct. I went with -approvers instead of maintainers because thats how we do our own code owners file and its how the otel site handles adding other teams as code owners. In my opinion helm and operator Approvers are totally qualified to review PRs for this distro.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@TylerHelmuth is there anything we need to do to resolve the error message github is giving us here?
Screenshot 2024-04-04 at 11 06 11 AM

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ya, we need to give those groups access to this repo. I'll open a community issue for that.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

10 changes: 8 additions & 2 deletions .github/workflows/base-ci-goreleaser.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,19 @@ on:
distribution:
required: true
type: string
goos:
required: true
type: string
goarch:
required: true
type: string

jobs:
check-goreleaser:
strategy:
matrix:
GOOS: [ linux, windows, darwin ]
GOARCH: [ "386", amd64, arm64, ppc64le, arm, s390x ]
GOOS: ${{ fromJSON( inputs.goos) }}
GOARCH: ${{ fromJSON( inputs.goarch) }}
Comment on lines +20 to +21
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change (and the one in base-release), could be done in a separate PR if desired.

exclude:
- GOOS: darwin
GOARCH: "386"
Expand Down
10 changes: 8 additions & 2 deletions .github/workflows/base-release.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,19 @@ on:
distribution:
required: true
type: string
goos:
required: true
type: string
goarch:
required: true
type: string

jobs:
prepare:
strategy:
matrix:
GOOS: [linux, windows, darwin]
GOARCH: ["386", amd64, arm64, ppc64le, arm, s390x]
GOOS: ${{ fromJSON( inputs.goos) }}
GOARCH: ${{ fromJSON( inputs.goarch) }}
exclude:
- GOOS: darwin
GOARCH: "386"
Expand Down
14 changes: 14 additions & 0 deletions .github/workflows/ci-goreleaser-contrib.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,15 +5,29 @@ on:
branches: [main]
paths:
- "distributions/otelcol-contrib/**"
- "cmd/**"
- ".github/**"
- "scripts/**"
- "Makefile"
- "go.mod"
- "go.sum"
pull_request:
branches: [main]
paths:
- "distributions/otelcol-contrib/**"
- "cmd/**"
- ".github/**"
- "scripts/**"
- "Makefile"
- "go.mod"
- "go.sum"

jobs:
check-goreleaser:
name: Continuous Integration - Contrib - GoReleaser
uses: ./.github/workflows/base-ci-goreleaser.yaml
with:
distribution: otelcol-contrib
goos: '[ "linux", "windows", "darwin" ]'
goarch: '[ "386", "amd64", "arm64", "ppc64le", "arm", "s390x" ]'
secrets: inherit
14 changes: 14 additions & 0 deletions .github/workflows/ci-goreleaser-core.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,22 @@ on:
branches: [main]
paths:
- "distributions/otelcol/**"
- "cmd/**"
- ".github/**"
- "scripts/**"
- "Makefile"
- "go.mod"
- "go.sum"
pull_request:
branches: [main]
paths:
- "distributions/otelcol/**"
- "cmd/**"
- ".github/**"
- "scripts/**"
- "Makefile"
- "go.mod"
- "go.sum"


jobs:
Expand All @@ -17,4 +29,6 @@ jobs:
uses: ./.github/workflows/base-ci-goreleaser.yaml
with:
distribution: otelcol
goos: '[ "linux", "windows", "darwin" ]'
goarch: '[ "386", "amd64", "arm64", "ppc64le", "arm", "s390x" ]'
secrets: inherit
33 changes: 33 additions & 0 deletions .github/workflows/ci-goreleaser-k8s.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
name: Continuous Integration - k8s - GoReleaser

on:
push:
branches: [main]
paths:
- "distributions/otelcol-k8s/**"
- "cmd/**"
- ".github/**"
- "scripts/**"
- "Makefile"
- "go.mod"
- "go.sum"
pull_request:
branches: [main]
paths:
- "distributions/otelcol-k8s/**"
- "cmd/**"
- ".github/**"
- "scripts/**"
- "Makefile"
- "go.mod"
- "go.sum"

jobs:
check-goreleaser:
name: Continuous Integration - k8s - GoReleaser
uses: ./.github/workflows/base-ci-goreleaser.yaml
with:
distribution: otelcol-k8s
goos: '[ "linux" ]'
goarch: '[ "amd64", "arm64", "ppc64le", "s390x" ]'
secrets: inherit
2 changes: 2 additions & 0 deletions .github/workflows/release-contrib.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -10,5 +10,7 @@ jobs:
uses: ./.github/workflows/base-release.yaml
with:
distribution: otelcol-contrib
goos: '[ "linux", "windows", "darwin" ]'
goarch: '[ "386", "amd64", "arm64", "ppc64le", "arm", "s390x" ]'
secrets: inherit
permissions: write-all
2 changes: 2 additions & 0 deletions .github/workflows/release-core.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -10,5 +10,7 @@ jobs:
uses: ./.github/workflows/base-release.yaml
with:
distribution: otelcol
goos: '[ "linux", "windows", "darwin" ]'
goarch: '[ "386", "amd64", "arm64", "ppc64le", "arm", "s390x" ]'
secrets: inherit
permissions: write-all
16 changes: 16 additions & 0 deletions .github/workflows/release-k8s.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
name: Release k8s

on:
push:
tags: ["v*"]

jobs:
release:
name: Release Contrib
uses: ./.github/workflows/base-release.yaml
with:
distribution: otelcol-k8s
goos: '[ "linux" ]'
goarch: '[ "amd64", "arm64", "ppc64le", "s390x" ]'
secrets: inherit
permissions: write-all
TylerHelmuth marked this conversation as resolved.
Show resolved Hide resolved
3 changes: 2 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,8 @@ OTELCOL_BUILDER_VERSION ?= 0.97.0
OTELCOL_BUILDER_DIR ?= ${HOME}/bin
OTELCOL_BUILDER ?= ${OTELCOL_BUILDER_DIR}/ocb

DISTRIBUTIONS ?= "otelcol,otelcol-contrib"
DISTRIBUTIONS ?= "otelcol,otelcol-contrib,otelcol-k8s"
GEN_CONFIG_DISTRIBUTIONS ?= "otelcol,otelcol-contrib"
TylerHelmuth marked this conversation as resolved.
Show resolved Hide resolved

ci: check build
check: ensure-goreleaser-up-to-date
Expand Down
146 changes: 146 additions & 0 deletions distributions/otelcol-k8s/.goreleaser.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,146 @@
partial:
by: target
project_name: opentelemetry-collector-releases
env:
- COSIGN_YES=true
builds:
- id: otelcol-k8s
goos:
- linux
goarch:
- amd64
- arm64
- ppc64le
- s390x
dir: _build
binary: otelcol-k8s
ldflags:
- -s
- -w
flags:
- -trimpath
env:
- CGO_ENABLED=0
archives:
- id: otelcol-k8s
builds:
- otelcol-k8s
name_template: '{{ .Binary }}_{{ .Version }}_{{ .Os }}_{{ .Arch }}{{ if .Arm }}v{{ .Arm }}{{ end }}{{ if .Mips }}_{{ .Mips }}{{ end }}'
checksum:
name_template: '{{ .ProjectName }}_otelcol-k8s_checksums.txt'
dockers:
- goos: linux
goarch: amd64
dockerfile: Dockerfile
image_templates:
- otel/opentelemetry-collector-k8s:{{ .Version }}-amd64
- otel/opentelemetry-collector-k8s:latest-amd64
- ghcr.io/open-telemetry/opentelemetry-collector-releases/opentelemetry-collector-k8s:{{ .Version }}-amd64
- ghcr.io/open-telemetry/opentelemetry-collector-releases/opentelemetry-collector-k8s:latest-amd64
build_flag_templates:
- --pull
- --platform=linux/amd64
- --label=org.opencontainers.image.created={{.Date}}
- --label=org.opencontainers.image.name={{.ProjectName}}
- --label=org.opencontainers.image.revision={{.FullCommit}}
- --label=org.opencontainers.image.version={{.Version}}
- --label=org.opencontainers.image.source={{.GitURL}}
use: buildx
- goos: linux
goarch: arm64
dockerfile: Dockerfile
image_templates:
- otel/opentelemetry-collector-k8s:{{ .Version }}-arm64
- otel/opentelemetry-collector-k8s:latest-arm64
- ghcr.io/open-telemetry/opentelemetry-collector-releases/opentelemetry-collector-k8s:{{ .Version }}-arm64
- ghcr.io/open-telemetry/opentelemetry-collector-releases/opentelemetry-collector-k8s:latest-arm64
build_flag_templates:
- --pull
- --platform=linux/arm64
- --label=org.opencontainers.image.created={{.Date}}
- --label=org.opencontainers.image.name={{.ProjectName}}
- --label=org.opencontainers.image.revision={{.FullCommit}}
- --label=org.opencontainers.image.version={{.Version}}
- --label=org.opencontainers.image.source={{.GitURL}}
use: buildx
- goos: linux
goarch: ppc64le
dockerfile: Dockerfile
image_templates:
- otel/opentelemetry-collector-k8s:{{ .Version }}-ppc64le
- otel/opentelemetry-collector-k8s:latest-ppc64le
- ghcr.io/open-telemetry/opentelemetry-collector-releases/opentelemetry-collector-k8s:{{ .Version }}-ppc64le
- ghcr.io/open-telemetry/opentelemetry-collector-releases/opentelemetry-collector-k8s:latest-ppc64le
build_flag_templates:
- --pull
- --platform=linux/ppc64le
- --label=org.opencontainers.image.created={{.Date}}
- --label=org.opencontainers.image.name={{.ProjectName}}
- --label=org.opencontainers.image.revision={{.FullCommit}}
- --label=org.opencontainers.image.version={{.Version}}
- --label=org.opencontainers.image.source={{.GitURL}}
use: buildx
- goos: linux
goarch: s390x
dockerfile: Dockerfile
image_templates:
- otel/opentelemetry-collector-k8s:{{ .Version }}-s390x
- otel/opentelemetry-collector-k8s:latest-s390x
- ghcr.io/open-telemetry/opentelemetry-collector-releases/opentelemetry-collector-k8s:{{ .Version }}-s390x
- ghcr.io/open-telemetry/opentelemetry-collector-releases/opentelemetry-collector-k8s:latest-s390x
build_flag_templates:
- --pull
- --platform=linux/s390x
- --label=org.opencontainers.image.created={{.Date}}
- --label=org.opencontainers.image.name={{.ProjectName}}
- --label=org.opencontainers.image.revision={{.FullCommit}}
- --label=org.opencontainers.image.version={{.Version}}
- --label=org.opencontainers.image.source={{.GitURL}}
use: buildx
docker_manifests:
- name_template: otel/opentelemetry-collector-k8s:{{ .Version }}
image_templates:
- otel/opentelemetry-collector-k8s:{{ .Version }}-amd64
- otel/opentelemetry-collector-k8s:{{ .Version }}-arm64
- otel/opentelemetry-collector-k8s:{{ .Version }}-ppc64le
- otel/opentelemetry-collector-k8s:{{ .Version }}-s390x
- name_template: otel/opentelemetry-collector-k8s:latest
image_templates:
- otel/opentelemetry-collector-k8s:latest-amd64
- otel/opentelemetry-collector-k8s:latest-arm64
- otel/opentelemetry-collector-k8s:latest-ppc64le
- otel/opentelemetry-collector-k8s:latest-s390x
- name_template: ghcr.io/open-telemetry/opentelemetry-collector-releases/opentelemetry-collector-k8s:{{ .Version }}
image_templates:
- ghcr.io/open-telemetry/opentelemetry-collector-releases/opentelemetry-collector-k8s:{{ .Version }}-amd64
- ghcr.io/open-telemetry/opentelemetry-collector-releases/opentelemetry-collector-k8s:{{ .Version }}-arm64
- ghcr.io/open-telemetry/opentelemetry-collector-releases/opentelemetry-collector-k8s:{{ .Version }}-ppc64le
- ghcr.io/open-telemetry/opentelemetry-collector-releases/opentelemetry-collector-k8s:{{ .Version }}-s390x
- name_template: ghcr.io/open-telemetry/opentelemetry-collector-releases/opentelemetry-collector-k8s:latest
image_templates:
- ghcr.io/open-telemetry/opentelemetry-collector-releases/opentelemetry-collector-k8s:latest-amd64
- ghcr.io/open-telemetry/opentelemetry-collector-releases/opentelemetry-collector-k8s:latest-arm64
- ghcr.io/open-telemetry/opentelemetry-collector-releases/opentelemetry-collector-k8s:latest-ppc64le
- ghcr.io/open-telemetry/opentelemetry-collector-releases/opentelemetry-collector-k8s:latest-s390x
signs:
- cmd: cosign
args:
- sign-blob
- --output-signature
- ${artifact}.sig
- --output-certificate
- ${artifact}.pem
- ${artifact}
signature: ${artifact}.sig
artifacts: all
certificate: ${artifact}.pem
docker_signs:
- args:
- sign
- ${artifact}
artifacts: all
sboms:
- id: archive
artifacts: archive
- id: package
artifacts: package
17 changes: 17 additions & 0 deletions distributions/otelcol-k8s/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
FROM alpine:3.16 as certs
TylerHelmuth marked this conversation as resolved.
Show resolved Hide resolved
RUN apk --update add ca-certificates

FROM scratch

ARG USER_UID=10001
USER ${USER_UID}

COPY --from=certs /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt
COPY --chmod=755 otelcol-k8s /otelcol-k8s
ENTRYPOINT ["/otelcol-k8s"]
# `4137` and `4318`: OTLP
# `55678`: OpenCensus
# `55679`: zpages
# `6831`, `14268`, and `14250`: Jaeger
# `9411`: Zipkin
EXPOSE 4317 4318 55678 55679 6831 14268 14250 9411
TylerHelmuth marked this conversation as resolved.
Show resolved Hide resolved
18 changes: 18 additions & 0 deletions distributions/otelcol-k8s/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# OpenTelemetry Collector Kubernetes Distro

This distribution is made specifically to be used in a Kubernetes Cluster to monitor Kubernetes and services running in Kubernetes. It contains a subset of components from [OpenTelemetry Collector](https://github.com/open-telemetry/opentelemetry-collector) and [OpenTelemetry Collector Contrib](https://github.com/open-telemetry/opentelemetry-collector-contrib).
TylerHelmuth marked this conversation as resolved.
Show resolved Hide resolved
TylerHelmuth marked this conversation as resolved.
Show resolved Hide resolved

## Components

The full list of components is available in the [manifest](manifest.yaml)
TylerHelmuth marked this conversation as resolved.
Show resolved Hide resolved

### Rules for Component Inclusion

- Only includes components from Contrib and Core.
- Only components that are Alpha or higher.
- All receivers must facilitate the collection of data that is generated by Kubernetes or services running in Kubernetes
- Although you could run a collector in Kubernetes to reach out to any endpoint, this distribution is not targeting that use case. This distribution is for monitoring Kubernetes and the processes it runs.
TylerHelmuth marked this conversation as resolved.
Show resolved Hide resolved
- All components must be vendor-neutral.
- Only exporters that use OTLP are allowed.
- To facilitate troubleshooting, the debug exporter and file exporter are exceptions .
TylerHelmuth marked this conversation as resolved.
Show resolved Hide resolved
- All technologies OTel promised to support will be included as receivers. Specifically this means the `jaegerreceiver`, `zipkinreceiver`, `opencensusreceiver`, and `prometheusreceiver` will be included.
jpkrohling marked this conversation as resolved.
Show resolved Hide resolved
Loading