Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove CCD secret mutation. #420

Merged
merged 3 commits into from
Sep 3, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion example/controller-registration.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ providerConfig:
chart: H4sIAAAAAAAAA+0ca2/cNjKf9SsIuUXbQ6R9eO0EAnKoG7up7xJ7YfvSOxRFoJXoXcVaURUlO26a/37Dl0S9Vis7ddJGAwNeUZzhkJwZDodDLd3ExxFOLPwuxRENSGTFCbkOfCha49QNR4/uDWOAJ3t7/D9A9T//PdmdTaZ70/19Vj7Z3x3PHqG9+zfdDRlN3QShRwkh6aZ6Xe//orDsnP/nKzdJ7Vt3Hd61DTbB+7NZ6/xPd2fl+Z9C0fgRGn/MjrbBFz7/bhy8xgmbdwddTww3jvNHc2KPTcPH1EuCOOVFB+gnHK6Rx0QCXZIEpSuMXkgRQq+YvKC5FB+US5QRuWvsoE5RM65V02Mb2jY+9dh8CdCt/z7x7CW5Txtd+r+3+6Ri/5/MxnuD/j8EjEZL4iyZBLgpRnSFLA+Ztj2Cv2sc+SQZLYN0lS1sj6xHSliKHyvXuxopdMsjUZqQMATZSfAyoCmUgkTZQLYsVMhGX33ruSkSLb0+Ojs/Pj35Tj7id+46DvGojRxbi9Bz8XIeuhF2BNHj6DJxoVbmpVmiCn8myRVOxINpGKMRmgPP7hJLK4YjdxFiikrDkMUxkRZOFgbRkhs7jyQJ9lJU8IZKvBmxTv3zN2Hd+p9imAwYFXpnT7C3/zcd746ng//3ENBn/t+scBjDEm2ncS9fsMP+T6bT/cr8T/eng/1/EHj/3kI+vgwijEzmppnI+vDB6HbVGB6sD7y2oRMJ3QUOqQ2OpH2FbwU5/pAtcBJhkCM7ICPWVIlGC4lrN8wkT+/foyDywszPObWRRNzASB23yiCj4qCWGrJ93lK9F0EEwhN5mKPbZzjELsX2CTDXyFnOWrCGFUJwhhB7E1yilUvnCbx/h0y6cqd7+w40+5o1D02x+nbqLlGOESdBlF4i82v6/de0WjPBMaFBSpLbTSSgj7iJoHNngtBZrd/VCfFxHJLbNY5S6efnwkFHsPXQh+tTa8WXA33sP/g8l8Fy7cYWn/xr8INIYhGYzZskSHFrjKDL/5/t71bs/xNWNNj/BwBpfUpa/ZpP7KmaV2H7SmGCqyDyHeaCgzy8cmODSYrvpq4DlkBs9putdbPgSCQKrnODKeXFwsgIw+w0mHNG/g8oBFlO0YzVVuzwFumbspQ66A9GZGOvy+Q0o/app+yjwp30v2c0sEP/Z5NJ1f/bnewO+v8g8LEUO5eNP1WZRSu5CiMAy7L4f70jXHBtJcd2LtrUlgSU1NteSDIfvA83jFfuhBPKh0Du78VgZGJ/b1TspaTnhQHwCjUjMCJQTfQQ+K2UO7wUmPU8HLNyYCy9uI0x5UOV4N+yIME+Mjvo23UCKKA5vtnFXxO+ZJkPsirtyZWG2Y8dHTHn47e476gARr92GULe3iJLaNqzRY7Tr02BUl5SmqVq7Xor8JyP+Qqm+CwVcu1Jyf9YQGwjcuti1tzyJXTlxg3DY5iWJHJDsTMpmGh7v4GfVpKtrLGGdtCPx/99dQTDu4YVvIVbvEwwpYeYpkHEtTTns/5mA4cNZDbyhlPPVwpNwXWAgVaPTPNdSk+Uxay2BJi2RLHzmoUkAroLhjJIb7uxZUVNjF3vKosLTqi3wn4WtjMiEGxVT2fDx2HqnkduTFckneMkIH4XmQYURbEySH6WEFrwKQKxjfRVZzmGLSu2S29rddVU7CZpwGZY2PYNMlEmVcErxGMybZAP7lXOszCckzDwGqeyUqW9S3nFc+wlOIVOecTH/iHxrtQK9a/z05Na06K6GuQWrI2cdTf4d3WNvwjo4/8XAZx+G4DN/v9kMtmrnv/v7u4N5/8PArrbrCJxwgM+zGd7613An+L70xh7rOEEXweMz58CZpRvXwbrALy1MX8Tg/V0acmOycLnJItS0SgFXtgW35F+VOqtXm7Hx74goDRBEtAGhfv0UURS5WSolWbL8Eq+7K+wd0WztRZq40rYHDcpTcO3PICLvrIvJJf2DzDwczddIXOryJ35He+yCD4DDzpflZWthdWNW8M7MLsVWzC6LnCmRhiGKwk8teqfewkIeLQsLf4JgUornOUHEQ4yG+L3ZgsGZTQZTppkuKi0w+LcbhamSLLABiwmrJtBpB0XWwkIZLDGLdTZubPkp9yjOTuQ1rmqzYlSKlUhwukNSUCZa/vdlCguLBgeihMQBFD/MCQ32N8O3wcp74cRZwtQSEvW6Y2dBNcsxWArdPDjSZZ4MHh1QoBmgWtKUmqxw5xiAKzUi63ZbLeg3GEVnuZ+rbRQXC1gol3YdyX5RFhdJlMAV8zyzNfOXj58cGqvxQmMWabT4nUKjLjkb+b6TNZrUKRCfCw0anAOhDlawUY90WqWXQY9DwRoQpt6XUsaBOsyCPGzEewdRs1DIw3BSAv6VMmwVmKWAQLtvGMFXpYkMDdWgtkDNECflX1byVf+m2PbBeb5beRRfVxYSysMjv8Cu6mVL2/PNqxuTYhAG99YAdv8XsMIUcaf38pcjmdzvGOJdi6wqq0EpbyX/kNRxu8ajGAZEWiFxFiExKxi4WttgKOcKoyDHKFK+4Yn6/TvgcDr4vwGL0Dxr5T89ZzLCrY0Gsxe61zJWs/l9pFXkqa7RM0PKFugNFUpdU6+LqKQLBbxFlYTZD4222jJtpsI/SxftVCpuCnlJafUlnxlLcAIWq7vs5DJM6d1udrkQUgFCdNVOzXxvomR3DbJw2S907nVl+90XBxd6zZO2OaXRweHR2dvjl4ePb84Pj15c3Lw6uh8fvD8KK+JEE8F+BEWa0crZKEwHPpn+LJcKsuZR+PknqKdS9xd/UPF7/GrgxdHr4HZ07M3p6+Pzn4+O76o8eqgEU9C0w6/Ro2nYZsmKQyuYSQpnSdkgfU+rtI0flGEFgTEvL8jMWu/l19xx6ZzahmwSBTr5U8XF3PtRRAFaeCGhzh0b6UldNBknNdIsOsHvXllWLcPwuqeoTdA61KoLIywGhq53KmYVznbxtxwFzMlHgkddPF8Xo0w5c6SjpMXNsXGCow/UKQiYOOGCBiDaxJma/yKbcMauiyMqcbqmlUUatPtGtxXjdpOo5uYqamSVo/J0GkUgrPF9gTt6sTmJ/DwgecxwifdbuEOci8vmTDdOnkJEx//ALz3g4ZXKD+IOMxgi7U8F0Fd+HXM12BZfPQOe5l+ILUjx4W7ueeljbJ6yQeEbZqP3sXMTusb3aKGha7wbWs6VZ5wVcNDSDgU0Co6jhpec5PW0CBrcovkrTJaSmISkuXtvxmvZjmpa0VoyidC4ggBrrnzFQn01JGpbmS2PjFVILeQr4gPeLOpsm29xHs74e7Pb5eybOB9iA5/ttAn/gt2B9zJJOOXgRaZv8TbBYI773/MKvc/4MeT6RD/fQiQpmWZom9ZqKopevodmlRTwGIePhhdTxYgIipgPCf+YS4eP3Dx+Dwix7Cd/E/kXrtByLZCnDzNFp0dvnfE+K9g+vrof7JwvbtcBO3Qf9D6qv5PJ+Mng/4/BLD0KV2z+Ry7WboiSfC7uNJ09ZT7RUV2WAhjhpMzEuI++t1Hc5MsZB6XxbK6XiQki7n7ZSEtk6sc2TVKWxZWVaQjLDLvCqe0XjKCaU8z/QU0ngS4oaRclfITntKDXsETgyMftBhjQ4mO50eUhbkSv/qoVyoH6hrL9OoiKlb6XbwGD24hh4oZau78B1T8uGGWjv+K819ZDPOM61PSFmuvz4gIBft5aZkJ8x9mnbhHYAhkNo6Uwzpdvk5UqHmwJePeaa1X92xDlfNlSbyrBL+hKnvgSwfLGOQjXLeyeYi3eS7ahts0+T92VCtlUE1Qq+YKBH8d8D1b6baiXiEONO3SXmjnKm2jky/dtPI4gh2qGwa/KymE/UMktZHyvJJcK8RWSNaSB2dSFfJz6NrziHpuiCU5vrWm+oMr9tklxQTlw7UCFn6E/onyokbt1VuyED/AEy5+jGAnKQQ8S/lFURmU8fTEUVEddqSB31HHA7bIWo0wvzwTFG/lbHAHLNhKh2QioE3dmCtm4wwyzBqpHZFIazE/KrgMPNRmlmW1FKxmI3WV+1ca+JuAqUTf3ihSdr828yclcqqgLF1drat0sK7GebrYFvT8LRYyqKMvTcVjH1t+n5V0i6XpDgyApSJMfTfZFSCbglMegpap6sKSdI2r2J/UCTbsYNsXjj7LojBMIWYm+14e1Q/C3vxpjhU0IQ8r1JBt4NDIs+E1l6+DH9hZvYWVj5sJgXxeinp292ebjeCndpv/NtBn/yeX1d5bwI7933RvVs3/m+4P938eBhrv/0iN/fjRm1qO3DZJO5cJWbPMgNC3WCIQPztD3/zy3lTHWqZjXjyfm49N9s50tjse+/DrN/044IlDGPuWSACzQGzY8mHJbKESY1U+yofij6uc34kXdVq4iY+HGqB89i0VMpMcaDEzaLdy0lI6FwceBVETBqiGWGmPLWOAwPO4zF6cggdg8cUx57TIA4EhDFgWaeEbGeLQ6OL08NRBF6uAisWWbb8SAn41u/kDHiT4Q2AgfZ6IGBEUkmiJE2AHvHefZf+xD8dcZjy5Bp3xmyysaI1ciighEftf+7TO99cT+8ke+ywAWmAcgSfDNcq3+0mK8ISserd5mluTQN+J/kdTgrvEd1N+U036KMfsHL4I3G6X2ZsfyFvy8H822+XjUD4550eHfJR6HcR/ahu/Ce6w/stt9fZuQNf5z3Q2qaz/s+l4yP9/ENi0/iuP/ZMe4nzqAfqbQ7f+i8SL+3wAtEP/d8f7Vf0fz/aH858HAZF2ziMlKs3cQcuVlygHTYa5mnLBK5+EYmvj0kF8rWDb/FhLQj8Ib9xbahj6MauDJkYRn0HvPxiG5iI46On46dgoUu94wcQwtIxZ+UmA/ERF7CkqWcAi8a58TLKhYh6XF3Wa8q8dtMuyW8SRyqY229KeHXTphhQbRj3N10G//GpUknZ5mbGDmvJ+2GcQWFZTIO+D72gZQLGbUZFhzJMHDZH9JAb8TJ/u4guPhYOu/1yEZDFau8y9Gi2yIAQ3mpEeidugLI/fUMlnGlUhQ0tCliF+U9yLELiWu/b3ZxKNy425yz43KwryD8FO7MnEfvfX7tWk1ivzn89Yz6bihW3bhlHyHx1D5AqqhE/mjRo7O1CWsrMDyj9MKXXlMcL20kZU3XRa3CLu1BfXigBTKRYjDHTE5aj8opLC5S8NdRNb3Z2SSY2Gl7PW/FWLpm9awD6JqTurNHpLSaR0pfi+RGMN/uWHyVikkMnPMky4zpW/fsD0ovKFgMY7/6zeho8WiNfFdf7KPfXiKr8qUJfzBX+l+/b5bXtDZcXVb9AblSZKF+Hza/DCRIjC6oV1ZiwbrvocX55AI9B9zOOsjbfA2+6AG/lH/4T0YTkYysjLr6oM/tgAAwwwwAADDDDAAAMMMMAAAwwwwAADDDDAAAMMMMDnDf8HJasSEQB4AAA=
values:
image:
tag: v0.24.4
tag: v0.24.5
---
apiVersion: core.gardener.cloud/v1beta1
kind: ControllerRegistration
Expand Down
1 change: 0 additions & 1 deletion go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -158,7 +158,6 @@ require (
istio.io/api v1.19.2-0.20231011000955-f3015ebb5bd4 // indirect
istio.io/client-go v1.19.3 // indirect
k8s.io/apiserver v0.29.2 // indirect
k8s.io/cluster-bootstrap v0.28.3 // indirect
k8s.io/gengo v0.0.0-20230829151522-9cce18d56c01 // indirect
k8s.io/helm v2.17.0+incompatible // indirect
k8s.io/klog v1.0.0 // indirect
Expand Down
2 changes: 0 additions & 2 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -2023,8 +2023,6 @@ k8s.io/autoscaler/vertical-pod-autoscaler v1.0.0 h1:y0TgWoHaeYEv3L1MfLC+D2WVxyN1
k8s.io/autoscaler/vertical-pod-autoscaler v1.0.0/go.mod h1:w6/LjLR3DPQd57vlgvgbpzpuJKsCiily0+OzQI+nyfI=
k8s.io/client-go v0.28.3 h1:2OqNb72ZuTZPKCl+4gTKvqao0AMOl9f3o2ijbAj3LI4=
k8s.io/client-go v0.28.3/go.mod h1:LTykbBp9gsA7SwqirlCXBWtK0guzfhpoW4qSm7i9dxo=
k8s.io/cluster-bootstrap v0.28.3 h1:hGK3mJsmVGGvRJ61nyQcYNR9g/IYax75TbJcylTmZts=
k8s.io/cluster-bootstrap v0.28.3/go.mod h1:s1B3FTw713b9iw67yGFiVF3zCfw5obrZXWl3EMelvdg=
k8s.io/code-generator v0.28.3 h1:I847QvdpYx7xKiG2KVQeCSyNF/xU9TowaDAg601mvlw=
k8s.io/code-generator v0.28.3/go.mod h1:A2EAHTRYvCvBrb/MM2zZBNipeCk3f8NtpdNIKawC43M=
k8s.io/component-base v0.28.3 h1:rDy68eHKxq/80RiMb2Ld/tbH8uAE75JdCqJyi6lXMzI=
Expand Down
157 changes: 0 additions & 157 deletions pkg/webhook/shoot/mutator.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,25 +2,10 @@ package shoot

import (
"context"
"errors"
"fmt"
"net/url"
"slices"
"strings"

extensionswebhook "github.com/gardener/gardener/extensions/pkg/webhook"
gardencorev1beta1 "github.com/gardener/gardener/pkg/apis/core/v1beta1"
extensionsv1alpha1 "github.com/gardener/gardener/pkg/apis/extensions/v1alpha1"
resourcesv1alpha1 "github.com/gardener/gardener/pkg/apis/resources/v1alpha1"
"github.com/gardener/gardener/pkg/component/extensions/operatingsystemconfig/downloader"
"github.com/metal-stack/gardener-extension-provider-metal/pkg/apis/metal/helper"

kutil "github.com/gardener/gardener/pkg/utils/kubernetes"

metalv1alpha1 "github.com/metal-stack/gardener-extension-provider-metal/pkg/apis/metal/v1alpha1"

extensionscontroller "github.com/gardener/gardener/extensions/pkg/controller"
"github.com/gardener/gardener/pkg/utils"
"github.com/go-logr/logr"
appsv1 "k8s.io/api/apps/v1"
corev1 "k8s.io/api/core/v1"
Expand Down Expand Up @@ -64,14 +49,6 @@ func (m *mutator) Mutate(ctx context.Context, new, _ client.Object) error {
extensionswebhook.LogMutation(logger, x.Kind, x.Namespace, x.Name)
return m.mutateVPNShootDeployment(ctx, x)
}
case *corev1.Secret:
// TODO: remove this once gardener-node-agent is in use
// the purpose of this hack is to enable the cloud-config-downloader to pull the hyperkube image from
// a registry mirror in case this shoot cluster is configured with networkaccesstype restricted/forbidden
err = m.mutateCloudConfigDownloaderHyperkubeImage(ctx, x)
if err != nil {
return fmt.Errorf("mutating cloud config downlader secret failed %w", err)
}
}
return nil
}
Expand Down Expand Up @@ -99,137 +76,3 @@ func (m *mutator) mutateVPNShootDeployment(_ context.Context, deployment *appsv1

return nil
}

const (
gardenerRegistry = "eu.gcr.io"
hyperkubeImage = "/gardener-project/hyperkube"

// this should be the final destination
newGardenerRegistry = "europe-docker.pkg.dev"
newHyperkubeImage = "/gardener-project/releases/hyperkube"
)

func (m *mutator) mutateCloudConfigDownloaderHyperkubeImage(ctx context.Context, secret *corev1.Secret) error {
if secret.Labels["gardener.cloud/role"] != "cloud-config" {
return nil
}

shootName, err := extractShootNameFromSecret(secret)
if err != nil {
return err
}

cluster := &extensionsv1alpha1.Cluster{}
if err := m.client.Get(ctx, kutil.Key(shootName), cluster); err != nil {
return err
}

shoot, err := extensionscontroller.ShootFromCluster(cluster)
if err != nil {
return fmt.Errorf("unable to decode cluster.Spec.Shoot.Raw %w", err)
}

if len(shoot.Spec.Provider.Workers) == 0 {
m.logger.Info("workerless shoot, nothing to do here", "shoot", shootName)
return nil
}

cloudProfile := &gardencorev1beta1.CloudProfile{}
err = helper.DecodeRawExtension(&cluster.Spec.CloudProfile, cloudProfile, m.decoder)
if err != nil {
return err
}

cloudProfileConfig, err := helper.DecodeCloudProfileConfig(cloudProfile)
if err != nil {
return err
}

infrastructureConfig := &metalv1alpha1.InfrastructureConfig{}
err = helper.DecodeRawExtension(shoot.Spec.Provider.InfrastructureConfig, infrastructureConfig, m.decoder)
if err != nil {
return err
}

_, p, err := helper.FindMetalControlPlane(cloudProfileConfig, infrastructureConfig.PartitionID)
if err != nil {
return err
}

controlPlaneConfig := &metalv1alpha1.ControlPlaneConfig{}
err = helper.DecodeRawExtension(shoot.Spec.Provider.ControlPlaneConfig, controlPlaneConfig, m.decoder)
if err != nil {
return err
}

if controlPlaneConfig.NetworkAccessType == nil || *controlPlaneConfig.NetworkAccessType == metalv1alpha1.NetworkAccessBaseline {
// this shoot does not have networkaccesstype restricted or forbidden specified, nothing to do here
return nil
}

if p.NetworkIsolation == nil || len(p.NetworkIsolation.RegistryMirrors) == 0 {
m.logger.Info("no registry mirrors specified in this shoot, nothing to do here", "shoot", shootName)
return nil
}

var (
networkIsolation = p.NetworkIsolation
destinationRegistry string
)

for _, registry := range networkIsolation.RegistryMirrors {
if slices.Contains(registry.MirrorOf, gardenerRegistry) {
parsed, err := url.Parse(registry.Endpoint)
if err != nil {
return fmt.Errorf("unable to parse registry endpoint:%w", err)
}
destinationRegistry = parsed.Host
break
}
}
if destinationRegistry == "" {
err := errors.New("no matching destination registry detected for the hyperkube image")
m.logger.Error(err, "please check the networkisolation configuration", "shoot", shootName)
return err
}

m.logger.Info("mutate secret", "shoot", shootName, "secret", secret.Name)

raw, ok := secret.Data[downloader.DataKeyScript]
if ok {
script := string(raw)
newScript := strings.ReplaceAll(script, gardenerRegistry+hyperkubeImage, destinationRegistry+hyperkubeImage)
newScript = strings.ReplaceAll(newScript, newGardenerRegistry+newHyperkubeImage, destinationRegistry+newHyperkubeImage)
secret.Data[downloader.DataKeyScript] = []byte(newScript)
secret.Annotations[downloader.AnnotationKeyChecksum] = utils.ComputeChecksum(newScript)
}
return nil
}

func extractShootNameFromSecret(secret *corev1.Secret) (string, error) {
// resources.gardener.cloud/origin: shoot--test--fra-equ01-8fef639c-bbe4-4c6f-9656-617dc4a4efd8-gardener-soil-test:shoot--pjb9j2--forbidden/shoot-cloud-config-execution
origin, ok := secret.Annotations[resourcesv1alpha1.OriginAnnotation]
if !ok {
return "", fmt.Errorf("no matching annotation found to identify the shoot namespace")
}

// does not work
// shootName, _, err := resourcesv1alpha1helper.SplitOrigin(origin)
// if err != nil {
// return "", fmt.Errorf("no matching content found in origin annotation to get shoot namespace %w", err)
// }

// resources.gardener.cloud/origin: shoot--test--fra-equ01-8fef639c-bbe4-4c6f-9656-617dc4a4efd8-gardener-soil-test:shoot--pjb9j2--forbidden/shoot-cloud-config-execution
_, firstpart, found := strings.Cut(origin, ":")
if !found {
return "", fmt.Errorf("no matching content found in origin annotation to get shoot namespace")
}
shootName, _, found := strings.Cut(firstpart, "/")
if !found {
return "", fmt.Errorf("no matching content found in origin annotation to get shoot namespace")
}
if len(shootName) == 0 {
return "", fmt.Errorf("could not find shoot name for webhook request")
}
return shootName, nil
}
41 changes: 0 additions & 41 deletions pkg/webhook/shoot/mutator_test.go

This file was deleted.