Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Chore/sc 110915 azure login #60

Draft
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

jules-casoli
Copy link
Contributor

State of this PR :

It is paused

Starting with kubectl 1.26, cloud-dependent methods of authenticating (used by aks and gke) where deprecated. We did the transition for gke, and though that it was needed for aks, but, as it turns out, everything still seems to be working fine (tested with plugin version 1.2.0, using kubectl 1.26 creating a k8s cluster at version 1.26.0)
So, for now, this developpement is stalled.

How auth currently works

We create a kube_config file, with at least the following section:

users:
- name: clusterAdmin_jcasoli-fm_jcasoli-aks
  user:
    client-certificate-data: LS0tLS1CRUdJTiBSU0EgU<-SomeLongBase64EncodedCert==
    client-key-data: LS0tLS1CRUdJTiBSU0EgU<-SomeLongBase64EncodedKey==
    token: cm0xpg7siubjx<-SomeToken

in practice, this means we authenticate using a (probably derived only once) given token. The associated identity is clusterAdmin, and we create (or maybe it already exists ?) a clusterrolebinding on the cluster with full rights:

> kubectl describe clusterrolebinding aks-cluster-admin-binding
Name:         aks-cluster-admin-binding
Labels:       addonmanager.kubernetes.io/mode=Reconcile
              kubernetes.io/cluster-service=true
Annotations:  <none>
Role:
  Kind:  ClusterRole
  Name:  cluster-admin
Subjects:
  Kind  Name          Namespace
  ----  ----          ---------
  User  clusterAdmin
  User  clusterUser

What this PR changes

The kube_config is changed to look like this :

users:
- name: clusterAdmin_jcasoli-fm_jcasoli-aks
  user:
    exec:
      apiVersion: client.authentication.k8s.io/v1beta1
      args:
      - get-token
      - --login
      - msi
      - --server-id
      - 6dae42f8-4368-4678-94ff-3960e28e3630
      command: kubelogin

Meaning that we are going to use kubelogin to fetch a token. Note that there is several ways to identify oneself to get this token (dentity-resource-id, client-id, etc...), but the gist of it is that we are going to use an IAM identity, most probably the one set-up by DCS. I will be refered to a dss-id-jcasoli.
(as a aside 6dae42f8-4368-4678-94ff-3960e28e3630 is a hard-coded azure constant, it just means ~"aks service")
Note that, for kubelogin to work with Azure identites, we must setup our cluster to use Azure AD authentication. This is automatically set-up at the cluster creation by the plugin.
image
The token we get is an Oauth token, granted to dss-id-jcasoli. <= very important
This token will be handled by aks control plane, but there is two possibility here:

  1. It is handled by the k8s "standard" control plane directly. This correspond to the Kubernetes RBAC option :
    image
  2. It is handled by azure IAM, which then will send back an authorisation token to be used by k8s, still to the name of dss-id-jcasoli. This is the Azure RBAC option :
    image

Both options have issues, which we don't know how to handle automatically for now. (the following are from memory, some of it may be slightly incorrect)

  1. If we use Kubernetes RBAC, the k8s control plane will receive a token that is valid for user dss-id-jcasoli. However, it does not know this user, and we hence have to add it / add a corresponding clusterbindingrole. This could be done using aks cli, but, at that moment, the only iam identity we have is dss-id-jcasoli, which lacks enough rights to create the clusterbindingrole
  2. If we use Azure RBAC, we encounter mostly the same issue, but with the additional indirection of Azure AD.

Indeed, DCS' doc says that dss-id-jcasoli should be a Contributor of the k8s Cluster (actually of the whole ressource group), which seems to be equivalent to giving hime the Azure Kubernetes Service Contributor Role rights, while we probably would need Azure Kubernetes Service RBAC Admin

@shortcut-integration
Copy link

This pull request has been linked to Shortcut Story #110915: [Azure] support latest cluster version.

@jules-casoli jules-casoli marked this pull request as draft March 14, 2023 12:45
@thtrunck
Copy link
Contributor

thtrunck commented Mar 17, 2023

If we want to add Azure AAD RBAC (but we don´t currently) we want to have a selector between the local K8S RBAC and AAD. Local account should be the default and we should keep the setup done by Azure in that case.
On the AAD case we need a strong ID to do the correct setup and it's unclear we want to pass that on the cluster creation. SO maybe we need more doc for the admin to do that or something in FM doing the correct setup and giving us an ID that we can pass to DSS.

Let's keep this PR for reference but supporting AAD auth isn't straightforward so we should wait for a strong business case before doing that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants