add leave steps, make join script re-runnable

docs/tutorials/JoinOnPremClustersToFleet.md

@@ -70,3 +70,44 @@ The `JOINED` column will be `True` once both fleet networking member agent chart
 member agents are able to communicate with fleet hub cluster.
 The column can take upto a minute to populate. The `JOINED` column indicates that all three fleet member agents have all joined once.
 The column is not meant for tracking each member agent's health status.
+
+> **Note:** Once all the on-prem clusters have joined, ensure you follow the steps below to make the clusters leave before re-running the script.
+
+# Steps to make on-prem cluster leave the Fleet hub cluster
+
+Delete the `MemberCluster` resource for a particular on-prem cluster in the hub cluster.
+
+The join script in the fleet repo creates `MemberCluster` resource with the same name as your on-prem cluster.
+Replace <cluster-name> with the name of your on-prem cluster.
+
+```
+kubectl config use-context hub
+kubectl delete membercluster <cluster-name>
+```
+
+Once the above delete command completes the on-prem cluster has successfully left the Fleet hub cluster. 
+But we still need to clean-up residual resources on the hub and on-prem clusters.
+
+> **Note:** There is a case where `MemberCluster` resource deletion is stuck, this occurs because we didn't install all the member agents required. 
+> If this case occurs run the following command,
+
+```
+kubectl delete internalmembercluster <cluster-name> -n fleet-member-<cluster-name>
+```
+
+This ensures the `MemberCluster` can be deleted so the on-prem cluster can successfully leave the Fleet hub cluster.
+
+# Clean up resources created by the join scripts
+
+We create all resources used for joining in a namespace called `connect-to-fleet`.
+Replace <cluster-name> with the name of your on-prem cluster.
+
+```
+kubectl config use-context hub
+kubectl delete secret <cluster-name>-hub-cluster-access-token -n connect-to-fleet
+kubectl delete serviceaccount <cluster-name>-hub-cluster-access -n connect-to-fleet
+kubectl config use-context <cluster-name>
+helm uninstall member-agent
+helm uninstall member-net-controller-manager
+helm uninstall mcs-controller-manager
+```

hack/Azure/setup/joinMC.sh

@@ -10,11 +10,15 @@ export HUB_CLUSTER_ADDRESS=$(kubectl config view -o jsonpath="{.clusters[?(@.nam
 echo "Switching into hub cluster context..."
 kubectl config use-context $HUB_CLUSTER_CONTEXT
 
-echo "Delete existing namespace to host resources required to connect to fleet"
-kubectl delete namespace connect-to-fleet --ignore-not-found=true
+export NOT_FOUND="not found"
+export CONNECT_TO_FLEET=connect-to-fleet
 
 echo "Create namespace to host resources required to connect to fleet"
-kubectl create namespace connect-to-fleet
+if [[ $NOT_FOUND == *$(kubectl get namespace $CONNECT_TO_FLEET)* ]]; then
+  kubectl create namespace $CONNECT_TO_FLEET
+else
+  echo "namespace $CONNECT_TO_FLEET already exists"
+fi
 
 for MC in "${@:3}"; do
 
@@ -30,23 +34,32 @@ export SERVICE_ACCOUNT="$MEMBER_CLUSTER-hub-cluster-access"
 # Note that if you choose a different value, commands in some steps below need to be
 # modified accordingly.
 echo "Creating member service account..."
-kubectl create serviceaccount $SERVICE_ACCOUNT -n connect-to-fleet
+if [[ $NOT_FOUND == *$(kubectl get serviceaccount $SERVICE_ACCOUNT -n $CONNECT_TO_FLEET)* ]]; then
+  kubectl create serviceaccount $SERVICE_ACCOUNT -n $CONNECT_TO_FLEET
+else
+  echo "member service account $SERVICE_ACCOUNT already exists in namespace $CONNECT_TO_FLEET"
+fi
 
 echo "Creating member service account secret..."
 export SERVICE_ACCOUNT_SECRET="$MEMBER_CLUSTER-hub-cluster-access-token"
+if [[ $NOT_FOUND == *$(kubectl get secret $SERVICE_ACCOUNT_SECRET -n $CONNECT_TO_FLEET)* ]]; then
 cat <<EOF | kubectl apply -f -
 apiVersion: v1
 kind: Secret
 metadata:
     name: $SERVICE_ACCOUNT_SECRET
-    namespace: connect-to-fleet
+    namespace: $CONNECT_TO_FLEET
     annotations:
         kubernetes.io/service-account.name: $SERVICE_ACCOUNT
 type: kubernetes.io/service-account-token
 EOF
+else
+  echo "member service account secret $SERVICE_ACCOUNT_SECRET already exists in namespace $CONNECT_TO_FLEET"
+fi
 
 echo "Creating member cluster CR..."
-export TOKEN="$(kubectl get secret $SERVICE_ACCOUNT_SECRET -n connect-to-fleet -o jsonpath='{.data.token}' | base64 --decode)"
+export TOKEN="$(kubectl get secret $SERVICE_ACCOUNT_SECRET -n $CONNECT_TO_FLEET -o jsonpath='{.data.token}' | base64 --decode)"
+if [[ $NOT_FOUND == *$(kubectl get membercluster $MEMBER_CLUSTER)* ]]; then
 cat <<EOF | kubectl apply -f -
 apiVersion: cluster.kubernetes-fleet.io/v1beta1
 kind: MemberCluster
@@ -56,10 +69,13 @@ spec:
     identity:
         name: $MEMBER_CLUSTER-hub-cluster-access
         kind: ServiceAccount
-        namespace: connect-to-fleet
+        namespace: $CONNECT_TO_FLEET
         apiGroup: ""
     heartbeatPeriodSeconds: 15
 EOF
+else
+  echo "member cluster CR $MEMBER_CLUSTER already exists"
+fi
 
 # # Install the member agent helm chart on the member cluster.