AV-203833: Initial commit for restricting fqdn #1520

akshayhavile · 2024-09-02T06:21:54Z

TODO:

Adding more UTs

Dhivyaaj · 2024-09-02T06:21:58Z

No JIRA Ids found for the PR. Jira id is mandatory to update fix version in jira. Please update respective Jira id in PR title or commit message if the PR is intented for default branches of repo. For mandatory exemptions, comment trigger phrase 'skip jira-id-check' in PR. For manual trigger, comment trigger phrase 'run jira-id-check' in PR.

Dhivyaaj · 2024-09-08T17:29:12Z

No JIRA Ids found for the PR. Jira id is mandatory to update fix version in jira. Please update respective Jira id in PR title or commit message if the PR is intented for default branches of repo. For mandatory exemptions, comment trigger phrase 'skip jira-id-check' in PR. For manual trigger, comment trigger phrase 'run jira-id-check' in PR.

akshayhavile · 2024-09-08T17:29:56Z

skip jira-id-check

Dhivyaaj · 2024-09-08T17:30:29Z

No JIRA Ids found for the PR. Jira id is mandatory to update fix version in jira. Please update respective Jira id in PR title or commit message if the PR is intented for default branches of repo. For mandatory exemptions, comment trigger phrase 'skip jira-id-check' in PR. For manual trigger, comment trigger phrase 'run jira-id-check' in PR.

pkoshtavmware · 2024-09-11T07:12:35Z

Could you please resolve lint issues @akshayhavile ?

internal/k8s/ako_init.go

DixitAakash

Overall changes look good.

helm/ako/templates/configmap.yaml

internal/k8s/ako_init.go

tests/evhtests/l7_evh_graph_test.go

internal/objects/uniquehostname.go

internal/k8s/ako_init.go

pkoshtavmware

LGTM

pkoshtavmware

LGTM

abhishekbsingh · 2024-09-23T06:03:46Z

internal/cache/controller_obj_cache.go

+		utils.AviLog.Debugf("Populating Ingress mapping for host : %s", host)
+		for _, ing := range ings {
+			namespace, _, name := lib.ExtractTypeNameNamespace(ing)
+			// Fetch ingress using clientset. From informer couldn't fetch it.


Instead of making Get calls using k8s client, should we start the informers before calling PopulateCache and use them here?

Populate cache may fail and then AKO will error out from there. So my intention was to avoid initialising delay by keeping booting sequence same.
What is your thoughts on this?

abhishekbsingh · 2024-09-23T06:14:20Z

internal/k8s/ako_init.go

+				if lib.AKOFQDNReusePolicy() == lib.FQDNReusePolicyStrict {
+					isValid = isRouteAcceptedWithFQDNRestriction(key, routeObj)
+					if isValid {
+						utils.AviLog.Debugf("Route %s is added to active list. Enqueuing it", key)


IMO, we should log the routes/ingress which we are not adding to the active list.

abhishekbsingh · 2024-09-23T06:17:39Z

internal/k8s/controller.go

-		c.workqueue[bkt].AddRateLimited(key)
-		lib.IncrementQueueCounter(utils.ObjectIngestionLayer)
-		utils.AviLog.Debugf("key: %s, msg: %s for namespace: %s", key, msg, namespace)
+		isAdded := true


nit: isAdded -> toBeAdded

abhishekbsingh · 2024-09-23T06:23:23Z

internal/k8s/controller.go

+				utils.AviLog.Debugf("key: %s, msg: ADD", key)
+			} else {
+				// update the status - already host claimed
+				status.UpdateRouteStatusWithErrMsg(key, route.Name, namespace, lib.HostAlreadyClaimed)


above code block can be restructured for readability and it also removes use of isAdded

if lib.AKOFQDNReusePolicy() == lib.FQDNReusePolicyStrict && !isRouteAcceptedWithFQDNRestriction(key, route) { status.UpdateRouteStatusWithErrMsg(key, route.Name, namespace, lib.HostAlreadyClaimed) return } bkt := utils.Bkt(namespace, numWorkers) if !lib.HasValidBackends(route.Spec, route.Name, namespace, key) { status.UpdateRouteStatusWithErrMsg(key, route.Name, namespace, lib.DuplicateBackends) } c.workqueue[bkt].AddRateLimited(key) lib.IncrementQueueCounter(utils.ObjectIngestionLayer) utils.AviLog.Debugf("key: %s, msg: ADD", key)

similar changes can be done in other places as well

abhishekbsingh · 2024-09-23T06:30:57Z

internal/k8s/controller.go

+					isOldAccepted := isRouteAcceptedWithFQDNRestriction(key, oldRoute)
+					isNewAccepted := isRouteAcceptedWithFQDNRestriction(key, newRoute)
+					if isOldAccepted || isNewAccepted {
+						c.workqueue[bkt].AddRateLimited(key)


If old hostname was accepted and the hostname has changed, we should be calling
objects.SharedUniqueNamespaceLister().DeleteHostnameToRoute(route.Spec.Host, routeNamespaceName)

Also, I am not clear on the handling of newRoute/Ingress is done if the new hostname is already claimed post update - I think it should be handled like a Deletion event, but right now, I think it will become a no OP as EnqueuIng will skip it before processing and the corresponding VS in Avi will remain up and running.
We need to test this case

When key gets enqueued, model will not be created in graph layer. But object present in rest layer cache, so it will be delete call.

abhishekbsingh · 2024-09-23T06:54:20Z

internal/nodes/avi_model_evh_nodes.go

@@ -1083,24 +1098,27 @@ func ProcessInsecureHostsForEVH(routeIgrObj RouteIngressModel, key string, parse

 		// Create one evh child per host and associate http policies for each path.
 		modelGraph := aviModel.(*AviObjectGraph)
-		modelGraph.BuildModelGraphForInsecureEVH(routeIgrObj, host, infraSetting, key, pathsvcmap)
+		flagIngToProcess = modelGraph.BuildModelGraphForInsecureEVH(routeIgrObj, host, infraSetting, key, pathsvcmap)


given the called function is only returning true, do we really need this flag?

abhishekbsingh · 2024-09-23T07:07:04Z

internal/nodes/hostname_cache.go

+		if len(ingresses) > 0 {
+			nsIngressname := strings.Split(ingresses[0], "/")
+			if len(nsIngressname) > 0 {
+				if nsIngressname[0] != namespace {


Should we have this check here? I think we can filter the routes/ingresses in ingestion layer itself

This is done for multipath ingress definition where some hosts are valid and other are valid to add extra layer of blocking.

abhishekbsingh · 2024-09-23T08:55:41Z

internal/lib/constants.go

@@ -207,6 +208,8 @@ const (
 	VrfContextObjectNotFoundError              = "VrfContext object not found"
 	NetworkNotFoundError                       = "Network object not found"
 	CtrlVersion_22_1_6                         = "22.1.6"
+	FQDNReusePolicyStrict                      = "strict"
+	FQDNReusePolicyOpen                        = "interNamespaceallowed"


interNamespaceallowed -> internamespaceallowed

abhishekbsingh · 2024-09-23T09:10:15Z

internal/objects/uniquehostname.go

+	// each entry will be objecttype/route-namespace/route-name
+	RouteNSRouteName string
+	// store creation time
+	CreationTime metav1.Time


why do we need the creation time, not sure if it's used anywhere

Previously we had active and inactive list. When all routes from active list gets deleted, we had to take out route with earliest creation time so that VS for that route will be created. But we have taken out functionality to dynamically updates routes. In future, if we decide to re-introduce functionality for that purpose we have kept this functionality.

abhishekbsingh · 2024-09-23T09:17:27Z

tests/evhtests/l7_evh_graph_test.go

+	g.Expect(len(nodes)).To(gomega.Equal(1))
+	g.Expect(len(nodes[0].EvhNodes)).To(gomega.Equal(0))
+
+	err = KubeClient.NetworkingV1().Ingresses("red").Delete(context.TODO(), "ingress-multi2", metav1.DeleteOptions{})


We should check that model corresponding to this Ingress is not found and the status is updated correctly.

model will be same as that for default namespace ingress. we are checking number of evhVs node and number of PG refs, and httpps ref... which should be 1.

akshayhavile · 2024-09-24T10:29:04Z

internal/cache/controller_obj_cache.go

@@ -73,6 +75,7 @@ func NewAviObjCache() *AviObjCache {
 	c.VrfCache = NewAviCache()
 	c.PKIProfileCache = NewAviCache()
 	c.ClusterStatusCache = NewAviCache()
+	c.FQDNPolicyCache = NewAviCache()


akshayhavile · 2024-09-24T10:34:12Z

internal/cache/controller_obj_cache.go

+		utils.AviLog.Debugf("Populating Ingress mapping for host : %s", host)
+		for _, ing := range ings {
+			namespace, _, name := lib.ExtractTypeNameNamespace(ing)
+			// Fetch ingress using clientset. From informer couldn't fetch it.


Populate cache may fail and then AKO will error out from there. So my intention was to avoid initialising delay by keeping booting sequence same.
What is your thoughts on this?

akshayhavile · 2024-09-24T15:42:07Z

internal/k8s/controller.go

+					isOldAccepted := isRouteAcceptedWithFQDNRestriction(key, oldRoute)
+					isNewAccepted := isRouteAcceptedWithFQDNRestriction(key, newRoute)
+					if isOldAccepted || isNewAccepted {
+						c.workqueue[bkt].AddRateLimited(key)


When key gets enqueued, model will not be created in graph layer. But object present in rest layer cache, so it will be delete call.

akshayhavile · 2024-09-26T06:52:52Z

internal/k8s/controller.go

-		c.workqueue[bkt].AddRateLimited(key)
-		lib.IncrementQueueCounter(utils.ObjectIngestionLayer)
-		utils.AviLog.Debugf("key: %s, msg: %s for namespace: %s", key, msg, namespace)
+		isAdded := true


akshayhavile · 2024-09-26T08:41:42Z

internal/k8s/controller.go

+					isOldAccepted := isRouteAcceptedWithFQDNRestriction(key, oldRoute)
+					isNewAccepted := isRouteAcceptedWithFQDNRestriction(key, newRoute)
+					if isOldAccepted || isNewAccepted {
+						c.workqueue[bkt].AddRateLimited(key)


akshayhavile · 2024-09-26T09:43:52Z

internal/cache/controller_obj_cache.go

@@ -55,6 +56,7 @@ type AviObjCache struct {
 	VsCacheMeta        *AviCache
 	VsCacheLocal       *AviCache
 	ClusterStatusCache *AviCache
+	FQDNPolicyCache    *AviCache


akshayhavile · 2024-09-26T14:20:11Z

internal/lib/lib.go

@@ -1795,6 +1797,10 @@ func ValidateSvcforClass(key string, svc *corev1.Service) bool {
 	return false
 }

+func AKOFQDNReusePolicy() string {
+	return AKOControlConfig().GetAKOFQDNReusePolicy()
+}


akshayhavile · 2024-09-26T14:54:10Z

internal/objects/uniquehostname.go

+	// each entry will be objecttype/route-namespace/route-name
+	RouteNSRouteName string
+	// store creation time
+	CreationTime metav1.Time


Previously we had active and inactive list. When all routes from active list gets deleted, we had to take out route with earliest creation time so that VS for that route will be created. But we have taken out functionality to dynamically updates routes. In future, if we decide to re-introduce functionality for that purpose we have kept this functionality.

akshayhavile · 2024-09-26T14:54:35Z

internal/nodes/avi_model_evh_nodes.go

@@ -1216,6 +1240,7 @@ func (o *AviObjectGraph) BuildModelGraphForInsecureEVH(routeIgrObj RouteIngressM
 	evhNode.AddFQDNAliasesToHTTPPolicy(evhNode.VHDomainNames, key)
 	evhNode.AviMarkers.Host = evhNode.VHDomainNames
 	objects.SharedCRDLister().UpdateFQDNToAliasesMappings(host, evhNode.VHDomainNames)
+	return true


akshayhavile · 2024-09-27T07:47:11Z

tests/evhtests/l7_evh_graph_test.go

+	g.Expect(len(nodes)).To(gomega.Equal(1))
+	g.Expect(len(nodes[0].EvhNodes)).To(gomega.Equal(0))
+
+	err = KubeClient.NetworkingV1().Ingresses("red").Delete(context.TODO(), "ingress-multi2", metav1.DeleteOptions{})


model will be same as that for default namespace ingress. we are checking number of evhVs node and number of PG refs, and httpps ref... which should be 1.

akshayhavile requested review from arihantg, aaha97, abhishekbsingh, pkoshtavmware, DixitAakash and ashishnayak96 September 2, 2024 06:21

akshayhavile assigned abhishekbsingh, pkoshtavmware and DixitAakash Sep 2, 2024

vmwclabot added the cla-not-required label Sep 2, 2024

akshayhavile force-pushed the AV-203833-master branch from e9ef70f to d00bcbf Compare September 8, 2024 17:29

akshayhavile changed the title ~~Initial commit for restricting fqdn~~ AV-203833: Initial commit for restricting fqdn Sep 8, 2024

akshayhavile force-pushed the AV-203833-master branch from d00bcbf to abd581d Compare September 8, 2024 17:39

DixitAakash reviewed Sep 12, 2024

View reviewed changes

internal/k8s/ako_init.go Show resolved Hide resolved

DixitAakash previously approved these changes Sep 12, 2024

View reviewed changes

akshayhavile dismissed DixitAakash’s stale review via 649c0c5 September 15, 2024 13:03

akshayhavile force-pushed the AV-203833-master branch from abd581d to 649c0c5 Compare September 15, 2024 13:03

DixitAakash previously approved these changes Sep 16, 2024

View reviewed changes

pkoshtavmware reviewed Sep 17, 2024

View reviewed changes

akshayhavile dismissed DixitAakash’s stale review via 99b8f60 September 17, 2024 11:20

akshayhavile force-pushed the AV-203833-master branch from 649c0c5 to 99b8f60 Compare September 17, 2024 11:20

pkoshtavmware reviewed Sep 18, 2024

View reviewed changes

internal/objects/uniquehostname.go Outdated Show resolved Hide resolved

internal/k8s/ako_init.go Show resolved Hide resolved

pkoshtavmware previously approved these changes Sep 18, 2024

View reviewed changes

akshayhavile dismissed pkoshtavmware’s stale review via 2d1824f September 20, 2024 07:03

akshayhavile force-pushed the AV-203833-master branch from 99b8f60 to 2d1824f Compare September 20, 2024 07:03

akshayhavile added 2 commits September 20, 2024 07:06

Initial commit for restricting fqdn

f3f6cf9

Functionality restriction to EVH only

0597f20

akshayhavile force-pushed the AV-203833-master branch from 2d1824f to fe03487 Compare September 20, 2024 07:11

Review comments

fe03487

pkoshtavmware previously approved these changes Sep 20, 2024

View reviewed changes

DixitAakash previously approved these changes Sep 23, 2024

View reviewed changes

abhishekbsingh requested changes Sep 23, 2024

View reviewed changes

Review comments addressed 2

6860f26

akshayhavile dismissed stale reviews from DixitAakash and pkoshtavmware via 6860f26 September 27, 2024 07:45

akshayhavile commented Sep 27, 2024

View reviewed changes

AV-203833: Initial commit for restricting fqdn #1520

Are you sure you want to change the base?

AV-203833: Initial commit for restricting fqdn #1520

Conversation

akshayhavile commented Sep 2, 2024 • edited Loading

Dhivyaaj commented Sep 2, 2024

Dhivyaaj commented Sep 8, 2024

akshayhavile commented Sep 8, 2024

Dhivyaaj commented Sep 8, 2024

pkoshtavmware commented Sep 11, 2024

DixitAakash left a comment

Choose a reason for hiding this comment

pkoshtavmware left a comment

Choose a reason for hiding this comment

pkoshtavmware left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

akshayhavile commented Sep 2, 2024 •

edited

Loading