NETOBSERV-1500 : Refactoring of transform network API

[OlivierCazade]

Description

Created different type for the different transform rules

Previously the NetworkTransformRule had all fields at the same level, but they were not all used by all transform network types and the Parameter field was used differently depending on the type.

The struct go from this:

type NetworkTransformRule struct {
	Input           string        `yaml:"input,omitempty" json:"input,omitempty" doc:"entry input field"`
	Output          string        `yaml:"output,omitempty" json:"output,omitempty" doc:"entry output field"`
	Type            string        `yaml:"type,omitempty" json:"type,omitempty" enum:"TransformNetworkOperationEnum" doc:"one of the following:"`
	Parameters      string        `yaml:"parameters,omitempty" json:"parameters,omitempty" doc:"parameters specific to type"`
	Assignee        string        `yaml:"assignee,omitempty" json:"assignee,omitempty" doc:"value needs to assign to output field"`
	KubernetesInfra *K8sInfraRule `yaml:"kubernetes_infra,omitempty" json:"kubernetes_infra,omitempty" doc:"Kubernetes infra rule specific configuration"`
	Kubernetes      *K8sRule      `yaml:"kubernetes,omitempty" json:"kubernetes,omitempty" doc:"Kubernetes rule specific configuration"`
}

To this :

type NetworkTransformRule struct {
	Type            string                 `yaml:"type,omitempty" json:"type,omitempty" enum:"TransformNetworkOperationEnum" doc:"one of the following:"`
	KubernetesInfra *K8sInfraRule          `yaml:"kubernetes_infra,omitempty" json:"kubernetes_infra,omitempty" doc:"Kubernetes infra rule configuration"`
	Kubernetes      *K8sRule               `yaml:"kubernetes,omitempty" json:"kubernetes,omitempty" doc:"Kubernetes rule configuration"`
	AddSubnet       *NetworkAddSubnetRule  `yaml:"add_subnet,omitempty" json:"add_subnet,omitempty" doc:"Add subnet rule configuration"`
	AddLocation     *NetworkGenericRule    `yaml:"add_location,omitempty" json:"add_location,omitempty" doc:"Add location rule configuration"`
	AddIPCategory   *NetworkGenericRule    `yaml:"add_ip_category,omitempty" json:"add_ip_category,omitempty" doc:"Add ip category rule configuration"`
	AddService      *NetworkAddServiceRule `yaml:"add_service,omitempty" json:"add_service,omitempty" doc:"Add service rule configuration"`
}

Now every rule type has its own structure.

Dependencies

n/a

Checklist

If you are not familiar with our processes or don't know what to answer in the list below, let us know in a comment: the maintainers will take care of that.

• Will this change affect NetObserv / Network Observability operator? If not, you can ignore the rest of this checklist.
• Is this PR backed with a JIRA ticket? If so, make sure it is written as a title prefix (in general, PRs affecting the NetObserv/Network Observability product should be backed with a JIRA ticket - especially if they bring user facing changes).
• Does this PR require product documentation?
  • If so, make sure the JIRA epic is labelled with "documentation" and provides a description relevant for doc writers, such as use cases or scenarios. Any required step to activate or configure the feature should be documented there, such as new CRD knobs.
• Does this PR require a product release notes entry?
  • If so, fill in "Release Note Text" in the JIRA.
• Is there anything else the QE team should know before testing? E.g: configuration changes, environment setup, etc.
  • If so, make sure it is described in the JIRA ticket.
• QE requirements (check 1 from the list):
  • Standard QE validation, with pre-merge tests unless stated otherwise.
  • Regression tests only (e.g. refactoring with no user-facing change).
  • No QE (e.g. trivial change with high reviewer's confidence, or per agreement with the QE team).

[OlivierCazade]

/hold
This is breaking change in the configuration API, we need to synchronize

[codecov]

Codecov Report

Attention: Patch coverage is 65.21739% with 24 lines in your changes are missing coverage. Please review.

Project coverage is 67.22%. Comparing base (ab65bc4) to head (d688fe3).
Report is 3 commits behind head on main.

Files	Patch %	Lines
pkg/pipeline/transform/transform_network.go	45.71%	15 Missing and 4 partials ⚠️
pkg/pipeline/transform/kubernetes/enrich.go	77.27%	2 Missing and 3 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #580      +/-   ##
==========================================
+ Coverage   66.97%   67.22%   +0.24%     
==========================================
  Files         104      104              
  Lines        7446     7459      +13     
==========================================
+ Hits         4987     5014      +27     
+ Misses       2162     2143      -19     
- Partials      297      302       +5     

Flag	Coverage Δ
unittests	67.22% <65.21%> (+0.24%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[jotak]

Not yet reviewed in deep, but the approach looks good to me
specializing each rule params via their own struct rather than forcing a generic approach looks much better!
I'll have some work that depends on this, for NETOBSERV-1426

[jpinsonneau]

It seems we often have same type of rules applied in a row; on both source & destination for example.

Would it make sense to allow arrays on the objects above ?

Then you will be able to call something like:

api.NetworkTransformRules{{
    Type: api.AddKubernetesRuleType,
    Kubernetes: []api.K8sRule{
      api.K8sRule{
          Input:  "SrcAddr",
          Output: "SrcK8S",
      }, 
      api.K8sRule{
          Input:  "DstAddr",
          Output: "DstK8S",
      },
    },
}}})

We could even get rid of the type by just providing objects 🤔

[OlivierCazade]

We could remove the type field but this would change consistency with the rest of the API that is using it.

About allowing an array, this mean we would have two nested level of rules instead of having one flat array of rules. This add complexity and I am not sure to see a value in doing it.

[jotak]

lgtm
I don't have a preference regarding Julien's suggestion (having an inner slice in k8s rules) .. it avoids some repetition but makes having an additional nested slice, hard to say which is simpler.
before merging we need to have the operator PR ready for the breaking change

[jotak]

/lgtm

[jotak]

@OlivierCazade can you edit the description to briefly explain what are the breaking changes, and give an example of how to convert an old config to a new one? This would help users who are hit by the breaking change to quickly figure out how to handle it

[jotak]

I'm adding no-qe / no-doc as there's no user facing changes - I tested myself along with netobserv/network-observability-operator#562

[memodi]

/ok-to-test

[github-actions]

New image:
quay.io/netobserv/flowlogs-pipeline:380f2f0

It will expire after two weeks.

To deploy this build, run from the operator repo, assuming the operator is running:

USER=netobserv VERSION=380f2f0 make set-flp-image

[jotak]

/lgtm

[OlivierCazade]

/approve

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: OlivierCazade

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [OlivierCazade]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment