OVS Traffic Mirroring Proposal (#227)

* add traffic mirroring proposal

Signed-off-by: Leonardo Rossetti <[email protected]>

* Add pros/cons + implementation details + question sections

Signed-off-by: Stefano Cappa <[email protected]>

* update proposal based on the discussion

Signed-off-by: Stefano Cappa <[email protected]>

* fix doc titles

Signed-off-by: Stefano Cappa <[email protected]>

* feat: merge ovs-mirror repo in ovs-cni with mirrorUuid fix + golang generics

Signed-off-by: Stefano Cappa <[email protected]>

* fix: invert ingress and egress boolean passed to CheckMirrorProducerWithPorts function

Signed-off-by: Stefano Cappa <[email protected]>

* feat: upgrade go to 1.18 also in hack folder to be able to use it while testing

Signed-off-by: Stefano Cappa <[email protected]>

* fix: restore cleanPorts when Netns is empty to pass tests without errors

Signed-off-by: Stefano Cappa <[email protected]>

* fix: small improvement to traffic-mirroring doc

Signed-off-by: Stefano Cappa <[email protected]>

* feat: add unit tests for mirror-producer

Signed-off-by: Stefano Cappa <[email protected]>

* feat: huge improvements in mirror-producer unit tests

Signed-off-by: Stefano Cappa <[email protected]>

* fix: add public ^Cthods documentation

Signed-off-by: Stefano Cappa <[email protected]>

* fix: refactor variable names to pass 'go lint'

Signed-off-by: Stefano Cappa <[email protected]>

* fix: use unique names for container and bridge to prevent errors when you run all tests together

Signed-off-by: Stefano Cappa <[email protected]>

* fix: remove bad test

Signed-off-by: Stefano Cappa <[email protected]>

* feat: add a test to verify cmdDel behaviour when a mirror has also output_port

Signed-off-by: Stefano Cappa <[email protected]>

* feat: add test in mirror-producer when both ingress and egress are false

Signed-off-by: Stefano Cappa <[email protected]>

* fix: the logic to check if a mirror has output_port == the current portUUID was totally wrong

Signed-off-by: Stefano Cappa <[email protected]>

* chore: add comment

Signed-off-by: Stefano Cappa <[email protected]>

* refactor: add comment to all utility functions and remove number from test steps

Signed-off-by: Stefano Cappa <[email protected]>

* refactor: improve errore message

Signed-off-by: Stefano Cappa <[email protected]>

* refactor: fix typo

Signed-off-by: Stefano Cappa <[email protected]>

* feat: add mirror-consumer tests

Signed-off-by: Stefano Cappa <[email protected]>

* fix: use different mirror names for producer and consumer

Signed-off-by: Stefano Cappa <[email protected]>

* refactor: use the right plugin names in tests

Signed-off-by: Stefano Cappa <[email protected]>

* chore: update gitignore to ignore also producer and consumer files used in build

Signed-off-by: Stefano Cappa <[email protected]>

* feat: add ovs-mirror plugins to ovs-cni.yml.in and define new env vars

Signed-off-by: Stefano Cappa <[email protected]>

* refactor: rename ovs-cni-mirror in ovs-mirror everywhere

Signed-off-by: Stefano Cappa <[email protected]>

* feat: experimental impl of e2e tests for mirrors

Signed-off-by: Stefano Cappa <[email protected]>

* refactor: rename again all mirror plugins from ovs-mirror-* to ovs-cni-mirror-*

Signed-off-by: Stefano Cappa <[email protected]>

* refactor: move mirror cmds into a subfolder called mirror

Signed-off-by: Stefano Cappa <[email protected]>

* fix: remove 'apk update' and fix net interface in tcpdump command

Signed-off-by: Stefano Cappa <[email protected]>

* fix: write tcpdump output as text, not in binary format

Signed-off-by: Stefano Cappa <[email protected]>

* fix: fix paths in gitignore

Signed-off-by: Stefano Cappa <[email protected]>

* fix: start tcpdump as pod command

Signed-off-by: Stefano Cappa <[email protected]>

* fix: re-enable all tests + cleanup

Signed-off-by: Stefano Cappa <[email protected]>

* fix: testFunction name of mirror_test

Signed-off-by: Stefano Cappa <[email protected]>

* fix: fix typos

Signed-off-by: Stefano Cappa <[email protected]>

* tests: add a sleep before reading tcpdump log

Signed-off-by: Leonardo Rossetti <[email protected]>

* fix: prevResult is not available in cmdDel if cniversion < 0.4.0, so we need to save and read from cache prevResult

Signed-off-by: Stefano Cappa <[email protected]>

* fix: start consumer pod before the 2 producers in e2e tests. Also, run tcpdump without -c.

Signed-off-by: Stefano Cappa <[email protected]>

* fix: use the right plugin names

Signed-off-by: Stefano Cappa <[email protected]>

* fix: rename 'ovs-cni-mirror-*' in 'ovs-mirror-*' because '-cni' is redundant

Signed-off-by: Stefano Cappa <[email protected]>

* fix: change function 'ReadTCPDumpFromPod' to be more generic (ReadFileFromPod) passing the filename as argument

Signed-off-by: Stefano Cappa <[email protected]>

* fix: typo in MirrorNetConf comment

Signed-off-by: Stefano Cappa <[email protected]>

* fix: copy and paste error in test 'Describe'

Signed-off-by: Stefano Cappa <[email protected]>

* fix: copy and paste errors in suite_test files for both producer and consumer

Signed-off-by: Stefano Cappa <[email protected]>

* refactor: extract IsMirrorUsed from DeleteMirror method to make the behaviour cleaner

Signed-off-by: Stefano Cappa <[email protected]>

* fix: remove useless calls to 'checkPortsInMirrors', because that function is already called in testAdd

Signed-off-by: Stefano Cappa <[email protected]>

* refactor: move duplicated code in a helper function

Signed-off-by: Stefano Cappa <[email protected]>

* refactor: move common mirror tests function into a dedicated package

Signed-off-by: Stefano Cappa <[email protected]>

* fix: add missing comments to exported types in testhelpers

Signed-off-by: Stefano Cappa <[email protected]>

* fix: golint error

Signed-off-by: Stefano Cappa <[email protected]>

* fix: add -cni- in hack/build-manifests.sh because in makefile 'docker-build-%' appends 'ovs-cni-'

Signed-off-by: Stefano Cappa <[email protected]>

* fix: update examples/ovs-cni.yml

Signed-off-by: Stefano Cappa <[email protected]>

* feat: add cleanEmptyMirrors

Signed-off-by: Stefano Cappa <[email protected]>

* feat: add cleanEmptyMirrors unit tests

Signed-off-by: Stefano Cappa <[email protected]>

* fix: ovs-vsctl command to add mirror requires 'add' and not 'set', otherwise all existing mirrors are removed

Signed-off-by: Stefano Cappa <[email protected]>

* feat: add cleanEmptyMirrors test to cover cmdDel case

Signed-off-by: Stefano Cappa <[email protected]>

* feat: add external_ids to Mirror table

Signed-off-by: Stefano Cappa <[email protected]>

* feat: add tests to check the behaviour of cleanMirrors with owner in external_ids

Signed-off-by: Stefano Cappa <[email protected]>

* refactor: move duplicated code in testhelpers

Signed-off-by: Stefano Cappa <[email protected]>

* fix: It texts

Signed-off-by: Stefano Cappa <[email protected]>

* fix: It texts again

Signed-off-by: Stefano Cappa <[email protected]>

Signed-off-by: Leonardo Rossetti <[email protected]>
Signed-off-by: Stefano Cappa <[email protected]>
Co-authored-by: Stefano Cappa <[email protected]>

.gitignore

@@ -1,13 +1,17 @@
 # built binaries
 cmd/marker/marker
 cmd/plugin/plugin
+cmd/mirror/consumer/consumer
+cmd/mirror/producer/producer
 
 # Temporary build files
 build/_output
 
 # used in build
 cmd/marker/.version
 cmd/plugin/.version
+cmd/mirror/consumer/.version
+cmd/mirror/producer/.version
 
 # local cluster
 _kubevirtci/

cmd/mirror/consumer/Dockerfile

@@ -0,0 +1,3 @@
+FROM registry.access.redhat.com/ubi8/ubi-minimal
+COPY consumer /ovs-mirror-consumer
+COPY .version /.version

cmd/mirror/consumer/main.go

@@ -0,0 +1,28 @@
+// Copyright 2018-2019 Red Hat, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package main
+
+import (
+	"github.com/containernetworking/cni/pkg/skel"
+	"github.com/containernetworking/cni/pkg/version"
+	"github.com/containernetworking/plugins/pkg/utils/buildversion"
+
+	plugin "github.com/k8snetworkplumbingwg/ovs-cni/pkg/mirror-consumer"
+)
+
+// mirror-consumer
+func main() {
+	skel.PluginMain(plugin.CmdAdd, plugin.CmdCheck, plugin.CmdDel, version.PluginSupports("0.3.0", "0.3.1", "0.4.0"), buildversion.BuildString("OVS mirror consumer"))
+}

cmd/mirror/producer/Dockerfile

@@ -0,0 +1,3 @@
+FROM registry.access.redhat.com/ubi8/ubi-minimal
+COPY producer /ovs-mirror-producer
+COPY .version /.version

cmd/mirror/producer/main.go

@@ -0,0 +1,28 @@
+// Copyright 2018-2019 Red Hat, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package main
+
+import (
+	"github.com/containernetworking/cni/pkg/skel"
+	"github.com/containernetworking/cni/pkg/version"
+	"github.com/containernetworking/plugins/pkg/utils/buildversion"
+
+	plugin "github.com/k8snetworkplumbingwg/ovs-cni/pkg/mirror-producer"
+)
+
+// ovs-mirror-producer
+func main() {
+	skel.PluginMain(plugin.CmdAdd, plugin.CmdCheck, plugin.CmdDel, version.PluginSupports("0.3.0", "0.3.1", "0.4.0"), buildversion.BuildString("OVS mirror producer"))
+}