Add Cilium CNI option

[sholdee]

Proposed Changes

• Adds Cilium CNI initialization option
• Disabled by default unless cilium_iface is defined, overrides flannel and calico
• Default settings assume nodes on same subnet and leverage native routing with auto direct routes, kube-proxy replacement, BPF masquerade, hybrid DSR, and maglev LB algorithm
• Routed mode available for environments requiring it
• Provides BGP control plane option with LB IPAM, replacing MetalLB when enabled
• BGP control plane advertises LB services and pod CIDRs by default, allowing one to enable native routing mode in environments with cross-subnet cluster node traffic
• Hubble observability platform enabled by default
• Adds molecule scenario
• Adds new scenario to CI test workflow
• Tested with MetalLB in native L2 mode
• Tested with Cilium in BGP control plane mode
• Tested molecule scenario
• Maintains integrity of interface contract and is non-breaking

Why Cilium?

• Performance: Cilium is built from the ground up on eBPF, ensuring leading edge performance and scalability.
• Maturity and Stability: Cilium is a CNCF "Graduated" maturity level CNI implementation and is widely adopted for production workloads by many of the largest cloud providers in the industry.
• Advanced Policies: Cilium provides advanced layer 4, layer 3, and layer 7 policy capabilities.
• Visibility and Monitoring: Cilium features Hubble observability platform, built on eBPF, providing live visibility into your cluster network activity.
• Customization: Cilium is highly customizable for advanced use-cases post-initialization.

Checklist

• Tested locally
• Ran site.yml playbook
• Ran reset.yml playbook
• Did not add any unnecessary changes
• Ran pre-commit install at least once before committing (actually did this time)
• 🚀

[timothystewart6]

@sholdee This is awesome! Thank you so much!!

[timothystewart6]

👍