Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Adding DPDK support in UPF #57

Merged
merged 21 commits into from
Jan 9, 2024
Merged
Show file tree
Hide file tree
Changes from 11 commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 17 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,23 @@ juju deploy sdcore-upf --trust --channel=edge

### Exposing the UPF externally

If a load balancer such as `metallb` is present, the charm will configure an externally accessible service port with the load balancer upon install of the charm.
If a load balancer such as `metallb` is present, the charm will configure an externally accessible
service port with the load balancer upon install of the charm.

### Running UPF in DPDK mode

By default, UPF runs in `af_packet` mode. To run UPF in `dpdk` mode, `upf-mode` config option
should be used, i.e.:

```shell
juju deploy sdcore-upf --trust --channel=edge --config upf-mode="dpdk" --config enable-hugepages=True --config access-interface-mac-address="00:b0:d0:63:c2:26" --config core-interface-mac-address="00:b0:d0:63:c2:36"
Gmerold marked this conversation as resolved.
Show resolved Hide resolved
```

As shown in the example above, when running UPF in `dpdk` mode, it is necessary to enable
HugePages and pass the MAC addresses of the `Access` and `Core` interfaces.

For detailed instructions on running UPF in `dpdk` mode please visit
[How-to: Running UPF in DPDK mode](https://canonical-charmed-5g.readthedocs-hosted.com/en/latest/how-to/running_upf_in_dpdk_mode/).

## Image

Expand Down
50 changes: 33 additions & 17 deletions config.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,10 @@
options:
upf-mode:
type: string
default: af_packet
description: |
Either `af_packet` (default) or `dpdk`.
Please see README.md to learn about all required parameters.
gruyaume marked this conversation as resolved.
Show resolved Hide resolved
Gmerold marked this conversation as resolved.
Show resolved Hide resolved
dnn:
type: string
default: internet
Expand All @@ -7,25 +13,14 @@ options:
type: string
default: 192.168.251.0/24
description: gNodeB subnet.
core-interface:
type: string
description: Interface on the host to use for the Core Network.
core-ip:
type: string
default: 192.168.250.3/24
description: IP address used by the UPF's Core interface.
core-gateway-ip:
type: string
default: 192.168.250.1
description: Gateway IP address to the Core Network.
core-interface-mtu-size:
type: int
description: |
MTU for the core interface (1200 <= MTU <= 65535) in bytes.
If not specified, Multus will use its default value (typically 1500).
access-interface:
type: string
description: Interface on the host to use for the Access Network.
access-interface-mac-address:
type: string
description: |
MAC address of the UPF's Access interface.
Required only if `upf-mode` is `dpdk`.
access-ip:
type: string
default: 192.168.252.3/24
Expand All @@ -39,9 +34,30 @@ options:
description: |
MTU for the access interface (1200 <= MTU <= 65535) in bytes.
If not specified, Multus will use its default value (typically 1500).
core-interface:
type: string
description: Interface on the host to use for the Core Network.
core-interface-mac-address:
type: string
description: |
MAC address of the UPF's Core interface.
Required only if `upf-mode` is `dpdk`.
core-ip:
type: string
default: 192.168.250.3/24
description: IP address used by the UPF's Core interface.
core-gateway-ip:
type: string
default: 192.168.250.1
description: Gateway IP address to the Core Network.
core-interface-mtu-size:
type: int
description: |
MTU for the core interface (1200 <= MTU <= 65535) in bytes.
If not specified, Multus will use its default value (typically 1500).
external-upf-hostname:
type: string
description: |-
description: |
Externally accessible FQDN for the UPF.
If not provided, it will default to the LoadBalancer Service hostname.
If that is not available, it will default to the internal
Expand Down
42 changes: 27 additions & 15 deletions lib/charms/kubernetes_charm_libraries/v0/multus.py
Gmerold marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ def __init__(self, *args):
container_name=self._container_name,
cap_net_admin=True,
privileged=True,
network_annotations=[
network_annotations_func=[
NetworkAnnotation(
name=NETWORK_ATTACHMENT_DEFINITION_NAME,
interface=INTERFACE_NAME,
Expand Down Expand Up @@ -94,7 +94,7 @@ def _on_config_changed(self, event: EventBase):
import logging
from dataclasses import asdict, dataclass
from json.decoder import JSONDecodeError
from typing import Callable, Union
from typing import Callable, List, Optional, Union

import httpx
from lightkube import Client
Expand Down Expand Up @@ -123,7 +123,7 @@ def _on_config_changed(self, event: EventBase):

# Increment this PATCH version before using `charmcraft publish-lib` or reset
# to 0 if you are raising the major API version
LIBPATCH = 8
LIBPATCH = 9


logger = logging.getLogger(__name__)
Expand All @@ -148,10 +148,20 @@ def __eq__(self, other):
class NetworkAnnotation:
"""NetworkAnnotation."""

NETWORK_ANNOTATION_RESOURCE_KEY = "k8s.v1.cni.cncf.io/networks"

name: str
interface: str
mac: Optional[str] = None
ips: Optional[List[str]] = None

def dict(self) -> dict:
"""Returns a NetworkAnnotation in the form of a dictionary.

dict = asdict
Returns:
dict: Dictionary representation of the NetworkAnnotation
"""
return {key: value for key, value in asdict(self).items() if value}


class KubernetesMultusError(Exception):
Expand Down Expand Up @@ -351,7 +361,7 @@ def patch_statefulset(
template=PodTemplateSpec(
metadata=ObjectMeta(
annotations={
"k8s.v1.cni.cncf.io/networks": json.dumps(
NetworkAnnotation.NETWORK_ANNOTATION_RESOURCE_KEY: json.dumps(
[
network_annotation.dict()
for network_annotation in network_annotations
Expand Down Expand Up @@ -446,22 +456,23 @@ def _pod_is_patched(
return False
return True

@staticmethod
def _annotations_contains_multus_networks(
self, annotations: dict, network_annotations: list[NetworkAnnotation]
annotations: dict, network_annotations: list[NetworkAnnotation]
) -> bool:
if "k8s.v1.cni.cncf.io/networks" not in annotations:
if NetworkAnnotation.NETWORK_ANNOTATION_RESOURCE_KEY not in annotations:
return False
try:
if json.loads(annotations["k8s.v1.cni.cncf.io/networks"]) != [
if json.loads(annotations[NetworkAnnotation.NETWORK_ANNOTATION_RESOURCE_KEY]) != [
network_annotation.dict() for network_annotation in network_annotations
]:
return False
except JSONDecodeError:
return False
return True

@staticmethod
def _container_security_context_is_set(
self,
containers: list[Container],
container_name: str,
cap_net_admin: bool,
Expand Down Expand Up @@ -494,7 +505,7 @@ def __init__(
self,
charm: CharmBase,
network_attachment_definitions_func: Callable[[], list[NetworkAttachmentDefinition]],
network_annotations: list[NetworkAnnotation],
network_annotations_func: Callable[[], list[NetworkAnnotation]],
container_name: str,
refresh_event: BoundEvent,
cap_net_admin: bool = False,
Expand All @@ -506,7 +517,8 @@ def __init__(
charm: Charm object
network_attachment_definitions_func: A callable to a function returning a list of
`NetworkAttachmentDefinition` to be created.
network_annotations: List of NetworkAnnotation.
network_annotations_func: A callable to a function returning a list
of `NetworkAnnotation` to be added to the container.
container_name: Container name
cap_net_admin: Container requires NET_ADMIN capability
privileged: Container requires privileged security context
Expand All @@ -516,7 +528,7 @@ def __init__(
super().__init__(charm, "kubernetes-multus")
self.kubernetes = KubernetesClient(namespace=self.model.name)
self.network_attachment_definitions_func = network_attachment_definitions_func
self.network_annotations = network_annotations
self.network_annotations_func = network_annotations_func
self.container_name = container_name
self.cap_net_admin = cap_net_admin
self.privileged = privileged
Expand All @@ -534,7 +546,7 @@ def _configure_multus(self, event: BoundEvent) -> None:
if not self._statefulset_is_patched():
self.kubernetes.patch_statefulset(
name=self.model.app.name,
network_annotations=self.network_annotations,
network_annotations=self.network_annotations_func(),
container_name=self.container_name,
cap_net_admin=self.cap_net_admin,
privileged=self.privileged,
Expand Down Expand Up @@ -608,7 +620,7 @@ def _statefulset_is_patched(self) -> bool:
"""Returns whether statefuset is patched with network annotations and capabilities."""
return self.kubernetes.statefulset_is_patched(
name=self.model.app.name,
network_annotations=self.network_annotations,
network_annotations=self.network_annotations_func(),
container_name=self.container_name,
cap_net_admin=self.cap_net_admin,
privileged=self.privileged,
Expand All @@ -618,7 +630,7 @@ def _pod_is_ready(self) -> bool:
"""Returns whether pod is ready with network annotations and capabilities."""
return self.kubernetes.pod_is_ready(
pod_name=self._pod,
network_annotations=self.network_annotations,
network_annotations=self.network_annotations_func(),
container_name=self.container_name,
cap_net_admin=self.cap_net_admin,
privileged=self.privileged,
Expand Down
3 changes: 3 additions & 0 deletions requirements.txt
Original file line number Diff line number Diff line change
@@ -1,7 +1,10 @@
httpx
gruyaume marked this conversation as resolved.
Show resolved Hide resolved
ipaddress
ops
jinja2
lightkube
lightkube-models
macaddress
pydantic
pytest-interface-tester
PyYAML>=6.0.1
Expand Down
Loading
Loading