Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Adding DPDK support in UPF #57

Merged
merged 21 commits into from
Jan 9, 2024
Merged

Conversation

Gmerold
Copy link
Contributor

@Gmerold Gmerold commented Nov 16, 2023

Description

This PR adds DPDK support to UPF.

Most important changes:

  1. NADs creation has been refactored to support vfioveth type required by DPDK
  2. StatefulSet network annotations creation has been refactored
  3. NetworkAnnotation dataclass has 2 additional fields required by DPDK: mac and ips
  4. DPDKStatefulSetUpdater has been added to handle StatefulSet updates required to run UPF in DPDK mode
  5. New charm configs added

Design decissions:

  1. DPDKStatefulSetUpdater is not a library, because at this point we are not aware of anyone else who could potentially leverage such a library. Regular class seems good enough at this point.
  2. DPDKStatefulSetUpdater makes bess container a privileged container. That's because in DPDK mode, bess needs to access physical address of HugePages. Ideally IPC_LOCK capability should be enough, but it's not. Similar issue has been described here.

Checklist:

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have made corresponding changes to the documentation
  • I have added tests that validate the behaviour of the software
  • I validated that new and existing unit tests pass locally with my changes
  • Any dependent changes have been merged and published in downstream modules
  • I have bumped the version of the library

@Gmerold Gmerold requested a review from a team as a code owner November 16, 2023 13:02
@Gmerold Gmerold changed the title Feat: Adding DPDK support in UPF feat: Adding DPDK support in UPF Nov 16, 2023
lib/charms/kubernetes_charm_libraries/v0/multus.py Outdated Show resolved Hide resolved
src/charm.py Outdated Show resolved Hide resolved
src/dpdk_statefulset_updater.py Outdated Show resolved Hide resolved
requirements.txt Outdated Show resolved Hide resolved
src/dpdk_statefulset_updater.py Outdated Show resolved Hide resolved
src/charm.py Outdated Show resolved Hide resolved
config.yaml Outdated Show resolved Hide resolved
requirements.txt Show resolved Hide resolved
src/dpdk_statefulset_updater.py Outdated Show resolved Hide resolved
src/dpdk_statefulset_updater.py Outdated Show resolved Hide resolved
src/dpdk_statefulset_updater.py Outdated Show resolved Hide resolved
src/dpdk_statefulset_updater.py Outdated Show resolved Hide resolved
src/dpdk_statefulset_updater.py Outdated Show resolved Hide resolved
tests/unit/test_charm.py Outdated Show resolved Hide resolved
tests/unit/test_charm.py Outdated Show resolved Hide resolved
@Gmerold Gmerold requested a review from gruyaume November 28, 2023 07:25
tests/unit/test_charm.py Outdated Show resolved Hide resolved
@gatici
Copy link
Contributor

gatici commented Nov 28, 2023

After deploying the charm with DPDK mode, bessd container can not start with following Permission Deny error.

2023-11-28T19:58:45.592Z [pebble] Service "bessd" starting: /bin/bessd -f -grpc-url=0.0.0.0:10514 
2023-11-28T19:58:45.706Z [bessd] F1128 19:58:45.706557    53 bessd.cc:211] flock(pidfile=/var/run/bessd.pid): Permission denied [13]
2023-11-28T19:58:45.706Z [bessd] *** Check failure stack trace: ***
2023-11-28T19:58:45.727Z [bessd] F1128 19:58:45.727789    53 debug.cc:407] Backtrace (recent calls first) ---
2023-11-28T19:58:45.727Z [bessd] (0): /bin/bessd(_ZN6google15ErrnoLogMessageD1Ev+0xae) [0x55de3962323e]
2023-11-28T19:58:45.727Z [bessd] (1): /bin/bessd(_ZN4bess5bessd21TryAcquirePidfileLockEi+0x163) [0x55de38cf6863]
2023-11-28T19:58:45.727Z [bessd] (2): /bin/bessd(_ZN4bess5bessd19CheckUniqueInstanceERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0x49) [0x55de38cf6989]
2023-11-28T19:58:45.727Z [bessd] (3): /bin/bessd(main+0xe3) [0x55de38a65d73]
2023-11-28T19:58:45.727Z [bessd] (4): /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf1) [0x7f92cd8f5081]
2023-11-28T19:58:45.727Z [bessd] (5): /bin/bessd(_start+0x2c) [0x55de38cbbefc]
2023-11-28T19:58:45.727Z [bessd] *** Check failure stack trace: ***

src/charm.py Show resolved Hide resolved
@Gmerold
Copy link
Contributor Author

Gmerold commented Nov 29, 2023

After deploying the charm with DPDK mode, bessd container can not start with following Permission Deny error.

2023-11-28T19:58:45.592Z [pebble] Service "bessd" starting: /bin/bessd -f -grpc-url=0.0.0.0:10514 
2023-11-28T19:58:45.706Z [bessd] F1128 19:58:45.706557    53 bessd.cc:211] flock(pidfile=/var/run/bessd.pid): Permission denied [13]
2023-11-28T19:58:45.706Z [bessd] *** Check failure stack trace: ***
2023-11-28T19:58:45.727Z [bessd] F1128 19:58:45.727789    53 debug.cc:407] Backtrace (recent calls first) ---
2023-11-28T19:58:45.727Z [bessd] (0): /bin/bessd(_ZN6google15ErrnoLogMessageD1Ev+0xae) [0x55de3962323e]
2023-11-28T19:58:45.727Z [bessd] (1): /bin/bessd(_ZN4bess5bessd21TryAcquirePidfileLockEi+0x163) [0x55de38cf6863]
2023-11-28T19:58:45.727Z [bessd] (2): /bin/bessd(_ZN4bess5bessd19CheckUniqueInstanceERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0x49) [0x55de38cf6989]
2023-11-28T19:58:45.727Z [bessd] (3): /bin/bessd(main+0xe3) [0x55de38a65d73]
2023-11-28T19:58:45.727Z [bessd] (4): /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf1) [0x7f92cd8f5081]
2023-11-28T19:58:45.727Z [bessd] (5): /bin/bessd(_start+0x2c) [0x55de38cbbefc]
2023-11-28T19:58:45.727Z [bessd] *** Check failure stack trace: ***

Is this strictly confined Microk8s that you're running there? If yes, it won't work.

@gatici
Copy link
Contributor

gatici commented Nov 29, 2023

After deploying the charm with DPDK mode, bessd container can not start with following Permission Deny error.

2023-11-28T19:58:45.592Z [pebble] Service "bessd" starting: /bin/bessd -f -grpc-url=0.0.0.0:10514 
2023-11-28T19:58:45.706Z [bessd] F1128 19:58:45.706557    53 bessd.cc:211] flock(pidfile=/var/run/bessd.pid): Permission denied [13]
2023-11-28T19:58:45.706Z [bessd] *** Check failure stack trace: ***
2023-11-28T19:58:45.727Z [bessd] F1128 19:58:45.727789    53 debug.cc:407] Backtrace (recent calls first) ---
2023-11-28T19:58:45.727Z [bessd] (0): /bin/bessd(_ZN6google15ErrnoLogMessageD1Ev+0xae) [0x55de3962323e]
2023-11-28T19:58:45.727Z [bessd] (1): /bin/bessd(_ZN4bess5bessd21TryAcquirePidfileLockEi+0x163) [0x55de38cf6863]
2023-11-28T19:58:45.727Z [bessd] (2): /bin/bessd(_ZN4bess5bessd19CheckUniqueInstanceERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0x49) [0x55de38cf6989]
2023-11-28T19:58:45.727Z [bessd] (3): /bin/bessd(main+0xe3) [0x55de38a65d73]
2023-11-28T19:58:45.727Z [bessd] (4): /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf1) [0x7f92cd8f5081]
2023-11-28T19:58:45.727Z [bessd] (5): /bin/bessd(_start+0x2c) [0x55de38cbbefc]
2023-11-28T19:58:45.727Z [bessd] *** Check failure stack trace: ***

Is this strictly confined Microk8s that you're running there? If yes, it won't work.

Yes, Microk8s version is 1.27-strict/stable.

README.md Show resolved Hide resolved
config.yaml Outdated Show resolved Hide resolved
@Gmerold Gmerold merged commit adf2dd5 into main Jan 9, 2024
23 checks passed
@Gmerold Gmerold deleted the TELCO-464-adding-dpdk-support-to-upf branch January 9, 2024 16:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants