Skip to content

[LE-3812] RDMA/mana_ib: use the correct page table index based on hardware page size #511

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

shreeya-patel98
Copy link

@shreeya-patel98 shreeya-patel98 commented Aug 19, 2025

Commit message

RDMA/mana_ib: use the correct page table index based on hardware page size
jira LE-3812
commit-author Long Li <[email protected]>
commit 9e517a8
upstream-diff There were build failures and conflicts due to the
following missing commits :-
e849aad ("RDMA/mana_ib: Fix bug in creation of dma regions")
071bade ("net: mana: Add support for page sizes other than 4KB on ARM64")
5861cd1 ("net: mana: Enable MANA driver on ARM64 with 4K page size")

MANA hardware uses 4k page size. When calculating the page table index,
it should use the hardware page size, not the system page size.

	Cc: [email protected]
Fixes: 0266a17 ("RDMA/mana_ib: Add a driver for Microsoft Azure Network Adapter")
	Signed-off-by: Long Li <[email protected]>
Link: https://patch.msgid.link/[email protected]
	Signed-off-by: Leon Romanovsky <[email protected]>
(cherry picked from commit 9e517a8)
	Signed-off-by: Shreeya Patel <[email protected]>

----------------------------------------------------------------------------------------------------------
RDMA/mana_ib: use the correct page size for mapping user-mode doorbell page
jira LE-3812
commit-author Long Li <[email protected]>
commit 4a3b99b

When mapping doorbell page from user-mode, the driver should use the system
page size as this memory is allocated via mmap() from user-mode.

	Cc: [email protected]
Fixes: 0266a17 ("RDMA/mana_ib: Add a driver for Microsoft Azure Network Adapter")
	Signed-off-by: Long Li <[email protected]>
Link: https://patch.msgid.link/[email protected]
	Signed-off-by: Leon Romanovsky <[email protected]>
(cherry picked from commit 4a3b99b)
	Signed-off-by: Shreeya Patel <[email protected]>
	

Kernel build logs

/mnt/scratch/workspace/sig-cloud-8/kernel-src-tree
Skipping make mrproper
[TIMER]{MRPROPER}: 0s
x86_64 architecture detected, copying config
'configs/kernel-x86_64.config' -> '.config'
Setting Local Version for build
CONFIG_LOCALVERSION="-shreeya_sig-cloud-8_4.18.0-553.62.1.el8_10-fb6d8f948c9f"
Making olddefconfig
scripts/kconfig/conf  --olddefconfig Kconfig
#
# configuration written to .config
#
Starting Build
scripts/kconfig/conf  --syncconfig Kconfig
  UPD     include/config/kernel.release
  DESCEND objtool
  DESCEND bpf/resolve_btfids
  UPD     include/generated/utsrelease.h
  CALL    scripts/checksyscalls.sh
  CHK     include/generated/compile.h
  CC      init/version.o
  AR      init/built-in.a
  CC      kernel/sys.o
  CC      crypto/fips.o
  CC      drivers/base/firmware_loader/main.o
  AR      crypto/built-in.a
  AR      drivers/base/firmware_loader/built-in.a
  AR      drivers/base/built-in.a
  CC      net/ethtool/ioctl.o
  CC      kernel/trace/trace.o
  
  <--snip-->
  
  INSTALL sound/usb/usx2y/snd-usb-usx2y.ko
  INSTALL sound/virtio/virtio_snd.ko
  INSTALL sound/x86/snd-hdmi-lpe-audio.ko
  INSTALL sound/xen/snd_xen_front.ko
  INSTALL virt/lib/irqbypass.ko
  DEPMOD  4.18.0-shreeya_sig-cloud-8_4.18.0-553.62.1.el8_10-fb6d8f948c9f+
[TIMER]{MODULES}: 8s
Making Install
sh ./arch/x86/boot/install.sh 4.18.0-shreeya_sig-cloud-8_4.18.0-553.62.1.el8_10-fb6d8f948c9f+ arch/x86/boot/bzImage \
	System.map "/boot"
[TIMER]{INSTALL}: 17s
Checking kABI
kABI check passed
Setting Default Kernel to /boot/vmlinuz-4.18.0-shreeya_sig-cloud-8_4.18.0-553.62.1.el8_10-fb6d8f948c9f+ and Index to 1
The default is /boot/loader/entries/753870ec7b134d7582d817dd9bf35992-4.18.0-shreeya_sig-cloud-8_4.18.0-553.62.1.el8_10-fb6d8f948c9f+.conf with index 1 and kernel /boot/vmlinuz-4.18.0-shreeya_sig-cloud-8_4.18.0-553.62.1.el8_10-fb6d8f948c9f+
The default is /boot/loader/entries/753870ec7b134d7582d817dd9bf35992-4.18.0-shreeya_sig-cloud-8_4.18.0-553.62.1.el8_10-fb6d8f948c9f+.conf with index 1 and kernel /boot/vmlinuz-4.18.0-shreeya_sig-cloud-8_4.18.0-553.62.1.el8_10-fb6d8f948c9f+
Generating grub configuration file ...
done
Hopefully Grub2.0 took everything ... rebooting after time metrices
[TIMER]{MRPROPER}: 0s
[TIMER]{BUILD}: 269s
[TIMER]{MODULES}: 8s
[TIMER]{INSTALL}: 17s
[TIMER]{TOTAL} 297s
Rebooting in 10 seconds

kernel-build.log

Kselftests

shreeya@spatel-dev-bom ~/c/w/sig-cloud-8> grep '^ok ' kselftest-before.log | wc -l && grep '^ok ' kselftest-after.log | wc -l
217
217
shreeya@spatel-dev-bom ~/c/w/sig-cloud-8> grep '^not ok ' kselftest-before.log | wc -l && grep '^not ok ' kselftest-after.log | wc -l
57
57

kselftest-after.log
kselftest-before.log

Kernel Commit Check

shreeya@spatel-dev-bom ~/c/w/s/kernel-src-tree-tools (mainline) [1]> python3 check_kernel_commits.py --repo ../kernel-src-tree --pr_branch {shreeya}_sig-cloud-8/4.18.0-553.62.1.el8_10 --base_branch origin/sig-cloud-8/4.18.0-553.62.1.el8_10 --markdown
> ✅ **All referenced commits exist upstream and have no Fixes: tags.**

jira LE-3812
commit-author Haiyang Zhang <[email protected]>
commit 40a1d11

Change the Kconfig dependency, so this driver can be built and run on ARM64
with 4K page size.
16/64K page sizes are not supported yet.

	Signed-off-by: Haiyang Zhang <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
	Signed-off-by: Jakub Kicinski <[email protected]>
(cherry picked from commit 40a1d11)
	Signed-off-by: Shreeya Patel <[email protected]>
jira LE-3812
commit-author Haiyang Zhang <[email protected]>
commit 382d174

As defined by the MANA Hardware spec, the queue size for DMA is 4KB
minimal, and power of 2. And, the HWC queue size has to be exactly
4KB.

To support page sizes other than 4KB on ARM64, define the minimal
queue size as a macro separately from the PAGE_SIZE, which we always
assumed it to be 4KB before supporting ARM64.

Also, add MANA specific macros and update code related to size
alignment, DMA region calculations, etc.

	Signed-off-by: Haiyang Zhang <[email protected]>
	Reviewed-by: Michael Kelley <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
	Signed-off-by: Jakub Kicinski <[email protected]>
(cherry picked from commit 382d174)
	Signed-off-by: Shreeya Patel <[email protected]>
jira LE-3812
commit-author Konstantin Taranov <[email protected]>
commit e02497f

Use ib_umem_dma_offset() helper to calculate correct dma offset.

Fixes: 0266a17 ("RDMA/mana_ib: Add a driver for Microsoft Azure Network Adapter")
	Signed-off-by: Konstantin Taranov <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
	Signed-off-by: Leon Romanovsky <[email protected]>
(cherry picked from commit e02497f)
	Signed-off-by: Shreeya Patel <[email protected]>
…l page

jira LE-3812
commit-author Long Li <[email protected]>
commit 4a3b99b

When mapping doorbell page from user-mode, the driver should use the system
page size as this memory is allocated via mmap() from user-mode.

	Cc: [email protected]
Fixes: 0266a17 ("RDMA/mana_ib: Add a driver for Microsoft Azure Network Adapter")
	Signed-off-by: Long Li <[email protected]>
Link: https://patch.msgid.link/[email protected]
	Signed-off-by: Leon Romanovsky <[email protected]>
(cherry picked from commit 4a3b99b)
	Signed-off-by: Shreeya Patel <[email protected]>
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR addresses page size handling in the MANA RDMA driver to correctly use hardware page size (4KB) instead of system page size for hardware interactions, while maintaining system page size for user-mode mappings.

  • Introduces MANA-specific page size constants and macros for 4KB hardware pages
  • Updates all DMA region and queue size calculations to use hardware page size
  • Fixes user-mode doorbell page mapping to use system page size
  • Enables MANA driver support on ARM64 with non-big-endian configurations

Reviewed Changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
include/net/mana/gdma.h Adds MANA page size constants and macros for 4KB hardware pages
include/net/mana/mana.h Updates EQ_SIZE to use MANA_PAGE_SIZE instead of PAGE_SIZE
drivers/net/ethernet/microsoft/mana/*.c Replaces PAGE_* macros with MANA_PAGE_* equivalents for hardware operations
drivers/net/ethernet/microsoft/Kconfig Enables MANA driver on ARM64 (non-big-endian) platforms
drivers/infiniband/hw/mana/main.c Fixes page table index calculation and user-mode doorbell mapping

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

Copy link
Collaborator

@PlaidCat PlaidCat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in the final commit we don't need to know that there was context outside of this commit.

upstream-diff There were build failures and conflicts due to the
following missing commits :-
https://github.com/ctrliq/kernel-src-tree/commit/e849aadfd5936e4e717107cccc811e34280c0bd8 ("RDMA/mana_ib: Fix bug in creation of dma regions")
https://github.com/ctrliq/kernel-src-tree/commit/071badedd6b952ae335c9a5a435e031dcc42793d ("net: mana: Add support for page sizes other than 4KB on ARM64")
https://github.com/ctrliq/kernel-src-tree/commit/5861cd1208d91f2b9adcb99ef2ffd0c209972496 ("net: mana: Enable MANA driver on ARM64 with 4K page size")

This is just for why is the upstream differnt from this commit. The context, correctly called out is for the PR.

… size

jira LE-3812
commit-author Long Li <[email protected]>
commit 9e517a8

MANA hardware uses 4k page size. When calculating the page table index,
it should use the hardware page size, not the system page size.

	Cc: [email protected]
Fixes: 0266a17 ("RDMA/mana_ib: Add a driver for Microsoft Azure Network Adapter")
	Signed-off-by: Long Li <[email protected]>
Link: https://patch.msgid.link/[email protected]
	Signed-off-by: Leon Romanovsky <[email protected]>
(cherry picked from commit 9e517a8)
	Signed-off-by: Shreeya Patel <[email protected]>
@shreeya-patel98 shreeya-patel98 force-pushed the {shreeya}_sig-cloud-8/4.18.0-553.62.1.el8_10 branch from 84c6f2f to b36d604 Compare August 19, 2025 14:19
Copy link
Collaborator

@PlaidCat PlaidCat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

Copy link

@thefossguy-ciq thefossguy-ciq left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚤

Copy link
Collaborator

@bmastbergen bmastbergen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🥌

@shreeya-patel98 shreeya-patel98 merged commit a54465d into sig-cloud-8/4.18.0-553.62.1.el8_10 Aug 19, 2025
2 checks passed
@shreeya-patel98 shreeya-patel98 deleted the {shreeya}_sig-cloud-8/4.18.0-553.62.1.el8_10 branch August 19, 2025 18:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

4 participants