Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

IB test fails for ibv_rc_pingpong and ibv_read_bw under RHEL6 #47

Open
cywong opened this issue Feb 13, 2016 · 1 comment
Open

IB test fails for ibv_rc_pingpong and ibv_read_bw under RHEL6 #47

cywong opened this issue Feb 13, 2016 · 1 comment

Comments

@cywong
Copy link

cywong commented Feb 13, 2016

I followed instruction from https://github.com/SoftRoCE/rxe-dev/wiki/rxe-dev:-Home to configure and
install SoftROCE, rping and ibv_write_bw works but no response for ibv_rc_pingpong and ibv_read_bw.

How to troubleshoot this issue?

Environment
Two VMs of CentOS 6.7 (Using rxe 4.0.0+ kernel)
SoftROCE:
rxe-dev: ec91e94
librxe-dev: fd829a95921cfc3f8e1108fcf84d7e8ac64f4bca

rxe_cfg status

Name  Link  Driver  Speed  NMTU  IPv4_addr        RDEV  RMTU
eth0  yes   e1000          1500  192.168.122.219  rxe0  1024  (3)

ibv_rc_pingpong

[root@test]# ibv_rc_pingpong -d rxe0 -g 0 <Peer IP>
  local address:  LID 0x0000, QPN 0x000011, PSN 0x017d96, GID fe80::5054:ff:fe6c:16f1
  remote address: LID 0x0000, QPN 0x000011, PSN 0xd7af63, GID fe80::5054:ff:fead:b03
**--Hang--**

ibv_read_bw

Device not recognized to implement inline feature. Disabling it
.....
---------------------------------------------------------------------------------------
                    RDMA_Read BW Test
 Dual-port       : OFF      Device         : rxe0
 Number of qps   : 1        Transport type : IB
 Connection type : RC       Using SRQ      : OFF
 CQ Moderation   : 100
 Mtu             : 1024[B]
 Link type       : Ethernet
 Gid index       : 0
 Outstand reads  : 128
 rdma_cm QPs     : OFF
 Data ex. method : Ethernet
---------------------------------------------------------------------------------------
 local address: LID 0000 QPN 0x0011 PSN 0xcb231f OUT 0x80 RKey 0x000411 VAddr 0x007f696bb5e000
 GID: 00:00:00:00:00:00:00:00:00:00:255:255:192:168:122:30
 remote address: LID 0000 QPN 0x0011 PSN 0x71e12d OUT 0x80 RKey 0x000411 VAddr 0x007f83f17ff000
 GID: 00:00:00:00:00:00:00:00:00:00:255:255:192:168:122:219
---------------------------------------------------------------------------------------
 #bytes     #iterations    BW peak[MB/sec]    BW average[MB/sec]   MsgRate[Mpps]
**--Hang--**

/var/log/messages
test kernel: no qp matches qpn 0x11

@cywong cywong changed the title IB test hang for ibv_rc_pingpong and ibv_read_bw under RHEL6 IB test fails for ibv_rc_pingpong and ibv_read_bw under RHEL6 Feb 13, 2016
@psebexen
Copy link
Contributor

I notice in your pasted input below ibv_rc_pingpong you include the argument '-g 0'. If you have not already, please try running the tests with '-g 1' on both machines.
You should be able list all of the GID entries with cat /sys/class/infiniband/rxe0/ports/1/gids/*. The first entry (number 0) is the link local GID only; entry 1 should be used for testing purposes.

The ibv_read_bw test may benefit from adding the '-R' flag. This will attempt to connect using rdma_cm instead of the default Ethernet approach.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants