Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ICMP with not response found #252

Open
lucascgo opened this issue Apr 24, 2024 · 7 comments
Open

ICMP with not response found #252

lucascgo opened this issue Apr 24, 2024 · 7 comments

Comments

@lucascgo
Copy link

I have a setup with two VFs binded to DPDK. I want to do a simple ping test using one of this VFs (port 0), inside my network, like this:

+-------------+        +-------------+      +-------------+      
| DPDK Port 0 |--------|   Gateway   |------|    DUT      |      
| 10.60.1.45  |        |  10.60.1.1  |      | 10.60.1.151 |      
+-------------+        +-------------+      +-------------+

Below the pktgen startup:

$ sudo pktgen -l 0-1 -n 2 -- -P -T -N -m "1.[0-1]"                                                                                                    
                                                                                                                                                                                       
*** Copyright(c) <2010-2023>, Intel Corporation. All rights reserved.                                                                                                                  
*** Pktgen  created by: Keith Wiles -- >>> Powered by DPDK <<<                                                                                                                         
                                                                                                                                                                                       
EAL: Detected CPU lcores: 16                                                                                                                                                           
EAL: Detected NUMA nodes: 1                                                                                                                                                            
EAL: Detected shared linkage of DPDK                                                                                                                                                   
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket                                                                                                                                  
EAL: Selected IOVA mode 'VA'                                                                                                                                                           
EAL: VFIO support initialized                                                                                                                                                          
EAL: Using IOMMU type 1 (Type 1)                                                                                                                                                       
EAL: Probe PCI driver: net_bnxt (14e4:16dc) device: 0000:65:02.0 (socket 0)                                                                                                            
EAL: Probe PCI driver: net_bnxt (14e4:16dc) device: 0000:65:03.0 (socket 0)                                                                                                            
TELEMETRY: No legacy callbacks, legacy socket not created                                                                                                                              
Lua 5.3.3  Copyright (C) 1994-2016 Lua.org, PUC-Rio                                                                                                                                    
    0: net_bnxt        0      0   14e4:16dc/0000:65:02.0                                                                                                                               
    1: net_bnxt        0      0   14e4:16dc/0000:65:03.0                                                                                                                               
                                                                                                                                                                                       
Initialize Port 0 -- RxQ 1, TxQ 1                                                                                                                                                      
   Src MAC 62:52:b3:8c:eb:0f <Promiscuous is Enabled>                                                                                                                                  
Initialize Port 1 -- RxQ 1, TxQ 1                                                                                                                                                      
   Src MAC 8a:82:0e:c2:bf:9e <Promiscuous is Enabled>

Below the pktgen running:

| Ports 0-1 of 2   <Main Page>  Copyright(c) <2010-2023>, Intel Corporation
  Port:Flags        : 0:PE-I---      Single 1:P------      Single
Link State          :         <UP-10000-FD>         <UP-10000-FD>     ---Total Rate---
Pkts/s Rx           :                     1                     0                    1
       Tx           :                     1                     0                    1
MBits/s Rx/Tx       :                   0/0                   0/0                  0/0
Pkts/s Rx Max       :                     4                     4                    5
       Tx Max       :                     4                     0                    4
Broadcast           :                     0                     0
Multicast           :                  3648                 14144
Sizes 64            :                  3520                 19008
      65-127        :                  9472                  1408
      128-255       :                     0                     0
      256-511       :                     0                   704
      512-1023      :                     0                     0
      1024-1518     :                     0                     0
Runts/Jumbos        :                 640/0                   0/0
ARP/ICMP Pkts       :             4032/8704               13440/0
Errors Rx/Tx        :                   0/0                   0/0
Total Rx Pkts       :                   212                   330
      Tx Pkts       :                   144                     0
      Rx/Tx MBs     :                   0/0                   0/0
TCP Flags           :                .A....                .A....
TCP Seq/Ack         :           74616/74640           74616/74640
Pattern Type        :               abcd...               abcd...
Tx Count/% Rate     :         Forever /100%         Forever /100%
Pkt Size/Rx:Tx Burst:           64 / 64: 64           64 / 64: 64
TTL/Port Src/Dest   :        64/ 1234/ 5678        64/ 1234/ 5678
Pkt Type:VLAN ID    :       IPv4 / TCP:0001       IPv4 / TCP:0001
802.1p CoS/DSCP/IPP :             0/  0/  0             0/  0/  0
VxLAN Flg/Grp/vid   :      0000/    0/    0      0000/    0/    0
IP  Destination     :           10.60.1.151           192.168.0.1
    Source          :         10.60.1.45/24        192.168.1.1/24
MAC Destination     :     52:54:00:80:1f:bd     62:52:b3:8c:eb:0f
    Source          :     62:52:b3:8c:eb:0f     8a:82:0e:c2:bf:9e
NUMA/Vend:ID/PCI    :0/14e4:16dc/0000:65:020/14e4:16dc/0000:65:03.0
-- Pktgen 24.03.1 (DPDK 24.07.0-rc0)  Powered by DPDK  (pid:404102) -----------

Pktgen:/> set 0 src ip 10.60.1.45/24
Pktgen:/> set 0 src mac 62:52:b3:8c:eb:0f
Pktgen:/> set 0 dst ip 10.60.1.151
Pktgen:/> set 0 dst mac 52:54:00:80:1f:bd
Pktgen:/> enable 0 icmp
Pktgen:/> enable 0 process
Pktgen:/> start 0 arp request
Pktgen:/> ping4 0
Pktgen:/>

The DUT do not receive the response:

$ ping -I ens10 10.60.1.45
PING 10.60.1.45 (10.60.1.45) from 10.60.1.151 ens10: 56(84) bytes of data.
--- 10.60.1.45 ping statistics ---
1487 packets transmitted, 0 received, 100% packet loss, time 1521653ms

Looking with tcpdump, I received the packets, but they appear to be incomplete:

22:22:29.644258 IP 10.60.1.151 > 10.60.1.45: ICMP echo request, id 265, seq 101, length 64
22:22:29.644547 IP 10.60.1.45 > 10.60.1.151: ICMP echo reply, id 265, seq 101, length 64
22:22:30.668310 IP 10.60.1.151 > 10.60.1.45: ICMP echo request, id 265, seq 102, length 64
22:22:30.668626 IP 10.60.1.45 > 10.60.1.151: ICMP echo reply, id 265, seq 102, length 64
22:22:31.692325 IP 10.60.1.151 > 10.60.1.45: ICMP echo request, id 265, seq 103, length 64
22:22:31.692602 IP 10.60.1.45 > 10.60.1.151: ICMP echo reply, id 265, seq 103, length 64
22:22:32.716315 IP 10.60.1.151 > 10.60.1.45: ICMP echo request, id 265, seq 104, length 64
22:22:32.716840 IP 10.60.1.45 > 10.60.1.151: ICMP echo reply, id 265, seq 104, length 64
22:22:33.740396 IP 10.60.1.151 > 10.60.1.45: ICMP echo request, id 265, seq 105, length 64
22:22:33.740752 IP 10.60.1.45 > 10.60.1.151: ICMP echo reply, id 265, seq 105, length 64

The Wireshark:
Screenshot from 2024-04-24 19-27-23

What could it be?
@KeithWiles
Copy link
Collaborator

Maybe wrong here, but the Gateway needs two ports and each port needs to be on a different subnet to allow the gateway to route between the two networks, which also means the DPDK and DUT needs to be on different subnets. I also assume the three devices are the only three connected and we do not have a switch connecting all three devices. Maybe the gateway is really a bridge or L2 switch then this will be fine.

10.60.1.45/24 <==> 10.60.1.1/24 GW 10.60.2.1/24 <===> 10.60.2.151/24

It appears the the ping response is working, but I see a lot of Runts on port 0. I see the ICMP counter is 8704 does that counter increase when doing a ping4?

If you stop DPDK rebind the port to the linux kernel and use ping from that machine does ping work?

@lucascgo
Copy link
Author

Hi @KeithWiles, thanks for your reply.

I wrote gateway, but is a real switch.
The infrastructure is like this:

+-------------+       +-------------+
| DPDK Port 0 |       | DUT Kernel  |
| 10.60.1.45  |       | 10.60.1.151 |
+-------------+       +-------------+
       |                     |       
       |        VLAN         |       
+-----------------------------------+
|               SWITCH              |
|             10.60.1.1             |
+-----------------------------------+

Yes, the ICMP Pkts counter increase with ping4 0 command. And the Runts appear only in the first time when I change from Kernel to DPDK and run the Pktgen.

If I rebind to the linux kernel the machine with DPDK:

$ sudo dpdk-devbind.py -b bnxt_en 0000:65:02.0 0000:65:03.0
$ sudo ip addr add 10.60.1.45/24 dev ens2f0v0

And I do the ping from DUT:

$ ping -I ens10 10.60.1.45
PING 10.60.1.45 (10.60.1.45) from 10.60.1.151 ens10: 56(84) bytes of data.
64 bytes from 10.60.1.45: icmp_seq=1 ttl=64 time=0.309 ms
64 bytes from 10.60.1.45: icmp_seq=2 ttl=64 time=0.261 ms
64 bytes from 10.60.1.45: icmp_seq=3 ttl=64 time=0.385 ms
64 bytes from 10.60.1.45: icmp_seq=4 ttl=64 time=0.399 ms

The funny thing is that it wasn't necessary the command sudo ip link set ens2f0v0 up to ping works. And the interface is DOWN on system:

$ ip -c a | grep ens
3: ens2f0np0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    inet 10.60.1.44/24 scope global ens2f0np0
4: ens2f1np1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    inet 10.30.1.44/24 scope global ens2f1np1
365: ens2f0v0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    inet 10.60.1.45/24 scope global ens2f0v0
366: ens2f1v0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000

The interfaces:

┌───────────┬──────────────┬───────────────────────┐
│ ens2f0np0 │ PF1          │ Switch VLAN 10.60.1.x │
├───────────┼──────────────┼───────────────────────┤
│ ens2f1np1 │ PF2          │ Switch VLAN 10.30.1.x │
├───────────┼──────────────┼───────────────────────┤
│ ens2f0v0  │ VF1 from PF1 │ Switch VLAN 10.60.1.x │
├───────────┼──────────────┼───────────────────────┤
│ ens2f1v0  │ VF2 from PF2 │ Switch VLAN 10.30.1.x │
└───────────┴──────────────┴───────────────────────┘

@lucascgo lucascgo reopened this Apr 25, 2024
@KeithWiles
Copy link
Collaborator

Can you send me the dump.pcap file you are showing above or one like it?

@lucascgo
Copy link
Author

Of course, here: dumps.tar.gz

@KeithWiles
Copy link
Collaborator

Looks like the checksum is wrong in the reply.

@lucascgo
Copy link
Author

And why this wrong checksum? Maybe version or compilation issue?

@KeithWiles
Copy link
Collaborator

Some times wireshark reports checksum wrong because the NIC is doing the checksuming in hardware and it not a real problem. Not sure why in your case.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@lucascgo @KeithWiles