Skip to content

FRR AFI integration

Renato Westphal edited this page Aug 10, 2017 · 3 revisions

Table of Contents

1. Introduction

This page describes a proof-of-concept demonstration of how the FRR[1] open-source routing stack can be integrated with vMX/AFI[2], leveraging Juniper's forwarding plane technology to create a high-performance software router.

2. Setup

The official AFI git repository[2] contains a detailed step-by-step on how to setup an AFI Development Environment using Docker containers.

For this proof-of-concept demonstration, however, we need to use the following AFI's repository/branch (step 3):

$ git clone https://github.com/opensourcerouting/AFI.git
$ cd AFI
$ git checkout -b linux-routing-stack origin/linux-routing-stack

In this branch, we have the afi-client-linux client developed for this proof of concept, as well as all FRR daemons installed and ready to be used in the container.

3. AFI client

The afi-client-linux program created for this proof of concept is extremely simple. It does two different things:

  • Uses tap interfaces to punt control packets (e.g. ARP, BGP) to the Linux network stack;
  • Listens to netlink messages and mirrors the Linux routing table into vMX's AFT.

A tap interface needs to be created for each one of the eight vMX's ge-0/0/N interfaces and assigned the same MAC of the corresponding VFP interface. Then, whenever a packet is punted by the AFI client, it's written on the tap interface corresponding to the port where the packet arrived (e.g. tap3 for ge-0/0/3 or port 3). At this point, the packet is injected in the Linux network stack (e.g. ARP Request). In the same way, if any packet is sent on this tap interface (e.g. ARP Reply), the AFI client will read this packet and forward it out in the appropriate interface in the host (e.g. veth3). This simple logic allows the AFI client to have a bidirectional communication channel with the Linux host, making it possible to use Linux as a slow path for control packets.

The other thing the AFI client does is to monitor all NEIGHBOR and ROUTE netlink messages from the kernel and replicate the Linux routing table into the fast path. Monitoring NEIGHBOR messages is necessary because, when installing a route using AFI, we need to specify the MAC address of the nexthop and not its IP. This means that the AFI client needs to keep track of the kernel's ARP table in order to be able to map IP addresses into MAC addresses.

4. Testing

In this section we present three simple examples showing the AFI/FRR integration in action. In all tests the goal is the same: make vMX route data packets using routes learned through FRR.

In order to avoid having to bootstrap additional containers or virtual machines, we make use of Linux Network Namespaces to simulate a network topology with completely independent routing stacks. This is what the container's network topology looks like:

    +--------------+--------------------------------------+
    |   vrouter0   |  Global netns   +--------VFP-------+ |
    |              |     +-----+     |                  | |
    |       veth   |veth0|     |     |p0 / ge-0/0/0     | |
    |       o------o-----o     o-----o                  | |
    |       .3     |     +-----+     | .1               | |
    |o lo          |    vmx_link0    |                  | |
    |10.0.255.0/32 | 103.30.0.0/24   |                  | |
    +--------------+                 |                  | |
    |   vrouter1   |                 |                  | |
    |              |     +-----+     |                  | |
    |       veth   |veth1|     |     |p1 / ge-0/0/1     | |
    |       o------o-----o     o-----o                  | |
    |       .3     |     +-----+     | .1               | |
    |o lo          |    vmx_link1    |                  | |
    |10.0.255.1/32 | 103.30.10.0/24  |                  | |
    +--------------+                 |                  | |
    |   vrouter2   |                 |                  | |
    |              |     +-----+     |                  | |
    |       veth   |veth2|     |     |p2 / ge-0/0/2     | |
    |       o------o-----o     o-----o                  | |
    |       .3     |     +-----+     | .1               | |
    |o lo          |    vmx_link2    |                  | |
    |10.0.255.2/32 | 103.30.20.0/24  |                  | |
    +--------------+                 |                  | |
    |   vrouter3   |                 |                  | |
    |              |     +-----+     |                  | |
    |       veth   |veth3|     |     |p3 / ge-0/0/3     | |
    |       o------o-----o     o-----o                  | |
    |       .3     |     +-----+     | .1               | |
    |o lo          |    vmx_link3    |                  | |
    |10.0.255.3/32 | 103.30.30.0/24  |                  | |
    +--------------+                 +------------------+ |
    |                                                     |
    |              o lo 10.0.255.100/32                   |
    +-----------------------------------------------------+

A lot of details that are not relevant to this section were omitted for the sake of simplicity (e.g. the VCP virtual machine). veth interface pairs are used to allow communication between the network namespaces. Everything is set up automatically by the setup_vmx.sh script.

Before proceeding with the tests, enter the following commands to create the runtime directories used by the FRR daemons:

# for i in $(seq 0 3) ; do eval "mkdir -p /var/run/frr/vrouter${i}" ; done
# chown -R frr:frr /var/run/frr/

And start the AFI client in the global network namespace:

# cd example-clients/afi-client-linux
# make
# ./run-afi-client-linux 128.0.0.16:50051 128.0.0.16:9002

4.1. Control Plane

Using FRR, the goal is to learn a route to all loopbacks (10.0.255.N/32) in all nodes of the network (vrouters and vMX). This section will demonstrate how to do that using OSPFv2, IS-IS and eBGP, but any other routing protocol could be used.

In this section we'll limit ourselves to check the if routing protocols converged properly and installed the desired routes in the Linux routing table. When a routing protocol converges on top of the tap interfaces, it means that the glue code between the AFI client and the Linux kernel is working properly.

4.1.1. OSPFv2

FRR configuration

  • vMX's ospfd (/etc/frr/ospfd.conf):
log file /tmp/frr-global-ospfd.log
!
router ospf
 router-id 10.0.255.100
 network 10.0.255.100/32 area 0
 network 103.30.0.0/24 area 0
 network 103.30.10.0/24 area 0
 network 103.30.20.0/24 area 0
 network 103.30.30.0/24 area 0
!
  • vrouter0's ospfd (/etc/frr/vrouter0/ospfd.conf):
log file /tmp/frr-vrouter0-ospfd.log
!
router ospf
 router-id 10.0.255.0
 network 10.0.255.0/32 area 0
 network 103.30.0.0/24 area 0
!
  • vrouter1's ospfd (/etc/frr/vrouter1/ospfd.conf):
log file /tmp/frr-vrouter1-ospfd.log
!
router ospf
 router-id 10.0.255.1
 network 10.0.255.1/32 area 0
 network 103.30.10.0/24 area 0
!
  • vrouter2's ospfd (/etc/frr/vrouter2/ospfd.conf):
log file /tmp/frr-vrouter2-ospfd.log
!
router ospf
 router-id 10.0.255.2
 network 10.0.255.2/32 area 0
 network 103.30.20.0/24 area 0
!
  • vrouter3's ospfd (/etc/frr/vrouter3/ospfd.conf):
log file /tmp/frr-vrouter3-ospfd.log
!
router ospf
 router-id 10.0.255.3
 network 10.0.255.3/32 area 0
 network 103.30.30.0/24 area 0
!

Starting the FRR daemons

1 - Start zebra and ospfd in the global network namespace:

# zebra -d
# ospfd -d

2 - Start zebra and ospfd in the vrouters:

# for i in $(seq 0 3) ; do \
    eval "ip netns exec vrouter${i} zebra -d -z /var/run/frr/vrouter${i}/zserv.api --vty_socket /var/run/frr/vrouter${i} -i /var/run/frr/vrouter${i}/zebra.pid -f /etc/frr/vrouter${i}/zebra.conf" ;\
    eval "ip netns exec vrouter${i} ospfd -d -z /var/run/frr/vrouter${i}/zserv.api --vty_socket /var/run/frr/vrouter${i} -i /var/run/frr/vrouter${i}/ospfd.pid -f /etc/frr/vrouter${i}/ospfd.conf" ;
done

Verification

Open vtysh and check if OSPF converged in the network:

vmx-frr-vtysh# show ip ospf neighbor 

Neighbor ID     Pri State           Dead Time Address         Interface            RXmtL RqstL DBsmL
10.0.255.0        1 Full/Backup       38.699s 103.30.0.3      tap0:103.30.0.1          0     0     0
10.0.255.1        1 Full/Backup       38.752s 103.30.10.3     tap1:103.30.10.1         0     0     0
10.0.255.2        1 Full/Backup       38.809s 103.30.20.3     tap2:103.30.20.1         0     0     0
10.0.255.3        1 Full/Backup       38.871s 103.30.30.3     tap3:103.30.30.1         0     0     0

Note that the OSPF adjacencies are being created on top of the tap interfaces as expected.

Now check the OSPF Link State Database" (LSDB):

vmx-frr-vtysh# show ip ospf database 

       OSPF Router with ID (10.0.255.100)

                Router Link States (Area 0.0.0.0)

Link ID         ADV Router      Age  Seq#       CkSum  Link count
10.0.255.0      10.0.255.0       388 0x80000005 0x36d2 2
10.0.255.1      10.0.255.1       388 0x80000005 0xfdf3 2
10.0.255.2      10.0.255.2       388 0x80000005 0xc515 2
10.0.255.3      10.0.255.3       388 0x80000005 0x8d36 2
10.0.255.100    10.0.255.100     392 0x8000000d 0x6dc8 5

                Net Link States (Area 0.0.0.0)

Link ID         ADV Router      Age  Seq#       CkSum
103.30.0.1      10.0.255.100     392 0x80000001 0xb933
103.30.10.1     10.0.255.100     392 0x80000001 0x5988
103.30.20.1     10.0.255.100     392 0x80000001 0xf8dd
103.30.30.1     10.0.255.100     392 0x80000001 0x9833

Check the OSPF routes in the RIB:

vmx-frr-vtysh# show ip route ospf 
Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, P - PIM, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel,
       > - selected route, * - FIB route

O>* 10.0.255.0/32 [110/10000] via 103.30.0.3, tap0, 00:07:17
O>* 10.0.255.1/32 [110/10000] via 103.30.10.3, tap1, 00:07:17
O>* 10.0.255.2/32 [110/10000] via 103.30.20.3, tap2, 00:07:17
O>* 10.0.255.3/32 [110/10000] via 103.30.30.3, tap3, 00:07:17
O   10.0.255.100/32 [110/0] is directly connected, lo, 00:08:07
O   103.30.0.0/24 [110/10000] is directly connected, tap0, 00:08:07
O   103.30.10.0/24 [110/10000] is directly connected, tap1, 00:08:07
O   103.30.20.0/24 [110/10000] is directly connected, tap2, 00:08:07
O   103.30.30.0/24 [110/10000] is directly connected, tap3, 00:08:07

Using the ip route command we can see that zebra installed the OSPF routes in the Linux kernel:

bash:~# ip route | grep tap
10.0.255.0 via 103.30.0.3 dev tap0  proto 188  metric 20 
10.0.255.1 via 103.30.10.3 dev tap1  proto 188  metric 20 
10.0.255.2 via 103.30.20.3 dev tap2  proto 188  metric 20 
10.0.255.3 via 103.30.30.3 dev tap3  proto 188  metric 20 
103.30.0.0/24 dev tap0  proto kernel  scope link  src 103.30.0.1 
103.30.10.0/24 dev tap1  proto kernel  scope link  src 103.30.10.1 
103.30.20.0/24 dev tap2  proto kernel  scope link  src 103.30.20.1 
103.30.30.0/24 dev tap3  proto kernel  scope link  src 103.30.30.1

4.1.2. IS-IS

FRR configuration

  • vMX's isisd (/etc/frr/isisd.conf):
log file /tmp/frr-global-isisd.log
!
interface lo
 ip router isis 1
 isis passive
!
interface tap0
 ip router isis 1
!
interface tap1
 ip router isis 1
!
interface tap2
 ip router isis 1
!
interface tap3
 ip router isis 1
!
router isis 1
 net 47.0023.0000.0000.0000.0000.0000.0000.1900.0010.00
 is-type level-1
!
  • vrouter0's isisd (/etc/frr/vrouter0/isisd.conf):
log file /tmp/frr-vrouter0-isisd.log
!
interface lo
 ip router isis 1
 isis passive
!
interface veth
 ip router isis 1
!
router isis 1
 net 47.0023.0000.0000.0000.0000.0000.0000.1900.0001.00
 is-type level-1
!
  • vrouter1's isisd (/etc/frr/vrouter1/isisd.conf):
log file /tmp/frr-vrouter1-isisd.log
!
interface lo
 ip router isis 1
 isis passive
!
interface veth
 ip router isis 1
!
router isis 1
 net 47.0023.0000.0000.0000.0000.0000.0000.1900.0002.00
 is-type level-1
!
  • vrouter2's isisd (/etc/frr/vrouter2/isisd.conf):
log file /tmp/frr-vrouter2-isisd.log
!
interface lo
 ip router isis 1
 isis passive
!
interface veth
 ip router isis 1
!
router isis 1
 net 47.0023.0000.0000.0000.0000.0000.0000.1900.0003.00
 is-type level-1
!
  • vrouter3's isisd (/etc/frr/vrouter3/isisd.conf):
log file /tmp/frr-vrouter3-isisd.log
!
interface lo
 ip router isis 1
 isis passive
!
interface veth
 ip router isis 1
!
router isis 1
 net 47.0023.0000.0000.0000.0000.0000.0000.1900.0004.00
 is-type level-1
!

Starting the FRR daemons

1 - Start zebra and isisd in the global network namespace:

# zebra -d
# isisd -d

2 - Start zebra and isisd in the vrouters:

# for i in $(seq 0 3) ; do \
    eval "ip netns exec vrouter${i} zebra -d -z /var/run/frr/vrouter${i}/zserv.api --vty_socket /var/run/frr/vrouter${i} -i /var/run/frr/vrouter${i}/zebra.pid -f /etc/frr/vrouter${i}/zebra.conf" ;\
    eval "ip netns exec vrouter${i} isisd -d -z /var/run/frr/vrouter${i}/zserv.api --vty_socket /var/run/frr/vrouter${i} -i /var/run/frr/vrouter${i}/isisd.pid -f /etc/frr/vrouter${i}/isisd.conf" ;
done

Verification

Open vtysh and check if IS-IS converged in the network:

vmx-frr-vtysh# show isis neighbor 
Area 1:
  System Id           Interface   L  State        Holdtime SNPA
  isisd               tap0        1  Up           30       3226.0a2e.ccf0
  isisd               tap1        1  Up           30       3226.0a2e.ccf1
  isisd               tap2        1  Up           30       3226.0a2e.ccf2
  isisd               tap3        1  Up           30       3226.0a2e.ccf3

Check the IS-IS LSP database:

vmx-frr-vtysh# show isis database 
Area 1:
IS-IS Level-1 link-state database:
LSP ID                  PduLen  SeqNumber   Chksum  Holdtime  ATT/P/OL
isisd.00-00                97   0x00000003  0x0cf1    1088    0/0/0
isisd.82-00                51   0x00000001  0x4aa9    1084    0/0/0
isisd.00-00                97   0x00000003  0xd01b    1088    0/0/0
isisd.85-00                51   0x00000001  0x36b8    1084    0/0/0
isisd.00-00                97   0x00000003  0x9544    1088    0/0/0
isisd.88-00                51   0x00000001  0x22c7    1084    0/0/0
isisd.00-00                97   0x00000003  0x5a6d    1088    0/0/0
isisd.8b-00                51   0x00000001  0x0ed6    1084    0/0/0
isisd.00-00          *    154   0x00000003  0x92b8    1107    0/0/0
    9 LSPs

Check the IS-IS routes in the RIB:

vmx-frr-vtysh# show ip route isis 
Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, P - PIM, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel,
       > - selected route, * - FIB route

I>* 10.0.255.0/32 [115/20] via 103.30.0.3, tap0, 00:01:57
I>* 10.0.255.1/32 [115/20] via 103.30.10.3, tap1, 00:01:56
I>* 10.0.255.2/32 [115/20] via 103.30.20.3, tap2, 00:01:56
I>* 10.0.255.3/32 [115/20] via 103.30.30.3, tap3, 00:01:56
I   103.30.0.0/24 [115/20] via 103.30.0.3, tap0 inactive, 00:01:57
I   103.30.10.0/24 [115/20] via 103.30.10.3, tap1 inactive, 00:01:56
I   103.30.20.0/24 [115/20] via 103.30.20.3, tap2 inactive, 00:01:56
I   103.30.30.0/24 [115/20] via 103.30.30.3, tap3 inactive, 00:01:56

Using the ip route command we can see that zebra installed the IS-IS routes in the Linux kernel:

bash:~# ip route | grep tap
10.0.255.0 via 103.30.0.3 dev tap0  proto 187  metric 20 
10.0.255.1 via 103.30.10.3 dev tap1  proto 187  metric 20 
10.0.255.2 via 103.30.20.3 dev tap2  proto 187  metric 20 
10.0.255.3 via 103.30.30.3 dev tap3  proto 187  metric 20 
103.30.0.0/24 dev tap0  proto kernel  scope link  src 103.30.0.1 
103.30.10.0/24 dev tap1  proto kernel  scope link  src 103.30.10.1 
103.30.20.0/24 dev tap2  proto kernel  scope link  src 103.30.20.1 
103.30.30.0/24 dev tap3  proto kernel  scope link  src 103.30.30.1 

4.1.3. BGP

FRR configuration

  • vMX's bgpd (/etc/frr/bgpd.conf):
log file /tmp/frr-global-bgpd.log
!
router bgp 100
 neighbor 103.30.0.3 remote 10
 neighbor 103.30.10.3 remote 11
 neighbor 103.30.20.3 remote 12
 neighbor 103.30.30.3 remote 13
 !
 address-family ipv4 unicast
  network 10.0.255.100/32
 exit-address-family
 !
!
  • vrouter0's bgpd (/etc/frr/vrouter0/bgpd.conf):
log file /tmp/frr-vrouter0-bgpd.log
!
router bgp 10
 neighbor 103.30.0.1 remote 100
 !
 address-family ipv4 unicast
  network 10.0.255.0/32
 exit-address-family
!
  • vrouter1's bgpd (/etc/frr/vrouter1/bgpd.conf):
log file /tmp/frr-vrouter1-bgpd.log
!
router bgp 11
 neighbor 103.30.10.1 remote 100
 !
 address-family ipv4 unicast
  network 10.0.255.1/32
 exit-address-family
!
  • vrouter2's bgpd (/etc/frr/vrouter2/bgpd.conf):
log file /tmp/frr-vrouter2-bgpd.log
!
router bgp 12
 neighbor 103.30.20.1 remote 100
 !
 address-family ipv4 unicast
  network 10.0.255.2/32
 exit-address-family
!
  • vrouter3's bgpd (/etc/frr/vrouter3/bgpd.conf):
log file /tmp/frr-vrouter3-bgpd.log
!
router bgp 13
 neighbor 103.30.30.1 remote 100
 !
 address-family ipv4 unicast
  network 10.0.255.3/32
 exit-address-family
!

Starting the FRR daemons

1 - Start zebra and bgpd in the global network namespace:

# zebra -d
# bgpd -d

2 - Start zebra and bgpd in the vrouters:

# for i in $(seq 0 3) ; do \
    eval "ip netns exec vrouter${i} zebra -d -z /var/run/frr/vrouter${i}/zserv.api --vty_socket /var/run/frr/vrouter${i} -i /var/run/frr/vrouter${i}/zebra.pid -f /etc/frr/vrouter${i}/zebra.conf" ;\
    eval "ip netns exec vrouter${i} bgpd -d -z /var/run/frr/vrouter${i}/zserv.api --vty_socket /var/run/frr/vrouter${i} -i /var/run/frr/vrouter${i}/bgpd.pid -f /etc/frr/vrouter${i}/bgpd.conf" ;
done

Verification

Open vtysh and check if BGP converged in the network:

vmx-frr-vtysh# show bgp ipv4 unicast summary 
BGP router identifier 10.0.255.100, local AS number 100 vrf-id 0
BGP table version 5
RIB entries 9, using 1296 bytes of memory
Peers 4, using 78 KiB of memory

Neighbor        V         AS MsgRcvd MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd
103.30.0.3      4         10       9      10        0    0    0 00:02:55            1
103.30.10.3     4         11       9      10        0    0    0 00:02:54            1
103.30.20.3     4         12       9      10        0    0    0 00:02:54            1
103.30.30.3     4         13       9      10        0    0    0 00:02:54            1

Total number of neighbors 4

Check the BGP routing table:

vmx-frr-vtysh# show bgp ipv4 unicast 
BGP table version is 5, local router ID is 10.0.255.100
Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
              i internal, r RIB-failure, S Stale, R Removed
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*> 10.0.255.0/32    103.30.0.3               0             0 10 i
*> 10.0.255.1/32    103.30.10.3              0             0 11 i
*> 10.0.255.2/32    103.30.20.3              0             0 12 i
*> 10.0.255.3/32    103.30.30.3              0             0 13 i
*> 10.0.255.100/32  0.0.0.0                  0         32768 i

Displayed  5 routes and 5 total paths

Check the BGP routes in the RIB:

vmx-frr-vtysh# show ip route bgp 
Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, P - PIM, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel,
       > - selected route, * - FIB route

B>* 10.0.255.0/32 [20/0] via 103.30.0.3, tap0, 00:03:14
B>* 10.0.255.1/32 [20/0] via 103.30.10.3, tap1, 00:03:14
B>* 10.0.255.2/32 [20/0] via 103.30.20.3, tap2, 00:03:14
B>* 10.0.255.3/32 [20/0] via 103.30.30.3, tap3, 00:03:14

Using the ip route command we can see that zebra installed the BGP routes in the Linux kernel:

bash:~# ip route | grep tap
10.0.255.0 via 103.30.0.3 dev tap0  proto 186  metric 20 
10.0.255.1 via 103.30.10.3 dev tap1  proto 186  metric 20 
10.0.255.2 via 103.30.20.3 dev tap2  proto 186  metric 20 
10.0.255.3 via 103.30.30.3 dev tap3  proto 186  metric 20 
103.30.0.0/24 dev tap0  proto kernel  scope link  src 103.30.0.1 
103.30.10.0/24 dev tap1  proto kernel  scope link  src 103.30.10.1 
103.30.20.0/24 dev tap2  proto kernel  scope link  src 103.30.20.1 
103.30.30.0/24 dev tap3  proto kernel  scope link  src 103.30.30.1

4.2. Data Plane

Now that we have a route to all loopbacks in all nodes (be it using OSPFv2, IS-IS or eBGP), we need to test our fast path.

Looking at the output of our AFI client, we can see that it's monitoring all routes installed in the Linux kernel and replicating them in the fast path (vASIC inside vMX):

[netlink-route] add: 10.0.255.0/32 dev 4 via 103.30.0.3
[netlink-route] add: 10.0.255.1/32 dev 7 via 103.30.10.3
[netlink-route] add: 10.0.255.2/32 dev 11 via 103.30.20.3
[netlink-route] add: 10.0.255.3/32 dev 14 via 103.30.30.3
[netlink-neighbor] add: ip 103.30.0.3 lladdr 32:26:0a:2e:cc:f0 interface veth0
[afi-route] install: 10.0.255.0/32 via 32:26:0a:2e:cc:f0
[netlink-neighbor] add: ip 103.30.10.3 lladdr 32:26:0a:2e:cc:f1 interface veth1
[afi-route] install: 10.0.255.1/32 via 32:26:0a:2e:cc:f1

Note that the AFI client only installs routes in the fast path once it can resolve their nexthops into MAC addresses. If the AFI client doesn't know the MAC addresses corresponding to the nexthop of a route, it has to wait until the route is used in the slow path to trigger the ARP protocol and only then install the route in the fast path (using the MAC address learned through netlink);

Now start an iperf server in vrouter0 (bound to the loopback address):

bash:~# ip netns exec vrouter0 iperf -s -B 10.0.255.0
------------------------------------------------------------
Server listening on TCP port 5001
Binding to local address 10.0.255.0
TCP window size: 85.3 KByte (default)
------------------------------------------------------------
[  4] local 10.0.255.0 port 5001 connected with 10.0.255.1 port 5001
[ ID] Interval       Transfer     Bandwidth
[  4]  0.0-13.0 sec  1.50 MBytes   967 Kbits/sec

And an iperf client in vrouter1 (bound to the loopback address):

bash:~# ip netns exec vrouter1 iperf -c 10.0.255.0 -B 10.0.255.1
------------------------------------------------------------
Client connecting to 10.0.255.0, TCP port 5001
Binding to local address 10.0.255.1
TCP window size: 85.0 KByte (default)
------------------------------------------------------------
[  3] local 10.0.255.1 port 5001 connected with 10.0.255.0 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-12.3 sec  1.50 MBytes  1.02 Mbits/sec

The successful connectivity proves that vMX is routing the data packets using FRR's learned routes.

Using tcpdump on the tap interfaces we can see that only control packets are being punted to the Linux network stack. Example:

# tcpdump -i tap0 -n -l
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on tap0, link-type EN10MB (Ethernet), capture size 262144 bytes
04:00:09.224517 IP 103.30.0.1.179 > 103.30.0.3.40970: Flags [P.], seq 3254258104:3254258123, ack 123198898, win 227, options [nop,nop,TS val 29349949 ecr 29334949], length 19: BGP
04:00:09.224849 IP 103.30.0.3.40970 > 103.30.0.1.179: Flags [P.], seq 1:20, ack 19, win 229, options [nop,nop,TS val 29349949 ecr 29349949], length 19: BGP
04:00:09.224859 IP 103.30.0.1.179 > 103.30.0.3.40970: Flags [.], ack 20, win 227, options [nop,nop,TS val 29349949 ecr 29349949], length 0
04:00:14.356557 ARP, Request who-has 103.30.0.1 tell 103.30.0.3, length 28
04:00:14.356568 ARP, Reply 103.30.0.1 is-at 32:26:0a:2e:aa:f0, length 28

5. Future Work

  • Add support for IPv6 and basic MPLS LSR functionality;
  • Add support for ECMP routes;
  • Embed an AFI client inside zebra as a module. This way it would be easier to add support for advanced features, like VRF and MPLS L2/L3 VPNs. It would also allow zebra to know when a route/pseudowire/etc failed to be installed in the fast path and act appropriately;
  • Performance testing using Ixia's IxNetwork.

6. References