l3 Configuration

Overview

IPv4 routing enables you to decide to which IPv4 neighbour a packet is forwarded, based on it's IPv4 header (see rfc791).

When an IP address in assigned to a Switchdev interface or its master bridge - a routing interface is created.
For each address and its broadcast and network addresses, traps are configured in PP to make sure that appropriate packets are delivered to the kernel.
For example:
ip addr add 192.168.1.1/24 sw1p1
ip link set dev sw1p1 up
ip addr add 192.168.2.1/24 sw1p2
ip link set dev sw1p1 up

NOTE: A configuration is applied only after the interface is set to admin UP.

IPv6

IPv6 main purpose is the same as IPv4 - decide, to which neighbour packet
will be forwarded, based on it's IPv6 header (see rfc8200).

NOTE: the main difference with IPv4 is that IPv6 must have Link-Local (FE80::/10)
address for correct operating. This address is used for (neigh/router solicit).

IPv6 ADDRCONF

IPv6 has ADDRCONF mechanism. This is kernel feature, that generate Link-Local
address automatically.
Example:
#Check there is no addressess
ip -6 addr show sw1p30
#Then set interface UP and check generated address
ip link set sw1p30 up
ip -6 addr show sw1p30
20: sw1p30: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state
UP group default qlen 2048
link/ether 18:be:92:11:d6:71 brd ff:ff:ff:ff:ff:ff
inet6 fe80::1abe:92ff:fe11:d671/64 scope link
valid_lft forever preferred_lft forever
NOTE: On this step RIF is created. Packets can be routed, according to ipv6 routes table,
if you already have added routes and enabled /proc/sys/net/ipv6/conf/all/forwarding.

Auto adding Link-Local address can be configured:
#Enable auto adding (default)
echo 0 >/proc/sys/net/ipv6/conf/all/addr_gen_mode
#Disable auto adding
echo 1 >/proc/sys/net/ipv6/conf/all/addr_gen_mode

On link DOWN IPv6 removes addresses by default. This behaviour can be configured with
/proc/sys/net/ipv6/conf/all/keep_addr_on_down.

IPv6 Global Unicast

To configure global unicast address on interface (subnet, that can be routed from
different link) use following commnds:
#Add Global Unicast address and subnet
ip -6 addr add ::a:1:1/120 dev sw1p30
#Enable forwarding
echo 1 >/proc/sys/net/ipv6/conf/all/forwarding

IPv6 routes

Blackhole route:
ip -6 ro add blackhole ::0606:0606

Local route:
ip -6 ro add local ::0707:0707 dev lo

Connected route:
ip -6 ro add ::0808:0800/120 dev sw1p30

Nexthop route:
ip -6 ro add ::0909:0900/120 via ::a:1:2 dev sw1p30

IPv6 neighbours

Static neighbour:
ip -6 ne add ::0808:0809 lladdr 00:99:99:99:99:99 dev sw1p30

IPv6 RIF deletion

To delete addresses on interface and related routes with neighbours – set link DOWN:
ip l s sw1p30 down

!!IMPORTANT!!
NOTE: flushing addresses on interface (ip a f sw1p30) doesn’t lead to
routes/neighbours deletion, as it implemented in IPv4:
ip l s sw1p30 up
ip -6 a a ::5001/120 dev sw1p30
ip -6 ro add ::6600/120 via ::5002
ip a f sw1p30
ip -6 ro
...
::6600/120 via ::5002 dev sw1p30 metric 1024 trap rt_offload pref medium
...

Bridge Routing Interface

To create router interfaces on top of bridge netdev, an IP address has to be assigned.
For .1Q bridge, a router interface can be created for each of its upper VLAN devices.
For .1D a router interface can be created to the bridge itself only.

.1D Bridge Example:

ip link add name br0 type bridge
ip link set sw1p3 master br0
ip link set sw1p3 up
ip addr add 192.168.3.1/24 dev br0
ip link set br0 up

.1Q Bridge Example:

ip link add name br0 type bridge vlan filtering 1
ip link set sw1p4 master br0
ip link set sw1p4 up
bridge vlan add vid 10 dev sw1p14 untagged pvid
ip link add link br0 name br0.10 type vlan id 10
bridge vlan add dev br0 vid 10 self
ip addr add 192.1684.1/24 dev br0.10

Routes configuration

Static routes are added using the ip route command, provided by the iprote2 package.

Add connected route

ip route add 1.1.1.1/24 dev sw1p1

Add nexthop route

ip route add 4.4.4.4 via 1.1.1.2

Add ECMP route

ip route add 4.4.4.5 nexthop via 1.1.1.3 nexthop via 1.1.1.4

Add blackhole route

ip route add blackhole 3.3.3.3

Show main routing table

ip route show

Example output:

1.1.1.0/24 dev sw1p1 scope link linkdown rt_trap
blackhole 3.3.3.3 rt_offload
4.4.4.4 via 1.1.1.2 dev sw1p1 trap linkdown rt_offload
4.4.4.5 linkdown rt_offload
        nexthop via 1.1.1.3 dev sw1p1 weight 1 trap linkdown
        nexthop via 1.1.1.4 dev sw1p1 weight 1 trap linkdown

Static Route Flags

There are two types of flags:

LPM: Notify about entry in HW LPM (with rt_ prefix). This flag is supported on all route types
nexthop: Notify about nexthop entry in HW (without rt_ prefix). This flag is supported only for nexthop routes

Scheme: flags on nexthop route:

4.4.4.4 via 1.1.1.2 dev sw1p1 trap linkdown rt_offload
                               A               A
                               |               |
                           nexthop flag     LPM flag

Scheme: flags on ECMP route:

                      LPM flag
                       /
4.4.4.5 linkdown rt_offload
        nexthop via 1.1.1.3 dev sw1p1 weight 1 trap linkdown
        nexthop via 1.1.1.4 dev sw1p1 weight 1 trap linkdown
                                                A
                                                |
                                            nexthop flag

Table: LPM/nexthop flags combinations on nexthop route

|-------------------+-------------------------+----------------------------|
| LPM\nexthop       | trap                    | offload                    |
|-------------------+-------------------------+----------------------------|
| <empty>           | Route is overlapped     | Route is overlapped        |
|-------------------+-------------------------+----------------------------|
| rt_offload_failed | Unknown behaviour.      | Unknown behaviour.         |
|                   | LPM in HW inconsistent. | LPM in HW inconsistent.    |
|-------------------+-------------------------+----------------------------|
| rt_offload        | Packets trapped         | Packets fully routed by HW |
|-------------------+-------------------------+----------------------------|

Table: LPM/nexthop flags combinations on nexthop route via non-switch interface

|-------------------+-------------------------|
| LPM\nexthop       | <empty>                 |
|-------------------+-------------------------|
| <empty>           | Route is overlapped     |
|-------------------+-------------------------|
| rt_trap           | Packets trapped         |
|-------------------+-------------------------|
| rt_offload_failed | Unknown behaviour.      |
|                   | LPM in HW inconsistent. |
|-------------------+-------------------------|

Table: LPM/nexthop flags combinations on connected route

|-------------------+-------------------------|
| LPM\nexthop       | <empty>                 |
|-------------------+-------------------------|
| <empty>           | Route is overlapped     |
|-------------------+-------------------------|
| rt_trap           | Packets trapped         |
|-------------------+-------------------------|
| rt_offload_failed | Unknown behaviour.      |
|                   | LPM in HW inconsistent. |
|-------------------+-------------------------|

Table: LPM/nexthop flags combinations on local route

|-------------------+-------------------------|
| LPM\nexthop       | <empty>                 |
|-------------------+-------------------------|
| <empty>           | Route is overlapped     |
|-------------------+-------------------------|
| rt_trap           | Packets trapped         |
|-------------------+-------------------------|
| rt_offload_failed | Unknown behaviour.      |
|                   | LPM in HW inconsistent. |
|-------------------+-------------------------|

Table: LPM/nexthop flags combinations on blackhole route

|-------------------+-------------------------|
| LPM\nexthop       | <empty>                 |
|-------------------+-------------------------|
| <empty>           | Route is overlapped     |
|-------------------+-------------------------|
| rt_offload        | Packets dropped by HW   |
|-------------------+-------------------------|
| rt_offload_failed | Unknown behaviour.      |
|                   | LPM in HW inconsistent. |
|-------------------+-------------------------|

Example

The following is an example of a simple static routing configuration.

Our topology:

10.1.1.2       10.1.1.1   11.1.1.1  11.1.1.2   10.2.1.1   10.2.1.2
|HS0|---------------|Router0|-----+------|Router1|-----------|HS1|
                                   \
                                    \
                             11.1.1.3\
                                |Router2|--------|HS2|
                                   10.3.1.1   10.3.1.2

Our device is "Router0". So the next configuration is related to "Router0".

Add addresses to interfaces: ip addr add 10.1.1.1/24 dev sw1p1 ip addr add 11.1.1.1/24 dev sw1p31

Add nexthop routes: ip route add 10.2.1/24 via 11.1.1.2 ip route add 10.3.1/24 via 11.1.1.3

See routing table:

# ip route show
10.1.1.0/24 dev sw1p1 proto kernel scope link src 10.1.1.1 rt_trap
11.1.1.0/24 dev sw1p31 proto kernel scope link src 11.1.1.1 rt_trap
10.2.1.0/24 via 11.1.1.2 dev sw1p31 offload rt_offload
10.3.1.0/24 via 11.1.1.3 dev sw1p31 offload rt_offload

Limitations and Assumptions

Linux Kernel forwarding MUST be enabled.
Multicast routing is not supported.
At least one switch port should be added (enslaved) to the bridge before assigning the IP address on the bridge. Otherwise the router interface for the bridge is not created.
Used only local and main tables with default rules

Provide feedback

Saved searches

Use saved searches to filter your results more quickly