NAT problem with different mtu

[qinghon]

Describe the bug
Is there any way to solve the mtu problem?
When using the pppoe network in a conventional router scenario, the pppoe interface requires mtu 1492 or less, and the network interface is usually 1500. This results in that when acting as a ppp <-> lan nat gateway, the gateway will not split the packets, and there will be no pmtu. As a result, the lan interface mtu cannot be used normally when it exceeds 1492.

Expected behavior
A clear and concise description of what you expected to happen.

Configuration
Command-line options:

einat -i ppp0 --ports 1024-65535 --hairpin-if eth0

Target Machine (please complete the following information):

• Architecture: [aarch64]
• Linux distribution: [debian 12]
• Kernel version: [6.6.39-ophub]
• einat version: [0.1.2]

Additional context
Add any other context about the problem here, e.g. network interface information, firewall(iptables/nftables) configuration.

4: eth0@if4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 4a:a8:91:57:c3:07 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 192.168.1.1/24 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::48a8:91ff:fe57:c307/64 scope link 
       valid_lft forever preferred_lft forever
5: eth1@if4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 32:07:ba:d8:ce:a7 brd ff:ff:ff:ff:ff:ff link-netnsid 0
6: ppp0: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1492 qdisc fq_codel state UNKNOWN group default qlen 3
    link/ppp 
    inet 112.202.172.190 peer 10.160.22.1/32 scope global ppp0
       valid_lft forever preferred_lft forever

[EHfive]

the gateway will not split the packets, and there will be no pmtu

So the PMTU discovery is not working in your setup, thus resulting fragmentation failure in the sending end?

I manually set MTU of WAN ethernet(not ppp) interface to smaller than LAN interface and the ICMP PMTU error packet is correctly send back to the sending end by Linux kernel on gateway.

$ ip link set dev wan mtu 1492
# manual PMTU discovery from LAN
$ ping -M do -s 1472 example.com
PING example.com 1472(1500) bytes of data.
From <LAN interface IP>: icmp_seq=1 Frag needed and DF set (mtu = 1492)
ping: local error: Message too long, mtu=1492

Or you are expecting packets larger than 1492 without "Don't Fragment" set to be fragemented regardless while forwarding from lan to ppp interface?

If that is the case, unfortunately einat is not able/very difficult to implement IP fragmentation due to limitation of eBPF. If the kernel or LAN interface is not fragmenting packets, than it would be remain unfragmented regardless the packet would be forwarded to einat hooked WAN interface or not.

And einat only alters L3 IP fileds and L4 port fileds in IP packets without incresing packet size, so it should be transparent to kernel network stack.

So I guess it's more like a problem of PMTU error message not being received by sending end in your setup? Could you detail the problems you encountered more?

Also maybe aligning the MTU to the minimum of all interfaces helps, e.g. set MTU of your LAN interfaces to 1492.

[EHfive]

Also try TCP MSS Clamping, https://wiki.nftables.org/wiki-nftables/index.php/Mangling_packet_headers#Mangling_TCP_options

Seems like it's the exact solution to mitigate PPPoE MTU problem if PMTUD was indeed working there.

This was configured by default on OpenWrt, so I guess that's why there was no such report from OpenWrt user.

[qinghon]

Or you are expecting packets larger than 1492 without "Don't Fragment" set to be fragemented regardless while forwarding from lan to ppp interface?
not this fragement

It is true that there is a problem with the remote server replying to mss, after configuring tcp mss clamping it works fine.
Thank you for your answer.