Improving UDP or VPN Performance

[AKHwyJunkie]

Greetings. I have kind of an advanced question, in the odd chance someone else has figured out how to best solve this.

I've found that performance when using any kind of VPN or tunneling is quite poor with OpenMPTCPRouter in the traffic flow. (So, a tunnel inside of router->VPS flow.) It doesn't seem to matter if the tunnel transport is UDP or TCP. I've tried Wireguard and OpenVPN, using both UDP and TCP transport. Both uplink and downlink are impacted. The best results are from the native port forwarding capability, like from VPS->OpenMPTCPRouter, but even the throughput performance with this is somewhat lackluster. The results are dramatic, like 1/10th of total potential throughput. The traffic does aggregate and I get decent throughput when using tunnels on each connection without OpenMPTCPRouter in the mix.

I originally thought this was related to the issue with UDP performance being quite poor. But, I can demonstrate it even when TCP is used for the tunnel transport. I was surprised to see similar performance issues with the native VPN->Router integration as well.

I've taken care to adjust MTU/MSS for tunnel-in-tunnel traffic, such that final byte size is 1500 bytes. I've also tried multiple proxy types, settling on XRay-VLESS. I see the issue with both the VPN tunnel type as OpenVPN TCP or Glorytun TCP. (I'd like to try Glorytun UDP, but I'm seeing what I think is a bug where the tunnel doesn't form properly when this type is used.) I haven't seen any improvement by adjusting Congestion Control, I'm using the default BBR. VPS and router is fine and has plenty of resources, I've also tried across VPS' on different providers. Both VPS script and OpenMPTCPRouter are on latest available versions.

I am probably using this platform in a way that most don't, so this is kind of a hail Mary for more ideas. I've exhausted everything I can think of at this point. I can pin the tunnels to the individual connections easily enough, and may just do that, but I'd like to see the aggregated throughput.

[AKHwyJunkie]

I wanted to follow up on this as I found a few "key" things that needed to be adjusted in my installation. I am now able to get relatively full throughput for my use cases. For the sake of future people that might be hitting the same thing, I wanted to document my findings. (I hate unsolved posts!)

To help clarify my situation, I have multiple methods of getting into my network. I operate multiple site-to-site tunnels (using Wireguard) and also can leverage OpenMPTCPRouter's methods as well. Prior to my discoveries here, both of these methods were relatively poor in throughput and had a number of problems. Fortunately, I found two individual settings that helped me get better performance over both options.

• For general Wireguard performance, I had enabled the option for "V2Ray/Xray UDP" thinking it might help with Wireguard's UDP transit. Turns out, that actually whacks UDP performance and things are generally better without it. This is found in System->OpenMPTCPRouter->Advanced Setttings.

The above change allowed me to get relatively full throughput of inside-out traffic, specifically traffic that was transited via Wireguard. I verified this using several iPerf tests.

Even with that, though, things like inbound PAT over these Wireguard tunnels was still generally poor in performance. (For example, I use these tunnels to back up my home data to the cloud nightly.) This doesn't really make any sense to me, but that was my experience. I tried to use the built-in NAT/PAT capabilities, but these were similarly poor in performance. Until I discovered:

• You can enable "Use V2Ray/Xray" for inbound PAT configurations. Like with the rest of OpenMPTCPRouter, I've found much higher performance levels using Xray. Enabling this on the system's native NAT/PAT entries allowed me to get full throughput. This is found in Network->Firewall->Port Forwards->Edit your PAT->Advanced Settings. (Then "Use V2Ray/Xray")

This was the most optimal configuration I could find, allowing full aggregated throughput of the OpenMPTCPRotuer->VPS link as was technically possible. My PAT's now fully aggregate uplink and downlink bandwidth, as expected. It's unclear to me exactly how the traffic is transited to/from the VPS without this option, but clearly it doesn't use the primary settings for the router.

Hopefully this helps some "future me" that might be struggling and is looking for potential solutions.

[mrkaffeine92]

I am glad to see someone playing around with that corner of the panel. I have been tinkering with congestion control and queue discipline among others, such as making ends meet by sending data more so - where greater bandwidth is available from the peers. However, I did not notice much in that department.
I have, through my research, also found that BBR works the best. I am still hesitant to say which scheduler - however I assume this has more to do with each person's use case and their overhead.

I did notice that you are quite meticulous regarding your observations and analysis, so I'd like to ask what you are looking to get out of it...is it the most speed you can? Because I have noticed, for example, I can play around and get the most speed out of certain settings when doing a speed test, but bufferbloat and my own experience is to the contrary.
Would be happy to check in with you about what your expectations and share some insights I've come across as well.

-In terms of which Proxy and VPN to use - I have not gotten the hang of that too much yet. I will have to run some apps like TCPViewer to map where most of my traffic is coming from and from who.

[AKHwyJunkie]

I'm interested in this project as it's pretty niche. I'm a network engineer by trade, even doing this stuff (link aggregation) professionally, and I have a deep background in solving complex network problems. I'm not a programmer, so while I may not be able to contribute to the project with code, I can certainly contribute with knowledge and testing. I also have a lot of interest in solving the digital divide, having been on the "wrong" side of it for ages.

As for what I want to see, really it's about performance. Link aggregation should be able to achieve just that, link aggregation. That's across standard TCP/IP protocols (even something obscure like Wireguard) and both ingress/egress situations. My biggest issue right now is that OpenMPTCPRouter adds a ton of latency to my connections at times and I've yet to be able to solve that.

I haven't found massive differences/gains with the TCP schedulers. There's a ton of performance that can be gained by changing the proxy. The "newer" ones (like XRay, VRay and variants) are most interesting/performant in my testing. As for congestion protocols, I've been dealing with this forever (with high latency connections) and have been impressed by BBR (over cubic) for many years now.

Cheers and thanks for weighing in!

[mrkaffeine92]

Do you have Discord or IRC?

[AKHwyJunkie]

Same username on discord!