🐛 BUG: wakes up the CPU a lot

[hmoffatt]

What version of nebula are you using? (nebula -version)

1.6.1

What operating system are you using?

Linux

Describe the Bug

powertop shows that nebula generates hundreds of events per second, which stops the computer sleeping more. strace shows that it is sleeping for only 100ms between events.

Logs from affected hosts

Nothing logged while it's up and running and connected.

Config files from affected hosts

Just a plain end node connected to two lighthouses.

[johnmaguire]

@hmoffatt This has come up before in #983. As before, it's not clear to me what a successful outcome for this ticket looks like. Without comparing power consumption to other similar utilities, I'm not sure that there's excessive power consumption occurring here.

Have you performed this test against any similar tools? Please see also some of the tuning you can perform here: #983 (comment)

Keep in mind also that Nebula's CPU usage (and power usage) will scale with how much traffic it's passing.

[hmoffatt]

I don't have any figures from other utilities, but powertop shows nebula is waking 40-50 times per second even when there's no user traffic on the interface, which is the highest of any process on the machine. None of the tuneables listed in #983 would seem to add up to anything like that many wakeups; which would you suggest trying first?

[johnmaguire]

@hmoffatt Without testing other similar software, it's hard for me to know if that's a reasonable number or not. (Apologies if this is obvious to you - but I've not spent time tuning processes for power usage in the past.) Can you also share how you've determined there is no traffic on the interface?

Regarding the timers, you could try changing them all to something very high and seeing what happens. If that makes a big difference, then you can start trying them one-by-one. There are only 6 listed timers, so it shouldn't be hard to enumerate them all.

[johnmaguire]

@hmoffatt I just noticed your original issue mentions you're running Nebula 1.6.1. This build is 2 years old so please make sure future testing is against the latest version of Nebula, currently v1.9.4.

Additionally, when reviewing the issue, I noticed...

None of the tuneables listed in #983 would seem to add up to anything like that many wakeups; which would you suggest trying first?

You stated you were seeing wakeups approximately every 100ms:

powertop shows that nebula generates hundreds of events per second, which stops the computer sleeping more. strace shows that it is sleeping for only 100ms between events.

As mentioned, I would try each of the tuneables, but one in particular jumps out as your lower bound, so you may wish to try it first:

Keeping track of and pushing forward in-flight handshakes (handshakes.try_interval, defaults to 100ms)

[hmoffatt]

Thanks for following up @johnmaguire . Unfortunately v1.6.1 is all that's packaged for Debian currently.

I just installed v1.9.4 from the release on GitHub, and now powertop reports 25-30 wake ups per second. (The rest of my network is still running v1.6.1 at this point.) Then I set handshakes.try_interval to 1000ms and now I see 10-20 wake ups per second, and a decent reduction in estimated power!

powertop also thinks there's quite a lot of power used by the tun interface, even when it's not carrying any packets. I don't know how it calculates that, so I think I'll ignore it.