Performance problem with the new version of Rocket

[leonardoblanco]

Hi, I was testing the performance of a simple helloworld service and for an old version of Rocket the performance was great, but for current version is not so good enough.

https://github.com/leonardoblanco/rocket-energy

This code, is a simple helloworld use rocket 0.3 and rust 1.27.0 (build with "rustup default nightly-2018-04-04"), try to build and test with wrk:

$ wrk -d1m http://localhost

I can achieve around 50k Requests/sec: 50164.66

https://github.com/leonardoblanco/rocket-hello

This other code, is a simple helloworld use the current version of rocket 0.4.1 and rust 1.37.0 (build with "rustup default nightly"), try to build and test with wrk:

$ wrk -d1m http://localhost:8000

I can achieve only around 500 Requests/sec: 544.82

The diference is big and I don't know if the problem is with Rocket or Rust, but I can't achieve the same performance that I had with the old version with the new version.

I'm doing all things right I guess and the environment where I ran the tests is the same.

Any ideas?

[jebrosen]

Are those with --release? The second number is shockingly low and not what I have experienced. It's also a much wider difference than I am seeing when I test locally: 100x as many requests in 0.3 vs 0.4, compared to my 4x as many.

After some testing this appears to be a performance regression between hyper 0.10.13 and 0.10.14, which can be demonstrated in your rocket-energy example with an edited Cargo.lock file. Maybe hyperium/hyper@2b83792 ends up performing badly for this demo's workload? I don't want to spend a lot of time on this because we are in the process of moving to hyper 0.12 (#1008) -- I'm seeing on the order of four times as many requests per second in the hello_world example with hyper 0.12 compared to 0.4, and that's with some known unresolved inefficiencies.

[leonardoblanco]

@jebrosen yes I built with --release and run with rocket_env=prod.

I tested the new version of rocket 0.4.1 with rust 1.37.0 with the old version of hyperium/hyper as you suggest and the performance was great again, I achieve around 44k requests/sec: 44003.23

Now let us see if with the new hyper 0.12 we will keep the performance.

Thanks @jebrosen

[vegai]

This might be linked somehow to the poor performance of Rocket in Techempower's benchmarks, e.g. here: https://www.techempower.com/benchmarks/#section=test&runid=5cc1a314-4e0c-4294-b8f1-cb607e0a5440&hw=ph&test=fortune -- also an abnormally high amount of failures.

[jebrosen]

This might be linked somehow to the poor performance of Rocket in Techempower's benchmarks

I suspect bugs in keep-alive are more likely to be the cause of those failures (see also #580, #928)

---

From where I stand, trying to improve performance on hyper 0.10 (which is barely maintained) is not worth the effort compared to moving to and improving hyper 0.12. As such, I'm going to close this issue. #17 and #1008 are good places to watch for further progress.