Improving load times for large tests

[sunder-gather]

Hi all! First of all, thanks for building such a wonderful tool!

I've a large test suite (about 5200 code generated test cases, 1.7M on disk) which takes extremely long to load, it looks like Hurl loads the full file, parses the entire file and then executes each request. Is there a way to improve on this by executing each request as they're parsed/found so as to save test times? Thanks!

[jcamiel]

Hi, thanks for the feedback!

Yes, you're right, Hurl parses the entire file, then run the all the entries once (it's like a comp language where a file is compiled before execution). We might try to see what we can improve with your use case, but don't expect anything soon because it's a "stucturing" decision for the software (would you be able to share one of your big test file with us so we can see if the parsing is not slowed by a performance issue?)

That said:

• imagine we parse one request then execute it before parsing the second request: you'll be able to see request executed sooner instead of waiting a long time. But, if your test is ok (all the asserts succeeded), the total duration (parsing + execution) will be the same as the current model with a big file. You will be able to see failure quicker however.
• as you're generating test cases, maybe you can try to break your tests in multiple files, if it is possible. Tests can be executed with --glob, for instance hurl --test --glob **/*.hurl. You have to group your requests in one file if you want to share your cookies, but in a stateless env, you can try breaking your tests as you're already generating them.

[jcamiel]

Hi @sunder-gather It looks effectively that there is a performance bug in the current Hurl parser. We have a pending PR to fix it; now I'm able to parse a 100,000 loc file in less than a second. I will ping you as soon as the fix is on our master branch to see if it corrects your issue.
Thanks for the feedback!

[u89012]

Hi @jcamiel! I took your advice and split the generated tests into multiple files but I'm happy to see this issue has now been resolved! 100K loc in <1 sec is impressive! Thanks again for Hurl!