Tests are incorrectly split when there are a lot of slow skipped tests

[knedlsepp]

I'm running pytest-split on a test suite that has a lot of slow tests which are pytest.marked as slow and skipped by default. I can include all these tests by using a custom pytest --all-tests option.
The issue I'm seeing now is that when I create a .test_durations file via pytest --all-tests --store-durations then the regular pytest --splits 4 ... run without the --all-tests will be unbalanced. It seems as though pytest-split is taking into account the runtime of the skipped tests and thus failing to produce a balanced split. A workaround is to store one .tests_durations file for pytest --all-tests and a separate file for the plain pytest, but I think ideally this should not be required.