Add altp package for parallel execution

[acerv]

This patch adds altp package that is an alternative implementation of ltp package, using asyncio for parallel tests execution. In order to use it, please set ASYNC_RUN before run.

The altp package also introduces a new UI for parallel execution and it replaces paramiko library with asyncssh, since paramiko doesn't support coroutines.

Beware this is an EXPERIMENTAL support and it's not the ending version.

[richiejp]

🤯

I guess you really just need this to be tested by current users, then merge it with the original ltp package? Or is there some reason to keep them separate (e.g. sh over serial port being incompatible with parallel)? In which case should features be removed from ltp if they are in altp?

On a shallow level I think this is all correct. The only major hurdle now is LTX.

[acerv]

Unfortunately non-async code doesn't match really well with asyncio, so we need a new reimplementation of "ltp" package, that is called "altp" in this case.
That's why we have two folders. But the idea is to move into altp and forget about ltp package in the future.
And yes, we need to test it properly :)

[richiejp]

If altp will replace ltp, then I would suggest skipping to that.

For testing I would suggest asking anyone using runltp-ng to try rerunning some tests with this branch. (e.g. SUSE public cloud team, if they are using it).

Of course it depends on how difficult it is to do these things Vs. having two LTP backends and asking people to try the alt backend. I just think you need to avoid confusion.

[acerv]

I agree with replacement, but the whole code has been rewritten and I'm not 100% sure it's going to be stable enough.
That's why two folders and 1 switch to test the new implementation. For me it's almost zero effort to keep both, I just apply a patch inside both folders.

[richiejp]

What do you think about bringing your LTX rewrite into this code base?

I think that even if people only want the Python component or only the C component, it's better for us to have a single repository for organisation. Most likely LTX and runltp development will need to advance in lock step unless we formalise the communication protocol which I do not see as necessary for now.

[acerv]

Good point. I'm actually thinking about it.
By talking with @pevik , we also considered the hypothesis to generalize runltp-ng and being able to run literally any kind of testing suite from any kind of testing framework. This is possible due to some modifications I'm creating and I will share when the first prototype is ready. We basically have to get rid to specific LTP stuff we have in runltp-ng.

The idea is to create a plain new project based on the asynchronous version of runltp-ng, which generalizes not only SUT, but also the Framework we are using (plugin based). This project would suit even better with ltx, that is a service created specifically for general sync/parallel execution. In this way we can run multiple testings suites, such as LTP, liburing, kselftests and so on, to generate the same report and to parse it in the same way, using SUT implementations we have (Qemu, SSH, LTX, host, etc). I will share some code soon.

[richiejp]

Yes, that would be usefull for e.g. running the Syzkaller reproducers. A plugin could be created for runltp instead of trying to force them into an LTP wrapper which tries to mimic what the very complicated Syzkaller executor does. The advantage being runltp can snapshot and revert the VM after each reproducer similar to Amazon Snapchange, but probably simpler.

[acerv]

We have the first fork implementation here: https://github.com/acerv/kirk