implement support for interactive debugger

[ssbarnea]

I thin that maybe is time to implement a workaround for allowing us to start the debugger on a failure. See https://bugs.launchpad.net/testrepository/+bug/902881

Instructions from https://wiki.openstack.org/wiki/Testr#Debugging_.28pdb.29_Tests are outdated as they do not include syntax changes (list instead of list-tests command)

This would be of great use for developers.

Ideally we should have a command line parameter (and environment variable) which would enable the alternative behaviour of starting the debugger on failed tests. Environment variable use is really useful as it avoids having to touch the code base to activate it. For example ansible can enable the debugger just by defining ANSIBLE_STRATEGY=debug which will stop on each failed tasks, no need to add params to calls (which an be hidden inside scripts). Once the developer defines the magic variable, it informs that he wants to debug failures.

Maybe we could even trigger this post-execution, when we already know which tests failed, so we would run again in non-parallel mode those that failed (or just call python -m testtools.run discover --load-list ... with the list of failed tests)

[mtreinish]

Is this basically the same thing as #14 or does it differ somehow?

[ssbarnea]

They are clearly related but I am not sure if identical. Shortly if I could define a STESTR_STRATEGY=debug that makes it start the debugger on failure I would extremely happy.

Now if I think a little bit more there is something different. As far as I know we cannot really use the debugger when we run tests in parallel and disabling parallel run is not something we want.

Thus I think this feature could be implemented in a smart way: only if you have test failures, rerun those with debug enabled. I am aware about the small chance that they would pass when run sequentially, but we can live with this risk.

If you think these are duplicated feel free to close this one and lets move the useful feedback into one ticket.

[aspiers]

I don't think they are identical. AFAICS #14 would still require you to instrument your code with pdb.set_trace(), whereas the suggestion here seems to propose automatically dropping into a debugger on test failure without having to instrument - and I think that's a worthwhile idea.

[aspiers]

So I just tried to use stestr with pdbpp (which is far better than pdb) and had mixed success. A basic stestr run my.test.case fails as expected unless -n is used, and then it works OK-ish, but interactive history editing and completion is completely broken :-(

In comparison it works perfectly when running python -m testtools.run my.test.case in the exact same virtualenv. So something in stestr or subunit is screwing with the interactive terminal functionality. This is really frustrating. A partial workaround is to wrap the run via rlwrap but that only brings interactive editing/history, not completion.