Ability to specify the return code for success

[pytoxbot]

• Bitbucket: https://bitbucket.org/hpk42/tox/issue/134
• Originally reported by: @lvh
• Originally created at: 2013-11-01T19:11:43.738

Right now, tox assumes that a return code of 0 means "success". That's almost always true, but I've ran into a case where it isn't.

While, arguably, that's a bug in the thing that I'm running, it would be nice if there was a way to specify the expected return code, so I don't have to do crude hacks like:

    sh -c "pyroma . || (( $? == 10 ))"

[pytoxbot]

Original comment by @lvh

Right. In my case, it's not super important; most of pyroma complains about is nice-to-haves, not fail-the-build. Invert would be marginally better for me, but accept all return codes would work fine too :)

[pytoxbot]

Original comment by @hpk42

Question is if ! (or another character) should mean "invert return codeoraccept all return codes``.

[pytoxbot]

Original comment by @lvh

Prefix with a !? It'd work for my use case, I'm not complaining, but I'm mostly worried about people who have a case where only one particular (or a small set of) retcodes are valid. But, sure, those people could use the shell.py thing :)

[pytoxbot]

Original comment by @hpk42

You can already use an arbitrary code script from the dir where tox.ini is located:

#!python

[testenv]
commands = python {toxinidir}/path/to/shell.py ret=10 actual_command ...

so you can wrap command execution. That's not very convenient for a somewhat common case of declaring a non-zero returncode as acceptable, however.

I am sometimes pondering implementing a subset of bash functionality so that it works also on windows. Something like this:

#!python

commands = 
        cd {toxinidir}/tests
        export SOMEVAR=...
        py.test ...

And allowing some || (( $? == 10 )) or so syntax would then be natural i guess. But i also find it awkward. What about using more "make" like syntax? See http://stackoverflow.com/questions/1545985/gnu-make-inverting-sub-process-success for examples.

[pytoxbot]

Original comment by @lvh

Also, people may want more complex expressions there, but I don't know if you want to allow that. I suppose at that point you're already doing arbitrary code execution, so what's another eval? :)

[pytoxbot]

Original comment by @lvh

Hm. It's a hard issue. IIUC stuff is now interpolated, split and then passed to popen, so maybe something that's unlikely to occur in that string now.

Also, maybe we need to add other metadata later.

So, one thing that you could do is pretend it's a shell:

pyroma . || (( $? == 10 ))

Maybe with a specific prefix that means "execute this in a shell", like:

(sh)pyroma . || (( $? == 10 ))

Upside:

1. Will magically keep working if stuff ever gets executed later under /bin/sh
2. You can do anything

Two downsides:

1. Ugly as sin
2. Not obvious to newcomers
3. You can do anything
4. Hard to extend unless you have deep shell wizard-fu

Another option would be something along the lines of:

pyroma # retcode=10

This can later be extended to something like # retcode=10,quantum_transmogrifiers=100.

I'm not a huge fan of either of these, but that's what came up in my head :)

[pytoxbot]

Original comment by @hpk42

any suggestion for syntax?

[obestwalter]

The interest in this seems to be minor, so I would propose to close this as wontfix or at least not introduce it in the old format but only in #999

[obestwalter]

cc @lvh

[lvh]

Sure, I have no more interest in this. I fixed it, years ago, by fixing the return code of said piece of software.

[gaborbernat]

I'll propose as won't fix this. Return code zero is success and we should prefer that, I would propose people to wrap call if they really have to support software with non zero exit value.