Better test descriptions and tool to generate test list

[axic]

The state tests have a _info section with a comment field, however that rarely contains anything.

It would be nice if there would be a mandatory description field explaining what the test supposed to do and a tool run on the CI would actually generate a table of test name + description + path 😉

[axic]

Example:

name	description	path
codesizeOOGInvalidSize	Create oversized (400000 bytes) contract via a creation transaction	stCodeSizeLimit/codesizeOOGInvalidSizeFiller.json

[holgerd77]

I would very much support this and go as far to suggest that you should make this one of your projects with strategic priority (whatever this means, sounded great 😄 ).

This ethereum/tests repository is wonderful and one of technical parts of the Ethereum ecosystem I admire the most for its thought-throughness and clarity. And this was probably the starting point for various clients to be implemented.

In everyday live this excitement cools down a bit 😺 since I have already spent countless hours figuring out the purpose of test cases (especially if one is not a fluent LLL reader) to get a grip where to start debugging.

A simple comment on every test case would be extremely valuable here. I know this is not the most rewarding kind of work, but if this is done in some various not-too-extensive stages this should be manageable.

[axic]

Related: #340.

[ehildenb]

#461 is now checking that the _info block with the fields source and sourceHash are present. @zixuanzh can look into supporting the remaining fields and fixing tests that do not have them.

Also, since the script ./test.py is now asserting that those fields exist, we can have a reliable way to list this information in the same tool, using ./test.py info instead of ./test.py list.

[winsvega]

that would be a great research work. I think with the coverage analizer we can run and see which tests are covering what exactly.
and would be nice to have a quiery engine for that

[winsvega]

Since yul tests there are many description comments in the yul file itself outside of json fields. The tests classification remains open