templates: Break out build/test summaries

[mrbazzan]

Closes #290

[mrbazzan]

@spbnick
Please review when you can.

[mrbazzan]

This is so the summaries will align in the mail. Something like this;

Build:   1V 2F
 Test:   4V 3E

[spbnick]

Thank you for this PR, @mrbazzan! Sorry it's taking me so long. I've been very busy with #311, which I hope to finish this week, but next week I'm on vacation, so it would take me still longer to get to it.

[mrbazzan]

I've been very busy with #311

#311 👍🏿 💯

next week, I'm on vacation, so it would take me still longer to get to it.

@spbnick No problem. Have a nice vacation :)

[spbnick]

Thank you for working on this @mrbazzan, and waiting for me to get to this PR ☺️

I would say this is a pretty good start, and it works almost as needed, it seems. However, this could produce unaligned output in some cases (sample1.json.gz), like this:

        Builds:   ❌ 2  ✅ 208
         Tests:   ❌ 14  ✅ 9

While other cases (sample2.json.gz) look better, but still not perfect:

        Builds:   ❌ 34  ✅ 867
         Tests:   ❌ 47  ✅ 45  🚧❌ 6  🚧✅ 4

It would be good to instead get each emoji aligned on its own column, as seen in the current builds summary section, including using the ➖ character where results are missing, and to have numbers aligned right in each field.

I.e., for both of the outputs above, respectively:

        Builds:   ❌  2  ✅ 208
         Tests:   ❌ 14  ✅   9

        Builds:   ❌ 34  ✅ 867
         Tests:   ❌ 47  ✅  45  🚧❌ 6  🚧✅ 4

You can generate emails for the above results using commands like this:

zcat sample1.json.gz | kcidb-notify '>revision#' > output.eml

Finally, for easier review and manipulating changes later, it's better to split your changes into commits refactoring the existing code step-by-step, with explanations, and commits that build upon the refactoring to introduce actual behavior changes, instead of grouping commits by changed areas/files. I.e. in your case you could have a few commits extracting the sub macros and adding new macros, which shouldn't be changing anything, and could be tested not to, and then have commits switching to the new behavior in the overview. Could you do that?

Thank you!

[mrbazzan]

Thank you for working on this @mrbazzan, and waiting for me to get to this PR relaxed

👍🏿 . I'll start refactoring the PR this week.
It might take some time as I'm busy with some personal things...

[mrbazzan]

Thanks a lot for the sample data.

[mrbazzan]

Hi @spbnick,
How's it going? I have been able to refactor the commits as you mentioned above, but I have issues properly formatting each emoji on its own column in the OVERVIEW section.
This is mostly because the data used for the Test and Build are from different methods. Any help/insight will be appreciated

[spbnick]

Hi @mrbazzan! Nice to have you back 🙂 What exact problems do you have? Would you like to perhaps push your code so far, so that we can discuss it?

[mrbazzan]

@spbnick
The Build and Test in the OVERVIEW section for each object type is still pretty much unaligned.

[spbnick]

Thank you @mrbazzan, and sorry for the late review - this week was hard 😂

I like the new commit breakdown, thank you!

I think what you need to achieve consistent formatting is to generate a single format string accommodating both test and builds summary lines for a container, and taking an optional container name (e.g. architecture name). So basically calculate the maximum count for each category (PASS/FAIL/DONE/SKIP/UNKNOWN/etc. multiplied by NOT-WAIVED/WAIVED/UNKNOWN) across both builds and tests, omitting categories with zero counts, and generate the format string with appropriate width. Then have a macro taking the format strings and counts for corresponding categories (as a list, or a dictionary?) and outputting the summary line.

E.g. have a macro calculate counts for tests for a container and returning a list, or a dictionary in a particular format, and have another doing the same for builds, returning the same format. Add them up to generate the format string with another macro, and pass both of these count list/dictionaries separately to the macro for producing the corresponding summary line.

Make sure the formatting string/macro also accepts the optional container name (e.g. architecture), and then you would also be able to use them for other summaries. Or maybe even forgo passing around the format string and just pass around the counters (and the container name), and embed the format string generation in the summary-formatting macro.

In any case, I would try going this way. If this turns out to be too cumbersome/awkward in Jinja2, then perhaps we could compromise and produce a somewhat uglier, but much simpler to code summary 😁

[spbnick]

I think this should be named something like container_status, since this is producing a build status summary for a container, not one particular build. Besides the file is already called build.j2 and having "build" in the name would be repeating ourselves.

[spbnick]

I think this should be called architecture_status, as in "producing a summary for builds in an "architecture" container". However, it might turn out that we don't need this particular separate macro after all.

[mrbazzan]

Thank you @mrbazzan, and sorry for the late review - this week was hard joy

👍🏿
Hi @spbnick, the last two months have been really hectic. Thanks for being patient while I address this PR

I think what you need to achieve consistent formatting is to generate a single format string accommodating both test and builds summary lines for a container, and taking an optional container name (e.g. architecture name). So basically calculate the maximum count for each category (PASS/FAIL/DONE/SKIP/UNKNOWN/etc. multiplied by NOT-WAIVED/WAIVED/UNKNOWN) across both builds and tests, omitting categories with zero counts, and generate the format string with appropriate width. Then have a macro taking the format strings and counts for corresponding categories (as a list, or a dictionary?) and outputting the summary line.

E.g. have a macro calculate counts for tests for a container and returning a list, or a dictionary in a particular format, and have another doing the same for builds, returning the same format. Add them up to generate the format string with another macro, and pass both of these count list/dictionaries separately to the macro for producing the corresponding summary line.

Thanks 👍🏿

Make sure the formatting string/macro also accepts the optional container name (e.g. architecture), and then you would also be able to use them for other summaries. Or maybe even forgo passing around the format string and just pass around the counters (and the container name), and embed the format string generation in the summary-formatting macro.

I'm still confused about this part. I'm sure it has something to do with being able to use the build format string with the various architectures and having them properly aligned, right?. After I get the initial feedback, I think it should be possible to improve the new implementation to add this :)

CC #320.
Thanks a lot for this thorough explanation. It took me a while before I actually understood your approach to solving the problem.
#320 is more or less similar to the approach you have explained above; it uses custom filters to perform most of the functionality instead of doing in the .j2 files. I also separated out the overview functionality(I'm sure you might be skeptical about this approach) to stand alone without interfering with the previous logic.

I should probably draft this PR while you check out the new one.