cdash_analyze_and_report.py: HTML special characters in (e.g.,) build names are not escaped when generating the report

[skyreflectedinmirrors]

I have been experimenting with cdash_analyze_and_report.py to provide better reporting of my CDash project, and have got things mostly working. One thing I have noticed however, is that certain links in the generated HTML file are not escaped properly, for example:

Because I named my build "Linux--with-", I often have a "++" as a part of the build name. When CDAR generates the links for such a build, you land on a filter page like e.g.:

I.e., the offending special characters ('++' in this case, but could be any HTML protected characters generally) are not displayed, and you get a filter that displays no builds.

I think solving this would be as simple as using something like urllib.parse to escape the user provided strings (i.e., filterFields, and friends) in the various QueryUrl constructors.

This is my first contribution to TriBITS, so please let me know if I've missed any of the guidelines. If I'm on the right track here, I'm happy to work on a fix / tests, and submit a PR.

[bartlettroscoe]

Hello @arghdos, yes, that is a problem in general. If you can post a PR that fixes this using standard Python 3.x, then that would be great!

And if you have other suggestions for this tool, let me know. (I have a long list of features that would make this tool more useful for various use cases that I should post to a GitHub issue.)

[skyreflectedinmirrors]

Will do! If you write up your list of features, send me a link. I had a few thoughts myself, mainly around generating summary statistics (e.g., for issue trackers on different JIRA instances, for different library components, etc.) but would be good to see your list as it's not yet clear to me where the role(s) of the CDash dashboard ends and the report generation begin.

[bartlettroscoe]

Will do! If you write up your list of features, send me a link.

@arghdos, I did a dump into the new epic:

• EPIC: cdash_analyze_and_report.py: Many possible improvements #578

Also note there are a number of existing Issues shown in this query.

The tool cdash_analyze_and_report.py is really not at a point where just anyone can pick it up and use it as it is lacking a good bit of documentation and it needs more basic error checking of input (both from the input options and files and from data downloaded from CDash).

The majority of that will likely never get done but some of that would be very useful for some customers and use cases. But I may get some funding to do some of that next FY to work on this for some new customers.

I had a few thoughts myself, mainly around generating summary statistics (e.g., for issue trackers on different JIRA instances, for different library components, etc.)

Go ahead and post issues for any ideas you have that might be useful for your use cases (and you might be willing to work on).

Also note that we can consider breaking the code and tests for cdash_analyze_and_report.py (and related tools like CreateIssueTrackerFromCDashQuery.py and cdash_build_testing_date.py) out into its own GitHub repo. (But there are some Python utility modules that are used by other TriBITS Python tools that would need to be addressed in some way.)

[skyreflectedinmirrors]

The tool cdash_analyze_and_report.py is really not at a point where just anyone can pick it up and use it as it is lacking a good bit of documentation and it needs more basic error checking of input (both from the input options and files and from data downloaded from CDash).

It definitely could use some love, but FWIW the conference publication from a few years back was enough to get me up and running (more or less). Most useful to me would be some information on the general architecture, though perhaps I'll start writing things as I tinker.

Also note that we can consider breaking the code and tests for cdash_analyze_and_report.py (and related tools like CreateIssueTrackerFromCDashQuery.py and cdash_build_testing_date.py) out into its own GitHub repo.

This would be useful, as it's the only component of TriBITS I'm using right now (though, it makes me wonder how much more of the infrastructure I could re-use).

[bartlettroscoe]

Also note that we can consider breaking the code and tests for cdash_analyze_and_report.py (and related tools like CreateIssueTrackerFromCDashQuery.py and cdash_build_testing_date.py) out into its own GitHub repo.

This would be useful, as it's the only component of TriBITS I'm using right now (though, it makes me wonder how much more of the infrastructure I could re-use).

If there is a good bit more work to be done, then we can make this happen. The only reason this code exists in the TriBITS at all is because it uses some of the Python utilities in TriBITS used by other tools in TriBITS. (So we may need to break this into a few different GitHub repos to resolve the dependency issues.)

[skyreflectedinmirrors]

Sure, and for the moment I'm happy enough with a copy of TriBITS in my repo.

[bartlettroscoe]

@arghdos, with the merge of #579, please confirm that this issue is resolved for you.

[skyreflectedinmirrors]

Confirmed

…

________________________________ From: Roscoe A. Bartlett ***@***.***> Sent: Tuesday, May 9, 2023 5:33 PM To: TriBITSPub/TriBITS ***@***.***> Cc: Curtis, Nicholas ***@***.***>; Mention ***@***.***> Subject: Re: [TriBITSPub/TriBITS] cdash_analyze_and_report.py: HTML special characters in (e.g.,) build names are not escaped when generating the report (Issue #577) Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding. @arghdos<https://github.com/arghdos>, with the merge of #579<#579>, please confirm that this issue is resolved for you. — Reply to this email directly, view it on GitHub<#577 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ABRKDCLRTDJPY2KX7OUZ7ATXFKZ2RANCNFSM6AAAAAAX2J7XRY>. You are receiving this because you were mentioned.Message ID: ***@***.***>