[Feature] Refactor Logging System: Reduce Noise & Add more informational logs

[svteb]

rafal-lal: actively updating this comment while I'm working on the issue

Is your feature request related to a problem? Please describe.
There are currently too many logs that do not provide any useful information and merely cause noise. Certain logs are incorrectly categorized (INFO should be DEBUG and vice versa). Additionally it is hard to determine the cause of failure in many tests since we do not log the state of the cluster after failure (test resources, deployments, etc.)

Describe the solution you'd like

Noise:

• Consider shortening the log header itself, the current format is I, [2025-01-03 07:23:11 +00:00 #189258] INFO -- , it is quite apparent that we do not need to specify I and INFO as they are the same. The UTC offset and process PID also seems useless.

• Utilize the Log.for() to scope the logs more appropriately (it takes considerable time to figure out where a log is coming from in the first place)

• The cnf_install (formerly cnf_setup) execution prints 4 lines of logs every second. Should be drastically reduced. - PR: Decrease number of logs during cnf_install and cnf_uninstall cnf-testsuite/kubectl_client#15

• Miscellaneous logs with no apparent meaning should be removed/refactored.

• Certain logs print out entire network statuses, manifests, process information and the like. These can be significantly reduced by only printing out information relevant to the test.

• Certain logs could be batched together into a single array (there are various for loops where an IP address/network status are printed on separate lines when looking for a "match").

• In various parts of the testsuite a Unable to use a TTY - input is not a terminal or the right kind of file error is printed out, determine the cause and remove it.

• Determine case by case which logs really belong to INFO and DEBUG categories.

• Format the printed logs in a logical manner (use \n in case of multiline outputs, e.g. so that manifests are properly formatted and the first line is not offset).

• Most of the noise comes from the libraries (shards) that testsuite makes use of (rafal-lal: updating list live as I'm going through libraries).

  • helm - PR in progress: Helm module refactors cnf-testsuite/helm#6
  • kubectl-client in progress, PR not yet created

Information:

• After failure the cluster resources should be printed out (figuring this out could prove to be more difficult than apparent).
• Attempt to make the INFO logs more "humanly" readable.

Describe alternatives you've considered

• As the testsuite slowly transitions into a more OOP model, we could consider implementing an inheritable class to ease log scoping for classes (research more into Log.for and Log.builder).

Additional context
This ticket will be updated as new problems are found, please comment if you have any other ideas/findings. As these changes do not only span tens of files but also numerous libraries, the changes should not come in a single pull request, but be split, to allow a sensible review process.