[JENKINS-33952] "Test Statistics Grid" shows all in "0" and "0%" including total as 0 tests when build never passed

[TobiX]

Hi,

There is a problem in the latest release:

If you have job that history is kept let say for 14 days and it was never been green the "Test Statistics Grid" shows all in "0" and "0%" including total as 0 tests

PS: here is a screenshot with calculations to confirm the bug - http://screencast.com/t/1QkzO8Ggy - the stats are completely wrong

---

Originally reported by ndinev, imported from: "Test Statistics Grid" shows all in "0" and "0%" including total as 0 tests when build never passed

• assignee: tgr
• status: Reopened
• priority: Minor
• resolution: Unresolved
• imported: 2022-10-30

[TobiX]

vandyev:

Addressed in pull request #42

[TobiX]

vandyev:

Resolved in 2.9.8

[TobiX]

vandyev:

2.9.8

[TobiX]

ndinev:

I tested again with version 2.9.9
Unfortunately the problem still exists - Failed tests form test sets that never passed are not included in the chart
http://screencast.com/t/orRVRGU53

[TobiX]

vandyev:

What plugin are you using to run/tabulate the test statistics? The Dashboard view is pretty dumb; it just reports whatever the test result actions from the plugin report. I can try and confirm/diagnose, but I'd like to make sure I replicate your environment.

[TobiX]

ndinev:

Not sure what you are asking, but I am using simple dashboard view with Test Statistics Chart and Test Trend Chart
Whenever there is a NA in "Last Success" column the result of this test set is ignored in Test Statistics Chart.

However it is important to note that Test Trend Chart (seems to) show correctly the number of failing test. This is strange since I would expect that trend chart uses same data as pie chart but shows them over time.

Hope this helps

[TobiX]

darthopto:

I am having the same problem with the Test Statistics Grid. The numbers for one job have never populated and the others seem to not update.

[TobiX]

vandyev:

Neven,

Definitely agree that it looks like something is wonky. Can you throw the "Test Statistics Grid" up on your dashboard, and show me the result of that? It shows all the individual jobs being tallied up, and that might help diagnose the issue. I don't see anywhere in the code where there should be an issue specific to never-passed jobs, so I'd like to try and get more data.

UPDATE: I went through your latest screenshot, and it doesn't look like the "successful" tests for those two jobs are being included in the chart either. Is there anything special about those jobs? What plugin is publishing those test results?

[TobiX]

vandyev:

Curtis,

Regarding the Grid, what type of tests are being published? I fixed an issue with the TAP plugin where it wasn't publishing its results in a way that the grid liked. I'd recommend opening a separate defect report for that one, tied to whatever test tool is trying to publish results. You can reference https://issues.jenkins-ci.org/browse/JENKINS-22016 which was the fix for the TAP plugin.

[TobiX]

jdemelo:

I'm seeing the same problem. I think it stems from the fact that TestUtil.summarizeJob() calls

job.getAction(TestResultProjectAction.class)

In my debugging, job.getAction() leads to the following calls:

job.getAllActions()
  - Actionable.createFor()
    - SimpleBuildStep.LastBuildActionFactory.createFor()

Inside LastBuildActionFactory.createFor(), you can see that project actions will only be added to the collection returned if getLastSuccessfulBuild() is not null. Because of this, the dashboard's calls to job.getAction(TestResultProjectAction.class) return null and a blank test result is added to the summary.

My hunch is that in general, only showing information from the last successful build isn't what people are looking for. In my case, I always want to see information from the last build that produced a test result, even if that build failed.

You might be able to borrow from here, and call it here, instead of job.getAction(TestResultProjectAction.class).

[TobiX]

neel_narute:

Hi,

I'm also facing the same issue. Do we have any update on this?

"Test Statistics Grid" showing '0' for failed test cases.

 

Regards,

Neel

[TobiX]

mekhdievzr:

Good day, guys! 

 

Same issue

Jenkins ver - 2.150.1

Plugin ver - 2.10

[TobiX]

monsam:

any update on this issue?