Welcome to Stats Service Discussions!

[cedricbonhomme]

👋 Welcome!

We’re using Discussions as a place to connect with other members of our community. We hope that you:

• Ask questions you’re wondering about.
• Share ideas.
• Engage with other community members.
• Welcome others and are open-minded. Remember that this is a community we build together 💪.

To get started, comment below with an introduction of yourself and tell us about what you do with this community.

[pundorra]

Hi!

I took a look at the stats ( https://dashboard.monarc.lu/stats/ ) and I have some comments to make the page more useful to the MONARC user. The idea of the page has a lot of potential but its message should be made stronger.

Here are some things to consider to improve about the sections Trends for Threats and for Trends for Vulnerabilities:

• there is not enough information on the dataset that generates this:

  • What is in the data?
  • Where does the data come from? Answering this question is important for this page.
  • Why just 2 instances from the many MONARC instances?

• the trends for both vulnerabilities and threats are almost constant. Is that news? What the trends show now is that there were some changes in November and twice in January. Is this what we really want to say? It could be that something happened in Nov and January and maybe it is interesting, but I think that is less important when the dataset is that small.

• the graphs for trends and vulns have many more colours in the legend than in the lines, is that an overlap? It is confusing to see more labels than lines.

• the green label in threats and the red label in vulns are associated with some hex code, so they are not understandable to the innocent reader.

• the scoring on the y axis is not explained and would need to be (averages per date of the different averageRate for threats and vulnerabilities -- can it be a bit clearer on how the averageRate was calculated?)

• are we really interested to see all the data points per day? I think we should show an aggregate per week or month, from the point of view of the user.

• a concern is that there seem to be no data points between December 8 and Dec. 16 included, or Dec. 19 and Jan 4. Is that the case?

• what is/might be in the data but not sufficiently emphasised:

  • top 5 threats / top 5 vulns in terms of averages - maybe it can be shown with a pie chart, for example.
  • lowest threats and vulns in terms of averages - same as above.
  • there are some clusters (the lines between 2.5 -> 3, the lines between 1.5 -> 2, and the lines around the score of 1.0). It needs to be seen what the clusters mean (if they mean anything)
  • where there were changes in the threats and vulns reported.

• an interpretation of the data shown in the graphs, and changes with respect to the last reporting period. Maybe we can have an update every quarter, for example.

Some of these observations can also apply to the Risks section. In addition:

• the legend is a bit incomplete, I suppose it means that the stronger colour means "current" and lighter colour means "residual".
• colouring: why is medium coloured with red and low with orange?
• for the low risks: why is residual higher than current? :)
• visually, the graphs are taking a lot of space compared to what they say.
• why are the y scales up to 16 and 25 actually, and what does that mean?
• is the residual high risk 0 for the operational risk?
• what is/might be in the data but not sufficiently emphasised:
  • in terms of quantity, most risks are around the medium area (or low? now I am confused about the intention of the bar chart)
  • all operational risks rated high are taken care of (maybe?)

Here it went, I hope it was useful.

[cedricbonhomme]

I will answer with separated comments. It easier.

So first point, about the data. The button at the top of the main page is a link to:
https://dashboard.monarc.lu/help#how-does-it-work
it is supposed to explain a bit more. There is also the section "What kind of data is envolved ?" But still it can be improved.

Why two instances? Because it's just a test with data of CASES ;-) Later, it should be more like this:
https://monarc-stats-service.readthedocs.io/en/latest/architecture.html#organization-level
organizations will push data, if they want to. So here, it would be, eventually, my.monarc.lu minus some clients.

[pundorra]

Yes, I couldn't have missed the button at the top, but when you click on it, it doesn't say what the data is, but only "Most of this data is already publicly available and bundled with MONARC." Indeed, it could be a bit more descriptive about the dataset -- just to calm down any MONARC user who might prefer assurance of the confidentiality / anonymity the tool offers.

Yeah, 2 instances is better than none :D It would be great to have more, we'll work on that.

(Btw, minor typo: it's not "envolved" but "involved" there)

[cedricbonhomme]

Arf, fixed. thanks.

[cedricbonhomme]

Trends are almost constant, yes. It's due to the data (some old analysis not updated since some time). On my local instance it's more funky. Again, it's almost data from MyPrint.

[pundorra]

It would be actually interesting to know what makes users refresh their analyses. Don't know how we can get this data other than by launching a survey.

[cedricbonhomme]

the green label in threats and the red label in vulns are associated with some hex code, so they are not understandable to the innocent reader.

These are UUIDs (not exadecimal) of objects that we were not able to find the label "on the fly". I'll change this.

To explain a bit more. The charts request the data to the backend via an API and as result they receive UUIDs (of vulns, threats, etc.) and values (calculated on server side). The JavaScript part in your browser will translate the UUIDs to human readable labels. This is just used for the legend. It's done here. (example: https://objects.monarc.lu/api/v2/object/?uuid=b402d557-4576-11e9-9173-0800277f0571&language=DE)
And as you can see, an async function is querying MOSP in order to find a label in a specific language. But it can happen that the object is not available on MOSP. So we can not find the label based on the UUID. So it just display the UUID ;-) And again the data is not good.

It's not big deal. Because to be honest it is even not needed to call MOSP in order to get the label. Labels are normally available directly for Stats Service. The backend of MONARC FO is responsible of collecting stats (for example daily) and to send the stats to Stats Service. And MONARC is sending the labels to Stats Service. (But I can not be sure in which language).
It was a use case for the MOSP API. I'll improve this. Anyway this code might disappear ;-)
(fyi stats are stored in a jsonb field. Consequently the format is flexible. The MONARC stats "collector" can just send a valid JSON data, with ou without labels....)

And about the colors for the first two charts, it's simply coming from this definition. Kind of random pastel colors.

The charts for the risks are using the same color code than MONARC normally (rgb(253, 102, 31), rgb(255, 188, 28) rgb(214, 241, 7)).

[pundorra]

nice, thanks for this explanation, I understand better now!

[cedricbonhomme]

the scoring on the y axis is not explained and would need to be (averages per date of the different averageRate for threats and vulnerabilities -- can it be a bit clearer on how the averageRate was calculated?)

Yes I agree. I thought about this, and I realized it is not much more clear in the MONARC dashboard. We need a clear definition first I think. I also looked in our documentation. Sadly it is not explained simply.

[pundorra]

If anybody starts to look into this, I can assist & help discuss.

[cedricbonhomme]

are we really interested to see all the data points per day? I think we should show an aggregate per week or month, from the point of view of the user.

I agree. It will be better with "zoom-able" charts. It's not big deal I think. Also there is the possibility to reduce the window of time. Already possible in backend.

[cedricbonhomme]

top 5 threats / top 5 vulns in terms of averages - maybe it can be shown with a pie chart, for example.

good idea for the pie chart. And why not with only very few slices. I wanted to make this page quite "high level". The top threats/vulns should be clear.
Or we can keep the page /stats as it is (and in the future more detailed). And the simple pie charts would go at the root / .

[pundorra]

Sure, I think the point is to have a visual that conveys something useful about the stats, that the common user didn't know before, and that doesn't take too much effort to see.

[cedricbonhomme]

So I won't comment much more about the data, they are not good. It would be more interesting with currently active risk analysis (with a freshness threshold, to not have a dashboard based on outdated data - because we know the threats and vulnerabilities are changing so fast xD ).

[cedricbonhomme]

for the low risks: why is residual higher than current? :)

yes, it's fun. It's because these values are averages with data across several analysis. Maybe for a single analysis you would have less residual than current. But by making an average on several analysis you can have a higher number of residual risks...
This graph clearly must be explained ;-) or replaced.

[pundorra]

yes, agree, we need to refine this a little, not to create confusion for the "innocent reader"