Metricbeat disk usage visualization not working correctly with interval:auto (TSVB)

[stacey-gammon]

I'm not sure if this is a TSVB error with the interval: auto setting, or a case of metricbeat default visualizations not setting the right defaults, but I am able to repro the issue mentioned in this discuss ticket: https://discuss.elastic.co/t/disappearing-disk-usage/129095

cc @simianhacker @elastic/beats

[simianhacker]

This has to do with the frequency of the data vs. the auto bucket size. If your data is coming in every 1 minute but your auto bucket size ends up being 10 seconds. Then when your page loads it will sometimes be zero ( 83% of the time it WILL be zero), just depends if the data shows up in the last bucket we display or not. When you customize the data frequency for Metricbeat to 1m you also need to update all the TSVB based visualization to use >=1m for the interval. At this time there is no way to fix this other than setting the interval under panel options to >=1m without some kind of communication from the Metricbeat process notifying Kibana's visualization that the frequency is lower then the visualizations, maybe the "data sources" feature will fix this.

I'm inclined to close this issue because TSVB is working as expected, if anything this should be a feature enhancement for having mechanism for Kibana where Metricbeat can update all the intervals when it detects a change in the publish interval config.

[stacey-gammon]

I think the bug then is that metricbeat doesn't set this interval in their default dashboard. Seems like their dashboard should work out of the box, and if the metricbeat default interval is 1m then so should the default visualization use 1m interval.

[simianhacker]

Yeah... the question is, if they run metricbeat --setup then change the interval because they realized they don't have enough storage to use the default interval, should they run metricbeat --setup again?

Probably a question for @monicasarbu or @tsg

[stacey-gammon]

Hm, yea I just don't recall touching system.yml to some custom interval, so it seemed to me that they are mismatched out of the box. Once a user changes the default .yml settings, then I can see it being reasonable to tell them they have to modify the visualization as well (though would also be bevery cool if metricbeat --setup updated it for them)

[simianhacker]

I wish there was a way to move this issue to the beats repo, this more of an issue with the defaults and less of an issue with anything in Kibana. I guess I will just create a new issue and link to this one.

[ruflin]

I agree on the part that our "dashboards" should set work well with our defaults and we must fix that. But it would be great to have a feature in Kibana that detects the interval and does the right thing. I'm thinking to send up the interval of each event under a certain field like event.interval. Based on this field Kibana could make a decision on what bucket size to use or show errors when a bucket size too small is used. Our users even change the interval over time so not requiring a setup call (which will overwrite all changes) would be great. Otherwise we probably would need an API to set the interval.

As this is out of scope of this issue I wonder if we should close this and open a new one for an enhancement request?

[exekias]

Would it make sense to take the last known value in this cases? I understand the last bucket is empty for the given interval and bucket size, but some of the previous buckets must contain the value we want.

[cdvv7788]

Any news on this issue? I still have issues with some visualizations (specially on disk). If I select "today" for the range, everything appears blank (My processes have been running for around 1 hour), so I have to select something smaller...the process to actually get data is kinda confusing.

[ensemblebd]

Honestly I feel like there is some sort of tiny discrepancy going on here.
Because I've installed metricbeat w/ default dashboards before - and never had this problem.

But installed it today on a new multi-server setup, 1 elk, and 2 other servers (1 windows 1 linux) on a LAN.
v7.15 across the board.
Have verified the timezones are all good across each, net sync'd for consistency.
But 90% of the dashboard visualizations (which come set to Auto by default), show nothing at all, or show disparate data (like dots on a area/line/bar chart). The ones that are gauges intermittently show data or not, but, 90+% of the time show nothing.
I suppose 90% of the imported visuals are TSVB.

I had to do both of these adjustments (against nearly all visualizations):

• Change auto to >=10s, the default interval of metricbeat for system metrics.
• Check the box for "Drop last bucket?".

The latter being the most critical it seems. When looking at discovery of course you see it pop every 10 seconds with data.
But looking at the visualization editor, that gap on the end (as it waits for new data), between what is effectively "now" and then, apparently results in miscalculations due to the final bucket being empty?
The line/area charts show it best, where the graph visually stops at the end, showing visually the gap in data. When set to auto + no drop.

That's why I feel like there's something more going on here.
Surely... if the default is Auto + no bucket drop, and thousands of people use metricbeat and tsvb gauges, surely... there would be more people in this thread right now.

I mean the only alternative to "something more going on", would be that the documentation of metricbeat needs updated to tell people to go and modify nearly all imported dashboards to have a accurate frequency per their config and bucket drop on?
That doesn't seem logical considering.

[stratoula]

@drewdaemon I feel we already have an issue about this, right?

[drewdaemon]

@stratoula I don't think we have an issue for this dashboard, but this is the same issue we see in elastic/integrations#1437

[stratoula]

I am closing this as there is already an issue for that, it is how the last value mode works in TSVB which is very confusing. Hopefully with Lens we are not going to have this problem