Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for make-series, series_fir, series_subtract #57

Open
lothar7 opened this issue Sep 3, 2024 · 3 comments
Open

Support for make-series, series_fir, series_subtract #57

lothar7 opened this issue Sep 3, 2024 · 3 comments

Comments

@lothar7
Copy link

lothar7 commented Sep 3, 2024

If would be nice if we could generate moving averages using kusto-loco as described here
https://www.aizoo.info/post/kusto-lightning-fact-6-series-fir-generating-moving-averages-for-time-series-data

Might be other ways of doing it though that perhaps doesn't need all these functions

@NeilMacMullen
Copy link
Owner

make_series etc offer a huge amount of flexibillity but look like they are correspondingly difficult to implement. If all you want to do is effectively run a function over a sliding window and extend the result into an additional column then it could be done with a custom function that contains an accumulator.

The main problem with this approach is that there's currently no signal to reset the function implementation so you'd have to create a new context before each query or perform the reset manually. It's also a lot less flexible - performing calculations over grouped sets would required special logic inside the function

@lothar7
Copy link
Author

lothar7 commented Sep 4, 2024

In my specific usecase i need to calculate a moving average for the last x days as a timeseries so i am looking for ways of solving it. If it has to be a custom function then so be it. (Althoufh it would be nice if it was possible using just plain kusto). Can you elaborate on the special logic you would want for grouped sets?

@NeilMacMullen
Copy link
Owner

The CustomFunctions sample code contains this example method.

So a simple variation like

[KustoImplementation(Keyword = "rolledAverage")]
public partial class RolledAverageFunction
{
   private Dictionary<string,CircularBuffer> groupWindows. = new....
    public static string Impl(string groupid,long n)
    {
      var window = groupWindows.GetOrAdd(groupId);
     window.Add(n);
     return window.Average();
    }
}

would kind of work as a hack. As I say, the main issue is that there is currently no way to reset the window buffers between queries (unless you fetch them from a static provider and poke that from outside the query engine).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants