i need create a prefix with the format yyyy/MM/dd

[ghost]

I need to create a prefix that allows me to have the year, month and day as a variable so that my logstash process does not analyze the millions of records I have in my storage but only uses the folder of the current day.

i tried with this but doesnt work

        prefix => "%{YEAR}/%{MONTHNUM}/%{MONTHDAY}/"

[janmg]

Apart from prefix which is the starting directory of what you want to process there is also path_filters which does an fnmatch using a glob. I don't know the details, but I image something like /202[4-9]/ will look at new files and stores the size in the registry just in case the file grows.
https://www.rubydoc.info/stdlib/core/File.fnmatch

Because logstash is giving me problems with ruby dependancies, I have started rewriting the plugin in golang
https://github.com/janmg/blob-to-queue/
There I'm pondering if keeping a list of all the files and their sizes in a registry, just like done in the original plugin https://github.com/Azure/azure-diagnostics-tools/tree/master/Logstash/logstash-input-azureblob
Perhaps using a directory counter based on year month day hour makes more sense also for nsgflowlogs