Add parameter to skip ignoring files

In our use-case we have a race condition. We are creating files, then we are setting up privileges. In function 'extend_paths' we can see that if file is not readable once it is ignored, so it happened to us, that some logs were ignored forever. I think that introducing such parameter should be the easiest way to avoid similar problems in the future. Signed-off-by: Artur Laskowski <[email protected]>
fluent · Dec 11, 2023 · 173860d · 173860d
1 parent 7e9eba7
commit 173860d
Showing 1 changed file with 3 additions and 1 deletion.
diff --git a/lib/fluent/plugin/in_tail.rb b/lib/fluent/plugin/in_tail.rb
@@ -113,6 +113,8 @@ def initialize
     config_param :follow_inodes, :bool, default: false
     desc 'Maximum length of line. The longer line is just skipped.'
     config_param :max_line_size, :size, default: nil
+    desc 'Skip ignoring files.'
+    config_param :skip_ignoring_files, :bool, defaults: false
 
     config_section :parse, required: false, multi: true, init: true, param_name: :parser_configs do
       config_argument :usage, :string, default: 'in_tail_parser'
@@ -303,7 +305,7 @@ def expand_paths
                   true
                 end
               else
-                if is_file
+                if is_file && !skip_ignoring_files?
                   unless @ignore_list.include?(p)
                     log.warn "#{p} unreadable. It is excluded and would be examined next time."
                     @ignore_list << p if @ignore_repeated_permission_error