Skip to content

Commit

Permalink
Revert backup chunk specification for suppressing useless chunk backu…
Browse files Browse the repository at this point in the history
…p sets

"Fluent::Unrecoverable Error" make the log set backup per a whole chunk unit, not only relevant log message.
It cause irrelevant logs lost due to whole chunk backup, in other words, not sending and backup whole chunk data set including the bigger log than 256 kb.

In addition the chunk backup file is remained as useless things, because there is no solution to restore the backup sets now.
Refer fluent/fluentd#3813
  • Loading branch information
bysnupy authored Jul 11, 2022
1 parent 7287d1a commit 0947b7d
Showing 1 changed file with 2 additions and 3 deletions.
5 changes: 2 additions & 3 deletions lib/fluent/plugin/out_cloudwatch_logs.rb
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,6 @@ module Fluent::Plugin
class CloudwatchLogsOutput < Output
Fluent::Plugin.register_output('cloudwatch_logs', self)

class TooLargeEventError < Fluent::UnrecoverableError; end

helpers :compat_parameters, :inject, :formatter

DEFAULT_BUFFER_TYPE = "memory"
Expand Down Expand Up @@ -379,7 +377,8 @@ def put_events_by_chunk(group_name, stream_name, events)
while event = events.shift
event_bytesize = event[:message].bytesize + EVENT_HEADER_SIZE
if MAX_EVENT_SIZE < event_bytesize
raise TooLargeEventError, "Log event in #{group_name} is discarded because it is too large: #{event_bytesize} bytes exceeds limit of #{MAX_EVENT_SIZE}"
log.warn "Log event in #{group_name} is discarded because it is too large: #{event_bytesize} bytes exceeds limit of #{MAX_EVENT_SIZE}"
break
end

new_chunk = chunk + [event]
Expand Down

0 comments on commit 0947b7d

Please sign in to comment.