Skip to content

Commit 116d0d6

Browse files
spairydaipom
andauthored
YAML config: Supports parsing array format (#5126)
<!-- Thank you for contributing to Fluentd! Your commits need to follow DCO: https://probot.github.io/apps/dco/ And please provide the following information to help us make the most of your pull request: --> **Which issue(s) this PR fixes**: None. **What this PR does / why we need it**: Supports parsing array of basic data types when analyzing YAML configuration files. The current behavior of Fluentd: * Support YAML Array format for `$args` only. * Other Array options must be specified in a comma-separated format. * `retryable_response_codes: 503, 504` * If there’s only a single value, it must be specified as String or with the trailing comma. * `retryable_response_codes: "503"` * `retryable_response_codes: 503,` This supports YAML Array format for all options. ``` retryable_response_codes: - 503 ``` ``` retryable_response_codes: [503] ``` Note: This PR doesn't address the issue where setting an Int directly to an Array option causes ConfigError: `retryable_response_codes: 503` (The `C` case in #5126 (comment)) It could be a different issue. **Docs Changes**: Not Need **Release Note**: Same as the title. --------- Signed-off-by: Hear_Y <[email protected]> Signed-off-by: Daijiro Fukuda <[email protected]> Co-authored-by: Daijiro Fukuda <[email protected]>
1 parent dc21d83 commit 116d0d6

File tree

2 files changed

+159
-2
lines changed

2 files changed

+159
-2
lines changed

lib/fluent/config/yaml_parser/parser.rb

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -144,8 +144,12 @@ def section_build(name, config, indent: 0, arg: nil)
144144

145145
config.each do |key, val|
146146
if val.is_a?(Array)
147-
val.each do |v|
148-
sb.add_section(section_build(key, v, indent: indent + @base_indent))
147+
if section?(val.first)
148+
val.each do |v|
149+
sb.add_section(section_build(key, v, indent: indent + @base_indent))
150+
end
151+
else
152+
sb.add_line(key, val)
149153
end
150154
elsif val.is_a?(Hash)
151155
harg = val.delete('$arg')
@@ -164,6 +168,10 @@ def section_build(name, config, indent: 0, arg: nil)
164168

165169
SectionBuilder.new(name, sb, indent, arg)
166170
end
171+
172+
def section?(value)
173+
value.is_a?(Array) or value.is_a?(Hash)
174+
end
167175
end
168176
end
169177
end

test/test_config.rb

Lines changed: 149 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -343,6 +343,155 @@ def test_check_not_fetchd
343343
10.times { match_conf['type'] }
344344
assert_equal before_size, match_conf.unused.size
345345
end
346+
347+
data(
348+
"One String for $arg" => <<~CONF,
349+
config:
350+
- source:
351+
$type: sample
352+
tag: test
353+
- match:
354+
$type: stdout
355+
$tag: test.**
356+
buffer:
357+
$arg: tag
358+
$type: memory
359+
flush_mode: immediate
360+
CONF
361+
"Comma-separated String for $arg" => <<~CONF,
362+
config:
363+
- source:
364+
$type: sample
365+
tag: test
366+
- match:
367+
$type: stdout
368+
$tag: test.**
369+
buffer:
370+
$arg: tag, time
371+
$type: memory
372+
timekey: 1h
373+
flush_mode: immediate
374+
CONF
375+
"One-liner Array for $arg" => <<~CONF,
376+
config:
377+
- source:
378+
$type: sample
379+
tag: test
380+
- match:
381+
$type: stdout
382+
$tag: test.**
383+
buffer:
384+
$arg: [tag, time]
385+
$type: memory
386+
timekey: 1h
387+
flush_mode: immediate
388+
CONF
389+
"Multi-liner Array for $arg" => <<~CONF,
390+
config:
391+
- source:
392+
$type: sample
393+
tag: test
394+
- match:
395+
$type: stdout
396+
$tag: test.**
397+
buffer:
398+
$arg:
399+
- tag
400+
- time
401+
$type: memory
402+
timekey: 1h
403+
flush_mode: immediate
404+
CONF
405+
"One String for normal Array option" => <<~CONF,
406+
config:
407+
- source:
408+
$type: sample
409+
tag: test
410+
- match:
411+
$type: stdout
412+
$tag: test.**
413+
format:
414+
$type: csv
415+
fields: message
416+
CONF
417+
"Comma-separated String for normal Array option" => <<~CONF,
418+
config:
419+
- source:
420+
$type: sample
421+
tag: test
422+
- match:
423+
$type: stdout
424+
$tag: test.**
425+
inject:
426+
time_key: timestamp
427+
time_type: string
428+
format:
429+
$type: csv
430+
fields: timestamp, message
431+
CONF
432+
"One-liner Array for normal Array option" => <<~CONF,
433+
config:
434+
- source:
435+
$type: sample
436+
tag: test
437+
- match:
438+
$type: stdout
439+
$tag: test.**
440+
inject:
441+
time_key: timestamp
442+
time_type: string
443+
format:
444+
$type: csv
445+
fields: [timestamp, message]
446+
CONF
447+
"Multi-liner Array for normal Array option" => <<~CONF,
448+
config:
449+
- source:
450+
$type: sample
451+
tag: test
452+
- match:
453+
$type: stdout
454+
$tag: test.**
455+
inject:
456+
time_key: timestamp
457+
time_type: string
458+
format:
459+
$type: csv
460+
fields:
461+
- timestamp
462+
- message
463+
CONF
464+
"Multiple sections" => <<~CONF,
465+
config:
466+
- source:
467+
$type: sample
468+
tag: test
469+
- match:
470+
$type: copy
471+
$tag: test.**
472+
store:
473+
- $type: relabel
474+
$label: "@foo"
475+
- $type: relabel
476+
$label: "@bar"
477+
- label:
478+
$name: "@foo"
479+
config:
480+
- match:
481+
$type: stdout
482+
$tag: test.**
483+
- label:
484+
$name: "@bar"
485+
config:
486+
- match:
487+
$type: stdout
488+
$tag: test.**
489+
CONF
490+
)
491+
test "Can parse config without error" do |conf|
492+
write_config "#{TMP_DIR}/config.yaml", conf
493+
read_config("#{TMP_DIR}/config.yaml", use_yaml: true)
494+
end
346495
end
347496

348497
def write_config(path, data, encoding: 'utf-8')

0 commit comments

Comments
 (0)