does fluentd plugin support creation of dynamic data streams in opensearch ?

[moharana-subhashree]

I'm trying to make fluentd ingesting the container logs to opensearch data streams.

The expectation is that while creating data stream it has to append the current date to the data stream name.

If I use below configurations in match section, i:e logstash_format is true and include_timestamp is true , I expect data stream name shall be created with data stream name appended with timestamp values, as it works for normal index creations in opensearch.

but I find the data stream name is only getting created as .ds-<data stream name>-000001 only as it is configured in the match section even though logstash_format is true and include_timestamp is true is configured.

below configuration used:

  <source>
    @type dummy
    tag dummy.logs
    dummy {"message":"Sample log message"}
    interval 10s
  </source>



  <match **>
    @type opensearch_data_stream
    @log_level info
    host indexsearch
    port 9200
    user <<indexsearch_username>>
    password <<indexsearch_password>>
    ca_file <<isroot_cert>>
    scheme https
    ssl_verify true
    ssl_version TLSv1_2
    data_stream_name sample-ds
    logstash_format true
    include_timestamp true
    time_key @timestamp
    time_key_exclude_timestamp true
    logstash_dateformat %Y.%m.%d
  </match>

does these properties hold good for creating a data stream also?

[moharana-subhashree]

hi, can someone please help addressing the above asked query ?