AWS CloudWatch Logs - create infrequent access type with create_missing_group

[hawkesn]

A note for the community

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Use Cases

When creating AWS CloudWatch log groups, there are two types: standard and infrequent access (see: https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/CloudWatch_Logs_Log_Classes.html).

By default, the create_missing_group will select the standard type. There are no options to select which type of log group to create.

Attempted Solutions

Workaround

We can create the log group manually through the AWS CLI or interface and then disable create_missing_group. We point vector to the pre-created log group and it would work.

This works if you have a low number of log groups, but if you have hundreds or if the log groups are dynamically generated, this is not feasible.

Proposal

Have an option to select the type of log group via the create_missing_group

References

No response

Version

0.42.0

[PriceHiller]

I'll try to work on this tomorrow if nobody has by then.

Question about implementation for a vector dev (@jszwedko ?):

1. Would we do a breaking change here and make the create_missing_group take a string of either Standard or Infrequent for the log groups?

2. Or would we allow true & false values still, but also allow the strings Standard and Infrequent as well for the value for create_missing_group? I think I have a sketchy way of doing this by wrangling the Serde deserializer into a horrible shape. This would avoid a breaking change.

3. Or do we create another entry like log_group_type or similar and allow one of Standard or Infrequent there and only use that entry if create_missing_group is enabled?

Of the three, I'm partial to the first option, but I don't want to introduce two breaking changes in a single 7-day period.

[pront]

I think the least intrusive way is option (3) and at the same time we can mark create_missing_group as deprecated (from what I am reading it is not needed if the new log_group_type is set)