Data Profile: AWS Cloud Trail Logs

[dforsber]

AWS Cloud Trail records all the API actions and you can create trails to export these logs onto S3.
https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-create-a-trail-using-the-console-first-time.html

Create a Data Profile for Boiling Insights that reads these logs, compacts and optimises them into Parquet files with the default year/month/day/hour partitioning on S3. Create example visualisations and the needed aggregation tables for them.

[dforsber]

There's a start here already:
https://github.com/boilingdata/boiling-insights/tree/main/data-profiles/aws-cloudtrail-logs

[dforsber]

AWS CloudTrail logs ETL works. There is no aggregated tables or charts yet, though.