Skip to content

Automatically forward CloudWatch logs posted by Amazon services to PaperTrail

License

Notifications You must be signed in to change notification settings

Signiant/PaperWatch

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

34 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

PaperWatch

A tool for automatically forwarding CloudWatch logs from your lambda functions to Papertrail.

Setup

Follow these steps to set up and deploy PaperWatch to AWS:

1. Configure

The two lambda functions reads the following configuration information from config/papertrail.json

Key Required Type Description
host Yes String The paper trail endpoint's address
port Yes Number The paper trail endpoint's port
consumer Yes String The name of the consumer function. Used by the Subscribe function to avoid subscribing the consumer to itself. Do not change unless you are modifying the CFN template.
retentionPeriod No Number If included, updates the CloudWatch log group's retention period (in days)
sources Yes Array of Objects List of log sources. A log source is identified by the prefix property. Log's generated by a source are passed to it's consumer function. Default configuration includes log sources for lambda functions and api gateway
exclude No Array of Strings List of lambda function names that the Subscribe function should ignore
logLevelExtractor Yes RegExp Regular expression that matches the incoming CloudWatch logs and extracts the log level from them. The extracted log level will be used in the PaperTrail log. Wrap the log level part in a regex group and you are good to go
defaultLogLevel Yes String If logLevelExtractor is unable to match, use the level provided here.
Example config:
{
  "host": "samplehost.papertrailapp.com",
  "port": 12345,
  "retentionPeriod": 3
  "sources": [
    {
      "prefix": "/aws/lambda/",
      "consumer": "LambdaLogConsumer"
    },
    {
      "prefix": "API-Gateway-Execution-Logs_",
      "consumer": "APIGatewayLogConsumer"
    }
  ]
  "exclude": [
    "TestFunction",
    "LoudFunction"
  ],
  "logLevelExtractor": ".* - (error|warn|info|verbose|debug|emerg|alert|crit|notice): ",
  "defaultLogLevel": "info"
}

2. Build

Running the build script will install dependencies and create the .zip deployment package (as lambda.zip)

paperwatch $ npm run build

3. Deploy

First, upload the deployment package to an s3 bucket. Take note of the bucket name, you will need this later. This can be done with the aws-cli as follows:

paperwatch $ aws s3 cp ./lambda.zip s3://<BUCKET_NAME>

Next, create a new cloudformation stack using from the template at cfn/deploy.cfn.json.
Enter a stack name. You will also be prompted to enter the following parameters:

  • SourceBucket - the name of the s3 bucket you uploaded the deployment package to
  • SourceKey - name of the deployment package in the s3 bucket (lambda.zip), including any prefixes if applicable

Once deployed, logs will be forwarded to papertrail for all new lambda functions.
Existing functions must be subscribed manually.

Manual Subscription

To manually subscribe a function, navigate to the CloudWatch console and follow these steps:

  1. Locate the log group for the function you wish to subscribe, selecting it by checking the box on the left
  2. Open the Actions dropdown and choose Subscription filters -> Create lambda subscription filter
  3. From the Lambda Function dropdown, select the consumer function (LambdaLogConsumer or APIGatewayLogConsumer by default)
  4. From the Log Format dropdown, select Other and click next
  5. For the Subscription filter name enter Papertrail
  6. Click Start Streaming

PaperWatch will now begin forwarding the functions logs to Papertrail

About

Automatically forward CloudWatch logs posted by Amazon services to PaperTrail

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published