Question on flow mode / River syntax for parsing Faro agent logs

[davisford]

Hello, we have enabled agent in our own cluster, and I have enabled the faro-web-sdk on the client side. It is sending the logs as JSON strings. I would like to use the River function json_decode to parse the JSON and add some additional labels, but not sure the best way to proceed.

Here's the River config for the agent:

faro.receiver "default" {    
    server {
        listen_address = "0.0.0.0"
        cors_allowed_origins = ["*"]
    }
    
    extra_log_labels = {
        app_name = "frontend",
        app_environment = "dev",
        source = "faro-sdk",
    }
    
    output {
        logs = [loki.write.default.receiver]
    }
}

loki.write "default" {
    endpoint {
        url = "http://loki:3100/loki/api/v1/push"
    }
}

loki.echo "test" {}

Here's a sample of what a log message ends up looking like in Grafana/Loki:

You can see the three extra_log_labels that were added there. What I want to do is parse the log message from JSON into an object and add some of the fields in that object to become extra_log_labels.

Considering that the JSON content of the log message looks like this:

{
    "0": "testing faro",
    "_meta": {
        "runtime": "Browser",
        "browser": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36",
        "name": "index.tsx",
        "parentNames": [
            "bridge"
        ],
        "date": "2024-02-04T18:08:30.160Z",
        "logLevelId": 3,
        "logLevelName": "INFO",
        "path": {
            "fullFilePath": "https://localhost:4200/build/chunks/chunk-WBYRT6AL.js",
            "fileName": "chunk-WBYRT6AL.js",
            "fileNameWithLine": "chunk-WBYRT6AL.js:2994",
            "fileColumn": "18",
            "fileLine": "2994",
            "filePath": "/build/chunks/chunk-WBYRT6AL.js",
            "filePathWithLine": "/build/chunks/chunk-WBYRT6AL.js:2994"
        }
    }
}

Would I need to do something like the following, if I wanted to add, let's say the name field in the JSON as an extra label in Loki?

      extra_log_labels = json_decode(faro.receiver.default.message)["_meta"]["name"]

Also, I read the json_decode function can fail. Is there a safe way to do this to test if the value exists in the object? Some messages that come through to Loki through Faro are kind=measurement, or kind=event. Can I do this in a way that I test for kind=log -- then I can control if the field inside the JSON will always exist?

[titaneric]

Following river script should meet your need.

Note that I use multiple pipelines to process the faro SDK log.
First, I add additional labels for faro SDK with app_name, app_environment, and source.
Then, I match app_name="frontend" and extract kind into new labels.
At last, I match kind="log" to extract your JSON message, and extract to new labels, too.

logging {
    level  = "debug"
    format = "logfmt"
}

faro.receiver "default" {    
    server {
        listen_address = "0.0.0.0"
        listen_port = "12347"
        cors_allowed_origins = ["*"]
    }
    
    extra_log_labels = {
        app_name = "frontend",
        app_environment = "dev",
        source = "faro-sdk",
    }
    
    output {
        logs = [loki.process.logfmt.receiver]
    }
}

loki.process "logfmt" {
    forward_to = [loki.process.json.receiver]
  
    stage.match {
        selector = "{app_name=\"frontend\"}"
    
        stage.logfmt {
            mapping = { "kind" = ""}
        }
        stage.match {
            selector = "{kind=\"log\"}"
        
            stage.logfmt {
                mapping = { "message" = "" }
            
            }
            stage.json {
                source      = "message"
                expressions = {name = "_meta.name"}
            }

            stage.labels {
                values = {
                    name = "",
                }
            }
        }

        stage.labels {
            values = {
                kind  = "",         
            }
        }
    }
}
loki.process "json" {
    forward_to = [loki.echo.echo.receiver, loki.write.local_loki.receiver]
  
        stage.match {
            selector = "{kind=\"log\"}"
        
            stage.logfmt {
                mapping = { "message" = "" }
            
            }
            stage.json {
                source      = "message"
                expressions = {name = "_meta.name"}
            }

            stage.labels {
                values = {
                    name = "",
                }
            }
        }
}
loki.echo "echo" {
}
loki.write "local_loki" {
    endpoint {
        url = "http://gateway:3100/loki/api/v1/push"
    }
}

[davisford]

Thanks @titaneric -- also dropping this Slack chan response in here as it may prove useful info for anyone else looking to do similar: