V2 Consideration: Matching Arrays with sigma

[humpalum]

Matching Arrays with sigma

Problem

Sigma currently lacks the capability to match values within arrays in log events. This limitation restricts its effectiveness in matching complex log data that includes arrays. For example:

{
    "message": "Some Message",
    "process": "Some Process",
    "connections": [
        "123.1.1.1",
        "123.1.2.2",
        "123.3.3.3"
    ]
}

It is not possible to match a value in that array for example with:

connection: "123.1.1.1"

Proposed Solution

Enhance Sigma with key modifiers for array matching, such as |arrayAny, |arrayAll, |arrayOne, and |arrayNone. This would allow for more flexible and powerful matching scenarios.

connections|arrayAny: 123.1.1.1

This would return true if any element in the connections array matches "123.1.1.1".

connections|arrayAll|startswith: 123

This would return true if all elements in the array start with "123".

--> Inspired by https://expr-lang.org/docs/Language-Definition#array-functions

Further Enhancement

Even further we could allow matching of nested keys on arrays of objects:

{
    "message": "Some Message",
    "process": "Some Process",
    "connections": [
        {"protocol":"TCP","ip":"123.1.1.1"},
        {"protocol":"TCP","ip":"123.1.2.2"},
        {"protocol":"UDP","ip":"123.3.3.3"}
    ]
}

Allowing to "chain" multiple keys would allow to match on array of objects. Taking above example, we could write detections like this:

# Check if there is one UDP connection beginning with 123
connections|arrayAny:
    protocol: "UDP"
    ip|startswith: "123"
# --> Would return true

or

# Check if all connections beginning with 123 are using TCP
connections|arrayAll:
    protocol: "TCP"
    ip|startswith: "123"
# --> Would return false

I can see how this is a big change of current sigma philosophy and I am not sure how implementable this is in backends but taking elasticsearch as an example, it would work on nested objects: Nested Fields