[processor/filter] 'and' operator is not working proper when we use '!=' in conditions

[vaibhhavv]

Component(s)

processor/filter

What happened?

Description

I am using OTel v0.111.0, Helm chart v0.108.0.
I am utilising filter processor, and observed that when I utilise the filter processor with

1. and operator along with ==, then it works perfect
2. But if use and operator along with !=, then it does not filter out things correctly.

Note: In my case, I tested it for traces

Steps to Reproduce

Suppose you are receiving traces, on your OTel and you want to filter services on both below resource attributes and use them in pipeline

1. my.resource.attributes1 = value1
2. my.resource.attributes2 = value2

Working Filter

filter/working-filter:
      error_mode: ignore
      traces:
        span:
          - resource.attributes["my.resource.attributes1"] == "value1" and resource.attributes["my.resource.attributes2"] == "value2"

Not Working Filter

filter/not-working-filter:
      error_mode: ignore
      traces:
        span:
          - resource.attributes["my.resource.attributes1"] != "value1" and resource.attributes["my.resource.attributes2"] != "value2"

pipelines:
      traces/pipeline1:
        exporters:
          - otlphttp
        processors:
          - filter/working-filter
          - batch
        receivers:
          - otlp

      traces/pipeline2:
        exporters:
          - otlphttp
        processors:
          - filter/not-working-filter
          - batch
        receivers:
          - otlp

Expected Result

1. Pipeline1 should drop the services that have both attributes, rest others should get exporter. This is working as expected.
2. Pipeline2 should drop all the services other than the ones that contain both the attributes and forward them to the exporter.

Actual Result

1. Pipeline1 works as expected.
2. Pipeline2 is not working as expected and not dropping all other services. Mostly with != operator, I see it not works as expected.

Collector version

v0.111.0

Environment information

Environment

OS: (e.g., "Ubuntu 20.04")
Compiler(if manually compiled): (e.g., "go 14.2")

OpenTelemetry Collector configuration

No response

Log output

No response

Additional context

No response

[github-actions]

Pinging code owners:

• processor/filter: @TylerHelmuth @boostchicken

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[bacherfl]

Hi @vaibhhavv !
I'm currently looking into this, and have a question regarding the expected result (2):

Pipeline2 should drop all the services other than the ones that contain both the attributes and forward them to the exporter.

Do I understand this correctly that only traces where my.resource.attributes1 == value1 and my.resource.attributes2 == value2 should be propagated further? So for example, an item with my.resource.attributes1: value1, and my.resource.attributes2: x should be filtered?

[vaibhhavv]

Hey @bacherfl 😄
You understood it correctly.

For example, if a trace has resource attributes:

1. my.resource.attributes1 == value1 and my.resource.attributes2 == value2
   Expected Result: These traces/spans should be forwarded to the exporter.

2. my.resource.attributes1 == value1 and my.resource.attributes2 == X OR
   my.resource.attributes1 == X and my.resource.attributes2 == value2OR
   my.resource.attributes1 == XYZ and my.resource.attributes2 == XYZ
   Expected Result: These traces/spans should be dropped.

[bacherfl]

Thanks for the quick reply! I think in that case you need to adapt the condition, as with the current statement the condition requires both of the attribute values to be different in order to filter out the item. So, instead of

- resource.attributes["my.resource.attributes1"] != "value1" and resource.attributes["my.resource.attributes2"] != "value2"

Try to change it to the following:

- resource.attributes["my.resource.attributes1"] != "value1" or resource.attributes["my.resource.attributes2"] != "value2"

This way, the items will get filtered in case either attributes or attributes2 have a different value

[vaibhhavv]

@bacherfl I quickly checked it out. With or it works as I was expecting. It filtered out things correctly.

But now the question that arises in my head is: normally or checks at least for any one condition to get true.
Then how it is checking both conditions strictly just with or( it is doing the work that is expected from and)

- resource.attributes["my.resource.attributes1"] != "value1" or resource.attributes["my.resource.attributes2"] != "value2"

[bacherfl]

@bacherfl I quickly checked it out. With or it works as I was expecting. It filtered out things correctly.

But now the question that arises in my head is: normally or checks at least for any one condition to get true. Then how it is checking both conditions strictly just with or( it is doing the work that is expected from and)

In this case, the expectation was to filter an item in case either attribute1 or attribute2 does not have the expected value, so therefore there was no need to strictly check both conditions, as e.g. attribute1: X is sufficient to filter the item, regardless of the value of attribute2. Contrary to that, the statement using and needs both conditions to be true, so therefore an item would only be filtered if both attributes differ from the expected values.

[vaibhhavv]

But @bacherfl, If you see the requirement, I want those traces that have both my.resource.attributes1 = value1 and my.resource.attributes2 = value2 resource attributes.
In my understanding, it should be a strict match because I want both resources. That is the reason I was going with and.
I still have no idea how or is filtering them out, Could you give some more insights, please?

[bacherfl]

But @bacherfl, If you see the requirement, I want those traces that have both my.resource.attributes1 = value1 and my.resource.attributes2 = value2 resource attributes. In my understanding, it should be a strict match because I want both resources. That is the reason I was going with and. I still have no idea how or is filtering them out, Could you give some more insights, please?

Yes, so using the or condition will ensure that items will be filtered out if either attribute1 or attribute2 is not matched. So implicitly this means that only items where both attributes match will be passed through. This might become a bit more intuitive by applying De Morgan's law to the expression by turning it into a negated expression, which means that

attribute1 != value1 or attribute2 != value2

is equivalent to

!(attribute1 == value1 and attribute2 == value2)

The latter expression might make it a bit more apparent that items will get filtered out if not both of the attributes match (i.e. if they don't fulfil the strict requirement of both attributes having the expected value)

Note: The second expression is just a theoretic example, not sure if this works in the OTTL syntax

[vaibhhavv]

ohhh @bacherfl, thanks for recalling De Morgan's law. 🙌
Now, the implementation is pretty much clear. Many thanks for the support! 😄