CMP-3566: Ensure a CLF exists and check for secure URLs #14031

yuumasato · 2025-10-21T19:22:26Z

Description:

Add an unfiltered yamlpath to check for existence of ClusterLogForwarders.
Copy the existing check and add the check for CLF existence

Rationale:

The cluster can be configured with log forwarders that are by default secure and use TLS, for example the AzureMonitor log forwarder. They don't have an URL key.
The existing check only checks the url of CLF that have it. If a CLF doesn't have it, we don't check it.
Problem is that we cannot differentiate between non-existent CLF, and CLF without url key. So we need to add a check for CLF existence.
Fixes https://issues.redhat.com/browse/CMP-3566

Change rule's existence checks to any_exist. This means that any ClusterLogForwarder that has a url key, needs to comply. If no such forwarder exists, it is fine. Previously, the rule would enforce that at least one such forwarder existed.

xiaojiey · 2025-10-22T04:28:10Z

verification pass. More details seen from the Jira ticket https://issues.redhat.com/browse/CMP-3566

Create a clusterlogforwarder with azMinitor;
Create two ssb: one with downstream ocp4-stig, the other one with upstream-ocp4-stig
Check the test result:
% oc get ccr | grep audit-log-forwarding-uses-tls
ocp4-stig-audit-log-forwarding-uses-tls FAIL medium
upstream-ocp4-stig-audit-log-forwarding-uses-tls PASS medium

ggbecker · 2025-10-22T14:02:23Z

/packit retest-failed

rhmdnd · 2025-10-22T15:21:59Z

applications/openshift/logging/audit_log_forwarding_uses_tls_observability_api/rule.yml

    # By using the objects filter we ensure we are getting the object to query for its url.
    filepath: "{{{ openshift_filtered_path('/apis/observability.openshift.io/v1/namespaces/openshift-logging/clusterlogforwarders', 'try [.items[].spec.outputs[][]|objects|select(.url != null).url] catch []') }}}"
    yamlpath: "[:]"
+    check_existence: any_exist


Does this mean the rule will pass if no log forwarding is enabled?

https://github.com/OVALProject/Language/blob/master/docs/oval-common-schema.md#ExistenceEnumeration

Do we want at_least_one_exists here?

Does this mean the rule will pass if no log forwarding is enabled?

Yes, 🫠

But at_least_one_exists doesn't work either:
If only AzureMonitor is configured, no ClusterLogForwarder will be returned, and the posture is still PASS.

@rhmdnd But If I'm pedantic, this rule is about ensuring that log forwarding uses TLS.
If no logs are forwarded, is it a PASS or FAIL?

There is a rule about enabling log forwarding: https://github.com/ComplianceAsCode/content/blob/master/applications/openshift/api-server/audit_log_forwarding_enabled/rule.yml

Unfortunately it is not automated.

When at_least_one_exists was set and only AzureMonitor is configured, the rule will FAIL.

% oc get ccr -n openshift-compliance| grep audit-log-forwarding-uses-tls ocp4-stig-audit-log-forwarding-uses-tls FAIL medium upstream-ocp4-stig-audit-log-forwarding-uses-tls FAIL medium % oc get clusterlogforwarders.observability.openshift.io clf-71770 -o=jsonpath={.spec.outputs} -n openshift-logging | jq -r [ { "azureMonitor": { "authentication": { "sharedKey": { "key": "shared_key", "secretName": "azure-secret-71770" } }, "customerId": "816936a6-c9f8-40f2-bda1-986582acd354", "logType": "case71770app_log" }, "name": "azure-app", "type": "azureMonitor" }, { "azureMonitor": { "authentication": { "sharedKey": { "key": "shared_key", "secretName": "azure-secret-71770" } }, "customerId": "816936a6-c9f8-40f2-bda1-986582acd354", "logType": "case71770infra_log" }, "name": "azure-infra", "type": "azureMonitor" }, { "azureMonitor": { "authentication": { "sharedKey": { "key": "shared_key", "secretName": "azure-secret-71770" } }, "customerId": "816936a6-c9f8-40f2-bda1-986582acd354", "logType": "case71770audit_log" }, "name": "azure-audit", "type": "azureMonitor" } ]

Let me try one thing.

This should be addressed now, this rule ensures that at least one secure CLF exists.

yuumasato · 2025-10-23T18:27:00Z

Maybe the same fix should be done for the old api too?
I'm looking into that

yuumasato · 2025-10-27T13:57:03Z

Maybe the same fix should be done for the old api too? I'm looking into that

Given that support for Logging Operator 5.x is ending soon:
https://access.redhat.com/support/policy/updates/openshift_operators

And that version 6.x only use observability API:
https://docs.redhat.com/pt-br/documentation/red_hat_openshift_logging/6.0/html-single/upgrading_logging/index#changes-in-logging-6_upgrading-to-logging-6

I'll not update the old logging rule.

yuumasato · 2025-10-27T13:57:42Z

I'll not update the old logging rule.

@xiaojiey @Anna-Koudelkova Are you okay with this?

xiaojiey · 2025-10-27T14:05:52Z

I'll not update the old logging rule.

@xiaojiey @Anna-Koudelkova Are you okay with this?
With the current approach, he rule works when azminitor/lokistack configured in the clusterlogforwarder.
The biggest issue with the current approach is: if no clusterlogforwarder configured at all, the rule will PASS. I think with this scenario it should return FAIL.
However, I tried with at_least_one_exists, it doesn't work, neither. When azminitor/lokistack configured in the clusterlogforwarder, the rule always FAIL.
Honestly speaking, I don't have good solution for this issue.

The yaml template can't generate the OVAL check we need to be able to assess the clusterlogforwarders correctly. Wee need a check that is capable of ensuring that: - there is at least one clusterlogforwarder - Any clusterlogforwarder object that has .url, is secure

xiaojiey · 2025-10-28T04:37:21Z

verification FAIL for the unsecure url udp://rsyslog.e2e-test-vector-syslog-cnrqt.svc:514:

No CLF(clusterlogforwarder) exists - rule FAIL
CLF with unsecure url udp://rsyslog.e2e-test-vector-syslog-cnrqt.svc:514 - rule should FAIL, but return PASS.
It is the same test result when using a url with "http" prefix.

% oc get -n openshift-logging clusterlogforwarders                                                                                         
NAME        AGE
clf-60699   38m
% oc get clusterlogforwarders.observability.openshift.io -n openshift-logging clf-60699 -o=jsonpath={.spec.outputs} | jq -r
[
  {
    "name": "rsyslog",
    "syslog": {
      "rfc": "RFC3164",
      "url": "udp://rsyslog.e2e-test-vector-syslog-lmv5k.svc:514"
    },
    "type": "syslog"
  }
]
% oc get ccr -n openshift-compliance| grep audit-log-forwarding-uses-tls
ocp4-stig-audit-log-forwarding-uses-tls                         FAIL     medium
upstream-ocp4-stig-audit-log-forwarding-uses-tls                PASS     medium

CLF with secure url - rule PASS
CLF with azminitor - rule PASS
CLF with lokistack - rule PASS

The cluster logging operator added a bunch of new forwarding implementations that allow users to ship their logs to various platforms and components. Some ClusterLogForwarders use a url, others use a host. In cases where we can inspect the url, we want to check to make sure it's not using an insecure protocol (http, tcp, udp). In other cases, where the forwarder is shipping logs to a dedicated service (like AzureMonitor), we can't actually inspect the protocol because the Azure forwarding configuration only exposes a `host` attribute, which doesn't use a protocol as part of the host string. Instead, it's baked into the forwarding implementation. This adds complexity to the rule because we need to: - Check that at least one ClusterLogForwarder exists - Each ClusterLogForwarder is configured to encrypt traffic to the forwarding endpoint - Short-circuit the check for special case forwarders, like AzureMonitor, that don't specify the protocol in the endpoint url/host Instead of looking for secure endpoints in each forwarder, which aren't implemented consistently, this commit reverses the logic so that it asserts no "insecure" endpoints are in a forwarding configuration. This works better for cases like AzureMonitor because if the rule doesn't find a `url` in the forwarder, is has nothing to compare the protocol check to, which means it passes. If a forwarder is configured to use plain old `http`, it will fail because the check asserts none exist against regular expression modeling unencrypted protocols. At the same time, we're maintaining the behavior where the rule fails is no forwarders exist at all. I believe this is ultimately due to the fact that "any_exists" OVAL checks will PASS if no pattern matches are made (filtering a log forwarder with url=http://example.com will not match a regular expression only looking for secure protocols, resuling in a PASS when it should actually fail due to how "any_exists" handles non-existent matches).

openshift-ci · 2025-11-04T23:00:40Z

@yuumasato: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/e2e-aws-openshift-node-compliance	`9073244`	link	true	`/test e2e-aws-openshift-node-compliance`

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

rhmdnd · 2025-11-04T23:22:20Z

/packit retest-failed

xiaojiey · 2025-11-05T13:10:24Z

Verification fail.
For the latest commit, all scenarios work except the scenario when CLF using azureMonitor/loki

No CLF exists - return FAIL, should FAIL
CLF with unsecure url - return FAIL, should FAIL
CLF with secure url - return PASS, should PASS
CLF with azminitor - return FAIL, should PASS
CLF with lokistack - return FAIL, should PASS

yuumasato added this to the 0.1.79 milestone Oct 21, 2025

yuumasato requested a review from xiaojiey October 21, 2025 19:22

yuumasato added the OpenShift OpenShift product related. label Oct 21, 2025

yuumasato requested review from Vincent056 and rhmdnd October 22, 2025 12:57

rhmdnd reviewed Oct 22, 2025

View reviewed changes

yuumasato changed the title ~~CMP-3566: Change rule checking log forwarder uses tls existence key~~ CMP-3566: Ensure a CLF exists and check for secure URLs Oct 27, 2025

rhmdnd force-pushed the fix-logforwarder-uses-tls branch from 283557f to 9073244 Compare November 4, 2025 21:30

CMP-3566: Ensure a CLF exists and check for secure URLs #14031

Are you sure you want to change the base?

CMP-3566: Ensure a CLF exists and check for secure URLs #14031

Conversation

yuumasato commented Oct 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description:

Rationale:

Uh oh!

xiaojiey commented Oct 22, 2025

Uh oh!

ggbecker commented Oct 22, 2025

Uh oh!

rhmdnd Oct 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

yuumasato Oct 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

yuumasato Oct 23, 2025

Choose a reason for hiding this comment

Uh oh!

xiaojiey Oct 27, 2025

Choose a reason for hiding this comment

Uh oh!

yuumasato Oct 27, 2025

Choose a reason for hiding this comment

Uh oh!

yuumasato Oct 27, 2025

Choose a reason for hiding this comment

Uh oh!

yuumasato commented Oct 23, 2025

Uh oh!

yuumasato commented Oct 27, 2025

Uh oh!

yuumasato commented Oct 27, 2025

Uh oh!

xiaojiey commented Oct 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

xiaojiey commented Oct 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

openshift-ci bot commented Nov 4, 2025

Uh oh!

rhmdnd commented Nov 4, 2025

Uh oh!

xiaojiey commented Nov 5, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

yuumasato commented Oct 21, 2025 •

edited

Loading

rhmdnd Oct 22, 2025 •

edited

Loading

yuumasato Oct 23, 2025 •

edited

Loading

xiaojiey commented Oct 27, 2025 •

edited

Loading

xiaojiey commented Oct 28, 2025 •

edited

Loading