User EnvoyFilter targeting Istio Ingress Gateway should result in Istio CR in Warning state

[strekm]

Description

Extend Istio CR reconciliation to check if user created Istio EnvoyFilter configuring Istio Ingress Gateway is present on a cluster. Such Istio EnvoyFilter can significantly alter or even break Istio Ingress Gateway. User should be notified that he applied potentially dangerous configuration. When such Istio EnvoyFilter is detected during a reconciliation Istio CR status should be set to Warning state. Additionally new specific condition should be introduced to simplify troubleshooting. Description message should clearly state EF name, namespace and fact targeting Istio Ingress Gateway.

Misconfigured EF is rejected by Istio Ingress Gateway although error occurs when Istio Ingress Gateway is restarted. Reasons for restarting Istio Ingress Gateway are related to configuration changes as well as Istio upgrades. When this happens Istio CR should be in error state.

Reasons

Making user more aware of potential harmful global configuration.

ToDos [Developer]

• Get familiar with code
• Add tests for all relevant cases
• Implement status / condition for when EF is applied on IGW
• Think about integration tests
• Test on SKR, see if you can see previous message for potential problems on IGW restart
• Docs, RN
• Inform SRE / think about troubleshooting guide

PRs

• ...

ACs [PO]

• Istio CR in Warning state if user create EF targeting Ingress Gateway is present on a cluster
• Istio CR in Error state if Ingress Gateway / Istiod can't be started
• troubleshooting guide created

DoD [Developer & Reviewer]

• Provide unit and integration tests.
• Provide documentation.
• Verify if the solution works for both open-source Kyma and SAP BTP, Kyma runtime.
• If you changed the resource limits, explain why it was needed.
• If the default configuration of Istio Operator has been changed, you performed a manual upgrade test to verify that the change can be rolled out correctly.
• Verify that your contributions don't decrease code coverage. If they do, explain why this is the case.
• Add release notes.