Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

http: add StringMatcher in HeaderMatcher and deprecate the old fields (exact, prefix, etc.) #17119

Merged
merged 16 commits into from
Jul 15, 2021
Merged
Show file tree
Hide file tree
Changes from 10 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion api/envoy/config/rbac/v3/rbac.proto
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,10 @@ option (udpa.annotations.file_status).package_version_status = ACTIVE;
// permissions:
// - and_rules:
// rules:
// - header: { name: ":method", exact_match: "GET" }
// - header:
// name: ":method"
// string_match:
// exact: "GET"
// - url_path:
// path: { prefix: "/products" }
// - or_rules:
Expand Down
5 changes: 4 additions & 1 deletion api/envoy/config/rbac/v4alpha/rbac.proto

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

34 changes: 28 additions & 6 deletions api/envoy/config/route/v3/route_components.proto
Original file line number Diff line number Diff line change
Expand Up @@ -1857,7 +1857,7 @@ message RateLimit {
// value.
//
// [#next-major-version: HeaderMatcher should be refactored to use StringMatcher.]
// [#next-free-field: 13]
// [#next-free-field: 14]
message HeaderMatcher {
option (udpa.annotations.versioning).previous_message_type = "envoy.api.v2.route.HeaderMatcher";

Expand All @@ -1872,12 +1872,16 @@ message HeaderMatcher {
// Specifies how the header match will be performed to route the request.
oneof header_match_specifier {
// If specified, header match will be performed based on the value of the header.
string exact_match = 4;
// This field is deprecated. Please use :ref:`string_match <envoy_v3_api_field_config.route.v3.HeaderMatcher.string_match>`.
string exact_match = 4
[deprecated = true, (envoy.annotations.deprecated_at_minor_version) = "3.0"];

// If specified, this regex string is a regular expression rule which implies the entire request
// header value must match the regex. The rule will not match if only a subsequence of the
// request header value matches the regex.
type.matcher.v3.RegexMatcher safe_regex_match = 11;
// This field is deprecated. Please use :ref:`string_match <envoy_v3_api_field_config.route.v3.HeaderMatcher.string_match>`.
type.matcher.v3.RegexMatcher safe_regex_match = 11
[deprecated = true, (envoy.annotations.deprecated_at_minor_version) = "3.0"];

// If specified, header match will be performed based on range.
// The rule will match if the request header value is within this range.
Expand All @@ -1898,28 +1902,46 @@ message HeaderMatcher {

// If specified, header match will be performed based on the prefix of the header value.
// Note: empty prefix is not allowed, please use present_match instead.
// This field is deprecated. Please use :ref:`string_match <envoy_v3_api_field_config.route.v3.HeaderMatcher.string_match>`.
//
// Examples:
//
// * The prefix *abcd* matches the value *abcdxyz*, but not for *abcxyz*.
string prefix_match = 9 [(validate.rules).string = {min_len: 1}];
string prefix_match = 9 [
deprecated = true,
(validate.rules).string = {min_len: 1},
(envoy.annotations.deprecated_at_minor_version) = "3.0"
];

// If specified, header match will be performed based on the suffix of the header value.
// Note: empty suffix is not allowed, please use present_match instead.
// This field is deprecated. Please use :ref:`string_match <envoy_v3_api_field_config.route.v3.HeaderMatcher.string_match>`.
//
// Examples:
//
// * The suffix *abcd* matches the value *xyzabcd*, but not for *xyzbcd*.
string suffix_match = 10 [(validate.rules).string = {min_len: 1}];
string suffix_match = 10 [
deprecated = true,
(validate.rules).string = {min_len: 1},
(envoy.annotations.deprecated_at_minor_version) = "3.0"
];

// If specified, header match will be performed based on whether the header value contains
// the given value or not.
// Note: empty contains match is not allowed, please use present_match instead.
// This field is deprecated. Please use :ref:`string_match <envoy_v3_api_field_config.route.v3.HeaderMatcher.string_match>`.
//
// Examples:
//
// * The value *abcd* matches the value *xyzabcdpqr*, but not for *xyzbcdpqr*.
string contains_match = 12 [(validate.rules).string = {min_len: 1}];
string contains_match = 12 [
deprecated = true,
(validate.rules).string = {min_len: 1},
(envoy.annotations.deprecated_at_minor_version) = "3.0"
];

// If specified, header match will be performed based on the string match of the header value.
type.matcher.v3.StringMatcher string_match = 13;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@snowp is copying the StringMatcher proto to the xds repo in cncf/xds#8. It might be a good idea to wait for that to land and then use the new type here, since that will save us the trouble of migrating this later.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the current StringMatcher is used everywhere in many places, is it better to wait and change all usages at once? It is already a large change for control plane to use the StringMatcher to replace the deprecated fields, just want to make it less complicated by not depending on the extra xds repo for now.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe that @snowp was going to change the existing code that uses StringMatcher to be templated so that it would work with either copy of StringMatcher. Once that's done, I think it should be trivial to use the new type here.

I think this just comes down to a question of timing. Snow, how soon do you think you can land the StringMatcher change, including any necessary templatizing of code in Envoy?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm busy the next two days for company perf season, but I'll have more time to work on it starting Wednesday. Early review of #17096 could be helpful in speeding this up to make sure that there is agreement on the direction.

}

// If specified, the match result will be inverted before checking. Defaults to false.
Expand Down
41 changes: 6 additions & 35 deletions api/envoy/config/route/v4alpha/route_components.proto

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions api/envoy/type/matcher/v3/string.proto
Original file line number Diff line number Diff line change
Expand Up @@ -62,8 +62,8 @@ message StringMatcher {
string contains = 7 [(validate.rules).string = {min_len: 1}];
}

// If true, indicates the exact/prefix/suffix matching should be case insensitive. This has no
// effect for the safe_regex match.
// If true, indicates the exact/prefix/suffix/contains matching should be case insensitive. This
Copy link
Contributor Author

@yangminzhu yangminzhu Jun 29, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note: the containts field is also supported but just not documented in the API.

// has no effect for the safe_regex match.
// For example, the matcher *data* will match both input string *Data* and *data* if set to true.
bool ignore_case = 6;
}
Expand Down
4 changes: 2 additions & 2 deletions api/envoy/type/matcher/v4alpha/string.proto

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 2 additions & 1 deletion configs/envoy_double_proxy.template.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,8 @@
"@type": type.googleapis.com/envoy.extensions.filters.http.health_check.v3.HealthCheck
pass_through_mode: false
headers:
- exact_match: /healthcheck
- string_match:
exact: /healthcheck
name: :path
- name: envoy.filters.http.buffer
typed_config:
Expand Down
3 changes: 2 additions & 1 deletion configs/envoy_front_proxy.template.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,8 @@
pass_through_mode: false
headers:
- name: ":path"
exact_match: "/healthcheck"
string_match:
exact: "/healthcheck"
- name: envoy.filters.http.buffer
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.http.buffer.v3.Buffer
Expand Down
6 changes: 4 additions & 2 deletions configs/envoy_service_to_service.template.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,8 @@
prefix: "/"
headers:
- name: content-type
exact_match: application/grpc
string_match:
exact: application/grpc
route:
cluster: local_service_grpc
- match:
Expand All @@ -39,7 +40,8 @@
pass_through_mode: true
headers:
- name: ":path"
exact_match: "/healthcheck"
string_match:
exact: "/healthcheck"
cache_time: 2.5s
- name: envoy.filters.http.buffer
typed_config:
Expand Down
3 changes: 2 additions & 1 deletion configs/terminate_http2_post.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,8 @@ static_resources:
prefix: "/"
headers:
- name: ":method"
exact_match: "POST"
string_match:
yangminzhu marked this conversation as resolved.
Show resolved Hide resolved
exact: "POST"
route:
cluster: service_google
upgrade_configs:
Expand Down
18 changes: 12 additions & 6 deletions docs/root/configuration/http/http_filters/tap_filter.rst
Original file line number Diff line number Diff line change
Expand Up @@ -78,11 +78,13 @@ An example POST body:
- http_request_headers_match:
headers:
- name: foo
exact_match: bar
string_match:
exact: bar
- http_response_headers_match:
headers:
- name: bar
exact_match: baz
string_match:
exact: baz
output_config:
sinks:
- streaming_admin: {}
Expand All @@ -103,11 +105,13 @@ Another example POST body:
- http_request_headers_match:
headers:
- name: foo
exact_match: bar
string_match:
exact: bar
- http_response_headers_match:
headers:
- name: bar
exact_match: baz
string_match:
exact: baz
output_config:
sinks:
- streaming_admin: {}
Expand Down Expand Up @@ -143,7 +147,8 @@ Another example POST body:
- http_request_headers_match:
headers:
- name: foo
exact_match: bar
string_match:
exact: bar
- http_request_generic_body_match:
patterns:
- string_match: test
Expand Down Expand Up @@ -242,7 +247,8 @@ An static filter configuration to enable streaming output looks like:
http_response_headers_match:
headers:
- name: bar
exact_match: baz
string_match:
exact: baz
output_config:
streaming: true
sinks:
Expand Down
6 changes: 4 additions & 2 deletions docs/root/configuration/operations/tools/router_check.rst
Original file line number Diff line number Diff line change
Expand Up @@ -69,10 +69,12 @@ expects a cluster name match of "instant-server".::
path_redirect: ...,
request_header_matches:
- name: ...,
exact_match: ...
string_match:
exact: ...
response_header_matches:
- name: ...,
exact_match: ...
string_match:
exact: ...
- name: ...,
presence_match: ...

Expand Down
4 changes: 4 additions & 0 deletions docs/root/version_history/current.rst
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,7 @@ New Features
* http: added a new option to upstream HTTP/2 :ref:`keepalive <envoy_v3_api_field_config.core.v3.Http2ProtocolOptions.connection_keepalive>` to send a PING ahead of a new stream if the connection has been idle for a sufficient duration.
* http: added the ability to :ref:`unescape slash sequences <envoy_v3_api_field_extensions.filters.network.http_connection_manager.v3.HttpConnectionManager.path_with_escaped_slashes_action>` in the path. Requests with unescaped slashes can be proxied, rejected or redirected to the new unescaped path. By default this feature is disabled. The default behavior can be overridden through :ref:`http_connection_manager.path_with_escaped_slashes_action<config_http_conn_man_runtime_path_with_escaped_slashes_action>` runtime variable. This action can be selectively enabled for a portion of requests by setting the :ref:`http_connection_manager.path_with_escaped_slashes_action_sampling<config_http_conn_man_runtime_path_with_escaped_slashes_action_enabled>` runtime variable.
* http: added upstream and downstream alpha HTTP/3 support! See :ref:`quic_options <envoy_v3_api_field_config.listener.v3.UdpListenerConfig.quic_options>` for downstream and the new http3_protocol_options in :ref:`http_protocol_options <envoy_v3_api_msg_extensions.upstreams.http.v3.HttpProtocolOptions>` for upstream HTTP/3.
* http: added :ref:`string_match <envoy_v3_api_field_config.route.v3.HeaderMatcher.string_match>` in the header matcher.
* input matcher: a new input matcher that :ref:`matches an IP address against a list of CIDR ranges <envoy_v3_api_file_envoy/extensions/matching/input_matchers/ip/v3/ip.proto>`.
* jwt_authn: added support to fetch remote jwks asynchronously specified by :ref:`async_fetch <envoy_v3_api_field_extensions.filters.http.jwt_authn.v3.RemoteJwks.async_fetch>`.
* listener: added ability to change an existing listener's address.
Expand All @@ -129,3 +130,6 @@ Deprecated
* dns_filter: the field :ref:`known_suffixes <envoy_v3_api_field_data.dns.v3.DnsTable.known_suffixes>` is deprecated. The internal data management of the filter has changed and the filter no longer uses the known_suffixes field.
* dynamic_forward_proxy: the field :ref:`use_tcp_for_dns_lookups <envoy_v3_api_field_extensions.common.dynamic_forward_proxy.v3.DnsCacheConfig.use_tcp_for_dns_lookups>` is deprecated in favor of :ref:`dns_resolution_config <envoy_v3_api_field_extensions.common.dynamic_forward_proxy.v3.DnsCacheConfig.dns_resolution_config>` which aggregates all of the DNS resolver configuration in a single message.
* http: :ref:`xff_num_trusted_hops <envoy_v3_api_field_extensions.filters.network.http_connection_manager.v3.HttpConnectionManager.xff_num_trusted_hops>` is deprecated in favor of :ref:`original IP detection extensions<envoy_v3_api_field_extensions.filters.network.http_connection_manager.v3.HttpConnectionManager.original_ip_detection_extensions>`.
* http: The HeaderMatcher fields :ref:`exact_match <envoy_v3_api_field_config.route.v3.HeaderMatcher.exact_match>`, :ref:`safe_regex_match <envoy_v3_api_field_config.route.v3.HeaderMatcher.safe_regex_match>`,
:ref:`prefix_match <envoy_v3_api_field_config.route.v3.HeaderMatcher.prefix_match>`, :ref:`suffix_match <envoy_v3_api_field_config.route.v3.HeaderMatcher.suffix_match>` and
:ref:`contains_match <envoy_v3_api_field_config.route.v3.HeaderMatcher.contains_match>` are deprecated by :ref:`string_match <envoy_v3_api_field_config.route.v3.HeaderMatcher.string_match>`.
5 changes: 4 additions & 1 deletion generated_api_shadow/envoy/config/rbac/v3/rbac.proto

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 4 additions & 1 deletion generated_api_shadow/envoy/config/rbac/v4alpha/rbac.proto

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading