Skip to content

Commit 6465760

Browse files
authored
Merge pull request #4422 from shuqz/shuqz-setup-conformance
[feat gw-api]add conformance test setup
2 parents 6d25794 + d39c15a commit 6465760

File tree

4 files changed

+151
-10
lines changed

4 files changed

+151
-10
lines changed

conformance/README.md

Lines changed: 121 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,121 @@
1+
2+
# Gateway API Conformance Testing
3+
4+
This directory contains conformance tests for the AWS Load Balancer Controller's Gateway API implementation. These tests validate compliance with the [Kubernetes Gateway API specification](https://gateway-api.sigs.k8s.io/concepts/conformance/).
5+
6+
## Prerequisites
7+
8+
- Kubernetes cluster (v1.19+)
9+
- kubectl configured to access your cluster
10+
- Gateway API CRDs installed
11+
- Go 1.21+ (for running tests locally)
12+
13+
## Setup
14+
15+
### 1. Install AWS Load Balancer Controller
16+
17+
Follow the [official installation guide](https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.14/deploy/installation/) to deploy the controller to your cluster.
18+
19+
### 2. Create a GatewayClass
20+
21+
Create a GatewayClass resource for conformance testing:
22+
23+
```yaml
24+
apiVersion: gateway.networking.k8s.io/v1
25+
kind: GatewayClass
26+
metadata:
27+
name: REPLACE_WITH_GATEWAY_CLASS_NAME
28+
spec:
29+
controllerName: ingress.k8s.aws/alb
30+
```
31+
32+
Apply it:
33+
```bash
34+
kubectl apply -f gatewayclass.yaml
35+
```
36+
37+
### 3. Configure Controller Deployment
38+
39+
Edit the controller deployment to enable Gateway API features:
40+
41+
```bash
42+
kubectl edit deployment -n kube-system aws-load-balancer-controller
43+
```
44+
45+
Ensure these arguments are present:
46+
```yaml
47+
spec:
48+
template:
49+
spec:
50+
containers:
51+
- args:
52+
- --default-target-type=ip
53+
- --feature-gates=NLBGatewayAPI=true,ALBGatewayAPI=true
54+
```
55+
56+
## Running Tests
57+
58+
### Run a Single Test Case
59+
60+
Useful for debugging specific functionality:
61+
62+
```bash
63+
go test ./conformance -run TestConformance -v -args \
64+
--gateway-class=REPLACE_WITH_GATEWAY_CLASS_NAME \
65+
--run-test=HTTPRouteExactPathMatching --allow-crds-mismatch=true --cleanup-base-resources=false \
66+
--debug=true
67+
```
68+
69+
**Arguments explained:**
70+
- `--gateway-class`: Name of the GatewayClass to use for testing
71+
- `--run-test`: Specific test to run (e.g., HTTPRouteExactPathMatching)
72+
- `--allow-crds-mismatch=true`: Allow CRD version mismatches between test suite and cluster
73+
- `--cleanup-base-resources=false`: Keep resources after test for debugging
74+
- `--debug=true`: Enable verbose debug logging
75+
76+
### Run a Conformance Profile
77+
78+
Run a complete profile (e.g., GATEWAY-HTTP) with custom configuration:
79+
80+
```bash
81+
go test -v ./conformance \
82+
--gateway-class=REPLACE_WITH_YOUR_GW_CLASS \
83+
--allow-crds-mismatch=true --cleanup-base-resources=false \
84+
--conformance-profiles=GATEWAY-HTTP \
85+
--supported-features=Gateway,HTTPRoute \
86+
--skip-tests=GatewaySecretInvalidReferenceGrant \
87+
--debug=true
88+
```
89+
90+
**Arguments explained:**
91+
- `--conformance-profiles`: Profile to test (GATEWAY-HTTP, GATEWAY-GRPC, etc.)
92+
- `--supported-features`: Comma-separated list of features your implementation supports
93+
- `--skip-tests`: Comma-separated list of tests to skip
94+
95+
96+
## Exempt Features
97+
98+
Features not yet implemented (tests will be skipped):
99+
- `GatewayStaticAddresses`
100+
- `GatewayHTTPListenerIsolation`
101+
- `HTTPRouteRequestMultipleMirrors`
102+
- `HTTPRouteRequestTimeout`
103+
- `HTTPRouteBackendTimeout`
104+
- `HTTPRouteParentRefPort`
105+
- continue updating...
106+
107+
## Troubleshooting
108+
109+
### CRD Version Mismatch
110+
111+
If you see CRD version errors, use `--allow-crds-mismatch=true` or update your Gateway API CRDs:
112+
```bash
113+
kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.0.0/standard-install.yaml
114+
```
115+
116+
### Target Group Port Empty
117+
If you see
118+
```
119+
"error":"TargetGroup port is empty. When using Instance targets, your service must be of type 'NodePort' or 'LoadBalancer'"
120+
```
121+
Make sure argument `--default-target-type=ip` is added to deployment

conformance/conformance_test.go

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
package conformance
2+
3+
import (
4+
"testing"
5+
"time"
6+
7+
"sigs.k8s.io/gateway-api/conformance"
8+
"sigs.k8s.io/gateway-api/conformance/utils/suite"
9+
)
10+
11+
func TestConformance(t *testing.T) {
12+
options := conformance.DefaultOptions(t)
13+
14+
// TODO: SkipTests, SupportedFeatures, ExemptFeatures needs to be updated after we conduct all conformance tests
15+
// Below is only an example for now
16+
// Configure skip tests, supported features and exempt features
17+
options.SkipTests = suite.ParseSkipTests("GatewaySecretInvalidReferenceGrant")
18+
options.SupportedFeatures = suite.ParseSupportedFeatures("Gateway,HTTPRoute")
19+
options.ExemptFeatures = suite.ParseSupportedFeatures("GatewayStaticAddresses,GatewayHTTPListenerIsolation")
20+
21+
// Configure timeout config
22+
options.TimeoutConfig.GatewayStatusMustHaveListeners = 10 * time.Minute // we need to wait for LB to be provisioned before updating gateway listener status
23+
options.TimeoutConfig.GatewayListenersMustHaveConditions = 10 * time.Minute
24+
options.TimeoutConfig.NamespacesMustBeReady = 10 * time.Minute
25+
options.TimeoutConfig.DefaultTestTimeout = 10 * time.Minute
26+
27+
conformance.RunConformanceWithOptions(t, options)
28+
}

go.mod

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -119,6 +119,7 @@ require (
119119
github.com/mattn/go-colorable v0.1.14 // indirect
120120
github.com/mattn/go-isatty v0.0.20 // indirect
121121
github.com/mattn/go-runewidth v0.0.9 // indirect
122+
github.com/miekg/dns v1.1.65 // indirect
122123
github.com/mitchellh/copystructure v1.2.0 // indirect
123124
github.com/mitchellh/go-wordwrap v1.0.1 // indirect
124125
github.com/mitchellh/reflectwalk v1.0.2 // indirect
@@ -160,6 +161,7 @@ require (
160161
go.yaml.in/yaml/v2 v2.4.2 // indirect
161162
go.yaml.in/yaml/v3 v3.0.3 // indirect
162163
golang.org/x/crypto v0.40.0 // indirect
164+
golang.org/x/mod v0.26.0 // indirect
163165
golang.org/x/oauth2 v0.30.0 // indirect
164166
golang.org/x/sync v0.16.0 // indirect
165167
golang.org/x/sys v0.34.0 // indirect

go.sum

Lines changed: 0 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -29,8 +29,6 @@ github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5 h1:0CwZNZbxp69SHPd
2929
github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5/go.mod h1:wHh0iHkYZB8zMSxRWpUBQtwG5a7fFgvEO+odwuTv2gs=
3030
github.com/asaskevich/govalidator v0.0.0-20230301143203-a9d515a09cc2 h1:DklsrG3dyBCFEj5IhUbnKptjxatkF07cF2ak3yi77so=
3131
github.com/asaskevich/govalidator v0.0.0-20230301143203-a9d515a09cc2/go.mod h1:WaHUgvxTVq04UNunO+XhnAqY/wQc+bxr74GqbsZ/Jqw=
32-
github.com/aws/aws-sdk-go-v2 v1.36.3 h1:mJoei2CxPutQVxaATCzDUjcZEjVRdpsiiXi2o38yqWM=
33-
github.com/aws/aws-sdk-go-v2 v1.36.3/go.mod h1:LLXuLpgzEbD766Z5ECcRmi8AzSwfZItDtmABVkRLGzg=
3432
github.com/aws/aws-sdk-go-v2 v1.39.2 h1:EJLg8IdbzgeD7xgvZ+I8M1e0fL0ptn/M47lianzth0I=
3533
github.com/aws/aws-sdk-go-v2 v1.39.2/go.mod h1:sDioUELIUO9Znk23YVmIk86/9DOpkbyyVb1i/gUNFXY=
3634
github.com/aws/aws-sdk-go-v2/config v1.27.27 h1:HdqgGt1OAP0HkEDDShEl0oSYa9ZZBSOmKpdpsDMdO90=
@@ -39,12 +37,8 @@ github.com/aws/aws-sdk-go-v2/credentials v1.17.27 h1:2raNba6gr2IfA0eqqiP2XiQ0UVO
3937
github.com/aws/aws-sdk-go-v2/credentials v1.17.27/go.mod h1:gniiwbGahQByxan6YjQUMcW4Aov6bLC3m+evgcoN4r4=
4038
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.16.11 h1:KreluoV8FZDEtI6Co2xuNk/UqI9iwMrOx/87PBNIKqw=
4139
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.16.11/go.mod h1:SeSUYBLsMYFoRvHE0Tjvn7kbxaUhl75CJi1sbfhMxkU=
42-
github.com/aws/aws-sdk-go-v2/internal/configsources v1.3.34 h1:ZK5jHhnrioRkUNOc+hOgQKlUL5JeC3S6JgLxtQ+Rm0Q=
43-
github.com/aws/aws-sdk-go-v2/internal/configsources v1.3.34/go.mod h1:p4VfIceZokChbA9FzMbRGz5OV+lekcVtHlPKEO0gSZY=
4440
github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.9 h1:se2vOWGD3dWQUtfn4wEjRQJb1HK1XsNIt825gskZ970=
4541
github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.9/go.mod h1:hijCGH2VfbZQxqCDN7bwz/4dzxV+hkyhjawAtdPWKZA=
46-
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.6.34 h1:SZwFm17ZUNNg5Np0ioo/gq8Mn6u9w19Mri8DnJ15Jf0=
47-
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.6.34/go.mod h1:dFZsC0BLo346mvKQLWmoJxT+Sjp+qcVR1tRVHQGOH9Q=
4842
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.9 h1:6RBnKZLkJM4hQ+kN6E7yWFveOTg8NLPHAkqrs4ZPlTU=
4943
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.9/go.mod h1:V9rQKRmK7AWuEsOMnHzKj8WyrIir1yUJbZxDuZLFvXI=
5044
github.com/aws/aws-sdk-go-v2/internal/ini v1.8.0 h1:hT8rVHwugYE2lEfdFE0QWVo81lF7jMrYJVDWI+f+VxU=
@@ -55,8 +49,6 @@ github.com/aws/aws-sdk-go-v2/service/appmesh v1.27.7 h1:q44a6kysAfej9zZwRnraOg9s
5549
github.com/aws/aws-sdk-go-v2/service/appmesh v1.27.7/go.mod h1:ZYSmrgAMp0rTCHH+SGsoxZo+PPbgsDqBzewTp3tSJ60=
5650
github.com/aws/aws-sdk-go-v2/service/ec2 v1.173.0 h1:ta62lid9JkIpKZtZZXSj6rP2AqY5x1qYGq53ffxqD9Q=
5751
github.com/aws/aws-sdk-go-v2/service/ec2 v1.173.0/go.mod h1:o6QDjdVKpP5EF0dp/VlvqckzuSDATr1rLdHt3A5m0YY=
58-
github.com/aws/aws-sdk-go-v2/service/elasticloadbalancingv2 v1.45.0 h1:RB7V8wT9ypjE/YJVBgKjoydTOh4IFoqceGiKxFH70mY=
59-
github.com/aws/aws-sdk-go-v2/service/elasticloadbalancingv2 v1.45.0/go.mod h1:xnCC3vFBfOKpU6PcsCKL2ktgBTZfOwTGxj6V8/X3IS4=
6052
github.com/aws/aws-sdk-go-v2/service/elasticloadbalancingv2 v1.51.0 h1:Zy1yjx+R6cR4pAwzFFJ8nWJh4ri8I44H76PDJ77tcJo=
6153
github.com/aws/aws-sdk-go-v2/service/elasticloadbalancingv2 v1.51.0/go.mod h1:RuZwE3p8IrWqK1kZhwH2TymlHLPuiI/taBMb8vrD39Q=
6254
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.11.3 h1:dT3MqvGhSoaIhRseqw2I0yH81l7wiR2vjs57O51EAm8=
@@ -79,8 +71,6 @@ github.com/aws/aws-sdk-go-v2/service/wafregional v1.23.3 h1:7dr6En0/6KRFoz8VmnYk
7971
github.com/aws/aws-sdk-go-v2/service/wafregional v1.23.3/go.mod h1:24TtlRsv4LKAE3VnRJQhpatr8cpX0yj8NSzg8/lxOCw=
8072
github.com/aws/aws-sdk-go-v2/service/wafv2 v1.51.4 h1:1khBA5uryBRJoCb4G2iR5RT06BkfPEjjDCHAiRb8P3Q=
8173
github.com/aws/aws-sdk-go-v2/service/wafv2 v1.51.4/go.mod h1:QpFImaPGKNwa+MiZ+oo6LbV1PVQBapc0CnrAMRScoxM=
82-
github.com/aws/smithy-go v1.22.2 h1:6D9hW43xKFrRx/tXXfAlIZc4JI+yQe6snnWcQyxSyLQ=
83-
github.com/aws/smithy-go v1.22.2/go.mod h1:irrKGvNn1InZwb2d7fkIRNucdfwR8R+Ts3wxYa/cJHg=
8474
github.com/aws/smithy-go v1.23.0 h1:8n6I3gXzWJB2DxBDnfxgBaSX6oe0d/t10qGz7OKqMCE=
8575
github.com/aws/smithy-go v1.23.0/go.mod h1:t1ufH5HMublsJYulve2RKmHDC15xu1f26kHCp/HgceI=
8676
github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=

0 commit comments

Comments
 (0)