Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Validate config using STDIN and /dev/fd/0 #10296

Closed
wants to merge 52 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
44d133d
Trying to validate config using STDIN and /dev/fd/0
ryanrolds Nov 11, 2024
83f05f1
Merge main into rolds/envoy_large_validation
soloio-bulldozer[bot] Nov 11, 2024
59a8257
Merge refs/heads/main into rolds/envoy_large_validation
soloio-bulldozer[bot] Nov 11, 2024
43d5512
Adding change log
ryanrolds Nov 11, 2024
3af084f
Merge branch 'rolds/envoy_large_validation' of ssh://github.com/solo-…
ryanrolds Nov 11, 2024
d9c1429
Merge refs/heads/main into rolds/envoy_large_validation
soloio-bulldozer[bot] Nov 12, 2024
5857e78
Merge refs/heads/main into rolds/envoy_large_validation
soloio-bulldozer[bot] Nov 12, 2024
b3587da
Merge refs/heads/main into rolds/envoy_large_validation
soloio-bulldozer[bot] Nov 12, 2024
ec010fa
Merge refs/heads/main into rolds/envoy_large_validation
soloio-bulldozer[bot] Nov 12, 2024
4ee4b20
Fixed for tests and new test for large configs
ryanrolds Nov 12, 2024
8493f69
Merge branch 'rolds/envoy_large_validation' of ssh://github.com/solo-…
ryanrolds Nov 12, 2024
9f50557
Minor adjustments
ryanrolds Nov 12, 2024
81d9d2c
Update changelog/v1.18.0-beta34/validate-large-configs.yaml
ryanrolds Nov 12, 2024
3306ab1
Remove uneeded env var setting
ryanrolds Nov 12, 2024
2e45b4c
Merge branch 'rolds/envoy_large_validation' of ssh://github.com/solo-…
ryanrolds Nov 12, 2024
41ba11c
Merge refs/heads/main into rolds/envoy_large_validation
soloio-bulldozer[bot] Nov 12, 2024
32c8ab1
Adjsuted how we work out the cluster context in kube2e tests
ryanrolds Nov 12, 2024
7430975
Moved kubernetes e2e test
ryanrolds Nov 12, 2024
d70b482
Merge branch 'rolds/envoy_large_validation' of ssh://github.com/solo-…
ryanrolds Nov 12, 2024
aee9427
Merge refs/heads/main into rolds/envoy_large_validation
soloio-bulldozer[bot] Nov 12, 2024
127d879
Adding changelog file to new location
Nov 13, 2024
6044d43
Deleting changelog file from old location
Nov 13, 2024
be5bb4f
Turned of strict mode for the full envoy tests
ryanrolds Nov 14, 2024
6792374
Merge branch 'rolds/envoy_large_validation' of ssh://github.com/solo-…
ryanrolds Nov 14, 2024
22fabd1
Added make target to help keep versions uniform
ryanrolds Nov 14, 2024
34a71c1
Removed an extra VS from large config test
ryanrolds Nov 14, 2024
b772cb1
Merge branch 'main' into rolds/envoy_large_validation
ryanrolds Nov 14, 2024
3fe5d33
Removed a superceeded test and embed line
ryanrolds Nov 14, 2024
0446380
Moved changelog
ryanrolds Nov 14, 2024
eba6c0b
Readded changelog
ryanrolds Nov 14, 2024
69b546b
Adding changelog file to new location
Nov 14, 2024
19c5c4b
Deleting changelog file from old location
Nov 14, 2024
45ea63b
Increased validating webhook timeout in tests and added note to full …
ryanrolds Nov 14, 2024
8cfc522
Merge branch 'main' into rolds/envoy_large_validation
ryanrolds Nov 14, 2024
ebee9c3
Removed logging line that was writing whole request body for validati…
ryanrolds Nov 14, 2024
c22c171
Adding another VS to the test data
ryanrolds Nov 14, 2024
32a1121
Merge branch 'main' into rolds/envoy_large_validation
ryanrolds Nov 15, 2024
a82c97c
Partial work
ryanrolds Nov 15, 2024
ec7e518
Partial chnages
ryanrolds Nov 18, 2024
c6b0908
Merge branch 'main' into rolds/envoy_large_validation
ryanrolds Nov 18, 2024
156445d
Undoing a change
ryanrolds Nov 18, 2024
f663644
Undoing some changes
ryanrolds Nov 18, 2024
d536408
Merge branch 'main' into rolds/envoy_large_validation
ryanrolds Nov 18, 2024
ae17147
Merge branch 'main' into rolds/envoy_large_validation
nfuden Nov 25, 2024
aae3e5c
changelog: move and make more user facing
nfuden Nov 25, 2024
0f19869
Merge branch 'main' into rolds/envoy_large_validation
nfuden Nov 25, 2024
f8bf01f
Adding changelog file to new location
Nov 25, 2024
e878603
Deleting changelog file from old location
Nov 25, 2024
b2911bf
Merge branch 'main' into rolds/envoy_large_validation
nfuden Nov 26, 2024
f6fd30b
Merge branch 'main' into rolds/envoy_large_validation
nfuden Nov 27, 2024
28590be
Adding changelog file to new location
Dec 2, 2024
b5ac234
Deleting changelog file from old location
Dec 2, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -287,10 +287,9 @@ run-hashicorp-e2e-tests: GINKGO_FLAGS += --label-filter="end-to-end && !performa
run-hashicorp-e2e-tests: test

.PHONY: run-kube-e2e-tests
run-kube-e2e-tests: TEST_PKG = ./test/kube2e/$(KUBE2E_TESTS) ## Run the Kubernetes E2E Tests in the {KUBE2E_TESTS} package
run-kube-e2e-tests: TEST_PKG = ./test/kube2e/$(KUBE2E_TESTS) ## Run the legacy Kubernetes E2E Tests in the {KUBE2E_TESTS} package
run-kube-e2e-tests: test


#----------------------------------------------------------------------------------
# Go Tests
#----------------------------------------------------------------------------------
Expand Down Expand Up @@ -1080,6 +1079,9 @@ endif # distroless images
CLUSTER_NAME ?= kind
INSTALL_NAMESPACE ?= gloo-system

kind-setup:
VERSION=${VERSION} CLUSTER_NAME=${CLUSTER_NAME} ./ci/kind/setup-kind.sh

kind-load-%-distroless:
kind load docker-image $(IMAGE_REGISTRY)/$*:$(VERSION)-distroless --name $(CLUSTER_NAME)

Expand Down
9 changes: 9 additions & 0 deletions changelog/v1.18.0-rc4/validate-large-configs.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
changelog:
- type: FIX
issueLink: https://github.com/solo-io/solo-projects/issues/7089
resolvesIssue: false
description: >-
Fix the validation of large configurations when using envoy validation.
This was rarely seen in practice but occurred more often with the new fullEnvoyConfig validation.
Previously if the configuration grew too large translation would be blocked.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

14 changes: 12 additions & 2 deletions projects/envoyinit/pkg/runner/run.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import (
"log"
"os"
"syscall"
"time"

"github.com/rotisserie/eris"
"github.com/solo-io/gloo/pkg/utils/cmdutils"
Expand All @@ -30,8 +31,17 @@ const (
func RunEnvoyValidate(ctx context.Context, envoyExecutable, bootstrapConfig string) error {
logger := contextutils.LoggerFrom(ctx)

validateCmd := cmdutils.Command(ctx, envoyExecutable, "--mode", "validate", "--config-yaml", bootstrapConfig, "-l", "critical", "--log-format", "%v")
if err := validateCmd.Run(); err != nil {
logger.Debugf("starting full envoy validation with size %d", len(bootstrapConfig))

validateCmd := cmdutils.Command(ctx, envoyExecutable, "--mode", "validate", "--config-path", "/dev/fd/0",
"-l", "critical", "--log-format", "%v")
validateCmd = validateCmd.WithStdin(bytes.NewBufferString(bootstrapConfig))

start := time.Now()
err := validateCmd.Run()
logger.Debugf("full envoy validation of %d size completed in %s", len(bootstrapConfig), time.Since(start))

if err != nil {
if os.IsNotExist(err) {
// log a warning and return nil; will allow users to continue to run Gloo locally without
// relying on the Gloo container with Envoy already published to the expected directory
Expand Down
4 changes: 4 additions & 0 deletions projects/gloo/api/v1/settings.proto
Original file line number Diff line number Diff line change
Expand Up @@ -920,6 +920,10 @@ message GatewayOptions {
//
// This feature is disabled by default and is not recommended for production deployments unless
// the performance implications are well understood and acceptable.
//
// Large configurations can take more than 10 seconds to validate, causing the validating webhook to timeout.
// When enabling this feature, consider increasing the timeout for the validating webhook
// (`.Values.gateway.validation.webhook.timeoutSeconds`).
google.protobuf.BoolValue full_envoy_validation = 14;
}

Expand Down
4 changes: 4 additions & 0 deletions projects/gloo/pkg/api/v1/settings.pb.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

7 changes: 6 additions & 1 deletion test/kube2e/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,10 @@
> This directory houses legacy tests. All new tests should instead be added to the `test/kubernetes/e2e` directory.

# Kubernetes End-to-End tests

> These are our legacy Kubernetes E2E tests. We are migrating them to `../kubernetes/e2e`. Create new E2E tests there
> using the new framework.

See the [developer kube-e2e testing guide](/devel/testing/kube-e2e-tests.md) for more information about the philosophy of these tests.

*Note: All commands should be run from the root directory of the Gloo repository*
Expand Down Expand Up @@ -68,7 +72,7 @@ To run the regression tests, your kubeconfig file must point to a running Kubern

Use the same command that CI relies on:
```bash
KUBE2E_TESTS=<test-to-run> make run-kube-e2e-tests
CLUSTER_NAME=solo-test-cluster KUBE2E_TESTS=<test-to-run> make run-kube-e2e-tests
```

#### Test Environment Variables
Expand All @@ -81,6 +85,7 @@ The below table contains the environment variables that can be used to configure
| WAIT_ON_FAIL | 0 | Set to 1 to prevent Ginkgo from cleaning up the Gloo Edge installation in case of failure. Useful to exec into inspect resources created by the test. A command to resume the test run (and thus clean up resources) will be logged to the output. |
| TEAR_DOWN | false | Set to true to uninstall Gloo after the test suite completes |
| RELEASED_VERSION | '' | Used by nightlies to tests a specific released version. 'LATEST' will find the latest release |
| CLUSTER_NAME | kind | Used to control which Kind cluster to run the tests inside |

#### Common Test Errors
`getting Helm chart version: expected a single entry with name [gloo], found: 5`\
Expand Down
4 changes: 3 additions & 1 deletion test/kube2e/gateway/gateway_suite_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ import (
"github.com/solo-io/gloo/test/helpers"
"github.com/solo-io/gloo/test/kube2e"
"github.com/solo-io/gloo/test/kube2e/helper"
testruntime "github.com/solo-io/gloo/test/kubernetes/testutils/runtime"
skhelpers "github.com/solo-io/solo-kit/test/helpers"

. "github.com/onsi/ginkgo/v2"
Expand Down Expand Up @@ -73,7 +74,8 @@ func StartTestHelper() {
}

// We rely on the "new" kubernetes/e2e setup code, since it incorporates controller-runtime logging setup
clusterContext := cluster.MustKindContext("kind")
runtimeContext := testruntime.NewContext()
clusterContext := cluster.MustKindContext(runtimeContext.ClusterName)

resourceClientset, err = kube2e.NewKubeResourceClientSet(ctx, clusterContext.RestConfig)
Expect(err).NotTo(HaveOccurred(), "can create kube resource client set")
Expand Down
4 changes: 3 additions & 1 deletion test/kube2e/gloo/gloo_suite_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ import (
"github.com/solo-io/gloo/test/helpers"
"github.com/solo-io/gloo/test/kube2e"
"github.com/solo-io/gloo/test/kube2e/helper"
testruntime "github.com/solo-io/gloo/test/kubernetes/testutils/runtime"
glootestutils "github.com/solo-io/gloo/test/testutils"
"github.com/solo-io/go-utils/testutils"

Expand Down Expand Up @@ -73,7 +74,8 @@ var _ = BeforeSuite(func() {
}

// We rely on the "new" kubernetes/e2e setup code, since it incorporates controller-runtime logging setup
clusterContext := cluster.MustKindContext("kind")
runtimeContext := testruntime.NewContext()
clusterContext := cluster.MustKindContext(runtimeContext.ClusterName)

resourceClientset, err = kube2e.NewKubeResourceClientSet(ctx, clusterContext.RestConfig)
Expect(err).NotTo(HaveOccurred(), "can create kube resource client set")
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -77,3 +77,16 @@ func (s *testingSuite) TestRejectInvalidTransformation() {
s.Assert().Contains(output, "Failed to parse response template: Failed to parse "+
"header template ':status': [inja.exception.parser_error] (at 1:92) expected statement close, got '%'")
}

// TestLargeConfiguration checks webhook accepts large configuration when fullEnvoyValidation=true
func (s *testingSuite) TestLargeConfiguration() {
s.T().Cleanup(func() {
err := s.testInstallation.Actions.Kubectl().DeleteFileSafe(s.ctx, validation.LargeConfiguration, "-n",
s.testInstallation.Metadata.InstallNamespace)
s.Assertions.NoError(err, "can delete large configuration")
})

err := s.testInstallation.Actions.Kubectl().ApplyFile(s.ctx, validation.LargeConfiguration, "-n",
s.testInstallation.Metadata.InstallNamespace)
s.Assert().NoError(err)
}
Loading