Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Race between TestAdminClearEmail and TestIssuanceCertStorageFailed #7838

Open
pgporada opened this issue Nov 21, 2024 · 0 comments · May be fixed by #7840
Open

Race between TestAdminClearEmail and TestIssuanceCertStorageFailed #7838

pgporada opened this issue Nov 21, 2024 · 0 comments · May be fixed by #7840
Assignees
@jprenken
Labels
area/admin-revoker starter Ideal issues for folks getting familiar with Boulder

Comments

@pgporada
Copy link
Member

pgporada commented Nov 21, 2024 •
edited
Loading

Randomly the following error will be thrown in CI.

--- FAIL: TestAdminClearEmail (0.04s)
    admin_test.go:43: clearing email via admin tool (./bin/admin -config test/config/admin.json -dry-run=false update-email -address [email protected] -clear): 01:37:56.655147 6 admin 8MWbnQQ Debug server listening on :8014
        01:37:56.655258 6 admin p6yD7gM Versions: admin=(Unspecified Unspecified) Golang=(go1.23.1) BuildHost=(Unspecified)
        01:37:56.655406 3 admin 9siVpw4 [AUDIT] unable to boot debug server on :8014: listen tcp :8014: bind: address already in use
        : exit status 1
FAIL

TestAdminClearEmail and TestIssuanceCertStorageFailed both exist in package integration and are run in parallel. I think golang runs all t.Parallel() at the package level up to the maximum number of tests allowed to run simultaneously.

Each test is using the same admin.json which sets its debug port to 8014/tcp.

user at work in ~/work/boulder/test/integration on main*
$ grep -ri admin
...
admin_test.go:	config := fmt.Sprintf("%s/%s", os.Getenv("BOULDER_CONFIG_DIR"), "admin.json")
...
cert_storage_failed_test.go:	config := fmt.Sprintf("%s/%s", os.Getenv("BOULDER_CONFIG_DIR"), "admin.json")
...

There's a race between these tests starting up the process and attempting to claim the port.

We could potentially pass --debug-addr ${IP}:${RANDOM_PORT} to each startup to bypass this race.
@pgporada pgporada added area/admin-revoker starter Ideal issues for folks getting familiar with Boulder labels Nov 21, 2024
jprenken added a commit that referenced this issue Nov 22, 2024
@jprenken
admin: Add debug-addr CLI flag
1839f92 
If provided, the CLI flag will override the value in the config file.

Use the CLI flag to de-conflict parallel integration tests.

Fixes #7838
@jprenken jprenken self-assigned this Nov 22, 2024
@jprenken jprenken linked a pull request Nov 22, 2024 that will close this issue
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jprenken jprenken

Labels
area/admin-revoker starter Ideal issues for folks getting familiar with Boulder
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

admin: Add debug-addr CLI flag letsencrypt/boulder
2 participants
@pgporada @jprenken