Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use --exodus-commit=phase1 in pulp hook [RHELDST-20490] #28

Merged
merged 1 commit into from
Oct 19, 2023

Conversation

rohanpm
Copy link
Member

@rohanpm rohanpm commented Oct 2, 2023

When this hook is active, the control flow is:

  1. pubtools-exodus: create publish
  2. rhsm-pulp: add content to publish
  3. pubtools-exodus: commit publish

Problem: step 2 might succeed from Pulp's point of view, but then fail at step 3 leaving the content not actually published to the CDN storage. As Pulp believes the publishes were successful, it would then skip publish of certain files at the next attempt, leading to missing content.

The newly introduced concept of a phase1 commit should fix this by allowing step 2 to request that all "phase 1 content" (e.g. RPMs, but not repodata) are flushed to the CDN storage before proceeding. Go ahead and start using that via the new related exodus-rsync argument.

While it would appear to make sense to do this at every publish, an env var is added as an escape hatch to go back to the old behavior, just in case something doesn't work as expected or upgrades are performed in the wrong order.

When this hook is active, the control flow is:

1. pubtools-exodus: create publish
2. rhsm-pulp: add content to publish
3. pubtools-exodus: commit publish

Problem: step 2 might succeed from Pulp's point of view, but then fail
at step 3 leaving the content not actually published to the CDN storage.
As Pulp believes the publishes were successful, it would then skip
publish of certain files at the next attempt, leading to missing
content.

The newly introduced concept of a phase1 commit should fix this by
allowing step 2 to request that all "phase 1 content" (e.g. RPMs, but
not repodata) are flushed to the CDN storage before proceeding. Go ahead
and start using that via the new related exodus-rsync argument.

While it would appear to make sense to do this at *every* publish, an
env var is added as an escape hatch to go back to the old behavior, just
in case something doesn't work as expected or upgrades are performed in
the wrong order.
@rohanpm
Copy link
Member Author

rohanpm commented Oct 5, 2023

Holding back on submit of this until the latest exodus-rsync is deployed everywhere.

@rohanpm rohanpm merged commit de5506e into release-engineering:main Oct 19, 2023
2 checks passed
@rohanpm rohanpm deleted the commit-phase1 branch October 19, 2023 23:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants