Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(explicitRollback): Add configurable timeout for serverGroup lookup from Clouddriver API #4686

Merged
merged 7 commits into from
Apr 2, 2024

Conversation

christosarvanitis
Copy link
Member

When a user performs an Ad-Hoc Rollback through Deck, Orca is expecting to receive a response from Clouddriver about the serverGroup details in a hardcoded 5 seconds timeout.
Although in most cases this is enough, in higher scale environments with thousands of AWS/ECS accounts and especially in applications that have a large amount of serverGroups the 5 sec timeout is too strict resulting in error during the serverGroup lookup.
TimeoutFailure

With this change we can configure the timeout (in seconds) through Orca config - while defaulting to the previous timeout of 5secs.

@christosarvanitis christosarvanitis changed the title fix(explicitRollback): Add configurable timeout for serverGroup rollback from Clouddriver API fix(explicitRollback): Add configurable timeout for serverGroup lookup from Clouddriver API Mar 29, 2024
.get() // not sure what would cause the Optional to not be present but we would catch and log it
} catch(Exception e) {
log.error('Could not generate resize stage because there was an error looking up {}', serverGroupName, e)
throw new SpinnakerException("failed to look up ${serverGroupName}", e)
throw new SpinnakerException("Failed Clouddriver look up for ${serverGroupName} in ${rollbackTimeout} sec. Consider increasing rollback.explicitRollback.timeout in orca profile.", e)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good log message here.

Copy link
Contributor

@ovidiupopa07 ovidiupopa07 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🚀

@jasonmcintosh jasonmcintosh added the ready to merge Approved and ready for merge label Apr 2, 2024
@mergify mergify bot added the auto merged Merged automatically by a bot label Apr 2, 2024
@mergify mergify bot merged commit 2b8af02 into spinnaker:master Apr 2, 2024
4 checks passed
@jasonmcintosh
Copy link
Member

jasonmcintosh commented Apr 2, 2024

@Mergifyio backport release-1.32.x release-1.33.x release 1-3.1.x

Copy link
Contributor

mergify bot commented Apr 2, 2024

backport release-1.32.x release-1.33.x release 1-3.1.x

❌ No backport have been created

GitHub error: Branch not found

  • Backport to branch 1-3.1.x failed

GitHub error: Branch not found

mergify bot pushed a commit that referenced this pull request Apr 2, 2024
…p from Clouddriver API (#4686)

* fix(explicitRollback): Add configurable timeout for serverGroup rollback from Clouddriver API

* fix(explicitRollback): Fix tests

* fix(explicitRollback): Add configurable timeout for serverGroup lookup from Clouddriver API

* fix(explicitRollback): Add configurable timeout for serverGroup lookup from Clouddriver API

* fix(explicitRollback): Add configurable timeout for serverGroup lookup from Clouddriver API

---------

Co-authored-by: Jason <[email protected]>
(cherry picked from commit 2b8af02)
mergify bot pushed a commit that referenced this pull request Apr 2, 2024
…p from Clouddriver API (#4686)

* fix(explicitRollback): Add configurable timeout for serverGroup rollback from Clouddriver API

* fix(explicitRollback): Fix tests

* fix(explicitRollback): Add configurable timeout for serverGroup lookup from Clouddriver API

* fix(explicitRollback): Add configurable timeout for serverGroup lookup from Clouddriver API

* fix(explicitRollback): Add configurable timeout for serverGroup lookup from Clouddriver API

---------

Co-authored-by: Jason <[email protected]>
(cherry picked from commit 2b8af02)
@jasonmcintosh
Copy link
Member

jasonmcintosh commented Apr 2, 2024

@Mergifyio backport release-1-31.x

Copy link
Contributor

mergify bot commented Apr 2, 2024

backport release 1-31.x

❌ No backport have been created

  • Backport to branch release failed

GitHub error: Branch not found

  • Backport to branch 1-31.x failed

GitHub error: Branch not found

Copy link
Contributor

mergify bot commented Apr 2, 2024

backport release-1-31.x

❌ No backport have been created

  • Backport to branch release-1-31.x failed

GitHub error: Branch not found

@jasonmcintosh
Copy link
Member

https://github.com/Mergifyio backport release-1.31.x

Copy link
Contributor

mergify bot commented Apr 2, 2024

backport release-1-31.x

❌ No backport have been created

  • Backport to branch release-1-31.x failed

GitHub error: Branch not found

Copy link
Contributor

mergify bot commented Apr 2, 2024

backport release-1.31.x

✅ Backports have been created

mergify bot pushed a commit that referenced this pull request Apr 2, 2024
…p from Clouddriver API (#4686)

* fix(explicitRollback): Add configurable timeout for serverGroup rollback from Clouddriver API

* fix(explicitRollback): Fix tests

* fix(explicitRollback): Add configurable timeout for serverGroup lookup from Clouddriver API

* fix(explicitRollback): Add configurable timeout for serverGroup lookup from Clouddriver API

* fix(explicitRollback): Add configurable timeout for serverGroup lookup from Clouddriver API

---------

Co-authored-by: Jason <[email protected]>
(cherry picked from commit 2b8af02)
Copy link
Contributor

mergify bot commented Apr 2, 2024

backport release-1-31.x

❌ No backport have been created

  • Backport to branch release-1-31.x failed

GitHub error: Branch not found

mergify bot added a commit that referenced this pull request Apr 2, 2024
…p from Clouddriver API (#4686) (#4690)

* fix(explicitRollback): Add configurable timeout for serverGroup rollback from Clouddriver API

* fix(explicitRollback): Fix tests

* fix(explicitRollback): Add configurable timeout for serverGroup lookup from Clouddriver API

* fix(explicitRollback): Add configurable timeout for serverGroup lookup from Clouddriver API

* fix(explicitRollback): Add configurable timeout for serverGroup lookup from Clouddriver API

---------

Co-authored-by: Jason <[email protected]>
(cherry picked from commit 2b8af02)

Co-authored-by: Christos Arvanitis <[email protected]>
mergify bot added a commit that referenced this pull request Apr 2, 2024
…p from Clouddriver API (#4686) (#4692)

* fix(explicitRollback): Add configurable timeout for serverGroup rollback from Clouddriver API

* fix(explicitRollback): Fix tests

* fix(explicitRollback): Add configurable timeout for serverGroup lookup from Clouddriver API

* fix(explicitRollback): Add configurable timeout for serverGroup lookup from Clouddriver API

* fix(explicitRollback): Add configurable timeout for serverGroup lookup from Clouddriver API

---------

Co-authored-by: Jason <[email protected]>
(cherry picked from commit 2b8af02)

Co-authored-by: Christos Arvanitis <[email protected]>
mergify bot added a commit that referenced this pull request Apr 2, 2024
…p from Clouddriver API (#4686) (#4691)

* fix(explicitRollback): Add configurable timeout for serverGroup rollback from Clouddriver API

* fix(explicitRollback): Fix tests

* fix(explicitRollback): Add configurable timeout for serverGroup lookup from Clouddriver API

* fix(explicitRollback): Add configurable timeout for serverGroup lookup from Clouddriver API

* fix(explicitRollback): Add configurable timeout for serverGroup lookup from Clouddriver API

---------

Co-authored-by: Jason <[email protected]>
(cherry picked from commit 2b8af02)

Co-authored-by: Christos Arvanitis <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
auto merged Merged automatically by a bot ready to merge Approved and ready for merge target-release/1.34
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants