[Terrafrom] Add rate limiting #1084
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mk-armah
requested changes
Oct 22, 2024
There was a problem hiding this comment.
Great work 👏🏽, left few comments. Also I was wondering if we could use aiolimiter to control requests easily since terraform rate limit is time bound
| # https://developer.hashicorp.com/terraform/cloud-docs/api-docs#rate-limiting | ||
| RATE_LIMIT_PER_SECOND = 30 | ||
| RATE_LIMIT_BUFFER = 5 # Buffer to avoid hitting the exact limit | ||
| MAX_CONCURRENT_REQUESTS = 10 |
There was a problem hiding this comment.
10 is how many workspaces we will process concurrently when fetching runs (or enriching these workspaces).
| json=json_data, | ||
| ) | ||
| response.raise_for_status() | ||
| async with self.semaphore: |
There was a problem hiding this comment.
Is there a concurrency constraint on the Terraform Cloud API as well ?
| workspaces[i : i + CHUNK_SIZE] | ||
| for i in range(0, len(workspaces), CHUNK_SIZE) | ||
| ]: | ||
| chunk_results = await gather(*[fetch_runs_for_workspace(w) for w in chunk]) |
There was a problem hiding this comment.
Is there a specific reason for replacing as_completed with gather ? waiting for all tasks within a chunk to complete before moving on to the next chunk appears to impede performance in this context. Please correct me.
Comment on lines
+43
to
+66
|
|
||
| self.rate_limit = RATE_LIMIT_PER_SECOND | ||
| self.rate_limit_remaining = RATE_LIMIT_PER_SECOND | ||
| self.rate_limit_reset: float = 0.0 | ||
| self.last_request_time = time.time() | ||
| self.request_times: list[float] = [] | ||
| self.semaphore = asyncio.Semaphore(MAX_CONCURRENT_REQUESTS) | ||
| self.rate_limit_lock = asyncio.Lock() | ||
|
|
||
| async def wait_for_rate_limit(self) -> None: | ||
| async with self.rate_limit_lock: | ||
| current_time = time.time() | ||
| self.request_times = [t for t in self.request_times if current_time - t < 1] | ||
|
|
||
| if len(self.request_times) >= RATE_LIMIT_PER_SECOND: | ||
| wait_time = 1 - (current_time - self.request_times[0]) | ||
| if wait_time > 0: | ||
| logger.info( | ||
| f"Rate limit reached, waiting for {wait_time:.2f} seconds" | ||
| ) | ||
| await asyncio.sleep(wait_time) | ||
| self.request_times = self.request_times[1:] | ||
|
|
||
| self.request_times.append(current_time) |
|
close for fresh impl in #1110 with aiolimiter |
phalbert
deleted the
PORT-10857-overcome-terraform-clouds-api-rate-limits
branch
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
What - This PR implements rate limiting logic in the Terraform Cloud integration to address frequent
429 Too Many Requestswarnings in the logs.Why - We were experiencing excessive
429 Too Many Requestswarnings, indicating that our integration was exceeding Terraform Cloud's API rate limits. This was cluttering the logs and potentially impacting performance.How - Implemented a more robust rate limiting mechanism in the
TerraformClientclass:wait_for_rate_limitmethod to enforce rate limits.send_api_requestmethod to use the new rate limiting logic.Type of change
All tests should be run against the port production environment(using a testing org).
Core testing checklist
Integration testing checklist
examplesfolder in the integration directory.Preflight checklist
Screenshots
[Include screenshots of logs showing reduced 429 warnings, if available]
API Documentation