Github action fails for lage PRs

[freewill777]

Hi,

for PRs containing 50+ files github action fails due to rate-limiting error received from openAI API., can someone make some improvements in this direction?

Thanks!

[villesau]

OpenAI seems to return following headers:

X-Ratelimit-Limit-Requests: 200
X-Ratelimit-Limit-Tokens: 40000
X-Ratelimit-Remaining-Requests: 199
X-Ratelimit-Remaining-Tokens: 39742
X-Ratelimit-Reset-Requests: 300ms
X-Ratelimit-Reset-Tokens: 387ms

If anyone has good solutions in mind for rate limiting, I'm open for suggestions (and pull requests)! :)

[gad2103]

@villesau you could consider using the chat model and re-prompting with chunks of files until that whole PR has been consumed by OpenAI. If i'm reading the issue correctly, the rate limit is on the single request?

[villesau]

@gad2103 can you elaborate a bit? Maybe a rudimentary example payload? Now it reviews chunks one by one which might cause the action to hit rate limits if the pr has many chunks. I'm very open for any ideas to improve the action<->OpenAI communication as it definitely has it's problems, such as not understanding the whole context of the pull request and so on.

[ajuarez-btf]

Hi everybody,

I'm getting a rate limit response but in my case it's from Github API. This is because the Action is sending a lot of comments in a single request. I'm gonna try to send the comments split into different requests.