Issue with Finance API Quota and TooManyRequests Error When Fetching Financial Events

[ssaqibshirazi]

We are encountering issues while using the Amazon Selling Partner Finance API for retrieving financial events. Despite adhering to the provided rate limits, the API frequently returns a TooManyRequests (429) error. The issue occurs specifically with the endpoint:

https://sellingpartnerapi-na.amazon.com/finances/v0/financialEvents?MaxResultsPerPage=100&PostedAfter=2024-12-12&PostedBefore=2024-12-13

Steps to Reproduce:

1. Use the Finance API to request financial events data for the specified time period.
2. Set MaxResultsPerPage=100 in the query parameters.
3. Make sequential calls, including passing NextToken for paginated results.

Observed Behavior:

1. The API returns a TooManyRequests (429) error even with reasonable call intervals.
2. There is no clear guidance or consistent behavior for handling this rate limit.
3. The Retry-After header is either missing or not sufficiently clear.

Expected Behavior:
The API should allow requests within the rate limit guidelines.
Clear documentation or headers (like Retry-After) should be provided for handling rate limits effectively.

Additional Context:
. API requests include MaxResultsPerPage set to 100.
. Retrying the request after a delay doesn't consistently resolve the issue.
. This problem started occurring as of December 10, 2024.

Please provide assistance or clarification on:

1. Current rate limits for the Finance API.
2. Recommendations for efficiently handling paginated results to avoid TooManyRequests errors.
3. Guidance on interpreting and using the Retry-After header, if applicable.
4. Whether additional quota is required for high-volume use cases.

Looking forward to your response. Thank you!

[MeenaAmz]

@ssaqibshirazi I believe you are referring to listFinancialEvents operation to get the financial events. The rate limit for this operation (and other operations) is defined in the API reference. SP-API does not provide a Retry-After header, but x-amzn-RateLimit-Limit header is provided in the response that also provides the operation's rate limits per account-application pair.

To better understand how rate limits work and strategies on how to handle rate limiting, please review these resources,

1. Error Handling Best Practices
2. Usage plans and rate limits
3. Optimize rate limits for application workloads

Specifically for finances API, you can also review the different tutorials provided in the use case guide and see if you can use financial event groups to potentially reduce the volume of results and pages returned.

Thanks
Meena
Selling Partner Developer Services

[chris1out]

We are also seeing this. It started showing up for us December 3rd. We’ve made no code changes and all of a sudden Amazon is throttling at an extreme rate. The health dashboard was showing red for several days, but now shows green. It is absolutely still broken.

[kwedaras]

Can confirm I experience the same issue. We did not make any changes to the code. Happening for a couple weeks already.
#4461 (comment)

[chris1out]

We finally got our issue resolved, but I'm not happy with the way this works now. After you make your first request and you get a next token, when you go to make the request with that token, if you get "Invalid Input", try the request again with a "MaxResultsPerPage" of 50. If that fails, try again with 25, if that fails try again with 1. If any of those succeed, go back to not passing the parameter so as you don't spend all day pulling 1 result at a time. This appears to be Amazon making an undocumented change on how large a result can be (and of course not telling us that's the issue in the error). Note this does not fix the "too many requests", but does seem to solve the "invalid input".

[saeedasma]

We are also seeing the same issue with the ListFinancialEvents call since the last 6 days now. Did anyone find a solution to this?

Thanks,
Asma

[ElyeM]

We are facing the same issue, it started around Black Friday, any solution? we are constantly getting hit by 429s.

@MeenaAmz we didn't change anything on our end, we are following all the rate limit rules, something changed on amazons side and it started rate limiting way too strict, we cant get the full data for a settlement.

[MeenaAmz]

Thanks for bringing this up. I'm following up with the backend team on this and will get back once I get an update.

Thanks
Meena
Selling Partner Developer Services

[ssaqibshirazi]

I have enhanced my code with the following improvements:

1. Rate Limiter: Implemented a mechanism to effectively manage API rate limits.
2. Retry Logic: Added exponential backoff to gracefully handle 429 Too Many Requests responses.
3. Structured Logging: Improved observability by logging critical events such as successes, rate limit handling, and errors.

These changes helped me resolve the 429 Too Many Requests issue.

I hope this approach helps you resolve similar issues on your end.

Please let me know if it works for you.

// A rate limiter implementation to handle API rate limits.
// Retry logic with exponential backoff for 429 (Too Many Requests) responses.
// Enhanced structured logging for better observability and debugging.

public class RateLimiter
{
    private readonly SemaphoreSlim _semaphore;
    private readonly int _rateLimit;
    private readonly TimeSpan _rateLimitInterval;

    public RateLimiter(int requestsPerSecond)
    {
        _rateLimit = requestsPerSecond;
        _rateLimitInterval = TimeSpan.FromSeconds(1.0 / _rateLimit);
        _semaphore = new SemaphoreSlim(_rateLimit);
    }

    public async Task AcquireAsync()
    {
        await _semaphore.WaitAsync();
        _ = Task.Delay(_rateLimitInterval).ContinueWith(_ => _semaphore.Release());
    }

    public static async Task HandleRateLimitExceededAsync(int retryCount)
    {
        int backoffTime = (int)Math.Pow(2, retryCount) * 1000; // Exponential backoff
        await Task.Delay(backoffTime);
    }
}

// Main API logic with retry and rate-limiting
public async Task<AMZ_SPAPI_Res_ListFinancialEvents> GetFinancialEventsAsync(
    string region, 
    string companyID, 
    string startDate, 
    string endDate, 
    string nextToken, 
    int maxRetries = 5)
{
    var rateLimiter = new RateLimiter(2); // Allow 2 requests per second
    AMZ_SPAPI_Res_ListFinancialEvents financialEventsResponse = new AMZ_SPAPI_Res_ListFinancialEvents();

    try
    {
        await GetAuthenticationAsync(region, companyID);

        var client = new RestClient();
        RestRequest request;

        if (string.IsNullOrEmpty(nextToken))
        {
            request = new RestRequest(
                $"{ENDPOINT}/finances/v0/financialEvents?MaxResultsPerPage=100&PostedAfter={Convert.ToDateTime(startDate):yyyy-MM-dd}&PostedBefore={Convert.ToDateTime(endDate):yyyy-MM-dd}",
                Method.Get
            );
        }
        else
        {
            request = new RestRequest(
                $"{ENDPOINT}/finances/v0/financialEvents?MaxResultsPerPage=100&NextToken={WebUtility.UrlEncode(nextToken)}",
                Method.Get
            );
        }

        request.AddHeader("Accept", "application/json");
        request.AddHeader("x-amz-access-token", GetTokenFromCache(companyID));
        request.AddHeader("X-Amz-Security-Token", GetSessionTokenFromCache(companyID));

        bool isSuccessful = false;
        int retryCount = 0;

        while (!isSuccessful && retryCount <= maxRetries)
        {
            await rateLimiter.AcquireAsync();

            var response = await client.ExecuteAsync(request);

            _logger.LogWarning($"Response Successful: {response.IsSuccessful}");

            if (response.IsSuccessful)
            {
                financialEventsResponse = JsonConvert.DeserializeObject<AMZ_SPAPI_Res_ListFinancialEvents>(response.Content);
                isSuccessful = true;
            }
            else if (response.StatusCode == HttpStatusCode.TooManyRequests)
            {
                _logger.LogWarning($"Rate limit exceeded. Retrying attempt {retryCount + 1}...");
                retryCount++;
                await RateLimiter.HandleRateLimitExceededAsync(retryCount);
            }
            else
            {
                _logger.LogError($"Error during API call: {response.Content}");
                break;
            }

            if (!isSuccessful && retryCount >= maxRetries)
            {
                _logger.LogError("Exceeded maximum retry attempts.");
            }
        }
    }
    catch (Exception ex)
    {
        _logger.LogError($"Exception: {ex.Message}");
    }

    return financialEventsResponse;
}

[MeenaAmz]

Hello Everyone,

Thanks for your patience. The throttling issue has been resolved and you should not see increased level of throttling anymore. Please validate and open a support case if you still see issues.

Thanks
Meena

[ElyeM]

Thanks @MeenaAmz I can confirm that it is much better now.