Issue with revalidation

[alacroix]

Hey guys, happy new year!

I have a recent issue with ISR. For some reasons, I have some pages in the revalidation queue that are 404 at the time (scrapers or else that try random pages maybe). For what I understand from the code here, if the response is not 200, it will add the record to the failed records object that will be retried later with the partial failure feature implemented in the PR #410.
As a result, I have a queue that piles up as I go along and legit revalidations are stuck behind duplicates of Not Found pages.

Do you have any advice / ideas on how to fix it? And why we need a 200 for the response of a revalidation (can't we just ignore the response status?)

Here is my configuration (mostly default) for my SQS Queue and trigger for the lambda (based on the reference implementation) :

Thanks!

[conico974]

I have some pages in the revalidation queue that are 404 at the time (scrapers or else that try random pages maybe)

This should not cause these page to be in the revalidation queue, the only thing that i could think of that could cause a 404 in the queue would be the 404 route itself being revalidated or a route that was previously generated as a 200 and is now a 404.

And why we need a 200 for the response of a revalidation (can't we just ignore the response status?)

This one is tricky, we cannot just ignore other status code, a 500 or a 404 both could be a temporary issue and could cause the issue mentionned in the PR if not handled later.

As a workaround for now you could have a custom converter and not return failed records (the default converter is the sqs-revalidate), but first things is to understand why these records fail (you should see the failed one in cloudwatch logs)