You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
// has a retry after? 503 (Service Unavailable) or 429 (Too Many Requests)
// but also a permanent redir (301)
String retry_after = mergedMD.getFirstValue("Retry-After");
if (retry_after != null){
// in seconds?
if (retry_after.matches("[0-9]+")) {
int rety_after_secs = Integer.parseInt(retry_after);
}
else {
// expressed as a date
// or a delay in secs
SimpleDateFormat format = new SimpleDateFormat("EEE, dd MMM yyyy HH:mm:ss zzz");
try {
Date d = format.parse(retry_after);
// pass the date
d.toInstant().toString();
}
catch (ParseException pe) {
LOG.debug("Incorrect date found for retry-after: {} - {}",retry_after, url);
}
}
}
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Retry-After
if a site is asking for a delay, we should enforce it in the internal queues of the FetcherBolt.
The text was updated successfully, but these errors were encountered: