Possible enhancements for --from-quarantine

[philbudne]

The --from-quarantine option on a worker (queue reader) causes input to be taken from the PROG-quar queue instead of PROG-in as usual to allow re-processing of stories that were "kicked out" of normal processing for some extraordinary reason, or excessive retries.

There are two issues/inconvenieces:

1. If the issue that caused the quarantine for some (or all) stories still exists, the stories will be re-written to the quarantine queue, and the program will loop endlessly.
2. on the other hand, if all stories are processed successfully, the program will eventually block waiting on an empty queue. Typing ^C will exit with a KeyboardInterrupt exception, which in production or staging will cause a sentry.io alert!

The first could be fixed by adding one or more additional x-mc-.... RabbitMQ headers in Worker._exc_headers that identify the hostname and process-id where the exception was caught. Checking that the message header

1. x-mc-when is >= the start time of the current process
2. AND that x-mc-host & x-mc-pid (or a combined x-mc-host-pid string) match the current process
   can be taken to mean all messages that were in the quarantine queue at program start have been checked, and any remaining are still problematic.

The second could be handled by setting a Unix "alarm" signal (reset each time a message is received), such than when the alarm fires (due to no messages received in the timeout period) the input loop is exited (the indexer/scripts/qutil.py program does this by calling basic_cancel).