Skip to content

Failed Download Handling (FDH)

Jump to bottom Edit New page
hugepants edited this page Oct 3, 2015 · 14 revisions

Introduction

The proliferation of fakes, incompletes, corrupt files, codec scams, propagation issues, upload glitches and DMCA takedowns can break the automation process and require manual intervention.

Failed Download Handling (FDH) in PVR automation apps solves this problem. It varies in implementation but typically functions as follows:

  • Downloader encounters a bad download and sets its status to failed
  • Post-processing script notifies the PVR app or the app scans downloader history for failed entries
  • PVR app blacklists the release from appearing in future search results
  • PVR app searches for an alternative
  • Alternative is sent to downloader
  • Repeat until a good download is done
  • If there are no alternatives, keep searching periodically for new results

If you don't use PVR apps or your apps don't support FDH, an alternative way to utilize FDH is to have the indexer do it via the X-Failure header.

Interface methods

When a downloader encounters a failed download and updates its job status to 'failed', there needs to be a communication channel to interface between the downloader and PVR app in order to perform the steps outlined above. There are two methods:

  1. On-demand fail notify with a post-processing script (e.g. with sabToSickBeard.py or nzbToMedia.py)
  • A script in the downloader runs once at the end of the download job and notifies the PVR app of a failed download.
  1. History scanning
  • The PVR app frequently scans the downloader's history to find failed entries.

For the most efficient and reliable interfacing, on-demand fail notify is recommended. While History scanning is not as detrimental as folder scanning is, frequent API polling presents more opportunities for errors to occur. If you're using post-processing scripting for other purposes like on-demand processing, then you may as well use it for on-demand fail notify too.

nzbToMedia is a richly featured suite of scripts that is ideal for FDH. In addition to providing "fail notice" interfacing, it can detect bad downloads more effectively with ffprobe media checking, as well as many other features.

Sick Beard and its forks

The original, standard version, midgetspy/Sick-Beard, does not have FDH.

Tolstyak was the first to develop FDH for Sick Beard in March 2012. Several PRs were made to midgetspy/Sick-Beard but they were never merged so development moved to his fork archived at Tolstyak/Sick-Beard. This fork regularly pulled in updates from upstream, so it was very similar to the standard version but with the addition of FDH. Development of this fork ended in 2013 but it still works today and its FDH code continues to work great.

Tolstyak's FDH code was added to mr-orange/Sick-Beard and continued in SickRage (based on Mr Orange).

Of all the Sick Beard+FDH forks, SickRage and SickGear are the only forks in active development. However, the discontinued "Mr Orange TPB" and "Tolstyak" forks are valid working alternatives.

For Sick Beard forks with FDH, enable: Settings > Search settings > Episode Search > Enable Failed Download Handling

Sick Beard forks with FDH can only use post-processing scripts for failed interfacing. History scanning is not supported. As scripting is superior, this is not an issue. You can use one of the following scripts:

  1. sabToSickBeard.py
  • Included with Sick Beard forks. Basic FDH support.
  1. nzbToMedia.py
  • Third party suite of scripts. Full FDH support with many additional features. Recommended.

nzbToMedia autoProcessMedia.cfg settings for Sick Beard forks here.

Known Sick Beard forks

Version FDH Details In active dev?
midgetspy/Sick-Beard No Standard version No
Tolstyak/Sick-Beard Yes Similar to standard version No
mr-orange/Sick-Beard: ThePirateBay Branch: Yes Torrent support, updated UI No
mr-orange/Sick-Beard: Pistachitos Branch ? ? Yes
thorli/Sick-Beard ? ? No
SickRage Yes Based on Mr Orange with more added features Yes
SickGear Yes ? Yes

Sonarr

Sonarr added its own implementation of FDH in October 2013. It is in active development, gaining popularity, and is widely praised.

As Sonarr is a standalone project with a dedicated website, FDH is prominently advertised on its front page. Because of this, FDH for TV automation is commonly associated with Sonarr, yet Sick Beard forks such as Tolstyak's had FDH a year before Sonarr did.

From a pure FDH standpoint, midgetspy/Sick Beard has been unusable for years, prompting an exodus to Sonarr. But contrary to popular belief, good FDH alternatives have always been available via the various Sick Beard forks which continue to have excellent FDH to this day.

Sonarr supports history scanning but not on-demand fail notify. For standard post-processing it uses the less efficient folder scanning but this can be disabled to allow nzbToMedia to be used as a post-processing script in the downloader for on-demand processing and for the other (non-FDH) nzbToMedia features.

Couch Potato

The standard version, RuudBurger/CouchPotatoServer, supports FDH and is in active development. Therefore there is no need for forks so this is the recommended version to use.

Couch Potato supports History scanning for FDH as standard, but this can be disabled so you can use nzbToMedia for on-demand fail notify and on-demand post-processing.

Downloader configuration

###SABnzbd Switches > Queue

Abort jobs that cannot be completed = Enable

Action when unwanted extension detected = Abort

Unwanted extensions = exe, com, cmd, bat

Switches > Post-Processing

Post-Process Only Verified Jobs = Disable

Special

empty_postproc = Enable

new_nzb_on_failure = Disable. To avoid clashing with Indexer FDH.

Also see: https://github.com/clinton-hall/nzbToMedia/wiki/sabnzbd

NZBGet

See: https://github.com/clinton-hall/nzbToMedia/wiki/nzbget

Indexer failed download handling

Some indexers have support for FDH on the server-side. The FDH code is running on the indexer's server so you don't need to have FDH locally with a PVR app. There are pros and cons to this which we will get to later.

When a NZB is pushed/uploaded from an indexer to your downloader, or pulled/downloaded to your downloader, this is done over HTTP. HTTP can supply metadata alongside a transfer in the form of headers. A custom header called X-Failure is sent by the indexer's server with each NZB file transfer.

Indexer > NZB file > Downloader

Indexer > X-Failure header > Downloader

The X-Failure HTTP header contains a unique URL, for example: https://some-indexer-here/fail/12345678

If a download fails, the downloader 'visits' this URL. This sends a signal to the indexer that the download has failed. The indexer has its own proprietary FDH code which pushes an alternative NZB to the downloader for downloading.

Pros

  • No requirement to run PVR apps locally. File processing such as renaming can be handled with downloader features or post-processing scripts.
  • Can be easier to set up

Cons

  • The FDH code running on the indexer is proprietary code that cannot be audited
  • Updates and fixes can only be made by the indexer
  • If the indexer site is down or disappears, nothing will work, whereas a PVR app can use an alternative indexer
  • Requires NZB push so external access to your downloader needs to be set up

To enable in SABnzbd, set this in Config > Special:

new_nzb_on_failure = Enable

Add a custom footer
Add a custom sidebar
Clone this wiki locally