Investigate exposing to the jobs the possibility of querying if the item with a given key already exists

[ivan-brko]

Use case:
If we are writing a parser which parses some news site and we don't want to to store duplicates, at the moment we will still send requests for these duplicate articles only to have them discarded later on when storage to DB occurs.
Instead of this, investigate adding a function like following to job API which would allow the job/scripts themselves to to not query duplicates:

def get_nonduplicate_list(list_containing_all_keys) -> list_containing_only_non_dupes