Exclude false positives?

[andromidasj]

Hey! I did a dry run of the script over a folder of .SRTs and skimmed through the results before running the script fully. It was 99% great, but there were a few false positives that I noticed, like in the example below:

WARNING: dry run: nothing was altered.
    INFO: [---------------------------------------------------------------------------------]
    INFO: loading subtitle: test.srt
    INFO: now cleaning subtitle: test.srt
    INFO: Done. Cleaning report:
          | 4 deleted blocks and 1 warnings remaining.
          |
          |     [---------Removed Blocks----------]
          |     30
          |     00:01:16,946 --> 00:01:19,274
          |     S01E05
          |     The Cabin
          |     reasons: (global_purge3, nearby_ad, adjacent_ad)
          |
          |     31
          |     00:01:19,294 --> 00:01:22,116
          |     sync and corrected by ninh
          |     www.addic7ed.com
          |     reasons: (global_purge4, en_warn1, en_warn1, en_warn2, global_warn1, global_warn1, nearby_ad, adjacent_ad, similar_content, wedged_block)
          |
          |     32
          |     00:01:22,977 --> 00:01:24,444
          |     Roland, I just
          |     need the door fixed.
          |     reasons: (en_warn1, nearby_ad, adjacent_ad)
          |
          |     469
          |     00:21:26,036 --> 00:21:28,698
          |     sync and corrected by ninh
          |     www.addic7ed.com
          |     reasons: (global_purge4, en_warn1, en_warn1, en_warn2, global_warn1, global_warn1, close_to_end, similar_content)
          |     [---------------------------------]
 

As you can see, blocks 31 and 469 should definitely be removed, but block 32 should stay. (I'm indifferent about block 30)

Is there a way to run the script and ignore those false positives? I'm glad the dry run caught these, but I'm not sure where to go from here 🙏

[KBlixt]

Hi,

I can take a look at this in the evening.

What I'll probably do is introduce a option to the script that make it ignore the received point from being adjecent to an add.

This will cause a few more false negatives, but should resolve this false positive as well as a few others.

If you have more false positives, I can take a look at them and see if there is anything I can do about them.

The alternative is to run the script and verify all removed blocks and recover false positives by copy pasting them in.

The --end-report option is quite useful for finding false positives and false negatives.

[andromidasj]

Sounds good, thanks for looking into this, @KBlixt!

I had a random idea that might be able to help people run a bulk job without needing to copy/paste and re-add false positives:

What if the script could create a quick SHA-1 hash of each block as it scans for ads, and display this hash (or a truncated portion of it) in the --dry-run output. Then, when a user scans through the dry run output to verify the results, they can note down a list of the hashes that they want the script to ignore. Then the script could be run with an argument like --ignore-hashes followed by a list of hashes which the script will skip over as it generates the hashes.

Again, just a random idea with a probably complicated execution and performance hit, but it could help solve the issue of handling false positives with minimal user intervention.

[KBlixt]

sorry for the late fix, I've made an update to prevent the the false positive you provided.

I've had some ideas of building some kind of white-listing solution or a more integrated way to resolve false positives by a simple press of a button. but for now I'm improving the script with small adjustments and maybe I'll get to it in the winter.

If you have any more false positives, please send them, fixing them will provide better results for everyone. :)

[andromidasj]

Can confirm that it works great now on the same sample I used earlier. Thanks!