Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[REQUEST] Option to keep stalled torrents. #84

Closed
devanteweary opened this issue Jun 23, 2024 · 72 comments · Fixed by #86 or #95
Closed

[REQUEST] Option to keep stalled torrents. #84

devanteweary opened this issue Jun 23, 2024 · 72 comments · Fixed by #86 or #95
Assignees
Labels
enhancement New feature or request

Comments

@devanteweary
Copy link

devanteweary commented Jun 23, 2024

Hey, touching on my questions from earlier, thought I'd make some suggestions here.
I don't know if you like 1 post per topic. Sorry if it seems like I'm spamming!

So this suggestion is:

  • Option to NOT delete stalled torrents. Only add them.

Sort of like there's an option to never delete seeding torrents, have the same option for torrents when searching for new ones to replace.

It might look like this:

[Radarr-1080.Torrent]
AutoDelete = false
IgnoreTorrentsYoungerThan = 86400
MaximumETA = -1
MaximumDeletablePercentage = 0.75
DoNotRemoveSlow = true

KeepStalled = true

And the behavior would be:

  • qBitrr sees a stalled torrent.
  • Adds that to the blocklist as normal.
  • Searches for a new release as normal.
  • Keeps the stalled torrent as well.

This way, the old one has a chance to complete one day, but you still have qBitrr looking for new ones.

Some uses cases... sometimes there will be a torrent that has stalled for a week or two, but by the grace of the torrent gods, someone will connect with the rest.

I see there's the qbitrr-ignored tag, but that's not really the same thing I believe.
That just causes qBitrr to ignore that one being stalled if I'm not mistaken.

This would be super helpful for torrent from private trackers that are stalled.
If one is stalled, I don't want it delete because it'll probably give me a hit and run.
But still want qBitrr to look for a different release.

No delete. Just add! :P'

OK thank you!

@Feramance
Copy link
Owner

This is doable, shouldn't be terrible to implement. At first I'll implement this using a tag, proceeding from there I'll make the changes for the no tagging version.

This change will also need to adjust the search behaviour as currently qBitrr only searches for items that aren't in the queue for Radarr/Sonarr

@devanteweary
Copy link
Author

Dang dude you're awesome. Check your Patreon messages, by the way.

@Feramance Feramance self-assigned this Jun 24, 2024
@Feramance Feramance linked a pull request Jun 24, 2024 that will close this issue
@Feramance
Copy link
Owner

This specific issue should be resolved currently in docker tag pr-86. Give it a try and let me know how it goes

@devanteweary
Copy link
Author

Great and thank you.

I tested it out with each of the following:

  • AllowedStalled = true

and

  • StalledDelay = 0
  • StalledDelay = -1
  • StalledDelay = 86400

Looks like it's just ignoring the new settings outright.
It's still going in and removing the stalled torrent and re-searching for a new one.

According to the logs, it's qBitrr v4.7.5-7c72ac4
Although I think that version number is the same as it was yesterday.

When I have it create the config.rename_me, it does in fact have the new AllowedStalled and StalledDelay fields.

Hope this helps!

@Feramance
Copy link
Owner

The version number for now will remain the same, only the hash will change, until the change is completed. Can you set logs to trace and send over a log?

@Feramance
Copy link
Owner

If not tonight, tomorrow, I'll be doing some testing myself too

@devanteweary
Copy link
Author

devanteweary commented Jun 24, 2024

Whoops shoulda had that for you already.

Here are the logs: qBitrr Logs
Also included my config.toml in there.

The one I'm looking at is Lionheart.
I know there are no good seeds out there so it's always gonna get to around 90% and then stall, so it's a good test.


By the way, the reason I can keep reusing Lionheart is because even though qBitrr keeps adding it to the blocklist and multiple entries at that, it seems like Radarr is ignoring the blocklist and re-adding it to qbittorrent everytime. Funny thing is in an interactive search, it DOES show the blocklisted icon. Was gonna bring it up later as an issue but didn't want to complicate things. Just mentioning right now in case you were wondering how I'm using the same torrent test over and over.

image

@Feramance
Copy link
Owner

The reason for grabbing the same torrent is due to use of multiple trackers which would have the same torrent. For Radarr/Sonarr each torrent on each tracker is different, so this happens a lot

@Feramance
Copy link
Owner

Also, let the trace logs run for 15-30 mins before sending them over. There's no info to work with in the logs you sent me

@devanteweary
Copy link
Author

devanteweary commented Jun 24, 2024

Also, let the trace logs run for 15-30 mins before sending them over. There's no info to work with in the logs you sent me

OK, I let it run for about an hour: qBitrr Logs

SHould show it replacing Lionheart every 30 minutes.

@Feramance
Copy link
Owner

I went over everything and it just dawned on me. Can you test again tagging the torrent with the new tag qBitrr-allowed_stalled? Once it's working with the tag I can get to completing work on the tagless update

@Feramance
Copy link
Owner

Made some additional changes, kindly update your container and test again please

@Feramance
Copy link
Owner

If you can send me the magnet link to the Lionheart torrent so I can test please

@devanteweary
Copy link
Author

devanteweary commented Jun 26, 2024

If you can send me the magnet link to the Lionheart torrent so I can test please

OK all done. Same link: qBitrr Logs
Magnet is in there too.

  • I assigned the tag in qBittorrent.
  • I forced updated the container under the same pr-86 tag.
  • Now it isn't doing anything with Lionheart.
  • The log show that it seems like it's trying to grab a couple of random movies that already exist. Such as Bambi, The Thing, Metroplois, and a few others since zipping this log up. But I don't think it's actually performing those actions. (I see nothing being added to qbittorrent and the progress text doesn't show up in Radarr.)
  • I don't remember it sleeping for only 5 seconds each time but maybe I'm wrong on that.

Hope it helps!

Oh and if you want a quicker response, I'm DevanteWeary on Discord.

@Feramance
Copy link
Owner

I noticed an issue in the logs you sent, I'll fix it and hopefully the logic will be functional

@Feramance
Copy link
Owner

Update to the latest version, I think it is now resolved and functional. Once confirmed I'll push the official update and continue work on the tagless update

@Feramance
Copy link
Owner

Though remind me, you wanted a search to still occur if the torrent is stalled right?

@devanteweary
Copy link
Author

Though remind me, you wanted a search to still occur if the torrent is stalled right?

OK. I did the same update. Forced updated the pr-86 tag.
Unfortunately it still is having issues.

This time I tried three scenarios so there are three sets of logs: qBitrr Logs
I also threw another movie in the mix - Anchorman. What I did was add a torrent I knew would be stalled for it. (no seeders).

Here's what happened:

logs_minus_one_no_tag: StalledDelay set to -1. Did not assign tag.

  • No change. Still continues to remove stalled torrent, adds it to blocklist, and re-search new one (which ends up being the same exact torrent even though it's on the blocklist).

logs_zero_no_tag: StalledDelay set to 0. Did not assign tag.

  • Same results.

logs_zero_using_tags: StalledDelay set to 0. Manually assigned qBitrr-allowed_stalled tag.

  • Removes stalled torrent, adds it to blocklist, re-searches, and the new one does NOT get auto-assigned any tags.

Also, with Anchorman, I noticed it will remove the stalled torrent, add it to blocklist, and re-add a new torrent that has a bunch of seeders and actively downloads (desired behavior). Then if I pause it, it'll get removed on the next loops (which I set to 5 minutes).

One more thing, each time I tested, I removed all the blocklist entries as well. Oh and I let them run for a while. At least 30 minutes each time I think.

OK that's it for now!

@devanteweary
Copy link
Author

I just had another movie downloading, had plenty of seeds and was actively downloading.

Just on a hunch, I ran qBitrr and sure enough, it removed that torrent, blocklisted it, and re-searched it.

And I forgot to answer you...

Yeah what I was looking for originally is for it to go ahead and blocklist and re-search stalled torrents.
Just not remove them from qbittorrent. Allowing the stalled torrent and newly searched replacement torrent to run side by side.

@Feramance
Copy link
Owner

Noted, have you tried with a price value for a delay? Say 60 (minutes)?

@Feramance
Copy link
Owner

Otherwise I think I can have this ready by Sunday at the most (maybe Monday)

@devanteweary
Copy link
Author

devanteweary commented Jun 28, 2024

Noted, have you tried with a price value for a delay? Say 60 (minutes)?

I just tried with 3600 (1 hour) and it just did the same thing. Removed torrent (ignoring the `StalledDelay = 3600'), blocklist, re-search (adding the same exact torrent, ignoring Radarr's blocklist).

The odd thing about the blocklist situation is it seems like this is the only movie it's ignoring the blocklist for.
I can tell it's re-adding the same one because it's the same hash that it's blocking.

I'll do some more testing on that front, but seems like a Radarr issue.

If you ever want to remote into my server and see what it's doing in realtime, just let me know here or Discord (DevanteWeary).

@Feramance
Copy link
Owner

Noted, I've made some changes and added more logs. I'm also testing this functionality with the Lionheart torrent you provided and things seem to be working correctly now. Please update and let me know, trace logs are now clearer on what the new functionality is doing

@Feramance
Copy link
Owner

Also the delay is in minutes, so 3600 would be equal to 60 hours. I've updated the config description

@devanteweary
Copy link
Author

devanteweary commented Jun 29, 2024

Noted, I've made some changes and added more logs. I'm also testing this functionality with the Lionheart torrent you provided and things seem to be working correctly now. Please update and let me know, trace logs are now clearer on what the new functionality is doing

OK updated the pr_86 again and still didn't work.

Also note how the last time Lionheart we removed and re-searched, it was at 54% so was actively downloading. It stalls at about 94ish%.

Oh and I changed the StalledDelay to 60.

@devanteweary
Copy link
Author

devanteweary commented Jun 30, 2024

I also tested out some other movies: Bad Moms and Wayne's World.

  • It removed the stalled ones.
  • It also removed the PAUSED ones.

Then re-searched those as well.
And finally, I tagged them in this test.

More qBitrr Logs

image

So to sum up, the current issues are:

  • It's removing paused torrents.
  • It's removing active downloads.
  • It's ignoring AllowStalled.
  • It's ignoring StalledDelay (if I understand correctly what it's supposed to do)
  • In the case of Lionheart, it's re-searching the exact same torrent (and hash) even though it gets blacklisted.

That's it!

@Feramance
Copy link
Owner

Ok so going through your logs I can deduce the following:

The stalled delay is working, I wasn't clear on how it works. It checks for the time delay from the time the torrent was added and when looking at your logs, the ones removed had been added more than 60 minutes prior to the run, so that logic is functional.

What's definitely wrong is the processing of paused torrents, that I'll need to adjust for sure

@devanteweary
Copy link
Author

devanteweary commented Jun 30, 2024

That's what I understood. In other words, if a torrent is only 57 minutes old, it will not be removed/replaced IF stalledDelay is set to 60.

I haven't looked at those logs in particular but I can tell you it's ignoring the 60 minutes rule for sure.
For example, it's just replacing the Lionheart every loop (5 minutes - ultimately I'll set it to 60 minutes).

And remember, it was ignoring the setting even when I had it set to 60 hours too.

Sorry man I know you've been working hard on it.
And I'm always open to you remoting in and taking a look for yourself firsthand.

I did one quick test just now.
I had Lionheart in there since 9pm last night.
I just now turned on qBitrr and it immediately removed Lionheart then re-searched and added it back.
Then after the 5 minute loop, it removed and re-searched Lionheart again. And I noticed Lionheart wasn't even stalled yet. It was still actively downloading.

Here's a screenshot of a second or two before it removed Lionheart, and then a screenshot of the log right after.

Right before:
image

And right after:
image

And of course the actual logs.
This is a brand new set of logs cleared before the test: qBitrr Logs - Quick Test

@Feramance
Copy link
Owner

Can you send me your config again please to check a few things? And logs too. Also sorry for the delayed reply, had a hectic weekend

@Feramance
Copy link
Owner

Feramance commented Jul 8, 2024

Ignore the previous message, I found the logs. I made some changes (logging updates too) and the crashing error should be fixed, take another look please. Also delay -1 seems to be working correctly. I do plan on seeing how to add timestamps at some point but it hasn't been an issue given that actions which are time dependant are logged appropriately. use feramance/qbitrr:nightly

@devanteweary
Copy link
Author

devanteweary commented Jul 8, 2024

Ignore the previous message, I found the logs. I made some changes (logging updates too) and the crashing error should be fixed, take another look please. Also delay -1 seems to be working correctly. I do plan on seeing how to add timestamps at some point but it hasn't been an issue given that actions which are time dependant are logged appropriately. use feramance/qbitrr:nightly

OK, updated to nightly and here's what I got.
I included logs and my config in each archive.

tldr;

  • Paused untagged are removed. Mentioned because status is PAUSED_DOWNLOAD when its removed.
  • Tagged are NOT removed, but also not blocklisted/re-searched.
  • StalledDelay seems to be taken in account correctly.

-- [ Delay 0 - Untagged ] -- qBitrr Logs
Loop 1 - LH (stalled): Removed, blocked, re-searched.
Loop 2 - LH (stalled): Removed, blocked, re-searched.
Loop 3 - LH (stalled): Removed, blocked, re-searched.
Loop 4 - LH (paused): Not removed.
Loop 5 - LH (paused): Removed, blocked, re-searched.
Loop 6 - LH (paused): Removed, blocked, re-searched.
Loop 7 - LH (paused): Removed, blocked, re-searched.

-- [ Delay 0 - Tagged ] -- qBitrr Logs
Loop 1 - LH (stalled): Untouched and not blocked.re-searched.
Loop 2 - LH (stalled): Untouched and not blocked.re-searcheD
Loop 3 - LH (stalled): Untouched and not blocked.re-searcheD

-- [ Delay 1440 - Mixed ] -- qBitrr Logs
Loop 1 - LH (stalled, tagged): Removed, blocked, re-searched.
Loop 2 - LH (dling, untagged): Untouched and not blocked/re-searched.
Loop 3 - LH (paused, untagged): Untouched and not blocked/re-searched.
Loop 3 - LH (paused, tagged): Untouched and not blocked/re-searched.

@Feramance
Copy link
Owner

Ok so I reopened a PR to handle things better for updates so use feramance/qbitrr:pr-84 again please goiung forward.

Overall the only incorrect behaviour I found was the paused download status handling. If you manually tag a torrent the logic won't work correctly as the torrent is blocked and researched only when the tag is added, not if it already there (to avoid blocklisting and researching on every loop run). Otherwise, besides the paused download, it seems to me that the expected behaviour was correct, please correct me if I am wrong.

@devanteweary
Copy link
Author

devanteweary commented Jul 9, 2024

Ok so I reopened a PR to handle things better for updates so use feramance/qbitrr:pr-84 again please goiung forward.

Overall the only incorrect behaviour I found was the paused download status handling. If you manually tag a torrent the logic won't work correctly as the torrent is blocked and researched only when the tag is added, not if it already there (to avoid blocklisting and researching on every loop run). Otherwise, besides the paused download, it seems to me that the expected behaviour was correct, please correct me if I am wrong.

Hey just to make sure, do you want me to use pr-84 or is it pr-86?

About the manual tagging... OK I actually didn't understand that before so thank you!
So in that case, the one I labeled [ Delay 0 - Untagged ] above is the most valid test, right?

If so, the issue it seems to be having goes back to the original intent of this feature request: it's removing stalled torrents from qBittorrent instead of keeping them.

Current behavior:

  • ✔ Block torrent in Radarr/Sonarr.
  • ✔ Re-search for a new one.
  • ❌ qBitrr removes the torrent from qBittorrent.

Desired behavior:

  • ✔ Block torrent in Radarr/Sonarr.
  • ✔ Re-search for a new one.
  • ✔ Do NOT remove stalled torrent from qBittorrent. Keep it in there.

So if I had to put it in options form, the desired behavior would look like this:
BlockStalled = true
ReSearchStalled = true
RemoveStalled = false

@Feramance
Copy link
Owner

Both are wrong actually, it's feramance/qBitrr:pr-95

@Feramance
Copy link
Owner

Regarding the removing of the torrent from qbittorrent when blocking, I'll investigate further, need to consult the Arr APIs

@Feramance Feramance linked a pull request Jul 9, 2024 that will close this issue
@Feramance Feramance added bug Something isn't working enhancement New feature or request and removed bug Something isn't working labels Jul 9, 2024
@devanteweary
Copy link
Author

Hey, came across this: line 43

Sorry if this is nothing, just saw it and wanted to throw it out there!

@Feramance
Copy link
Owner

After looking into pyarr (the arr api used by qBitrr) further, it seems that the delete queue isn't working as intended. After some manual tested I found a solution and have implemented it. Kindly update to the latest pr-95 build and let me know how it goes.

@Feramance
Copy link
Owner

Current behavior:

  • ✔ Block torrent in Radarr/Sonarr.
  • ✔ Re-search for a new one.
  • ❌ qBitrr removes the torrent from qBittorrent.

Desired behavior:

  • ✔ Block torrent in Radarr/Sonarr.
  • ✔ Re-search for a new one.
  • ✔ Do NOT remove stalled torrent from qBittorrent. Keep it in there.

I think this last update should meet the desired behaviour, as such I have started work on the TAGLESS update and made a good amount of progress already

@devanteweary
Copy link
Author

I think this last update should meet the desired behaviour, as such I have started work on the TAGLESS update and made a good amount of progress already

Thank you so much for sticking with it!!! You're amazing man.
OK some good news and bad news.

Good news is it worked the first time, but bad news is after that, it stopped working and also did a couple things it didn't do before, like say it was blocking/researching but didn't actually do it.


-- [ Delay 0 ] <-- Worked perfectly!!! - qBitrr Logs
Loop 1 - LH (#1)(stalled): tagged as allowed, NOT removed, blocklisted, re-searched, new torrent added
Loop 2 - LH (#1)(stalled): already checked
Loop 2 - LH (#1)(downloading): not stalled
Loop ? - LH (#1)(stalled): already checked
- LH (#2)(uploading): tagged as imported

-- [ Delay 1440 ] - qBitrr Logs
Ignored delay, removed torrent, but did not block and did not re-search
Log says deleted stale torrent: stalled state

-- [ Delay 0 - IgnoreYoungerThan 1200 ] - qBitrr Logs
After 10min, just skipped

-- [ Delay 0 - Added torrent after launching qBitrr ] - qBitrr Logs
qBitrr tagged the torrent, then didn't do anything with it.

-- [ Delay 0 - Added torrent before launching qBitrr ] - qBitrr Logs
Loop 1 - LH (#1): Skipping file check
Loop 2 - LH (#1)(stalled, manually removed tag): qBitrr added tag, did not block, did not research
Loop 3 - LH (#1)(stalled, still using tag from loop 2): skipped again

-- [ Delay 0 - I Forced Update of qBitrr ] - qBitrr Logs
Loop 1 - LH (#1)(stalled): tagged, said it was blocking and researching but didnt actually do it in Radarr
Loop 2 - LH (#1): skipped

aaaaaand I just tried a test... nothing special just wanted to since it's now hours later from those previous test...

-- [ Delay 0 ] - qBitrr Logs
Loop 1 - LH (#1)(stalled): tagged, said it was blocking and re-searching but didn't actually do it in Radarr
Loop 2 - LH (#1): skipped

@Feramance
Copy link
Owner

Ok so after checking all the logs and logic, I determined that I think the test with the 1440 delay needed a restart or needed the update to work as intended (I have added logs to log the stalled delay value for this reason). In addition when having a 0 delay the skipping is intended if its already been processed or has no changes required, so that's normal (skipping means 'let it be' essentially.

Lastly, the only real issue I saw was the not blocking and searching, I have made a small adjustment with added logs to hopefully fix the issue and if not at least better determine what's going on. So please UPDATE first, then test... Fingers crossed

@devanteweary
Copy link
Author

OK I wanna saaaay.... looking good so far!
Lionheart worked perfectly.
I went ahead and unleashed it on my series and that seems to be going great too.

I don't know man, I wanna say delay0 is a success!
As for an actual delay, I tested StalledDelay = 20 and...

  • Tagged torrent but did not do anything else.
  • After the 20 minutes was over, it removed the torrent and did NOT blocklist/re-search. (should not have removed)
  • Log said it was removing stale torrent.
  • qBitrr Logs

But for the intent of keeping stalled torrents while searching for new ones... looks like you got it working!!

I wanted to ask... so I just assumed by removing the qBitManager folder, I was "starting fresh" i.e. removing the flag that said a torrent had already been check. But I don't think that's the case at all, is it?

@Feramance
Copy link
Owner

Very happy to hear, and regarding the researching, it'll only work if qBitrr itself tags the torrent, not if you do. You should never need to tag anything except for the ignore tag, but I'll take a look tomorrow before merging and getting down to having a function tagless version.

Regarding your question about deleting the folder, the way qBitrr works, it doesn't really care about the folder, it rebuilds all the data it needs with every database update to keep up to date with the Arr apps. In short, restarting it is enough to literally start fresh with whatever configs you have set in your config.toml

@Feramance
Copy link
Owner

Also to clarify, stalled delay 20 means remove after 20 minutes from torrent added time of stalled. Some scenarios:

  • if torrent is 24 mind old when it stalled, then the 20 mins limit is already exceeded and is removed and re searched
  • if torrent is 10 mins old then torrent is researched and allowed to remain for an additional 10 minutes before being removed

Re search happens when the torrent is tagged as stalled only, or when removed if it exists in the Arr app's queue

@devanteweary
Copy link
Author

Oh snap OK no I was totally mistaken.
I thought it was a "don't even re-search, block, even LOOK at this torrent until it's at least 20 minutes old."

Well that's a cool option then.
You've basically let us keep stalled ones in qbittorrent but also given us a way to remove them after a certain amount of time.

OK OK sorry one more thing. In qBittorrent, I don't think the status "Downloading metadata" is considered stalled even if it's been that way for a while. I see that those torrents show up in the Downloading filter and not the Stalled filter, so assuming that is relevant?

@Feramance
Copy link
Owner

Feramance commented Jul 15, 2024

I forgot the exact rule but after a while if a torrent is still stuck on downloading metadata, it should be detected as stalled yes, if not I'll amend it

@Feramance
Copy link
Owner

So just to clarify regarding the Downloading Metadata, the way it's handled is if the download is younger than the config IgnoreTorrentsYoungerThan then it ignores it. If it exceeds the time configured and it's still Downloading Metadata, then its marked as stalled.

@devanteweary
Copy link
Author

Just a follow up.
This function has been working like a charm!

@Feramance
Copy link
Owner

Very happy to hear. Sorry on the delay for the tagless, been very busy lately and had to resolve some issues

@norimicry
Copy link

norimicry commented Aug 26, 2024

Trying to use this feature to prevent H&R but it seems to be deleting the stalled torrents right away. Am I doing something wrong?

ReSearchStalled = true
StalledDelay = 11520 (required time to prevent H&R for tracker)

@Feramance
Copy link
Owner

Did you restart qBitrr after changing these settings? And if so, please attach trace logs with this enabled. Thanks

@Feramance Feramance reopened this Aug 26, 2024
@Feramance
Copy link
Owner

Closing due to no response

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
3 participants