Telegram Project - Hours and hours spent on one post/api call

[bigchipbag]

Using the latest version of the warrior, There are some items that take hours to complete. I am not sure if this is expected behaviour and just what happens maybe with some gigantic telegram channels, but this is behaviour which prevents the warrior from cleanly shutting down. I requested to shutdown minutes ago and it just sits here continuing.

Ideally these would not prevent clean shutdowns and could potentially be partially-submitted to the tracker, not sure what happens but I have to often click shutdown immediately and wonder if someone else gets stuck with this task again

[JustAnotherArchivist]

Yeah, these are known and caused by a handful of channels with a very large number of comments per post. Unfortunately, there's no real alternative to this deep chain of requests without missing data. We know of three particularly problematic channels so far, two of which are currently not being processed at all due to this. The third has much fewer comments than those two, but it can still run for a long time unfortunately. I'll check again which one is causing it this time.

Partial submissions are not currently possible. Either the item (or rather, multi-item, since you get more than one item at once) gets completed or everything fails and needs to be redone from the start. Shutting down uncleanly will indeed lead to the items getting retried. That's unfortunate, but don't worry about it (too much). If you need to shut down your machine, just do.

[bigchipbag]

Great, thank you for the clarification. I wanted to check to see if it was an issue, and yes I do eventually close it out after waiting a bit to see if it can cleanly shut down.