Adding support for RAR 5.0 format

[DoggoKt]

Hey, I really like this projcet and I've studied a bunch of the RAR format lately. I've tried using your package on a rar5 file and realized it didn't work because, well, there's a few main changes in parsing 5.0 files. The spec is https://www.rarlab.com/technote.htm, but honestly reading the unrar source is a little easier.

1. CRC-32 instead of CRC-16, meaning the CRC parts of headers are 4 bytes, not 2 anymore.
2. The order is different. They kept the "old" rar4.0 order in the signature aka magic bytes for obvious reasons.
3. There is not a fixed size for anything except for the CRC signatures - you read seven bits from a byte, and the top one says if to read more or not. Not difficult to implement, just very different.
4. The signature is one more byte longer (not the fixed string, anymore, there's an extra 0x01)

This is not a feature request, more of an announcement that I'll be adding the support for it. It should be also automatically detectable from the first 7 bytes due to different signatures. As said, kudos to you, really cool projet and good work. I'll try to work on it as much as I can. Not that interested in making it browser-ready, but I've thought about adding stremio support (with a similar system to rar-http).

[1313]

Cool, thanks for letting me know, should be fairly simple to write a new set of header parsers. I'm afraid I don't have the time at the moment but feel free to submit a PR and I can guide you on the way.

Thanks for the kind words 🤘

[jaruba]

bump, we hit this case recently too..

[jaruba]

@DoggoKt oh, i just saw you said "but I've thought about adding stremio support (with a similar system to rar-http)", so i guess we are working for the same goal 😅

[DoggoKt]

@jaruba Oh hey. I'm still working on the RAR5 parsing, I have about 20 commits not pushed. Lot of them are also just fixes and little tweaks to the main source code. It should be integrated and of course backwards-compatible so if you're up to adding features, feel free :)

Cheers!

[jaruba]

@DoggoKt i am very limited in my understanding of the archive related logic and RAR5, i can tell you where we are at after a very long time:

• we can stream a video file from a nntp engine (usenet) with range support (so seeking works with the video)
• we can stream RAR files (but not RAR5) with multi-part (through a nntp engine or http) and range support
• we can (probably) stream zip files (no multi-part support) and without range support (no video seeking) through nntp or http(s) with zip-http

i think that my conclusion months ago was that there is a technical limitation to range requests for zip streams, but i am not knowledgeable enough in the field to ever be sure, i have found some cases of zip and multi-part 7z on usenet too

i have also played around a bit with par2 parchive parsing and verification, maybe repairs too

i have implemented multi-server support for usenet, so if a chunk is missing a backbone would be used..

there is a lot of refactoring to be done, but RAR5 (which seems pretty common) is probably the biggest roadblock for Stremio atm

[DoggoKt]

Oh damn, streaming multi-part files is something I'd absolutely appreciate. Ideally I'd connect with real-debrid since that opens the biggest amount of sources. I'll work on the RAR5 and I'll ping you in the upcoming pull request. Very interested in how this will end up!

…

Am 4. 12. 2024 um 20:25 schrieb Alexandru Branza ***@***.***>:  @DoggoKt i am very limited in my understanding of the archive related logic an RAR5, i can tell you where i am at after a very long time: we can stream a video file from a nntp engine (usenet) with range support (so seeking works with the video) we can stream RAR files (but not RAR5) and multi-part (through a nntp engine or http) we can (probably) stream zip files (no multi-part support) and without range support (no video seeking) through nntp or http(s) i think that my conclusion months ago is that there is a technical limitation to range requests for zip streams, but i am not knowledgeable enough in the field to ever be sure, i have found some cases of zip and multi-part 7z on usenet too i have also player around a bit with par2 parchive parsing and verification, maybe repairs too i have implemented multi-server support for usenet, so if a chunk is missing a backbone would be used.. there is a lot of refactoring to be done, but RAR5 which seems pretty common is probably the biggest roadblock for Stremio atm — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.