Support encrypted archives

[tripleee]

As a stopgap measure, I would like for the documentation to be explicit about this limitation.

Adding "support reading encrypted archives" and "support writing encrypted archives" to the TODO list in README.rst would be a good first step towards improving visibility.

vnix$ python nst.py demo.zip
> {'_ArchiveEntryItReadable__is_consumed': False, '_ArchiveEntry__entry_res': c_void_p(34891728), '_ArchiveEntry__reader_res': 34889568, '_ArchiveEntry__is_consumed': False}
[moo.txt] SIZE=(4)
Traceback (most recent call last):
  File "nst.py", line 15, in <module>
    for entry in arch.file_pour(argv[1]):
  File "/home/tripleee/zip/env-libarchive/lib/python2.6/site-packages/libarchive/adapters/archive_read.py", line 384, in _pour
    "Pour failed: (%d) [%s]" % (r, message))
libarchive.exception.ArchiveError: Pour failed: (-25) [Encrypted file is unsupported]

[tripleee]

#8 includes an attempt to add the stopgap measure to document the absence of crypto functionality.

[dsoprea]

You're more then welcome to investigate and contribute information on how to implement encryption. This might encourage me to actually work on it (I don't, myself, require encryption). You might also open your own pull-request if you'd like to attempt to implement it yourself (with my support, of course).

As is often the case with many adapter libraries, it shouldn't be considered a flaw if not all functionality is supported. It wasn't my goal to be comprehensive. Rather, my goal was to get the ball rolling on this being a community effort and to contribute things as I had personal need for them or the community voiced an overwhelming need for.

We all have our priorities, and I'm sure you'll understand if I reserve my effort for things to which I can make the most impact for people.

[tripleee]

Definitely understood. My situation is that I am evaluating my options, and the lack of encryption support is a showstopper for my current task. I cannot commit to developing this particular library at this point in time (looks like for now I'll just have to use 7z in a subprocess to solve my immediate problem) but I am definitely interested in seeing this move forward, and would like to be able to work on this particular area if I can get management approval etc.

In the meantime, I appreciate that you obviously have your own set of priorities. Filing a bug so that somebody can take the ball on an issue is, I think, an important step towards building a community around this code (and perhaps also detach it somewhat from your person, and your priorities).

Now, on the pull request I created, is this an acceptable contribution, or would you like to see something done differently? I have a couple of additional changes in the pipeline but I wanted to fly a test balloon your way before investing more time.

[dsoprea]

Yes, that's fine. I can merge after I do a cursory overview in a few
moments.

I'd like to frame a solution for you, but calling 7Z directly might be
prudent if you need something instantly.

Are you principally interested in 7Z encrypted reads, writes, or both? Have
you checked whether this is supported by libarchive, in general?

Dustin
On Jul 3, 2015 2:30 AM, "tripleee" [email protected] wrote:

Definitely understood. My situation is that I am evaluating my options,
and the lack of encryption support is a showstopper for my current task. I
cannot commit to developing this particular library at this point in time
(looks like for now I'll just have to use 7z in a subprocess to solve my
immediate problem) but I am definitely interested in seeing this move
forward, and would like to be able to work on this particular area if I can
get management approval etc.

In the meantime, I appreciate that you obviously have your own set of
priorities. Filing a bug so that somebody can take the ball on an issue
is, I think, an important step towards building a community around this
code (and perhaps also detach it somewhat from your person, and your
priorities).

Now, on the pull request I created, is this an acceptable contribution, or
would you like to see something done differently? I have a couple of
additional changes in the pipeline but I wanted to fly a test balloon your
way before investing more time.

—
Reply to this email directly or view it on GitHub
#6 (comment)
.

[tripleee]

Are you principally interested in 7Z encrypted reads, writes, or both?

I'm going to need both.

Have you checked whether this is supported by libarchive, in general?

Good question! A cursory review of the documentation offers no insights so I'm guessing maybe it isn't.

[dsoprea]

It doesn't seem supported, which is both too bad and ironic since I originally started this project because I had a similar need to yours: libarchive was the most general solution to support 7Z.

I inquired: libarchive/libarchive#579

[dsoprea]

They couldn't have wanted to help less. I looked into it, and it looks like libarchive is really only useful for writing/reading encrypted zip files: libarchive/libarchive#579

zip is probably not an option for you, assuming you actually want solid encryption.

Sorry for the inconvenience.

[dsoprea]

Though I went through a related investigation before I started PyEasyArchive to determine how to manage 7Z files from Python and only ended-up with libarchive, I did another check just now. There still appears to be virtually no library support for reading 7Z (unencrypted, at all). The 7z tool build-process only produces a bunch of object-files and a handful of executables.

It seems like there's no practical interest in supporting encryption directly in archives. I've recently become a committed fan of XZ (which is LZMA/LZMA 2, and essentially a simpler version of 7Z with an interface similar to GZ and BZ). It's a fairly new implementation, I believe, and, even there, there's zero support for encryption. Unlike 7Z, it does properly support Unix file-ownership, though. Also unlike 7Z, it's natively supported in Python in the lzma package as of 3.3 .

If you have any choice in the matter, you might consider using XZ in a TAR -> COMPRESS -> ENCRYPT process. It would greatly simplify your efforts and remove your dependency on what looks to be a decreasingly-popular format (keeping in mind that I have loved 7Z and, until recently, have been using it nearly exclusively for nine-years). You could use the tarfile package to produce a TAR file, push it through the LZMA package to render an XZ, and then encrypt (naming it something like "xze"). Because of the order of the steps, above, you can also pipe the output of the decryption directly into TAR (using 'xJ') to decompress by hand.

Note that you can also use PyEasyArchive for the TAR-XZ creation, but I'd always recommend the native Python support first.

I'm going to close this issue unless there's something we missed.

[tripleee]

Thanks for looking into this. Your results already clarify my options. Tar is not an realistic alternative, but maybe zip is.