feat: faster ZipCrypto decryption (via Rust)

This decreases the time to decrypt ZipCrypto ZIPs by approximately a factor of
10 (measured using 100MiB of random data zipped).

This uses pyo3 to make Rust code available to Python. However, recent versions
of pyo3 don't support Python 3.6 (and older don't seem to support Python 3.12
onwards), and judging the factor of 10 worth it to drop support. This is why we
dropped Python 3.6 support in an earlier commit.

Co-authored-by: Uka Osim <[email protected]>
Co-authored-by: Michal Charemza <[email protected]>

.github/workflows/deploy-package-to-pypi.yml

@@ -5,29 +5,161 @@ on:
     types: [published]
 
 jobs:
-  build:
+  build-source:
+    name: Build source package
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/checkout@v3
+      - uses: actions/checkout@v4
 
-      - uses: actions/setup-python@v3
+      - name: Update version in pyproject.toml from current git tag
+        run: |
+          sed -i "s/0\\.0\\.0\\.dev0/${GITHUB_REF_NAME}/g" pyproject.toml
+
+      - uses: actions/setup-python@v4
+        with:
+          python-version: 3.13
+
+      - name: Build package
+        run: |
+          pip install build
+          python -m build --sdist
+
+      - uses: actions/upload-artifact@v4
         with:
-          python-version: 3.11
+          name: source
+          path: ./dist
+
+  build-linux:
+    name: Build Linux wheels
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        image:
+          - "manylinux2014_x86_64"
+          - "musllinux_1_1_x86_64"
+          - "manylinux2014_aarch64"
+          - "musllinux_1_1_aarch64"
+        folder:
+          - "cp37-cp37m"
+          - "cp38-cp38"
+          - "cp39-cp39"
+          - "cp310-cp310"
+          - "cp311-cp311"
+          - "cp312-cp312"
+          - "cp313-cp313"
+
+    steps:
+      - uses: actions/checkout@v4
 
       - name: Update version in pyproject.toml from current git tag
+        run: |
+          sed -i "s/0\\.0\\.0\\.dev0/${GITHUB_REF_NAME}/g" pyproject.toml
+
+      - name: Set up QEMU
+        uses: docker/setup-qemu-action@v3
+        with:
+          platforms: arm64
+
+      - name: Build packages
         run: >-
-          sed -i "s/0\\.0\\.0\\.dev0/${GITHUB_REF/refs\/tags\/v/}/g" pyproject.toml
+          docker run --rm -v ${{ github.workspace }}:/app quay.io/pypa/${{ matrix.image }} bash -c '
+            cd /app &&
+            curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y &&
+            . "$HOME/.cargo/env" &&
+            /opt/python/${{ matrix.folder }}/bin/python -m build --wheel
+            auditwheel repair $(ls dist/*.whl) &&
+            rm dist/*.whl &&
+            cp wheelhouse/*.whl dist
+          '
 
-      - run: |
+      - uses: actions/upload-artifact@v4
+        with:
+          name: linux-${{ matrix.image }}-$${{ matrix.folder }}
+          path: ./dist
+
+  build-macos:
+    name: Build macOS wheels
+    strategy:
+      matrix:
+        os:
+          - "macos-12"
+          - "macos-13"
+          - "macos-14"  # ARM
+        python-version:
+          - "3.7.1"
+          - "3.8.10"
+          - "3.9.13"
+          - "3.10.11"
+          - "3.11.9"
+          - "3.12.6"
+          - "3.13.0"
+        exclude:
+          - python-version: "3.7.1"
+            os: "macos-14"
+    runs-on: '${{ matrix.os }}'
+
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v4
+        with:
+          python-version: '${{ matrix.python-version }}'
+
+      - name: Update version in pyproject.toml from current git tag
+        run: |
+          sed -i "" "s/0\\.0\\.0\\.dev0/${GITHUB_REF_NAME}/g" pyproject.toml
+
+      - name: Build package
+        run: |
+          pip install build
+          python -m build --wheel
+
+      - uses: actions/upload-artifact@v4
+        with:
+          name: ${{ matrix.os }}-${{ matrix.python-version }}
+          path: ./dist
+
+  build-windows:
+    name: Build Windows wheels
+    strategy:
+      matrix:
+        os:
+          - "windows-2019"
+        python-version:
+          - "3.7.1"
+          - "3.8.0"
+          - "3.9.0"
+          - "3.10.0"
+          - "3.11.0"
+          - "3.12.0"
+          - "3.13.0"
+    runs-on: '${{ matrix.os }}'
+
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v4
+        with:
+          python-version: '${{ matrix.python-version }}'
+
+      - name: Update version in pyproject.toml from current git tag
+        run: |
+          (Get-Content pyproject.toml).Replace('0.0.0.dev0', $Env:GITHUB_REF_NAME) | Set-Content pyproject.toml
+
+      - name: Build package
+        run: |
           pip install build
-          python -m build
+          python -m build --wheel
 
-      - uses: actions/upload-artifact@v3
+      - uses: actions/upload-artifact@v4
         with:
+          name: ${{ matrix.os }}-${{ matrix.python-version }}
           path: ./dist
 
   deploy:
-    needs: [build]
+    needs:
+      - build-source
+      - build-linux
+      - build-macos
+      - build-windows
     environment:
       name: pypi
       url: https://pypi.org/project/stream-unzip/
@@ -37,9 +169,20 @@ jobs:
     permissions:
       id-token: write
     steps:
-      - uses: actions/download-artifact@v3
+      - uses: actions/download-artifact@v4
+        with:
+          path: ./dist
+
+      # The "merge-multiple" option of download-artifact seems to cause corruption when there are
+      # multiple files of the same name, which happens because in some different macOS versions
+      # make the exact same Python package. So we avoid that and do a manual move of packages
+      # to the top level for upload
+      - name: Move packages to top level
+        run: |
+          find ./dist -mindepth 2 -type f -exec mv -t ./dist -i '{}' +
+          rm -R -- ./dist/*/
 
       - name: Publish package distributions to PyPI
         uses: pypa/gh-action-pypi-publish@release/v1
         with:
-          packages_dir: artifact/
+          packages_dir: ./dist/

Cargo.toml

@@ -0,0 +1,12 @@
+[package]
+name = "stream_unzip_zipcrypto_decrypt"
+version = "0.1.0"
+edition = "2021"
+
+[lib]
+name = "stream_unzip_zipcrypto_decrypt"
+crate-type = ["cdylib"]
+
+[dependencies]
+pyo3 = { version = "0.22.5", features = ["extension-module", "gil-refs"] }
+crc32fast = "1.4.2"

README.md

@@ -21,7 +21,7 @@ In addition to being memory efficient, stream-unzip supports:
 
 - WinZip-style AES-encrypted / password-protected ZIPs. Python's zipfile module cannot open AES-encrypted ZIPs.
 
-- Legacy-encrypted / password-protected ZIP files. This is also known as ZipCrypto/Zip 2.0.
+- Legacy-encrypted / password-protected ZIP files. This is also known as ZipCrypto/Zip 2.0. Decrypting ZipCrypto with stream-zip is approximately 10 times faster than Python's zipfile module.
 
 - ZIP files created by Java's ZipOutputStream that are larger than 4GiB. At the time of writing libarchive-based stream readers cannot read these without error.