Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Interpret data descriptors when reading zip file from (read, nonseek) stream #197

Open
wants to merge 23 commits into
base: master
Choose a base branch
from

Commits on Jun 21, 2024

  1. Added the method fn read_zipfile_from_seekablestream<S: Read + Seek>(…

    …reader: &mut S) that reads the next zipfile entry from a stream by potential parsing the data descriptor
    
    This is an alternative method to read a zip file. If possible, use the ZipArchive functions as some information will be missing when reading this manner.
    
    This method extends the existing read_zipfile_from_stream method when the stream is seekable.
    
    This method is superior to ZipArchive in the special case that the underlying stream implementation must buffer all seen/read data to provide seek back support and the memory consumption must be kept small. This could be the case when reading a ZipFile B.zip nested within a zip file A.zip, that is stored on disk. Since A is seekable when stored as file, A.zip can be read using ZipArchive. Be B a zip file stored in A, then we can read B's content using a decompressing reader. The problem is that the decompressing reader will not be seekable (due to decompression). When one would like to still read contents of B.zip without extracting A to disk, the file B.zip must be buffered in RAM. When using ZipArchive to read B.zip from RAM, the whole B.zip file must be buffered to RAM because the central directory of B.zip is located at the end of the file B.zip. This method will read B.zip from the start of the file and returning the first file entry found in B.zip. After the execution of this function and the ZipFile return value is dropped, the reader will be positioned at the end of the file entry. Since this function will never seek back to before the initial position of the stream when the function was called, the underlying stream implementation may discard, after dropping ZipFile, all buffered data before the current position of the stream. Summarizing: In given scenario, this method must not buffer the whole B.zip file to RAM, but only the first file entry.
    0xCCF4 committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    93a7f69 View commit details
    Browse the repository at this point in the history
  2. Merge branch 'refs/heads/master' into feature-read-from-seekable-stream

    # Conflicts:
    #	src/read.rs
    0xCCF4 committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    d146ff3 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    7f0d07b View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    59f1327 View commit details
    Browse the repository at this point in the history

Commits on Jun 22, 2024

  1. Added security risk documentation and untrusted value struct to encap…

    …sulate potential unsafe data
    0xCCF4 committed Jun 22, 2024
    Configuration menu
    Copy the full SHA
    3bef659 View commit details
    Browse the repository at this point in the history

Commits on Jun 23, 2024

  1. Library does not require Take<Read> anymore but instead accepts a tem…

    …plate T: Read in internal data structures
    
    Added UntrustedValue and MaybeUntrusted data types
    0xCCF4 committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    f6b5da9 View commit details
    Browse the repository at this point in the history
  2. Merge branch 'refs/heads/master' into feature-read-from-seekable-stream

    # Conflicts:
    #	src/read.rs
    0xCCF4 committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    e36ad61 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    47f718b View commit details
    Browse the repository at this point in the history

Commits on Jun 24, 2024

  1. CRC32 checksum is now late propagated

    Before: The CRC32 checksum had to be supplied before the stream is read. Though checked when EOF occurred.
    Now: The CRC32 checksum can be supplied before starting to read or after finishing reading from the stream.
    0xCCF4 committed Jun 24, 2024
    Configuration menu
    Copy the full SHA
    bf7a030 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    3265477 View commit details
    Browse the repository at this point in the history
  3. Run cargo fmt --all

    0xCCF4 committed Jun 24, 2024
    Configuration menu
    Copy the full SHA
    1d3afa9 View commit details
    Browse the repository at this point in the history

Commits on Jul 6, 2024

  1. Configuration menu
    Copy the full SHA
    ad3dbc0 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    fc83a70 View commit details
    Browse the repository at this point in the history

Commits on Jul 8, 2024

  1. Configuration menu
    Copy the full SHA
    6cc20f5 View commit details
    Browse the repository at this point in the history
  2. Merge remote-tracking branch 'origin/feature-read-from-seekable-strea…

    …m' into feature-read-from-seekable-stream
    0xCCF4 committed Jul 8, 2024
    Configuration menu
    Copy the full SHA
    c61a683 View commit details
    Browse the repository at this point in the history

Commits on Jul 15, 2024

  1. Configuration menu
    Copy the full SHA
    f0cb9f2 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    1ecad02 View commit details
    Browse the repository at this point in the history
  3. perf: use vecdequeue for look ahead buffer

    run cargo fmt & clippy
    0xCCF4 committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    549d1da View commit details
    Browse the repository at this point in the history
  4. Merge branch 'master' into feature-read-from-seekable-stream

    Signed-off-by: Chris Hennick <[email protected]>
    Pr0methean authored Jul 15, 2024
    Configuration menu
    Copy the full SHA
    7cd7937 View commit details
    Browse the repository at this point in the history

Commits on Jul 19, 2024

  1. Merge branch 'master' into feature-read-from-seekable-stream

    Signed-off-by: Chris Hennick <[email protected]>
    Pr0methean authored Jul 19, 2024
    Configuration menu
    Copy the full SHA
    a530f52 View commit details
    Browse the repository at this point in the history
  2. fix: FixedSizeBlock must extend Pod

    Signed-off-by: Chris Hennick <[email protected]>
    Pr0methean authored Jul 19, 2024
    Configuration menu
    Copy the full SHA
    6022136 View commit details
    Browse the repository at this point in the history

Commits on Aug 3, 2024

  1. Configuration menu
    Copy the full SHA
    a9b7bc6 View commit details
    Browse the repository at this point in the history

Commits on Aug 13, 2024

  1. Configuration menu
    Copy the full SHA
    ecf5588 View commit details
    Browse the repository at this point in the history