Skip to content

Commit

Permalink
tools/osbuild-mpp: hash mpp-embed urls from stream
Browse files Browse the repository at this point in the history
Currently if one uses `mpp-embed` with URLs, osbuild-mpp still wants to
download the full file just so it can hash it. Make this more efficient
by hashing from the stream instead, which `hashlib` natively supports.

This also makes osbuild-mpp work with large artifacts in environments
that may not have enough space to temporarily save the data.
  • Loading branch information
jlebon authored and supakeen committed Sep 18, 2024
1 parent 5973473 commit ed33869
Showing 1 changed file with 5 additions and 2 deletions.
7 changes: 5 additions & 2 deletions tools/osbuild-mpp
Original file line number Diff line number Diff line change
Expand Up @@ -1608,20 +1608,23 @@ class ManifestFileV2(ManifestFile):
f, _ = self.find_and_open_file(path, [], mode="rb", encoding=None)
with f:
data = f.read()
checksum = hashlib.sha256(data).hexdigest()
elif url:
response = urllib.request.urlopen(url)
data = response.fp.read()
h = hashlib.file_digest(response.fp, 'sha256')
checksum = h.hexdigest()
else:
data = bytes(text, "utf-8")
checksum = hashlib.sha256(data).hexdigest()

checksum = hashlib.sha256(data).hexdigest()
digest = "sha256:" + checksum

if url:
source = element_enter(self.sources, "org.osbuild.curl", {})
items = element_enter(source, "items", {})
items[digest] = url
else:
assert data is not None
encoded = base64.b64encode(data).decode("utf-8")
source = element_enter(self.sources, "org.osbuild.inline", {})
items = element_enter(source, "items", {})
Expand Down

0 comments on commit ed33869

Please sign in to comment.