Cannot download DB: process hangs on ~30Mb; error message: PROTOCOL_ERROR; received from peer #2382

this-username-has-been-taken · 2025-01-20T15:32:25Z

Hello! I was trying to update a vulnerability database and faced with the error: ERROR unable to update vulnerability database: unable to update vulnerability database: unable to download db: stream error: stream ID 1; PROTOCOL_ERROR; received from peer. This error occurs every time I try to update the DB and it started approximately a week ago.

What happened:
When trying to update the vulnerability database process hangs on 27-30Mb and after that the following error occurs:
[0061] ERROR unable to update vulnerability database: unable to update vulnerability database: unable to download db: stream error: stream ID 1; PROTOCOL_ERROR; received from peer.
However I can download the archive file manually - it loads without any issues.

Full log output:

root@somwhere:~# grype --version
grype 0.86.1
root@somwhere:~# grype db delete
Vulnerability database deleted
root@somwhere:~# grype db check
Updated DB version 5 was built on 2025-01-20 10:09:29 +0000 UTC
Updated DB URL: https://grype.anchore.io/databases/vulnerability-db_v5_2025-01-20T10:09:29Z_1737371089.tar.gz
You can run 'grype db update' to update to the latest db
root@somwhere:~# grype db update
 ✔ Vulnerability DB                [30 MB / 208 MB]
[0061] ERROR unable to update vulnerability database: unable to update vulnerability database: unable to download db: stream error: stream ID 1; PROTOCOL_ERROR; received from peer
root@somwhere:~# wget https://grype.anchore.io/databases/vulnerability-db_v5_2025-01-20T10:09:29Z_1737371089.tar.gz
--2025-01-20 18:25:30--  https://grype.anchore.io/databases/vulnerability-db_v5_2025-01-20T10:09:29Z_1737371089.tar.gz
Resolving grype.anchore.io (grype.anchore.io)... 188.114.99.224, 188.114.98.224, 2a06:98c1:3122:e000::, ...
Connecting to grype.anchore.io (grype.anchore.io)|188.114.99.224|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 208086502 (198M) [application/x-tar]
Saving to: ‘vulnerability-db_v5_2025-01-20T10:09:29Z_1737371089.tar.gz’

vulnerability-db_v5_2025-01-20T10:0 100%[=================================================================>] 198.45M  38.3MB/s    in 5.4s

2025-01-20 18:25:36 (36.5 MB/s) - ‘vulnerability-db_v5_2025-01-20T10:09:29Z_1737371089.tar.gz’ saved [208086502/208086502]

What you expected to happen:
I expect the vulnerability DB to be downloaded and updated without any errors.

How to reproduce it (as minimally and precisely as possible):

Clear the the vulnerability DB: grype db delete.
Run the update command: grype db update.
Observe the issue after approximately a minute.

Environment:

Output of grype version: 0.86.1
OS (e.g: cat /etc/os-release or similar):

PRETTY_NAME="Debian GNU/Linux 12 (bookworm)"
NAME="Debian GNU/Linux"
VERSION_ID="12"
VERSION="12 (bookworm)"
VERSION_CODENAME=bookworm
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"

The text was updated successfully, but these errors were encountered:

popey · 2025-01-20T15:52:12Z

Hi. Sorry to hear you're having trouble downloading the vulnerability database with grype.

I cannot reproduce on my Debian server in Germany, running multiple times. It's possible you're hitting a different CDN endpoint to me, however.

$ grype --version
grype 0.86.1
$ grype db delete
Vulnerability database deleted
$ grype db check
Updated DB version 5 was built on 2025-01-20 10:09:29 +0000 UTC
Updated DB URL: https://grype.anchore.io/databases/vulnerability-db_v5_2025-01-20T10:09:29Z_1737371089.tar.gz
You can run 'grype db update' to update to the latest db
$grype db update
 ✔ Vulnerability DB                [updated]
Vulnerability database updated to latest version!

Are you behind any kind of challenging firewall or proxy?

I appreciate that the wget worked, and grype did not. Is this a transient issue, or is it repeatedly unable to download?

this-username-has-been-taken · 2025-01-20T16:40:04Z

Thank you for the quick response!

The situations is getting a bit weird.
I have tried grype db update on 3 PCs: two of them are virtual Debian servers running on Proxmox and the third is my notebook running Ubuntu in the WSL (under Windows) mode:

VM 1: routed via VPN (IP from Sweden): got error.
VM 2: routed directly via ISP: got error.
Ubuntu WSL: routed via the same VPN (IP from Sweden): DB updated without any issues.

Looks like the error occurs only with Debian VMs. They are running behind an OPNSense gateway. There are no proxies in the network: VM -> Gateway (also VM) -> Proxmox -> physical adapter -> ISP.

When I get the error, I get it everytime: it is unconditional.

popey · 2025-01-20T17:09:52Z

I have seen this inside a Windows VM (qemu) on top of Ubuntu on a laptop in my office in the UK, with no weird networking. But that is when the wifi at the office is playing up. Now, when the network is not sluggish, I get no problem.

I just tested inside a proxmox VM on top of my debian box, which is about as close to your environment as I can get at the moment. It took about 10 seconds, but most of that was the unpack after the download.

time grype -vvv db update
[0000]  INFO grype version: 0.86.1
[0000] DEBUG config:
  log:
      quiet: false
      level: trace
      file: ""
  dev:
      profile: none
  db:
      cache-dir: /home/alan/.cache/grype/db
      update-url: https://toolbox-data.anchore.io/grype/databases/listing.json
      ca-cert: ""
      auto-update: true
      validate-by-hash-on-start: false
      validate-age: true
      max-allowed-built-age: 120h0m0s
      require-update-check: true
      update-available-timeout: 30s
      update-download-timeout: 5m0s
      max-update-check-frequency: 2h0m0s
  exp:
      dbv6: false
[0000] TRACE no max-frequency set for update check
[0000] DEBUG checking for available database updates
[0000] DEBUG found database update candidate: Listing(url=https://grype.anchore.io/databases/vulnerability-db_v5_2025-01-20T10:09:29Z_1737371089.tar.gz)
[0000] DEBUG cannot find existing metadata, using update...
[0000] DEBUG database update available: Listing(url=https://grype.anchore.io/databases/vulnerability-db_v5_2025-01-20T10:09:29Z_1737371089.tar.gz)
[0000]  INFO downloading new vulnerability DB
[0010]  INFO downloaded new vulnerability DB version=5 built="2025-01-20 10:09:29 +0000 UTC"
[0010] DEBUG completed db update check with result: Vulnerability database updated to latest version!

[0010] TRACE worker stopped component=eventloop
[0010] TRACE signal exit component=eventloop
Vulnerability database updated to latest version!

real    0m10.161s
user    0m5.172s
sys     0m2.715s

When things are super bad on our infrastructure/cdn, we typically get more shouting. I'm trying not to diminish the issue you're seeing, just pointing out we haven't had a lot of people shouting. Let's see if we get any more reports.

this-username-has-been-taken · 2025-01-20T18:42:00Z

I have also tried Grype for Windows running on my notebook, which is routed directly via ISP (the same IP and DNS as VM 2): everything works fine.
So far it looks like only Proxmox + Debian variant is messed up. It doesn't seem that there are any issues with CDN, Geo blocks or anything related to the delivery part. It looks like the problem exists with the VMs only, but I have no idea what's wrong: all other connections / software works fine.
I will try dig a bit further.
BTW, is there any other, more verbose output that grype db update -vvv. That one doesn't provide any useful information:

root@somwhere:~# grype db update -vvv
[0000]  INFO grype version: 0.86.1
[0000] DEBUG config:
  log:
      quiet: false
      level: trace
      file: ""
  dev:
      profile: none
  db:
      cache-dir: /root/.cache/grype/db
      update-url: https://toolbox-data.anchore.io/grype/databases/listing.json
      ca-cert: ""
      auto-update: true
      validate-by-hash-on-start: false
      validate-age: true
      max-allowed-built-age: 120h0m0s
      require-update-check: true
      update-available-timeout: 30s
      update-download-timeout: 5m0s
      max-update-check-frequency: 2h0m0s
  exp:
      dbv6: false
[0000] TRACE no max-frequency set for update check
[0000] DEBUG checking for available database updates
[0000] DEBUG found database update candidate: Listing(url=https://grype.anchore.io/databases/vulnerability-db_v5_2025-01-20T10:09:29Z_1737371089.tar.gz)
[0000] DEBUG cannot find existing metadata, using update...
[0000] DEBUG database update available: Listing(url=https://grype.anchore.io/databases/vulnerability-db_v5_2025-01-20T10:09:29Z_1737371089.tar.gz)
[0000]  INFO downloading new vulnerability DB
[0061] TRACE worker stopped component=eventloop
[0061] TRACE signal exit component=eventloop
[0061] ERROR unable to update vulnerability database: unable to update vulnerability database: unable to download db: stream error: stream ID 1; PROTOCOL_ERROR; received from peer

popey · 2025-01-20T19:07:45Z

BTW, is there any other, more verbose output that grype db update -vvv. That one doesn't provide any useful information:

Not that I am aware of, other than using external tools.

As a last resort, maybe use wireshark on the host, and monitor the network of the guest VM, and see what it's doing?

this-username-has-been-taken · 2025-01-22T15:12:53Z

The issue was with OPNSense gateway.
After I rebooted it Grype updated its DB without any problems.

@popey , thank you very much for your help and assistance.

this-username-has-been-taken added the bug Something isn't working label Jan 20, 2025

anchoretoolsops added this to OSS Jan 20, 2025

this-username-has-been-taken changed the title ~~CAnnot download DB: process hangs on ~30%; error message: PROTOCOL_ERROR; received from peer~~ Cannot download DB: process hangs on ~30%; error message: PROTOCOL_ERROR; received from peer Jan 20, 2025

this-username-has-been-taken changed the title ~~Cannot download DB: process hangs on ~30%; error message: PROTOCOL_ERROR; received from peer~~ Cannot download DB: process hangs on ~30Mb; error message: PROTOCOL_ERROR; received from peer Jan 20, 2025

this-username-has-been-taken closed this as completed Jan 22, 2025

github-project-automation bot moved this to Done in OSS Jan 22, 2025

spiffcs added the changelog-ignore Don't include this issue in the release changelog label Jan 22, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Cannot download DB: process hangs on ~30Mb; error message: PROTOCOL_ERROR; received from peer #2382

Cannot download DB: process hangs on ~30Mb; error message: PROTOCOL_ERROR; received from peer #2382

this-username-has-been-taken commented Jan 20, 2025 •

edited

Loading

popey commented Jan 20, 2025

this-username-has-been-taken commented Jan 20, 2025 •

edited

Loading

popey commented Jan 20, 2025

this-username-has-been-taken commented Jan 20, 2025

popey commented Jan 20, 2025

this-username-has-been-taken commented Jan 22, 2025

Cannot download DB: process hangs on ~30Mb; error message: PROTOCOL_ERROR; received from peer #2382

Cannot download DB: process hangs on ~30Mb; error message: PROTOCOL_ERROR; received from peer #2382

Comments

this-username-has-been-taken commented Jan 20, 2025 • edited Loading

popey commented Jan 20, 2025

this-username-has-been-taken commented Jan 20, 2025 • edited Loading

popey commented Jan 20, 2025

this-username-has-been-taken commented Jan 20, 2025

popey commented Jan 20, 2025

this-username-has-been-taken commented Jan 22, 2025

this-username-has-been-taken commented Jan 20, 2025 •

edited

Loading

this-username-has-been-taken commented Jan 20, 2025 •

edited

Loading