data2: wget cannot verify knb.ecoinformatics.org's certificate but curl does

[mamelara]

When attempting to retrieve the bin tarball from the knb.ecoinformatics.org/software/dist repository on data2.ess-dive.lbl.gov, wget fails with the following command in build.sh:

 # Get the metacat distribution
  if [ ! -f  "$DIR/${ARCHIVE}" ];
  then
      wget http://knb.ecoinformatics.org/software/dist/${ARCHIVE} -O $DIR/${ARCHIVE}

  fi

This results in this output:

INFO: Metacat major:2 minor:16
--2022-01-25 09:48:05--  http://knb.ecoinformatics.org/software/dist/metacat-bin-2.16.2.tar.gz
Resolving knb.ecoinformatics.org (knb.ecoinformatics.org)... 128.111.85.211
Connecting to knb.ecoinformatics.org (knb.ecoinformatics.org)|128.111.85.211|:80... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://knb.ecoinformatics.org/software/dist/metacat-bin-2.16.2.tar.gz [following]
--2022-01-25 09:48:05--  https://knb.ecoinformatics.org/software/dist/metacat-bin-2.16.2.tar.gz
Connecting to knb.ecoinformatics.org (knb.ecoinformatics.org)|128.111.85.211|:443... connected.
ERROR: cannot verify knb.ecoinformatics.org's certificate, issued by '/C=US/O=Let's Encrypt/CN=R3':
  Issued certificate has expired.
To connect to knb.ecoinformatics.org insecurely, use `--no-check-certificate'.

However, the LE cert for knb has not expired and curl validates the cert properly:

curl -v https://knb.ecoinformatics.org/software/dist/metacat-bin-2.16.2.tar.gz -o metacat-bin-2.16.2.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0* About to connect() to knb.ecoinformatics.org port 443 (#0)
*   Trying 128.111.85.211...
* Connected to knb.ecoinformatics.org (128.111.85.211) port 443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0* NSS: client certificate not found (nickname not specified)
* SSL connection using TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256
* Server certificate:
* 	subject: CN=knb.ecoinformatics.org
* 	start date: Jan 15 13:25:49 2022 GMT
* 	expire date: Apr 15 13:25:48 2022 GMT
* 	common name: knb.ecoinformatics.org
* 	issuer: CN=R3,O=Let's Encrypt,C=US
> GET /software/dist/metacat-bin-2.16.2.tar.gz HTTP/1.1
> User-Agent: curl/7.29.0
> Host: knb.ecoinformatics.org
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Tue, 25 Jan 2022 18:47:01 GMT
< Server: Apache/2.4.52 (Ubuntu)
< Last-Modified: Wed, 12 Jan 2022 18:49:43 GMT
< ETag: "1bae765b-5d56708738ce6"
< Accept-Ranges: bytes
< Content-Length: 464418395
< X-Frame-Options: sameorigin
< Access-Control-Allow-Origin:
< Access-Control-Allow-Headers: Authorization, Content-Type, Origin, Cache-Control
< Access-Control-Allow-Methods: GET, POST, PUT, OPTIONS
< Access-Control-Allow-Credentials: true
< Content-Type: application/x-gzip
<
{ [data not shown]
100  442M  100  442M    0     0  63.7M      0  0:00:06  0:00:06 --:--:-- 61.5M
* Connection #0 to host knb.ecoinformatics.org left intact

It may be that the root certificate that wget uses is outdated, so this may need to be updated, or we change the command to use curl instead.

To replicate, run the essdive-appstack build_failover.sh script and it should fail when attempting to retrieve the 2.16.2 tarball.

[mamelara]

Note: This seems to only be the case for data2. The NCEAS VM seems to work fine with wget.