MongoDB: cannot connect to MongoDB: server selection error: server selection timeout

[harishkumarrajasekaran]

I am getting socket issue in the new version 0.40.0, I was able to resolve the TLS issue by configuring the --web.config file, but the connection to the MongoDB deployment is showing error in Prometheus logs attached below. However, I can perform the connectivity through the same certificates without any issues locally. Any help would be appreciated. Thanks.

Percona prometheus version used: 0.40.0
MongoDB version: 6.0.6 Community Edition

LOG:
#########################
level=info ts=2024-01-22T12:14:54.307Z caller=tls_config.go:274 msg="Listening on" address=[::]:9216
level=info ts=2024-01-22T12:14:54.308Z caller=tls_config.go:310 msg="TLS is enabled." http2=false address=[::]:9216

time="2024-01-22T12:14:59Z" level=error msg="Cannot connect to MongoDB: cannot connect to MongoDB: server selection error: server selection timeout, current topology: { Type: Unknown, Servers: [{ Addr: xxx.svc.cluster.local:27017, Type: Unknown, Last error: connection(xxxx..svc.cluster.local:27017[-30]) socket was unexpectedly closed: EOF }, ] }"
##############################

Configurations:

extraArgs:

• --compatible-mode
• --collector.dbstats
• --collector.collstats-limit=0
• --collector.topmetrics
• --web.config=/tmp/webconfig.yaml
• --mongodb.uri=mongodb://user:pass@/admin?tls=true&tlsCertificateKeyFile=/tmp/tls-combined.pem&tlsCAFile=/tmp/ca.crt
• --log.level=debug
• --mongodb.direct-connect=false

web-config:

tls_server_config:
cert_file: "/tmp/tls-combined.pem"
key_file: "/tmp/tls.key"
client_auth_type: "RequireAndVerifyClientCert"
client_ca_file: "/tmp/ca.crt"
http_server_config:
http2: false

Originally posted by @harishkumarrajasekaran in #685 (comment)

[l0rdaxel]

I have a similar error, could you show me how you configured your mongodb_exporter.service? I am unable to activate TLS.

Thanks!

[winem]

We ran into the same error with the exporter in version 0.40. Can you please try to run the exporter in version 0.39.0 and see if that works? There is already a PR that addresses the issue with 0.40.0 but unfortunately it's not yet been released. Please see: #780

@adnull also provides a container that has the fix already implemented. I'd love to see a new release of this exporter to prevent others from running into this issue, too.

Is there anything that can be done to get a new release? I could not find any timeline or roadmap.

[adnull]

Hi. My docker version contains changes in multiple hosts resolving, i don't think it fixes TLS connection problem, but if it works we can compare what was changed.

[BupycHuk]

Hello @winem, we are going to release 0.41.0 in the middle of june.

[Joschi84]

Hello, I have the same problem.

The service configuration:
[Unit]
Description=MongoDB Exporter

[Service]
Type=simple
Restart=always
ExecStart=/usr/local/bin/mongodb_exporter
--mongodb.uri=mongodb://user:[email protected]:27017,url2.com:27017,url3.com:27017/admin?authSource=admin&replicaSet=replicaName&tls=true&tlsCAFile=/etc/mongodb/tls/monCA.crt&tlsCertificateKeyFile=/etc/mongodb/tls/monCl.pem
--mongodb.direct-connect=false

[Install]
WantedBy=multi-user.target

I checked with mongosh and connection was possible. Is the problem still a thing or I am doing something wrong?

Errors:
"MESSAGE" : "time="2024-10-23T13:28:42+02:00" level=error msg="Registry - Cannot get node type to check if this is a mongos : cannot get mongo node type from an empty client"
"MESSAGE" : "time="2024-10-23T13:30:12+02:00" level=error msg="Cannot connect to MongoDB: cannot connect to MongoDB: server selection error: server selection timeout, current topology: { Type: Unknown, Servers: [{ Addr: url1.com:27017, Type: Unknown, Last error: connection(url1:27017[-878]) socket was unexpectedly closed: EOF: connection(url1.com:27017[-878]) socket was unexpectedly closed: EOF }, { Addr: url2.com:27017, Type: Unknown, Last error: connection(url2:27017[-879]) socket was unexpectedly closed: EOF: connection(url2.com:27017[-879]) socket was unexpectedly closed: EOF }, { Addr: url3.com:27017, Type: Unknown, Last error: connection(url3.com:27017[-877]) socket was unexpectedly closed: EOF: connection(url3.com:27017[-877]) socket was unexpectedly closed: EOF }, ] }""

[adnull]

@Joschi84 do you use the latest release of the mongodb_exporter ( e.g. >= 0.41.0 ) ?

[Joschi84]

@adnull Wow you are fast. Yes I am using 0.41.2. Installed it today.

[adnull]

No idea yet. Could you show us the connection string and the options you've used to check it with the mongosh?

[adnull]

Also, what is the version of the server software and mongosh, that info would be useful to find a problem

[Joschi84]

Sure.
For mongodb-exporter configuration I already shared the connection string (made it a little bit more accurate):

--mongodb.uri=mongodb://user:[email protected]:27017,subdom2.url.com:27017,subdom3.url.com:27017/admin?authSource=admin&replicaSet=replicaName&tls=true&tlsCAFile=/etc/mongodb/tls/monCA.crt&tlsCertificateKeyFile=/etc/mongodb/tls/monCl.pem

and with mongosh I connect exactly like this, but successfull:

mongosh "mongodb://user:[email protected]:27017,subdom2.url.com:27017,subdom3.url.com:27017/admin?authSource=admin&replicaSet=replicaName&tls=true&tlsCAFile=/etc/mongodb/tls/monCA.crt&tlsCertificateKeyFile=/etc/mongodb/tls/monCl.pem"

subdom1.url.com:27017,subdom2.url.com:27017 and subdom3.url.com:27017 are on different servers. mongodb-exporter lives on the same server as prometheus. Any ideas? Do you have a working configuration with a replica set and required authentication?
Should I use a older version of mongod-exporter? Is there a save choice?

mongosh version 2.3.2
What do you mean with server software. I use Ubuntu 22.04.5 LTS on the server. Ahh and no docker or other container software

[Joschi84]

I changed things and reversed changes (around "admin?" - I wanted skip db choice because we want also see local db, correct?). So I am back at the start point but the error changed. I am sorry but maybe I skipped reload daemon on the way.

The error is
"MESSAGE" : "time="2024-10-23T17:13:29+02:00" level=error msg="Cannot connect to MongoDB: cannot connect to MongoDB: server selection error: server selection timeout, current topology: { Type: ReplicaSetNoPrimary, Servers: [{ Addr: subdomain1.url.com:27017, Type: Unknown, Last error: tls: failed to verify certificate: x509: certificate relies on legacy Common Name field, use SANs instead }, { Addr: subdomain2.url.com:27017, Type: Unknown, Last error: tls: failed to verify certificate: x509: certificate relies on legacy Common Name field, use SANs instead }, { Addr: subdomain3.url.com:27017, Type: Unknown, Last error: tls: failed to verify certificate: x509: certificate relies on legacy Common Name field, use SANs instead }, ] }""

So I probaly made a mistake at the creation of the certificates (used not the most upToDate method). So maybe this makes more sense for someone.

mongosh works with "mongodb://user:[email protected]:27017,subdom2.url.com:27017,subdom3.url.com:27017/admin?authSource=admin&replicaSet=replicaName&tls=true&tlsCAFile=/etc/mongodb/tls/monCA.crt&tlsCertificateKeyFile=/etc/mongodb/tls/monCl.pem"

and

mongodb.uri=mongodb://user:[email protected]:27017,subdom2.url.com:27017,subdom3.url.com:27017/admin?authSource=admin&replicaSet=replicaName&tls=true&tlsCAFile=/etc/mongodb/tls/monCA.crt&tlsCertificateKeyFile=/etc/mongodb/tls/monCl.pem

brings the above error. So mongosh seems to be more tolerant regarding this szenario.

I am sorry for the confusion.

[adnull]

@Joschi84 maybe to try using "&tlsInsecure=true" in the connection string? The problem seems that you don't have SANs strings in your server certificates.

[adnull]

Also this topic could be useful https://www.reddit.com/r/golang/comments/oqnm8n/x509_certificate_relies_on_legacy_common_name/

[Joschi84]

@adnull Thank you. Especially for your time. I now understand a little what the problem is. Your link is greatly appreciated.