[source-mongodb] SSL error when syncing MongoDb ReplicaSet without SSL/TLS enabled #48431
Labels
area/connectors
Connector related issues
community
connectors/source/mongodb
team/db-dw-sources
Backlog for Database and Data Warehouse Sources team
type/bug
Something isn't working
Comments
heruscode
added
area/connectors
marcosmarxm
added
team/db-dw-sources
|
@airbytehq/dbsources can someone take a look into this question? |
|
Airbyte enforces SSL, its hardcoded, new version of connector will allow to use custom cert, release Q1.2025 |
@kamil-lada will it be possible to disable SSL? I don't think a custom cert would work for me because my ReplicaSet doesn't even has SSL enabled |
|
@kamil-lada we have the same issue at our company. We need to sync data from a self-managed cluster that doesn't have TLS enabled. Would be nice if the connector would allow to disable TLS. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Connector Name
source-mongodb
Connector Version
1.5.12
What step the error happened?
During the sync
Relevant information
Airbyte version: 1.1.0
Connector version: 1.5.12
Hi! I am trying to sync a Self-Managed MongoDB Replica Set that do not have SSL/TLS enabled --> BigQuery
The MongoDB cluster is a 3 node ReplicaSet and I am trying to sync from primary node. There is no primary/secondary rotation events happening when I am doing the sync, it always error out.
I see that it is being able to extract the records(it shows "10 loaded" in the connection status) and when I go to BigQuery I also confirm that the records were correctly copied.
But after a few minutes the Sync fails with a SSL/TLS error making the Connection status to be "Failed Sync".
Is SSL/TLS still being enforced somehow?
Relevant log output
2024-11-08 20:28:52 source > INFO pool-2-thread-1 c.m.i.d.l.SLF4JLogger(info):71 MongoClient with metadata {"driver": {"name": "mongo-java-driver|sync", "version": "4.11.0"}, "os": {"type": "Linux", "name": "Linux", "architecture": "amd64", "version": "6.1.75+"}, "platform": "Java/Amazon.com Inc./21.0.5+11-LTS"} created with settings MongoClientSettings{readPreference=primary, writeConcern=WriteConcern{w=null, wTimeout=null ms, journal=null}, retryWrites=true, retryReads=true, readConcern=ReadConcern{level=null}, credential=MongoCredential{mechanism=null, userName='REDACTED-airby', source='admin', password=<hidden>, mechanismProperties=<hidden>}, transportSettings=null, streamFactoryFactory=null, commandListeners=[], codecRegistry=ProvidersCodecRegistry{codecProviders=[ValueCodecProvider{}, BsonValueCodecProvider{}, DBRefCodecProvider{}, DBObjectCodecProvider{}, DocumentCodecProvider{}, CollectionCodecProvider{}, IterableCodecProvider{}, MapCodecProvider{}, GeoJsonCodecProvider{}, GridFSFileCodecProvider{}, Jsr310CodecProvider{}, JsonObjectCodecProvider{}, BsonCodecProvider{}, EnumCodecProvider{}, com.mongodb.client.model.mql.ExpressionCodecProvider@67ec8477, com.mongodb.Jep395RecordCodecProvider@25d958c6, com.mongodb.KotlinCodecProvider@4482469c]}, loggerSettings=LoggerSettings{maxDocumentLength=1000}, clusterSettings={hosts=[cowboy5.mongo.REDACTED.com:27017], srvServiceName=mongodb, mode=SINGLE, requiredClusterType=UNKNOWN, requiredReplicaSetName='null', serverSelector='null', clusterListeners='[]', serverSelectionTimeout='30000 ms', localThreshold='15 ms'}, socketSettings=SocketSettings{connectTimeoutMS=10000, readTimeoutMS=0, receiveBufferSize=0, proxySettings=ProxySettings{host=null, port=null, username=null, password=null}}, heartbeatSocketSettings=SocketSettings{connectTimeoutMS=10000, readTimeoutMS=10000, receiveBufferSize=0, proxySettings=ProxySettings{host=null, port=null, username=null, password=null}}, connectionPoolSettings=ConnectionPoolSettings{maxSize=100, minSize=0, maxWaitTimeMS=120000, maxConnectionLifeTimeMS=0, maxConnectionIdleTimeMS=0, maintenanceInitialDelayMS=0, maintenanceFrequencyMS=60000, connectionPoolListeners=[], maxConnecting=2}, serverSettings=ServerSettings{heartbeatFrequencyMS=10000, minHeartbeatFrequencyMS=500, serverListeners='[]', serverMonitorListeners='[]'}, sslSettings=SslSettings{enabled=true, invalidHostNameAllowed=false, context=javax.net.ssl.SSLContext@40611548}, applicationName='null', compressorList=[], uuidRepresentation=STANDARD, serverApi=null, autoEncryptionSettings=null, dnsClient=null, inetAddressResolver=null, contextProvider=null} 2024-11-08 20:28:52 source > INFO pool-2-thread-1 c.m.i.d.l.SLF4JLogger(info):71 No server chosen by ReadPreferenceServerSelector{readPreference=primary} from cluster description ClusterDescription{type=UNKNOWN, connectionMode=SINGLE, serverDescriptions=[ServerDescription{address=cowboy5.mongo.REDACTED.com:27017, type=UNKNOWN, state=CONNECTING}]}. Waiting for 30000 ms before timing out 2024-11-08 20:28:52 source > INFO cluster-ClusterId{value='672e74845512566e79267c8e', description='null'}-cowboy5.mongo.REDACTED.com:27017 c.m.i.d.l.SLF4JLogger(info):76 Exception in monitor thread while connecting to server cowboy5.mongo.REDACTED.com:27017 com.mongodb.MongoSocketWriteException: Exception sending message at com.mongodb.internal.connection.InternalStreamConnection.translateWriteException(InternalStreamConnection.java:714) ~[mongodb-driver-core-4.11.0.jar:?] at com.mongodb.internal.connection.InternalStreamConnection.sendMessage(InternalStreamConnection.java:587) ~[mongodb-driver-core-4.11.0.jar:?] at com.mongodb.internal.connection.InternalStreamConnection.sendCommandMessage(InternalStreamConnection.java:419) ~[mongodb-driver-core-4.11.0.jar:?] at com.mongodb.internal.connection.InternalStreamConnection.sendAndReceive(InternalStreamConnection.java:364) ~[mongodb-driver-core-4.11.0.jar:?] at com.mongodb.internal.connection.CommandHelper.sendAndReceive(CommandHelper.java:102) ~[mongodb-driver-core-4.11.0.jar:?] at com.mongodb.internal.connection.CommandHelper.executeCommand(CommandHelper.java:49) ~[mongodb-driver-core-4.11.0.jar:?] at com.mongodb.internal.connection.InternalStreamConnectionInitializer.initializeConnectionDescription(InternalStreamConnectionInitializer.java:130) ~[mongodb-driver-core-4.11.0.jar:?] at com.mongodb.internal.connection.InternalStreamConnectionInitializer.startHandshake(InternalStreamConnectionInitializer.java:78) ~[mongodb-driver-core-4.11.0.jar:?] at com.mongodb.internal.connection.InternalStreamConnection.open(InternalStreamConnection.java:213) ~[mongodb-driver-core-4.11.0.jar:?] at com.mongodb.internal.connection.DefaultServerMonitor$ServerMonitorRunnable.lookupServerDescription(DefaultServerMonitor.java:196) ~[mongodb-driver-core-4.11.0.jar:?] at com.mongodb.internal.connection.DefaultServerMonitor$ServerMonitorRunnable.run(DefaultServerMonitor.java:156) ~[mongodb-driver-core-4.11.0.jar:?] at java.base/java.lang.Thread.run(Thread.java:1583) [?:?] Caused by: javax.net.ssl.SSLHandshakeException: Remote host terminated the handshake at java.base/sun.security.ssl.SSLSocketImpl.handleEOF(SSLSocketImpl.java:1714) ~[?:?] at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1514) ~[?:?] at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1421) ~[?:?] at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:455) ~[?:?] at java.base/sun.security.ssl.SSLSocketImpl.ensureNegotiated(SSLSocketImpl.java:922) ~[?:?] at java.base/sun.security.ssl.SSLSocketImpl$AppOutputStream.write(SSLSocketImpl.java:1291) ~[?:?] at com.mongodb.internal.connection.SocketStream.write(SocketStream.java:164) ~[mongodb-driver-core-4.11.0.jar:?] at com.mongodb.internal.connection.InternalStreamConnection.sendMessage(InternalStreamConnection.java:584) ~[mongodb-driver-core-4.11.0.jar:?] ... 10 more Suppressed: java.net.SocketException: Broken pipe at java.base/sun.nio.ch.SocketDispatcher.write0(Native Method) ~[?:?] at java.base/sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:62) ~[?:?] at java.base/sun.nio.ch.NioSocketImpl.tryWrite(NioSocketImpl.java:394) ~[?:?] at java.base/sun.nio.ch.NioSocketImpl.implWrite(NioSocketImpl.java:410) ~[?:?] at java.base/sun.nio.ch.NioSocketImpl.write(NioSocketImpl.java:440) ~[?:?] at java.base/sun.nio.ch.NioSocketImpl$2.write(NioSocketImpl.java:819) ~[?:?] at java.base/java.net.Socket$SocketOutputStream.write(Socket.java:1195) ~[?:?] at java.base/sun.security.ssl.SSLSocketOutputRecord.encodeAlert(SSLSocketOutputRecord.java:81) ~[?:?] at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:419) ~[?:?] at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:326) ~[?:?] at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:467) ~[?:?] at java.base/sun.security.ssl.SSLSocketImpl.ensureNegotiated(SSLSocketImpl.java:922) ~[?:?] at java.base/sun.security.ssl.SSLSocketImpl$AppOutputStream.write(SSLSocketImpl.java:1291) ~[?:?] at com.mongodb.internal.connection.SocketStream.write(SocketStream.java:164) ~[mongodb-driver-core-4.11.0.jar:?] at com.mongodb.internal.connection.InternalStreamConnection.sendMessage(InternalStreamConnection.java:584) ~[mongodb-driver-core-4.11.0.jar:?] at com.mongodb.internal.connection.InternalStreamConnection.sendCommandMessage(InternalStreamConnection.java:419) ~[mongodb-driver-core-4.11.0.jar:?] at com.mongodb.internal.connection.InternalStreamConnection.sendAndReceive(InternalStreamConnection.java:364) ~[mongodb-driver-core-4.11.0.jar:?] at com.mongodb.internal.connection.CommandHelper.sendAndReceive(CommandHelper.java:102) ~[mongodb-driver-core-4.11.0.jar:?] at com.mongodb.internal.connection.CommandHelper.executeCommand(CommandHelper.java:49) ~[mongodb-driver-core-4.11.0.jar:?] at com.mongodb.internal.connection.InternalStreamConnectionInitializer.initializeConnectionDescription(InternalStreamConnectionInitializer.java:130) ~[mongodb-driver-core-4.11.0.jar:?] at com.mongodb.internal.connection.InternalStreamConnectionInitializer.startHandshake(InternalStreamConnectionInitializer.java:78) ~[mongodb-driver-core-4.11.0.jar:?] at com.mongodb.internal.connection.InternalStreamConnection.open(InternalStreamConnection.java:213) ~[mongodb-driver-core-4.11.0.jar:?] at com.mongodb.internal.connection.DefaultServerMonitor$ServerMonitorRunnable.lookupServerDescription(DefaultServerMonitor.java:196) ~[mongodb-driver-core-4.11.0.jar:?] at com.mongodb.internal.connection.DefaultServerMonitor$ServerMonitorRunnable.run(DefaultServerMonitor.java:156) ~[mongodb-driver-core-4.11.0.jar:?] at java.base/java.lang.Thread.run(Thread.java:1583) [?:?] Caused by: java.io.EOFException: SSL peer shut down incorrectly at java.base/sun.security.ssl.SSLSocketInputRecord.read(SSLSocketInputRecord.java:494) ~[?:?] at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(SSLSocketInputRecord.java:483) ~[?:?] at java.base/sun.security.ssl.SSLSocketInputRecord.decode(SSLSocketInputRecord.java:160) ~[?:?] at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:111) ~[?:?] at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1506) ~[?:?] at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1421) ~[?:?] at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:455) ~[?:?] at java.base/sun.security.ssl.SSLSocketImpl.ensureNegotiated(SSLSocketImpl.java:922) ~[?:?] at java.base/sun.security.ssl.SSLSocketImpl$AppOutputStream.write(SSLSocketImpl.java:1291) ~[?:?] at com.mongodb.internal.connection.SocketStream.write(SocketStream.java:164) ~[mongodb-driver-core-4.11.0.jar:?] at com.mongodb.internal.connection.InternalStreamConnection.sendMessage(InternalStreamConnection.java:584) ~[mongodb-driver-core-4.11.0.jar:?] ... 10 more Stack Trace: com.mongodb.MongoSocketWriteException: Exception sending message at com.mongodb.internal.connection.InternalStreamConnection.translateWriteException(InternalStreamConnection.java:714) at com.mongodb.internal.connection.InternalStreamConnection.sendMessage(InternalStreamConnection.java:587) at com.mongodb.internal.connection.InternalStreamConnection.sendCommandMessage(InternalStreamConnection.java:419) at com.mongodb.internal.connection.InternalStreamConnection.sendAndReceive(InternalStreamConnection.java:364) at com.mongodb.internal.connection.CommandHelper.sendAndReceive(CommandHelper.java:102) at com.mongodb.internal.connection.CommandHelper.executeCommand(CommandHelper.java:49) at com.mongodb.internal.connection.InternalStreamConnectionInitializer.initializeConnectionDescription(InternalStreamConnectionInitializer.java:130) at com.mongodb.internal.connection.InternalStreamConnectionInitializer.startHandshake(InternalStreamConnectionInitializer.java:78) at com.mongodb.internal.connection.InternalStreamConnection.open(InternalStreamConnection.java:213) at com.mongodb.internal.connection.DefaultServerMonitor$ServerMonitorRunnable.lookupServerDescription(DefaultServerMonitor.java:196) at com.mongodb.internal.connection.DefaultServerMonitor$ServerMonitorRunnable.run(DefaultServerMonitor.java:156) at java.base/java.lang.Thread.run(Thread.java:1583) Caused by: javax.net.ssl.SSLHandshakeException: Remote host terminated the handshake at java.base/sun.security.ssl.SSLSocketImpl.handleEOF(SSLSocketImpl.java:1714) at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1514) at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1421) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:455) at java.base/sun.security.ssl.SSLSocketImpl.ensureNegotiated(SSLSocketImpl.java:922) at java.base/sun.security.ssl.SSLSocketImpl$AppOutputStream.write(SSLSocketImpl.java:1291) at com.mongodb.internal.connection.SocketStream.write(SocketStream.java:164) at com.mongodb.internal.connection.InternalStreamConnection.sendMessage(InternalStreamConnection.java:584) ... 10 more Suppressed: java.net.SocketException: Broken pipe at java.base/sun.nio.ch.SocketDispatcher.write0(Native Method) at java.base/sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:62) at java.base/sun.nio.ch.NioSocketImpl.tryWrite(NioSocketImpl.java:394) at java.base/sun.nio.ch.NioSocketImpl.implWrite(NioSocketImpl.java:410) at java.base/sun.nio.ch.NioSocketImpl.write(NioSocketImpl.java:440) at java.base/sun.nio.ch.NioSocketImpl$2.write(NioSocketImpl.java:819) at java.base/java.net.Socket$SocketOutputStream.write(Socket.java:1195) at java.base/sun.security.ssl.SSLSocketOutputRecord.encodeAlert(SSLSocketOutputRecord.java:81) at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:419) at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:326) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:467) ... 14 more Caused by: java.io.EOFException: SSL peer shut down incorrectly at java.base/sun.security.ssl.SSLSocketInputRecord.read(SSLSocketInputRecord.java:494) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(SSLSocketInputRecord.java:483) at java.base/sun.security.ssl.SSLSocketInputRecord.decode(SSLSocketInputRecord.java:160) at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:111) at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1506) ... 16 moreContribute
The text was updated successfully, but these errors were encountered: