Use Mongo as source and sink

[mongodbuser]

I have not been able to get this working with MongoDB as source as well as sink. These servers are not a part of the replica set and there is a need to replay all events as it is.

Questions:

1. I assume scheme registry is not applicable here. Please confirm
2. What is the definition of schema.name - Can any values be used here?

Following are the property files:

Source

name=mongodb-source-connector
connector.class=org.apache.kafka.connect.mongodb.MongodbSourceConnector
tasks.max=1
host=10.6.156.188
port=27017
batch.size=100
topic.prefix=testconn
schema.name=mongodbschema
databases=PocDB.myCollection

Sink

name=mongodb-sink-connector
connector.class=org.apache.kafka.connect.mongodb.MongodbSinkConnector
tasks.max=1
host=10.6.156.189
port=27017
bulk.size=100
mongodb.database=PocDB
mongodb.collections=myCollection
topics=testconn_PocDB_myCollection

Any help in this regard is appreciated.

Thanks

[xuxu0728]

hi,did you run it!When I run the mongodb as source,there is no work.Ａｎｄ the error which zookeeper output like this

INFO Got user-level KeeperException when processing sessionid:0x15ddf911b8a000b type:create cxid:0x2 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/consumers Error:KeeperErrorCode = NodeExists for /consumers (org.apache.zookeeper.server.PrepRequestProcessor)
[2017-08-14 15:13:53,344] INFO Got user-level KeeperException when processing sessionid:0x15ddf911b8a000b type:create cxid:0x1a zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/consumers/console-consumer-46190/owners/test1 Error:KeeperErrorCode = NoNode for /consumers/console-consumer-46190/owners/test1 (org.apache.zookeeper.server.PrepRequestProcessor)
[2017-08-14 15:13:53,347] INFO Got user-level KeeperException when processing sessionid:0x15ddf911b8a000b type:create cxid:0x1b zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/consumers/console-consumer-46190/owners Error:KeeperErrorCode = NoNode for /consumers/console-consumer-46190/owners (org.apache.zookeeper.server.PrepRequestProcessor)
[2017-08-14 15:14:53,203] INFO Got user-level KeeperException when processing sessionid:0x15ddf911b8a000b type:setData cxid:0x26 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/consumers/console-consumer-46190/offsets/test1/0 Error:KeeperErrorCode = NoNode for /consumers/console-consumer-46190/offsets/test1/0 (org.apache.zookeeper.server.PrepRequestProcessor)
[2017-08-14 15:14:53,206] INFO Got user-level KeeperException when processing sessionid:0x15ddf911b8a000b type:create cxid:0x27 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/consumers/console-consumer-46190/offsets Error:KeeperErrorCode = NoNode for /consumers/console-consumer-46190/offsets (org.apache.zookeeper.server.PrepRequestProcessor)
[2017-08-14 15:14:53,214] INFO Got user-level KeeperException when processing sessionid:0x15ddf911b8a000b type:setData cxid:0x2b zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/consumers/console-consumer-46190/offsets/test1/1 Error:KeeperErrorCode = NoNode for /consumers/console-consumer-46190/offsets/test1/1 (org.apache.zookeeper.server.PrepRequestProcessor)
[2017-08-14 15:14:53,216] INFO Got user-level KeeperException when processing sessionid:0x15ddf911b8a000b type:create cxid:0x2c zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/consumers/console-consumer-46190/offsets/test1 Error:KeeperErrorCode = NodeExists for /consumers/console-consumer-46190/offsets/test1 (org.apache.zookeeper.server.PrepRequestProcessor)

Could you help me?

[zoeminghong]

@mongodbuser can you show connect-avro-standalone.properties's code? About the schema.name, you see #19