-
Notifications
You must be signed in to change notification settings - Fork 27
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Mirrormaker 2 updates #25
Comments
Overall, I think there solution is just removing the Jackson 1.x classes, which I believe is what the Avro + Registry upgrade did. I'm not sold that the Kafka client version needs changing, but doesn't hurt, I suppose |
Thanks for the feedback. Actually, this is error I received while running current version on CP 5.5.0 and why I made this update:
In recent versions, schema registry dumped org.codehaus.jackson dependency, I don't think removing jackson 1.x (com.fasterxml.jackson package ) would solve this problem. I guess I could include org.codehaus.jackson in jar, but better solution seemed to update dependencies... |
Therefore the move to FasterXML makes sense that a class would not be found. The alternative solution to your PR would be to have shaded codehaus (Jackson 1.x) along with Avro 1.8.x ... And I have been seeing there is backwards compatibility issues around Avro 1.9.0 ... Not sure if those are resolved in 1.9.2 or not |
Yes, there was some backward compatibility between avro 1.9.0. and 1.8.x but they are fixed from version 1.9.1: |
Hi, like iturcino I was looking for exactly the same: using MM2 on confluent 5.5.0 cp. Initially after building the jar file from the master branch I encountered the same java.lang.NoClassDefFoundError: org/codehaus/jackson/JsonParseException error. After building from the pr/26 branch it seems to be running fine. So thanks both! I upvote fro merging this to master. |
Hi @gerardq, sorry, been busy with personal things, but to answer your question, yes, it's expected because the schema text is ran through an MD5 hash, which checks for uniqueness within a single registry. Same hash = same ID. That being said, I'm not sure I understand the use case of replicating within the same cluster when all you really need is a new consumer group from the same topic |
Thanks for quick answer. I was just testing in a single cluster for convenience. I'll do some testing with another cluster but currently it is under construction. |
You may find my blog post helpful for that |
Thanks, I first found your blog first and then second the code on github. |
I'm working on a project where we need to implement data mirroring with separate schema registry cluster per Kafka cluster, so came across your great Confluent blog and this repo. This is exactly what we need, we are using Mirrormaker 2 for this.
When I added this SMT to MM2 connector (with CP v.5.5.0, Kafka v.2.5.0) it didn't work, I got exception that is already documented here confluentinc/schema-registry#1222
So I updated dependencies, updated code and got it working on Mirror Maker 2
I can make a PR for this, what do you think?
Also, right after this, I'll make changes to #2 since this is needed also in my project, that's going to be another PR :)
The text was updated successfully, but these errors were encountered: