Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update message serialization used by Kafka #141

Open
emetsger opened this issue Oct 23, 2017 · 0 comments
Open

Update message serialization used by Kafka #141

emetsger opened this issue Oct 23, 2017 · 0 comments

Comments

@emetsger
Copy link
Contributor

Messages sent to Kafka are simply byte arrays with some additional metadata. The producers and consumers of messages are responsible for serializing and deserializing the byte arrays according to their needs.

RMap is using the built-in JVM Object serialization. While this is a pragmatic, short-term solution, it is not a long-term viable option. More robust serialization ought to be considered. Some specific options are:

  • RDF (use the native RDF serializations supported by RMap)
  • Apache Avro (provides for robust serialization and schema evolution for messages)
  • XStream (xml, json)
  • ...?
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant