-
Notifications
You must be signed in to change notification settings - Fork 166
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Recommend ZK for Pulsar standalone #871
Conversation
According to the issue it doesn't look to be a RocksDB metadata issue, rather a BK bug |
|
||
You may encounter issues with the default RocksDB metadata store after restarting Pulsar standalone. | ||
|
||
We recommend you consider using the following environment variable to use ZooKeeper as the metadata store: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think it's a good idea to recommend not to use a default setting. If we have a bug in default configuration, we must fix it. If it cannot be fixed, we will have to change the defaults.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It would be the best solution. @lhotari 's concern was that:
it would have to be documented since many might be using old versions and wondering about the resolution.
The problem in changing the default is that it would break existing "deployments" with data
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think it's a good idea to recommend not to use a default setting. If we have a bug in default configuration, we must fix it. If it cannot be fixed, we will have to change the defaults.
@merlimat I agree. Since we cannot change already released versions and users are hitting problems, I think it's worth documenting the workaround until there's a better solution. It seems that there isn't a nice way to solve this problem.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll merge this PR with the current content. We can come back to this later once there's a better resolution.
@merlimat you're right. I removed the mention of the wrong tracking issue. I'm a little confused about the different problems I encountered with Pulsar standalone. The issue with RocksDB I have:
I had an app that concurrently created 2 tenants, a few namespaces, and about 15 topics with roughly 50 partitions in total and created a single producer and consumer for each topic. The problem was fixed after I started to create resources sequentially. I can share a docker image with the app where the issue can be reproduced if it makes sense. |
I made a separate commit 96cdbf2 for fixing the chown |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, although as discussed there isn't a good solution and it's better to have the workaround documented.
✅ Contribution Checklist
chown
command because it didn't work on MacOS