Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

批量写的建议 #16

Open
michaelmao999 opened this issue Sep 2, 2019 · 2 comments
Open

批量写的建议 #16

michaelmao999 opened this issue Sep 2, 2019 · 2 comments
Labels
question Further information is requested

Comments

@michaelmao999
Copy link

  1. PartitionGroupStoreManager.write 方法可以同时写多条消息,但为什么每处理一条消息后都会唤醒刷新进程一次,而不是把这批消息处理完成后统一唤醒一次。
  2. 当前性能的瓶颈在于频繁的刷新上,而消息分组处理在ProduceManager.writeMessagesAsync, 如果可以把这部分工作后移到PartitionGroupStoreManager.write里面,那么同一分组上可以同时处理的消息会更多,它会减少flush次数,提升系统性能。
@michaelmao999
Copy link
Author

刷新磁盘消耗了太多CPU时间, 当存在很多队列的时候,即使没有任何数据更新,但因为监控线程timeout,也会导致刷新磁盘。LoopThread.run(). 建议实现一个独立的监控线程专门处理flush功能。

@liyue2008
Copy link
Contributor

刷盘时间间隔可以通过配置项store.flush.interval来控制。
默认的时间间隔设置的比较小的原因是为了保证客户端发送消息的时延足够小。

@liyue2008 liyue2008 added the question Further information is requested label Jan 17, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

2 participants