Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

May I ask whether BinLogStreamReader can control the detection time of binlog events? For example, it is June 29 today. I want to monitor events from 6.1, can it be realized by parameter skip_to_timestamp #404

Open
legend-xu opened this issue Jun 29, 2023 · 8 comments

Comments

@legend-xu
Copy link

May I ask whether BinLogStreamReader can control the detection time of binlog events? For example, it is June 29 today. I want to monitor events from 6.1, can it be realized by parameter skip_to_timestamp,look forward to your reply

@nkr-404-NotFound
Copy link

试试

    skip_to_timestamp = (datetime.datetime.now() - datetime.timedelta(minutes = 1)).timestamp()

你想退多久 delta 里面就设置多久

@nkr-404-NotFound
Copy link

我们只退 1分钟 =。=

@legend-xu
Copy link
Author

@nankingrider 你好,我按照这样写法试了,但是只对当天有效果,我试着用skip_to_timestamp = (datetime.datetime.now() - datetime.timedelta(days= 30)).timestamp(),依然是从当天最早的时间开始监测,请问是什么原因,我的BinLogStreamReader配置如下:
stream = BinLogStreamReader(
connection_settings=mysql_settings,
server_id=1024,
blocking=True, # 没有变更时阻塞
# resume_stream=True, # 重启后从最新的位置开始, 和skip_to_timestamp不同时使用
only_events=[WriteRowsEvent, UpdateRowsEvent], # 增改
only_schemas=schema_list, # 只查看当前schema
only_tables=table_list, # 只查看当前表
skip_to_timestamp=(datetime.datetime.now() - datetime.timedelta(days=30)).timestamp(),
# 这里传入时间戳(10位)
# log_file=start_file, # 指定binlog文件
# log_pos=end_file, # 指定
# fail_on_table_metadata_unavailable=True,
)

@nkr-404-NotFound
Copy link

如果我没看错的话, 他只能从 current binlog 开始

@nkr-404-NotFound
Copy link

image

或许你们的 master db 每天都会 rotate 一个新的 binlog 。。。 所以你总是只能从当天的binlog开始获取

@nkr-404-NotFound
Copy link

或许你可以传入 binlog file name 和 binlog pos。

@legend-xu
Copy link
Author

@nankingrider 好的,非常感谢

@dongwook-chan
Copy link
Collaborator

@legend-xu
Thank you for your question. My team and I will certainly look into this matter. Our initial approach will be to examine all events starting from the first file (instead of current file) within the available binary log files and apply a filter based on the date/time parameter that you have supplied. We will continue to keep you updated as we progress in addressing this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants