Commits on May 15, 2024

1. Implemented fetch and pull modes for the main application …

   MySQL connection configuration structure ('easymysql::connection_config')
   extended with three additional parameters:
   * 'connect_timeout' ('MYSQL_OPT_CONNECT_TIMEOUT')
   * 'read_timeout' ('MYSQL_OPT_READ_TIMEOUT')
   * 'write_timeout' ('MYSQL_OPT_WRITE_TIMEOUT')
   that are used to set corresponding MySQL client options via 'mysql_options()'.
   'main_config.json' sample configuration file and config file generation in
   'binlog_streaming.binsrv' MTR test case updated correspondingly.
   
   Main application extended with one more command line argument (the very first
   one) called ''<operation_mode>' with two allowed values ('fetch' and 'pull').
   
   In 'fetch' mode the application now connects to the remote MySQL server and
   reads all events from all binary logs till the very last one
   (till 'mysql_binlog_fetch()' returns an empty buffer) and exits. The replication
   channel is opened in non-blocking mode.
   
   In 'pull' mode the application also reads all available events till
   the very last one but instead of exiting immediately continues to wait
   for other events to come later. The replication channel is opened in blocking
   mode. In order to be able to gracefully terminate this process, we use
   'read_timeout' configuration parameter ('MYSQL_OPT_READ_TIMEOUT' MySQL client
   option) after which blocking 'mysql_binlog_fetch()' call will terminate with
   a timeout error. After this, the replication connection is closed and program
   enters the 'idle' state doing nothing. Currently the idle time is hardcoded to
   5 seconds. When this 'idle' time passes, the program attempts to re-establish
   MySQL server connection and continue replication. Not being able to do so, will
   not terminate the program. Instead, after another 'idle' period the next attempt
   to re-establish connection will be made. This model will allow us to react to
   termination requests (most probably to posix 'SIGINT' / 'SIGTERM' signals) not
   only in the binlog event processing loop but during the 'idle' time as well.
   
   Introduced 'binsrv::operation_mode_type' smart enum with two values ('fetch'
   and 'pull') that is used to specify main application operation mode.
   
   'easymysql::connection' now includes all the functionality available before in
   the 'easymysql::binlog' class. Established connection can now be switched to
   'replication' mode. 'easymysql::binlog' class removed completely.
   
   The method that switches connection to replication mode
   ('easymysql::connection::switch_to_replication()') accepts an extra parameter
   that specifies blocking / non-blocking replication mode.
   
   Added explicit 'flush()' calls at the end of 'do_write_data_to_stream()' for
   both 'binsrv::filesystem_storage_backend' and 'binsrv::s3_storage_backend' to
   make sure that buffered data is immediately written to the disk.
   
   'mysql-server' project GitHub links in the comments updated to version '8.0.37'.

   

   percona-ysorokin committed May 15, 2024
   Configuration menu

   • View commit details
   • Copy full SHA for a47fdc9
   • Browse repository at this point

   Copy the full SHA

   a47fdc9 View commit details

   Browse the repository at this point in the history