Remote Clone Flow

The current version is at https://github.com/facebook/mysql-5.6/wiki/Remote-Clone-Flow

In remote clone, client and donor are separate instances communicating through network.

The clone plugin defines and uses a RPC protocol with several COM_ commands. The storage engine interface does not deal with them directly, the map to the same handlerton API as the local clone which does not use the RPC protocol.

Starting the Clone

1. The client plugin establishes a regular MySQL protocol connection to the donor, then sends COM_CLONE MySQL protocol command to switch to the clone protocol.
2. The client plugin calls clone_apply_begin(HA_CLONE_MODE_VERSION). The storage engine creates a locator that does not point to any snapshot but contains version negotiation information instead.
3. The client plugin sends COM_INIT to the donor with the locator payload.
4. The donor plugin calls clone_begin(HA_CLONE_MODE_START). It checks the version information in the passed version locator. Then it creates a snapshot and prepares an actual clone locator. All subsequent flow uses this locator.
5. The donor plugin sends COM_RES_LOCS with the locators as the response to the previous COM_INIT.
6. The client plugin calls clone_apply_begin(HA_CLONE_MODE_START).

Data Copy

1. The client plugin sends COM_EXECUTE to the donor which asks to stream the data.
2. The donor plugin calls clone_copy (same as the local clone), whose output is then serialized into two responses:
   1. COM_RES_DATA_DESC for metadata
   2. COM_RES_DATA for the data itself
3. The client plugin passes the two response payload to clone_apply, which is then handled the same as for the local clone, with the additional requirement that any application of the memory data is acknowledge by sending COM_ACK to the donor.

Finishing the Clone

1. After donor completes sending all the data, the sends a COM_RES_COMPLETE packet.
2. The client plugin calls clone_apply_end.
3. The client plugin sends COM_EXIT to the donor.
4. The donor calls clone_end.

Multiple Threads

New threads are spawned by the plugin client-side. Each thread sends a COM_ATTACH command to the donor, which makes the donor to start a new thread on the donor and call clone_init(HA_CLONE_MODE_ATTACH) there. Then the threads go through the above sequence.

Restarts

To initiate a restart, the client prepares a restart locator that has both the snapshot ID and the application state which describes what exactly has been applied in this clone session so far, and sends it to the donor with a COM_REINIT command. To send the command, a new MySQL protocol session is created and COM_CLONE is issued to switch the session to the clone mode.

In-place Provisioning

The client drops any data in the case of an existing instance, and copies all the InnoDB non-system-schema data right into its final location. The system schema data is copied to files with a .clone suffix (i.e. mysql.ibd.clone), which are renamed into place upon server restart. The restart is performed by the clone itself.