Expand MySQL documentation (#210)

* Expanded MySQL Connection documentation
Added section for MySQL schema change handling

* Moved MySQL standard connection guide to an accordion

* Link to the required binlog configuration in the MySQL Aurora connection setup

* Added MySQL to SQLite type mappings

* Further tweaks to MySQL docs

* Updated note for binary types in MySQL
Changed MySQL Alpha -> Beta

Author: Rentacookie

architecture/powersync-protocol.mdx

@@ -67,4 +67,4 @@ Write checkpoints are used to ensure clients have synced their own changes back
 
 Creating a write checkpoint is a separate operation, which is performed by the client after all data has been uploaded. It is important that this happens after the data has been written to the backend source database.
 
-The server then keeps track of the current CDC stream position on the database (LSN in Postgres, resume token in MongoDB, or binlog position in MySQL), and notifies the client when the data has been replicated, as part of checkpoint data in the normal data stream.
+The server then keeps track of the current CDC stream position on the database (LSN in Postgres, resume token in MongoDB, or GTID + Binlog Position in MySQL), and notifies the client when the data has been replicated, as part of checkpoint data in the normal data stream.

images/installation/aurora-create-parameter-group.png

@@ -162,15 +162,50 @@ Also see:
 - [MongoDB Atlas Device Sync Migration Guide](/migration-guides/mongodb-atlas)
 - [MongoDB Setup](/installation/database-setup#mongodb)
 
-## <Icon icon="dolphin" iconType="solid" size={32}/> MySQL (Alpha) Specifics
+## <Icon icon="dolphin" iconType="solid" size={32}/> MySQL (Beta) Specifics
 
-1. Fill in your connection details from MySQL:
-   1. "**Name**" can be any name for the connection.
-   2. "**Host**" and "**Database name**" is the database to replicate.
-   3. "**Username**" and "**Password**" maps to your database user.
-2. Click **"Test Connection"** and fix any errors. If have any issues connecting, reach out to our support engineers on our [Discord server](https://discord.gg/powersync) or otherwise [contact us](/resources/contact-us).
-   1. Make sure that your database allows access to PowerSync's IPs — see [Security and IP Filtering](/installation/database-setup/security-and-ip-filtering)
-3. Click **"Save".**
+Select your MySQL hosting provider for steps to connect your newly-created PowerSync instance to your MySQL database:
+<AccordionGroup>
+<Accordion title="AWS Aurora">
+To enable binary logging and GTID replication in AWS Aurora, you need to create a [DB Parameter Group](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/AuroraMySQL.Reference.ParameterGroups.html)
+and configure it with the necessary parameters. Follow these steps:
+
+1. Navigate to [Amazon RDS console](https://console.aws.amazon.com/rds/)
+   In the navigation pane, choose Parameter groups and click on `Create Parameter Group`.
+   <Frame>
+       <img src="/images/installation/aurora-create-parameter-group.png" />
+   </Frame>
+2. Add all the required [binlog configuration](/installation/database-setup#binlog-configuration) parameters. For example:
+   <Frame>
+       <img src="/images/installation/aurora-set-gtid-flags.png" />
+   </Frame>
+3. Associate your newly created parameter group with your Aurora cluster:
+   1. In the navigation pane, choose Databases.
+   2. Select your Aurora cluster.
+   3. Choose Modify.
+   4. In the DB Parameter Group section, select the parameter group you created.
+   5. Click Continue and then Apply immediately.
+4. Whitelist PowerSync's IPs in your Aurora cluster's security group to allow access. See [Security and IP Filtering](/installation/database-setup/security-and-ip-filtering) for more details.
+5.
+</Accordion>
 
-PowerSync deploys and configures an isolated cloud environment for you, which can take a few minutes to complete.
+For other providers and self-hosted databases:
+<Accordion title="Other / Self-hosted">
+Fill in your MySQL connection details:
+    1. "**Name**", "**Host**", "**Port**", "**Database name**", "**Username**", "**Password**" are required.
+    2. "**Name**" can be any name for the connection.
+    3. "**Host**" the endpoint for your database.
+    4. "**Database name**" is the default database to replicate. Additional databases are derived by qualifying the tables in the sync rules.
+    5. "**Username**" and "**Password**" maps to your database user.
+    6. If you want to query your source database via the PowerSync Dashboard, enable "**Allow querying data from the dashboard?**".
+    7. Click **"Test Connection"** and fix any errors.
+    8. Click **"Save".**
+
+    PowerSync deploys and configures an isolated cloud environment for you, which can take a few minutes to complete.
+</Accordion>
+</AccordionGroup>
+
+<Info>
+    Make sure that your MySQL database allows access to PowerSync's IPs — see [Security and IP Filtering](/installation/database-setup/security-and-ip-filtering)
+</Info>
 

images/installation/aurora-set-gtid-flags.png

@@ -4,7 +4,7 @@ description: "Configure your backend database for PowerSync, including permissio
 sidebarTitle: "Overview"
 ---
 
-Jump to: [Postgres](#postgres) | [MongoDB](#mongodb) | [MySQL](#mysql-alpha)
+Jump to: [Postgres](#postgres) | [MongoDB](#mongodb) | [MySQL](#mysql-beta)
 
 import PostgresPowerSyncUser from '/snippets/postgres-powersync-user.mdx';
 import PostgresPowerSyncPublication from '/snippets/postgres-powersync-publication.mdx';
@@ -307,40 +307,77 @@ If you need to use private endpoints with MongoDB Atlas, see [Private Endpoints]
 
 For more information on migrating from Atlas Device Sync to PowerSync, see our [migration guide](/migration-guides/mongodb-atlas).
 
-## <Icon icon="dolphin" iconType="solid" size={32} /> MySQL (Alpha)
-
-<Info>
-  This section is a work in progress. More details for MySQL connections are coming soon. In the meantime, ask on our [Discord server](https://discord.gg/powersync) if you have any questions.
-</Info>
+## <Icon icon="dolphin" iconType="solid" size={32} /> MySQL (Beta)
 
 <Check>
   **Version compatibility**: PowerSync requires MySQL version 5.7 or greater.
 </Check>
 
-MySQL connections use the [binary log](https://dev.mysql.com/doc/refman/8.4/en/binary-log.html) to replicate changes.
+PowerSync reads from the MySQL [binary log](https://dev.mysql.com/doc/refman/8.4/en/binary-log.html) to replicate changes. We use a modified version of the [Zongji MySQL](https://github.com/powersync-ja/powersync-mysql-zongji) binlog listener to achieve this.
 
-Generally, this requires the following config:
+### Binlog Configuration
 
-- `gtid_mode` : `ON`
-- `enforce_gtid_consistency` : `ON`
-- `binlog_format` : `ROW`
+To ensure that PowerSync can read the binary log, you need to configure your MySQL server to enable binary logging and configure it with the following server command options:
+- [server_id](https://dev.mysql.com/doc/refman/8.4/en/replication-options.html#sysvar_server_id): Uniquely identifies the MySQL server instance in the replication topology. Default value is **1**.
+- [log_bin](https://dev.mysql.com/doc/refman/8.4/en/replication-options-binary-log.html#sysvar_log_bin): **ON**. Enables binary logging. Default is **ON** for MySQL 8.0 and later, but **OFF** for MySQL 5.7.
+- [enforce_gtid_consistency](https://dev.mysql.com/doc/refman/8.4/en/replication-options-gtids.html#sysvar_enforce_gtid_consistency): **ON**. Enforces GTID consistency. Default is **OFF**.
+- [gtid_mode](https://dev.mysql.com/doc/refman/8.4/en/replication-options-gtids.html#sysvar_gtid_mode): **ON**. Enables GTID based logging. Default is **OFF**.
+- [binlog_format](https://dev.mysql.com/doc/refman/8.4/en/replication-options-binary-log.html#sysvar_binlog_format): **ROW**. Sets the binary log format to row-based replication. This is required for PowerSync to correctly replicate changes. Default is **ROW**.
 
-PowerSync also requires a user with replication permissions on the database. An example:
+These can be specified in a MySQL [option file](https://dev.mysql.com/doc/refman/8.4/en/option-files.html):
+```
+server_id=<Unique Integer Value>
+log_bin=ON
+enforce_gtid_consistency=ON
+gtid_mode=ON
+binlog_format=ROW
+```
+
+### Database User Configuration
+PowerSync also requires a MySQL user with **REPLICATION** and **SELECT** permission on the source databases. These can be added by running the following SQL commands:
 
 ```sql
 -- Create a user with necessary privileges
-CREATE USER 'repl_user'@'%' IDENTIFIED BY 'good_password';
+CREATE USER 'repl_user'@'%' IDENTIFIED BY '<password>';
 
 -- Grant replication client privilege
 GRANT REPLICATION SLAVE, REPLICATION CLIENT ON *.* TO 'repl_user'@'%';
 
--- Grant access to the specific database
-GRANT ALL PRIVILEGES ON powersync.* TO 'repl_user'@'%';
+-- Grant select access to the specific database
+GRANT SELECT ON <source_database>.* TO 'repl_user'@'%';
 
 -- Apply changes
 FLUSH PRIVILEGES;
 ```
 
+It is possible to constrain the MySQL user further and limit access to specific tables. Care should be taken to ensure that all the tables in the sync rules are included in the grants.
+```sql
+-- Grant select to the users and the invoices tables in the source database
+GRANT SELECT ON <source_database>.users TO 'repl_user'@'%';
+GRANT SELECT ON <source_database>.invoices TO 'repl_user'@'%';
+
+-- Apply changes
+FLUSH PRIVILEGES;
+```
+
+### Additional Configuration (optional)
+#### Binlog
+
+The binlog can be configured to limit logging to specific databases. By default, events for tables in all the databases on the MySQL server will be logged.
+- [binlog-do-db](https://dev.mysql.com/doc/refman/8.4/en/replication-options-binary-log.html#option_mysqld_binlog-do-db): Only updates for tables in the specified database will be logged.
+- [binlog-ignore-db](https://dev.mysql.com/doc/refman/8.4/en/replication-options-binary-log.html#option_mysqld_binlog-ignore-db): No updates for tables in the specified database will be logged.
+
+Examples:
+```
+# Only row events for tables in the user_db and invoices_db databases will appear in the binlog.
+binlog-do-db=user_db
+binlog-do-db=invoices_db
+```
+```
+# Row events for tables in the user_db will be ignored. Events for any other database will be logged.
+binlog-ignore-db=user_db
+```
+
 ## Next Step
 
 Next, connect PowerSync to your database:

installation/database-connection.mdx

@@ -3,7 +3,7 @@ title: "Quickstart Guide / Installation Overview"
 sidebarTitle: "Quickstart / Overview"
 ---
 
-PowerSync is designed to be stack agnostic, and currently supports [Postgres](/installation/database-setup#postgres), [MongoDB](/installation/database-setup#mongodb) and [MySQL](/installation/database-setup#mysql-alpha) (alpha) as the backend source database, and has the following official client-side SDKs available:
+PowerSync is designed to be stack agnostic, and currently supports [Postgres](/installation/database-setup#postgres), [MongoDB](/installation/database-setup#mongodb) and [MySQL](/installation/database-setup#mysql-beta) (Beta) as the backend source database, and has the following official client-side SDKs available:
 - [**Flutter**](/client-sdk-references/flutter) (mobile and [web](/client-sdk-references/flutter/flutter-web-support)) 
 - [**React Native**](/client-sdk-references/react-native-and-expo) (mobile and [web](/client-sdk-references/react-native-and-expo/react-native-web-support)) 
 - [**JavaScript Web**](/client-sdk-references/javascript-web) (including integrations for React & Vue) 

installation/database-setup.mdx

@@ -22,7 +22,7 @@ PowerSync is designed to be backend database agnostic, and currently supports:
 
   <Card title="MongoDB" icon="leaf" href="/installation/database-setup/#mongodb" />
 
-  <Card title="MySQL (Alpha)" icon="dolphin" href="/installation/database-setup/#mysql-alpha" />
+  <Card title="MySQL (Beta)" icon="dolphin" href="/installation/database-setup/#mysql-beta" />
 </CardGroup>
 
 ### Supported Client SDKs

installation/quickstart-guide.mdx

@@ -45,7 +45,7 @@ Below is a summary of the current main PowerSync features and their release stat
 | **Category / Item**                                | **Status**      |
 | -------------------------------------------------- | -------------- |
 | **Database Connectors**                              |                |
-| MySQL                                               | Alpha          |
+| MySQL                                               | Beta          |
 | MongoDB                                             | V1             |
 | Postgres                                            | V1             |
 |                                                    |                |

intro/powersync-overview.mdx

@@ -149,11 +149,75 @@ Renaming an unsynced collection to a name that is included in the Sync Rules tri
 Circular renames (e.g., renaming `todos` → `todos_old` → `todos`) are not directly supported. To reprocess the database after such changes, a Sync Rules update must be deployed.
 
 
-## <Icon icon="dolphin" iconType="solid" size={32} /> MySQL (Alpha) Specifics
+## <Icon icon="dolphin" iconType="solid" size={32} /> MySQL (Beta) Specifics
+PowerSync keeps the [sync buckets](/usage/sync-rules/organize-data-into-buckets) up to date with any incremental data changes as recorded in the MySQL [binary log](https://dev.mysql.com/doc/refman/8.4/en/binary-log.html).
+The binary log also provides DDL (Data Definition Language) query updates, which include:
 
-<Info>
-  This section is a work in progress. More details for MySQL connections are coming soon. In the meantime, ask on our [Discord server](https://discord.gg/powersync) if you have any questions.
-</Info>
+1. Creating, dropping or renaming tables.
+
+2. Truncating tables. (Not technically a schema change, but they appear in the query updates regardless.)
+
+3. Changing replica identity of a table. (Creation, deletion or modification of primary keys, unique indexes, etc.)
+
+4. Adding, dropping, renaming or changing the types of columns.
+
+For MySQL, PowerSync detects schema changes by parsing the DDL queries in the binary log. It may not always be possible to parse the DDL queries correctly, especially if they are complex or use non-standard syntax.
+In such cases, PowerSync will ignore the schema change, but will log a warning with the schema change query. If required, the schema change would then need to be manually
+handled by redeploying the sync rules. This triggers a re-replication.
+
+### MySQL schema changes affecting Sync Rules
+
+#### DROP table
+
+PowerSync will detect when a table is dropped, and automatically remove the data from the sync buckets.
+
+#### CREATE table
+
+Table creation is detected and handled the first time row events for the new table appear on the binary log.
+
+#### TRUNCATE table
+
+PowerSync will detect truncate statements in the binary log, and consequently remove all data from the sync buckets for that table.
+
+#### RENAME table
+
+A renamed table is handled similarly to dropping the old table, and then creating a new table with existing data under the new name.
+This may be a slow operation if the table is large, since the "new" table has to be re-replicated. Replication will be blocked until the new table is replicated.
+
+#### Change REPLICA IDENTITY
+
+The replica identity of a table is considered to be changed if either:
+
+1. The type of replica identity changes (`DEFAULT`, `INDEX`, `FULL`, `NOTHING`).
+
+2. The name or type of columns which form part of the replica identity changes.
+
+The latter can happen if:
+
+1. Using `REPLICA IDENTITY FULL`, and any column is added, removed, renamed, or the type changed.
+
+2. Using `REPLICA IDENTITY DEFAULT`, and the type of any column in the primary key is changed.
+
+3. Using `REPLICA IDENTITY INDEX`, and the type of any column in the replica index is changed.
+
+4. The primary key or replica index is removed or changed.
+
+When the replication identity changes, the entire table is replicated again. This may be a slow operation if the table is large, and all other replication will be blocked until the table is replicated again.
+
+Sync rules affected by schema changes will fail "soft" — an alert would be generated, but the system will continue processing changes.
+
+#### Column changes
+
+Column changes such as adding, dropping, renaming columns, or changing column types, are detected by PowerSync but will generally not result in re-replication. (Unless the replica identity was affected as described above).
+
+Adding a column with a `NULL` default value will generally not cause issues. Existing records will have a missing value instead of `NULL` value, but those are generally treated the same on the client.
+
+Adding a column with a different default value, whether it's a static or computed value, will not have this default automatically replicated for existing rows. To propagate this value, make an update to every existing row.
+
+Removing a column will not have the values automatically removed for existing rows on PowerSync. To propagate the change, make an update to every existing row.
+
+Changing a column type, and/or changing the default value of a column using an `ALTER TABLE` statement, will not be automatically replicated to PowerSync.
+In some cases, the change will have no effect on PowerSync (for example, changing between `VARCHAR` and `TEXT` types). When the values are expected to change, make an update to every existing row to propagate the changes.
 
 ## See Also