[DNM] Implement dialect "tidb+pymysql"/"tidb+mysqldb" for sqlalchemy

[JaySon-Huang]

What's changed

• Introduce class TiDBDialect_mysqldb/TiDBDialect_pymysql for connection string "tidb+mysqldb"/"tidb+pymysql"
  • These two dialect classes for tidb overwrite the ddl_compiler by TiDBDDLCompiler so we can generate the correct SQL for:
    • altering tiflash replica
    • add vector index
• Introduce class VectorIndex and TiFlashReplica so that we can define them using the sqlalchemy framework
• Introduce function get_declarative_base so that users can utilize it to define a table model with VectorIndex. See the examples/orm-sqlalchemy-quickstart/sqlalchemy-quickstart.py or tests/sqlalchemy/test_sqlalchemy.py for details.

And we can support other tidb-specified features like AUTO_RANDOM in the sqlalchemy-tidb framework

Local test result

>  TEST_TIDB_HOST=127.0.0.1 TEST_TIDB_PORT=8042 tox -e py39
py39: install_package> python -I -m pip install --force-reinstall --no-deps /Users/jayson/Projects/tidb-vector-python/.tox/.tmp/package/12/tidb_vector-0.0.0.tar.gz
py39: commands[0]> pytest tests
=================================================================================================================================== test session starts ===================================================================================================================================
platform darwin -- Python 3.9.6, pytest-8.3.3, pluggy-1.5.0
cachedir: .tox/py39/.pytest_cache
rootdir: /Users/jayson/Projects/tidb-vector-python
configfile: pyproject.toml
collected 40 items                                                                                                                                                                                                                                                                        

tests/integrations/test_utils.py .                                                                                                                                                                                                                                                  [  2%]
tests/integrations/test_vector_client.py ..........                                                                                                                                                                                                                                 [ 27%]
tests/peewee/test_peewee.py .............                                                                                                                                                                                                                                           [ 60%]
tests/sqlalchemy/test_sqlalchemy.py ................                                                                                                                                                                                                                                [100%]

==================================================================================================================================== warnings summary =====================================================================================================================================
tests/sqlalchemy/test_sqlalchemy.py::TestSQLAlchemy::test_insert_get_record
  /Users/jayson/Projects/tidb-vector-python/tests/sqlalchemy/test_sqlalchemy.py:57: SAWarning: Dialect tidb:pymysql will not make use of SQL compilation caching as it does not set the 'supports_statement_cache' attribute to ``True``.  This can have significant performance implications including some performance degradations in comparison to prior SQLAlchemy versions.  Dialect maintainers should seek to set this attribute to True after appropriate development and testing for SQLAlchemy 1.4 caching support.   Alternatively, this attribute may be set to False which will disable this warning. (Background on this warning at: https://sqlalche.me/e/20/cprf)
    session.query(Item1Model).delete()

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
============================================================================================================================= 40 passed, 1 warning in 55.69s ==============================================================================================================================
.pkg: _exit> python /Users/jayson/Projects/tidb-vector-python/lib/python3.13/site-packages/pyproject_api/_backend.py True poetry.core.masonry.api
  py39: OK (58.66=setup[2.77]+cmd[55.89] seconds)
  congratulations :) (58.72 seconds)

Inspired from

• https://github.com/xzkostyan/clickhouse-sqlalchemy
• https://github.com/cockroachdb/sqlalchemy-cockroachdb

[JaySon-Huang]

/cc @wd0517 @Mini256 @breezewish

[breezewish]

Very impressive improvement!! I like the way how user use it now.

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[wd0517]

Great job!
However, I think this should be merged to https://github.com/pingcap/sqlalchemy-tidb. We also need to maintain the sqlalchemy-tidb repository again and ensure it works for both sqlalchemy 1.4 and 2.0. Any suggestions?

BTW, a simplified and MySQL-compatible syntax for creating vector indexe would be more preferable.

[JaySon-Huang]

BTW, a simplified and MySQL-compatible syntax for creating vector indexes would be more preferable.

Yes, we have further plan for making it more easier for creating vector indexes with some optional parameters in the SQL.

However, I think this should be merged to https://github.com/pingcap/sqlalchemy-tidb. We also need to maintain the sqlalchemy-tidb repository again and ensure it works for both sqlalchemy 1.4 and 2.0. Any suggestions?

Get it. I didn't notice there is such a lib there. I'll try to get the changes merged into sqlalchemy-tidb first.

[JaySon-Huang]

Only suite for sqlalchemy~2.0. Closing it because it take too much maintenance cost