Skip to content

Commit

Permalink
Docs: Improve Internals page, to educate better about differences to PG
Browse files Browse the repository at this point in the history
  • Loading branch information
amotl committed Dec 4, 2024
1 parent 536a44f commit 55361ac
Show file tree
Hide file tree
Showing 2 changed files with 62 additions and 60 deletions.
48 changes: 1 addition & 47 deletions docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -57,47 +57,6 @@ Connect to CrateDB Cloud.
Connection conn = DriverManager.getConnection("jdbc:crate://example.aks1.westeurope.azure.cratedb.net:5432/?user=crate", connectionProps);
.. _details:

*******
Details
*******

Overview
========

For general purpose use, we recommend to use the official `PostgreSQL JDBC
Driver`_.

*This* JDBC driver is needed in certain scenarios like the one outlined at
`Apache Kafka, Apache Flink, and CrateDB`_. The background is that, when using
the ``postgresql://`` JDBC driver prefix, the `Apache Flink JDBC Connector`_
will implicitly select the corresponding dialect implementation for PostgreSQL.

In turn, this will employ a few behaviours that strictly expect a PostgreSQL
server on the other end, so that some operations will fail on databases
offering wire-compatibility with PostgreSQL, but do not provide certain
features like the `hstore`_ or `jsonb`_ extensions. Also, tools like `Dataiku`_
need this driver to implement transaction commands like ``ROLLBACK`` as a
no-op.


.. _differences:

Differences
===========

The driver is based upon a fork of the `PostgreSQL JDBC Driver`_, see `pgjdbc
driver fork`_. On a high-level perspective, this list enumerates a few
behavioral differences.

- The CrateDB driver deserializes objects to a Map, the official one treats them as JSON.
- A few metadata functions have been adjusted to better support CrateDB's type system.

Read up on further details at the :ref:`internals` section.



*************
Documentation
*************
Expand Down Expand Up @@ -160,26 +119,21 @@ The project is licensed under the terms of the Apache 2.0 license, like
`CrateDB itself <CrateDB source_>`_, see `LICENSE`_.


.. _Apache Flink JDBC Connector: https://github.com/apache/flink-connector-jdbc
.. _Apache Kafka, Apache Flink, and CrateDB: https://github.com/crate/cratedb-examples/tree/main/application/apache-kafka-flink
.. _Apache Kafka, Apache Flink, and CrateDB: https://github.com/crate/cratedb-examples/tree/main/framework/flink
.. _Basic example for connecting to CrateDB and CrateDB Cloud using JDBC: https://github.com/crate/cratedb-examples/tree/main/by-language/java-jdbc
.. _Build a data ingestion pipeline using Kafka, Flink, and CrateDB: https://dev.to/crate/build-a-data-ingestion-pipeline-using-kafka-flink-and-cratedb-1h5o
.. _CrateDB: https://crate.io/products/cratedb/
.. _CrateDB source: https://github.com/crate/crate
.. _Create an issue: https://github.com/crate/crate-jdbc/issues
.. _Dataiku: https://www.dataiku.com/
.. _development sandbox: https://github.com/crate/crate-jdbc/blob/master/DEVELOP.rst
.. _Flink example jobs for CrateDB: https://github.com/crate/cratedb-flink-jobs
.. _hosted on GitHub: https://github.com/crate/crate-jdbc/
.. _hstore: https://www.postgresql.org/docs/current/hstore.html
.. _JDBC: https://en.wikipedia.org/wiki/Java_Database_Connectivity
.. _JDBC 4.1 specification: https://download.oracle.com/otn-pub/jcp/jdbc-4_1-mrel-spec/jdbc4.1-fr-spec.pdf
.. _JDBC API documentation: https://docs.oracle.com/javase/8/docs/technotes/guides/jdbc/
.. _JDBC tutorial: https://docs.oracle.com/javase/tutorial/jdbc/basics/
.. _JDBC Type 4 driver: https://en.wikipedia.org/wiki/JDBC_driver#Type_4_driver_%E2%80%93_Database-Protocol_driver/Thin_Driver_(Pure_Java_driver)
.. _jsonb: https://www.postgresql.org/docs/current/datatype-json.html
.. _LICENSE: https://github.com/crate/crate-jdbc/blob/master/LICENSE
.. _pgjdbc driver fork: https://github.com/crate/pgjdbc
.. _PostgreSQL JDBC Driver: https://github.com/pgjdbc/pgjdbc
.. _PostgreSQL Wire Protocol: https://www.postgresql.org/docs/current/protocol.html
.. _sample application: https://github.com/crate/crate-sample-apps/tree/main/java-spring
Expand Down
74 changes: 61 additions & 13 deletions docs/internals.rst
Original file line number Diff line number Diff line change
@@ -1,40 +1,88 @@
.. _details:
.. _internals:

#########
Internals
#########

pgJDBC employs a few behaviours that strictly expect a PostgreSQL
server on the other end, so that some operations will fail on databases
offering wire-compatibility with PostgreSQL, but do not provide certain
features like the `hstore`_ or `jsonb`_ extensions.

The background is that, when using the ``postgresql://`` JDBC driver
prefix, downstream applications and frameworks will implicitly select
the corresponding dialect implementation for PostgreSQL.


*************
Compatibility
*************

- Tools like `Dataiku`_ need this driver to implement transaction commands
like ``ROLLBACK`` as a no-op.
- The `Apache Flink JDBC Connector`_ needs this driver to not select the
PostgreSQL dialect, see also `Apache Kafka, Apache Flink, and CrateDB`_.

.. note::

For basic and general purpose use, the official `PostgreSQL JDBC Driver`_
can also be used. Sometimes, it is easier because pgJDBC is included
into many applications out of the box.


.. _differences:
.. _implementations:
.. _jdbc-implementation:

What's inside
=============
*********************
Differences to pgJDBC
*********************

The driver is based upon a fork of the `PostgreSQL JDBC Driver`_, see `pgjdbc
driver fork`_, and adjusts a few details to compensate for behavioral
differences of CrateDB.
Please take notice of the corresponding implementation notes:

- `CallableStatement`_ is not supported, as CrateDB itself does not support
stored procedures.
- `DataSource`_ is not supported.
- `ParameterMetaData`_, e.g. as returned by `PreparedStatement`_, is not
supported.
- `ResultSet`_ objects are read only (``TYPE_FORWARD_ONLY``, ``CONCUR_READ_ONLY``),
so changes to a ``ResultSet`` are not supported.
- DDL and DML statements are supported through adjustments to the
`PgPreparedStatement`_ and `PgStatement`_ interfaces.
:Supported:

- A few metadata functions have been adjusted to better support CrateDB's type system.
- The CrateDB JDBC driver deserializes objects to a Map, pgJDBC treats them as JSON.
- DDL and DML statements are supported through adjustments to the
`PgPreparedStatement`_ and `PgStatement`_ interfaces.

:Unsupported:

- `CallableStatement`_ is not supported, as CrateDB itself does not support
stored procedures.
- `DataSource`_ is not supported.
- `ParameterMetaData`_, e.g. as returned by `PreparedStatement`_, is not
supported.
- `ResultSet`_ objects are read only (``TYPE_FORWARD_ONLY``, ``CONCUR_READ_ONLY``),
so changes to a ``ResultSet`` are not supported.

To learn further details about the compatibility with JDBC and PostgreSQL
features, see the specific code changes to the `PgConnection`_,
`PgDatabaseMetaData`_, and `PgResultSet`_ classes.



.. _Apache Flink JDBC Connector: https://github.com/apache/flink-connector-jdbc
.. _Apache Kafka, Apache Flink, and CrateDB: https://github.com/crate/cratedb-examples/tree/main/framework/flink
.. _CallableStatement: https://docs.oracle.com/javase/8/docs/api/java/sql/CallableStatement.html
.. _Dataiku: https://www.dataiku.com/
.. _DataSource: https://docs.oracle.com/javase/8/docs/api/javax/sql/DataSource.html
.. _hstore: https://www.postgresql.org/docs/current/hstore.html
.. _jsonb: https://www.postgresql.org/docs/current/datatype-json.html
.. _ParameterMetaData: https://docs.oracle.com/javase/8/docs/api/java/sql/ParameterMetaData.html
.. _pgjdbc driver fork: https://github.com/crate/pgjdbc
.. _PostgreSQL JDBC Driver: https://jdbc.postgresql.org/
.. _PreparedStatement: https://docs.oracle.com/javase/8/docs/api/java/sql/PreparedStatement.html
.. _ResultSet: https://docs.oracle.com/javase/8/docs/api/java/sql/ResultSet.html


.. _PgConnection: https://github.com/pgjdbc/pgjdbc/compare/REL42.2.5...crate:pgjdbc:REL42.2.5_crate?expand=1#diff-8ee30bec696495ec5763a3e1c1b216776efc124729f72e18dbaa35064af0aef0
.. _PgDatabaseMetaData: https://github.com/pgjdbc/pgjdbc/compare/REL42.2.5...crate:pgjdbc:REL42.2.5_crate?expand=1#diff-0571f8ac3385a7f7bb34e5c77f8afd24810311506989379c2e85c6c16eea6ce4
.. _PgResultSet: https://github.com/pgjdbc/pgjdbc/compare/REL42.2.5...crate:pgjdbc:REL42.2.5_crate?expand=1#diff-7e93771092eab9084402e3c7c81319a1f037febdc7614264329bd29f11d39ef2
.. _PgPreparedStatement: https://github.com/pgjdbc/pgjdbc/compare/REL42.2.5...crate:pgjdbc:REL42.2.5_crate?expand=1#diff-d4946409bd7c59e525f34b4c974a3df76638dc84adc060cc5d13d5409c6aeb21
.. _PgStatement: https://github.com/pgjdbc/pgjdbc/compare/REL42.2.5...crate:pgjdbc:REL42.2.5_crate?expand=1#diff-2abcc60e1b1ef8eeadd6372bf7afd0c0ebae0ebd691b0965fc914fea794eb6d0
.. _PreparedStatement: https://docs.oracle.com/javase/8/docs/api/java/sql/PreparedStatement.html
.. _ResultSet: https://docs.oracle.com/javase/8/docs/api/java/sql/ResultSet.html

0 comments on commit 55361ac

Please sign in to comment.