Skip to content

Commit

Permalink
[DOP-11905] Prepare for release
Browse files Browse the repository at this point in the history
  • Loading branch information
dolfinus committed Feb 5, 2024
1 parent 4796345 commit 631cb1d
Show file tree
Hide file tree
Showing 6 changed files with 29 additions and 22 deletions.
28 changes: 28 additions & 0 deletions docs/changelog/0.10.1.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
0.10.1 (2024-02-05)
===================

Features
--------

- Add support of ``Incremental Strategies`` for ``Kafka`` connection:

.. code-block:: python
reader = DBReader(
connection=Kafka(...),
source="topic_name",
hwm=AutoDetectHWM(name="some_hwm_name", expression="offset"),
)
with IncrementalStrategy():
df = reader.run()
This lets you resume reading data from a Kafka topic starting at the last committed offset from your previous run. (:github:pull:`202`)
- Add ``has_data``, ``raise_if_no_data`` methods to ``DBReader`` class. (:github:pull:`203`)
- Updare VMware Greenplum connector from ``2.1.4`` to ``2.3.0``. This implies:
* Greenplum 7.x support
* `Kubernetes support <https://docs.vmware.com/en/VMware-Greenplum-Connector-for-Apache-Spark/2.3/greenplum-connector-spark/configure.html#k8scfg>`_
* New read option `gpdb.matchDistributionPolicy <https://docs.vmware.com/en/VMware-Greenplum-Connector-for-Apache-Spark/2.3/greenplum-connector-spark/options.html#distpolmotion>`_
which allows to match each Spark executor with specific Greenplum segment, avoiding redundant data transfer between Greenplum segments
* Allows overriding `Greenplum optimizer parameters <https://docs.vmware.com/en/VMware-Greenplum-Connector-for-Apache-Spark/2.3/greenplum-connector-spark/options.html#greenplum-gucs>`_ in read/write operations (:github:pull:`208`)
- ``Greenplum.get_packages()`` method now accepts optional arg ``package_version`` which allows to override version of Greenplum connector package. (:github:pull:`208`)
1 change: 1 addition & 0 deletions docs/changelog/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@

DRAFT
NEXT_RELEASE
0.10.1
0.10.0
0.9.5
0.9.4
Expand Down
14 changes: 0 additions & 14 deletions docs/changelog/next_release/202.feature.rst

This file was deleted.

1 change: 0 additions & 1 deletion docs/changelog/next_release/203.feature.rst

This file was deleted.

6 changes: 0 additions & 6 deletions docs/changelog/next_release/208.feature.1.rst

This file was deleted.

1 change: 0 additions & 1 deletion docs/changelog/next_release/208.feature.2.rst

This file was deleted.

0 comments on commit 631cb1d

Please sign in to comment.