Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

depr(dask,pandas): deprecate the dask and pandas backends #9888

Merged
merged 1 commit into from
Aug 22, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 0 additions & 2 deletions docs/_tabsets/install.qmd
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,6 @@ from textwrap import dedent
backends = [
{"name": "BigQuery", "module": "bigquery"},
{"name": "ClickHouse", "module": "clickhouse"},
{"name": "Dask", "module": "dask"},
{"name": "DataFusion", "module": "datafusion"},
{"name": "Druid", "module": "druid"},
{"name": "DuckDB", "module": "duckdb"},
Expand All @@ -20,7 +19,6 @@ backends = [
{"name": "MSSQL", "module": "mssql"},
{"name": "MySQL", "module": "mysql"},
{"name": "Oracle", "module": "oracle"},
{"name": "pandas", "module": "pandas"},
{"name": "Polars", "module": "polars"},
{"name": "PostgreSQL", "module": "postgres"},
{"name": "PySpark", "module": "pyspark"},
Expand Down
7 changes: 7 additions & 0 deletions docs/backends/dask.qmd
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,13 @@

![](https://img.shields.io/badge/memtables-native-green?style=flat-square) ![](https://img.shields.io/badge/inputs-CSV | Parquet-blue?style=flat-square) ![](https://img.shields.io/badge/outputs-CSV | pandas | Parquet | PyArrow-orange?style=flat-square)

::: {.callout-warning}
## The Dask backend is slated for removal in Ibis 10.0
We recommend using one of our other backends.

Many workloads work well on the DuckDB and Polars backends, for example.
:::

## Install

Install Ibis and dependencies for the Dask backend:
Expand Down
8 changes: 8 additions & 0 deletions docs/backends/pandas.qmd
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,14 @@

![](https://img.shields.io/badge/memtables-native-green?style=flat-square) ![](https://img.shields.io/badge/inputs-CSV | Parquet-blue?style=flat-square) ![](https://img.shields.io/badge/outputs-CSV | pandas | Parquet | PyArrow-orange?style=flat-square)

::: {.callout-warning}
## The Pandas backend is slated for removal in Ibis 10.0
We recommend using one of our other backends.

Many workloads work well on the DuckDB and Polars backends, for example.
:::


## Install

Install Ibis and dependencies for the pandas backend:
Expand Down
6 changes: 2 additions & 4 deletions ibis/backends/dask/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -48,16 +48,14 @@ def do_connect(
>>> ibis.dask.connect(data)

"""
if dictionary is None:
dictionary = {}
super().do_connect(dictionary)

for k, v in dictionary.items():
for k, v in self.dictionary.items():
if not isinstance(v, (dd.DataFrame, pd.DataFrame)):
raise TypeError(
f"Expected an instance of 'dask.dataframe.DataFrame' for {k!r},"
f" got an instance of '{type(v).__name__}' instead."
)
super().do_connect(dictionary)

def disconnect(self) -> None:
pass
Expand Down
5 changes: 5 additions & 0 deletions ibis/backends/pandas/__init__.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
from __future__ import annotations

import warnings
from functools import lru_cache
from typing import TYPE_CHECKING, Any

Expand Down Expand Up @@ -51,6 +52,10 @@ def do_connect(
<ibis.backends.pandas.Backend at 0x...>

"""
warnings.warn(
f"The {self.name} backend is slated for removal in 10.0.",
DeprecationWarning,
)
self.dictionary = dictionary or {}
self.schemas: MutableMapping[str, sch.Schema] = {}

Expand Down
2 changes: 2 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -385,6 +385,8 @@ filterwarnings = [
"ignore:Passing a BlockManager to DataFrame is deprecated:DeprecationWarning",
# snowpark logging warnings
"ignore:The 'warn' method is deprecated, use 'warning' instead:DeprecationWarning",
# dask and pandas backend deprecation
'ignore:The (dask|pandas) backend is slated for removal in 10\.0:DeprecationWarning',
]
empty_parameter_set_mark = "fail_at_collect"
markers = [
Expand Down