Skip to content

10.0.0

Latest
Compare
Choose a tag to compare
@ibis-project-bot ibis-project-bot released this 06 Feb 21:07
· 39 commits to main since this release

10.0.0 (2025-02-06)

⚠ BREAKING CHANGES

  • api: change as_interval unit argument to be positional-only
  • api: change as_timestamp unit argument to be positional-only
  • api: standardize unnest and pivot_longer signatures
  • api: remove deprecated Table.relabel method
  • api: standardize StringValue method signatures
  • api: standardize NumericValue methods
  • api: make GeoSpatialValue.contains positional-only
  • api: make Table.describe quantile argument keyword-only
  • api: remove deprecated Table.relabel method
  • api: make Table.drop_null/Table.fill_null/Table.window_by/Table.alias argument positional-only
  • api: make Table.sample fraction argument positional-only
  • api: make Table.aggregate metrics argument positional-only
  • api: make Table set operation methods positional-only
  • api: make Table.cast and Table.try_cast methods positional-only
  • api: make nth positional-only
  • api: make isin/notin/cases/identical_to positional-only
  • api: make null-related methods and null function positional-only
  • api: make Value.cast and Value.try_cast positional-only
  • internals: make Value.name positional-only
  • internals: make Expr.pipe positional-only
  • internals: make Expr.equals positional-only
  • api: align signatures of to_json methods
  • api: align signatures of to_delta methods
  • api: align signatures of to_csv/to_csv_dir methods
  • api: align signatures of to_parquet/to_parquet_dir methods
  • api: align .sql method signatures across polars and sql as well as the Table method
  • api: top-level connect method now takes its first argument as positional-only
  • duckdb: align signatures of read_sqlite/read_mysql/read_postgres methods in the duckdb backend
  • api: align signatures of read_delta method; sources are positional-only, everything else is required-keyword
  • api: canonicalize has_operation backend method; single argument is positional-only
  • api: canonicalize read_kafka and to_kafka methods of the PySpark backend
  • api: canonicalize drop_table_or_view method of the impala backend
  • api: canonicalize to_geo signature of the the DuckDB backend
  • api: canonicalize read_geo signature of the the DuckDB backend
  • api: align signatures of list_catalogs; like` argument is now keyword-only
  • bigquery: canonicalize set_database signature
  • api: make list_databases arguments all required-keyword
  • risingwave: canonicalize signatures of risingwave-specific create_* methods
  • polars: canonicalize signature of read_pandas method
  • api: align signatures of drop_table method; name is positional-only; the rest are keyword-only
  • api: align signatures of create_catalog and drop_catalog methods; name is positional-only; the rest are keyword-only
  • api: compile method is now the same across backends
  • api: align signatures of create_table method; name is positional-only; obj is positional-or-keyword; the rest are keyword-only
  • api: align signatures of create_view method; name is positional-only; obj is positional-or-keyword; the rest are keyword-only
  • api: align signatures of drop_view method; name is positional-only; the rest are keyword-only
  • api: align signatures of truncate_table method; name is positional-only; the rest are keyword-only
  • api: align signatures of insert method; name is positional-only; obj is positional-or-keyword; the rest are keyword-only
  • api: align signatures of read_json method; sources are positional-only, everything else is required-keyword
  • api: align signatures of read_csv method; sources are positional-only, everything else is required-keyword
  • api: align signatures of read_parquet method; sources are positional-only, everything else is required-keyword
  • api: align signatures of to_torch method
  • api: align signatures of to_polars method
  • api: align signatures of Backend.list_tables method; all arguments are now keyword-only
  • api: align signatures of Backend.table method; name is positional-only; everything else is required-keyword
  • api: align signatures of create_database and drop_database; name is positional-only; everything else is required-keyword
  • api: standardize MapValue method signatures
  • api: standardize ArrayValue method signatures
  • api: type argument of struct function is now required-keyword
  • api: standardize TemporalValue APIs
  • api: where argument of aggregate functions is now required-keyword
  • api: hashbytes and hexdigest are now positional-only
  • api: standardize how argument to join methods as keyword-only and standardize remaining arguments
  • api: ibis.coalesce/ibis.greatest/ibis.least are now positional-only
  • api: Expr.ifelse is now positional-only
  • api: top-level set operation functions are now positional-only
  • api: set_backend and get_backend functions are now positional-only
  • api: ntile function and method is now positional-only
  • api: ibis.preceding/ibis.following` are now positional-only
  • api: expr argument of ibis.asc/ibis.desc is now positional-only; nulls_first is keyword-only
  • api: data argument of ibis.memtable is now positional-only; the rest are keyword-only
  • api: pairs argument of ibis.schema is now positional-only; the rest are keyword-only
  • api: ibis.param is now positional-only
  • api: n argument in Table.limit and Table.head is now required-positional
  • api: offset argument in Table.limit is now required-keyword
  • api: temporal window expression APIs now require all arguments as keywords
  • api: to_pyarrow and to_pyarrow_batches requires expr as positional-only and keyword for everything else
  • api: to_pandas_batches requires expr as positional-only
  • api: execute and to_pandas methods now require expr as positional-only
  • api: distance is now a required keyword argument for the d_within api
  • duckdb: The duckdb backend's read_csv method accepts only DuckDB types for the values components of the columns and types arguments. You may need need to adjust existing code. For example, the string "float64" should be replaced with the string "double".
  • duckdb: The read_in_memory method is removed from the duckdb backend. Use ibis.memtable instead.
  • api: The how parameter of the Value.arbitrary method is removed. call Value.first or Value.last explicitly
  • api: The StringValue.initcap method is removed. Use StringValue.capitalize instead.
  • api: IntegerValue.label is redundant with the IntegerValue.cases method, use that instead. Replace expr.label(labels) with expr.cases(*enumerate(labels))
  • register: The deprecated register method has been removed. Please use the file-specific read_* methods instead. For in-memory objects, pass them to ibis.memtable or create_table.
  • duckdb: Special handling of the temp_directory argument passed to Ibis is removed in favor of passing the argument through directly to duckdb.connect. Interior nodes of directory trees must be created, e.g., using Path.mkdir(exists_ok=True, parents=True), mkdir -p, etc.
  • config: option_context is removed. Use contextlib.contextmanager to create your own version of this functionality if necessary.
  • duckdb: The DuckDB lower bound has been bumped to a version that has storage backwards compatibility. You may need to migrate your DuckDB database files.
  • api: has_name has always returned True since 9.0. It is safe to remove any calls to has_name.
  • backends: execute now returns non-numpy objects for scalar values.
  • api: ibis.negate is removed. Use the negate method on a
    specific column, instead.
  • api: All ibis.geo_* functions are removed. Equivalent
    methods are available on all geo columns.
  • api: where is removed. Use ibis.ifelse instead.
  • value: Value.greatest and Value.least are removed. Use
    ibis.greatest and ibis.least, instead.
  • joins: Passing a pyarrow.Table or a pandas.DataFrame as
    the right-hand-side of a join is no longer supported.

To join against in-memory data, you can pass the in-memory object to
ibis.memtable or con.create_table and use the resulting table object
instead.

Issues closed

  • api: Removed hierarchical usage of schema.
    Ibis uses the following naming conventions:

    • schema: a mapping of column names to datatypes
    • database: a collection of tables
    • catalog: a collection of databases
  • mysql: Ibis now uses the MySQLdb driver. You may need to install MySQL client libraries to build the extension.

  • padding: String padding operations now follow Python semantics and leave strings greater than the padding length untouched.

  • pandas: The pandas backend is removed. Note that pandas DataFrames are STILL VALID INPUTS AND OUTPUTS and will remain so for the foreseeable future. Please use one of the other local backends like DuckDB, Polars, or DataFusion to perform operations directly on pandas DataFrames.

  • dask: The dask backend is removed. Please use one of the
    other backends that Ibis supports.

  • api: remove deprecated where methodism (886b2d1)

  • api: remove top-level negate function (c8c37dd)

  • api: remove top-level geo functions (6b187c3)

  • backends: convert scalars to non-numpy python objects (#10233) (df08d5e)

  • duckdb: bump version lower bound to 0.10 (8dbbc8b)

  • mysql: port to MySQLdb instead of pymysql (#10077) (2b6633c), closes #10055

  • value: remove deprecated greatest and least methods (65f0973)

Features

  • add get_backend to expr (#10228) (4b34a61)
  • api: add distinct option to collect (13cf036)
  • api: add name kwarg to Table.value_counts() (#10361) (12e6057)
  • api: add StringValue.as_time for parsing strings into times (#10278) (9134ef5)
  • api: add to_geo methods for writing geospatial output (#10299) (9f565a9), closes #10296
  • api: add support for passing an optional index parameter to array map and filter (#10205) (dfe7c34)
  • api: move from .case() to .cases() (#9096) (54889db)
  • arrays: add modes array aggregation (#10737) (6603c6c)
  • athena: add amazon athena backend (#10631) (66b3ed2)
  • bigquery, impala, mssql, oracle, postgres: compile Table.sample to native TABLESAMPLE syntax when possible (321a3b5)
  • bigquery: non-nullable schema support for embedded fields in struct (#10189) (b16db6a)
  • databricks: add the databricks backend (#10223) (0733705)
  • databricks: isolate volume creation (#10432) (3cdccfb)
  • datafusion: add ArrayDistinct operation (#10336) (4491a89)
  • datafusion: enable reading multiple paths in read_csv (#10317) (b57be01)
  • datatypes: add support for fixed length arrays (#10729) (d17d53e)
  • datatypes: allow trailing comma in struct string shorthands (#10393) (858deb0)
  • deps: install project and dev and test dependencies in devcontainer (#10544) (e593495), closes #10532
  • deps: support pyarrow 18 (#10381) (0d95336)
  • dtypes: allow passing nullable kwarg to string parsed dtypes (#10632) (4583dee), closes #10243
  • duckdb: add to_json() to Table and duckdb backend (b17c28d)
  • duckdb: allow casting to geometry from text (#10221) (3e73479)
  • duckdb: forward UDF configuration dict as kwargs during registration (#10358) (02efbb2)
  • export: implement the Column.to_list() API (#10498) (d11a292), closes #10102
  • geospatial: accept geopandas GDFs in memtable (56ec371)
  • memtable: add pyarrow.dataset support (#10206) (428d1a3)
  • polars: add Intersection and Difference ops (#10623) (69b848a)
  • polars: add StringFind operation (#10624) (5e8855d)
  • polars: add TimeFromHMS op (#10635) (ff885fb)
  • polars: add ArrayContains operation (eb14daa)
  • polars: add ArrayDistinct operation (#10334) (5657d21)
  • polars: add ArrayIndex operation (ab1df94)
  • polars: add ArrayIntersect operation (#10335) (efbf99d)
  • polars: add ArrayRemove operation (#10322) (81d3179)
  • polars: add ArraySlice operation (224bfc3)
  • polars: add ArrayStringJoin op (#10638) (42dd858)
  • polars: add ArrayUnion operation (#10330) (4d5e847)
  • polars: add bitwise operations (bit_and, bit_or, bit_xor) (#10634) (56a66de)
  • polars: allow user to specify "engine" kwarg (#10151) (3877d6d)
  • polars: implement sample (c7c900b)
  • postgres: use psycopg rather than psycopg2 (#10659) (85b449e)
  • pyarrow: support arrow PyCapsule interface in more places (#9663) (c9238bd)
  • pyspark: add official support and ci testing with spark connect (#10187) (abb5593)
  • python: support python 3.13 (#10582) (35f2b9d)
  • risingwave: support include_null in first/last aggs (d3e4104)
  • types: add typing overloads to common .cast() dtypes (#10682) (1394a2a)

Bug Fixes

  • api: ensure consistent typing in round output type (#10351) (b2b0925)
  • api: ensure memtable schema and columns match (#10617) (f1837b7)
  • api: use to_pyarrow() instead of execute() when pretty printing scalars (10b38ee)
  • athena: implement proper support for inserting data (#10770) (622de09)
  • athena: use exists instead of replace for create_database (#10767) (3c13d67)
  • backends: ensure that analytic functions do not receive a window frame (#10739) (85d5d68)
  • clickhouse: fix truncating to date from a timestamp (#10220) (24e5395)
  • compiler: respect order of ops in FloorDivide (#10353) (af24c8e)
  • compiler: use the correct exception type for a failed import (#10430) (00d83f3)
  • datafusion: handle array concat like everything else (f6b4ca4)
  • datatype-parsing: ensure that geospatial types are round trippable through the data type parser (#10171) (1c91d65), closes #10170
  • datatypes: ensure that decimals can be upcast when source precision, scale are lte to their target fields (#10470) (5ce906a), closes #10467
  • datatypes: give a more useful error message when trying to cast structs with different fields (#10214) (d3496fd)
  • datatypes: return pd.Timestamp or pd.Series[datetime64] for date.to_pandas() (#8784) (2e56acb)
  • deps: bump duckdb to 1.1.2 (#10311) (198c562)
  • deps: bump the upper bound of duckdb to allow easier upgrades between ibis releases (#10786) (db0e0c2)
  • deps: enable running uv lock command in devcontainer (#10576) (caf3daf), closes #10575
  • deps: regen requirements-dev.txt (dbe43cf)
  • deps: remove the upper bounds for pyarrow (#10503) (24c0aca), closes #10492
  • deps: update dependency black to v25 (#10746) (2da91fb)
  • deps: update dependency datafusion to v41 (#10147) (e7cfc11)
  • deps: update dependency datafusion to v42 (#10304) (ae30920)
  • deps: update dependency datafusion to v43 (#10573) (f26b9e6)
  • deps: update dependency snowflake-connector-python to v3.12.3 [security] (#10366) (d841789)
  • deps: update dependency sqlglot to >=23.4,<25.22 (#10109) (44c4de8)
  • deps: update dependency sqlglot to >=23.4,<25.23 (#10176) (6042bf4)
  • deps: update dependency sqlglot to >=23.4,<25.24 (#10230) (021df72)
  • deps: update dependency sqlglot to >=23.4,<25.25 (#10237) (c898cbc)
  • deps: update dependency sqlglot to >=23.4,<25.26 (#10314) (43dfe38)
  • deps: update dependency sqlglot to >=23.4,<25.27 (#10348) (3252b38)
  • deps: update dependency sqlglot to >=23.4,<25.28 (#10356) (fbf214d)
  • deps: update dependency sqlglot to >=23.4,<25.29 (#10375) (6fffa50)
  • deps: update dependency sqlglot to >=23.4,<25.30 (#10444) (72e6f7b)
  • deps: update dependency sqlglot to >=23.4,<25.31 (#10472) (321a382)
  • deps: update dependency sqlglot to >=23.4,<25.32 (#10502) (18dbfaf)
  • deps: update dependency sqlglot to >=23.4,<25.33 (#10527) (0721d11)
  • deps: update dependency sqlglot to >=23.4,<25.35 (2f2c834)
  • deps: update dependency sqlglot to >=23.4,<26.2 (#10662) (1be3509)
  • deps: update dependency sqlglot to >=23.4,<26.3 (#10668) (c987927)
  • deps: update dependency sqlglot to >=23.4,<26.4 (#10715) (30b3822)
  • deps: update dependency sqlglot to v26 (437146a)
  • dev-env: fix aarch64 nix builds (#10795) (985942d)
  • docs: add null examples header and expected failure (#10496) (b1be5ea)
  • docs: render func args as args, not list of args (51d90a7)
  • druid, risingwave: better error on unsupported random/sample methods (5a47668)
  • duckdb: ensure that duckdb columns argument to read_csv accepts duckdb syntax not ibis syntax (#10696) (83bed74)
  • duckdb: handle null propagation correctly in array concatenation (14620bb)
  • duckdb: return null typed pyarrow arrays and disable creating tables with all null columns in duckdb (#9810) (b2c0989)
  • duckdb: thread udf parameters through (9377966)
  • duckdb: use simple GEOMETRY type for all geospatial data (#10324) (#10333) (0b98112)
  • geometry: make output handling robust to text or bytes (49b404a)
  • graphviz: ensure all lines are left-justified (#10783) (3e86e2d)
  • insert: user can specify insert table in specified database (#10285) (0e848d0)
  • interactive-repr: ensure that null scalars can be repr'd interactively (#10784) (5f1c75a), closes #10780
  • joins: allow chaining positional and cross joins (#10122) (e969eda)
  • make memtable cleanup tests work (#10283) (11f8921)
  • memtables: track memtables with a weakset to allow overwriting tables with the same name but different data (#10133) (bf0a666)
  • mssql: allow temp=None (#10289) (ea1c179)
  • mssql: catch and bubble up exceptions from .sql (#10332) (80ad348), closes #10331
  • mssql: ensure ibis.random() generates a new value per call (#10173) (1667f43)
  • mssql: escape special characters in passwords (#10437) (caf3632), closes /stackoverflow.com/questions/78531086/pyodbc-connection-string-correctly-escaping-password-with-special-characters/78532507#78532507
  • mssql: exclude window frame from rank (#10302) (244876a), closes #10291
  • mssql: support new sqlglot temporal types (150eff3)
  • mssql: use nvarchar to avoid non ascii be question mark (564594f)
  • mysql: add dtype mapping for mediumint (6d22c5c)
  • mysql: generate IF NOT EXISTS if force=True (#10785) (d613b55)
  • mysql: handle database names that must be quoted in list_tables (#10466) (23c0e81)
  • polars: ensure that null literals are not mistakenly made non-null (d4d4bcc)
  • polars: handle pl.Array (#10260) (aadae58)
  • polars: use elementwise flatten to flatten nested arrays (#10168) (f97bb77), closes #10135
  • postgres: add dtype mapping for postgres OID type (b279fb2)
  • postgres: dedent in udfs (3f5d090)
  • postgres: do not use schema when renaming a table for overwrite purposes (#10771) (c7ea505)
  • postgres: fix insertion of NaT/None into timestamp columns (#10748) (847ed85)
  • postgres: make null handling in array concat consistent with all other backends (6e51493)
  • pyspark: generate IF NOT EXISTS if force=True (#10782) (c7d1d38)
  • repr: remove expression printing from exception message (#10130) (8a260a8)
  • snowflake, bigquery: minor adjustments for passing in dialect to sg.func (#10407) (210ab3c)
  • snowflake: apply casting logic for json output to scalars (#10202) (91764d3)
  • snowflake: ensure that paths are URIs on all platforms (#10775) (42cbb5d)
  • snowflake: ensure that timestamps are properly timezoned (#10732) (02628e8)
  • snowflake: make semantics of array filtering match everything else (#10469) (bad487b)
  • snowflake: only compile sample to TABLESAMPLE on physical tables (#10218) (bdb718d)
  • snowflake: use as_posix instead of as_uri to avoid escaping special characters (#10792) (ec41d4c)
  • sql: include dialect when creating sqlglot functions (#10397) (36c99ba)
  • sql: remove constants in order_by calls during select merging (#10475) (6e693b7), closes #10428
  • sql: standardize NULL handling of argmin/argmax (#10227) (51335ed)
  • trino,pyspark: improve null handling in array filter (#10448) (860b9ca)
  • trino: remove invalid ranking function window frames (#10363) (348500c)
  • udf: make udfs impure and avoid merging selects with impure functions (1693984)

Documentation

  • add callout notes for some APIs changes and additions (#10551) (2e1aa79)
  • add JSON API docs (#10722) (a9d6611)
  • algolia: also add functions to algolia index (5ae697e)
  • algolia: feed the index a little snack (#10210) (10f7f5b)
  • api: avoid quartodoc warning about missing parameter (499ccc6)
  • api: deprecate to_array in favor of as_scalar in docstring (#10508) (62e8b6c)
  • api: indicate that re_search need only match part of a string (#10402) (78d88a5)
  • api: use memtable for pipe docstring (41ea0c7)
  • avoid needing to render API docs for any preview/render invocation (#10106) (34c465c)
  • bigquery: add update-adc flag to gcloud auth login (#10172) (82e9ba0)
  • blog: classification metrics on the backend (#10501) (aafb30f)
  • blog: fix metric print output in classification blog (f832547)
  • blog: ibis, duckdb geo and lonboard for overture maps (#10215) (5f18747)
  • blog: walking talking cube (#10160) (ea16887)
  • build: fetch all commits to enable proper dynamic versioning in the docs build (#10184) (c62efce)
  • campaign-finance-blog-post: fix typo (e438bf3)
  • campaign-finance-blog-post: use nullif (80027bf)
  • clarify distinct in difference(), add examples (#10716) (b711f39)
  • clickhouse: entry into the accursed (#10174) (5a2e6e8)
  • code_of_conduct: update committee members and reporting email (#10117) (dcdeaea)
  • contribute: include instructions to use Pixi (#10504) (db8af10)
  • correct link generated in operations matrix (#10345) (c7c929d)
  • datafusion: add datafusion nyc presentation (#10141) (913bc55)
  • datafusion: add imdb live demo reference to end of presentation (#10162) (1d965c7)
  • datafusion: assorted edits to datafusion meetup talk (#10144) (c6008e8)
  • datafusion: update talk title (#10150) (c799f6b)
  • deduplicate rendering of cloud support policy (#10637) (bb0f354)
  • dev: add some hard-won pytest tips and tricks (#10224) (d8638b6)
  • development: update tool matrices for accuracy (fb4e45d)
  • dropdowns: make dropdowns scrollable and easier to see in navigation bar (#10090) (c8932c8)
  • druid: fix connect kwarg example (#10372) (7dac420), closes #10368
  • duckdb: remove warning of duckdb not supporting chunk_size parameter (af9c2a2), closes /github.com/duckdb/duckdb/pull/6840#issuecomment-1481387587 #10443
  • examples: count, percent_rank, cume_dist, ntile example usage (#10435) (291bb69)
  • examples: add asof_join weather example (#10507) (7595ca6)
  • examples: add having example for GroupedTable (#10457) (130e4d5)
  • examples: add Value.asc and Value.desc examples (#10515) (41c8f23)
  • examples: add as_interval IntegerValue example (a28ceb1)
  • examples: add as_timestamp IntegerValue example (e693e71)
  • examples: add cumsum, cummean, and sum examples (#10474) (2650f7c)
  • examples: add DayOfWeek method examples (#10394) (6e08195)
  • examples: add distinct argument example for intersect (ac99ebb)
  • examples: add example and see also ref for compile (e69348b)
  • examples: add example and see also ref for ibis.to_sql (9877766)
  • examples: add example and see also ref for Table.execute (bb40a50)
  • examples: add hashbytes example (#10537) (0b99e47)
  • examples: add ibis.get_backend examples (#10489) (4d99018)
  • examples: add IntegerColumn bitwise aggregegate examples (a189c5a)
  • examples: add mean, bucket, histogram examples (3a1bb36)
  • examples: add remaining non-TimeValue temporal examples (#10401) (c55cbd9)
  • examples: add remaining selectors examples (#10421) (a340e4f)
  • examples: add rest arg examples for intersect (e663faf)
  • examples: add selector example for none (#10425) (9f60ca3)
  • examples: add std, var, corr, and cov usage examples (161c61f)
  • examples: add temporal day_of_year example (#10376) (40c14c3)
  • examples: add temporal strftime examples (#10362) (11b4e3a)
  • examples: add temporal truncate examples (#10367) (41a9b7d)
  • examples: add temporal interval examples (#10383) (5eed3f7)
  • examples: add type example (#10420) (9d1e8cc)
  • examples: add unbind example (#10454) (40a30c1)
  • examples: add usage examples for get_backend and get_name (#10449) (ae1bb20)
  • examples: align Table.get_backend with ibis.get_backend examples (#10495) (11947d8)
  • examples: cummin, cummax, lead, lag, nth example usage (452f807)
  • expand ibis-for-sql-users set operator examples (#10395) (5c14965)
  • fix typo in query docstring (#10308) (04f8a5f)
  • fix: skip doctest for SqlString return (efbe5b1)
  • how-to: fix the ffill/bfill how-to guide (70de8db)
  • how-to: marimo integration (#10392) (e1f9124)
  • impala: remove table methods from backend docs (#10557) (1163156), closes #9840
  • render: fix rendering of backend method docstrings (#10313) (a4fe3ee)
  • repl: bump duckdb pyodide wheel to 1.1.2 (#10423) (38ccb4d)
  • reword backend counting to nearly 20 from 20+ (#10533) (52b5353)
  • security: update security report address to point to private Zulip channel (#10118) (5f8a6c9)
  • superduperdb name change (#10250) (76940ce)
  • support-matrix: hide internal operations from the support matrix (#10352) (c53ce33)
  • synonyms: add synonym list to redirect searches with no results (#10163) (78fd5c6)
  • table-expr: include inherited methods (all to_* methods) (#10164) (b39ec14)
  • tutorial: fix typo in ibis-for-sql-users.qmd (#10549) (6cf461f)
  • update ci-analysis post (#10290) (22dcce1)
  • update README backends (#10259) (41462c5)
  • use python3.11 in doctests to avoid impyla import issue (#10464) (d60c370)

Refactors

  • api: compile method is now the same across backends (2a2e3c5)
  • api: data argument of ibis.memtable is now positional-only; the rest are keyword-only (869fe02)
  • api: distance is now a required keyword argument for the d_within api (d9f1692)
  • api: execute and to_pandas methods now require expr as positional-only (d8d6852)
  • api: Expr.ifelse is now positional-only (8b094d7)
  • api: expr argument of ibis.asc/ibis.desc is now positional-only; nulls_first is keyword-only (633ccab)
  • api: hashbytes and hexdigest are now positional-only (fd5e7e3)
  • api: ibis.coalesce/ibis.greatest/ibis.least are now positional-only (8cc7216)
  • api: ibis.param is now positional-only (6d5f036)
  • api: n argument in Table.limit and Table.head is now required-positional (1e3473a)
  • api: ntile function and method is now positional-only (a33ccb3)
  • api: offset argument in Table.limit is now required-keyword (9c159a7)
  • api: pairs argument of ibis.schema is now positional-only; the rest are keyword-only (a252ee5)
  • api: set_backend and get_backend functions are now positional-only (6b458b7)
  • api: to_pandas_batches requires expr as positional-only (d7fb0b1)
  • api: to_pyarrow and to_pyarrow_batches requires expr as positional-only and keyword for everything else (892a3cf)
  • api: type argument of struct function is now required-keyword (931dcbc)
  • api: where argument of aggregate functions is now required-keyword (5b29be3)
  • api: align .sql method signatures across polars and sql as well as the Table method (007afcf)
  • api: align signatures of Backend.list_tables method; all arguments are now keyword-only (03a4d8a)
  • api: align signatures of Backend.table method; name is positional-only; everything else is required-keyword (22de2b5)
  • api: align signatures of create_catalog and drop_catalog methods; name is positional-only; the rest are keyword-only (0f8be2c)
  • api: align signatures of create_database and drop_database; name is positional-only; everything else is required-keyword (f5fae11)
  • api: align signatures of create_table method; name is positional-only; obj is positional-or-keyword; the rest are keyword-only (895a0af)
  • api: align signatures of create_view method; name is positional-only; obj is positional-or-keyword; the rest are keyword-only (fae9f76)
  • api: align signatures of drop_table method; name is positional-only; the rest are keyword-only (4acc7e1)
  • api: align signatures of drop_view method; name is positional-only; the rest are keyword-only (4488e62)
  • api: align signatures of insert method; name is positional-only; obj is positional-or-keyword; the rest are keyword-only (98dc49f)
  • api: align signatures of list_catalogs; like` argument is now keyword-only (456d579)
  • api: align signatures of read_csv method; sources are positional-only, everything else is required-keyword (681378c)
  • api: align signatures of read_delta method; sources are positional-only, everything else is required-keyword (ed3df4b)
  • api: align signatures of read_json method; sources are positional-only, everything else is required-keyword (643516a)
  • api: align signatures of read_parquet method; sources are positional-only, everything else is required-keyword (63d44e6)
  • api: align signatures of to_csv/to_csv_dir methods (9a641f8)
  • api: align signatures of to_delta methods (3440922)
  • api: align signatures of to_json methods (a557414)
  • api: align signatures of to_parquet/to_parquet_dir methods (f099499)
  • api: align signatures of to_polars method (6b055b1)
  • api: align signatures of to_torch method (ce2f81d)
  • api: align signatures of truncate_table method; name is positional-only; the rest are keyword-only (d6e411b)
  • api: canonicalize drop_table_or_view method of the impala backend (b87418d)
  • api: canonicalize has_operation backend method; single argument is positional-only (6b53a21)
  • api: canonicalize read_geo signature of the the DuckDB backend (38d2190)
  • api: canonicalize read_kafka and to_kafka methods of the PySpark backend (bab767d)
  • api: canonicalize to_geo signature of the the DuckDB backend (c8589b2)
  • api: canonicalize first argument of Backend.from_connection (cb28632)
  • api: canonicalize first argument of Backend.from_connection (00ff77e)
  • api: change as_interval unit argument to be positional-only (eda91b2)
  • api: change as_timestamp unit argument to be positional-only (f58c9d5)
  • api: ibis.preceding/ibis.following` are now positional-only (8ec0d11)
  • api: make GeoSpatialValue.contains positional-only (64825db)
  • api: make isin/notin/cases/identical_to positional-only (13695f1)
  • api: make list_databases arguments all required-keyword (9beb279)
  • api: make nth positional-only (dd0bbc2)
  • api: make Table.aggregate metrics argument positional-only (028fc6c)
  • api: make Table.cast and Table.try_cast methods positional-only (3bfcc0c)
  • api: make Table.describe quantile argument keyword-only (94758b1)
  • api: make Table.drop_null/Table.fill_null/Table.window_by/Table.alias argument positional-only (a1731a8)
  • api: make Table.sample fraction argument positional-only (a1fa4b0)
  • api: make Table set operation methods positional-only (f363a07)
  • api: make Value.cast and Value.try_cast positional-only (75c9e6b)
  • api: make null-related methods and null function positional-only (da4cb1e)
  • api: remove has_name (#10246) (adc2fd0)
  • api: remove IntegerValue.label (6278f7e)
  • api: remove StringValue.initcap method (0ba40da)
  • api: remove deprecated Table.relabel method (1480cf7)
  • api: remove deprecated Table.relabel method (1ae4025)
  • api: remove schema (#10149) (9642182)
  • api: remove the how parameter from Value.arbitrary (64521bc)
  • api: standardize ArrayValue method signatures (bc4c7a3)
  • api: standardize how argument to join methods as keyword-only and standardize remaining arguments (08cf816)
  • api: standardize MapValue method signatures (7313dfc)
  • api: standardize NumericValue methods (e0460b5)
  • api: standardize StringValue method signatures (cd7c1b6)
  • api: standardize TemporalValue APIs (72c14d7)
  • api: standardize unnest and pivot_longer signatures (62af6fa)
  • api: temporal window expression APIs now require all arguments as keywords (12a6074)
  • api: top-level connect method now takes its first argument as positional-only (a7a050a)
  • api: top-level set operation functions are now positional-only (38c275b)
  • bigquery: canonicalize set_database signature (6777877)
  • bigquery: remove unnecessary and misspelled bigquery string find implementation (#10119) (0558319)
  • compiler: fix Compiler.dialect type hint, remove unneeded duckdb compilation rule for RegexReplace (#10400) (a806691)
  • config: remove option_context (4f15905)
  • dask: remove the dask backend (d66dff5)
  • duckdb: align signatures of read_sqlite/read_mysql/read_postgres methods in the duckdb backend (a522187)
  • duckdb: pass temp_directory through as-is to duckdb.connect (4ea041f)
  • duckdb: remove the read_in_memory method from the duckdb backend (8343c98)
  • duckdb: replace register usage with read (56b8ce4)
  • impala: remove impala table (#9840) (e38eb83), closes #9839 #9365
  • internals: make Expr.equals positional-only (024ccc6)
  • internals: make Expr.pipe positional-only (413e52e)
  • internals: make Value.name positional-only (3abddd1)
  • internals: move _is_null_literal to its call site (78edd68)
  • joins: require explicit abstract table as RHS of joins (#9661) (8166717), closes #9571
  • padding: follow python string padding conventions (#10096) (b31fcc6)
  • pandas: remove the pandas backend (93f8e10)
  • polars: canonicalize signature of read_pandas method (f5a4c79)
  • pyarrow-format: avoid constructing unnecessary array to produce a scalar (#10204) (c9e5d58)
  • register: remove deprecated register method (#10545) (aa60584)
  • risingwave: canonicalize signatures of risingwave-specific create_* methods (778ea46)
  • risingwave: decouple the risingwave backend from the postgres backend (#10669) (bd30ba5), closes #10659
  • snowflake: remove some unnecessary checking of input in the snowflake compiler (4fceebf)
  • sql-compiler: pull up argmin and argmax into the base compiler (ef6634c)
  • sql: centralize compilation of sample with a shared rewrite rule (b95a036)
  • sql: clean up unnecessary use of explicit visit methods (2acaa16)
  • sql: simplify paren handling for binary ops (721ba4e)
  • table-api: unify exception type for all backends to TableNotFound when a table does not exist (#9695) (0c49e3b)

Performance

  • api: return tuple from Table.columns instead of list (7d89a39)
  • clickhouse: avoid unnecessary construction of a pyarrow table when inserting a pandas dataframe (#10287) (7fd710c)
  • duckdb: reduce branching factor for ArrayDistinct (9b7a377)
  • duckdb: use builtin negative array slicing in newer duckdbs (b91e3dc)

Deprecations

  • structs: slate destructure for removal in 11.0 (e0bc9b7)