Add a way to test for list equality

[m-legrand]

Problem description

I'm currently working with a dataset that contains list columns, and I was surprised not to find an easy way to test for list equality:

>>> import polars as pl
>>> data = pl.DataFrame({"x": [[1], [1, 2], [2, 3]]}, schema={"x": pl.List(int)})
... data
shape: (3, 1)
┌───────────┐
│ x         │
│ ---       │
│ list[i64] │
╞═══════════╡
│ [1]       │
│ [1, 2]    │
│ [2, 3]    │
└───────────┘
>>> data.filter(pl.col("x") == pl.lit([1]))
ArrowErrorException: NotYetImplemented("Casting from Int64 to 
    LargeList(Field { name: \"item\", data_type: Int64, is_nullable: true, metadata: {} }) 
    not supported")

Maybe I missed the right incantation from the Expr.list namespace?
In the meantime I went with the following utility function:

def filter_list_equal(df: pl.DataFrame, colname: str, values: list) -> pl.DataFrame:
    col = pl.col(colname)
    lf = df.lazy()
    lf = lf.filter(col.list.lengths() == pl.lit(len(values)))
    for i, v in enumerate(values):
        lf = lf.filter(col.list[i] == pl.lit(v))
    return lf.collect()

[ion-elgreco]

You need to wrap [1] in a list otherwise it's interpreted as an int #7879. Also you need to add it before you filter on it. Not sure why it's not working within the filter. I think I saw an issue about this before.

data.with_columns(pl.lit([[1]]).alias('y')).filter(pl.col('x') == pl.col('y'))

[ritchie46]

Something seems to go wrong when we inline the predicate. Will take a look later

[m-legrand]

Having to assign a new column (and delete it afterwards) also makes for a more cumbersome user experience.
Not even mentioning having to come up with column names I'm sure my input dataframe doesn't already have!

[cmdlineluser]

This was asked again today on stackoverflow: https://stackoverflow.com/questions/77002768/how-to-filter-a-polars-dataframe-with-list-type-columns

The current workaround for adding a new column seems to be (casting to numerics if necessary e.g. str -> cat) and .hash()

df.filter(pl.col("x").hash() != pl.lit([[1]]).hash())

# shape: (2, 1)
# ┌───────────┐
# │ x         │
# │ ---       │
# │ list[i64] │
# ╞═══════════╡
# │ [1, 2]    │
# │ [2, 3]    │
# └───────────┘

Was also asked a couple of weeks ago: https://stackoverflow.com/questions/76875762/filter-on-listint64-dtype-in-polars