Filter rows using a bitwise operator

[Gigioliva]

Research

• I have searched the above polars tags on Stack Overflow for similar questions.

• I have asked my usage related question on Stack Overflow.

Link to question on Stack Overflow

No response

Question about Polars

Is it possible to apply a filter on rows using the bitwise operator? For example:

import polars

df = polars.DataFrame({"a": [2, 4, 8, 9], "b": ["a", "b", "c", "d"]})
print(df)

# should return the lines for which "a" & 4 is 0
# so all except the second row
df = df.filter((polars.col("a") & 2**2) == 0)
print(df)

[cmdlineluser]

Not sure if there is a more direct way - all I could find regarding bitwise was a feature request: #5009

>>> df.filter(pl.col("a").map(lambda col: np.bitwise_and(col, 2**2) == 0))
shape: (3, 2)
┌─────┬─────┐
│ a   | b   │
│ --- | --- │
│ i64 | str │
╞═════╪═════╡
│ 2   | a   │
├─────┼─────┤
│ 8   | c   │
├─────┼─────┤
│ 9   | d   │
└─────┴─────┘

[alexander-beedie]

I think we should probably look at exposing expression-level bitwise functionality on a dedicated Expr.bit namespace.

Note that there is some bitwise functionality available at the Series level where there isn't the same potential for ambiguity between logical/bitwise &, but it's not ideal if you want to use expressions (or the lazy API):

df.filter( (pl.col("a") & pl.Series([4]*len(df))) == 0)

# shape: (3, 2)
# ┌─────┬─────┐
# │ a   ┆ b   │
# │ --- ┆ --- │
# │ i64 ┆ str │
# ╞═════╪═════╡
# │ 2   ┆ a   │
# │ 8   ┆ c   │
# │ 9   ┆ d   │
# └─────┴─────┘

[FObersteiner]

Related? #6311

[stinodego]

Looks like this has been implemented. Running your example gives the expected output:

shape: (3, 2)
┌─────┬─────┐
│ a   ┆ b   │
│ --- ┆ --- │
│ i64 ┆ str │
╞═════╪═════╡
│ 2   ┆ a   │
│ 8   ┆ c   │
│ 9   ┆ d   │
└─────┴─────┘