Decimal data type corruption

[cmartin1968]

Checks

• I have checked that this issue has not already been reported.
• I have confirmed this bug exists on the latest version of Polars.

Reproducible example

import polars as pl

df = pl.DataFrame({'a': [1,1,1,2,2,2],
                   'b': [1, 1, 1, 2, 2,3], 
                   'x': [2.4, 6.7, None, 5.0, 2.3,5.5],
})

print(df)
# col x as float
print(df.select(pl.col('a'),pl.col('b'),pl.col('x').filter(pl.col('x').is_not_null()).last().over(['a','b'])))

# col x as decimal
print(df.select(pl.col('a'),pl.col('b'),pl.col('x').cast(pl.Decimal(scale=4, precision=15)).filter(pl.col('x').is_not_null()).last().over(['a','b'])))

Log output

Issue description

When querying MySQL tables with DECIMAL data types, subsequent operations on those fields cause data corruption. In this example, when attempting to get the last non NULL value of column 'x' by unique groups of columns 'a' and 'b', the column with DECIMAL dtype is multplied by a factor of 10k

Expected behavior

shape: (6, 3)
┌─────┬─────┬──────┐
│ a ┆ b ┆ x │
│ --- ┆ --- ┆ --- │
│ i64 ┆ i64 ┆ f64 │
╞═════╪═════╪══════╡
│ 1 ┆ 1 ┆ 2.4 │
│ 1 ┆ 1 ┆ 6.7 │
│ 1 ┆ 1 ┆ null │
│ 2 ┆ 2 ┆ 5.0 │
│ 2 ┆ 2 ┆ 2.3 │
│ 2 ┆ 3 ┆ 5.5 │
└─────┴─────┴──────┘
shape: (6, 3)
┌─────┬─────┬─────┐
│ a ┆ b ┆ x │
│ --- ┆ --- ┆ --- │
│ i64 ┆ i64 ┆ f64 │
╞═════╪═════╪═════╡
│ 1 ┆ 1 ┆ 6.7 │
│ 1 ┆ 1 ┆ 6.7 │
│ 1 ┆ 1 ┆ 6.7 │
│ 2 ┆ 2 ┆ 2.3 │
│ 2 ┆ 2 ┆ 2.3 │
│ 2 ┆ 3 ┆ 5.5 │
└─────┴─────┴─────┘
shape: (6, 3)
┌─────┬─────┬───────────────┐
│ a ┆ b ┆ x │
│ --- ┆ --- ┆ --- │
│ i64 ┆ i64 ┆ decimal[15,4] │
╞═════╪═════╪═══════════════╡
│ 1 ┆ 1 ┆ 67000.0000 │
│ 1 ┆ 1 ┆ 67000.0000 │
│ 1 ┆ 1 ┆ 67000.0000 │
│ 2 ┆ 2 ┆ 23000.0000 │
│ 2 ┆ 2 ┆ 23000.0000 │
│ 2 ┆ 3 ┆ 55000.0000 │
└─────┴─────┴───────────────┘

Installed versions

--------Version info---------
Polars:              1.21.0
Index type:          UInt32
Platform:            Linux-5.14.0-362.13.1.el9_3.x86_64-x86_64-with-glibc2.34
Python:              3.11.5 (main, Sep  7 2023, 00:00:00) [GCC 11.4.1 20230605 (Red Hat 11.4.1-2)]
LTS CPU:             False

----Optional dependencies----
Azure CLI            <not installed>
adbc_driver_manager  0.6.0
altair               <not installed>
azure.identity       <not installed>
boto3                <not installed>
cloudpickle          <not installed>
connectorx           0.3.3a1
deltalake            <not installed>
fastexcel            <not installed>
fsspec               <not installed>
gevent               <not installed>
google.auth          <not installed>
great_tables         <not installed>
matplotlib           <not installed>
numpy                1.25.2
openpyxl             3.1.2
pandas               2.2.2
pyarrow              13.0.0
pydantic             <not installed>
pyiceberg            <not installed>
sqlalchemy           2.0.20
torch                <not installed>
xlsx2csv             0.8.3
xlsxwriter           3.1.3

[mcrumiller]

It looks like over may be the culprit, and messes up with the scale parameter:

import polars as pl

df = pl.DataFrame({"x": [1.0]})

# Without `over` we're ok
df.select(pl.col("x").cast(pl.Decimal(scale=2)).first())
# shape: (1, 1)
# ┌──────────────┐
# │ x            │
# │ ---          │
# │ decimal[*,2] │
# ╞══════════════╡
# │ 1.00         │
# └──────────────┘

# With `over`, number gets multiplied by scale parameter
df.select(pl.col("x").cast(pl.Decimal(scale=2)).first().over("x"))
# shape: (1, 1)
# ┌──────────────┐
# │ x            │
# │ ---          │
# │ decimal[*,2] │
# ╞══════════════╡
# │ 100.00       │
# └──────────────┘

# Scale = 5
df.select(pl.col("x").cast(pl.Decimal(scale=5)).first().over("x"))
# shape: (1, 1)
# ┌──────────────┐
# │ x            │
# │ ---          │
# │ decimal[*,5] │
# ╞══════════════╡
# │ 100000.00000 │
# └──────────────┘