bug: allow floats as timestamp column #777
Conversation
cla-bot
bot
added
the
cla-signed
| } else { | ||
| Ok(time.clone()) | ||
| } | ||
| } |
There was a problem hiding this comment.
A lot of this is halfway duplicated from the prepare_batch code. It's not a complete duplicate, but there's definitely room to simplify, but it would require more refactoring of the column_behavior code. Not sure how much we want to do now.
There was a problem hiding this comment.
The best options seem to be either:
- Drop the column behavior stuff and have both paths call
prepare_batch(the newer code). - Add a column behavior corresponding to
numeric_to_timestampwhichprepare_batchuses for the timestamp case.
Either would let us avoid adding as much code to column_behavior.
| @@ -24,8 +25,15 @@ pub enum ColumnBehavior { | |||
| /// Cast the given column to the given data type. | |||
There was a problem hiding this comment.
Do we need to use / change the column behavior? I think we could just use the prepare_batch method and avoid that. Alternatively, it seems like we could have a case here that called into the numeric_to_timestamp and at least shared that case (rather than adding more complexity to column behaviors).
| } else { | ||
| Ok(time.clone()) | ||
| } | ||
| } |
There was a problem hiding this comment.
The best options seem to be either:
- Drop the column behavior stuff and have both paths call
prepare_batch(the newer code). - Add a column behavior corresponding to
numeric_to_timestampwhichprepare_batchuses for the timestamp case.
Either would let us avoid adding as much code to column_behavior.
| ) | ||
| golden.jsonl(source) | ||
|
|
||
| async def test_time_column_as_float_can_cast_ns(golden) -> None: |
Allows
timestampcolumn to befloatfor parquet sources in the new prepare path.Also fixes the
time_unit, which was previously not being taken into account.Closes #776