Skip to content

Commit

Permalink
Improved null handling when converting from nu -> dataframe. (nushell…
Browse files Browse the repository at this point in the history
…#13855)

# Description
Fixes: nushell#12726 and nushell#13185

Previously converting columns that contained null caused polars to force
a dtype of object even when using a schema.

Now:
1. When using a schema, the type the schema defines for the column will
always be used.
2. When a schema is not used, the previous type is used when a value is
null.

# User-Facing Changes
- The type defined by the schema we be respected when passing in a null
value `[a]; [null] | polars into-df -s {a: str}` will create a df with
an str dtype column with one null value versus a column of type object.
- *BREAKING CHANGE* If you define a schema, all columns must be in the
schema.
  • Loading branch information
ayax79 authored Sep 16, 2024
1 parent 9ca0fb7 commit af77bc6
Show file tree
Hide file tree
Showing 2 changed files with 272 additions and 45 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -32,12 +32,9 @@ impl PluginCommand for ExprCount {
}

fn examples(&self) -> Vec<Example> {
// to add an example with a result that contains a null we will need to be able to
// allow null values to be entered into the dataframe from nushell
// and retain the correct dtype. Right now null values cause the dtype to be object
vec![Example {
description: "Count the number of non-null values in a column",
example: r#"[[a]; ["foo"] ["bar"]] | polars into-df
example: r#"[[a]; ["foo"] ["bar"] [null]] | polars into-df
| polars select (polars col a | polars count)
| polars collect"#,
result: Some(
Expand Down
Loading

0 comments on commit af77bc6

Please sign in to comment.