[BUG] concatenate_columns Ignoring null values is not available

[Fu-Jie]

Source Code

@pf.register_dataframe_method
@deprecated_alias(columns="column_names")
def concatenate_columns(
    df: pd.DataFrame,
    column_names: List[Hashable],
    new_column_name: Hashable,
    sep: str = "-",
    ignore_empty: bool = True,
) -> pd.DataFrame:

    if len(column_names) < 2:
        raise JanitorError("At least two columns must be specified")


################# Need  modified  ###################
    df[new_column_name] = (
        df[column_names].astype(str).fillna("").agg(sep.join, axis=1)
    )
######## ############ to ###############################
    df[new_column_name] = (
        df[column_names].astype('string').fillna("").agg(sep.join, axis=1)
    )


    if ignore_empty:

        def remove_empty_string(x):
            """Ignore empty/null string values from the concatenated output."""
            return sep.join(x for x in x.split(sep) if x)

        df[new_column_name] = df[new_column_name].transform(
            remove_empty_string
        )

    return df

Like this, np.nan becomes 'nan' in astype str

If change it to this, there is no problem

[samukweku]

thanks @Fu-Jie , do you mind raising a PR for this? It might be better to add an na argument, to decide what happens if nulls are present

[Fu-Jie]

thanks @Fu-Jie , do you mind raising a PR for this? It might be better to add an na argument, to decide what happens if nulls are present

I haven't done PR before, but I can try it. This is my solution, what do you think?

na = ['NaT','nan','<NA>']
df[new_column_name] = (
        df[column_names].astype(str).replace(na,"").agg(sep.join, axis=1)
    )

[samukweku]

give it a go @Fu-Jie ; the @pyjanitor-devs/core-devs is welcoming and will gladly guide you through the process. Have a look at the guide on how to submit a PR.