Skip to content

Commit

Permalink
fix drop duplicates
Browse files Browse the repository at this point in the history
  • Loading branch information
rwood-97 committed Nov 6, 2023
1 parent 6f1f96d commit c0ceec8
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion mapreader/download/sheet_downloader.py
Original file line number Diff line number Diff line change
Expand Up @@ -557,7 +557,7 @@ def _save_metadata(
if os.path.exists(out_filepath):
existing_metadata_df = pd.read_csv(out_filepath, sep=",", index_col=0)
metadata_df = pd.concat([existing_metadata_df, new_metadata_df], ignore_index=True)
metadata_df.drop_duplicates(keep="first", inplace=True)
metadata_df = metadata_df.loc[metadata_df.astype(str).drop_duplicates().index] # https://stackoverflow.com/questions/43855462/pandas-drop-duplicates-method-not-working-on-dataframe-containing-lists
else:
metadata_df = new_metadata_df

Expand Down

0 comments on commit c0ceec8

Please sign in to comment.