Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FIX] CSV Import: Add explicit datetime conversion #6696

Merged
merged 2 commits into from
Jan 15, 2024

Conversation

ales-erjavec
Copy link
Contributor

Issue

Fixes an error with dev version of pandas as seen on CI i.e.

FAIL: test_load_csv (Orange.widgets.data.tests.test_owcsvimport.TestUtils.test_load_csv)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/runner/work/orange3/orange3/.tox/beta/lib/python3.11/site-packages/Orange/widgets/data/tests/test_owcsvimport.py", line 449, in test_load_csv
    self.assertSequenceEqual(
AssertionError: Sequences differ: [dtype('O'), dtype('float64'), dtype('O'), Catego[100 chars]64')] != [dtype('<M8[ns]'), dtype('float64'), dtype('O'), [24 chars]64')]

First differing element 0:
dtype('O')
dtype('<M8[ns]')

+ [dtype('<M8[ns]'), dtype('float64'), dtype('O'), 'category', dtype('float64')]
- [dtype('O'),
-  dtype('float64'),
-  dtype('O'),
-  CategoricalDtype(categories=['one', 'three', 'two'], ordered=False, categories_dtype=object),
-  dtype('float64')]
Description of changes

Add explicit to_datetime conversion for requested datetime columns when read_csv failed to convert all values.
Also remove obsolete workaround float precision parsing.

Includes
  • Code changes
  • Tests
  • Documentation

Is already released and in requirements
On newer pandas (dev) read_csv leaves `parse_dates` columns which
failed (strict) parsing as object columns. Add explicit to_datetime
coercion when that happens.
Copy link

codecov bot commented Jan 5, 2024

Codecov Report

Merging #6696 (0bed791) into master (7b68883) will decrease coverage by 0.02%.
Report is 10 commits behind head on master.
The diff coverage is 75.00%.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #6696      +/-   ##
==========================================
- Coverage   88.08%   88.06%   -0.02%     
==========================================
  Files         322      322              
  Lines       70105    70257     +152     
==========================================
+ Hits        61755    61875     +120     
- Misses       8350     8382      +32     

@PrimozGodec PrimozGodec merged commit f5a0f58 into biolab:master Jan 15, 2024
25 of 27 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants