-
Notifications
You must be signed in to change notification settings - Fork 1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
adding duplicate data filter #30
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see a problem on the AutoFilterEngine.supported_filters
implementation, which is not part of this PR.
return self.reader_class().supported_filters()
reader_class
returns a generic Reader
, which does not require a supported_filters
method.
I think that reader_class
should returns a AutoFilterReader
.
I also see many looks filling up lists across the module, that are cleaner as list comprehensions.
Also, UnkownFilterException
is misspell.
@avaldebe Thanks for the excellent review. I added nearly all of your suggestions, and concerning typing fixed similar errors in other parts of the code. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
thanks for addressed the issues I raised
Looking again at the AutoFilterEngine base-class and your first comment, I agree with you and return now a AutoFilterReader instead of just a Reader. Sorry for misunderstanding when reading the first time, I got confused since the comment was not related to this PR. I will fix that in #32 |
This PR adds a duplicate filter to pyaro. By default, it filters duplicates of type "stations", "start_times", "end_times", but the user can configure other keys.
This PR starts also using typing-features not available in python 3.9. Following pyaerocom, python 3.9 is therefore no longer supported, and python 3.12 has been added to the list of supported python versions.
closes #22