-
Notifications
You must be signed in to change notification settings - Fork 118
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Increase test coverage + Fix save_model_to_hdf5
+ Improve is_remote_path
+ Fix is_remote_path
#900
Increase test coverage + Fix save_model_to_hdf5
+ Improve is_remote_path
+ Fix is_remote_path
#900
Conversation
Codecov ReportPatch coverage:
Additional details and impacted files@@ Coverage Diff @@
## main #900 +/- ##
==========================================
+ Coverage 76.78% 83.83% +7.04%
==========================================
Files 329 318 -11
Lines 31431 28731 -2700
Branches 6113 5486 -627
==========================================
- Hits 24134 24086 -48
+ Misses 5726 3140 -2586
+ Partials 1571 1505 -66
Flags with carried forward coverage won't be shown. Click here to find out more.
☔ View full report in Codecov by Sentry. |
saving
save_model_to_hdf5
+ Improve is_remote_path
+ Fix is_remote_path
save_model_to_hdf5
+ Improve is_remote_path
+ Fix is_remote_path
save_model_to_hdf5
+ Improve is_remote_path
+ Fix is_remote_path
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the PR!
keras_core/utils/file_utils.py
Outdated
|
||
# Specific patterns for supported remote paths | ||
supported_patterns = [ | ||
re.compile(r"^(gs|cns|cfs|http|https|ftp|s3)://.*$", re.IGNORECASE) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Which ones of these work with gfile
? Since we use gfile
for remote paths, we should only match the paths for which it will work (otherwise using local accessors which will raise an error is fine).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
WIP
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hello @fchollet,
I've refined the is_remote_path
function to only recognize gs://
and hdfs://
based on gfile
capabilities.
I've also added a warning message when encountering an unrecognized remote path. This provides immediate feedback to developers, helping them quickly identify and address potential issues.
Would appreciate your feedback on this approach.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
def is_remote_path(filepath):
filepath_str = str(filepath).strip()
supported_patterns = [re.compile(r"^(gs|hdfs)://.*$", re.IGNORECASE)]
for pattern in supported_patterns:
if pattern.match(filepath_str):
return True
# Log or print the error message without raising an exception
warning_msg = (
f"Warning: The path '{filepath_str}' is not recognized as a "
f"supported remote path by gfile. Supported paths are: "
f"{', '.join(['gs://', 'hdfs://'])}"
)
print(warning_msg)
return False
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM -- thanks!
@fchollet |
save_model_to_hdf5
+ Improve is_remote_path
+ Fix is_remote_path
save_model_to_hdf5
+ Improve is_remote_path
+ Fix is_remote_path
save_model_to_hdf5
+ Improve is_remote_path
+ Fix is_remote_path
save_model_to_hdf5
+ Improve is_remote_path
+ Fix is_remote_path
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you -- great work!
This PR aims to increase the robustness of the codebase by improving test coverage, fixing existing functions, and adding new tests to validate the changes made.