Add logging framework #131
Conversation
* progress on tesscut code * send warnings, return array of filepaths * More generalization, print messages * comment fix * style fixes fix type annotation * rename module, use same thread number for cube_cut Use json.load() documentation, set threads to 8 for cube_cut * unit tests, don't allow cutout size of 0 * fix example in docstring * Use mock in unit tests * rename mock * documentation, fix cache position * remove thread pool, filter ffi footprint files in test data * Add cachetools as requirement * work-around for footprint files using TAP * Remove mock from footprint tests * Delete sample footprints * remove cloud function, make crossmatch and ffi fetch functions public * Remove CIRCLE from example inputs * parametrize tests
There was a problem hiding this comment.
All your changes here seem reasonable to me. I'm not sure if there were any print statements when errors occurred, but if so, log.error might be helpful to capture them and differentiate from info messages. Also, are there any messages we'd like to display to the user even when verbosity is off? If so, we could reshuffle the levels from info/warning to debug/info.
I agree that verbose is a little more user-friendly than dealing with log levels. If you wanted to retain a verbose flag but remove the boilerplate use in all the functions, one option could be to set up an astrocut user configuration file, like an ini or yaml, and move the verbose flag there. This would allow _handle_verbose to be set once globally. The upside is it's a convenient place to have all user config in one place. A user could also set it and forget it. The downside is it can be a bit clunkier for users to change the verbosity on the fly when running a function. I'm not suggesting we need to make any changes here though, but it could be something to consider in the future, especially if we think the list of user configurable settings might grow.
astrocut/utils/utils.py
Outdated
| @@ -216,3 +217,7 @@ def get_fits(cutout_hdus, center_coord=None, output_path=None): | |||
| return cutout_hdulist | |||
|
|
|||
|
|
|||
| def _handle_verbose(verbose: bool): | |||
| """Set the log level according to the verbose parameter""" | |||
| level = logging.INFO if verbose else logging.WARNING | |||
There was a problem hiding this comment.
I think we should rework the levels here.
verbose should turn on DEBUG level.
and the default should probably include INFO
so anything that was always printed to the screen should be INFO or above. Anything that was gated by verbose should be DEBUG.
There was a problem hiding this comment.
Changed in newest commit!
astropy apparently has a config system already. i am not familiar with it, but perhaps we could also use it (or maybe |
astrocut/asdf_cutouts.py
Outdated
| if verbose and not is_anon: | ||
| print(f'Attempting to access private S3 bucket: {s3_path.bucket}') | ||
| if not is_anon: | ||
| log.info(f'Attempting to access private S3 bucket: {s3_path.bucket}') |
There was a problem hiding this comment.
instead of formatting the string and passing it into the logger, as you have here with the fstring, and below with the str.format() calls, it is best practice to let the logger do the formatting. this is for security (in some places we do call astrocut as a web service, so some of these inputs may be untrusted and we don't want them logged onto a server), but there's another reason as well. If the logging level is skipped (e.g. Debug) the logger can avoid doing any str formatting if it's passed in this way, whereas if you format the string before passing it into the logger, you are always doing that (small) bit of work.
so in this case, my suggestion would be:
| log.info(f'Attempting to access private S3 bucket: {s3_path.bucket}') | |
| log.info('Attempting to access private S3 bucket: %s', s3_path.bucket') |
And should be applied to all the log formatting here.
There was a problem hiding this comment.
Since AstropyLogger doesn't support lazy formatting, I configured a new logger for astrocut with a similar formatter.
|
I created a new ticket to set up a user configuration file here: https://jira.stsci.edu/browse/ASB-29237 |
Adds a logging framework. A few things to note:
INFO, and most of the messages are printed to theDEBUGlevel.verboseparameter to log intermediate information. I didn't want to change the API by removing that parameter, so I wrote a function to set the log level based on the value ofverbose. I'm conflicted on whether to deprecate theverboseparameters or leave them in for the sake of user-friendliness. As it is now, theverboseparameter overrides any level that the user sets the log to. Still, I think more users would know how to useverboserather than set the log level themselves. Any thoughts are appreciated!__init__.pyto better separate logic.test_asdf_cut.pysince one of the tests was leaving behind an output file.