Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improving readability/consistency of errors/exceptions in gempyor. #313

Merged
merged 53 commits into from
Nov 8, 2024
Merged
Show file tree
Hide file tree
Changes from 12 commits
Commits
Show all changes
53 commits
Select commit Hold shift + click to select a range
452902f
Update parameters.py
emprzy Aug 15, 2024
9a7f4a3
Update parameters.py
emprzy Aug 15, 2024
c37a605
Merge branch 'HopkinsIDD:main' into gempyor_error_improvements
emprzy Sep 11, 2024
d063762
Improved readability/consistency of errors/exceptions in 5 gempyor fi…
emprzy Sep 16, 2024
0cdeed1
Merge branch 'HopkinsIDD:main' into gempyor_error_improvements
emprzy Sep 20, 2024
add67d1
Attempting to fix errors by changing the regexs in some of the test f…
emprzy Sep 20, 2024
3b4e490
Another attempt to fix RegExs in test files
emprzy Sep 27, 2024
e3df5d8
Update initial_conditions.py
emprzy Oct 2, 2024
2066ee4
Update parameters.py
emprzy Oct 2, 2024
2b3feed
Update flepimop/gempyor_pkg/tests/parameters/test_parameters_class.py
emprzy Oct 4, 2024
72aaa84
Small adjustments to compartments.py
emprzy Oct 4, 2024
4639721
Some more small syntactical adjustments.
emprzy Oct 4, 2024
064ec7e
Update flepimop/gempyor_pkg/src/gempyor/compartments.py
emprzy Oct 14, 2024
f614127
Continuing to evolve error/exception messags
emprzy Oct 16, 2024
9c20318
Merge branch 'gempyor_error_improvements' of https://github.com/emprz…
emprzy Oct 16, 2024
3b20e8a
Fixing reg-exs
emprzy Oct 16, 2024
afee285
Updating RegEx
emprzy Oct 16, 2024
2eaa427
Fixing RegExs again UGH!
emprzy Oct 16, 2024
48a78ee
I'm losing my mind with this RegEx
emprzy Oct 16, 2024
6e1410f
Update test_parameters_class.py
emprzy Oct 16, 2024
5c09a3c
Update flepimop/gempyor_pkg/tests/parameters/test_parameters_class.py
emprzy Oct 16, 2024
2785437
Shortening a verbose error
emprzy Oct 16, 2024
4cc651f
RegEx update
emprzy Oct 16, 2024
be85d72
Regex edit..
emprzy Oct 18, 2024
fb549c4
Regex edit
emprzy Oct 18, 2024
135aa54
Update flepimop/gempyor_pkg/src/gempyor/compartments.py
emprzy Oct 18, 2024
681ff79
Update flepimop/gempyor_pkg/src/gempyor/compartments.py
emprzy Oct 18, 2024
1549b5b
Update flepimop/gempyor_pkg/src/gempyor/compartments.py
emprzy Oct 18, 2024
cbfd093
Update flepimop/gempyor_pkg/src/gempyor/compartments.py
emprzy Oct 18, 2024
ccfb6e0
Update flepimop/gempyor_pkg/src/gempyor/config_validator.py
emprzy Oct 18, 2024
e83f250
Update flepimop/gempyor_pkg/src/gempyor/config_validator.py
emprzy Oct 18, 2024
6eed1e3
Update flepimop/gempyor_pkg/src/gempyor/config_validator.py
emprzy Oct 18, 2024
08e25a3
Update flepimop/gempyor_pkg/src/gempyor/config_validator.py
emprzy Oct 18, 2024
e68dc8a
Update flepimop/gempyor_pkg/src/gempyor/config_validator.py
emprzy Oct 18, 2024
960f4d2
Update flepimop/gempyor_pkg/src/gempyor/config_validator.py
emprzy Oct 18, 2024
2b75d6a
Update flepimop/gempyor_pkg/src/gempyor/config_validator.py
emprzy Oct 18, 2024
5d999dd
Update flepimop/gempyor_pkg/src/gempyor/config_validator.py
emprzy Oct 18, 2024
a15184e
Update flepimop/gempyor_pkg/src/gempyor/config_validator.py
emprzy Oct 18, 2024
2b2a281
Update flepimop/gempyor_pkg/src/gempyor/config_validator.py
emprzy Oct 18, 2024
76b5adf
Update flepimop/gempyor_pkg/src/gempyor/initial_conditions.py
emprzy Oct 18, 2024
a0ae5e3
Update flepimop/gempyor_pkg/src/gempyor/config_validator.py
emprzy Oct 18, 2024
0c14d66
Update flepimop/gempyor_pkg/src/gempyor/config_validator.py
emprzy Oct 18, 2024
9a6ba69
Update flepimop/gempyor_pkg/src/gempyor/config_validator.py
emprzy Oct 18, 2024
4ca01af
Update flepimop/gempyor_pkg/src/gempyor/config_validator.py
emprzy Oct 18, 2024
88e310d
Update flepimop/gempyor_pkg/src/gempyor/inference.py
emprzy Oct 18, 2024
01d11a7
Update parameters.py and test_parameters_class.py
emprzy Oct 18, 2024
fd049dd
Merge branch 'HopkinsIDD:main' into gempyor_error_improvements
emprzy Nov 4, 2024
7096ef1
Updated compartments.py, config_validaor.py, inference.py, parameters…
emprzy Nov 4, 2024
ef62ec3
Merge branch 'main' into gempyor_error_improvements
emprzy Nov 6, 2024
99b4aec
Update parameters.py
emprzy Nov 6, 2024
d2f8cc2
Applying `black` to this branch
emprzy Nov 6, 2024
cde6acb
linting my code with `black`
emprzy Nov 8, 2024
7b4f6e8
Capitalization correct in a RegEx in `test_ic.py`
emprzy Nov 8, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 35 additions & 20 deletions flepimop/gempyor_pkg/src/gempyor/compartments.py
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ def __init__(self, seir_config=None, compartments_config=None, compartments_file
self.times_set += 1

if self.times_set == 0:
raise ValueError("Compartments object not set, no config or file provided")
raise ValueError("Compartments object not set, no config or file provided.")
return

def constructFromConfig(self, seir_config, compartment_config):
Expand Down Expand Up @@ -375,7 +375,9 @@ def get_comp_idx(self, comp_dict: dict, error_info: str = "no information") -> i
comp_idx = self.compartments[mask].index.values
if len(comp_idx) != 1:
raise ValueError(
f"The provided dictionary does not allow to isolate a compartment: {comp_dict} isolate {self.compartments[mask]} from options {self.compartments}. The get_comp_idx function was called by'{error_info}'."
f"The provided dictionary does not allow to isolate a compartment: {comp_dict} "
f" isolate {self.compartments[mask]} from options {self.compartments}. "
f"The get_comp_idx function was called by '{error_info}'."
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like there's a double space between the first and second line. I'm also not super familiar with the context here, but can this be edited for grammar as well. Something like: "The provided dictionary does not allow an isolated compartment: {comp_dict}. It is isolated from the options {self.compartments}. Additional info: {error_info}."?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I happen to be looking at this particular exception again for reasons unrelated to this PR and I now think this should be a LookupError: https://docs.python.org/3/library/exceptions.html#LookupError. I think that better describes the issue at hand here.

)
return comp_idx[0]

Expand All @@ -394,8 +396,7 @@ def get_transition_array(self):
if self.compartments["name"][compartment] == elem:
rc = compartment
if rc == -1:
print(self.compartments)
raise ValueError(f"Could not find {colname} defined by {elem} in compartments")
raise ValueError(f"Could not find {colname} defined by {elem} in {self.compartments}.")
emprzy marked this conversation as resolved.
Show resolved Hide resolved
transition_array[cit, it] = rc

unique_strings = []
Expand Down Expand Up @@ -424,8 +425,11 @@ def get_transition_array(self):
candidate = reduce(lambda a, b: a + "*" + b, elem)
candidate = candidate.replace(" ", "")
# candidate = candidate.replace("*1", "")
if not candidate in unique_strings:
raise ValueError("Something went wrong")
if candidate not in unique_strings:
raise ValueError(
f"Candidate '{candidate}' from 'rate' column is not in the list of unique strings. "
f"Unique strings: {unique_strings}"
emprzy marked this conversation as resolved.
Show resolved Hide resolved
)
rc = [it for it, x in enumerate(unique_strings) if x == candidate][0]
transition_array[2][it] = rc

Expand Down Expand Up @@ -464,8 +468,11 @@ def get_transition_array(self):
candidate = reduce(lambda a, b: a + "*" + b, y)
candidate = candidate.replace(" ", "")
# candidate = candidate.replace("*1", "")
if not candidate in unique_strings:
raise ValueError("Something went wrong")
if candidate not in unique_strings:
raise ValueError(
f"Proportion exponent '{candidate}' is not found in the list of unique strings. "
f"Unique strings: {unique_strings}"
emprzy marked this conversation as resolved.
Show resolved Hide resolved
)
rc = [it for it, x in enumerate(unique_strings) if x == candidate][0]
proportion_info[2][proportion_compartment_index] = rc
proportion_compartment_index += 1
Expand All @@ -490,7 +497,10 @@ def get_transition_array(self):
if self.compartments["name"][compartment] == elem3:
rc = compartment
if rc == -1:
raise ValueError(f"Could not find proportional_to {elem3} in compartments")
raise ValueError(
f"Could not find proportional_to {elem3} in compartments. "
f"Available compartments: {self.compartments}"
emprzy marked this conversation as resolved.
Show resolved Hide resolved
)
emprzy marked this conversation as resolved.
Show resolved Hide resolved

proportion_array[proportion_index] = rc
proportion_index += 1
Expand Down Expand Up @@ -554,7 +564,10 @@ def parse_parameter_strings_to_numpy_arrays_v2(self, parameters, parameter_names
f = sp.sympify(formula, locals=symbolic_parameters_namespace)
parsed_formulas.append(f)
except Exception as e:
print(f"Cannot parse formula: '{formula}' from parameters {parameter_names}")
print(
f"Cannot parse formula: '{formula}' :"
f"from parameters: {parameter_names}."
emprzy marked this conversation as resolved.
Show resolved Hide resolved
)
raise (e) # Print the error message for debugging

# the list order needs to be right.
Expand Down Expand Up @@ -600,10 +613,10 @@ def parse_parameter_strings_to_numpy_arrays(
not operators
): # empty list means all have been tried. Usually there just remains one string in string_list at that time.
raise ValueError(
f"""Could not parse string {string_list}.
This usually mean that '{string_list[0]}' is a parameter name that is not defined
or that it contains an operator that is not in the list of supported operator: ^,*,/,+,-.
The defined parameters are {parameter_names}."""
f"Could not parse string {string_list}. "
f"This usually mean that '{string_list[0]}' is a parameter name that is not defined "
emprzy marked this conversation as resolved.
Show resolved Hide resolved
f"or that it contains an operator that is not in the list of supported operators: ^,*,/,+,-. "
emprzy marked this conversation as resolved.
Show resolved Hide resolved
f"The defined parameters are {parameter_names}."
)

split_strings = [x.split(operators[0]) for x in string_list]
Expand Down Expand Up @@ -710,12 +723,14 @@ def list_access_element_safe(thing, idx, dimension=None, encapsulate_as_list=Fal
try:
return list_access_element(thing, idx, dimension, encapsulate_as_list)
except Exception as e:
print(f"Error {e}:")
print(f">>> in list_access_element_safe for {thing} at index {idx}")
print(">>> This is often, but not always because the object above is a list (there are brackets around it).")
print(">>> and in this case it is not broadcast, so if you want to it to be broadcasted, you need remove the brackets around it.")
print(f"dimension: {dimension}")
raise e
raise Exception(
f"Error {e}: "
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Saying "error" in an error message is redundant, and also unclear in this context. Maybe something like "Raised exception: e"?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

n.b., there are some other case to apply this style to.

f"in list_access_element_safe for {thing} at index {idx}. "
f"This is often, but not always because the object above is a list (there are brackets around it). "
f"and in this case it is not broadcast, so if you want to it to be broadcasted, you need remove the brackets around it. "
f"dimension: {dimension}."
)



def list_access_element(thing, idx, dimension=None, encapsulate_as_list=False):
Expand Down
63 changes: 46 additions & 17 deletions flepimop/gempyor_pkg/src/gempyor/config_validator.py
Original file line number Diff line number Diff line change
Expand Up @@ -42,17 +42,17 @@ def validate_initial_file_check(cls, values):
initial_conditions_file = values.get('initial_conditions_file')
initial_file_type = values.get('initial_file_type')
if method in {'FromFile', 'SetInitialConditions'} and not initial_conditions_file:
raise ValueError(f'Error in InitialConditions: An initial_conditions_file is required when method is {method}')
raise ValueError(f"Error in InitialConditions: An initial_conditions_file is required when method is {method}")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
raise ValueError(f"Error in InitialConditions: An initial_conditions_file is required when method is {method}")
raise ValueError(f"An initial_conditions_file is required when method is {method}.")

Saying error in an exception is redundant. The output will list the fact that this is inside the InitialConditionsConfig.validate_initial_file_check method.

if method in {'InitialConditionsFolderDraw','SetInitialConditionsFolderDraw'} and not initial_file_type:
raise ValueError(f'Error in InitialConditions: initial_file_type is required when method is {method}')
raise ValueError(f"Error in InitialConditions: initial_file_type is required when method is {method}")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
raise ValueError(f"Error in InitialConditions: initial_file_type is required when method is {method}")
raise ValueError(f"An initial_file_type is required when method is {method}.")

return values

@model_validator(mode='before')
def plugin_filecheck(cls, values):
method = values.get('method')
plugin_file_path = values.get('plugin_file_path')
if method == 'plugin' and not plugin_file_path:
raise ValueError('Error in InitialConditions: a plugin file path is required when method is plugin')
raise ValueError(f"Error in InitialConditions: a plugin file path is required when method is plugin.")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
raise ValueError(f"Error in InitialConditions: a plugin file path is required when method is plugin.")
raise ValueError(f"A plugin file path is required when method is 'plugin'.")

return values


Expand All @@ -70,19 +70,31 @@ def validate_seedingfile(cls, values):
seeding_file_type = values.get('seeding_file_type')
seeding_file = values.get('seeding_file')
if method == 'PoissonDistributed' and not lambda_file:
raise ValueError(f'Error in Seeding: A lambda_file is required when method is {method}')
raise ValueError(
f"Error in Seeding: A lambda_file is required when method is {method} "
f"Current value: {lambda_file}."
)
if method == 'FolderDraw' and not seeding_file_type:
raise ValueError('Error in Seeding: A seeding_file_type is required when method is FolderDraw')
raise ValueError(
f"Error in Seeding: A seeding_file_type is required when method is FolderDraw"
f"Current value: {seeding_file_type}."
)
if method == 'FromFile' and not seeding_file:
raise ValueError('Error in Seeding: A seeding_file is required when method is FromFile')
raise ValueError(
f"Error in Seeding: A seeding_file is required when method is FromFile "
f"Current value: {seeding_file}."
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Like before, can drop the "error in location" portion of the exception message, python will take care of that for you.

return values

@model_validator(mode='before')
def plugin_filecheck(cls, values):
method = values.get('method')
plugin_file_path = values.get('plugin_file_path')
if method == 'plugin' and not plugin_file_path:
raise ValueError('Error in Seeding: a plugin file path is required when method is plugin')
raise ValueError(
f"Error in Seeding: a plugin file path is required when method is plugin "
f"Current value: {plugin_file_path!r}. Please specify the path to the plugin file."
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Drop the "error in location" and also there is an issue with the grammar. "Current" doesn't need to be capitalized and probably can do away with the colon.

)
return values

class IntegrationConfig(BaseModel):
Expand All @@ -107,11 +119,11 @@ def check_distr(cls, values):
b = values.get('b')
if distr != 'fixed':
if not mean and not sd:
raise ValueError('Error in value: mean and sd must be provided for non-fixed distributions')
raise ValueError(f"Mean and sd must be provided for non-fixed distributions.")
if distr == 'truncnorm' and not a and not b:
raise ValueError('Error in value: a and b must be provided for truncated normal distributions')
raise ValueError(f"a and b must be provided for truncated normal distributions.")
if distr == 'fixed' and not value:
raise ValueError('Error in value: value must be provided for fixed distributions')
raise ValueError(f"Value must be provided for fixed distributions")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
raise ValueError(f"Value must be provided for fixed distributions")
raise ValueError(f"A value must be provided for fixed distributions.")

return values

class BaseParameterConfig(BaseModel):
Expand All @@ -130,7 +142,10 @@ def which_value(cls, values):
value = values.get('value') is not None
timeseries = values.get('timeseries') is not None
if value and timeseries:
raise ValueError('Error in seir::parameters: your parameter is both a timeseries and a value, please choose one')
raise ValueError(
f"Configuration error in seir::parameters: your parameter is both a timeseries and a value, please choose one. "
f"Current values - value: {values.get('value')!r}, timeseries: {values.get('timeseries')!r}."
)
return values


Expand Down Expand Up @@ -203,7 +218,11 @@ def which_source(cls, values):
incidence = values.get('incidence')
prevalence = values.get('prevalence')
if incidence and prevalence:
raise ValueError('Error in outcomes::source. Can only be incidence or prevalence, not both.')
raise ValueError(
emprzy marked this conversation as resolved.
Show resolved Hide resolved
f"Configuration error in outcomes::source."
f"Value can only be incidence or prevalence, not both."
f"Current values - incidence: {values.get('incidence')!r}, prevalence: {values.get('prevalence')!r}."
)
return values

# @model_validator(mode='before') # DOES NOT WORK
Expand Down Expand Up @@ -256,7 +275,7 @@ def check_paramfromfile_type(cls, values):
param_subpop_file = values.get('param_subpop_file') is not None

if param_from_file and not param_subpop_file:
raise ValueError(f"Error in outcome: 'param_subpop_file' is required when 'param_from_file' is True")
raise ValueError(f"Error in outcome: 'param_subpop_file' is required when 'param_from_file' is True.")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as before with the drop "error in location" and there's no need to use an f-string here.

return values

class ResampleConfig(BaseModel):
Expand Down Expand Up @@ -320,9 +339,15 @@ def verify_inference(cls, values):
inference_present = values.get('inference') is not None
start_date_groundtruth = values.get('start_date_groundtruth') is not None
if inference_present and not start_date_groundtruth:
raise ValueError('Inference mode is enabled but no groundtruth dates are provided')
raise ValueError(
f"Inference mode is enabled, but no groundtruth dates are provided."
f"Please provide groundtruth dates."
)
elif start_date_groundtruth and not inference_present:
raise ValueError('Groundtruth dates are provided but inference mode is not enabled')
raise ValueError(
f"Groundtruth dates are provided, but inference mode is not enabled."
f"Please enable inference mode."
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need for the f-strings and there's a missing space after the first period.

return values

@model_validator(mode='before')
Expand All @@ -331,13 +356,17 @@ def check_dates(cls, values):
end_date = values.get('end_date')
if start_date and end_date:
if end_date <= start_date:
raise ValueError('end_date must be greater than start_date')
raise ValueError(
f"`end_date` ({end_date}) must be later than `start_date` ({start_date}))."
)
return values

@model_validator(mode='before')
def init_or_seed(cls, values):
init = values.get('initial_conditions')
seed = values.get('seeding')
if not init or seed:
emprzy marked this conversation as resolved.
Show resolved Hide resolved
raise ValueError('either initial_conditions or seeding must be provided')
raise ValueError(
f"At least one of `initial_conditions` or `seeding` must be provided."
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

f-string is not needed here and the indentation looks a bit off?

return values
6 changes: 3 additions & 3 deletions flepimop/gempyor_pkg/src/gempyor/inference.py
Original file line number Diff line number Diff line change
Expand Up @@ -268,9 +268,9 @@ def autodetect_scenarios(config):

if len(seir_modifiers_scenarios) != 1 or len(outcome_modifiers_scenarios) != 1:
raise ValueError(
f"Inference only support configurations files with one scenario, got"
f"seir: {seir_modifiers_scenarios}"
f"outcomes: {outcome_modifiers_scenarios}"
f"Inference only supports configuration files with one scenario, recieved:\n"
f"SEIR modifiers: {seir_modifiers_scenarios}."
f"Outcomes modifiers: {outcome_modifiers_scenarios}."
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's get rid of the random new line and see if this can be worked into one to two sentences.

)

return seir_modifiers_scenarios[0], outcome_modifiers_scenarios[0]
Expand Down
Loading
Loading