Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Comprehensive normalization utilities #147

Open
aimalz opened this issue Jan 20, 2023 · 2 comments
Open

Comprehensive normalization utilities #147

aimalz opened this issue Jan 20, 2023 · 2 comments
Assignees
Labels
bug Something isn't working enhancement New feature or request good first issue Good for newcomers

Comments

@aimalz
Copy link
Collaborator

aimalz commented Jan 20, 2023

normalize_interp1d only covers the overall integral. We should also have a generic normalization function that includes dealing with negative values (two possible ways?). This should be accompanied by a quant_gen-specific normalization method that checks that the integral between quantile locs satisfies the provided CDF values. Additionally, we need to renormalize if converting to a parameterization (e.g. interp or hist) that has a restricted range compared to the original to ensure the modified PDFs still normalize to 1.

@aimalz aimalz added bug Something isn't working good first issue Good for newcomers labels Jan 20, 2023
@aimalz aimalz self-assigned this Jan 20, 2023
@eacharles eacharles added enhancement New feature or request and removed bug Something isn't working labels Sep 12, 2023
@eacharles
Copy link
Collaborator

I'm not understanding what you want.

@aimalz
Copy link
Collaborator Author

aimalz commented Sep 14, 2023

Sorry that came out a bit scattered. Let me try that again:

Parameterizations that define a generic 1D function (e.g. spline, interp1d, etc.) rather than a function that inherently adheres to the definition of a PDF need an automatic check for and rectify violations of nonnegativity, in addition to integrability to unity. I labeled this issue as a bug because the presence of negative values or failure to integrate to unity over the provided range means that the ensembles are not actually filled with PDFs, with downstream effects including but not limited to nonsensical values for the metrics. This check should be performed when instantiating Ensembles under those parameterizations from user-provided values and when converting Ensembles to those parameterizations (from any starting parameterization, including the affected ones under different metadata values). I mentioned the limited existing functionality for integrability as a potential starting point in case it's deemed more advantageous to generalize that than to build something new.

The quantile-specific note addresses a special case of this check on the options for how to evaluate the .pdf() method, which may result in a violation of the definition of a PDF by either criteria above as well as the definition of the instance of the quantile parameterization in question via the integral between adjacent data values not necessarily being equal to the corresponding metadata values. However, it would be more appropriate for this to be a sub-item on #148 rather than lumped into this issue.

@aimalz aimalz added the bug Something isn't working label Sep 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working enhancement New feature or request good first issue Good for newcomers
Projects
None yet
Development

No branches or pull requests

2 participants