Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make a transformer to add a column w/ a standard value #3072

Closed
6 tasks done
Tracked by #2016
cmgosnell opened this issue Nov 21, 2023 · 2 comments
Closed
6 tasks done
Tracked by #2016

Make a transformer to add a column w/ a standard value #3072

cmgosnell opened this issue Nov 21, 2023 · 2 comments
Assignees
Labels
ferc1 Anything having to do with FERC Form 1

Comments

@cmgosnell
Copy link
Member

cmgosnell commented Nov 21, 2023

Tasks

this will be a bit of a weird transformer because one parameter will need to effect outcomes in many places across the transform. There will be a normal looking transform TransformParams & transform function & method that actually does the assignment of a new column. But this transform param will also need to be access in the methods/functions that muck with the table metadata and calculations.

@cmgosnell cmgosnell converted this from a draft issue Nov 21, 2023
@cmgosnell cmgosnell added the ferc1 Anything having to do with FERC Form 1 label Nov 21, 2023
@cmgosnell cmgosnell mentioned this issue Nov 21, 2023
8 tasks
@cmgosnell cmgosnell moved this from New to Backlog in Catalyst Megaproject Nov 21, 2023
@cmgosnell cmgosnell self-assigned this Feb 16, 2024
@cmgosnell cmgosnell moved this from Backlog to In progress in Catalyst Megaproject Feb 16, 2024
@cmgosnell
Copy link
Member Author

Okay in service of #3404 i learned that the calculation components are not always getting the dimension tags when we have applied them uniformly across all tables.

A lil example:

from pudl.output.ferc1 import NodeId
from pudl.etl import defs
_core_ferc1_xbrl__calculation_components = defs.load_asset_value("_core_ferc1_xbrl__calculation_components")
calc_idx = [c for c in list(NodeId._fields) if c != "table_name"] # remove tbl name bc long
parent_idx = [f"{c}_parent" for c in calc_idx]
_core_ferc1_xbrl__calculation_components[_core_ferc1_xbrl__calculation_components.xbrl_factoid_parent == "nuclear_fuel_net"][
      parent_idx + calc_idx
]

Image

I believe this dimensions are being dropped and/or not being assigned in add_calculation_corrections. But in general I've had a hard time seeing where we've added these uniform dimensions across the board and i think fixing this _correction problem will be much easier to do with a parameterized transform for this adding of uniform dimension columns.

@cmgosnell
Copy link
Member Author

this was closed via #3409

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ferc1 Anything having to do with FERC Form 1
Projects
Archived in project
Development

No branches or pull requests

1 participant