PUDL v2022.11.30

See the release notes for v2022.11.30 in our docs for a more narrative explanation of what has changed.

What's Changed

This is all the PRs that were merged since the last release, excluding those made by the @dependabot and @pre-commit-ci bots.

• Apply black autoformatting by @zaneselvans in #1543
• Apply black formatting by @zaneselvans in #1548
• Update to pip 22, setuptools 61. Add nbconvert to pudl-dev by @zaneselvans in #1565
• Add installation_year and construction_year to PPL by @katie-lamb in #1554
• Modify EPA CEMS ETL to facilitate Intake Catalog by @zaneselvans in #1563
• Hub EIA transition by @cmgosnell in #1575
• Add office hours scheduling links to README by @zaneselvans in #1582
• Rename tox virtualenv dir from .env_pudl to .env_tox by @zaneselvans in #1586
• Add rstcheck to our collection of linters by @zaneselvans in #1587
• Bring in year_state_filter tests & improvements from pudl_catalog by @zaneselvans in #1589
• Update maximum allowed version of setuptools to 62. by @zaneselvans in #1590
• Use partial function in map of EPA CEMS ETL by @zaneselvans in #1591
• Additional code formatting/linting without modernizing Python syntax by @zaneselvans in #1598
• Refactor labeling of true granularities with plant part to generator match function by @katie-lamb in #1447
• Re-gigger backfilling technology_description & make prime_mover_code an annually harvested column by @cmgosnell in #1600
• Remove some seldom used dependencies from pudl-dev environment.yml by @zaneselvans in #1615
• Fix breakage resulting from dask v2022.4.2 by @zaneselvans in #1618
• Cinco de Mayo 🇲🇽 by @zaneselvans in #1616
• Clean operational_status_code using metadata encoder by @cmgosnell in #1624
• add opex_nonfuel column to all FERC1 plant tables in output layer for all_plants_ferc1 table by @aesharpe in #1626
• Add installation_year and construction_year as plant part level by @katie-lamb in #1578
• Add ML for sustainable energy citation to bibliography by @zaneselvans in #1641
• Dependabot auto merge by @zaneselvans in #1655
• Small docs updates by @aesharpe in #1642
• Dynamically generate RSTs with new DataSource metadata by @katie-lamb in #1532
• Add DataSource Metadata for EPA-EIA Crosswalk by @aesharpe in #1676
• Address issue where 861 ETL fails w/o all years of data by @arengel in #1671
• update ferc-eia glue with fixes found from the FERC plant-ID-er by @cmgosnell in #1678
• Rework clean_merge_asof func by @katie-lamb in #1550
• Switch to the Furo Sphinx theme by @zaneselvans in #1680
• Require Python 3.10 and update to modern syntax by @zaneselvans in #1685
• Release notes for date_merge and default columns change for PPL and MCOE by @katie-lamb in #1690
• Apply Yaml pre-commit formatter by @bendnorman in #1689
• Fix all plants ferc1 by @aesharpe in #1656
• Require Python 3.10 in the pudl-dev conda environment by @zaneselvans in #1697
• Draft: add capacity mw to mcoe defaults and fix row counts in validation tests for eia tables by @katie-lamb in #1695
• GCE Deploy by @zaneselvans in #1627
• Add workflow_dispatch support to nightly builds by @bendnorman in #1702
• Add build-deploy-pudl.yml to main by @bendnorman in #1703
• Fix Github Ref bug by @bendnorman in #1704
• Unpin apt-get packages by @bendnorman in #1725
• Remove unnecessary packages from build system; specify backend. by @zaneselvans in #1743
• Update setuptools numpy by @zaneselvans in #1745
• Add gcs and bypass cache args to datastore cli by @bendnorman in #1740
• Fix dependabot automerge by @zaneselvans in #1753
• Bring new bot-auto-merge workflow into main by @zaneselvans in #1756
• Move slowly varying plant attributes from entity to annual plants table by @zaneselvans in #1749
• Fix bug in gens mega and plant part list creation by @katie-lamb in #1759
• Integrate EPA CEMS hourly emissions data for 2021 by @zaneselvans in #1778
• EIA923 early release, EIA860 early release, and 860m 2022-06 by @cmgosnell in #1834
• Avoid using Shapely v1.8.3 due to upstream bug / incompatibility by @zaneselvans in #1848
• Update eia923 raw inputs to include revisions made by EIA on 2022-08-11 by @zaneselvans in #1846
• Add missing columns and update EIA860, EIA860M and EIA923 data for 2021 by @cmgosnell in #1836
• Use gcs cache in ci by @zaneselvans in #1858
• Patch nightly build flakiness by @bendnorman in #1856
• Create a data_maturity label for EIA data by @cmgosnell in #1855
• Update bug report issue template by @zaneselvans in #1869
• Update ETL settings files to work with XBRL+DBF and new Ferc1Settings by @cmgosnell in #1886
• Xbrl steam but really by @cmgosnell in #1881
• Use internal zenodo-cache bucket for nightly builds by @bendnorman in #1880
• Encode balancing authority codes by @cmgosnell in #1897
• Fill in some null BA codes using BA names by @cmgosnell in #1906
• Implement drop_invalid_rows() for fuel_ferc1 table by @zaneselvans in #1903
• Split TableTransformer.transform() into 3 phases by @zaneselvans in #1900
• Prepare raw FERC XBRL DB's for publication with Datasette by @zschira in #1831
• Aggregate data_maturity in gfn_eia923, update EIA ETL debugging Notebook by @zaneselvans in #1915
• Use provision-micromamba and remove ferc1_solo ETL to speed up CI. by @zaneselvans in #1913
• Fill in pre-2013 BA codes by @cmgosnell in #1911
• Update the name of the EPA CAMD to EIA crosswalk data source. by @zaneselvans in #1918
• Add metadata & DOIs for EIA Bulk Electricity data source by @zaneselvans in #1922
• Integrate EIA-861 2021 Early Release data by @zaneselvans in #1921
• Updating 861 package_data for 2021 early release by @arengel in #1920
• Add epacems crosswalk to etl by @aesharpe in #1692
• Re-add and update the epacamd-eia crosswalk analysis module by @aesharpe in #1934
• Add updated crosswalk analysis back into dev by @aesharpe in #1938
• Ensure PUDL works with Pandas 1.5.0 by @zaneselvans in #1902
• Plant part updates to fix RMI CI memory issues by @katie-lamb in #1865
• Fix build error for epacamd_eia_test by @aesharpe in #1940
• Create github action to sync zenodo caches by @bendnorman in #1935
• Rollback pandas-1.5 by @bendnorman in #1945
• Automate docstring formatting by @zaneselvans in #1948
• New FERC 1 / XBRL Transform Unit Tests by @zaneselvans in #1919
• Add a static table of political subdivision attributes. by @zaneselvans in #1966
• Add release notes about the new political_subdivisions table. by @zaneselvans in #1967
• Integrate xbrl to dbf utility ID mapping by @cmgosnell in #1931
• Refactor FERC 1 transform; integrate XBRL for fuel & steam tables by @cmgosnell in #1721
• Remove derelict docker build workflow file by @zaneselvans in #2002
• Add labeling functions for manual maps by @cmgosnell in #1988
• EIA API Replacement by @TrentonBush in #1998
• EIA Bulk Electricity Aggregates by @TrentonBush in #1937
• Add 2021 861 to output by @aesharpe in #1990
• Integrating 2021 FERC1 XBRL hydro table by @cmgosnell in #1992
• Document new transforms & organize imports/logging by @zaneselvans in #1962
• Add release notes about EIA API removal / fuel price filling by @zaneselvans in #2008
• Remove EXPERIMENTAL from datasette title by @bendnorman in #2010
• Boiler Fuel Allocation Improvements by @grgmiller in #1608
• Bump allowed geopandas & pygeos conda versions by @zaneselvans in #2019
• Add 2021 eia923 eia860 final release by @aesharpe in #2017
• Docs housekeeping by @zaneselvans in #2039
• Add a draft PR template. by @zaneselvans in #2038
• Fix erroneous max row values in mcoe_test by @aesharpe in #2041
• Allocate fuel consumption by @cmgosnell in #1096
• Integrate pudl ID's from eia final updates into the xbrl branch by @cmgosnell in #2042
• Updated docs to include nightly build failure etiquette. by @zaneselvans in #2044
• remove pesky rounds on floats by @cmgosnell in #2052
• xbrl and dbf integration of pumped storage ferc1 table by @cmgosnell in #2005
• xbrl and dbf integration of purchased power ferc1 table by @cmgosnell in #2011
• First draft of plant_in_service_ferc1 DBF + XBRL transform by @zaneselvans in #2025
• Stop warning about duplicate record IDs & missing columns that don't matter by @zaneselvans in #2054
• track down and fix warnings in xbrl transform by @cmgosnell in #2057
• Fix bad multi-index construction that was scrambling XBRL columns. by @zaneselvans in #2059
• Transforms for small gens table XBRL by @aesharpe in #2035
• Integrate XBRL taxonomy metadata into plant_in_service transform by @zaneselvans in #2058
• add release notes for xbrl integration by @cmgosnell in #2063
• Xbrl nov archive by @cmgosnell in #2065
• Fix doc stings for small gens table transform by @aesharpe in #2068
• Resolve duplicate DBF-XBRL mappings and duplicate records in plant_in_service_ferc1 by @zaneselvans in #2067
• accumulation of any changes in the xbrl validation tests by @cmgosnell in #2071
• Initial FERC Form 1 XBRL Integration by @zschira in #1665
• Address missing fuel ferc1 records by @zaneselvans in #2092
• Add loglevel and logfile to all CLI commands by @bendnorman in #2091
• Add FERC XBRL DBs to datasette deployment. by @zaneselvans in #2095
• Release 2022.11.XX by @zaneselvans in #1681
• Add GCP region/zone env var to nightly build scripts by @bendnorman in #2101
• Packaging & release note tweaks for PUDL v2022.11.30 by @zaneselvans in #2098
• fix missing generator_id by @grgmiller in #2122
• v2022.11.30 by @zaneselvans in #2135

New Contributors

• @arengel made their first contribution in #1671
• @grgmiller made their first contribution in #1608

Full Changelog: v0.6.0...v2022.11.30