Add chemicals industry (new) #418

tud-mchen6 · 2024-07-25T15:43:01Z

Fixes #311. This is the newer version; discarding #348. Everything works, but still in progress of prettifying.

Checklist

Any checks which are not relevant to the PR can be pre-checked by the PR creator. All others should be checked by the reviewer. You can add extra checklist items here if required by the PR.

tud-mchen6 · 2024-07-25T15:48:00Z

My high-level questions, just to confirm:

Do we only consider energy demand? During the process of extracting electricity demand (and assuming all demands that could possibly be met by electricity are met by electricity), some demands that cannot be met by electricity will be filtered out.
Do we assume that all low enthalpy heat demand cannot be met by electricity?
Regarding the "energy demand" of Basic chemicals: what carriers are they?
On the one hand, we estimate electricity demand from jrc_energy, while on the other hand, we estimate Basic chemicals related demand from jrc_prod. Will there be overlap or double-counting?

irm-codebase

Nice job! However, I have several suggestions.

Biggest one is that I feel like you are not using the correct JRC function for the electricity consumption stuff. See the specific comment. Feel free to comment if you think I got something wrong!

irm-codebase · 2024-07-25T17:25:03Z

modules/industry/industry.smk

+    message: "Calculate energy demand for the 'Chemicals Industry' sector in JRC-IDEES."
+    conda: CONDA_PATH
+    # params:
+    #     config = config["params"]["config-chemicals-industry"]


Consider looking at which parameters should be user configurable and bring them to this level.

irm-codebase · 2024-07-25T17:29:15Z

modules/industry/scripts/chemicals_industry.py

+    """
+    # Get the electricity consumption (assuming every demand that could be met by electricity is met by electricity)
+    electrical_consumption = (
+        (jrc.replace_carrier_final_demand("Electricity", jrc_energy))


I feel in my bones that this is not the function you want to call here.
Check get_section_final_intensity_auxiliary_electric, which is what the steel sector uses.

This function gets electricity demand for stuff like motor drives, pumps and lighting.

The best way to answer this is probably looking at what the old code does for both chemicals and steel here, not "other" industry, which is unique.

I feel in my bones that this is not the function you want to call here. Check get_section_final_intensity_auxiliary_electric, which is what the steel sector uses.

This function gets electricity demand for stuff like motor drives, pumps and lighting.

As I understand, the function you suggest is giving an intensity value, i.e. twh/t. But what's needed here, at least according to the old code, is the absolute value of energy in twh (just an example of unit).

irm-codebase · 2024-07-25T17:33:03Z

modules/industry/scripts/chemicals_industry.py

+    chem_co2_consumption = chem_co2_consumption.assign_attrs(units="t").assign_coords(
+        carrier_name="co2"
+    )
+    # TODO: find out how co2 is called in jrc


As far as I remember, CO2 is not present in JRC, same with "methanol" above.
Instead, follow the same naming convention JRC uses (capital letters, etc).

Yeah, this is why we provide CO2 demand as constants in a dictionary earlier in the script.

irm-codebase · 2024-07-25T17:33:29Z

modules/industry/scripts/chemicals_industry.py

+    )
+    chem_energy_consumption = chem_energy_consumption.assign_attrs(
+        units="twh"
+    ).assign_coords(carrier_name="electricity")


Please keep the same JRC naming convention. See the iron and steel sector.

irm-codebase · 2024-07-25T17:33:51Z

modules/industry/scripts/chemicals_industry.py

+        .sum(dim=["section", "carrier_name"])
+        .drop_vars(["subsection", "cat_name"])
+        .assign_attrs(units="twh")
+        .assign_coords(carrier_name="space_heat")


More naming conventions. Please keep "Low enthalpy heat".

irm-codebase · 2024-07-25T17:36:10Z

modules/industry/scripts/chemicals_industry.py

+    )
+
+    # Prettify
+    total_consumption = jrc.standardize(total_consumption, "twh, t", "demand")


Since Chemicals industry is unique in that it also outputs demand in tonnes, please create a separate xarray variable for it with a clear and short name (see the jrc_energy dataset as an example).

This is to avoid ambiguous units: right now you cannot know were twh or tonnes apply unless you ask the person who made the code!

irm-codebase · 2024-07-25T17:37:52Z

modules/industry/scripts/utils/jrc_idees_parser.py

-    useful_dem_tot = jrc_energy["useful"].sum("carrier_name")
+    useful_dem_tot = jrc_energy["useful"].sum(
+        "carrier_name"
+    )  # MC note: including all carriers


Please remove these comments.
If the function is not descriptive enough, improve the docstring.

Easier to maintain down the line (and avoids black creating extra lines because of the comment).

irm-codebase · 2024-07-25T17:38:33Z

modules/industry/scripts/utils/jrc_idees_parser.py

-    assert useful_dem_tot.sum() < carrier_final_demand.sum(), "Creating energy!"
-
+    # assert useful_dem_tot.sum() < carrier_final_demand.sum(), "Creating energy!" # MC note: but useful_dem_tot includes all carrier demands while carrier_final_demand only contains the demand related to one specific carrier. Surely this is fair comparison?
+    # MC note: commented out because this is not correct for chemicals industry. but need to coordinate with Ivan regarding how this function should be written.


See my comment in the "other industry" PR:
#355 (comment)

irm-codebase · 2024-07-25T17:58:53Z

My high-level questions, just to confirm:

1. Do we only consider **energy** demand? During the process of extracting electricity demand (and assuming all demands that could possibly be met by electricity are met by electricity), some demands that cannot be met by electricity will be filtered out.

2. Do we assume that all low enthalpy heat demand cannot be met by electricity?

3. Regarding the "energy demand" of Basic chemicals: what carriers are they?

4. On the one hand, we estimate electricity demand from `jrc_energy`, while on the other hand, we estimate Basic chemicals related demand from `jrc_prod`. Will there be overlap or double-counting?

I'll try to help here, but I am unfamiliar with the chemicals section of the code. I can only help with the jrc helper functions and so on. If there are more questions, I can help when I'm back from my vacation.

For chemicals, no. Any "energy relevant" carrier matters here. I do not remember the SCEC specifics, but these could be: ammonia, H2, CO2, etc. Please check the SCEC code and include all the ones used there for Chem. Ind.
Low enthalpy heat can be met by whatever technology the model chooses (heatpumps, boilers, wood, etc). Just use the same method as the iron and steel sector.
Check the SCEC code please. That is probably there, or in the configuration files.
Please see the SCEC code, or ask @brynpickering if you suspect double counting is occurring. My guess is "no", if you use the correct subsections.

brynpickering

THanks for this @tud-mchen6! I would restructure the chemicals industry script a bit to move constants around and to distribute it into smaller functions that you can then give nice docstrings for (for future maintainability, e.g. so others don't wonder the same things you have wondered when making this contribution!)

brynpickering · 2024-07-29T07:23:25Z

modules/industry/scripts/chemicals_industry.py

+from utils import filling
+from utils import jrc_idees_parser as jrc
+
+H2_LHV_KTOE = 2.863  # 0.0333 TWh/kt LHV -> 2.863ktoe/kt


Some of these values are also in the steel script. To avoid the maintenance burden, perhaps we move these to a single file (e.g. utils/constants.py?)

brynpickering · 2024-07-29T07:32:41Z

modules/industry/scripts/chemicals_industry.py

+    will be required for iron ore processed using H-DRI, but is not required by EAF.
+
+    This function does the following:
+    1.


brynpickering · 2024-07-29T07:33:32Z

modules/industry/scripts/chemicals_industry.py

+        "Methanol": 1.373,
+    }
+
+    energy_demand = {  # MWh/t


Move to config (there are user assumptions here)

brynpickering · 2024-07-29T07:33:56Z

modules/industry/scripts/chemicals_industry.py

+    )
+
+    # Gather relevant params for Basic chemicals in the Chemicals Industry
+    h2_demand = {  # t/t


move to top of file (they are constants)

brynpickering · 2024-07-29T07:34:06Z

modules/industry/scripts/chemicals_industry.py

+        "Ammonia": 0.178,
+        "Methanol": 0.189,
+    }
+    methanol_demand = {


move to top of file (they are constants)

brynpickering · 2024-07-29T07:34:20Z

modules/industry/scripts/chemicals_industry.py

+        "Ammonia": 17,
+        "Methanol": 2,
+    }
+    molar_mass = {


move to top of file (they are constants)

brynpickering · 2024-07-29T07:34:24Z

modules/industry/scripts/chemicals_industry.py

+        "Methanol": 1.5,  # auxiliary demand, could just be assumed as already included in existing electricity demand
+    }
+
+    mass = {  # Bazzanella and Ausfelder, 2017


move to top of file (they are constants)

brynpickering · 2024-07-29T07:34:57Z

modules/industry/scripts/chemicals_industry.py

+        "Ammonia": 17.01,
+        "Methanol": 32.04,
+    }
+    moles = {k: mass[k] / molar_mass[k] for k in mass}


I would separate the mass-to-molar mass calc into its own function

brynpickering · 2024-07-29T07:36:54Z

modules/industry/scripts/chemicals_industry.py

+    new_chemicals_demand.to_netcdf(output_path)
+
+
+def transform_jrc_chemicals_subsector_demand(


I don't think we're using "subsector" any more. See the steel script

brynpickering · 2024-07-29T07:38:54Z

modules/industry/scripts/chemicals_industry.py

+    chem_co2_consumption = chem_co2_consumption.assign_attrs(units="t").assign_coords(
+        carrier_name="co2"
+    )
+    # TODO: find out how co2 is called in jrc


Yeah, this is why we provide CO2 demand as constants in a dictionary earlier in the script.

brynpickering · 2024-07-29T07:48:52Z

In answer to your questions:

Do we only consider energy demand? During the process of extracting electricity demand (and assuming all demands that could possibly be met by electricity are met by electricity), some demands that cannot be met by electricity will be filtered out.

We completely overhaul the chemical industry stuff and as a result only keep direct electricity consumption, which refers to all energy requirements to process chemicals. I believe that the non-electricity consumption in JRC-IDEES refers to creating relevant chemicals, which we completely replace with our own demands for CO2, hydrogen, and auxiliary electricity. This probably requires some review though.

Do we assume that all low enthalpy heat demand cannot be met by electricity?

It gets turned into "space heat demand" later, which can be met by any available technologies for that.

Regarding the "energy demand" of Basic chemicals: what carriers are they?

Electricity.

On the one hand, we estimate electricity demand from jrc_energy, while on the other hand, we estimate Basic chemicals related demand from jrc_prod. Will there be overlap or double-counting?

The electricity demand from jrc_energy refers to demand to process basic chemicals. The electricity demand we get from processing jrc_prod is for creating those basic chemicals. They should be mutually exclusive sets of demands that can be added together without double counting.

tud-mchen6 added 4 commits July 18, 2024 14:59

try-out commit chemicals industry

173dee9

snakefile update

ebb36f7

a working version

0e36c35

minor fix

ac2524e

tud-mchen6 requested review from irm-codebase, brynpickering and timtroendle July 25, 2024 15:43

tud-mchen6 self-assigned this Jul 25, 2024

irm-codebase suggested changes Jul 25, 2024

View reviewed changes

brynpickering requested changes Jul 29, 2024

View reviewed changes

irm-codebase mentioned this pull request Aug 27, 2024

Industry module functionality as compared to sector-coupled euro-calliope calliope-project/ec_modules#32

Open

6 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add chemicals industry (new) #418

Add chemicals industry (new) #418

tud-mchen6 commented Jul 25, 2024

tud-mchen6 commented Jul 25, 2024

irm-codebase left a comment •

edited

Loading

irm-codebase Jul 25, 2024

irm-codebase Jul 25, 2024

irm-codebase Jul 25, 2024

tud-mchen6 Aug 27, 2024

irm-codebase Jul 25, 2024

brynpickering Jul 29, 2024

irm-codebase Jul 25, 2024

irm-codebase Jul 25, 2024 •

edited

Loading

irm-codebase Jul 25, 2024 •

edited

Loading

irm-codebase Jul 25, 2024

irm-codebase Jul 25, 2024 •

edited

Loading

irm-codebase commented Jul 25, 2024 •

edited

Loading

brynpickering left a comment

brynpickering Jul 29, 2024

brynpickering Jul 29, 2024

brynpickering Jul 29, 2024

brynpickering Jul 29, 2024

brynpickering Jul 29, 2024

brynpickering Jul 29, 2024

brynpickering Jul 29, 2024

brynpickering Jul 29, 2024

brynpickering Jul 29, 2024

brynpickering Jul 29, 2024

brynpickering commented Jul 29, 2024

		new_chemicals_demand.to_netcdf(output_path)


		def transform_jrc_chemicals_subsector_demand(

Add chemicals industry (new) #418

Are you sure you want to change the base?

Add chemicals industry (new) #418

Conversation

tud-mchen6 commented Jul 25, 2024

Checklist

tud-mchen6 commented Jul 25, 2024

irm-codebase left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

irm-codebase Jul 25, 2024 • edited Loading

Choose a reason for hiding this comment

irm-codebase Jul 25, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

irm-codebase Jul 25, 2024 • edited Loading

Choose a reason for hiding this comment

irm-codebase commented Jul 25, 2024 • edited Loading

brynpickering left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

brynpickering commented Jul 29, 2024

irm-codebase left a comment •

edited

Loading

irm-codebase Jul 25, 2024 •

edited

Loading

irm-codebase Jul 25, 2024 •

edited

Loading

irm-codebase Jul 25, 2024 •

edited

Loading

irm-codebase commented Jul 25, 2024 •

edited

Loading