Options for regional equity and autarky (PyPSA#166)

* solve: add option for equity constraints

* solve: scale more

* prepare: add option to limit line/link capacity

* solve: add inflow to EQ constraints

* solve: reindex inflow to match load

* update config files and references

* add autarky option

* move release notes

* add spillage to equity requirements

* prepare: fix accidental code removal

* prepare: add country autarky option

* consider snapshot_weightings for inflow

* trigger CI

config.default.yaml

@@ -152,11 +152,13 @@ lines:
     300.: "Al/St 240/40 3-bundle 300.0"
     380.: "Al/St 240/40 4-bundle 380.0"
   s_max_pu: 0.7
+  s_nom_max: .inf
   length_factor: 1.25
   under_construction: 'zero' # 'zero': set capacity to zero, 'remove': remove, 'keep': with full capacity
 
 links:
   p_max_pu: 1.0
+  p_nom_max: .inf
   include_tyndp: true
   under_construction: 'zero' # 'zero': set capacity to zero, 'remove': remove, 'keep': with full capacity
 

config.tutorial.yaml

@@ -130,11 +130,13 @@ lines:
     300.: "Al/St 240/40 3-bundle 300.0"
     380.: "Al/St 240/40 4-bundle 380.0"
   s_max_pu: 0.7
+  s_nom_max: .inf
   length_factor: 1.25
   under_construction: 'zero' # 'zero': set capacity to zero, 'remove': remove, 'keep': with full capacity
 
 links:
   p_max_pu: 1.0
+  p_nom_max: .inf
   include_tyndp: true
   under_construction: 'zero' # 'zero': set capacity to zero, 'remove': remove, 'keep': with full capacity
 

doc/configtables/lines.csv

@@ -1,5 +1,6 @@
 ,Unit,Values,Description
 types,--,"Values should specify a `line type in PyPSA <https://pypsa.readthedocs.io/en/latest/components.html#line-types>`_. Keys should specify the corresponding voltage level (e.g. 220., 300. and 380. kV)","Specifies line types to assume for the different voltage levels of the ENTSO-E grid extraction. Should normally handle voltage levels 220, 300, and 380 kV"
 s_max_pu,--,"Value in [0.,1.]","Correction factor for line capacities (``s_nom``) to approximate :math:`N-1` security and reserve capacity for reactive power flows"
+s_nom_max,MW,"float","Global upper limit for the maximum capacity of each extendable line."
 length_factor,--,float,"Correction factor to account for the fact that buses are *not* connected by lines through air-line distance."
 under_construction,--,"One of {'zero': set capacity to zero, 'remove': remove completely, 'keep': keep with full capacity}","Specifies how to handle lines which are currently under construction."

doc/configtables/links.csv

@@ -1,4 +1,5 @@
 ,Unit,Values,Description
 p_max_pu,--,"Value in [0.,1.]","Correction factor for link capacities ``p_nom``."
+p_nom_max,MW,"float","Global upper limit for the maximum capacity of each extendable DC link."
 include_tyndp,bool,"{'true', 'false'}","Specifies whether to add HVDC link projects from the `TYNDP 2018 <https://tyndp.entsoe.eu/tyndp2018/projects/>`_ which are at least in permitting."
 under_construction,--,"One of {'zero': set capacity to zero, 'remove': remove completely, 'keep': keep with full capacity}","Specifies how to handle lines which are currently under construction."

doc/configtables/opts.csv

@@ -3,6 +3,8 @@ Trigger, Description, Definition, Status
 ``Co2L``, Add an overall absolute carbon-dioxide emissions limit configured in ``electricity: co2limit``. If a float is appended an overall emission limit relative to the emission level given in ``electricity: co2base`` is added (e.g. ``Co2L0.05`` limits emissisions to 5% of what is given in ``electricity: co2base``), ``prepare_network``: `add_co2limit() <https://github.com/PyPSA/pypsa-eur/blob/6b964540ed39d44079cdabddee8333f486d0cd63/scripts/prepare_network.py#L19>`_ and its `caller <https://github.com/PyPSA/pypsa-eur/blob/6b964540ed39d44079cdabddee8333f486d0cd63/scripts/prepare_network.py#L154>`_, In active use
 ``Ep``, Add cost for a carbon-dioxide price configured in ``costs: emission_prices: co2`` to ``marginal_cost`` of generators (other emission types listed in ``network.carriers`` possible as well), ``prepare_network``: `add_emission_prices() <https://github.com/PyPSA/pypsa-eur/blob/6b964540ed39d44079cdabddee8333f486d0cd63/scripts/prepare_network.py#L24>`_ and its `caller <https://github.com/PyPSA/pypsa-eur/blob/6b964540ed39d44079cdabddee8333f486d0cd63/scripts/prepare_network.py#L158>`_, In active use
 ``CCL``, Add minimum and maximum levels of generator nominal capacity per carrier for individual countries. These can be specified in the file linked at ``electricity: agg_p_nom_limits`` in the configuration. File defaults to ``data/agg_p_nom_minmax.csv``., ``solve_network``, In active use
+``EQ``, "Require each country or node to on average produce a minimal share of its total consumption itself. Example: ``EQ0.5c`` demands each country to produce on average at least 50% of its consumption; ``EQ0.5`` demands each node to produce on average at least 50% of its consumption.", ``solve_network``, In active use
+``ATK``, "Require each node to be autarkic. Example: ``ATK`` removes all lines and links. ``ATKc`` removes all cross-border lines and links.", ``prepare_network``, In active use
 ``BAU``, Add a per-``carrier`` minimal overall capacity; i.e. at least ``40GW`` of ``OCGT`` in Europe; configured in ``electricity: BAU_mincapacities``, ``solve_network``: `add_opts_constraints() <https://github.com/PyPSA/pypsa-eur/blob/6b964540ed39d44079cdabddee8333f486d0cd63/scripts/solve_network.py#L66>`_, Untested
 ``SAFE``, Add a capacity reserve margin of a certain fraction above the peak demand to which renewable generators and storage do *not* contribute. Ignores network., ``solve_network`` `add_opts_constraints() <https://github.com/PyPSA/pypsa-eur/blob/6b964540ed39d44079cdabddee8333f486d0cd63/scripts/solve_network.py#L73>`_, Untested
 ``carrier+factor``, "Alter the capital cost of a carrier by a factor. Example: ``solar+0.5`` reduces the capital cost of solar to 50\% of original values.", ``prepare_network``, In active use

doc/configuration.rst

@@ -18,7 +18,7 @@ Top-level configuration
 
 .. literalinclude:: ../config.default.yaml
    :language: yaml
-   :lines: 1-8,17,24-30
+   :lines: 5-12,21,28-34
 
 .. csv-table::
    :header-rows: 1
@@ -50,7 +50,7 @@ An exemplary dependency graph (starting from the simplification rules) then look
 
 .. literalinclude:: ../config.default.yaml
    :language: yaml
-   :lines: 10-15
+   :lines: 14-19
 
 .. csv-table::
    :header-rows: 1
@@ -66,7 +66,7 @@ Specifies the temporal range to build an energy system model for as arguments to
 
 .. literalinclude:: ../config.default.yaml
    :language: yaml
-   :lines: 19-22
+   :lines: 23-26
 
 .. csv-table::
    :header-rows: 1
@@ -80,7 +80,7 @@ Specifies the temporal range to build an energy system model for as arguments to
 
 .. literalinclude:: ../config.default.yaml
    :language: yaml
-   :lines: 32-50
+   :lines: 36-54
 
 .. csv-table::
    :header-rows: 1
@@ -97,7 +97,7 @@ Specifies the temporal range to build an energy system model for as arguments to
 
 .. literalinclude:: ../config.default.yaml
    :language: yaml
-   :lines: 57-70
+   :lines: 61-74
 
 .. csv-table::
    :header-rows: 1
@@ -114,7 +114,7 @@ Specifies the temporal range to build an energy system model for as arguments to
 
 .. literalinclude:: ../config.default.yaml
    :language: yaml
-   :lines: 72-89
+   :lines: 76-93
 
 .. csv-table::
    :header-rows: 1
@@ -126,7 +126,7 @@ Specifies the temporal range to build an energy system model for as arguments to
 
 .. literalinclude:: ../config.default.yaml
    :language: yaml
-   :lines: 72,90-102
+   :lines: 76,94-106
 
 .. csv-table::
    :header-rows: 1
@@ -138,7 +138,7 @@ Specifies the temporal range to build an energy system model for as arguments to
 
 .. literalinclude:: ../config.default.yaml
    :language: yaml
-   :lines: 72,103-116
+   :lines: 76,107-120
 
 .. csv-table::
    :header-rows: 1
@@ -150,7 +150,7 @@ Specifies the temporal range to build an energy system model for as arguments to
 
 .. literalinclude:: ../config.default.yaml
    :language: yaml
-   :lines: 72,117-136
+   :lines: 76,121-140
 
 .. csv-table::
    :header-rows: 1
@@ -162,7 +162,7 @@ Specifies the temporal range to build an energy system model for as arguments to
 
 .. literalinclude:: ../config.default.yaml
    :language: yaml
-   :lines: 72,137-143
+   :lines: 76,141-147
 
 .. csv-table::
    :header-rows: 1
@@ -176,7 +176,7 @@ Specifies the temporal range to build an energy system model for as arguments to
 
 .. literalinclude:: ../config.default.yaml
    :language: yaml
-   :lines: 145-152
+   :lines: 149-157
 
 .. csv-table::
    :header-rows: 1
@@ -190,7 +190,7 @@ Specifies the temporal range to build an energy system model for as arguments to
 
 .. literalinclude:: ../config.default.yaml
    :language: yaml
-   :lines: 154-157
+   :lines: 159-163
 
 .. csv-table::
    :header-rows: 1
@@ -204,7 +204,7 @@ Specifies the temporal range to build an energy system model for as arguments to
 
 .. literalinclude:: ../config.default.yaml
    :language: yaml
-   :lines: 159-162
+   :lines: 165-168
 
 .. csv-table::
    :header-rows: 1
@@ -218,7 +218,7 @@ Specifies the temporal range to build an energy system model for as arguments to
 
 .. literalinclude:: ../config.default.yaml
    :language: yaml
-   :lines: 164-165
+   :lines: 170-171
 
 .. csv-table::
    :header-rows: 1
@@ -232,7 +232,7 @@ Specifies the temporal range to build an energy system model for as arguments to
 
 .. literalinclude:: ../config.default.yaml
    :language: yaml
-   :lines: 167-179
+   :lines: 173-185
 
 .. csv-table::
    :header-rows: 1
@@ -254,7 +254,7 @@ Specifies the temporal range to build an energy system model for as arguments to
 
 .. literalinclude:: ../config.default.yaml
    :language: yaml
-   :lines: 181-189
+   :lines: 187-197
 
 .. csv-table::
    :header-rows: 1
@@ -266,7 +266,7 @@ Specifies the temporal range to build an energy system model for as arguments to
 
 .. literalinclude:: ../config.default.yaml
    :language: yaml
-   :lines: 181,190-206
+   :lines: 187,198-214
 
 .. csv-table::
    :header-rows: 1
@@ -280,7 +280,7 @@ Specifies the temporal range to build an energy system model for as arguments to
 
 .. literalinclude:: ../config.default.yaml
    :language: yaml
-   :lines: 208-342
+   :lines: 216-355
 
 .. csv-table::
    :header-rows: 1

doc/release_notes.rst

@@ -11,6 +11,12 @@ Release Notes
 Upcoming Release
 ================
 
+* An option is introduced which adds constraints such that each country or node produces on average a minimal share of its total consumption itself.
+  For example ``EQ0.5c`` set in the ``{opts}`` wildcard requires each country to produce on average at least 50% of its consumption. Additionally,
+  the option ``ATK`` requires autarky at each node and removes all means of power transmission through lines and links. ``ATKc`` only removes
+  cross-border transfer capacities. Moreover, line and link capacities can be capped in the ``config.yaml`` at
+  ``lines: s_nom_max:`` and ``links: p_nom_max`` (`#166 <https://github.com/PyPSA/pypsa-eur/pull/166>`_).
+
 * Added an option to alter the capital cost of carriers by a factor via ``carrier+factor`` in the ``{opts}`` wildcard. This can be useful for exploring uncertain cost parameters. Example: ``solar+0.5`` reduces the capital cost of solar to 50% of original values (`#167 <https://github.com/PyPSA/pypsa-eur/pull/167>`_).
 
 * Add compatibility for pyomo 5.7.0 in :mod:`cluster_network` and :mod:`simplify_network`.
@@ -62,7 +68,6 @@ PyPSA-Eur 0.2.0 (8th June 2020)
 
 * Updated ``conda`` environment regarding ``pypsa``, ``pyproj``, ``gurobi``, ``lxml``. This release requires PyPSA v0.17.0.
 
-
 PyPSA-Eur 0.1.0 (9th January 2020)
 ==================================
 

doc/tutorial.rst

@@ -47,47 +47,47 @@ The model can be adapted to only include selected countries (e.g. Germany) inste
 
 .. literalinclude:: ../config.tutorial.yaml
    :language: yaml
-   :lines: 16
+   :lines: 20
 
 Likewise, the example's temporal scope can be restricted (e.g. to a single month).
 
 .. literalinclude:: ../config.tutorial.yaml
    :language: yaml
-   :lines: 18-21
+   :lines: 22-25
 
 It is also possible to allow less or more carbon-dioxide emissions. Here, we limit the emissions of Germany 100 Megatonnes per year.
 
 .. literalinclude:: ../config.tutorial.yaml
    :language: yaml
-   :lines: 33
+   :lines: 35,37
 
 PyPSA-Eur also includes a database of existing conventional powerplants.
 We can select which types of powerplants we like to be included with fixed capacities:
 
 .. literalinclude:: ../config.tutorial.yaml
    :language: yaml
-   :lines: 47
+   :lines: 35,51
 
 To accurately model the temporal and spatial availability of renewables such as wind and solar energy, we rely on historical weather data.
 It is advisable to adapt the required range of coordinates to the selection of countries.
 
 .. literalinclude:: ../config.tutorial.yaml
    :language: yaml
-   :lines: 49-57
+   :lines: 53-61
 
 We can also decide which weather data source should be used to calculate potentials and capacity factor time-series for each carrier.
 For example, we may want to use the ERA-5 dataset for solar and not the default SARAH-2 dataset.
 
 .. literalinclude:: ../config.tutorial.yaml
    :language: yaml
-   :lines: 59,102-103
+   :lines: 63,106-107
 
 Finally, it is possible to pick a solver. For instance, this tutorial uses the open-source solvers CBC and Ipopt and does not rely
 on the commercial solvers Gurobi or CPLEX (for which free academic licenses are available).
 
 .. literalinclude:: ../config.tutorial.yaml
    :language: yaml
-   :lines: 158,167-168
+   :lines: 164,175-176
 
 .. note::
 
@@ -271,7 +271,7 @@ the wildcards given in ``scenario`` in the configuration file ``config.yaml`` ar
 
 .. literalinclude:: ../config.tutorial.yaml
    :language: yaml
-   :lines: 7-12
+   :lines: 13-18
 
 In this example we would not only solve a 6-node model of Germany but also a 2-node model.
 

scripts/prepare_network.py

@@ -149,6 +149,27 @@ def average_every_nhours(n, offset):
 
     return m
 
+def enforce_autarky(n, only_crossborder=False):
+    if only_crossborder:
+        lines_rm = n.lines.loc[
+                        n.lines.bus0.map(n.buses.country) !=
+                        n.lines.bus1.map(n.buses.country)
+                    ].index
+        links_rm = n.links.loc[
+                        n.links.bus0.map(n.buses.country) !=
+                        n.links.bus1.map(n.buses.country)
+                    ].index
+    else:
+        lines_rm = n.lines.index
+        links_rm = n.links.index
+    n.mremove("Line", lines_rm)
+    n.mremove("Link", links_rm)
+
+def set_line_nom_max(n):
+    s_nom_max_set = snakemake.config["lines"].get("s_nom_max,", np.inf)
+    p_nom_max_set = snakemake.config["links"].get("p_nom_max", np.inf)
+    n.lines.s_nom_max.clip(upper=s_nom_max_set, inplace=True)
+    n.links.p_nom_max.clip(upper=p_nom_max_set, inplace=True)
 
 if __name__ == "__main__":
     if 'snakemake' not in globals():
@@ -200,4 +221,11 @@ def average_every_nhours(n, offset):
     ll_type, factor = snakemake.wildcards.ll[0], snakemake.wildcards.ll[1:]
     set_transmission_limit(n, ll_type, factor, Nyears)
 
+    set_line_nom_max(n)
+
+    if "ATK" in opts:
+        enforce_autarky(n)
+    elif "ATKc" in opts:
+        enforce_autarky(n, only_crossborder=True)
+
     n.export_to_netcdf(snakemake.output[0])

scripts/solve_network.py

@@ -90,6 +90,7 @@
 
 import numpy as np
 import pandas as pd
+import re
 
 import pypsa
 from pypsa.linopf import (get_var, define_constraints, linexpr, join_exprs,
@@ -167,6 +168,34 @@ def add_CCL_constraints(n, config):
                                            '<=', maximum, 'agg_p_nom', 'max')
 
 
+def add_EQ_constraints(n, o, scaling=1e-1):
+    float_regex = "[0-9]*\.?[0-9]+"
+    level = float(re.findall(float_regex, o)[0])
+    if o[-1] == 'c':
+        ggrouper = n.generators.bus.map(n.buses.country)
+        lgrouper = n.loads.bus.map(n.buses.country)
+        sgrouper = n.storage_units.bus.map(n.buses.country)
+    else:
+        ggrouper = n.generators.bus
+        lgrouper = n.loads.bus
+        sgrouper = n.storage_units.bus
+    load = n.snapshot_weightings @ \
+           n.loads_t.p_set.groupby(lgrouper, axis=1).sum()
+    inflow = n.snapshot_weightings @ \
+             n.storage_units_t.inflow.groupby(sgrouper, axis=1).sum()
+    inflow = inflow.reindex(load.index).fillna(0.)
+    rhs = scaling * ( level * load - inflow )
+    lhs_gen = linexpr((n.snapshot_weightings * scaling,
+                       get_var(n, "Generator", "p").T)
+              ).T.groupby(ggrouper, axis=1).apply(join_exprs)
+    lhs_spill = linexpr((-n.snapshot_weightings * scaling,
+                         get_var(n, "StorageUnit", "spill").T)
+                ).T.groupby(sgrouper, axis=1).apply(join_exprs)
+    lhs_spill = lhs_spill.reindex(lhs_gen.index).fillna("")
+    lhs = lhs_gen + lhs_spill
+    define_constraints(n, lhs, ">=", rhs, "equity", "min")
+
+
 def add_BAU_constraints(n, config):
     mincaps = pd.Series(config['electricity']['BAU_mincapacities'])
     lhs = (linexpr((1, get_var(n, 'Generator', 'p_nom')))
@@ -211,6 +240,9 @@ def extra_functionality(n, snapshots):
         add_SAFE_constraints(n, config)
     if 'CCL' in opts and n.generators.p_nom_extendable.any():
         add_CCL_constraints(n, config)
+    for o in opts:
+        if "EQ" in o:
+            add_EQ_constraints(n, o)
     add_battery_constraints(n)