Remove report from all

.gitignore

@@ -10,6 +10,7 @@ __pycache__
 *dconf
 gurobi.log
 .vscode
+.idea
 *.orig
 
 /bak
@@ -25,6 +26,7 @@ gurobi.log
 /tmp
 doc/_build
 
+
 /scripts/old
 /scripts/create_scenarios.py
 /config/create_scenarios.py
@@ -56,6 +58,14 @@ dask-worker-space/
 publications.jrc.ec.europa.eu/
 d1gam3xoknrgr2.cloudfront.net/
 
+#latex files in report/
+report/*.aux
+report/*.bbl
+report/*.blg
+report/*.log
+report/*.out
+report/*.toc
+
 *.org
 
 *.nc

.reuse/dep5

@@ -34,3 +34,7 @@ License: CC0-1.0
 Files: borg-it
 Copyright: 2017-2024 The PyPSA-Eur Authors
 License: CC0-1.0
+
+Files: report/*
+Copyright: Open Energy Transition gGmbH
+License: MIT

README.md

@@ -13,96 +13,86 @@ SPDX-License-Identifier: CC-BY-4.0
 [![REUSE status](https://api.reuse.software/badge/github.com/pypsa/pypsa-eur)](https://api.reuse.software/info/github.com/pypsa/pypsa-eur)
 [![Stack Exchange questions](https://img.shields.io/stackexchange/stackoverflow/t/pypsa)](https://stackoverflow.com/questions/tagged/pypsa)
 
-# PyPSA-Eur: A Sector-Coupled Open Optimisation Model of the European Energy System
-
-PyPSA-Eur is an open model dataset of the European energy system at the
-transmission network level that covers the full ENTSO-E area. The model is suitable both for operational studies and generation and transmission expansion planning studies.
-The continental scope and highly resolved spatial scale enables a proper description of the long-range
-smoothing effects for renewable power generation and their varying resource availability.
-
-
-
-
-The model is described in the [documentation](https://pypsa-eur.readthedocs.io)
-and in the paper
-[PyPSA-Eur: An Open Optimisation Model of the European Transmission
-System](https://arxiv.org/abs/1806.01613), 2018,
-[arXiv:1806.01613](https://arxiv.org/abs/1806.01613).
-The model building routines are defined through a snakemake workflow.
-Please see the [documentation](https://pypsa-eur.readthedocs.io/)
-for installation instructions and other useful information about the snakemake workflow.
-The model is designed to be imported into the open toolbox
-[PyPSA](https://github.com/PyPSA/PyPSA).
-
-**WARNING**: PyPSA-Eur is under active development and has several
-[limitations](https://pypsa-eur.readthedocs.io/en/latest/limitations.html) which
-you should understand before using the model. The github repository
-[issues](https://github.com/PyPSA/pypsa-eur/issues) collect known topics we are
-working on (please feel free to help or make suggestions). The
-[documentation](https://pypsa-eur.readthedocs.io/) remains somewhat patchy. You
-can find showcases of the model's capabilities in the Joule paper [The potential
-role of a hydrogen network in
-Europe](https://doi.org/10.1016/j.joule.2023.06.016), another [paper in Joule
-with a description of the industry
-sector](https://doi.org/10.1016/j.joule.2022.04.016), or in [a 2021 presentation
-at EMP-E](https://nworbmot.org/energy/brown-empe.pdf). We do not recommend to
-use the full resolution network model for simulations. At high granularity the
-assignment of loads and generators to the nearest network node may not be a
-correct assumption, depending on the topology of the underlying distribution
-grid, and local grid bottlenecks may cause unrealistic load-shedding or
-generator curtailment. We recommend to cluster the network to a couple of
-hundred nodes to remove these local inconsistencies. See the discussion in
-Section 3.4 "Model validation" of the paper.
-
-
-![PyPSA-Eur Grid Model](doc/img/elec.png)
-
-The dataset consists of:
-
-- A grid model based on a modified [GridKit](https://github.com/bdw/GridKit)
-  extraction of the [ENTSO-E Transmission System
-  Map](https://www.entsoe.eu/data/map/). The grid model contains 7072 lines
-  (alternating current lines at and above 220kV voltage level and all high
-  voltage direct current lines) and 3803 substations.
-- The open power plant database
-  [powerplantmatching](https://github.com/PyPSA/powerplantmatching).
-- Electrical demand time series from the
-  [OPSD project](https://open-power-system-data.org/).
-- Renewable time series based on ERA5 and SARAH, assembled using the [atlite tool](https://github.com/PyPSA/atlite).
-- Geographical potentials for wind and solar generators based on land use (CORINE) and excluding nature reserves (Natura2000) are computed with the [atlite library](https://github.com/PyPSA/atlite).
-
-A sector-coupled extension adds demand
-and supply for the following sectors: transport, space and water
-heating, biomass, industry and industrial feedstocks, agriculture,
-forestry and fishing. This completes the energy system and includes
-all greenhouse gas emitters except waste management and land use.
-
-This diagram gives an overview of the sectors and the links between
-them:
-
-![sector diagram](doc/img/multisector_figure.png)
-
-Each of these sectors is built up on the transmission network nodes
-from [PyPSA-Eur](https://github.com/PyPSA/pypsa-eur):
-
-![network diagram](https://github.com/PyPSA/pypsa-eur/blob/master/doc/img/base.png?raw=true)
-
-For computational reasons the model is usually clustered down
-to 50-200 nodes.
-
-Already-built versions of the model can be found in the accompanying [Zenodo
-repository](https://doi.org/10.5281/zenodo.3601881).
-
-# Contributing and Support
-We strongly welcome anyone interested in contributing to this project. If you have any ideas, suggestions or encounter problems, feel invited to file issues or make pull requests on GitHub.
--   In case of code-related **questions**, please post on [stack overflow](https://stackoverflow.com/questions/tagged/pypsa).
--   For non-programming related and more general questions please refer to the [mailing list](https://groups.google.com/group/pypsa).
--   To **discuss** with other PyPSA users, organise projects, share news, and get in touch with the community you can use the [discord server](https://discord.com/invite/AnuJBk23FU).
--   For **bugs and feature requests**, please use the [PyPSA-Eur Github Issues page](https://github.com/PyPSA/pypsa-eur/issues).
-
-# Licence
-
-The code in PyPSA-Eur is released as free software under the
-[MIT License](https://opensource.org/licenses/MIT), see [`doc/licenses.rst`](doc/licenses.rst).
-However, different licenses and terms of use may apply to the various
-input data, see [`doc/data_sources.rst`](doc/data_sources.rst).
+# EU-Flex Platform
+<img src="https://raw.githubusercontent.com/open-energy-transition/oet-website/main/assets/img/oet-logo-red-n-subtitle.png" alt="Open Energy Transition Logo" width="260" height="100" align="right">
+
+
+## Overview
+The EU-Flex Platform is an implementation of ACER's EU-wide flexibility assessment platform based on PyPSA-Eur. This project develops a comprehensive modeling tool for assessing power system flexibility needs across the European Union, as mandated by Regulation (EU) 2024/1747.
+
+## Project Scope
+This repository covers Phase 1 (Implementation) and Phase 2 (Testing) of the EU-Flex Platform development:
+
+### Phase 1: Implementation (Dec 2024 - Nov 2025)
+- Development of core modeling capabilities using PyPSA
+- Implementation of flexibility assessment functionalities
+- Integration of various data sources and power system components
+- Development of APIs and user interfaces
+- Setting up security and operational protocols
+
+### Phase 2: Testing (Dec 2025 - June 2026)
+- Comprehensive testing with complete datasets
+- Performance optimization and refinement
+- Integration testing with real-world scenarios
+- System validation and quality assurance
+- Methodology alignment and adjustments
+
+## Key Features (WIP)
+- EU-wide power system modeling
+- Flexibility needs assessment across multiple timeframes
+- Integration of renewable energy sources
+- Analysis of demand-side response and storage options
+- Cross-border capacity evaluation
+- Stochastic analysis capabilities
+
+## Repository structure
+
+* `benchmarks`: will store `snakemake` benchmarks (does not exist initially)
+* `config`: configurations used in the study
+* `cutouts`: will store raw weather data cutouts from `atlite` (does not exist initially)
+* `data`: includes input data that is not produced by any `snakemake` rule
+* `doc`: includes all files necessary to build the `readthedocs` documentation of PyPSA-Eur
+* `envs`: includes all the `mamba` environment specifications to run the workflow
+* `logs`: will store log files (does not exist initially)
+* `notebooks`: includes all the `notebooks` used for ad-hoc analysis
+* `report`: contains all files necessary to build the report; plots and result files are generated automatically
+* `rules`: includes all the `snakemake`rules loaded in the `Snakefile`
+* `resources`: will store intermediate results of the workflow which can be picked up again by subsequent rules (does not exist initially)
+* `results`: will store the solved PyPSA network data, summary files and plots (does not exist initially)
+* `scripts`: includes all the Python scripts executed by the `snakemake` rules to build the model
+
+## Technology Stack
+- [OET/PyPSA-Eur](https://github.com/open-energy-transition/pypsa-eur) used via the [OET's soft-fork strategy](https://open-energy-transition.github.io/handbook/docs/Engineering/SoftForkStrategy). The model is described in the [documentation](https://pypsa-eur.readthedocs.io) and in the paper
+[PyPSA-Eur: An Open Optimisation Model of the European Transmission System](https://arxiv.org/abs/1806.01613), 2018, [arXiv:1806.01613](https://arxiv.org/abs/1806.01613).
+- [Snakemake](https://snakemake.readthedocs.io) for workflow management
+
+## Installation and Usage
+
+### 1. Installation
+
+Clone the repository:
+
+    git clone https://github.com/open-energy-transition/{{repository}}
+
+You need [mamba](https://mamba.readthedocs.io/en/latest/) to run the analysis. Users may also prefer to use [micromamba](https://mamba.readthedocs.io/en/latest/installation/micromamba-installation.html) or [conda](https://docs.conda.io/projects/conda/en/latest/index.html). Using `mamba`, you can create an environment from within you can run it:
+
+    mamba env create -f environment.yaml
+
+Activate the newly created `{{project_short_name}}` environment:
+
+    mamba activate {{project_short_name}}
+
+### 2. Run the analysis
+
+    snakemake -call
+
+This will run all analysis steps to reproduce results and build the report.
+
+To generate a PDF of the dependency graph of all steps `resources/dag.pdf` run:
+
+    snakemake -c1 dag
+
+<sup>*</sup> Open Energy Transition (g)GmbH, Königsallee 52, 95448 Bayreuth, Germany
+
+## Access and Usage
+This platform is being developed for ACER (European Union Agency for the Cooperation of Energy Regulators). Access and usage rights are restricted according to ACER's policies and regulations.

Snakefile

@@ -54,6 +54,7 @@ include: "rules/solve_electricity.smk"
 include: "rules/postprocess.smk"
 include: "rules/validate.smk"
 include: "rules/development.smk"
+include: "rules/report.smk"
 
 
 if config["foresight"] == "overnight":

config/config.default.yaml

@@ -710,6 +710,7 @@ sector:
   biogas_upgrading_cc: false
   conventional_generation:
     OCGT: gas
+  keep_existing_capacities: false
   biomass_to_liquid: false
   biomass_to_liquid_cc: false
   electrobiofuels: false

doc/configtables/sector.csv

@@ -178,6 +178,7 @@ biomass_spatial,--,"{true, false}",Add option for resolving biomass demand regio
 biomass_transport,--,"{true, false}",Add option for transporting solid biomass between nodes
 biogas_upgrading_cc,--,"{true, false}",Add option to capture CO2 from biomass upgrading
 conventional_generation,,,Add a more detailed description of conventional carriers. Any power generation requires the consumption of fuel from nodes representing that fuel.
+keep_existing_capacities,--,"{true, false}",Keep existing conventional carriers from the power model. Defaults to false.
 biomass_to_liquid,--,"{true, false}",Add option for transforming solid biomass into liquid fuel with the same properties as oil
 biomass_to_liquid_cc,--,"{true, false}",Add option for transforming solid biomass into liquid fuel with the same properties as oil with carbon capture
 biosng,--,"{true, false}",Add option for transforming solid biomass into synthesis gas with the same properties as natural gas

doc/release_notes.rst

@@ -479,6 +479,11 @@ PyPSA-Eur 0.12.0 (30th August 2024)
 * Address various deprecations.
 
 
+* Allow running the sector model for isolated non-EU28 countries, by filling missing sectoral
+  data with defaults, average EU values or zeros, if not available.
+
+* Enable retaining exisiting conventional capacities added in the power only model for sector coupeled applications.
+
 PyPSA-Eur 0.11.0 (25th May 2024)
 =====================================
 

report/references.bib

@@ -0,0 +1,8 @@
+% SPDX-FileCopyrightText:  Open Energy Transition gGmbH
+%
+% SPDX-License-Identifier: MIT
+
+@misc{ Nobody06,
+       author = "Nobody Jr",
+       title = "My Article",
+       year = "2006" }