Aggiunta dataset custom_powerplants #1347
Conversation
for more information, see https://pre-commit.ci
Feature added
There was a problem hiding this comment.
Hello @ValeMTo and many thanks for the contribution! I see that you aim tailor the model for DRC, and that looks like a great work which resonates with our current priorities to increase accuracy for each part of the world.
Your implementation looks nice, and I really love some ideas you have put inside. Have added some comments which main goal is to make your contribution helpful for anyone using the code. Let me know please what you think!
The comments may look being too many but my feeling is that they can be tackled relatively easily. The main part is to align on the architecture, especially for the parts which relate to region-specific data. I'm also available for a discussion if you feel it can be useful.
Great work, and looking forward to accommodate your contribution 😄
| ,Name,Fueltype,Technology,Set,Country,Capacity,Efficiency,Duration,Volume_Mm3,DamHeight_m,StorageCapacity_MWh,DateIn,DateRetrofit,DateOut,lat,lon,EIC,projectID,bus | ||
| 1,Victoria Falls,Hydro,Run-Of-River,PP,ZM,108,,,0,0,0,1936,1936,2036,-17.9305,25.8598,,, | ||
| 2,Zilo,Hydro,Reservoir,PP,CD,110,,,0,0,0,1952,1952,2052,-10.4998,25.4631,,, | ||
| 3,Mwadingusha,Hydro,Reservoir,PP,CD,68,,,0,0,0,1928,1928,2028,-10.7452,27.2447,,, |
There was a problem hiding this comment.
Thank you for sharing the data!
On the particular implementation, I fear there may be some technical issues... Usually, it's a not very good idea to git-track the data files as it can easily lead to an inflation of the repository
Though, we would love to have the data easily available, and a solution for that can be absolutely found. One possible solution would be to create a databundle, load it to Zenodo and add a rule to retrieve this dataset if the country is requested by a user (could be also beneficial as a part of dissemination work you are doing for your study)
| Processa i dati del file Excel e li combina con il DataFrame 'hydro'. | ||
| #fulmicotone |
There was a problem hiding this comment.
I love Italian but unfortunately it's against PEP8 coding standard to have comments in the language in any other language than English (as discriminative as it sounds...) Happy to support you with translation if needed
| @@ -541,7 +662,28 @@ def attach_hydro(n, costs, ppl): | |||
| * hydro_inflow_factor | |||
| ) | |||
|
|
|||
| # fulmicotone | |||
| excel_path = r"C:\Users\Davide\pypsa-earth-project\pypsa-earth\data\African_Hydropower_Atlas_v2-0_PoliTechM.xlsx" | |||
There was a problem hiding this comment.
To make the workflow run for everyone, we need to provide the datafile. Just to make sure it's possible in principle: would opening the data be complaint with the license restrictions?
| print(p_max_pu_ror.shape) # (numero_di_righe_di_final_data, 8760) | ||
| print(p_max_pu_ror.head()) # Anteprima dei dati |
There was a problem hiding this comment.
I guess calls of print() have been needed for debug which it perfectly reasonable. However, those calls should not go to the codebase. If you think it's worth to provide a user with some information please use logging functionality for that
| @@ -462,6 +462,124 @@ def attach_conventional_generators( | |||
| n.generators.loc[idx, attr] = values | |||
|
|
|||
|
|
|||
| def process_hydropower_data(excel_path, hydro, sheet_name): | |||
There was a problem hiding this comment.
I guess that is a custom function to work with African Hydropower Atlas, right? If that's the case it could be probably designed as a dedicated rule to produce a csv-file of custom powerplants data. What do you think?
A placeholder for such a file is available in data folder and gives a hint on the data structure expected by the workflow
| def calculate_scale(gegis_load, file_path_temba): | ||
| """ |
There was a problem hiding this comment.
A great idea on having in-built calculation of the demand scaling factor! The major comment on that is that it's crucial to make the approach applicable for any country of the world. Would you be interested in that?
| # Leggi la pagina "SpecifiedAnnualDemand" del file Excel | ||
| df_temba = pd.read_excel(file_path_temba, sheet_name="SpecifiedAnnualDemand") | ||
|
|
||
| # Rimuovi eventuali spazi all'inizio e alla fine dei nomi delle colonne | ||
| df_temba.columns = df_temba.columns.astype(str).str.strip() | ||
|
|
||
| # Crea una nuova colonna chiamata 'Country' contenente solo le prime due lettere della colonna 'FUEL' | ||
| df_temba["Country"] = df_temba["FUEL"].str[:2] | ||
|
|
||
| # Rimuovi la colonna originale 'FUEL' | ||
| df_temba = df_temba.drop(columns=["FUEL"]) |
There was a problem hiding this comment.
If feels like this part can be wrapped-up into a dedicated function to parse TEMBA data. It would improve readability of the code and also allow to an easy generalisation by replacing TEMBA-parser with one relevant for a particular region
| print("scale") | ||
| print(scale) |
| @@ -187,6 +257,7 @@ def build_demand_profiles( | |||
| start_date, | |||
| end_date, | |||
| out_path, | |||
| file_path_temba, # fulmicotone | |||
There was a problem hiding this comment.
Best would be to add file_path_temba to Snakemake inputs for build_demand_profile rule. It allows for more flexible and clean file management
| # fulmicotone | ||
| file_path_temba = r"C:\Users\Davide\Downloads\temba\TEMBA_SSP1-26.xlsx" | ||
| # year = 2030 |
There was a problem hiding this comment.
To clarify the above comment on Snakemake inputs: it's currently not obvious where file_path_temba is defined, and super easy to forget modify it if anything has been changed. Having this specification in Snakemake would facilitate keeping everything in sight
|
Thank @ekatef so much for reviewing our pull request and as soon as we can, we will fix the issues that you highlighted. Would it be better to cancel this PR and make a new one once we are ready? |
Ahhh, sorry!! I though you are seeking for the feedback 😄 No worries, and the great work, in any case! Sure, feel free to close this PR 🙂 On the technical side, it's usually easier when a PR deals with a single task. For example, it makes sense to open one PR on adding calculations for the scaling factors and another one on adding custom time-series for the runoff. In case you would be interested to contribute those features, it would be very much appreciated! |
|
Don't worry, you were right to think that we were looking for feedback. Indeed, we made a PR lol Thanks and sorry again!! |
fixing PR pypsa-meets-earth#1347
Good luck with your modeling study, and looking forward to accommodate your contributions when it'll be a right time for it 😄 |
Closes # (if applicable).
Changes proposed in this Pull Request
Checklist
envs/environment.yamlanddoc/requirements.txt.config.default.yamlandconfig.tutorial.yaml.test/(note tests are changing the config.tutorial.yaml)doc/configtables/*.csvand line references are adjusted indoc/configuration.rstanddoc/tutorial.rst.doc/release_notes.rstis amended in the format of previous release notes, including reference to the requested PR.