review/openfred_weatherdata #47

Ludee · 2020-02-06T09:49:49Z

Tables:
Note: metadata just differs at the "resources" key.

openfred_flags -> review metadata
openfred_locations -> review metadata
openfred_series -> review metadata
openfred_timespans -> review metadata
openfred_variables -> review metadata

Additional tables:

Keep these tables?:

tables are in schema model_draft

ToDo:

Create metadata @jh-RLI
Update metadata to v1.4.1 @jh-RLI
Check license info @ronny
Create UML @gnn -> @jh-RLI will provide a simple ERM diagram
Move into climate schema @Ludee
Update and release FRED-WAM @christian-rli
validate metadata using omi @jh-RLI
- set "alignment": "horizontal" to "alignment": null to avoid omi validation error.
Add metadata to OEP openfred tables @jh-RLI

The text was updated successfully, but these errors were encountered:

jh-RLI · 2020-02-06T14:24:30Z

@Ludee I think the openfred_powerplants is missing?

jh-RLI · 2020-02-06T15:02:01Z

Spoke to @Ludee the powerplants table is not part of the weatherdata set and is not to be considered here.

jh-RLI · 2020-02-06T15:47:32Z

The metadata-strings are created for all listed tables and linked in the issue description. The license information and a short description are still missing.

rpnaut · 2020-03-05T11:00:35Z

Regarding the openFRED weather data:

They have already an entry in the WDCC:

https://cera-www.dkrz.de/WDCC/ui/cerasearch/entry?acronym=coastDat3_COSMO-CLM_MERRA2

The description and key-words are defined already there:

Name: coastDat3_COSMO-CLM_MERRA2

Project: openFRED (open feed-in time series based on a Renewable Energy Database)

Summary: This is an atmospheric hindcast for Western Europe and the North Atlantic using COSMO-CLM version 5.0 with spectral nudging from 2002-2017. MERRA2 reanalysis data are used as forcing. Additionally transient and monthly aerosol data of the MACv2 climatology are prescribed. The model uses a rotated grid with 566 x 481 grid points and a grid point distance of 0.0625 degrees, the rotated North pole is located at 162.0 W, 39.25 N.
The published data excludes the sponge zone and have 526 x 441 grid points. In rotated coordinates the published simulation data extends from 22.64 W to 10.18 E, 11.2 S to 16.3 N, in geographical coordinates this corresponds to about 12 W to 30 E, 39 N to 60 N.

institution: Helmholtz-Zentrum Geesthacht, Institute of Coastal Research, Germany

source: int2lm_131101_2.00_clm4, COSMO-CLM5.0_clm14_aerosol_gust (available at DKRZ's LTA WDCC service)

contact: http://coastmod.hzg.de

originator: Ronny Petrik

crs: EPSG:4326 ContactDr. Ronny Petrik (✉ [email protected] )

Keyword(s): COSMO-CLM, hindcast, MERRA2, openSource, regional modelling, renewable energy

rpnaut · 2020-03-05T11:12:27Z

Bei den Attributen in Temporal stimmt was nicht: End ist 2018 und Resolution ist minimal 15 Minuten und maximal 60 Minuten.

"temporal": {
29 "referenceDate": null,
30 "timeseries": {
31 "start": "2002",
32 "end": "2017",
33 "resolution": "30 min",
34 "alignment": "horizontal",
35 "aggregationType": null
36 }
37 },

rpnaut · 2020-03-05T11:16:09Z

Die Lizenzen bei den Sources sind alle auch meiner Meinung nach NULL. Public Domain und Vertragsvereinbarungen können als Lizenzen nicht eingestuft werden.

Ludee · 2020-04-30T16:01:10Z

Project: openFRED (open feed-in time series based on a Renewable Energy Database)

The correct project name is open_FRED

Ludee · 2020-04-30T16:12:59Z

Die Lizenzen bei den Sources sind alle auch meiner Meinung nach NULL. Public Domain und Vertragsvereinbarungen können als Lizenzen nicht eingestuft werden.

https://github.com/OpenEnergyPlatform/data-preprocessing/blob/review/openfred/data-review/climate.openfred_weatherdata_locations.json#L38-L151

The metadata point sources/licenses means term of use and not the juridical license. Thus if it is licensed under PD/CC0 or other "quasi-licenses" like GeoNutzV it should be documented. It's not important if it is a real license or not.

Ludee · 2020-04-30T16:15:40Z

Keyword(s): COSMO-CLM, hindcast, MERRA2, openSource, regional modelling, renewable energy

It's not Open Source nor open-source. But never ever openSource. The source code to generate the weather data is not licensed with an open license!
It is Open Data!

Ludee · 2020-04-30T16:19:00Z

source: int2lm_131101_2.00_clm4, COSMO-CLM5.0_clm14_aerosol_gust (available at DKRZ's LTA WDCC service)

What is this? Is this an additional source?

rpnaut · 2020-05-01T09:52:56Z

Okay. At first we adapt the project explanation at WDCC regarding project name and ending of simulation.

At second, we will adapt the keyword "openSource" - huge mistake and never checked by myself.

rpnaut · 2020-05-01T12:09:13Z

Die Attribution
„coastDat3_COSMO-CLM_MERRA2 - open_FRED Weatherdata“ © Helmholtz-Zentrum Geesthacht | ODC-By-1.0"

ist korrekt.

rpnaut · 2020-05-01T12:12:08Z

I cannot push to the repository. Please try to get this working with my account rpnaut.

As long as this is not fixed, I keep the files on local devices.

rpnaut · 2020-05-03T09:11:06Z

I have to say that I do not know the difference between all the json-files:

openfred_flags -> review metadata
openfred_locations -> review metadata
openfred_series -> review metadata
openfred_timespans -> review metadata
openfred_variables -> review metadata

So, I only start to review the first one.

rpnaut · 2020-05-03T19:51:28Z

source: int2lm_131101_2.00_clm4, COSMO-CLM5.0_clm14_aerosol_gust (available at DKRZ's LTA WDCC service)

This is the hint on the source code (version of the model).

Ludee · 2021-03-24T18:40:31Z

Thanks for the reminder @Bachibouzouk
This review should be finalized.
@gnn @christian-rli can you help?

jh-RLI · 2021-04-23T09:14:10Z

I have to say that I do not know the difference between all the json-files:

openfred_flags -> review metadata
openfred_locations -> review metadata
openfred_series -> review metadata
openfred_timespans -> review metadata
openfred_variables -> review metadata

So, I only start to review the first one.

That's fine, the only difference is the data model description at the key "resources": ....). Each table is described with technical details, there is also a description for each field/column. This might be worth a review. I will use the current state for now.

jh-RLI · 2021-04-23T10:30:16Z

The OEP metadata validation tool omi does not allow "alignment":"horizontal". Just "left", "middle" or "right" are valid ATM. What would be the appropriate value?

"temporal": {
        .....
        "timeseries": {
            ....
            "alignment": "horizontal",
           .....
        }

jh-RLI · 2021-04-23T10:31:43Z

There is no "publicationDate": null
Edit: Set to date of this issue (2020-02-06)

jh-RLI · 2021-05-06T15:38:42Z

OEMetadata is now available on each open_FRED table (scroll down to "Metadata Information"):
openfred_flags
openfred_locations
openfred_series
openfred_timespans
openfred_variables

Also added similar metadata to the additional tables:
openfred_feedin_wind_2016
openfred_timeseries_ch

Note:

I had to use my backend DB access to update the Metadata since there are no permissions set on the tables. I'm not sure who created the tables? Please create an "open_FRED" user group on the OEP and add all contributor. Then the table owner needs to set the group to the table permissions. By that we can update the metadata and more easy next time.

You can create the goup (add new member) and add permissions to a table using the OEP website (oep account required).

@christian-rli

Ludee · 2021-05-11T19:23:58Z

Thanks and rights are granted.

Ludee · 2021-05-11T20:09:01Z

The table with the variables is missing the units. That's an important information. @gnn

https://openenergy-platform.org/dataedit/view/climate/openfred_variables

Ludee added help wanted Extra attention is needed documentation curation annotation, maintenance and management of a data set labels Feb 6, 2020

Ludee assigned gnn, Ludee, jh-RLI and christian-rli Feb 6, 2020

Ludee added a commit that referenced this issue Feb 6, 2020

Add template for fred wetaherdata #47

78db790

Ludee added a commit that referenced this issue Feb 6, 2020

add random infos to metadata #47

296bf8a

jh-RLI linked a pull request Feb 6, 2020 that will close this issue

Review/openfred #48

Open

Ludee added a commit that referenced this issue Feb 19, 2020

Update license information #47

cd32969

Ludee added a commit that referenced this issue Apr 30, 2020

Update all FRED metadata #47

1fd4c2e

Ludee mentioned this issue Jun 17, 2020

Validate and finalise metadata htw-pv3/weather-data#24

Open

3 tasks

Ludee assigned Bachibouzouk Mar 24, 2021

steull added review Data and Metadata review v1.4 metadata created with version 1.4 labels Mar 1, 2022

steull added a commit that referenced this issue Jul 4, 2022

Update metadata to v1.5.1 #47

1cee2b6

steull added v1.5.1 metadata created with version 1.5.1 and removed v1.4 metadata created with version 1.4 labels Jul 4, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

review/openfred_weatherdata #47

review/openfred_weatherdata #47

Ludee commented Feb 6, 2020 •

edited by jh-RLI

Loading

jh-RLI commented Feb 6, 2020

jh-RLI commented Feb 6, 2020

jh-RLI commented Feb 6, 2020

rpnaut commented Mar 5, 2020

rpnaut commented Mar 5, 2020

rpnaut commented Mar 5, 2020

Ludee commented Apr 30, 2020

Ludee commented Apr 30, 2020

Ludee commented Apr 30, 2020

Ludee commented Apr 30, 2020

rpnaut commented May 1, 2020 •

edited

Loading

rpnaut commented May 1, 2020

rpnaut commented May 1, 2020

rpnaut commented May 3, 2020

rpnaut commented May 3, 2020

Ludee commented Mar 24, 2021

jh-RLI commented Apr 23, 2021

jh-RLI commented Apr 23, 2021

jh-RLI commented Apr 23, 2021 •

edited

Loading

jh-RLI commented May 6, 2021 •

edited

Loading

Ludee commented May 11, 2021

Ludee commented May 11, 2021

review/openfred_weatherdata #47

review/openfred_weatherdata #47

Comments

Ludee commented Feb 6, 2020 • edited by jh-RLI Loading

jh-RLI commented Feb 6, 2020

jh-RLI commented Feb 6, 2020

jh-RLI commented Feb 6, 2020

rpnaut commented Mar 5, 2020

rpnaut commented Mar 5, 2020

rpnaut commented Mar 5, 2020

Ludee commented Apr 30, 2020

Ludee commented Apr 30, 2020

Ludee commented Apr 30, 2020

Ludee commented Apr 30, 2020

rpnaut commented May 1, 2020 • edited Loading

rpnaut commented May 1, 2020

rpnaut commented May 1, 2020

rpnaut commented May 3, 2020

rpnaut commented May 3, 2020

Ludee commented Mar 24, 2021

jh-RLI commented Apr 23, 2021

jh-RLI commented Apr 23, 2021

jh-RLI commented Apr 23, 2021 • edited Loading

jh-RLI commented May 6, 2021 • edited Loading

Note:

Ludee commented May 11, 2021

Ludee commented May 11, 2021

Ludee commented Feb 6, 2020 •

edited by jh-RLI

Loading

rpnaut commented May 1, 2020 •

edited

Loading

jh-RLI commented Apr 23, 2021 •

edited

Loading

jh-RLI commented May 6, 2021 •

edited

Loading