Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Write scalars as scalars and not as lists #76

Merged
merged 1 commit into from
Mar 22, 2024

Conversation

nailend
Copy link
Contributor

@nailend nailend commented Mar 15, 2024

In my current AP8 dataset I use, scalars are returns as list of scalar and also passed as list of scalars to tabular. which doesnt work.

this is a fix. I am not sure if this is actually necesary as @henhuy maybe already takes care of this in data_adapter

@nailend nailend requested a review from FelixMau March 15, 2024 09:15
@henhuy
Copy link
Contributor

henhuy commented Mar 15, 2024

I think they are grouped as period values, or not?
thougt we would refactor them to timeseries, as tabular cannot handle period values yet?

@nailend
Copy link
Contributor Author

nailend commented Mar 15, 2024

No its about single scalars like efficiencies which are constant. These would be translated in tabular to timeseries if necessary. But currently they would end up as list of lists on tabular and not as a list of values as they are in the datapackage as a list of one single value.

@FelixMau
Copy link
Contributor

If lists are found that do not match timeseries lengths they stay as they are because we cant actually know what those values are.
This issue is couriering due to unexpected data-format in BEV data

@nailend
Copy link
Contributor Author

nailend commented Mar 17, 2024

Regardless of the origin of this problem, I am pretty certain that a list of an iterable with length 1 eg. [[1,]] or [(1,)] is for no use. In my situation, data is uploaded to oep for each year as Type: float array and not as Type: float. This is therefore, a fix for wrong datatype upload.

I think they are grouped as period values, or not?
thougt we would refactor them to timeseries, as tabular cannot handle period values yet?

data-adapter-oemof does that for multiple years, but if the value for one year is a list (e.g. efficiency = [0.28]), it will be grouped as list of iterable (e.g. [[0.28,]. [0,28], ...).

I currently need this, once the uploaded data is flawless, this could be removed again.

@nailend nailend requested a review from henhuy March 17, 2024 16:23
@henhuy
Copy link
Contributor

henhuy commented Mar 18, 2024

I think the type float array is regarding a bandwidth. But in scenario data there should not be bandwidths anymore!
Thus, the APs should upload scenario data as well. In think we discussed on versioning it as s1.

@nailend
Copy link
Contributor Author

nailend commented Mar 21, 2024

I talked to Hedda. New data will take some time to be uploaded. I need this to go on with bev-development. I will therefore merge this in features/example_transport. You might revert this merge later.

Copy link
Contributor

@henhuy henhuy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like a good workaround!
I agree for merging it, as I cannot see that new data will be there in time...

@nailend nailend merged commit fabd98b into dev Mar 22, 2024
2 checks passed
@nailend nailend deleted the fix/write_scalars_not_as_lists branch March 22, 2024 08:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants