[energidataservice] Initial contribution

[jlaur]

This binding integrates the Danish service Energi Data Service, making data available as:

• Channels with current prices.
• An advanced channel with future prices in JSON format.
• Thing action for importing the prices directly into rules.

In this initial version, electricity prices are in focus, specifically the datasets Elspotprices and DatahubPricelist.

The binding is mainly targeting Denmark because of the tariffs and taxes provided which are specific to the Danish market and in currency DKK. However, spot prices can also be retrieved for other regions, for example Norway and Sweden.

Resolves #14222

Prerequisites

• Gson 2.10 is required for deserializing Java records. This dependency currently adds ~300 kB to the size. Hopefully Gson can be upgraded in core before 4.0 - see Upgrade Gson to 2.10 openhab-core#3321

Features

• Optimized service calls: Spot prices are retrieved once per day and tariffs only upon expiry (usually months/years).
• Error handing: Policies for retry strategies are in place to make sure data will be retrieved as soon as possible after failed calls. This includes exponential back-off with jitter.
• Grid company is selectable in configuration.
• Pre-configured filters are included for all known grid companies.
• Filters can be overridden by channel configuration.
• Spot prices and all tariffs are supported.
• Properties shows number of remaining calls as well as timestamp of last/next call.
• Actions for getting prices and performing calculations.

Note about VAT

See #14222 (comment)

#14529 is now merged, and everything related to VAT has been removed from the binding.

Testing

• 4.0: https://community.openhab.org/t/energi-data-service-binding-4-0-0-0-4-1-0-0/144370 (latest)
• 3.4: https://community.openhab.org/t/energi-data-service-binding-3-4-0-0-4-0-0-0/144381 (backported 27.01.2023, last updated 18.04.2023 23:11)

I have the 3.4.x backport running in my productive system with rules for automation based on the provided thing actions:
https://community.openhab.org/t/dishwasher-price-calculation-automation/139207/4

Further plans

I've been working on this extensively for a few weeks and all planned MVP features are now implemented. Therefore, I feel the time is right for publishing this pull request. Major changes at this stage are not planned, but can of course be triggered by pull request comments. 😉

A few notes on areas I might revisit on short term:

• Dynamic default value for configuration parameter, if possible: Currently the user has to select price area based on location, either DK1 for West of the Great Belt or DK2 for East of the Great Belt. I would like the value to be defaulted to a calculated value based on the position configured in openHAB's regional settings.
• With dynamic default configuration parameters, currency could also be defaulted based on Locale.
• Channel configuration: Offset for date query parameter filter override.
• Reconsider ways of making the data useful. Check Tibber and aWATTar.

On longer term, gas data could also be added, if there would be an interest in that.

Future

See openhab/openhab-core#3478 for discussions of more generic character.

[masipila]

I did not test this as as I currently have one openHab installation which controls the heating of our house so I did not want to goof with that. However, I read through your code, which seems quite well thought and comprehensive.

By reading the code, I can see that this stores the spot prices as JSON array:
channel-type.energidataservice.hourlyPrices.description = JSON array with hourly prices from 12 hours ago and onward.

Since this is not a proper time series, what is your approach for the basic need of creating a chart of the spot prices? Or is the intention that you will have a Rule that will read the spot prices from this JSON string, parse them and persist them as a real time series so that they can be plotted as a chart (among other needs for a proper time series)?

I mean a chart like this, where the blue area represents the spot prices and the yellow bars indicate the control points (hours) when house heating is allowed to be on.

Cheers,
Markus

Edit: Added an example chart

[jlaur]

I read through your code, which seems quite well thought and comprehensive.

Thanks!

By reading the code, I can see that this stores the spot prices as JSON array: channel-type.energidataservice.hourlyPrices.description = JSON array with hourly prices from 12 hours ago and onward.

Since this is not a proper time series, what is your approach for the basic need of creating a chart of the spot prices?

The binding is only a data provider. Charts can be created using other framework components such as persistence. Unfortunately it's currently not possible to provide data with a given timestamp (back and forward in time), so out of the box it will not be possible to create a chart of future prices. You can see historic prices by persisting CurrentSpotPrice, for example.

Persisting of historic and future prices would need to be resolved with openhab/openhab-core#3000 or something similar.

Or is the intention that you will have a Rule that will read the spot prices from this JSON string, parse them and persist them as a real time series so that they can be plotted as a chart (among other needs for a proper time series)?

The intention was to have some way of exposing future prices, and I didn't want to create tons of channels representing different hours. Initially I used this JSON array in rules to perform calculations. However, this could later be replaced by the action getPrices which is much faster. And finally, I created some calculation actions, so in my own current use cases, the calculations can be performed by the binding, and just called by rules. See https://community.openhab.org/t/dishwasher-price-calculation-automation/139207/4

I mean a chart like this, where the blue area represents the spot prices and the yellow bars indicate the control points (hours) when house heating is allowed to be on.

It looks cool, but this is a problem to be solved outside of the binding IMHO.

[masipila]

I agree that fetching the data from a 3rd party and persisting it are two different topics.

I just left this comment to the PR #3000. If the conclusion over there will be that bindings must not be able to persist data with future timestamps then I guess all that a binding like this can do is to fetch the data and store it as a serialized JSON array like you are doing here or alternatively have gazillion channels for different timestamps.

And to address the IMHO obvious need to persist that data, there would then have to be for example a Rule which reads the serialized array (or gazillion channels with different timestamps) and then persists the data independently from the Binding that fetched it.

Did I get this right?

Cheers,
Markus

Edit: Fixed the link to my comment.

[jlaur]

I just left this comment to the PR #3000. If the conclusion over there will be that bindings must not be able to persist data with future timestamps then I guess all that a binding like this can do is to fetch the data and store it as a serialized JSON array like you are doing here or alternatively have gazillion channels for different timestamps.

And to address the IMHO obvious need to persist that data, there would then have to be for example a Rule which reads the serialized array (or gazillion channels with different timestamps) and then persists the data independently from the Binding that fetched it.

Did I get this right?

The only thing the binding can do is make the data available. In addition to the two methods you summarized, it can also be made available through actions, which this binding is also offering. When data is made available, users can consume the data in any way they see fit. I don't have any recommendations here. I'm using rules, but the Rest API could also be used to access items, i.e. the serialized JSON.

[openhab-bot]

This pull request has been mentioned on openHAB Community. There might be relevant details there:

https://community.openhab.org/t/energi-data-service-binding-4-0-0-4-1-0/144370/1

[openhab-bot]

This pull request has been mentioned on openHAB Community. There might be relevant details there:

https://community.openhab.org/t/energi-data-service-binding-3-4-0-4-0-0/144381/1

[openhab-bot]

This pull request has been mentioned on openHAB Community. There might be relevant details there:

https://community.openhab.org/t/dishwasher-price-calculation-automation/139207/5

[jlaur]

@openhab/add-ons-maintainers - anyone up for a review, even a partial one? 😉

Type	Lines
Code	3686
OH-INF	309
README	399
Tests	5137

[jlaur]

@openhab/add-ons-maintainers - no one having time to pick up this PR for review? 🙂 Would be nice as baseline before 4.1 changes for currency, history, etc.

[fwolter]

This is a reasonable code to test ratio :)

[jlaur]

@fwolter - thanks for picking up this PR and starting a review. I have fixed your initial comments.

[kaikreuzer]

Would be nice as baseline before 4.1 changes for currency, history, etc.

I was actually not looking at it yet as I assumed you'll depend on openhab/openhab-core#3597.
But I guess it is only the advanced json channel that will get slightly redundant/outdated once we have forecasts in place and I assume you'll want to keep this in place even then anyhow.

[jlaur]

I was actually not looking at it yet as I assumed you'll depend on openhab/openhab-core#3597.

It doesn't depend on this, but that will make it possible to store future prices and see graphs with those. However, the binding can already perform calculations based on future prices, or you can make your own calculations based on future prices from the JSON channel or thing actions to retrieve those prices.

But I guess it is only the advanced json channel that will get slightly redundant/outdated once we have forecasts in place and I assume you'll want to keep this in place even then anyhow.

The JSON channel is already half redundant because of thing actions, but I think it still serves a purpose for some scenarios where you can get future prices without writing rules. However, with openhab/openhab-core#3597 I think it will be fully redundant.

[fwolter]

LGTM

[enter your binding's logo reminder here.]

[openhab-bot]

This pull request has been mentioned on openHAB Community. There might be relevant details there:

https://community.openhab.org/t/dishwasher-price-calculation-automation/139207/8