Stackgroup with variable positive and negative values - needs entities split, and defaults in every entity?

[ando2040]

Problem: Series bleed into each other because fill: tonexty default for stackgroups uses the upper extent of the previous series as y=0, so when a series swaps to negative, it fills over the previous one.

When I tried a workaround documented elsewhere, to filter positive/negative value series into separate stackgroups, the stackgroups didn't work - despite resample in the entity defaults.

The solution was to move all the filter defaults into every entity - a filter is not additive, it replaces all default filters even though the new filter is not one previously specified in the defaults. This causes a lot of duplication. Does this need explicitly adding to "gotchas?" It got me for a long time but maybe I'm dense... or could/should it be fixed? Personal preference I suspect?

Note that I have exploited a previous gotcha I added to the readme - in this case I want to display value of ys in a single legend whatever its value, and hide the negative legend, so I added name: before the filter, not after.

One remaining niggle: blue series shows a continuous line at y<0 even when y is positive, and red shows a continuous line at y>0 when y is negative. Ideas welcome. Am I even achieving this in the best/only way?

type: custom:plotly-graph
title: 7-day Import/Export history
layout:
  height: 350
entities:
  - entity: sensor.national_grid_grid_generation_ireland_mw
    name: |
      $ex 'Ireland(now) ' + (ys.at(-1)/1000).toFixed(2) + 'GW'
    filters:
      - filter: y > 0
      - resample: 1m
      - multiply: 0.001
    fillcolor: seagreen
    stackgroup: positive
  - entity: sensor.national_grid_grid_generation_ireland_mw
    filters:
      - filter: y < 0
      - resample: 1m
      - multiply: 0.001
    fillcolor: seagreen
    stackgroup: negative
    name: Ireland -
    showlegend: false
  - entity: sensor.national_grid_grid_generation_denmark_mw
    name: |
      $ex 'Denmark(now) ' + (ys.at(-1)/1000).toFixed(2) + 'GW'
    filters:
      - filter: y > 0
      - resample: 1m
      - multiply: 0.001
    fillcolor: red
    stackgroup: positive
  - entity: sensor.national_grid_grid_generation_denmark_mw
    filters:
      - filter: y < 0
      - resample: 1m
      - multiply: 0.001
    fillcolor: red
    stackgroup: negative
    name: Denmark -
    showlegend: false
  - entity: sensor.national_grid_grid_generation_france_mw
    name: |
      $ex 'France(now) ' + (ys.at(-1)/1000).toFixed(2) + 'GW'
    filters:
      - filter: y > 0
      - resample: 1m
      - multiply: 0.001
    fillcolor: blue
    stackgroup: positive
  - entity: sensor.national_grid_grid_generation_france_mw
    filters:
      - filter: y < 0
      - resample: 1m
      - multiply: 0.001
    fillcolor: blue
    stackgroup: negative
    name: France -
    showlegend: false
  - entity: sensor.national_grid_grid_generation_belgium_mw
    name: |
      $ex 'Belgium(now) ' + (ys.at(-1)/1000).toFixed(2) + 'GW'      
    filters:
      - filter: y > 0
      - resample: 1m
      - multiply: 0.001
    fillcolor: black
    stackgroup: positive
  - entity: sensor.national_grid_grid_generation_belgium_mw
    filters:
      - filter: y < 0
      - resample: 1m
      - multiply: 0.001
    fillcolor: black
    stackgroup: negative
    name: Belgium -
    showlegend: false
  - entity: sensor.national_grid_grid_generation_netherlands_mw
    name: |
      $ex 'Nlands(now) ' + (ys.at(-1)/1000).toFixed(2) + 'GW'
    filters:
      - filter: y > 0
      - resample: 1m
      - multiply: 0.001
    fillcolor: orange
    stackgroup: positive
  - entity: sensor.national_grid_grid_generation_netherlands_mw
    filters:
      - filter: y < 0
      - resample: 1m
    fillcolor: orange
    stackgroup: negative
    name: Nlands -
    showlegend: false
  - entity: sensor.national_grid_grid_generation_norway_mw
    name: |
      $ex 'Norway(now) ' + (ys.at(-1)/1000).toFixed(2) + 'GW'
    filters:
      - filter: y > 0
      - resample: 1m
      - multiply: 0.001
    fillcolor: darkred
    stackgroup: positive
  - entity: sensor.national_grid_grid_generation_norway_mw
    filters:
      - filter: y < 0
      - resample: 1m
      - multiply: 0.001
    fillcolor: darkred
    stackgroup: negative
    name: Norway -
    showlegend: false
defaults:
  entity:
    unit_of_measurement: GW
    line:
      width: 0
  yaxes:
    fixedrange: true
hours_to_show: 250
refresh_interval: 10

[dbuezas]

Does this need explicitly adding to "gotchas?"

Since it got you, it's worth adding, I'll approve the PR :)

[ando2040]

Submitted, thanks!

[dbuezas]

One remaining niggle: blue series shows a continuous line at y<0 even when y is positive, and red shows a continuous line at y>0 when y is negative

Maybe you can use tozeroy and make areas translucent?

The solution was to move all the filter defaults into every entity - a filter is not additive, it replaces all default filters even though the new filter is not one previously specified in the defaults.

You could put all reused filters in an array once at the root, the extra ones inside the entity and then load them via filters: $ex [...get('reused_filters'), ...get('.extra_filters')] (note the dot on extra_filters, which makes it relative to that part of the yaml)

[dbuezas]

Try this: #317 (comment)

[ando2040]

Thanks as ever David. This seems to work for the positive stackgroup.

    filters:
      - fn: |- 
          ({xs, ys}) => ({
              xs: xs.flatMap((x, i) => ys[i+1] === null ? [x,x]:[x]),
              ys: ys.flatMap((y, i) => ys[i+1] === null ? [y,null]:[y])
          })
      - resample: 1m
      - multiply: 0.001

I'm less convinced I've adapted it correctly for the negative stackgroup, still working on that...

[dbuezas]

Maybe you can use the same trick of adding a null value on the same x value whenever the series changes sign. This should make the line vertical so the area doesn't cover the other series

[ando2040]

There was a minor alteration for my use case ("seems to work" above proved not to be 100% on closer inspection), thanks as always for pointing me in the right direction!

This is the function I ended up with for positive values:

      - fn: |-
          ({xs, ys}) => ({
              xs: xs.flatMap((x, i) => ys[i] > 0 ? [x] : []),
              ys: ys.flatMap((y, i) => y > 0 ? [y] : [])
          })     

and for negative values:

      - fn: |-
          ({xs, ys}) => ({
              xs: xs.flatMap((x, i) => (ys[i] !== 0 && (ys[i+1] === null || ys[i+1] >= 0)) ? [x, x] : (ys[i] !== 0 ? [x] : [])),
              ys: ys.flatMap((y, i) => (y !== 0 && (ys[i+1] === null || ys[i+1] >= 0)) ? [y, null] : (y !== 0 ? [y] : []))
          })

I can't really take the coding credit/critique, this notation makes my brain hurt - it was mostly GPT but I was highly persistent and a diligent tester! I don't think either is consistently robust for all data, but it's only a chart! Here's the current result, there shouldn't be any series that is simultaneously above and below y=0, I think there might still be a bug or two in there but it's pretty close...

[dbuezas]

Glad to hear.

The filter you have for the positive ones is equivalent to:

- filter: y>0

And you can simplify (ys[i+1] === null || ys[i+1] >= 0) to !(ys[i+1] < 0) in the other one