Bug: colors in line charts are often very similar

[lucasrodes]

Description

Line colors in line charts colors are sometimes very similar depending on the entities selected.

Expected behavior

All lines in the chart should have colors that are easy to distinguish. This can be achieved by choosing from our palette smartly.

Steps to reproduce

Steps to reproduce the behavior:

1. Go to https://ourworldindata.org/grapher/excess-mortality-p-scores-projected-baseline
2. Follow the steps displayed in the first video below

Environment

For desktop, provide:

• OS: MacOS
• Browser: Arc
• Version: 1.57.1 (Chromium Engine Version 128.0.6613.85)

Additional context and screenshots

After playing with selecting and unselecting entities in this chart, I realized that the lines were not very distinct. I am a bit confused by the rules for the Grapher when choosing colors.

color-charts.mov

If one refresh, the chart is loaded with distinguishable colors (second video).

demo-color-2.mov

I think this often leads to DI charts with similar line colors.

[toni-sharpe]

This isn't a high priority comment. But I hope it is useful to you. If not and these sorts of comment are wasting your time or just annoying, ask me to stop, I won't be offended. I've tried to format it so it's not overwhelming but so there's some deeper dives possible for interested folk.

I'm assuming low knowledge and sticking at that pace, it's not directed at any one person's decision or work. A search of the grapher repo returns "192 matches across 51 files" for rgb and "313 matches across 55 files" for : # indicating a CSS hex declaration. 54 of those are .scss files. hsl is used 24 times in 4 places.

HSL in a layman style

tl;dr: HSL presents colour as a set of values designed for people, unlike HEX, designed for computers. It's syntactic sugar really, but improves the DX drastically. Hue is the colour of visible light, saturation is how much of it there is (fade to grey) and lightness (or luminescence) is how much light there is (fade to black).

Also it shows how the values can be manipulated with simple to understand arithmetic. Images taken from GIMP are used to demonstrate this, and there are some links to CSS tricks and MDN which get into the technical detail.

HSL: a broad overview

=====================================

HSL Definition:

Hue: is a wheel of colour that rolls round from red, through orange, to yellow, then green, blue, purple and finally back to red. Typically it runs through 360 degrees, like a compass. Note that there is no pink. Pink is really light red, you'll notice on some artists paints that the pinks are in the red range.

Saturation: is the amount of hue in the result. Usually a %. Set to 100 it's as much colour as possible, set to 0 it's grey-scale.

Lightness / Luminescence: Two words that mean roughly the same and start with L. Again, usually a % with 0 being absolute darkness, the absence of light. Technically this hasn't been achieved by humans although there are paints[1] that absorb 99.9% of light. 100 is the opposite. The result is a scale from black to white, which goes through all levels of the selected hue, at the selected saturation.

HSL is usually much simpler for people to understand, whereas mixtures of RGB are more computer focused. Monitors use a mixture of R, G and B to produce the range of colours, which we know from hex codes in CSS, whereas HSL puts the colour into one place which we understand because it is closer to what we see then uses saturation and lightness, again, a closer match to our experience with eyes, assuming sightedness.

It's also much easier to do arithmetic with.

[1] now isn't the time to get into why paint uses yellow and not green as a primary colour.

In CSS

I'll let the excellent CSS Tricks site explain how it works in CSS style sheets.

And the MDN docs, just because they're great.

CSS also has an alpha property is HSLA, that defines opaqueness.

An example

In these examples look at the bottom left for "Current" colour.

A hue change

You can see how the S and L portions are the same but the Hue selection is moved.

Grey-scale

Here we see the exact same Hue and Lightness as the previous example, but the removal of all saturation results in grey. This grey would be the same regardless of hue.

Black

Here's the same hue again, this time with no lightness, resulting in black. The same applies where this occurs with any hue.

Final note

HSL is really syntactic sugar for referring to light from a luminescent source, in a computer it's converted into RGB for display; this is different from light reflected off an object (like something painted) which is why RGB are primary on screens, and Red, Blue and Yellow are primary in paints.

I'll leave colour there for now, hope this explanation helps. The topic can take a life's work and I'm no expert by any means.

=====================================

An example of how this manifests on different charts

tl;dr: I take a few images from my own site to show how this looks, including the line graph which I've not actually got live yet in my work.

I've linked directly to the histogram page because it's probably the best one for demonstrating the ideas and code plus it's interactive, and using "deviation" will show the alpha level opacity which hsla provides.

HSL: some examples in charts

=====================================

Line graph

Seen as that's the subject, that's what I'll show first

Histogram

Though I'm also going to show this - IMO, it works very well with Histograms (therefore also Marimekko, Stacked ...

The first example doesn't need to toggle, spreading the colour works: 360 / 3 = quiet enough difference.

This set of two show with a force toggle and without - the code section will cover the toggle

=====================================

Code

A couple of functions from my work are defined, cleaned of some cruft and annotated. I use this in quite a few places.

Code

function calcContrastToggle({ i, total, useHueContrastToggle }) {
  // small sets don't need a toggle, see the first Histogram example
  // there are even cases where applying the toggle is detrimental 
  if (total <= CONTRAST_TOGGLE_MINIMUM) {
    return 0
  }
  
  // possibly a five minute job
  return (i % 2) * 180
}

export function calcAccessibleHue() {
  return function({ i, total, aLevel = 1 }) {
    // i just moves us round the wheel with the total defining how far on each step
    const hueCalced = calcHue({ i, total })

    // uses the above function
    const contrastToggleCalced = calcContrastToggle({ i, total, useHueContrastToggle })
    
    // hue + toggle gives us the highest contrast hue
    const hue = hueCalced + contrastToggleCalced
    
    // this line just ensures that we stay inside a circle 0 .. 360
    const hueCircled = hue > 360 ? hue % 360 : hue

    // and format, in my code the style tag is used
    return `hsla(${hueCircled} 80% 50% / ${aLevel})`
  }
}

What's notable about the code, apart from how simple it is, is that the colour choice is basic arithmetic and very clear to understand for the reader. This relates back to the earlier section about manipulation and being more human friendly than HEX (or RGB for that matter). It matches how artists and designers think so the developer is using a language that's understood by a broader set of colleagues who don't need to be technical.

Then if you're confronted by more challenging colour logic, like what might lie under support of the various colour interpretations people have, my hypothesis would be that the task would be easier using this tool than direct HEX values.

Final note: there are numerous accessibility problems with "just colour" solutions, but that's a separate topic.

[CGiattino]

I'd originally sent this message on Slack on 13 Sep. Just copying here so it's kept together with this issue.

I think there are two related things here:

1. does the grapher select colors the way we want it to, and do this consistently?
2. is our color palette as distinct as possible? particularly in the sequence of colors that grapher chooses

I wanted to add that I experienced a similar thing today, with grapher and similar (to my eye at least!) line colors being selected.

I’m curious if others see these colors as more distinct than I do? this could all just be me!

• I went to post Este’s DI on Chile solar on social media, but I noticed the line colors looked fairly similar to my eye — I see 4 broadly reddish ones (Chile, Spain, China, US) out of 6 lines

• so I went to the grapher chart to see if I could make those a bit more distinct
• I deselected the default countries and added in the ones from the DI, and it’s perhaps a bit better, but I still got what seem to me like 4 broadly reddish colors

• I clicked refresh to see if that would improve it, as it sometimes does, and that resulted in the following. I do think the way these countries are grouped makes this seem worse than it would otherwise, but I still see 3 reddish ones and 2 greenish ones

[toni-sharpe]

Thanks for sharing the Slack information, I'll make some notes on what I view shortly

Where I am I have a good viewing setup (MacBook, shaded area), plus I have eyes that are very good at seeing colour differences, these make me a poor test candidate.

All I'd add now is that the line thickness contributes - I think this is clear in my earlier comment where I show some examples of bar and line charts; and I'm very sure at 0 zoom Charlie's examples would be harder for me.

[toni-sharpe]

@CGiattino I came back to this thinking some simulations might be worth looking at. I ran them through the simulators found here at PillStone. The simulator is on this page. I ran all eight, which you'll find in the enclosed sections below, along with a histogram example of hue toggling and wheeling (from my site). Without knowing a great deal about the medical side of this topic I took it on trust, and high google position, but a good case in support of their breakdown is that it matches StoryBook's. Parlance is copied, but the topic's got some depth. The whole green and red "classic" puts forward complexities beyond this comment.

EDIT: Added a credible source for the terms used from the American Academy of Ophthalmology.

Without simulation

Green: Deuter: anomaly (weak) anopia (blind)

Red: Protan: anomaly (weak) anopia (blind)

Shows red deficiencies as simulation given the examples

Blue: Tritan: anomaly (weak) anopia (blind)

B&W & blue only

[lucasrodes]

I realized what the current logic of picking colors in line charts is.

Basically, colors are picked sequentially from our palette:

Say you've selected n entities for your chart. Then, the first n colors are picked. If you now unselect the nth entity and pick another one, this new entity will use the n+1 color in the palette.

In #4003, we are trying to optimize the colors for a selection of n entities. This means we've tried to make the first n (where n is any number 1-24) colors in the palette as distinct as possible. Therefore, unselections break this logic.

I've confirmed my hypothesis by playing with this chart and forcing the selection of colors 2, 6, 9 ,11, 13, 16, 19, 21, 24.

I see the motivation for this behavior: if we were to bring back unselected entities, they should retain the color they were first shown with. But I wonder if the cost of this (i.e., producing charts with sub-optimal color choices) is higher than the benefit.

My opinion: When an entity is unselected, it is totally fine for it to lose its "color slot." If it is unselected and selected right away, it will retain the "color slot" (I think that makes sense). But if it is unselected and some other countries are selected before it is selected again, it is fine for it to just have whatever color the palette specifies.

[lucasrodes]

To add to my previous comment, I think the current color-picking logic makes even less sense in the admin page (i.e. when we are creating a chart).

In a new chart, when choosing the entities to display, the colors picked should not be affected by whether the power user did unselections.

My sense, however, is that the colors that the power user sees when creating a new chart vs. the colors shown when the chart is actually public don't necessarily match. That is, the colors in the latter will be based on the order of our color palette (i.e., regardless of any unselections during chart-editing).