[Tidy] Remove setting of pio.templates.default on import

[antonymilne]

Description

After a lot of work, everything should work almost exactly the same as before 😅

Dashboard: everything should be exactly the same as before.

Outside dashboard:

• importing vizro shouldn't change plotly theme
• plots from outside vizro (including plotly.express) should not have vizro theme
• but you can do a pure plotly chart using template="vizro_light/vizro_dark" if you want
• plots from inside vizro (vizro.plotly.express and anything custom written in @capture) should have vizro theme
• this theme is vizro_dark by default but can be changed using pio.default.template = "vizro_light" OR setting template="vizro_light" in a plot OR setting fig.layout.template = "vizro_light" in a plot
• the theme is not changed globally, so if you do a vizro plot and then a pure plotly one then the pure plotly one shouldn't have the vizro theme
• similarly you could do template="vizro_light" and then template="vizro_dark" in the same notebook and it should respect those choices

How does templating work?

On vizro import, we register plotly themes vizro_dark and vizro_light, available as pio.templates["vizro_dark"] and pio.templates["vizro_light"]. The correct way to use these is:

• ideally through specifying a string template="vizro_dark/light"
• if the actual template itself is required, look it up as pio.templates["vizro_dark/light"]

Note the public API here is through pio.templates, NOT the underlying vizro._themes objects. If a user want to modify a template then the correct way to do it is to modify pio.templates["vizro_dark/light"], not the underlying vizro._themes objects. Whenever we consume the theme we take it from pio.templates, not vizro._themes. This ensures consistency with the plotly templating scheme and that user modifications are always applied correctly.

On vizro import, nothing else happens to do with templates. We don't set a default plotly template, just register them.

In a dashboard

• Developer sets Dashboard.theme="vizro_dark/light", which then sets pio.templates.default = dashboard.theme in Vizro.build. This default template is never unset (even by Vizro._reset). Remember this vizro_dark/light template could have been modified by developer but it still has the same name
• While dashboard is running, callbacks run Graph.__call__ to execute the figure function to get a go.Figure. Regardless of how this was produced, inside Graph.__call__ we take the user-specified template required_template and update fig.layout.template = required_template on the server before returning the graph. This is actually just a small optimisation to ensure there's no flickering on the frontend, which would happen if we relied on clientside callback for the initial graph to render in the correct theme
• Once graph has loaded and user clicks theme selector, a clientside callback update_graph_theme runs to do Plotly.relayout, which is basically equivalent to fig.layout.template = vizro_dark/light. The Python graph function is not re-run (and shouldn't be, since doing so would be very slow just to switch some colours). Ideally this same clientside callback would also be used on initial graph rendering to set the theme, but that introduces a small flicker, hence setting it on the backend in advance also

Outside a dashboard

Think "Jupyter notebook user" here, although there's nothing actually specific to Jupyter notebooks and we don't check to see if a Jupyter environment is active. The intention is that:

• Vizro plots render as vizro_dark unless the user is explicitly trying to do vizro_light
• you can copy and paste a plot into the dashboard and it will work exactly the same way

Plots that use capture("graph)" (could be vizro.plotly.express, other builtin vizro chart or custom user graph) work as follows.

• before chart function is run, we set pio.templates.default = "vizro_dark" unless it's already set to vizro_light
• chart function is run
• after chart function is run, pio.templates.default is reverted to whatever it was before
• then we apply fig.layout.template = vizro_dark/light to be exactly consistent with how the dashboard behaviour works. This ensures that if a user sets template=... inside their custom function it gets overridden, just like it would on a dashboard

TODO here:

• Tidy up PR description
• Finish tests
• Think about what different from now -> should just have affected jupyter case
• think about nested plot case
• consider doing the + - update ticket
• Docs

Screenshot

Notice

• I acknowledge and agree that, by checking this box and clicking "Submit Pull Request":

  • I submit this contribution under the Apache 2.0 license and represent that I am entitled to do so on behalf of myself, my employer, or relevant third parties, as applicable.
  • I certify that (a) this contribution is my original creation and / or (b) to the extent it is not my original creation, I am authorized to submit this contribution on behalf of the original creator(s) or their licensees.
  • I certify that the use of this contribution as authorized by the Apache 2.0 license does not violate the intellectual property rights of anyone else.
  • I have not referenced individuals, products or companies in any commits, directly or indirectly.
  • I have not added data or restricted code in any commits, directly or indirectly.

[huong-li-nguyen]

Your PR just reminded me of an issue.

Plotly Express charts come with their own color palettes and do not adopt the default color palettes specified in your template, even if you assign the template to the charts as you do in this PR. Consequently, removing the global theme has caused us to lose our default color palettes. For example, if you run the dev example, you'll notice that it defaults to Plotly's standard color palette.. 🙈

plotly/plotly_express#70

The only way to resolve this seems to be by setting the global theme based on the issue discussion above..or we try to set the color palettes in the charts again. (EDIT: I just tried that out, but it doesn't seem to work)

[antonymilne]

@huong-li-nguyen yes, I just realised that this morning as well 🤦 😱 😞 😬 I have some more thoughts on this now, let us discuss! I am keen for this to not drag on indefinitely since it's blocking other stuff and eating up lots of time.

[huong-li-nguyen]

First of all, incredible work! 🥳 💯 🚀

I’m genuinely impressed that you found a solution that addresses the notebook case, the dashboard case, and all the different user groups we've discussed. 🫠

I hope the added complexity is justified because even after our multiple discussions, I still needed all your comments to fully grasp what’s happening and why we’re doing things a certain way. Thanks for documenting everything so thoroughly!

I’ve manually tested all our demo dashboards and the following test cases, and everything works as expected! 🏁 ✅

Test Case	Action	Expected Result	✅
Dashboard with vizro_dark as the default theme	Set vizro_dark as the default theme and switch themes.	The initial theme should be vizro_dark. Switching themes should update the dashboard accordingly.	✅
Dashboard with vizro_light as the default theme	Set vizro_light as the default theme and switch themes.	The initial theme should be vizro_light. Switching themes should update the dashboard accordingly.	✅
Notebook with pure Plotly chart with no template	Import vizro and create plots using pure plotly or plotly.express.	These plots should not have the vizro theme applied.	✅
Notebook with pure Plotly chart with vizro_light/vizro_dark template	Create a pure Plotly chart using template="vizro_light" or template="vizro_dark".	The chart should adopt the specified vizro theme.	✅
Notebook with plots from vizro	Create plots using vizro.plotly.express and any custom plots written with @capture. Additionally check template argument.	These plots should have the vizro theme applied. The default theme is vizro_dark, but it can be changed using: - pio.templates.default = "vizro_light" - Setting template="vizro_light" in a plot - Setting fig.layout.template = "vizro_light" in a plot	✅
Theme persistence in mixed plotting	Create a vizro plot followed by a pure Plotly plot.	The vizro plot should have the vizro theme. The subsequent pure Plotly plot should not have the vizro theme applied, as the theme is not changed globally.	✅

[antonymilne]

Thank you for the amazing testing @huong-li-nguyen! I'm so pleased it worked and also that it made some sense from all my comments. I spent so long thinking about this I didn't want to forget about it if we look at this code again in future.

I've just updated the PR description - please can you read through and let me know if it makes sense and is consistent with your expectations also? Then I will add it to our dev docs, and a much more user-consumable version will go into the main docs in a separate PR.

[huong-li-nguyen]

I read through it several times 😄, and it all makes sense to me!

There is one tricky case where it might not be immediately clear what's happening. The good news is that this example works consistently in both the notebook and the dashboard, thanks to your workaround! 🥳

From a user perspective, I would wonder which template takes precedence. Given that I use @capture, the global theme should be set to vizro_dark. At the same time, I specify template="plotly", which one might assume has higher precedence. However, the result is a mix of both, which could be confusing. I get all the settings from vizro_dark, but the colors are from template="plotly".

@capture("graph")
def custom_scatter(data_frame):
    fig = px.scatter(data_frame, x="sepal_width", y="sepal_length", color="species", template="plotly")
    return fig

I'm not sure if it's worth explaining as this is already such an edge case (basically if someone wants to use a distinct chart template inside our Vizro dashboard). But based on your description above, my understanding is that if someone does want to change it, they would need to modify pio.templates["vizro_dark/light"] directly.

So, to get the unlikely case below working, one would have to do:
pio.templates["vizro_dark"] = pio.templates["plotly"]

And this indeed works ✅. So if I understood everything correctly, even this edge case does work as I expected it?

[antonymilne]

Yes, you understand things correctly, and this is actually maybe not even such an edge case if someone wants to modify several figures in the same way. In due course it will be explained in the docs.

What happens here is:

• pio.template.default = "vizro_dark"
• px.scatter(template="plotly") takes FULL precedence over that, as if you hadn't set pio.template.default in the first place. This is expected and not weird
• but then the fig.layout.template = "vizro_dark" takes PARTIAL precedence over that
• and theme switching will re-apply fig.layout.template = "vizro_dark/light"

The confusing thing is the PARTIAL above: it's not actually fully overriding the template in the way that you might expect. This is the thing I was talking about yesterday on slack and is a confusing feature of how plotly express work.

Completely independently of vizro, if you do this then the results will not be what you expect:

import plotly.io as pio
fig = px.scatter(px.data.iris(), x="petal_width", y="petal_length", color="species", template="plotly_dark")
# or could have set `pio.template.default = "plotly_dark"`, would have been exactly the same figure
fig.show() # figure is fully in plotly dark colours as expected
fig.layout.template = "ggplot2"
fig.show()

This will give a plot that's a weird hybrid where the template is ggplot2 but the marker colours have not changed from the plotly_dark ones. This is because the marker colours are encoded in fig.data rather than fig.layout and so doing fig.layout.template = "ggplot2" doesn't affect them.

One thing that is sort of possible is to do:

fig = px.scatter(px.data.iris(), x="petal_width", y="petal_length", color="species", template="plotly_dark")
# move all colours to the template so they *can* be overridden by fig.layout.template
fig = pio.to_templated(fig)
fig.layout.template = "ggplot2"
fig.show()

The problem with this is that you ned to specify the right stuff to not move to the template using to_templated(skip) or you'll lose other stuff that plotly express adds that we want to keep (like hover information formatting). And if a user has explicitly set color_discrete_map or similar in px.scatter then we don't want to move that to override that.

Soooo basically if you want to style your plot in a way that's not vizro_dark/light then your options are:

• modify vizro_dark/light theme - applies to all charts and works well
• specify color_discrete_map or whatever inside your custom chart function - this isn't overridden by our template
• apply your own template and then use carefully use to_templated to migrate only the bits you want overridden by vizro_dark/light to the template - applies to multiple charts, but requires a lot of plotly expertise so not likely anyone will do it

[huong-li-nguyen]

@antonymilne - that makes a lot of sense! And yes, I had in mind that this partial ambiguity comes from px and how they do the overrides. I think we are all clear from my side then 👍

[antonymilne]

Copying and pasting even more on this so it's all in one place for posterity...

import plotly.graph_objs as go
import vizro.models as vm
import vizro.plotly.express as px
from vizro import Vizro
from vizro.models.types import capture

import plotly.io as pio

pio.templates["vizro_dark"]["layout"]["colorway"] = ["red"]
pio.templates["vizro_light"]["layout"]["colorway"] = ["yellow"]

df = px.data.iris()

@capture("graph")
def my_graph_figure_px(data_frame):
    return px.scatter(data_frame, x="sepal_width", y="sepal_length")


@capture("graph")
def my_graph_figure_go(data_frame):
    return go.Figure(go.Scatter(x=data_frame["sepal_width"], y=data_frame["sepal_length"]))


page = vm.Page(
    title="Test",
    components=[
        vm.Graph(figure=px.scatter(df, x="sepal_width", y="sepal_length")),
        vm.Graph(figure=my_graph_figure_px(df)),
        vm.Graph(figure=my_graph_figure_go(df)),
    ],
    controls=[vm.Filter(column="species")],
)

dashboard = vm.Dashboard(pages=[page])

[maxschulz-COL]

Wow what an analysis you two, and I think I mostly understood it although I didn't test it manually because Li has already done so.

Well done 👏 🚀 I am super happy we achieved this and think I get it mostly. I have this one question where I am not sure what the contextmanager is trying to achieve, but happy to approve!

[petar-qb]

Awesome work @antonymilne and testing/investigation @huong-li-nguyen 🎉 💜

This solution is great! (and I think I understand all the changes 😄)

[antonymilne]

Thanks for the reviews all. @petar-qb your questions made me wonder something else which I'm going to document here since it didn't make sense to me (and still doesn't fully tbh, but it seems to work).

When we do fig.layout.template = "...", why doesn't this override everything there? e.g. If you've set a title then we obviously don't want to overwrite that by changing the template. Currently this works correctly but it's not obvious why.

There's several ways you can update a figure's template:

1. fig.update(layout_template=...)
2. fig.update_layout(template=...)
3. fig.layout.template=...
4. fig["layout"]["template"] = ...
5. probably many more variants on the above

What's clear from reading through the source code is that 1 and 2 are identical. It's not super clear whether 2, 3, and 4 are identical. There's maybe some subtle differences but not sure whether they make any difference in reality.

The most important thing is that the update methods have an argument overwrite which defaults to False:

If True, overwrite existing properties. If False, apply updates
            to existing properties recursively, preserving existing
            properties that are not specified in the update operation.

Method 3 that we're doing here seems to also act as if overwrite=False, which means that setting a template doesn't override the title for example. Not sure exactly how this works but the place to start looking is the @layout.setter in BaseFigure anyway.

[stichbury]

Same as before? LGTM! ⭐