Would a Matplotlib backend be useful?

[jonmmease]

While working on the VegaFusion mime renderer, it occurred to me that the ability to convert Vega-Lite to Vega and then pre-evaluate the data transformations in the Vega spec would make it relatively tractable to write translators from these pre-evaluated Vega specs to other plotting libraries, like matplotlib. (This would be for non-interactive charts only, I don't think translating interactivity to another library would be possible)

I'm curious whether folks would find it useful, in practice, to be able to convert (non-interactive) Altair Charts to Matplotlib figures. One motivation that comes to mind would be to take advantage of Matplotlib's support for LaTeX typesetting. And even though VlConvert makes static image export a lot easier, there are probably still advantages to using Matplotlib for creating certain figures for publications.

What do you think?

[jakevdp]

I think it would be very useful. I started trying it here but didn't get very far: https://github.com/altair-viz/altair_matplotlib

Feel free to revive that effort if you're interested!

[mattijn]

I would like the support of a non-interactive backend, but can you elaborate what you had in mind @jonmmease? Eg from Vega JSON to Python Matplotlib or from Vega JSON to a Matplotlib supported backend?

[jonmmease]

I'm picturing Python logic that inputs a pre-transformed JSON Vega spec and then builds up a Matplotlib figure object.

E.g.
This altair chart

import altair as alt
from vega_datasets import data

source = data.movies.url

chart = alt.Chart(source).mark_bar().encode(
    alt.X("IMDB_Rating:Q", bin=True),
    y='count()',
)

produces this Vega-Lite:

{
  "config": {
    "view": {
      "continuousWidth": 400,
      "continuousHeight": 300
    }
  },
  "data": {
    "url": "https://cdn.jsdelivr.net/npm/[email protected]/data/movies.json"
  },
  "mark": "bar",
  "encoding": {
    "x": {
      "bin": true,
      "field": "IMDB_Rating",
      "type": "quantitative"
    },
    "y": {
      "aggregate": "count",
      "type": "quantitative"
    }
  },
  "$schema": "https://vega.github.io/schema/vega-lite/v4.17.0.json"
}

Which compiles to this Vega:

{
  "$schema": "https://vega.github.io/schema/vega/v5.json",
  "background": "white",
  "padding": 5,
  "width": 400,
  "height": 300,
  "style": "cell",
  "data": [
    {
      "name": "source_0",
      "url": "https://cdn.jsdelivr.net/npm/[email protected]/data/movies.json",
      "format": {
        "type": "json"
      },
      "transform": [
        {
          "type": "extent",
          "field": "IMDB_Rating",
          "signal": "bin_maxbins_10_IMDB_Rating_extent"
        },
        {
          "type": "bin",
          "field": "IMDB_Rating",
          "as": [
            "bin_maxbins_10_IMDB_Rating",
            "bin_maxbins_10_IMDB_Rating_end"
          ],
          "signal": "bin_maxbins_10_IMDB_Rating_bins",
          "extent": {
            "signal": "bin_maxbins_10_IMDB_Rating_extent"
          },
          "maxbins": 10
        },
        {
          "type": "aggregate",
          "groupby": [
            "bin_maxbins_10_IMDB_Rating",
            "bin_maxbins_10_IMDB_Rating_end"
          ],
          "ops": [
            "count"
          ],
          "fields": [
            null
          ],
          "as": [
            "__count"
          ]
        },
        {
          "type": "filter",
          "expr": "isValid(datum[\"bin_maxbins_10_IMDB_Rating\"]) && isFinite(+datum[\"bin_maxbins_10_IMDB_Rating\"])"
        }
      ]
    }
  ],
  "marks": [
    {
      "name": "marks",
      "type": "rect",
      "style": [
        "bar"
      ],
      "from": {
        "data": "source_0"
      },
      "encode": {
        "update": {
          "fill": {
            "value": "#4c78a8"
          },
          "ariaRoleDescription": {
            "value": "bar"
          },
          "description": {
            "signal": "\"IMDB_Rating (binned): \" + (!isValid(datum[\"bin_maxbins_10_IMDB_Rating\"]) || !isFinite(+datum[\"bin_maxbins_10_IMDB_Rating\"]) ? \"null\" : format(datum[\"bin_maxbins_10_IMDB_Rating\"], \"\") + \" \u2013 \" + format(datum[\"bin_maxbins_10_IMDB_Rating_end\"], \"\")) + \"; Count of Records: \" + (format(datum[\"__count\"], \"\"))"
          },
          "x2": {
            "scale": "x",
            "field": "bin_maxbins_10_IMDB_Rating",
            "offset": 1
          },
          "x": {
            "scale": "x",
            "field": "bin_maxbins_10_IMDB_Rating_end"
          },
          "y": {
            "scale": "y",
            "field": "__count"
          },
          "y2": {
            "scale": "y",
            "value": 0
          }
        }
      }
    }
  ],
  "scales": [
    {
      "name": "x",
      "type": "linear",
      "domain": {
        "signal": "[bin_maxbins_10_IMDB_Rating_bins.start, bin_maxbins_10_IMDB_Rating_bins.stop]"
      },
      "range": [
        0,
        {
          "signal": "width"
        }
      ],
      "bins": {
        "signal": "bin_maxbins_10_IMDB_Rating_bins"
      },
      "zero": false
    },
    {
      "name": "y",
      "type": "linear",
      "domain": {
        "data": "source_0",
        "field": "__count"
      },
      "range": [
        {
          "signal": "height"
        },
        0
      ],
      "nice": true,
      "zero": true
    }
  ],
  "axes": [
    {
      "scale": "y",
      "orient": "left",
      "gridScale": "x",
      "grid": true,
      "tickCount": {
        "signal": "ceil(height/40)"
      },
      "domain": false,
      "labels": false,
      "aria": false,
      "maxExtent": 0,
      "minExtent": 0,
      "ticks": false,
      "zindex": 0
    },
    {
      "scale": "x",
      "orient": "bottom",
      "grid": false,
      "title": "IMDB_Rating (binned)",
      "labelFlush": true,
      "labelOverlap": true,
      "tickCount": {
        "signal": "ceil(width/10)"
      },
      "zindex": 0
    },
    {
      "scale": "y",
      "orient": "left",
      "grid": false,
      "title": "Count of Records",
      "labelOverlap": true,
      "tickCount": {
        "signal": "ceil(height/40)"
      },
      "zindex": 0
    }
  ]
}

VegaFusion would take this vega and pre-transform it into:

{
  "$schema": "https://vega.github.io/schema/vega/v5.json",
  "data": [
    {
      "name": "source_0",
      "values": [
        {
          "__count": 985,
          "bin_maxbins_10_IMDB_Rating": 6.0,
          "bin_maxbins_10_IMDB_Rating_end": 7.0
        },
        {
          "__count": 100,
          "bin_maxbins_10_IMDB_Rating": 3.0,
          "bin_maxbins_10_IMDB_Rating_end": 4.0
        },
        {
          "__count": 741,
          "bin_maxbins_10_IMDB_Rating": 7.0,
          "bin_maxbins_10_IMDB_Rating_end": 8.0
        },
        {
          "__count": 633,
          "bin_maxbins_10_IMDB_Rating": 5.0,
          "bin_maxbins_10_IMDB_Rating_end": 6.0
        },
        {
          "__count": 204,
          "bin_maxbins_10_IMDB_Rating": 8.0,
          "bin_maxbins_10_IMDB_Rating_end": 9.0
        },
        {
          "__count": 43,
          "bin_maxbins_10_IMDB_Rating": 2.0,
          "bin_maxbins_10_IMDB_Rating_end": 3.0
        },
        {
          "__count": 273,
          "bin_maxbins_10_IMDB_Rating": 4.0,
          "bin_maxbins_10_IMDB_Rating_end": 5.0
        },
        {
          "__count": 4,
          "bin_maxbins_10_IMDB_Rating": 9.0,
          "bin_maxbins_10_IMDB_Rating_end": 10.0
        },
        {
          "__count": 5,
          "bin_maxbins_10_IMDB_Rating": 1.0,
          "bin_maxbins_10_IMDB_Rating_end": 2.0
        }
      ]
    }
  ],
  "marks": [
    {
      "type": "rect",
      "name": "marks",
      "from": {
        "data": "source_0"
      },
      "encode": {
        "update": {
          "y": {
            "field": "__count",
            "scale": "y"
          },
          "x2": {
            "field": "bin_maxbins_10_IMDB_Rating",
            "scale": "x",
            "offset": 1
          },
          "y2": {
            "value": 0,
            "scale": "y"
          },
          "fill": {
            "value": "#4c78a8"
          },
          "ariaRoleDescription": {
            "value": "bar"
          },
          "x": {
            "field": "bin_maxbins_10_IMDB_Rating_end",
            "scale": "x"
          }
        }
      },
      "style": [
        "bar"
      ]
    }
  ],
  "scales": [
    {
      "name": "x",
      "type": "linear",
      "domain": [1, 10],
      "range": [0, 400],
      "bins": {"start":1,"step":1,"stop":10},
      "zero": false
    },
    {
      "name": "y",
      "type": "linear",
      "domain": [4, 985],
      "range": [300, 0],
      "zero": true,
      "nice": true
    }
  ],
  "axes": [
    {
      "scale": "y",
      "aria": false,
      "labels": false,
      "gridScale": "x",
      "maxExtent": 0,
      "ticks": false,
      "tickCount": 10,
      "zindex": 0,
      "domain": false,
      "grid": true,
      "minExtent": 0,
      "orient": "left"
    },
    {
      "scale": "x",
      "labelOverlap": true,
      "title": "IMDB_Rating (binned)",
      "grid": false,
      "labelFlush": true,
      "orient": "bottom",
      "tickCount": 40,
      "zindex": 0
    },
    {
      "scale": "y",
      "grid": false,
      "zindex": 0,
      "title": "Count of Records",
      "tickCount": 10,
      "orient": "left",
      "labelOverlap": true
    }
  ],
  "padding": 5,
  "style": "cell",
  "background": "white",
  "width": 400,
  "height": 300
}

Where the input datasets are values with no transforms, the scale ranges and domains are literal values, and all signals have been evaluated and inlined.

The idea would be to write Python logic that inputs simplified Vega specs (with the above properties) and builds up a Matplotlib figure using the regular matplotlib API. It's probably still harder than I'm imagining, but starting with this simplified Vega spec feels a lot more tractable than starting with the original Vega-Lite spec.

[mattijn]

Thanks for the response. It's basically just marks, scales and axes, but indeed still a non-trivial exercise. +1 for brevity though. It would be great if it can enable cross-pollination to Vega. Maybe also interesting for an academic project? Matplotlib Vega > Mega JSON.

[mattijn]

Hi @jonmmease, with permission, I am tagging @ksunden from the Matplotlib org in this discussion. There is an interest at the Matplotlib org to have a shareable interchange format of visualisations and this route might be a potential route to explore.

[clouds56]

I'm now looking for this.
I love the syntax and theme of altair, but I don't need interactive (and I found interactive might cost a lot in footprint and chart init time)

[clouds56]

Yes, I've tried this

import numpy as np
import matplotlib.pyplot as plt
import polars as pl

arr = np.zeros(3_000_000)
df = pl.DataFrame({
  'index': np.arange(len(arr)),
  'value': arr
})
plt.plot(df['index'], df['value']) # this takes 0.2s

import altair as alt
alt.data_transformers.enable('vegafusion')
alt.renderers.enable('png')

alt.Chart(df).mark_line().encode(x='index', y='value') # this takes 14.7s

on my M2 macOS.

[jonmmease]

It's not a finished project, but you could try out Avenger which a project I'm been toying around with to write a GPU accelerated Vega renderer for Altair.

[clouds56]

@jonmmease, Thanks for writing the amazing project, I think wgpu is a good way to go.
But it seems not mitigating the problem.
I've add alt.renderers.enable('avenger-png', scale=1) and the chart takes 34.9s to generate.

And I don't think matplotlib renders graphs in GPU. I've tried to write it in raw python (numpy), and this takes only 2.2s.

img = np.zeros((300, 500))
arr = np.random.rand(3_000_000)
x = (np.arange(len(arr)) / len(arr) * 500).astype(int)
y = (arr * 200).astype(int) + 50
init_y = 0
for i,j in zip(x, y):
  img[min(init_y, j):max(init_y, j)+1, i] = 1
  init_y = j
plt.imshow(img)

[jonmmease]

Thanks for giving it a try. Honestly, you are hitting the reason why I lost a bit of motivation on building Avenger. Avenger's rendering itself is dramatically faster than going through Vega's regular canvas rendering path, but there's a lot of work that Vega does upstream to build the scenegraph that's the input to the renderer, which takes a lot of time for relatively large datasets.

One reason is that Vega's internal architecture is designed to process data points one at a time in a streaming fashion, whereas for large static datasets it would be much faster to process them in column oriented form, as matplotlib does.

An approach I've been considering is to integrate Avenger into VegaFusion, and then give VegaFusion the ability to render marks to PNG images that are embedded in the chart specification that gets passed to the regular Vega renderer. I'm personally more inclined to go this direction once I have some time than the matplotlib direction, but I'd be happy to talk with anyone who was interested in exploring the matplotlib approach.

[clouds56]

Correct me if I'm wrong.
I found in any combination of (VegaFusion/vl-convert/avenger), there are several time consuming steps that can not be bypassed.

• convert table to/from json in vega_spec (931ms per one way for 1_000_000 points)
• generating svg (6.36s)
• render svg to png (failed since too many points)

If we would like to statically draw a graph (non-interactive), we may not need svg for intermediate representation and pass (pyarrow) buffer (but not json, including str or dict) between components (VegaFusion, vl-convert, and vega-lite.js).