Is it possible to get selections from python?

[cosmicfarmers]

Hi, thank you so much for the fantastic project!

Retrieving selections from an Altair chart is always something I need. I've experimented with Holoviz Panel, and it does allow me to access the selected items on an Altair chart.

However, with Vega Fusion, I'm struggling to figure out how to retrieve my selections using Python code.

My goal is to use the Vega Fusion chart to preview data and modify selections. Subsequently, I'd like to execute a more extensive query based on these selections in a separate Jupyter cell.

There is something like this, unfortunately it does not work with widget renderer though.
https://altair-viz.github.io/user_guide/jupyter_chart.html

If Vega Fusion does not have any interface yet, please let me know any hacky way to achieve it as a temporary workaround.

[cosmicfarmers]

instead of using vf.enable_widget(), and just displaying the altair chart,

i input my altair chart like:
vf.jupyter.VegaFusionWidget(my_chart, verbose=True)

Now I see my selection is shown in web browser's console.

Is there any workaround to get this selection in another jupyter cell?

[jonmmease]

Hi @jhyoo1982, glad to hear VegaFusion has been helpful for you!

As you noted, it's not possible, today, to use the VegaFusionWidget to access selections from Python. The JupyterChart, released in Altair 5.1, is designed for this purpose. Altair 5.1 also includes some integration with VegaFusion in the form of a "vegafusion" data transformer. You can combine JupyterChart and the "vegafusion" data transformer to get close to what you're asking for. For example:

import altair as alt
alt.data_transformers.enable("vegafusion")
...
alt.JupyterChart(chart)

When used this way, VegaFusion pre-transformes the data in the kernel before sending it to the client using the same logic as is performed for the VegaFusion mime renderer. The limitation of this approach is that any data that's used in a selection filter will be sent to the browser since there's no communication from the browser back to the python kernel. The VegaFusionWidget approach sends selection information from the browser back to the kernel so that the full dataset doesn't need to be sent to the browser.

My goal is to re-build VegaFusionWidget's functionality in Altair using JupyterChart and then deprecate VegaFusionWidget, but I don't have a timeline to share on that at this point.

[cosmicfarmers]

Thanks for the quick reply!
I understand what you are saying.

One last question,

data = pd.DataFrame({'a': list('CCCDDDEEE'),
                     'b': [2, 7, 4, 1, 2, 6, 8, 4, 7]})

dummy_picker = alt.selection_point(encodings=['x'])

chart = (
    alt.Chart(data)
        .transform_filter(dummy_picker)
        .encode(
            x='a',
            y='b',
            color=alt.condition(dummy_picker, alt.value('crimson'),  alt.value('grey'))           
        )
        .add_params(dummy_picker)
        .mark_bar()

)
ex_vf = vf.jupyter.VegaFusionWidget(chart, verbose=True)

Before making a selection on the chart ex_vf,

ex_vf._request_msg is empty like b''

But after making a selection by clicking a bar on the chart,
ex_vf._request_msg
shows some byte code like:
b'\n\x96\x10\n\x8b\x10\n\x9d\x01\n\x7f\n\x14\n\x10_parent_source_0\x10\x01*Y\nL_vegafusion_data/vegafusion-68924a75af09cd2af286c9300bb44036f3235016.feather\x18\xa0?"\x06\n\x04jsonB\x0c\n\nAsia/Seoul\x1a\x04\x08\x02\x10\x01 \xa4\xd0\xac\x8e\xd6\xec\xe9\x99\xb6\x01(\xcc\xed\xd3\xd9\xc6\xde\xc5\xc8\xa3\x01\n\xd6\t\n\xb9\t\n\x11\n\rparam_2_store\x10\x01\x1a\xa3\t\x12\xa0\t\xff\xff\xff\xff\xb8\x01\x00\x00\x10\x00\x00\x00\x00\x00\n\x00\x0c\x00\n\x00\t\x00\x04\x00\n\x00\x00\x00\x10\x00\x00\x00\x00\x01\x04\x00\x08\x00\x08\x00\x00\x00\x04\x00\x08\x00\x00\x00\x04\x00\x00\x00\x03\x00\x00\x00H\x01\x00\x00\\\x00\x00\x00\x04\x00\x00\x00\xd4\xfe\xff\xff\x18\x00\x00\x00\x0c\x00\x00\x00\x00\x00\x01\x0c8\x00\x00\x00\x01\x00\x00\x00\x08\x00\x00\x00\xc8\xfe\xff\xff\xf4\xfe\xff\xff\x14\x00\x00\x00\x0c\x00\x00\x00\x00\x00\x01\x05\x0c\x00\x00\x00\x00\x00\x00\ ...

is that related to web browser's console output?

VegaFusion(wasm): Sending data param_54_store
42.4c40952c6809b26ac6c0.js?v=4c40952c6809b26ac6c0:1 
[
  {
    "unit": "",
    "fields": [
      {
        "field": "a",
        "channel": "x",
        "type": "E"
      }
    ],
    "values": [
      "E"
    ]
  }
]

because what i need is that the evidence that i clicked E on x channel like:

   "values": [
      "E"
    ]

I don't know how to decode the byte array. Please let me know if it's possible.

[jonmmease]

Yeah, these are related. The bytes stored in _request_msg are serialized to the browser as an ipywidget traitlet. The browser deserializes them into a QueryResult protocol buffer object, and the values are stored inside there.

This QueryResult object is defined by the protocol buffer file at:

https://github.com/hex-inc/vegafusion/blob/f7a355f959a0e00db6daab828a711f2cca156d79/vegafusion-core/src/proto/services.proto#L21-L26

The deserialization happens in Rust, compiled to WASM, at

https://github.com/hex-inc/vegafusion/blob/f7a355f959a0e00db6daab828a711f2cca156d79/vegafusion-wasm/src/lib.rs#L178C29-L178C29.

So yeah, one option is that you could use the protoc command line tool to generate a Python interface to the *.proto files located in https://github.com/hex-inc/vegafusion/tree/main/vegafusion-core/src/proto, and then deserialize the _request_msg bytes object into a QueryResult Python object and then get the information you want out of there in Python. And I don't recall the syntax offhand, but you should also be able to register a custom callback function using the traitlets API that would get called whenever the value of _request_msg changes.

It's definitely a bit of a hack, but happy to answer more questions if that's something you want to try!

[cosmicfarmers]

Fantastic! It worked! Thanks a lot!!!

[cosmicfarmers]

I leave a message for those who also need to get selections in python side. @jonmmease Thanks again for saving my day!

Steps:

1. Install protoc: https://github.com/protocolbuffers/protobuf/releases
2. Clone vegafusion and copy all *.proto files to the current directory.

• git clone https://github.com/hex-inc/vegafusion.git
• cp vegafusion/vegafusion-core/src/proto/*.proto .

3. generate .py files to decode from all .proto files by:
   (maybe not all .proto files should be converted but there are some dependencies each other anyway.)

• e.g. protoc --python_out=. services.proto

5. import and parse QueryRequest by generated protoc decoder.
   For example,
   I made a function like:

def get_selection(vfw):
    import pyarrow as pa
    import services_pb2 # this is generated by protoc

    if (vfw._request_msg is None):
        return []
    
    query_request = services_pb2.QueryRequest()
    query_request.ParseFromString(vfw._request_msg)
    
    dfs = []
    for i, node in enumerate(query_request.task_graph_values.task_graph.nodes):
        if(not hasattr(node.task.value, 'table')):
            continue
        if(node.task.value.table == b''):
            continue

        #print(i)
        #print("-----------------")            
        value_bytes = node.task.value.table
        # Decode the Arrow IPC format data:
        buffer = pa.py_buffer(value_bytes)
        reader = pa.ipc.open_stream(buffer)
        try:
            record_batch = reader.read_next_batch()
            df = record_batch.to_pandas()
            if len(df) >= 1:
                dfs.append(df)
                #print(df)
        except StopIteration:
            #print("No more record batches to read.")
            df = None  # or some other default action

        #print("-----------------")
    return dfs

In jupyter, you can have a vegafusion widget chart by:

import vegafusion as vf
#my_altair_chart code here
my_altair_chart_in_vf = vf.jupyter.VegaFusionWidget(my_altair_chart, verbose=True)

after you display the vf chart, make some selections using mouse click.
after that, if you run this code in another jupyter cell, you will get selections you just made:
get_selection(my_altair_chart_in_vf)

You can modify get_selection function, if there is any exception case that I didn't cover in this code.

[jonmmease]

Great work @jhyoo1982! Glad this worked out for you

I'll try to remember to update this thread down the line when there's an easier approach.

[cosmicfarmers]

@jonmmease When you successfully remember this thread next time, here is another thing. :)

When I use .add_params(my_picker) and if the picker is defined with default selections like:
my_picker = alt.selection_point(fields=['my_field'], value=[{'my_field':some_val1}, {'my_field':some_val2}])
, both QueryRequest and QueryResult have nothing about the selection made, the default selections were applied in the chart by vega fusion widget, correctly, though.

I had multiple charts combined and also had multiple pickers but only pickers with default values set didn't have any evidence of selections I made in the _request_msg and _response_msg.

For now, I am not using selection with default value for this reason. (but it does work well without setting default values.)

Thanks for reading!