Few Code-Script Questions about Multimodal Late-Fusion

[net-zero-2050]

Hi, I hope you're doing well.

While reviewing your GitHub repo and paper on the "Multimodal Late-Fusion" study, I had a few follow-up questions about the code:

1. Regarding the GSP-ID input, the paper mentions the dimension as GSP (1, ), indicating it's a single 1-dimensional value. Could you clarify how this works? Are you embedding a value like GSP_ID = 47 directly into the OrderedDict for your output_network?

2. In multimodal.yaml, I noticed that the in_channels for UKV is set to 2, while ECMWF is 12. Where do these values come from? My understanding was that UKV has dimensions (11, 24, 24, 6) and ECMWF (11, 12, 12, 6), with in_channels representing the number of variables. For instance, the satellite is 11 because there are 11 channels....

3. You're using a batch size of 128, correct? Just to confirm, does this mean you're working with 128 Zarr files per source? Also, could you let me know where to find the code? I'm curious about whether you concatenate these 128 Zarr files into a DataFrame or how you ensure compatibility with your input dimensions (e.g., in_channels) in your ML code.

Thanks in advance for any clarification!

[emlaura]

Thanks for your question @net-zero-2050! I've flagged with our technical team and someone should be responding ASAP 😊

[peterdudfield]

Hi @net-zero-2050, thanks for getting in contact. Ill try to answer these, but @Sukh-P and @AUdaltsova please do jump in on anything

1. So the gsp_id is just one number, but its gets embedded into a higher dimension here. This should allow the model to learn specific behaviours for some gsps but also group the behaviour together of similar gsp_ids.

2. Do you mind sharing where multimodal.yaml file is? You are write the channel dimension is the first one in that block of four. Our latest model is shared here with various configuration files

3. Our batch size is normally a bit smaller than that, but it totally depends on your hardware and your model and data configuration. We work with 1 main zarr file, and then each sample take a bit of that data, ready to show to the model. We've used ocf_datapipes for this data pipeline, but we are currently moving over to ocf-data-sampler. In general we like to use a data configuration file for the data we load in, and save this in the model, so that we know exactly what the data and input dimensions are.

Hopefully these answers help a bit, but please let me know any follow up questions

[peterdudfield]

Thanks @net-zero-2050

1. Thanks for sharing the link - that is very much an example. I would recommend using the files here as they are one of our latest models.

2. So the one large zarr file covers a large amount of time and space ( e.g 5 years of time and the entire UK). Each sample you mention has a small portion of that, for example 8 hour and only a section of the UK. Probably good to note that with have a one large file for each data input, so 1 for satellite, and 1 for each nwp provider that we use.

Let us know if this helps, or you have any more questions

[net-zero-2050]

@peterdudfield, thank you for your response.

I have a few follow-up questions. In your published paper (link), you mention that the model is designed to predict 0 to 8 hours ahead (pg. 1). However, Figure 4 (pg. 10) shows an output dimension of (16, 3), which I assume represents the shape of the target variable, Y.

Could you clarify this? I couldn't find any details in your code regarding the structure or specifics of the target variable, Y.

[peterdudfield]

Thanks @net-zero-2050

Yea, so we do predictions in 30 minutes intervals so the 8 hours turns into 16 data points. The 3 comes from predicting 3 different probability levels, p10, p50 and p90. The target variable is still (16,1) but we use a quantile loss function. Does this make sense? Thanks

[net-zero-2050]

@peterdudfield
Amazingly clear answer... thank you always for your help!

[peterdudfield]

Thanks, do get in touch if you have any further questions