How to add covariates by fine-tuning?

[Riofd]

Hello, I wan to know if there is any easy way to add covariates for forecasting by fine-tuning? Since Chronos does not support covariates, I just want to add more channels as input by adding a "cross-attention" layer at the input stage. My idea is to do this special "cross-attention" after input embedding. I want to freeze most of the layers and just train the new "cross-attention" layer and fine-tune the last linear layer (named lm_head in Chronos).

[Otsutsukii]

Interesting idea, maybe something I will try later, except if the authors came up with better ideas

[Riofd]

I think this is a easy way for downstream forecasting with covariates, and I am trying to do so. Looking forward to your feedback on the results~