Try to reconstruct the exp

[YOU-k]

Hi there,
I was trying to test the model's performance with a reconstruction task. I tried to make this work with the TransformerModel used.
And when I load the weight using the function load_pretrained:
I get layers that start with 'encoder, value_encoder, transformer_encoder, decoder ' all loaded,

the scatter plot for the averaged expression of each gene, where the x-axis means the predicted values from the model, and y is the predicted value.

I find that in the perturbation fine-tune task, the decoder is not loaded. so, I reload the model with 'encoder, value_encoder, transformer_encoder,' loaded, so the decoder is randomly initialized, and this time, the scatter plot looks like this:

Could you please help me understand why loading the decoder weight could end up with this result? Am I using the weight in the wrong way? and how could I reconstruct the data?

Cheers,
Yue

[a93sokol]

Having the same issue, still relevant.
My question is rather, why when nothing is masked the reconstruction quality/ranking is so off?