Question about hierarchical Structure

[Titolasanta]

Hi, researching your paper I read that the used architecture utilizes a hierarchical Structure in a similar way as in the Hourglass transformer ( https://arxiv.org/abs/2110.13711 ). But for your case you are utilizing a encoder-decoder transformer, so how do you reshape what would usually be the input for your decoders which consists of both the output of the encoder which I would consider to be downsampled and the original input, which would be in the original input shape?

Thank you

[gaozhihan]

Thank you for your interest and your question. It may help to refer to the following piece of our implementation:

earth-forecasting-transformer/src/earthformer/cuboid_transformer/cuboid_transformer.py

Lines 3192 to 3195 in 7732b03

	if self.num_global_vectors > 0:
	dec_out = self.decoder(initial_z, mem_l, mem_global_vector_l)
	else:
	dec_out = self.decoder(initial_z, mem_l)

The decoder typically takes as input the outputs from each layer of the encoder, as well as a dummy input named initial_z that matches the shape of the encoder's final layer output.