Does the PAD token affect in fine-tuning?

[dafmdev]

We want to fine-tune Chronos but all our time series have lengths less than 512, which is the Chornos context window. Can this affect the performance of Chronos?

Is there information available related to this topic?

[lostella]

@dafmdev from a purely technical standpoint, training or fine-tuning with shorter series should not be an issue: PAD tokens will be used where needed, and the model will learn to ignore them.

But if you fine-tune on data with context length up to N < 512, then you probably want to also make sure to provide data with context length up to N at prediction time, since the model may not know what to do with longer contexts at that point.

[tomar-s]

Hi @lostella,
Can you please explain in more detail your response above with an example from the zero shot dataset like Tourism monthly where the length of series is 333. If I try to fine tune and then evaluate on the same dataset, what needs to be done differently for evaluation which was not done if it were a zero shot dataset used directly for evaluation?