The amount of pretraining data

[liu-jc]

Hi @abdulfatir ,
I would like to ask how many data chronos used for pretraining and how to get the numbers stated in the paper. As mentioned in Sec 5.1, 890K univariate time series are used (in total roughly 84B tokens). Are these numbers for original raw data but not directly for training?

In Sec 5.2, it states 10M TSMixup augmentations and 1M synthetic time series (so total 11M time series). In my view, these numbers are the data used for training. Am I correct?

Besides, I would like to ask how many observations/tokens are used for training given that we are using 11M time series? I noticed that synthetic time series have 1024 tokens each as shown in https://github.com/amazon-science/chronos-forecasting/blob/main/scripts/kernel-synth.py. But how about TSMixup augmentations?

[lostella]

Are these numbers for original raw data but not directly for training?

That's correct.

In Sec 5.2, it states 10M TSMixup augmentations and 1M synthetic time series (so total 11M time series). In my view, these numbers are the data used for training. Am I correct?

That's correct.

But how about TSMixup augmentations?

As stated in Algorithm 1 in the paper, TSMixup augmentations were generated with length uniformly sampled between 128 and 2048. That gives an average length of 1088. Therefore approximately 1088 * 10^7 observations were generated with this process.

[liu-jc]

Hi @lostella

Thanks for the prompt clarification! Therefore, total number of tokens/observations used in training is ~ 1088 * 10M + 1M * 1024 = ~11B. That is even less than the number of observations (84B) in raw data. Do I read it correctly? If no, could you pls help to correct me.

So, in my view, what you have done is like condensing/denoising the data while making them more diverse by using mixup? If possible, could you share the insights/rationales behind this design? 😃