Replacement of the gpt2 large model

[LunarKangaroo]

Hello, dear authors, I refer to the paper that chronos can be replaced with a pure decoder model, then I tried the causal based llama2 as well as gpt2, and both of them encountered the batchsize mismatch as below, it seems that the trainer module mistakenly treats the context as a predicted value with the target value to calculate the loss of cross-entropy. Do you know how to solve this problem?

[abdulfatir]

How exactly did you try to do that?

[LunarKangaroo]

Thank you for your reply! I just change the model_id,model_type and the config about the large_model, like this:

If I change to the seq2seq model( like t5,bart and so on) I don't get the error above. Have you encountered any of the above problems?

[abdulfatir]

Please share text when asking questions. Screenshots are difficult to parse and not helpful. For causal models, please check and modify the reference config: https://github.com/amazon-science/chronos-forecasting/blob/main/scripts/training/configs/chronos-gpt2.yaml

[LunarKangaroo]

I use your advice and try again, here is the same error occurred:
Traceback (most recent call last):
File "train.py", line 647, in
app()
File "/root/miniconda3/envs/chronos2/lib/python3.8/site-packages/typer/main.py", line 326, in call
raise e
File "/root/miniconda3/envs/chronos2/lib/python3.8/site-packages/typer/main.py", line 309, in call
return get_command(self)(*args, **kwargs)
File "/root/miniconda3/envs/chronos2/lib/python3.8/site-packages/click/core.py", line 1157, in call
return self.main(*args, **kwargs)
File "/root/miniconda3/envs/chronos2/lib/python3.8/site-packages/typer/core.py", line 661, in main
return _main(
File "/root/miniconda3/envs/chronos2/lib/python3.8/site-packages/typer/core.py", line 193, in _main
rv = self.invoke(ctx)
File "/root/miniconda3/envs/chronos2/lib/python3.8/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/root/miniconda3/envs/chronos2/lib/python3.8/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
File "/root/miniconda3/envs/chronos2/lib/python3.8/site-packages/typer/main.py", line 692, in wrapper
return callback(**use_params)
File "/root/miniconda3/envs/chronos2/lib/python3.8/site-packages/typer_config/decorators.py", line 92, in wrapped
return cmd(*args, **kwargs)
File "train.py", line 634, in main
trainer.train()
File "/root/miniconda3/envs/chronos2/lib/python3.8/site-packages/transformers/trainer.py", line 1859, in train
return inner_training_loop(
File "/root/miniconda3/envs/chronos2/lib/python3.8/site-packages/transformers/trainer.py", line 2203, in _inner_training_loop
tr_loss_step = self.training_step(model, inputs)
File "/root/miniconda3/envs/chronos2/lib/python3.8/site-packages/transformers/trainer.py", line 3138, in training_step
loss = self.compute_loss(model, inputs)
File "/root/miniconda3/envs/chronos2/lib/python3.8/site-packages/transformers/trainer.py", line 3161, in compute_loss
outputs = model(**inputs)
File "/root/miniconda3/envs/chronos2/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/root/miniconda3/envs/chronos2/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
return forward_call(*args, **kwargs)
File "/root/miniconda3/envs/chronos2/lib/python3.8/site-packages/torch/_dynamo/eval_frame.py", line 451, in _fn
return fn(*args, **kwargs)
File "/root/miniconda3/envs/chronos2/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/root/miniconda3/envs/chronos2/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
return forward_call(*args, **kwargs)
File "/root/miniconda3/envs/chronos2/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 921, in catch_errors
return callback(frame, cache_entry, hooks, frame_state, skip=1)
File "/root/miniconda3/envs/chronos2/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 786, in _convert_frame
result = inner_convert(
File "/root/miniconda3/envs/chronos2/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 400, in _convert_frame_assert
return _compile(
File "/root/miniconda3/envs/chronos2/lib/python3.8/contextlib.py", line 75, in inner
return func(*args, **kwds)
File "/root/miniconda3/envs/chronos2/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 676, in _compile
guarded_code = compile_inner(code, one_graph, hooks, transform)
File "/root/miniconda3/envs/chronos2/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 262, in time_wrapper
r = func(*args, **kwargs)
File "/root/miniconda3/envs/chronos2/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 535, in compile_inner
out_code = transform_code_object(code, transform)
File "/root/miniconda3/envs/chronos2/lib/python3.8/site-packages/torch/_dynamo/bytecode_transformation.py", line 1036, in transform_code_object
transformations(instructions, code_options)
File "/root/miniconda3/envs/chronos2/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 165, in _fn
return fn(*args, **kwargs)
File "/root/miniconda3/envs/chronos2/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 500, in transform
tracer.run()
File "/root/miniconda3/envs/chronos2/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 2149, in run
super().run()
File "/root/miniconda3/envs/chronos2/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 810, in run
and self.step()
File "/root/miniconda3/envs/chronos2/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 773, in step
getattr(self, inst.opname)(inst)
File "/root/miniconda3/envs/chronos2/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 489, in wrapper
return inner_fn(self, inst)
File "/root/miniconda3/envs/chronos2/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1219, in CALL_FUNCTION
self.call_function(fn, args, {})
File "/root/miniconda3/envs/chronos2/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 674, in call_function
self.push(fn.call_function(self, args, kwargs))
File "/root/miniconda3/envs/chronos2/lib/python3.8/site-packages/torch/_dynamo/variables/misc.py", line 523, in call_function
return self.fn(*args, **kwargs)
File "/root/miniconda3/envs/chronos2/lib/python3.8/site-packages/torch/_dynamo/variables/user_defined.py", line 183, in fake_cross_entropy_loss
return wrap_fx_proxy(
File "/root/miniconda3/envs/chronos2/lib/python3.8/site-packages/torch/_dynamo/variables/builder.py", line 1330, in wrap_fx_proxy
return wrap_fx_proxy_cls(target_cls=TensorVariable, **kwargs)
File "/root/miniconda3/envs/chronos2/lib/python3.8/site-packages/torch/_dynamo/variables/builder.py", line 1415, in wrap_fx_proxy_cls
example_value = get_fake_value(proxy.node, tx, allow_non_graph_fake=True)
File "/root/miniconda3/envs/chronos2/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 1714, in get_fake_value
raise TorchRuntimeError(str(e)).with_traceback(e.traceback) from None
File "/root/miniconda3/envs/chronos2/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 1656, in get_fake_value
ret_val = wrap_fake_exception(
File "/root/miniconda3/envs/chronos2/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 1190, in wrap_fake_exception
return fn()
File "/root/miniconda3/envs/chronos2/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 1657, in
lambda: run_node(tx.output, node, args, kwargs, nnmodule)
File "/root/miniconda3/envs/chronos2/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 1782, in run_node
raise RuntimeError(make_error_message(e)).with_traceback(
File "/root/miniconda3/envs/chronos2/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 1764, in run_node
return node.target(args, **kwargs)
File "/root/miniconda3/envs/chronos2/lib/python3.8/site-packages/torch/nn/functional.py", line 3086, in cross_entropy
return torch._C._nn.cross_entropy_loss(input, target, weight, _Reduction.get_enum(reduction), ignore_index, label_smoothing)
torch._dynamo.exc.TorchRuntimeError: Failed running call_function <function cross_entropy at 0x7f4378de2700>((FakeTensor(..., device='cuda:0', size=(16352, 4096), grad_fn=), FakeTensor(..., device='cuda:0', size=(2048,), dtype=torch.int64), None, None, -100, None, 'mean', 0.0), **{}):
Expected input batch_size (16352) to match target batch_size (2048).

from user code:
File "/root/miniconda3/envs/chronos2/lib/python3.8/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1338, in forward
loss = loss_fct(shift_logits.view(-1, shift_logits.size(-1)), shift_labels.view(-1))

Set TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 for more information

[abdulfatir]

Please share exact steps to reproduce this error. Did you try higher versions of python? 3.10, for example.

[LunarKangaroo]

The exact step is CUDA_VISIBLE_DEVICES=0 python fit_official2.py --config ./training/configs/chronos-gpt2.yaml --no-random-init --max-steps 1000 --learning-rate 0.001
I try to train GPT2 to forcast the ts. I will try higher version python later, Thank you for your reply!

[abdulfatir]

I just tested the following on my machine.

CUDA_VISIBLE_DEVICES=0 python scripts/training/train.py --config scripts/training/configs/chronos-gpt2.yaml --no-random-init --max-steps 1000 --learning-rate 0.001

It works as expected. Try changing your python version to 3.10.

Also, please note that we have not released the GPT2 variant of Chronos, so what you're doing will "fine-tune" the GPT2 language model on HuggingFace. Ideally you should first pretrain it on a large collection of time series data before fine-tuning on your dataset.