[BUG] TypeError: model must be a LightningModule or torch._dynamo.OptimizedModule, got TemporalFusionTransformer

[hagersalehahmed]

what is main reason of this error? 1 frames
/usr/local/lib/python3.10/dist-packages/pytorch_lightning/utilities/compile.py in _maybe_unwrap_optimized(model)
123 if isinstance(model, pl.LightningModule):
124 return model
--> 125 raise TypeError(
126 f"model must be a LightningModule or torch._dynamo.OptimizedModule, got {type(model).__qualname__}"
127 )

There is one subject_ID is 12345, I split data into 80% training and 20% testing

data['time_idx'] = range(1, len(data) + 1)
split_index = int(0.8 * len(data))
train_df = data[:split_index]
test_df = data[split_index:]
print(len(train_df))


scaler_rr = MinMaxScaler()
scaler_spo2 = MinMaxScaler()
train_df["RR"] = scaler_rr.fit_transform(train_df[["RR"]])
test_df["RR"] = scaler_rr.transform(test_df[["RR"]])

train_df["SpO2"] = scaler_spo2.fit_transform(train_df[["SpO2"]])
test_df["SpO2"] = scaler_spo2.transform(test_df[["SpO2"]])
test_df.head()


max_encoder_length = 5  # past
max_prediction_length = 3 #Future 


training_cutoff = train_df['time_idx'].max()-max_prediction_length 

train_df["time_idx"] = pd.factorize(train_df["time_idx"])[0]
test_df["time_idx"] = pd.factorize(test_df["time_idx"])[0]


max_encoder_length = 5
max_prediction_length = 3


training = TimeSeriesDataSet(
    train_df,
    time_idx="time_idx",
    target=["RR", "SpO2"],
    group_ids=["subject_id"],
    min_encoder_length=max_encoder_length // 2,
    max_encoder_length=max_encoder_length,
    min_prediction_length=1,
    max_prediction_length=max_prediction_length,
    static_categoricals=[],
    time_varying_known_reals=["time_idx"],
    time_varying_unknown_reals=["RR", "SpO2"],
    target_normalizer=MultiNormalizer(
        [
            GroupNormalizer(groups=["subject_id"], scale_by_group=True)
            for _ in ["RR", "SpO2"]
        ]
    ),
    add_relative_time_idx=True,
    add_target_scales=True,
    add_encoder_length=True,
)


testing = TimeSeriesDataSet.from_dataset(
    training,
    test_df,
    predict=True,
    stop_randomization=True,
)


batch_size = 16

train_dataloader = training.to_dataloader(
    train=True, batch_size=batch_size, num_workers=0
)

test_dataloader = testing.to_dataloader(
    train=False,  batch_size=batch_size*10, num_workers=0
)



trainer = pl.Trainer(
    accelerator="cpu",gradient_clip_val=0.1,
)


tft = TemporalFusionTransformer.from_dataset(
    training,
    # not meaningful for finding the learning rate but otherwise very important
    learning_rate=0.03,
    hidden_size=8,  # most important hyperparameter apart from learning rate
    # number of attention heads. Set to up to 4 for large datasets
    attention_head_size=1,
    dropout=0.1,  # between 0.1 and 0.3 are good values
    hidden_continuous_size=8,  # set to <= hidden_size
    loss=QuantileLoss(),
    optimizer="adam",
    
)
print(f"Number of parameters in network: {tft.size() / 1e3:.1f}k")

trainer.fit(
    tft,
    train_dataloaders=train_dataloader,
    val_dataloaders=test_dataloader,
)

[fnhirwa]

hey @hagersalehahmed would you please provide the full code example you were running for me to be able to reproduce the bug?
Thanks

[benHeid]

Based on the instantiation of your trainer and the error message I assume that you are using PyTorch-lightning. To fix this issue, please change to lightning.

@sktime/core-developers, this is an error that happens quiet often. Do you have an idea on how to make this more user-friendly?

[geetu040]

@sktime/core-developers, this is an error that happens quiet often. Do you have an idea on how to make this more user-friendly?

I don't really think we can do much about this. Since the base model for all forecasters uses lightning.pytorch.LightningModule, we cannot do much to this class to make it compatible with trainer of class pytorch_lightning.Trainer

[benHeid]

Sorry I was not that clear. It is not about making it compatible. It is about if we need to document it better somewhere. Or if there is a way to improve the error message (probably not)