Cast ScalarImage and LabelMap 'dtype' To A Different Tensor Type

[washieuanan]

Hi,

I've been working on a segmentation task using TorchIO. I've been trying to train a model but I keep on receiving the following error:

RuntimeError: expected scalar type Short but found Float

If I examine a ScalarImage using training_set[0]['mri'] after transformations, I receive the following output:

ScalarImage(shape: (1, 512, 512, 100); spacing: (0.47, 0.47, 3.00); orientation: RAS+; memory: 100.0 MiB; dtype: torch.FloatTensor)

Is there a way to cast the dtype of a ScalarImage and LabelMap from torch.FloatTensor to torch.ShortTensor? If there aren't any solutions within TorchIO, are there any other solutions that may work? Thanks in advance.

[fepegar]

Hi, @washieuanan. What were you trying to do? Can you post the full error traceback?

[washieuanan]

I was trying out TorchIO and using its tutorial. I attempted to train the model using my images but I received the following traceback error:

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-37-5ae6b15f4bc3> in <module>
      3 if train_whole_images:
      4     weights_stem = 'whole_images'
----> 5     train_losses, val_losses = train(num_epochs, training_loader, validation_loader, model, optimizer, weights_stem)
      6     checkpoint = {
      7         'train_losses': train_losses,

<ipython-input-35-b6b089d7836f> in train(num_epochs, training_loader, validation_loader, model, optimizer, weights_stem)
     69     train_losses = []
     70     val_losses = []
---> 71     val_losses.append(run_epoch(0, Action.VALIDATE, validation_loader, model, optimizer))
     72     for epoch_idx in range(1, num_epochs + 1):
     73         print('Starting epoch', epoch_idx)

<ipython-input-35-b6b089d7836f> in run_epoch(epoch_idx, action, loader, model, optimizer)
     53         optimizer.zero_grad()
     54         with torch.set_grad_enabled(is_training):
---> 55             logits = model(inputs)
     56             probabilities = F.softmax(logits, dim=CHANNELS_DIMENSION)
     57             batch_losses = get_dice_loss(probabilities, targets)

~/miniconda3/lib/python3.8/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
   1049         if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
   1050                 or _global_forward_hooks or _global_forward_pre_hooks):
-> 1051             return forward_call(*input, **kwargs)
   1052         # Do not call functions when jit is used
   1053         full_backward_hooks, non_full_backward_hooks = [], []

~/miniconda3/lib/python3.8/site-packages/unet/unet.py in forward(self, x)
    118 
    119     def forward(self, x):
--> 120         skip_connections, encoding = self.encoder(x)
    121         encoding = self.bottom_block(encoding)
    122         x = self.decoder(skip_connections, encoding)

~/miniconda3/lib/python3.8/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
   1049         if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
   1050                 or _global_forward_hooks or _global_forward_pre_hooks):
-> 1051             return forward_call(*input, **kwargs)
   1052         # Do not call functions when jit is used
   1053         full_backward_hooks, non_full_backward_hooks = [], []

~/miniconda3/lib/python3.8/site-packages/unet/encoding.py in forward(self, x)
     56         skip_connections = []
     57         for encoding_block in self.encoding_blocks:
---> 58             x, skip_connnection = encoding_block(x)
     59             skip_connections.append(skip_connnection)
     60         return skip_connections, x

~/miniconda3/lib/python3.8/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
   1049         if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
   1050                 or _global_forward_hooks or _global_forward_pre_hooks):
-> 1051             return forward_call(*input, **kwargs)
   1052         # Do not call functions when jit is used
   1053         full_backward_hooks, non_full_backward_hooks = [], []

~/miniconda3/lib/python3.8/site-packages/unet/encoding.py in forward(self, x)
    146             x += connection
    147         else:
--> 148             x = self.conv1(x)
    149             x = self.conv2(x)
    150         if self.downsample is None:

~/miniconda3/lib/python3.8/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
   1049         if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
   1050                 or _global_forward_hooks or _global_forward_pre_hooks):
-> 1051             return forward_call(*input, **kwargs)
   1052         # Do not call functions when jit is used
   1053         full_backward_hooks, non_full_backward_hooks = [], []

~/miniconda3/lib/python3.8/site-packages/unet/conv.py in forward(self, x)
     77 
     78     def forward(self, x):
---> 79         return self.block(x)
     80 
     81     @staticmethod

~/miniconda3/lib/python3.8/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
   1049         if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
   1050                 or _global_forward_hooks or _global_forward_pre_hooks):
-> 1051             return forward_call(*input, **kwargs)
   1052         # Do not call functions when jit is used
   1053         full_backward_hooks, non_full_backward_hooks = [], []

~/miniconda3/lib/python3.8/site-packages/torch/nn/modules/container.py in forward(self, input)
    137     def forward(self, input):
    138         for module in self:
--> 139             input = module(input)
    140         return input
    141 

~/miniconda3/lib/python3.8/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
   1049         if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
   1050                 or _global_forward_hooks or _global_forward_pre_hooks):
-> 1051             return forward_call(*input, **kwargs)
   1052         # Do not call functions when jit is used
   1053         full_backward_hooks, non_full_backward_hooks = [], []

~/miniconda3/lib/python3.8/site-packages/torch/nn/modules/conv.py in forward(self, input)
    585 
    586     def forward(self, input: Tensor) -> Tensor:
--> 587         return self._conv_forward(input, self.weight, self.bias)
    588 
    589 

~/miniconda3/lib/python3.8/site-packages/torch/nn/modules/conv.py in _conv_forward(self, input, weight, bias)
    580                 self.groups,
    581             )
--> 582         return F.conv3d(
    583             input, weight, bias, self.stride, self.padding, self.dilation, self.groups
    584         )

RuntimeError: expected scalar type Short but found Float

After transformations, the ScalarImage and LabelMap have a dtype of torch.FloatTensor. Is there a way to cast the Tensor type of the ScalarImage and LabelMap from torch.FloatTensor to torch.ShortTensor? Or should I try to change the model itself (I'm using the unet pip package).

[fepegar]

You can cast the data in the images accessing using the data property and set_data method, but you shouldn't need to. In line 55, you have logits = model(inputs). The type of inputs is probably short and that's causing the error:

In [1]: import unet
   ...: import torch
   ...: x = torch.rand(1, 1, 512, 512).short()
   ...: net = unet.UNet2D(padding=1)
   ...: net(x)

[...]
/usr/local/Caskroom/miniconda/base/envs/torchio/lib/python3.8/site-packages/torch/nn/modules/conv.py in _conv_forward(self, input, weight, bias)
    437                             weight, bias, self.stride,
    438                             _pair(0), self.dilation, self.groups)
--> 439         return F.conv2d(input, weight, bias, self.stride,
    440                         self.padding, self.dilation, self.groups)
    441

RuntimeError: expected scalar type Short but found Float

Please use logits = model(inputs.float()) instead and let us know if that helped.

[washieuanan]

Thanks @fepegar. It seems to have worked. Good to know that there's to way to cast the dtype.