'GELU' object has no attribute 'approximate'

[Sarthak2426]

i have followed the steps but its showing me this error 'GELU' object has no attribute 'approximate'

[chengshuxiao]

i have followed the steps but its showing me this error 'GELU' object has no attribute 'approximate'

You could try to switch your pytorch version to "1.11.0".

[ChuykoMihail]

Hello. Unfortunately, I need the latest version of torch for other functions in my project. Also version 1.11.0 is not available for download with pip and requires reinstallation of other libraries in my project. Is there anything else i could do? I didnt find good answer on the internet.
I did remove 'approximate' argument from venv/lib/python3.11/site-packages/torch/nn/modules/activation.py and it seems it worked but is there a better way?

[rgaignoux]

Hello. Same problem with pytorch 2.0.1

[alihajqani]

Hi guys, This error is because the GELU function in the old version of pytorch does not have the parameter approximate. I replaced the GELU function which is at:

‍‍‍‍env/lib/python3.11/site-packages/torch/nn/modules/activation.py

as following function and worked for me.

class GELU(Module):
    r"""Applies the Gaussian Error Linear Units function:

    .. math:: \text{GELU}(x) = x * \Phi(x)

    where :math:`\Phi(x)` is the Cumulative Distribution Function for Gaussian Distribution.

    Shape:
        - Input: :math:`(N, *)` where `*` means, any number of additional
          dimensions
        - Output: :math:`(N, *)`, same shape as the input

    .. image:: ../scripts/activation_images/GELU.png

    Examples::

        >>> m = nn.GELU()
        >>> input = torch.randn(2)
        >>> output = m(input)
    """
    def forward(self, input: Tensor) -> Tensor:
        return F.gelu(input)

I saw this solution here.