Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Unitest Error #176

Closed
wangtianrui opened this issue Apr 2, 2023 · 12 comments
Closed

Unitest Error #176

wangtianrui opened this issue Apr 2, 2023 · 12 comments
Assignees
@bamsumit
Labels
0-needs-review 1-bug Something isn't working

Comments

@wangtianrui
Copy link

Describe the bug
Follow the official tutorial. I installed the lava first. Then lava-dl was installed. But I couldn't pass the unit test.

To reproduce current behavior
Steps to reproduce the behavior:

  1. I installed the lava. And passed the unitest.
Required test coverage of 65.0% reached. Total coverage: 87.74%
=================================== 396 passed, 5 skipped, 4 warnings in 832.50s (0:13:52) ===================================
  1. I installed the lava-dl
export PYTHONPATH=/home/studios/lava/src
git clone [email protected]:lava-nc/lava-dl.git
cd lava-dl
curl -sSL https://install.python-poetry.org | python3 -
poetry config virtualenvs.in-project true
poetry install
pytest

but I cant pass the unitest:

The above exception was the direct cause of the following exception:
tests/lava/lib/dl/slayer/neuron/dynamics/test_adaptive_threshold.py:55: in <module>
    voltage = slayer.neuron.dynamics.leaky_integrator.dynamics(
src/lava/lib/dl/slayer/neuron/dynamics/leaky_integrator.py:95: in dynamics
    output = Accelerated.leaky_integrator.dynamics(
src/lava/lib/dl/slayer/utils/utils.py:14: in __get__
    return staticmethod(self.fget).__get__(None, owner)()
src/lava/lib/dl/slayer/neuron/dynamics/leaky_integrator.py:40: in leaky_integrator
    Accelerated.module = load(
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/utils/cpp_extension.py:1202: in load
    return _jit_compile(
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/utils/cpp_extension.py:1425: in _jit_compile
    _write_ninja_file_and_build_library(
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/utils/cpp_extension.py:1537: in _write_ninja_file_and_build_library
    _run_ninja_build(
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/utils/cpp_extension.py:1824: in _run_ninja_build
    raise RuntimeError(message) from e
E   RuntimeError: Error building extension 'dynamics_v5': [1/2] :/usr/local/cuda:/usr/local/cuda/bin/nvcc  -DTORCH_EXTENSION_NAME=dynamics_v5 -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -isystem /home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include -isystem /home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include -isystem /home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/TH -isystem /home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/THC -isystem :/usr/local/cuda:/usr/local/cuda/include -isystem /home/wang/.conda/envs/lava/include/python3.9 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_61,code=compute_61 -gencode=arch=compute_61,code=sm_61 --compiler-options '-fPIC' -std=c++14 -c /home/wang/codes/py/XTeam/studios/lava-dl/src/lava/lib/dl/slayer/neuron/dynamics/leaky_integrator.cu -o leaky_integrator.cuda.o 
E   FAILED: leaky_integrator.cuda.o 
E   :/usr/local/cuda:/usr/local/cuda/bin/nvcc  -DTORCH_EXTENSION_NAME=dynamics_v5 -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -isystem /home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include -isystem /home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include -isystem /home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/TH -isystem /home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/THC -isystem :/usr/local/cuda:/usr/local/cuda/include -isystem /home/wang/.conda/envs/lava/include/python3.9 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_61,code=compute_61 -gencode=arch=compute_61,code=sm_61 --compiler-options '-fPIC' -std=c++14 -c /home/wang/codes/py/XTeam/studios/lava-dl/src/lava/lib/dl/slayer/neuron/dynamics/leaky_integrator.cu -o leaky_integrator.cuda.o 
E   /bin/sh: 1: :/usr/local/cuda:/usr/local/cuda/bin/nvcc: not found
E   ninja: build stopped: subcommand failed.
_____ ERROR collecting tests/lava/lib/dl/slayer/neuron/dynamics/test_leaky_integrator.py _____
ImportError while importing test module '/home/wang/codes/py/XTeam/studios/lava-dl/tests/lava/lib/dl/slayer/neuron/dynamics/test_leaky_integrator.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/home/wang/.conda/envs/lava/lib/python3.9/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/lava/lib/dl/slayer/neuron/dynamics/test_leaky_integrator.py:55: in <module>
    output0 = slayer.neuron.dynamics.leaky_integrator.dynamics(
src/lava/lib/dl/slayer/neuron/dynamics/leaky_integrator.py:95: in dynamics
    output = Accelerated.leaky_integrator.dynamics(
src/lava/lib/dl/slayer/utils/utils.py:14: in __get__
    return staticmethod(self.fget).__get__(None, owner)()
src/lava/lib/dl/slayer/neuron/dynamics/leaky_integrator.py:40: in leaky_integrator
    Accelerated.module = load(
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/utils/cpp_extension.py:1202: in load
    return _jit_compile(
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/utils/cpp_extension.py:1450: in _jit_compile
    return _import_module_from_library(name, build_directory, is_python_module)
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/utils/cpp_extension.py:1844: in _import_module_from_library
    module = importlib.util.module_from_spec(spec)
E   ImportError: /home/wang/.cache/torch_extensions/py39_cu102/dynamics/dynamics_v5.so: cannot open shared object file: No such file or directory
________ ERROR collecting tests/lava/lib/dl/slayer/neuron/dynamics/test_resonator.py _________
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/utils/cpp_extension.py:1808: in _run_ninja_build
    subprocess.run(
/home/wang/.conda/envs/lava/lib/python3.9/subprocess.py:528: in run
    raise CalledProcessError(retcode, process.args,
E   subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:
tests/lava/lib/dl/slayer/neuron/dynamics/test_resonator.py:62: in <module>
    re, im = slayer.neuron.dynamics.resonator.dynamics(
src/lava/lib/dl/slayer/neuron/dynamics/resonator.py:118: in dynamics
    real, imag = _ResDynamics.apply(
src/lava/lib/dl/slayer/neuron/dynamics/resonator.py:196: in forward
    _real, _imag = Accelerated.resonator.fwd(
src/lava/lib/dl/slayer/utils/utils.py:14: in __get__
    return staticmethod(self.fget).__get__(None, owner)()
src/lava/lib/dl/slayer/neuron/dynamics/resonator.py:39: in resonator
    Accelerated.module = load(
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/utils/cpp_extension.py:1202: in load
    return _jit_compile(
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/utils/cpp_extension.py:1425: in _jit_compile
    _write_ninja_file_and_build_library(
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/utils/cpp_extension.py:1537: in _write_ninja_file_and_build_library
    _run_ninja_build(
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/utils/cpp_extension.py:1824: in _run_ninja_build
    raise RuntimeError(message) from e
E   RuntimeError: Error building extension 'dynamics_v6': [1/2] :/usr/local/cuda:/usr/local/cuda/bin/nvcc  -DTORCH_EXTENSION_NAME=dynamics_v6 -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -isystem /home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include -isystem /home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include -isystem /home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/TH -isystem /home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/THC -isystem :/usr/local/cuda:/usr/local/cuda/include -isystem /home/wang/.conda/envs/lava/include/python3.9 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_61,code=compute_61 -gencode=arch=compute_61,code=sm_61 --compiler-options '-fPIC' -std=c++14 -c /home/wang/codes/py/XTeam/studios/lava-dl/src/lava/lib/dl/slayer/neuron/dynamics/resonator.cu -o resonator.cuda.o 
E   FAILED: resonator.cuda.o 
E   :/usr/local/cuda:/usr/local/cuda/bin/nvcc  -DTORCH_EXTENSION_NAME=dynamics_v6 -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -isystem /home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include -isystem /home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include -isystem /home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/TH -isystem /home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/THC -isystem :/usr/local/cuda:/usr/local/cuda/include -isystem /home/wang/.conda/envs/lava/include/python3.9 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_61,code=compute_61 -gencode=arch=compute_61,code=sm_61 --compiler-options '-fPIC' -std=c++14 -c /home/wang/codes/py/XTeam/studios/lava-dl/src/lava/lib/dl/slayer/neuron/dynamics/resonator.cu -o resonator.cuda.o 
E   /bin/sh: 1: :/usr/local/cuda:/usr/local/cuda/bin/nvcc: not found
E   ninja: build stopped: subcommand failed.
====================================== warnings summary ======================================
../../../../../.conda/envs/lava/lib/python3.9/site-packages/pytest_cov/plugin.py:256
  /home/wang/.conda/envs/lava/lib/python3.9/site-packages/pytest_cov/plugin.py:256: PytestDeprecationWarning: The hookimpl CovPlugin.pytest_configure_node uses old-style configuration options (marks or attributes).
  Please use the pytest.hookimpl(optionalhook=True) decorator instead
   to configure the hooks.
   See https://docs.pytest.org/en/latest/deprecations.html#configuring-hook-specs-impls-using-markers
    def pytest_configure_node(self, node):

../../../../../.conda/envs/lava/lib/python3.9/site-packages/pytest_cov/plugin.py:265
  /home/wang/.conda/envs/lava/lib/python3.9/site-packages/pytest_cov/plugin.py:265: PytestDeprecationWarning: The hookimpl CovPlugin.pytest_testnodedown uses old-style configuration options (marks or attributes).
  Please use the pytest.hookimpl(optionalhook=True) decorator instead
   to configure the hooks.
   See https://docs.pytest.org/en/latest/deprecations.html#configuring-hook-specs-impls-using-markers
    def pytest_testnodedown(self, node, error):

tests/lava/lib/dl/netx/test_blocks.py:32
  /home/wang/codes/py/XTeam/studios/lava-dl/tests/lava/lib/dl/netx/test_blocks.py:32: PytestCollectionWarning: cannot collect test class 'TestRunConfig' because it has a __init__ constructor (from: tests/lava/lib/dl/netx/test_blocks.py)
    class TestRunConfig(RunConfig):

tests/lava/lib/dl/netx/test_hdf5.py:29
  /home/wang/codes/py/XTeam/studios/lava-dl/tests/lava/lib/dl/netx/test_hdf5.py:29: PytestCollectionWarning: cannot collect test class 'TestRunConfig' because it has a __init__ constructor (from: tests/lava/lib/dl/netx/test_hdf5.py)
    class TestRunConfig(RunConfig):

../../../../../.conda/envs/lava/lib/python3.9/site-packages/pkg_resources/__init__.py:121
  /home/wang/.conda/envs/lava/lib/python3.9/site-packages/pkg_resources/__init__.py:121: DeprecationWarning: pkg_resources is deprecated as an API
    warnings.warn("pkg_resources is deprecated as an API", DeprecationWarning)

../../../../../.conda/envs/lava/lib/python3.9/site-packages/pkg_resources/__init__.py:2870
  /home/wang/.conda/envs/lava/lib/python3.9/site-packages/pkg_resources/__init__.py:2870: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('mpl_toolkits')`.
  Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
    declare_namespace(pkg)

../../../../../.conda/envs/lava/lib/python3.9/site-packages/pkg_resources/__init__.py:2870
../../../../../.conda/envs/lava/lib/python3.9/site-packages/pkg_resources/__init__.py:2870
../../../../../.conda/envs/lava/lib/python3.9/site-packages/pkg_resources/__init__.py:2870
../../../../../.conda/envs/lava/lib/python3.9/site-packages/pkg_resources/__init__.py:2870
../../../../../.conda/envs/lava/lib/python3.9/site-packages/pkg_resources/__init__.py:2870
../../../../../.conda/envs/lava/lib/python3.9/site-packages/pkg_resources/__init__.py:2870
  /home/wang/.conda/envs/lava/lib/python3.9/site-packages/pkg_resources/__init__.py:2870: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('sphinxcontrib')`.
  Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
    declare_namespace(pkg)




FAIL Required test coverage of 45.0% not reached. Total coverage: 32.69%
================================== short test summary info ===================================
ERROR tests/lava/lib/dl/slayer/neuron/test_adrf.py - RuntimeError: Error building extension 'dynamics': [1/2] :/usr/local/cuda:/usr/local/cuda...
ERROR tests/lava/lib/dl/slayer/neuron/test_adrf_iz.py - RuntimeError: Error building extension 'dynamics_v1': [1/2] :/usr/local/cuda:/usr/local/c...
ERROR tests/lava/lib/dl/slayer/neuron/test_alif.py - RuntimeError: Error building extension 'dynamics_v2': [1/2] :/usr/local/cuda:/usr/local/c...
ERROR tests/lava/lib/dl/slayer/neuron/test_cuba.py
ERROR tests/lava/lib/dl/slayer/neuron/test_rf.py - RuntimeError: Error building extension 'dynamics_v3': [1/2] :/usr/local/cuda:/usr/local/c...
ERROR tests/lava/lib/dl/slayer/neuron/test_rf_iz.py
ERROR tests/lava/lib/dl/slayer/neuron/test_sigma_delta.py
ERROR tests/lava/lib/dl/slayer/neuron/dynamics/test_adaptive_phase_th.py
ERROR tests/lava/lib/dl/slayer/neuron/dynamics/test_adaptive_resonator.py - RuntimeError: Error building extension 'dynamics_v4': [1/2] :/usr/local/cuda:/usr/local/c...
ERROR tests/lava/lib/dl/slayer/neuron/dynamics/test_adaptive_threshold.py - RuntimeError: Error building extension 'dynamics_v5': [1/2] :/usr/local/cuda:/usr/local/c...
ERROR tests/lava/lib/dl/slayer/neuron/dynamics/test_leaky_integrator.py
ERROR tests/lava/lib/dl/slayer/neuron/dynamics/test_resonator.py - RuntimeError: Error building extension 'dynamics_v6': [1/2] :/usr/local/cuda:/usr/local/c...
!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 12 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!
============================== 12 warnings, 12 errors in 7.53s ===============================

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem. Remove section otherwise.

Environment (please complete the following information):

  • Linux version 4.15.0-142-generic (buildd@lgw01-amd64-039) (gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.12))
  • torch=1.12.1 cuda=11.3
  • Lava version [e.g. 0.6.0]
@wangtianrui wangtianrui added the 1-bug Something isn't working label Apr 2, 2023
@bamsumit
Copy link
Contributor

bamsumit commented Apr 3, 2023

Hi @wangtianrui it looks like you are trying to install it on a GPU machine, but you don't have a nvcc compiler installed or configured correctly.

This is the error message from your logs

E   /bin/sh: 1: :/usr/local/cuda:/usr/local/cuda/bin/nvcc: not found

The solution is to install nvcc compiler on your machine with the same cuda version as your cuda runtime.
Cuda runtime can be viewed as

nvidia-smi | grep CUDA

Verify nvcc installation by running

nvcc --version

Finally make sure torch cuda version also matches.

@bamsumit bamsumit self-assigned this Apr 3, 2023
@wangtianrui
Copy link
Author

wangtianrui commented Apr 4, 2023 •
edited
Loading

Thank you for your reply. I just confirmed it once. My nvcc and pytorch are working fine.

(lava) wang@speech-ubuntu:~/codes/py/studios/lava$ nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2021 NVIDIA Corporation
Built on Sun_Mar_21_19:15:46_PDT_2021
Cuda compilation tools, release 11.3, V11.3.58
Build cuda_11.3.r11.3/compiler.29745058_0

(lava) wang@speech-ubuntu:~/codes/py/studios/lava-dl$ which nvcc 
/usr/local/cuda/bin/nvcc

(lava) wang@speech-ubuntu:~/codes/py/studios/lava-dl$ nvidia-smi | grep CUDA
| NVIDIA-SMI 465.19.01    Driver Version: 465.19.01    CUDA Version: 11.3 

(lava) wang@speech-ubuntu:~/codes/py/studios/lava-dl$ python
Python 3.9.16 | packaged by conda-forge | (main, Feb  1 2023, 21:39:03) 
[GCC 11.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> torch.cuda.is_available()
True

I also try to check the /usr/local/cuda and /usr/local/cuda/bin/nvcc

(lava) wang@speech-ubuntu:~/codes/py/studios/lava-dl$ ls  /usr/local/cuda
bin  compute-sanitizer  DOCS  EULA.txt  extras  include  lib64  libnvvp  nsightee_plugins  nvml  nvvm  nvvm-prev  README  samples  share  src  targets  tools  version.json
(lava) wang@speech-ubuntu:~/codes/py/studios/lava-dl$ ls /usr/local/cuda/bin/
bin2c              crt       cuda-gdbserver                cu++filt   ncu                          nsight-sys     nsys-ui       nvdisasm      nv-nsight-cu-cli  nvvp
computeprof        cudafe++  cuda-install-samples-11.3.sh  cuobjdump  ncu-ui                       nsys           nvcc          nvlink        nvprof            ptxas
compute-sanitizer  cuda-gdb  cuda-memcheck                 fatbinary  nsight_ee_plugins_manage.sh  nsys-exporter  nvcc.profile  nv-nsight-cu  nvprune

And I can train models on gpu with regular pytorch codes. But I cant do that with lava-dl modules.

It looks like there are some problems with the nvcc lookup

Traceback (most recent call last):
  File "/home/wang/codes/py/XTeam/baseline_solution/sdnn/train.py", line 84, in <module>
    solver.train()
  File "/home/wang/codes/py/XTeam/baseline_solution/sdnn/base/Solver.py", line 103, in train
    train_avg_loss = self._run_one_epoch(epoch, training=True)
  File "/home/wang/codes/py/XTeam/baseline_solution/sdnn/base/Solver.py", line 151, in _run_one_epoch
    batch_loss = self.model(x.cuda(), y.cuda(), training).mean()
  File "/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/nn/parallel/data_parallel.py", line 168, in forward
    outputs = self.parallel_apply(replicas, inputs, kwargs)
  File "/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/nn/parallel/data_parallel.py", line 178, in parallel_apply
    return parallel_apply(replicas, inputs, kwargs, self.device_ids[:len(replicas)])
  File "/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/nn/parallel/parallel_apply.py", line 86, in parallel_apply
    output.reraise()
  File "/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/_utils.py", line 461, in reraise
    raise exception
RuntimeError: Caught RuntimeError in replica 0 on device 0.
Original Traceback (most recent call last):
  File "/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/utils/cpp_extension.py", line 1808, in _run_ninja_build
    subprocess.run(
  File "/home/wang/.conda/envs/lava/lib/python3.9/subprocess.py", line 528, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/nn/parallel/parallel_apply.py", line 61, in _worker
    output = module(*input, **kwargs)
  File "/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/wang/codes/py/XTeam/baseline_solution/sdnn/train.py", line 47, in forward
    denoised_abs = self.denoiser(noisy_abs)
  File "/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/wang/codes/py/XTeam/baseline_solution/sdnn/model.py", line 45, in forward
    x = block(x)
  File "/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/wang/codes/py/XTeam/studios/lava-dl/src/lava/lib/dl/slayer/block/base.py", line 518, in forward
    x = delay(x, 1)
  File "/home/wang/codes/py/XTeam/studios/lava-dl/src/lava/lib/dl/slayer/axon/delay.py", line 35, in delay
    return _delayFunctionNoGradient.apply(input, delay_val, sampling_time)
  File "/home/wang/codes/py/XTeam/studios/lava-dl/src/lava/lib/dl/slayer/axon/delay.py", line 158, in forward
    output = shift(input, delay_val, sampling_time)
  File "/home/wang/codes/py/XTeam/studios/lava-dl/src/lava/lib/dl/slayer/utils/time/shift.py", line 139, in shift
    return Accelerated.shift.shift(
  File "/home/wang/codes/py/XTeam/studios/lava-dl/src/lava/lib/dl/slayer/utils/utils.py", line 14, in __get__
    return staticmethod(self.fget).__get__(None, owner)()
  File "/home/wang/codes/py/XTeam/studios/lava-dl/src/lava/lib/dl/slayer/utils/time/shift.py", line 36, in shift
    Accelerated.module = load(
  File "/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/utils/cpp_extension.py", line 1202, in load
    return _jit_compile(
  File "/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/utils/cpp_extension.py", line 1425, in _jit_compile
    _write_ninja_file_and_build_library(
  File "/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/utils/cpp_extension.py", line 1537, in _write_ninja_file_and_build_library
    _run_ninja_build(
  File "/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/utils/cpp_extension.py", line 1824, in _run_ninja_build
    raise RuntimeError(message) from e
RuntimeError: Error building extension 'shift': [1/2] :/usr/local/cuda:/usr/local/cuda/bin/nvcc  -DTORCH_EXTENSION_NAME=shift -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -isystem /home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include -isystem /home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include -isystem /home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/TH -isystem /home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/THC -isystem :/usr/local/cuda:/usr/local/cuda/include -isystem /home/wang/.conda/envs/lava/include/python3.9 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_61,code=compute_61 -gencode=arch=compute_61,code=sm_61 --compiler-options '-fPIC' -std=c++14 -c /home/wang/codes/py/XTeam/studios/lava-dl/src/lava/lib/dl/slayer/utils/time/shift.cu -o shift.cuda.o 
FAILED: shift.cuda.o 
:/usr/local/cuda:/usr/local/cuda/bin/nvcc  -DTORCH_EXTENSION_NAME=shift -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -isystem /home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include -isystem /home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include -isystem /home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/TH -isystem /home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/THC -isystem :/usr/local/cuda:/usr/local/cuda/include -isystem /home/wang/.conda/envs/lava/include/python3.9 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_61,code=compute_61 -gencode=arch=compute_61,code=sm_61 --compiler-options '-fPIC' -std=c++14 -c /home/wang/codes/py/XTeam/studios/lava-dl/src/lava/lib/dl/slayer/utils/time/shift.cu -o shift.cuda.o 
/bin/sh: 1: :/usr/local/cuda:/usr/local/cuda/bin/nvcc: not found
ninja: build stopped: subcommand failed.

@wangtianrui
Copy link
Author

wangtianrui commented Apr 4, 2023 •
edited
Loading

In addition. My codes were running in conda's virtual environment without "source .venv/bin/activate". I wonder if there will be any unexpected problems with the conda env?

@wangtianrui
Copy link
Author

wangtianrui commented Apr 4, 2023 •
edited
Loading

update:

I reinstall the repo, and the errors still exist:

export CUDA_HOME=/usr/local/cuda-11.3
export PYTHONPATH=/home/wang/codes/py/XTeam/studios/lava/src
conda activate lava

(lava) wang@speech-ubuntu:~/codes/py/XTeam/studios/lava-dl$ nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2021 NVIDIA Corporation
Built on Sun_Mar_21_19:15:46_PDT_2021
Cuda compilation tools, release 11.3, V11.3.58
Build cuda_11.3.r11.3/compiler.29745058_0
(lava) wang@speech-ubuntu:~/codes/py/XTeam/studios/lava-dl$ echo $PYTHONPATH
/home/wang/codes/py/XTeam/studios/lava/src
(lava) wang@speech-ubuntu:~/codes/py/XTeam/studios/lava-dl$ nvidia-smi | grep CUDA
| NVIDIA-SMI 465.19.01    Driver Version: 465.19.01    CUDA Version: 11.3     |
(lava) wang@speech-ubuntu:~/codes/py/XTeam/studios/lava-dl$ python
Python 3.9.16 | packaged by conda-forge | (main, Feb  1 2023, 21:39:03) 
[GCC 11.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> torch.cuda.is_available()
True
FAIL Required test coverage of 45.0% not reached. Total coverage: 32.69%
==================================================== short test summary info ====================================================
ERROR tests/lava/lib/dl/slayer/neuron/test_adrf.py - RuntimeError: Error building extension 'dynamics': [1/2] /usr/local/cuda-11.3/bin/nvcc  -DTORCH_EXTENSION_NAME=dynamics -DTO...
ERROR tests/lava/lib/dl/slayer/neuron/test_adrf_iz.py - RuntimeError: Error building extension 'dynamics_v1': [1/2] /usr/local/cuda-11.3/bin/nvcc  -DTORCH_EXTENSION_NAME=dynamics_v...
ERROR tests/lava/lib/dl/slayer/neuron/test_alif.py - RuntimeError: Error building extension 'dynamics_v2': [1/2] /usr/local/cuda-11.3/bin/nvcc  -DTORCH_EXTENSION_NAME=dynamics_v...
ERROR tests/lava/lib/dl/slayer/neuron/test_cuba.py
ERROR tests/lava/lib/dl/slayer/neuron/test_rf.py - RuntimeError: Error building extension 'dynamics_v3': [1/2] /usr/local/cuda-11.3/bin/nvcc  -DTORCH_EXTENSION_NAME=dynamics_v...
ERROR tests/lava/lib/dl/slayer/neuron/test_rf_iz.py
ERROR tests/lava/lib/dl/slayer/neuron/test_sigma_delta.py
ERROR tests/lava/lib/dl/slayer/neuron/dynamics/test_adaptive_phase_th.py
ERROR tests/lava/lib/dl/slayer/neuron/dynamics/test_adaptive_resonator.py - RuntimeError: Error building extension 'dynamics_v4': [1/2] /usr/local/cuda-11.3/bin/nvcc  -DTORCH_EXTENSION_NAME=dynamics_v...
ERROR tests/lava/lib/dl/slayer/neuron/dynamics/test_adaptive_threshold.py - RuntimeError: Error building extension 'dynamics_v5': [1/2] /usr/local/cuda-11.3/bin/nvcc  -DTORCH_EXTENSION_NAME=dynamics_v...
ERROR tests/lava/lib/dl/slayer/neuron/dynamics/test_leaky_integrator.py
ERROR tests/lava/lib/dl/slayer/neuron/dynamics/test_resonator.py - RuntimeError: Error building extension 'dynamics_v6': [1/2] /usr/local/cuda-11.3/bin/nvcc  -DTORCH_EXTENSION_NAME=dynamics_v...
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 12 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
========================================== 12 warnings, 12 errors in 221.87s (0:03:41) ==========================================

@bamsumit
Copy link
Contributor

bamsumit commented Apr 4, 2023

Can you tell me the output of these commands.

which nvcc
ls /usr/local/cuda/bin/nvcc
ls /usr/local/cuda

@wangtianrui
Copy link
Author

Thank you for your reply. Here are the results of these commands:

(lava) wang@speech-ubuntu:~/codes/py/XTeam$ which nvcc
/usr/local/cuda/bin/nvcc
al/cuda(lava) wang@speech-ubuntu:~/codes/py/XTeam$ ls /usr/local/cuda/bin/nvcc
/usr/local/cuda/bin/nvcc
(lava) wang@speech-ubuntu:~/codes/py/XTeam$ ls /usr/local/cuda
bin                DOCS      extras   lib64    nsightee_plugins  nvvm       README   share  targets  version.json
compute-sanitizer  EULA.txt  include  libnvvp  nvml              nvvm-prev  samples  src    tools

@bamsumit
Copy link
Contributor

bamsumit commented Apr 4, 2023

Can you try this solution: pytorch/extension-cpp#51 (comment)

@wangtianrui
Copy link
Author

wangtianrui commented Apr 5, 2023 •
edited
Loading

I tried your solution, but it also dont work for me.

export CUDA_HOME=/usr/local/cuda-11.3
export PYTHONPATH=/home/wang/codes/py/XTeam/studios/lava/src
export PATH="$PATH:/usr/local/cuda-11.3/bin"
export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/local/cuda-11.3/lib64"

The error was changed but it's still related to "shift.cuda"

Traceback (most recent call last):
  File "/home/wang/codes/py/XTeam/baseline_solution/sdnn/train.py", line 84, in <module>
    solver.train()
  File "/home/wang/codes/py/XTeam/baseline_solution/sdnn/base/Solver.py", line 103, in train
    train_avg_loss = self._run_one_epoch(epoch, training=True)
  File "/home/wang/codes/py/XTeam/baseline_solution/sdnn/base/Solver.py", line 151, in _run_one_epoch
    batch_loss = self.model(x.cuda(), y.cuda(), training).mean()
  File "/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/nn/parallel/data_parallel.py", line 168, in forward
    outputs = self.parallel_apply(replicas, inputs, kwargs)
  File "/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/nn/parallel/data_parallel.py", line 178, in parallel_apply
    return parallel_apply(replicas, inputs, kwargs, self.device_ids[:len(replicas)])
  File "/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/nn/parallel/parallel_apply.py", line 86, in parallel_apply
    output.reraise()
  File "/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/_utils.py", line 461, in reraise
    raise exception
RuntimeError: Caught RuntimeError in replica 0 on device 0.
Original Traceback (most recent call last):
  File "/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/utils/cpp_extension.py", line 1808, in _run_ninja_build
    subprocess.run(
  File "/home/wang/.conda/envs/lava/lib/python3.9/subprocess.py", line 528, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/nn/parallel/parallel_apply.py", line 61, in _worker
    output = module(*input, **kwargs)
  File "/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/wang/codes/py/XTeam/baseline_solution/sdnn/train.py", line 47, in forward
    denoised_abs = self.denoiser(noisy_abs)
  File "/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/wang/codes/py/XTeam/baseline_solution/sdnn/model.py", line 45, in forward
    x = block(x)
  File "/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/wang/codes/py/XTeam/baseline_solution/sdnn/lava/lib/dl/slayer/block/base.py", line 518, in forward
    x = delay(x, 1)
  File "/home/wang/codes/py/XTeam/baseline_solution/sdnn/lava/lib/dl/slayer/axon/delay.py", line 35, in delay
    return _delayFunctionNoGradient.apply(input, delay_val, sampling_time)
  File "/home/wang/codes/py/XTeam/baseline_solution/sdnn/lava/lib/dl/slayer/axon/delay.py", line 158, in forward
    output = shift(input, delay_val, sampling_time)
  File "/home/wang/codes/py/XTeam/baseline_solution/sdnn/lava/lib/dl/slayer/utils/time/shift.py", line 139, in shift
    return Accelerated.shift.shift(
  File "/home/wang/codes/py/XTeam/baseline_solution/sdnn/lava/lib/dl/slayer/utils/utils.py", line 14, in __get__
    return staticmethod(self.fget).__get__(None, owner)()
  File "/home/wang/codes/py/XTeam/baseline_solution/sdnn/lava/lib/dl/slayer/utils/time/shift.py", line 36, in shift
    Accelerated.module = load(
  File "/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/utils/cpp_extension.py", line 1202, in load
    return _jit_compile(
  File "/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/utils/cpp_extension.py", line 1425, in _jit_compile
    _write_ninja_file_and_build_library(
  File "/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/utils/cpp_extension.py", line 1537, in _write_ninja_file_and_build_library
    _run_ninja_build(
  File "/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/utils/cpp_extension.py", line 1824, in _run_ninja_build
    raise RuntimeError(message) from e
RuntimeError: Error building extension 'shift': [1/2] /usr/local/cuda-11.3/bin/nvcc  -DTORCH_EXTENSION_NAME=shift -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -isystem /home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include -isystem /home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include -isystem /home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/TH -isystem /home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/THC -isystem /usr/local/cuda-11.3/include -isystem /home/wang/.conda/envs/lava/include/python3.9 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_61,code=compute_61 -gencode=arch=compute_61,code=sm_61 --compiler-options '-fPIC' -std=c++14 -c /home/wang/codes/py/XTeam/baseline_solution/sdnn/lava/lib/dl/slayer/utils/time/shift.cu -o shift.cuda.o 
FAILED: shift.cuda.o 
/usr/local/cuda-11.3/bin/nvcc  -DTORCH_EXTENSION_NAME=shift -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -isystem /home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include -isystem /home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include -isystem /home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/TH -isystem /home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/THC -isystem /usr/local/cuda-11.3/include -isystem /home/wang/.conda/envs/lava/include/python3.9 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_61,code=compute_61 -gencode=arch=compute_61,code=sm_61 --compiler-options '-fPIC' -std=c++14 -c /home/wang/codes/py/XTeam/baseline_solution/sdnn/lava/lib/dl/slayer/utils/time/shift.cu -o shift.cuda.o 
/home/wang/codes/py/XTeam/baseline_solution/sdnn/lava/lib/dl/slayer/utils/time/shift.cu: In function ‘torch::autograd::Variable shiftGlobal(torch::autograd::Variable, float, float)’:
/home/wang/codes/py/XTeam/baseline_solution/sdnn/lava/lib/dl/slayer/utils/time/shift.cu:169:39: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations]
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/ATen/core/TensorBody.h:235:1: note: declared here
   T * data() const {
 ^
/home/wang/codes/py/XTeam/baseline_solution/sdnn/lava/lib/dl/slayer/utils/time/shift.cu:169:62: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations]
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/ATen/core/TensorBody.h:235:1: note: declared here
   T * data() const {
 ^
/home/wang/codes/py/XTeam/baseline_solution/sdnn/lava/lib/dl/slayer/utils/time/shift.cu: In function ‘torch::autograd::Variable shift(torch::autograd::Variable, torch::autograd::Variable, float)’:
/home/wang/codes/py/XTeam/baseline_solution/sdnn/lava/lib/dl/slayer/utils/time/shift.cu:194:39: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations]
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/ATen/core/TensorBody.h:235:1: note: declared here
   T * data() const {
 ^
/home/wang/codes/py/XTeam/baseline_solution/sdnn/lava/lib/dl/slayer/utils/time/shift.cu:194:62: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations]
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/ATen/core/TensorBody.h:235:1: note: declared here
   T * data() const {
 ^
/home/wang/codes/py/XTeam/baseline_solution/sdnn/lava/lib/dl/slayer/utils/time/shift.cu:202:39: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations]
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/ATen/core/TensorBody.h:235:1: note: declared here
   T * data() const {
 ^
/home/wang/codes/py/XTeam/baseline_solution/sdnn/lava/lib/dl/slayer/utils/time/shift.cu:202:62: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations]
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/ATen/core/TensorBody.h:235:1: note: declared here
   T * data() const {
 ^
/home/wang/codes/py/XTeam/baseline_solution/sdnn/lava/lib/dl/slayer/utils/time/shift.cu:202:89: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations]
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/ATen/core/TensorBody.h:235:1: note: declared here
   T * data() const {
 ^
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h: In instantiation of ‘std::shared_ptr<torch::nn::Module> torch::nn::Cloneable<Derived>::clone(const c10::optional<c10::Device>&) const [with Derived = torch::nn::CrossMapLRN2dImpl]’:
/tmp/tmpxft_0000765a_00000000-6_shift.cudafe1.stub.c:21:27:   required from here
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:57:59: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, at::Tensor>’ to type ‘torch::OrderedDict<std::basic_string<char>, at::Tensor>&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:69:61: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >’ to type ‘torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h: In instantiation of ‘std::shared_ptr<torch::nn::Module> torch::nn::Cloneable<Derived>::clone(const c10::optional<c10::Device>&) const [with Derived = torch::nn::EmbeddingBagImpl]’:
/tmp/tmpxft_0000765a_00000000-6_shift.cudafe1.stub.c:21:27:   required from here
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:57:59: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, at::Tensor>’ to type ‘torch::OrderedDict<std::basic_string<char>, at::Tensor>&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:69:61: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >’ to type ‘torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h: In instantiation of ‘std::shared_ptr<torch::nn::Module> torch::nn::Cloneable<Derived>::clone(const c10::optional<c10::Device>&) const [with Derived = torch::nn::EmbeddingImpl]’:
/tmp/tmpxft_0000765a_00000000-6_shift.cudafe1.stub.c:21:27:   required from here
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:57:59: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, at::Tensor>’ to type ‘torch::OrderedDict<std::basic_string<char>, at::Tensor>&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:69:61: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >’ to type ‘torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h: In instantiation of ‘std::shared_ptr<torch::nn::Module> torch::nn::Cloneable<Derived>::clone(const c10::optional<c10::Device>&) const [with Derived = torch::nn::ParameterDictImpl]’:
/tmp/tmpxft_0000765a_00000000-6_shift.cudafe1.stub.c:21:27:   required from here
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:57:59: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, at::Tensor>’ to type ‘torch::OrderedDict<std::basic_string<char>, at::Tensor>&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:69:61: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >’ to type ‘torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h: In instantiation of ‘std::shared_ptr<torch::nn::Module> torch::nn::Cloneable<Derived>::clone(const c10::optional<c10::Device>&) const [with Derived = torch::nn::SequentialImpl]’:
/tmp/tmpxft_0000765a_00000000-6_shift.cudafe1.stub.c:21:27:   required from here
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:57:59: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, at::Tensor>’ to type ‘torch::OrderedDict<std::basic_string<char>, at::Tensor>&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:69:61: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >’ to type ‘torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h: In instantiation of ‘std::shared_ptr<torch::nn::Module> torch::nn::Cloneable<Derived>::clone(const c10::optional<c10::Device>&) const [with Derived = torch::nn::ModuleListImpl]’:
/tmp/tmpxft_0000765a_00000000-6_shift.cudafe1.stub.c:21:27:   required from here
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:57:59: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, at::Tensor>’ to type ‘torch::OrderedDict<std::basic_string<char>, at::Tensor>&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:69:61: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >’ to type ‘torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h: In instantiation of ‘std::shared_ptr<torch::nn::Module> torch::nn::Cloneable<Derived>::clone(const c10::optional<c10::Device>&) const [with Derived = torch::nn::ModuleDictImpl]’:
/tmp/tmpxft_0000765a_00000000-6_shift.cudafe1.stub.c:21:27:   required from here
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:57:59: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, at::Tensor>’ to type ‘torch::OrderedDict<std::basic_string<char>, at::Tensor>&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:69:61: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >’ to type ‘torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h: In instantiation of ‘std::shared_ptr<torch::nn::Module> torch::nn::Cloneable<Derived>::clone(const c10::optional<c10::Device>&) const [with Derived = torch::nn::TransformerDecoderImpl]’:
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/optim/sgd.h:49:48:   required from here
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:57:59: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, at::Tensor>’ to type ‘torch::OrderedDict<std::basic_string<char>, at::Tensor>&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:69:61: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >’ to type ‘torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h: In instantiation of ‘std::shared_ptr<torch::nn::Module> torch::nn::Cloneable<Derived>::clone(const c10::optional<c10::Device>&) const [with Derived = torch::nn::TransformerEncoderImpl]’:
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/optim/sgd.h:49:48:   required from here
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:57:59: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, at::Tensor>’ to type ‘torch::OrderedDict<std::basic_string<char>, at::Tensor>&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:69:61: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >’ to type ‘torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h: In instantiation of ‘std::shared_ptr<torch::nn::Module> torch::nn::Cloneable<Derived>::clone(const c10::optional<c10::Device>&) const [with Derived = torch::nn::TransformerDecoderLayerImpl]’:
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/optim/sgd.h:49:48:   required from here
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:57:59: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, at::Tensor>’ to type ‘torch::OrderedDict<std::basic_string<char>, at::Tensor>&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:69:61: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >’ to type ‘torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h: In instantiation of ‘std::shared_ptr<torch::nn::Module> torch::nn::Cloneable<Derived>::clone(const c10::optional<c10::Device>&) const [with Derived = torch::nn::TransformerEncoderLayerImpl]’:
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/optim/sgd.h:49:48:   required from here
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:57:59: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, at::Tensor>’ to type ‘torch::OrderedDict<std::basic_string<char>, at::Tensor>&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:69:61: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >’ to type ‘torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h: In instantiation of ‘std::shared_ptr<torch::nn::Module> torch::nn::Cloneable<Derived>::clone(const c10::optional<c10::Device>&) const [with Derived = torch::nn::GroupNormImpl]’:
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/optim/sgd.h:49:48:   required from here
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:57:59: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, at::Tensor>’ to type ‘torch::OrderedDict<std::basic_string<char>, at::Tensor>&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:69:61: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >’ to type ‘torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h: In instantiation of ‘std::shared_ptr<torch::nn::Module> torch::nn::Cloneable<Derived>::clone(const c10::optional<c10::Device>&) const [with Derived = torch::nn::LocalResponseNormImpl]’:
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/optim/sgd.h:49:48:   required from here
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:57:59: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, at::Tensor>’ to type ‘torch::OrderedDict<std::basic_string<char>, at::Tensor>&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:69:61: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >’ to type ‘torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h: In instantiation of ‘std::shared_ptr<torch::nn::Module> torch::nn::Cloneable<Derived>::clone(const c10::optional<c10::Device>&) const [with Derived = torch::nn::LayerNormImpl]’:
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/optim/sgd.h:49:48:   required from here
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:57:59: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, at::Tensor>’ to type ‘torch::OrderedDict<std::basic_string<char>, at::Tensor>&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:69:61: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >’ to type ‘torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h: In instantiation of ‘std::shared_ptr<torch::nn::Module> torch::nn::Cloneable<Derived>::clone(const c10::optional<c10::Device>&) const [with Derived = torch::nn::MultiheadAttentionImpl]’:
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/optim/sgd.h:49:48:   required from here
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:57:59: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, at::Tensor>’ to type ‘torch::OrderedDict<std::basic_string<char>, at::Tensor>&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:69:61: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >’ to type ‘torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h: In instantiation of ‘std::shared_ptr<torch::nn::Module> torch::nn::Cloneable<Derived>::clone(const c10::optional<c10::Device>&) const [with Derived = torch::nn::ThresholdImpl]’:
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/optim/sgd.h:49:48:   required from here
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:57:59: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, at::Tensor>’ to type ‘torch::OrderedDict<std::basic_string<char>, at::Tensor>&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:69:61: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >’ to type ‘torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h: In instantiation of ‘std::shared_ptr<torch::nn::Module> torch::nn::Cloneable<Derived>::clone(const c10::optional<c10::Device>&) const [with Derived = torch::nn::LogSoftmaxImpl]’:
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/optim/sgd.h:49:48:   required from here
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:57:59: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, at::Tensor>’ to type ‘torch::OrderedDict<std::basic_string<char>, at::Tensor>&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:69:61: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >’ to type ‘torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h: In instantiation of ‘std::shared_ptr<torch::nn::Module> torch::nn::Cloneable<Derived>::clone(const c10::optional<c10::Device>&) const [with Derived = torch::nn::SoftminImpl]’:
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/optim/sgd.h:49:48:   required from here
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:57:59: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, at::Tensor>’ to type ‘torch::OrderedDict<std::basic_string<char>, at::Tensor>&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:69:61: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >’ to type ‘torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h: In instantiation of ‘std::shared_ptr<torch::nn::Module> torch::nn::Cloneable<Derived>::clone(const c10::optional<c10::Device>&) const [with Derived = torch::nn::SoftmaxImpl]’:
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/optim/sgd.h:49:48:   required from here
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:57:59: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, at::Tensor>’ to type ‘torch::OrderedDict<std::basic_string<char>, at::Tensor>&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:69:61: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >’ to type ‘torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h: In instantiation of ‘std::shared_ptr<torch::nn::Module> torch::nn::Cloneable<Derived>::clone(const c10::optional<c10::Device>&) const [with Derived = torch::nn::GRUCellImpl]’:
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/optim/sgd.h:49:48:   required from here
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:57:59: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, at::Tensor>’ to type ‘torch::OrderedDict<std::basic_string<char>, at::Tensor>&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:69:61: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >’ to type ‘torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h: In instantiation of ‘std::shared_ptr<torch::nn::Module> torch::nn::Cloneable<Derived>::clone(const c10::optional<c10::Device>&) const [with Derived = torch::nn::LSTMCellImpl]’:
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/optim/sgd.h:49:48:   required from here
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:57:59: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, at::Tensor>’ to type ‘torch::OrderedDict<std::basic_string<char>, at::Tensor>&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:69:61: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >’ to type ‘torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h: In instantiation of ‘std::shared_ptr<torch::nn::Module> torch::nn::Cloneable<Derived>::clone(const c10::optional<c10::Device>&) const [with Derived = torch::nn::RNNCellImpl]’:
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/optim/sgd.h:49:48:   required from here
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:57:59: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, at::Tensor>’ to type ‘torch::OrderedDict<std::basic_string<char>, at::Tensor>&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:69:61: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >’ to type ‘torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h: In instantiation of ‘std::shared_ptr<torch::nn::Module> torch::nn::Cloneable<Derived>::clone(const c10::optional<c10::Device>&) const [with Derived = torch::nn::GRUImpl]’:
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/optim/sgd.h:49:48:   required from here
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:57:59: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, at::Tensor>’ to type ‘torch::OrderedDict<std::basic_string<char>, at::Tensor>&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:69:61: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >’ to type ‘torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h: In instantiation of ‘std::shared_ptr<torch::nn::Module> torch::nn::Cloneable<Derived>::clone(const c10::optional<c10::Device>&) const [with Derived = torch::nn::LSTMImpl]’:
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/optim/sgd.h:49:48:   required from here
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:57:59: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, at::Tensor>’ to type ‘torch::OrderedDict<std::basic_string<char>, at::Tensor>&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:69:61: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >’ to type ‘torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h: In instantiation of ‘std::shared_ptr<torch::nn::Module> torch::nn::Cloneable<Derived>::clone(const c10::optional<c10::Device>&) const [with Derived = torch::nn::RNNImpl]’:
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/optim/sgd.h:49:48:   required from here
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:57:59: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, at::Tensor>’ to type ‘torch::OrderedDict<std::basic_string<char>, at::Tensor>&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:69:61: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >’ to type ‘torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h: In instantiation of ‘std::shared_ptr<torch::nn::Module> torch::nn::Cloneable<Derived>::clone(const c10::optional<c10::Device>&) const [with Derived = torch::nn::FractionalMaxPool3dImpl]’:
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/optim/sgd.h:49:48:   required from here
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:57:59: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, at::Tensor>’ to type ‘torch::OrderedDict<std::basic_string<char>, at::Tensor>&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:69:61: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >’ to type ‘torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h: In instantiation of ‘std::shared_ptr<torch::nn::Module> torch::nn::Cloneable<Derived>::clone(const c10::optional<c10::Device>&) const [with Derived = torch::nn::FractionalMaxPool2dImpl]’:
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/optim/sgd.h:49:48:   required from here
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:57:59: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, at::Tensor>’ to type ‘torch::OrderedDict<std::basic_string<char>, at::Tensor>&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:69:61: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >’ to type ‘torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h: In instantiation of ‘std::shared_ptr<torch::nn::Module> torch::nn::Cloneable<Derived>::clone(const c10::optional<c10::Device>&) const [with Derived = torch::nn::ZeroPad2dImpl]’:
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/optim/sgd.h:49:48:   required from here
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:57:59: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, at::Tensor>’ to type ‘torch::OrderedDict<std::basic_string<char>, at::Tensor>&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:69:61: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >’ to type ‘torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h: In instantiation of ‘std::shared_ptr<torch::nn::Module> torch::nn::Cloneable<Derived>::clone(const c10::optional<c10::Device>&) const [with Derived = torch::nn::UnfoldImpl]’:
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/optim/sgd.h:49:48:   required from here
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:57:59: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, at::Tensor>’ to type ‘torch::OrderedDict<std::basic_string<char>, at::Tensor>&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:69:61: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >’ to type ‘torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h: In instantiation of ‘std::shared_ptr<torch::nn::Module> torch::nn::Cloneable<Derived>::clone(const c10::optional<c10::Device>&) const [with Derived = torch::nn::FoldImpl]’:
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/optim/sgd.h:49:48:   required from here
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:57:59: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, at::Tensor>’ to type ‘torch::OrderedDict<std::basic_string<char>, at::Tensor>&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:69:61: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >’ to type ‘torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h: In instantiation of ‘std::shared_ptr<torch::nn::Module> torch::nn::Cloneable<Derived>::clone(const c10::optional<c10::Device>&) const [with Derived = torch::nn::ConvTranspose3dImpl]’:
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/optim/sgd.h:49:48:   required from here
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:57:59: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, at::Tensor>’ to type ‘torch::OrderedDict<std::basic_string<char>, at::Tensor>&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:69:61: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >’ to type ‘torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h: In instantiation of ‘std::shared_ptr<torch::nn::Module> torch::nn::Cloneable<Derived>::clone(const c10::optional<c10::Device>&) const [with Derived = torch::nn::ConvTranspose2dImpl]’:
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/optim/sgd.h:49:48:   required from here
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:57:59: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, at::Tensor>’ to type ‘torch::OrderedDict<std::basic_string<char>, at::Tensor>&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:69:61: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >’ to type ‘torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h: In instantiation of ‘std::shared_ptr<torch::nn::Module> torch::nn::Cloneable<Derived>::clone(const c10::optional<c10::Device>&) const [with Derived = torch::nn::ConvTranspose1dImpl]’:
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/optim/sgd.h:49:48:   required from here
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:57:59: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, at::Tensor>’ to type ‘torch::OrderedDict<std::basic_string<char>, at::Tensor>&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:69:61: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >’ to type ‘torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h: In instantiation of ‘std::shared_ptr<torch::nn::Module> torch::nn::Cloneable<Derived>::clone(const c10::optional<c10::Device>&) const [with Derived = torch::nn::Conv3dImpl]’:
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/optim/sgd.h:49:48:   required from here
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:57:59: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, at::Tensor>’ to type ‘torch::OrderedDict<std::basic_string<char>, at::Tensor>&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:69:61: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >’ to type ‘torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h: In instantiation of ‘std::shared_ptr<torch::nn::Module> torch::nn::Cloneable<Derived>::clone(const c10::optional<c10::Device>&) const [with Derived = torch::nn::Conv2dImpl]’:
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/optim/sgd.h:49:48:   required from here
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:57:59: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, at::Tensor>’ to type ‘torch::OrderedDict<std::basic_string<char>, at::Tensor>&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:69:61: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >’ to type ‘torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h: In instantiation of ‘std::shared_ptr<torch::nn::Module> torch::nn::Cloneable<Derived>::clone(const c10::optional<c10::Device>&) const [with Derived = torch::nn::Conv1dImpl]’:
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/optim/sgd.h:49:48:   required from here
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:57:59: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, at::Tensor>’ to type ‘torch::OrderedDict<std::basic_string<char>, at::Tensor>&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:69:61: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >’ to type ‘torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h: In instantiation of ‘std::shared_ptr<torch::nn::Module> torch::nn::Cloneable<Derived>::clone(const c10::optional<c10::Device>&) const [with Derived = torch::nn::AdaptiveLogSoftmaxWithLossImpl]’:
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/optim/sgd.h:49:48:   required from here
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:57:59: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, at::Tensor>’ to type ‘torch::OrderedDict<std::basic_string<char>, at::Tensor>&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:69:61: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >’ to type ‘torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h: In instantiation of ‘std::shared_ptr<torch::nn::Module> torch::nn::Cloneable<Derived>::clone(const c10::optional<c10::Device>&) const [with Derived = torch::nn::BilinearImpl]’:
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/optim/sgd.h:49:48:   required from here
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:57:59: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, at::Tensor>’ to type ‘torch::OrderedDict<std::basic_string<char>, at::Tensor>&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:69:61: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >’ to type ‘torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h: In instantiation of ‘std::shared_ptr<torch::nn::Module> torch::nn::Cloneable<Derived>::clone(const c10::optional<c10::Device>&) const [with Derived = torch::nn::UnflattenImpl]’:
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/optim/sgd.h:49:48:   required from here
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:57:59: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, at::Tensor>’ to type ‘torch::OrderedDict<std::basic_string<char>, at::Tensor>&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:69:61: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >’ to type ‘torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h: In instantiation of ‘std::shared_ptr<torch::nn::Module> torch::nn::Cloneable<Derived>::clone(const c10::optional<c10::Device>&) const [with Derived = torch::nn::LinearImpl]’:
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/optim/sgd.h:49:48:   required from here
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:57:59: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, at::Tensor>’ to type ‘torch::OrderedDict<std::basic_string<char>, at::Tensor>&’
/home/wang/.conda/envs/lava/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:69:61: error: invalid static_cast from type ‘const torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >’ to type ‘torch::OrderedDict<std::basic_string<char>, std::shared_ptr<torch::nn::Module> >&’
ninja: build stopped: subcommand failed.

@bamsumit
Copy link
Contributor

bamsumit commented Apr 5, 2023

Okay that's progress. Can you uninstall torch, torchvision and torchaudio and reinstall it again targeting cuda 11.3 (I suspect it is installed for cuda 11.7, which is the default now)

  • pip install torch==1.12.1+cu113 torchvision==0.13.1+cu113 torchaudio==0.12.1 --extra-index-url https://download.pytorch.org/whl/cu113 OR
  • conda install pytorch==1.12.1 torchvision==0.13.1 torchaudio==0.12.1 cudatoolkit=11.3 -c pytorch

@wangtianrui
Copy link
Author

No. My torch, torchvision and torchaudio were targeting cuda 11.3

(lava) wang@speech-ubuntu:~/codes/py/XTeam$ conda list | grep torch
ffmpeg                    4.3                  hf484d3e_0    pytorch
pytorch                   1.12.1          py3.9_cuda11.3_cudnn8.3.2_0    pytorch
pytorch-mutex             1.0                        cuda    pytorch
torch                     1.12.1                   pypi_0    pypi
torchaudio                0.12.1               py39_cu113    pytorch
torchvision               0.13.1                   pypi_0    pypi

The normal pytorch models codes can be trained correctly, but the model with lava-dl cannot. The cause of the problem seems to be "lava/lib/dl/slayer/utils/time/shift.py".

@wangtianrui
Copy link
Author

Hello, I have solved the problem of training. I just put the src/lava/ folder directly under the project directory without installing the lava-dl. In this way, it can be trained without errors. I presume that some environment bindings are incompatible when installing lava.

Thank you for your reply. Wish you a happy life. @bamsumit

@bamsumit
Copy link
Contributor

bamsumit commented Apr 6, 2023

Great. Thanks for letting us know.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@bamsumit bamsumit

Labels
0-needs-review 1-bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@wangtianrui @bamsumit