Compile error with ctc_loss_layer

[mattroos]

I'm getting the errors below when compiling the requisite version of Caffe. Everything else seems fine prior to the ctc_loss_layer. I recently upgraded to Ubuntu 16.04, CUDA 9.0, and NVIDIA driver 384. Perhaps related to CUDA 9.0?

NVCC src/caffe/layers/ctc_loss_layer.cu
./include/caffe/util/ctc/detail/../contrib/moderngpu/include/device/intrinsics.cuh(115): warning: function "__shfl_up(float, unsigned int, int)"
/usr/local/cuda/include/sm_30_intrinsics.hpp(261): here was declared deprecated ("__shfl_up() is deprecated in favor of __shfl_up_sync() and may be removed in a future release (Use -Wno-deprecated-declarations to suppress this warning).")

./include/caffe/util/ctc/detail/../contrib/moderngpu/include/device/intrinsics.cuh(125): warning: function "__shfl_up(int, unsigned int, int)"
/usr/local/cuda/include/sm_30_intrinsics.hpp(175): here was declared deprecated ("__shfl_up() is deprecated in favor of __shfl_up_sync() and may be removed in a future release (Use -Wno-deprecated-declarations to suppress this warning).")

./include/caffe/util/ctc/detail/../contrib/moderngpu/include/device/intrinsics.cuh(126): warning: function "__shfl_up(int, unsigned int, int)"
/usr/local/cuda/include/sm_30_intrinsics.hpp(175): here was declared deprecated ("__shfl_up() is deprecated in favor of __shfl_up_sync() and may be removed in a future release (Use -Wno-deprecated-declarations to suppress this warning).")

./include/caffe/util/ctc/detail/gpu_ctc.h(18): warning: deleted functions are a C++11 feature

src/caffe/layers/ctc_loss_layer.cu(19): error: identifier "nullptr" is undefined

src/caffe/layers/ctc_loss_layer.cu(80): error: identifier "nullptr" is undefined
          detected during:
            instantiation of "void caffe::gpu_ctc(Dtype *, Dtype *, int *, int *, int *, int, int, Dtype *, int) [with Dtype=float]" 
(181): here
            instantiation of "void caffe::CTCLossLayer<Dtype>::Forward_gpu(const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &, const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &) [with Dtype=float]" 
(226): here

./include/caffe/util/ctc/detail/gpu_ctc.h(448): error: identifier "nullptr" is undefined
          detected during:
            instantiation of "ctcStatus_t caffe::compute_ctc_loss(const Dtype *, Dtype *, const int *, const int *, const int *, int, int, Dtype *, void *, ctcComputeInfo) [with Dtype=float]" 
src/caffe/layers/ctc_loss_layer.cu(136): here
            instantiation of "void caffe::gpu_ctc(Dtype *, Dtype *, int *, int *, int *, int, int, Dtype *, int) [with Dtype=float]" 
src/caffe/layers/ctc_loss_layer.cu(181): here
            instantiation of "void caffe::CTCLossLayer<Dtype>::Forward_gpu(const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &, const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &) [with Dtype=float]" 
src/caffe/layers/ctc_loss_layer.cu(226): here

./include/caffe/util/ctc/detail/gpu_ctc.h(305): error: identifier "constexpr" is undefined
          detected during:
            instantiation of "ctcStatus_t GpuCTC<ProbT>::compute_cost_and_score(const ProbT *, ProbT *, ProbT *, const int *, const int *, const int *, __nv_bool, __nv_bool) [with ProbT=float]" 
(457): here
            instantiation of "ctcStatus_t GpuCTC<ProbT>::cost_and_grad(const ProbT *, ProbT *, ProbT *, const int *, const int *, const int *) [with ProbT=float]" 
src/caffe/layers/ctc_loss_layer.cu(94): here
            instantiation of "ctcStatus_t caffe::compute_ctc_loss(const Dtype *, Dtype *, const int *, const int *, const int *, int, int, Dtype *, void *, ctcComputeInfo) [with Dtype=float]" 
src/caffe/layers/ctc_loss_layer.cu(136): here
            instantiation of "void caffe::gpu_ctc(Dtype *, Dtype *, int *, int *, int *, int, int, Dtype *, int) [with Dtype=float]" 
src/caffe/layers/ctc_loss_layer.cu(181): here
            instantiation of "void caffe::CTCLossLayer<Dtype>::Forward_gpu(const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &, const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &) [with Dtype=float]" 
src/caffe/layers/ctc_loss_layer.cu(226): here

./include/caffe/util/ctc/detail/gpu_ctc.h(305): error: expected a ";"
          detected during:
            instantiation of "ctcStatus_t GpuCTC<ProbT>::compute_cost_and_score(const ProbT *, ProbT *, ProbT *, const int *, const int *, const int *, __nv_bool, __nv_bool) [with ProbT=float]" 
(457): here
            instantiation of "ctcStatus_t GpuCTC<ProbT>::cost_and_grad(const ProbT *, ProbT *, ProbT *, const int *, const int *, const int *) [with ProbT=float]" 
src/caffe/layers/ctc_loss_layer.cu(94): here
            instantiation of "ctcStatus_t caffe::compute_ctc_loss(const Dtype *, Dtype *, const int *, const int *, const int *, int, int, Dtype *, void *, ctcComputeInfo) [with Dtype=float]" 
src/caffe/layers/ctc_loss_layer.cu(136): here
            instantiation of "void caffe::gpu_ctc(Dtype *, Dtype *, int *, int *, int *, int, int, Dtype *, int) [with Dtype=float]" 
src/caffe/layers/ctc_loss_layer.cu(181): here
            instantiation of "void caffe::CTCLossLayer<Dtype>::Forward_gpu(const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &, const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &) [with Dtype=float]" 
src/caffe/layers/ctc_loss_layer.cu(226): here

./include/caffe/util/ctc/detail/gpu_ctc.h(307): error: identifier "num_configs" is undefined
          detected during:
            instantiation of "ctcStatus_t GpuCTC<ProbT>::compute_cost_and_score(const ProbT *, ProbT *, ProbT *, const int *, const int *, const int *, __nv_bool, __nv_bool) [with ProbT=float]" 
(457): here
            instantiation of "ctcStatus_t GpuCTC<ProbT>::cost_and_grad(const ProbT *, ProbT *, ProbT *, const int *, const int *, const int *) [with ProbT=float]" 
src/caffe/layers/ctc_loss_layer.cu(94): here
            instantiation of "ctcStatus_t caffe::compute_ctc_loss(const Dtype *, Dtype *, const int *, const int *, const int *, int, int, Dtype *, void *, ctcComputeInfo) [with Dtype=float]" 
src/caffe/layers/ctc_loss_layer.cu(136): here
            instantiation of "void caffe::gpu_ctc(Dtype *, Dtype *, int *, int *, int *, int, int, Dtype *, int) [with Dtype=float]" 
src/caffe/layers/ctc_loss_layer.cu(181): here
            instantiation of "void caffe::CTCLossLayer<Dtype>::Forward_gpu(const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &, const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &) [with Dtype=float]" 
src/caffe/layers/ctc_loss_layer.cu(226): here

./include/caffe/util/ctc/detail/gpu_ctc.h(143): error: identifier "constexpr" is undefined
          detected during:
            instantiation of "ctcStatus_t GpuCTC<ProbT>::create_metadata_and_choose_config(const int *, const int *, const int *, size_t &) [with ProbT=float]" 
(414): here
            instantiation of "ctcStatus_t GpuCTC<ProbT>::compute_cost_and_score(const ProbT *, ProbT *, ProbT *, const int *, const int *, const int *, __nv_bool, __nv_bool) [with ProbT=float]" 
(457): here
            instantiation of "ctcStatus_t GpuCTC<ProbT>::cost_and_grad(const ProbT *, ProbT *, ProbT *, const int *, const int *, const int *) [with ProbT=float]" 
src/caffe/layers/ctc_loss_layer.cu(94): here
            instantiation of "ctcStatus_t caffe::compute_ctc_loss(const Dtype *, Dtype *, const int *, const int *, const int *, int, int, Dtype *, void *, ctcComputeInfo) [with Dtype=float]" 
src/caffe/layers/ctc_loss_layer.cu(136): here
            instantiation of "void caffe::gpu_ctc(Dtype *, Dtype *, int *, int *, int *, int, int, Dtype *, int) [with Dtype=float]" 
src/caffe/layers/ctc_loss_layer.cu(181): here
            instantiation of "void caffe::CTCLossLayer<Dtype>::Forward_gpu(const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &, const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &) [with Dtype=float]" 
src/caffe/layers/ctc_loss_layer.cu(226): here

./include/caffe/util/ctc/detail/gpu_ctc.h(143): error: expected a ";"
          detected during:
            instantiation of "ctcStatus_t GpuCTC<ProbT>::create_metadata_and_choose_config(const int *, const int *, const int *, size_t &) [with ProbT=float]" 
(414): here
            instantiation of "ctcStatus_t GpuCTC<ProbT>::compute_cost_and_score(const ProbT *, ProbT *, ProbT *, const int *, const int *, const int *, __nv_bool, __nv_bool) [with ProbT=float]" 
(457): here
            instantiation of "ctcStatus_t GpuCTC<ProbT>::cost_and_grad(const ProbT *, ProbT *, ProbT *, const int *, const int *, const int *) [with ProbT=float]" 
src/caffe/layers/ctc_loss_layer.cu(94): here
            instantiation of "ctcStatus_t caffe::compute_ctc_loss(const Dtype *, Dtype *, const int *, const int *, const int *, int, int, Dtype *, void *, ctcComputeInfo) [with Dtype=float]" 
src/caffe/layers/ctc_loss_layer.cu(136): here
            instantiation of "void caffe::gpu_ctc(Dtype *, Dtype *, int *, int *, int *, int, int, Dtype *, int) [with Dtype=float]" 
src/caffe/layers/ctc_loss_layer.cu(181): here
            instantiation of "void caffe::CTCLossLayer<Dtype>::Forward_gpu(const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &, const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &) [with Dtype=float]" 
src/caffe/layers/ctc_loss_layer.cu(226): here

./include/caffe/util/ctc/detail/gpu_ctc.h(144): error: identifier "cpu_buffer_size" is undefined
          detected during:
            instantiation of "ctcStatus_t GpuCTC<ProbT>::create_metadata_and_choose_config(const int *, const int *, const int *, size_t &) [with ProbT=float]" 
(414): here
            instantiation of "ctcStatus_t GpuCTC<ProbT>::compute_cost_and_score(const ProbT *, ProbT *, ProbT *, const int *, const int *, const int *, __nv_bool, __nv_bool) [with ProbT=float]" 
(457): here
            instantiation of "ctcStatus_t GpuCTC<ProbT>::cost_and_grad(const ProbT *, ProbT *, ProbT *, const int *, const int *, const int *) [with ProbT=float]" 
src/caffe/layers/ctc_loss_layer.cu(94): here
            instantiation of "ctcStatus_t caffe::compute_ctc_loss(const Dtype *, Dtype *, const int *, const int *, const int *, int, int, Dtype *, void *, ctcComputeInfo) [with Dtype=float]" 
src/caffe/layers/ctc_loss_layer.cu(136): here
            instantiation of "void caffe::gpu_ctc(Dtype *, Dtype *, int *, int *, int *, int, int, Dtype *, int) [with Dtype=float]" 
src/caffe/layers/ctc_loss_layer.cu(181): here
            instantiation of "void caffe::CTCLossLayer<Dtype>::Forward_gpu(const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &, const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &) [with Dtype=float]" 
src/caffe/layers/ctc_loss_layer.cu(226): here

./include/caffe/util/ctc/detail/gpu_ctc.h(154): error: no instance of overloaded function "std::min" matches the argument list
            argument types are: (int, <error-type>)
          detected during:
            instantiation of "ctcStatus_t GpuCTC<ProbT>::create_metadata_and_choose_config(const int *, const int *, const int *, size_t &) [with ProbT=float]" 
(414): here
            instantiation of "ctcStatus_t GpuCTC<ProbT>::compute_cost_and_score(const ProbT *, ProbT *, ProbT *, const int *, const int *, const int *, __nv_bool, __nv_bool) [with ProbT=float]" 
(457): here
            instantiation of "ctcStatus_t GpuCTC<ProbT>::cost_and_grad(const ProbT *, ProbT *, ProbT *, const int *, const int *, const int *) [with ProbT=float]" 
src/caffe/layers/ctc_loss_layer.cu(94): here
            instantiation of "ctcStatus_t caffe::compute_ctc_loss(const Dtype *, Dtype *, const int *, const int *, const int *, int, int, Dtype *, void *, ctcComputeInfo) [with Dtype=float]" 
src/caffe/layers/ctc_loss_layer.cu(136): here
            instantiation of "void caffe::gpu_ctc(Dtype *, Dtype *, int *, int *, int *, int, int, Dtype *, int) [with Dtype=float]" 
src/caffe/layers/ctc_loss_layer.cu(181): here
            instantiation of "void caffe::CTCLossLayer<Dtype>::Forward_gpu(const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &, const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &) [with Dtype=float]" 
src/caffe/layers/ctc_loss_layer.cu(226): here

./include/caffe/util/ctc/detail/gpu_ctc_kernels.h(234): error: space required between adjacent ">" delimiters of nested template argument lists (">>" is the right shift operator)
          detected during:
            instantiation of "ctcStatus_t GpuCTC<ProbT>::launch_alpha_beta_kernels<NT,VT>(const ProbT *, ProbT *, __nv_bool, __nv_bool) [with ProbT=float, NT=32, VT=1]" 
./include/caffe/util/ctc/detail/gpu_ctc.h(336): here
            instantiation of "ctcStatus_t GpuCTC<ProbT>::launch_gpu_kernels(const ProbT *, ProbT *, size_t, __nv_bool, __nv_bool) [with ProbT=float]" 
./include/caffe/util/ctc/detail/gpu_ctc.h(426): here
            instantiation of "ctcStatus_t GpuCTC<ProbT>::compute_cost_and_score(const ProbT *, ProbT *, ProbT *, const int *, const int *, const int *, __nv_bool, __nv_bool) [with ProbT=float]" 
./include/caffe/util/ctc/detail/gpu_ctc.h(457): here
            instantiation of "ctcStatus_t GpuCTC<ProbT>::cost_and_grad(const ProbT *, ProbT *, ProbT *, const int *, const int *, const int *) [with ProbT=float]" 
src/caffe/layers/ctc_loss_layer.cu(94): here
            instantiation of "ctcStatus_t caffe::compute_ctc_loss(const Dtype *, Dtype *, const int *, const int *, const int *, int, int, Dtype *, void *, ctcComputeInfo) [with Dtype=float]" 
src/caffe/layers/ctc_loss_layer.cu(136): here
            instantiation of "void caffe::gpu_ctc(Dtype *, Dtype *, int *, int *, int *, int, int, Dtype *, int) [with Dtype=float]" 
src/caffe/layers/ctc_loss_layer.cu(181): here
            instantiation of "void caffe::CTCLossLayer<Dtype>::Forward_gpu(const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &, const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &) [with Dtype=float]" 
src/caffe/layers/ctc_loss_layer.cu(226): here

./include/caffe/util/ctc/detail/gpu_ctc_kernels.h(300): error: space required between adjacent ">" delimiters of nested template argument lists (">>" is the right shift operator)
          detected during:
            instantiation of "ctcStatus_t GpuCTC<ProbT>::launch_alpha_beta_kernels<NT,VT>(const ProbT *, ProbT *, __nv_bool, __nv_bool) [with ProbT=float, NT=32, VT=1]" 
./include/caffe/util/ctc/detail/gpu_ctc.h(336): here
            instantiation of "ctcStatus_t GpuCTC<ProbT>::launch_gpu_kernels(const ProbT *, ProbT *, size_t, __nv_bool, __nv_bool) [with ProbT=float]" 
./include/caffe/util/ctc/detail/gpu_ctc.h(426): here
            instantiation of "ctcStatus_t GpuCTC<ProbT>::compute_cost_and_score(const ProbT *, ProbT *, ProbT *, const int *, const int *, const int *, __nv_bool, __nv_bool) [with ProbT=float]" 
./include/caffe/util/ctc/detail/gpu_ctc.h(457): here
            instantiation of "ctcStatus_t GpuCTC<ProbT>::cost_and_grad(const ProbT *, ProbT *, ProbT *, const int *, const int *, const int *) [with ProbT=float]" 
src/caffe/layers/ctc_loss_layer.cu(94): here
            instantiation of "ctcStatus_t caffe::compute_ctc_loss(const Dtype *, Dtype *, const int *, const int *, const int *, int, int, Dtype *, void *, ctcComputeInfo) [with Dtype=float]" 
src/caffe/layers/ctc_loss_layer.cu(136): here
            instantiation of "void caffe::gpu_ctc(Dtype *, Dtype *, int *, int *, int *, int, int, Dtype *, int) [with Dtype=float]" 
src/caffe/layers/ctc_loss_layer.cu(181): here
            instantiation of "void caffe::CTCLossLayer<Dtype>::Forward_gpu(const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &, const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &) [with Dtype=float]" 
src/caffe/layers/ctc_loss_layer.cu(226): here

./include/caffe/util/ctc/detail/gpu_ctc.h(468): error: identifier "nullptr" is undefined
          detected during:
            instantiation of "ctcStatus_t caffe::compute_ctc_loss(const Dtype *, Dtype *, const int *, const int *, const int *, int, int, Dtype *, void *, ctcComputeInfo) [with Dtype=float]" 
src/caffe/layers/ctc_loss_layer.cu(136): here
            instantiation of "void caffe::gpu_ctc(Dtype *, Dtype *, int *, int *, int *, int, int, Dtype *, int) [with Dtype=float]" 
src/caffe/layers/ctc_loss_layer.cu(181): here
            instantiation of "void caffe::CTCLossLayer<Dtype>::Forward_gpu(const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &, const std::vector<caffe::Blob<Dtype> *, std::allocator<caffe::Blob<Dtype> *>> &) [with Dtype=float]" 
src/caffe/layers/ctc_loss_layer.cu(226): here

13 errors detected in the compilation of "/tmp/tmpxft_00000db6_00000000-6_ctc_loss_layer.cpp1.ii".
Makefile:599: recipe for target '.build_release/cuda/src/caffe/layers/ctc_loss_layer.o' failed
make: *** [.build_release/cuda/src/caffe/layers/ctc_loss_layer.o] Error 1

[mattroos]

It seems totally unrelated, but I also get this warning when compiling the CPU code...

CXX src/caffe/layers/ctc_loss_layer.cpp
src/caffe/layers/ctc_loss_layer.cpp: In function ‘void caffe::cpu_ctc(Dtype*, Dtype*, int*, int*, int*, int, int, Dtype*, int) [with Dtype = float]’:
src/caffe/layers/ctc_loss_layer.cpp:150:53: warning: ‘cpu_alloc_bytes’ may be used uninitialized in this function [-Wmaybe-uninitialized]
     void* ctc_cpu_workspace = malloc(cpu_alloc_bytes);
                                                     ^
src/caffe/layers/ctc_loss_layer.cpp: In function ‘void caffe::cpu_ctc(Dtype*, Dtype*, int*, int*, int*, int, int, Dtype*, int) [with Dtype = double]’:
src/caffe/layers/ctc_loss_layer.cpp:150:53: warning: ‘cpu_alloc_bytes’ may be used uninitialized in this function [-Wmaybe-uninitialized]

[MichalBusta]

Probably related to gcc version / cuda version.
is cuda code compiled with -std=c++11 flag? (if no, pass it to cmake)

[mattroos]

Yes, at around line 455 in Makefile, I have

CXXFLAGS += -std=c++11

[YuliyaLi]

@mattroos hi, i have the same problem, did you solve it? Looking forward your repply, thank you.

[mattroos]

@YuliyaLi, unfortunately I don't recall. I eventually moved the work to a different machine, which may have been because of the troubles described in this issue. I don't have access to the earlier machine anymore, so I can't review the state of things.

[sureshlakhani]

Did you guys able to fix this issue? if yes then how? any pointers can help