Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

caffe fork runtest failed #2

Open
ardiya opened this issue Jun 2, 2017 · 10 comments
Open

caffe fork runtest failed #2

ardiya opened this issue Jun 2, 2017 · 10 comments

Comments

@ardiya
Copy link

ardiya commented Jun 2, 2017 •
edited
Loading

Hi, I want to post this in your caffe fork but there is no issue tab there.

So, I have tried to compile your latest caffe fork and encounter some failure when running `make runtest' there.

Do you have any suggestion? Perhaps something like revert to the stable commit or your DenseBlock_release1 will do?

The failure happen when testing DenseBlock layer and

[ RUN      ] DenseBlockLayerTest/0.TestDenseBlockBwd
/home/amax/ardiya/dncaffe/src/caffe/test/test_DenseBlock_layer.cpp:317: Failure
The difference between filter10layer3->diff_at(outCIdx,inCIdx,filterHIdx,filterWIdx) and filter10layer4->diff_at(outCIdx,inCIdx,filterHIdx,filterWIdx) is   0.55306005477905273, which exceeds 0.4, where
filter10layer3->diff_at(outCIdx,inCIdx,filterHIdx,filterWIdx) evaluates to 0.15142345428466797,
filter10layer4->diff_at(outCIdx,inCIdx,filterHIdx,filterWIdx) evaluates to 0.7044835090637207, and
0.4 evaluates to 0.40000000000000002.
/home/amax/ardiya/dncaffe/src/caffe/test/test_DenseBlock_layer.cpp:317: Failure
The difference between filter10layer3->diff_at(outCIdx,inCIdx,filterHIdx,filterWIdx) and filter10layer4->diff_at(outCIdx,inCIdx,filterHIdx,filterWIdx) is   0.54173290729522705, which exceeds 0.4, where
filter10layer3->diff_at(outCIdx,inCIdx,filterHIdx,filterWIdx) evaluates to -1.1120017766952515,
filter10layer4->diff_at(outCIdx,inCIdx,filterHIdx,filterWIdx) evaluates to -1.6537346839904785, and
0.4 evaluates to 0.40000000000000002.
/home/amax/ardiya/dncaffe/src/caffe/test/test_DenseBlock_layer.cpp:317: Failure
The difference between filter10layer3->diff_at(outCIdx,inCIdx,filterHIdx,filterWIdx) and filter10layer4->diff_at(outCIdx,inCIdx,filterHIdx,filterWIdx) is   1.1154916286468506, which exceeds 0.4, where
filter10layer3->diff_at(outCIdx,inCIdx,filterHIdx,filterWIdx) evaluates to -1.2618510723114014,
filter10layer4->diff_at(outCIdx,inCIdx,filterHIdx,filterWIdx) evaluates to -0.14635944366455078, and
0.4 evaluates to 0.40000000000000002.
/home/amax/ardiya/dncaffe/src/caffe/test/test_DenseBlock_layer.cpp:317: Failure
The difference between filter10layer3->diff_at(outCIdx,inCIdx,filterHIdx,filterWIdx) and filter10layer4->diff_at(outCIdx,inCIdx,filterHIdx,filterWIdx) is   0.5621497631072998, which exceeds 0.4, where
filter10layer3->diff_at(outCIdx,inCIdx,filterHIdx,filterWIdx) evaluates to -1.9996230602264404,
filter10layer4->diff_at(outCIdx,inCIdx,filterHIdx,filterWIdx) evaluates to -1.4374732971191406, and
0.4 evaluates to 0.40000000000000002.
/home/amax/ardiya/dncaffe/src/caffe/test/test_DenseBlock_layer.cpp:336: Failure
The difference between layer3localScaler->diff_at(0,channelIdx,0,0) and layer4localScaler->diff_at(0,channelIdx,0,0) is 1.100369930267334, which exceeds 0  .4, where
layer3localScaler->diff_at(0,channelIdx,0,0) evaluates to 4.863670825958252,
layer4localScaler->diff_at(0,channelIdx,0,0) evaluates to 3.763300895690918, and
0.4 evaluates to 0.40000000000000002.
/home/amax/ardiya/dncaffe/src/caffe/test/test_DenseBlock_layer.cpp:345: Failure
The difference between layer3localBias->diff_at(0,channelIdx,0,0) and layer4localBias->diff_at(0,channelIdx,0,0) is 0.55823898315429688, which exceeds 0.4  , where
layer3localBias->diff_at(0,channelIdx,0,0) evaluates to 38.147781372070312,
layer4localBias->diff_at(0,channelIdx,0,0) evaluates to 37.589542388916016, and
0.4 evaluates to 0.40000000000000002.
[  FAILED  ] DenseBlockLayerTest/0.TestDenseBlockBwd, where TypeParam = caffe::CPUDevice<float> (751 ms)

and the data layer.

[ RUN      ] DataLayerTest/0.TestReadCropTestLevelDB
/home/amax/ardiya/dncaffe/src/caffe/test/test_data_layer.cpp:229: Failure
Value of: blob_top_label_->cpu_data()[i]
  Actual: 1
Expected: i
Which is: 0
/home/amax/ardiya/dncaffe/src/caffe/test/test_data_layer.cpp:229: Failure
Value of: blob_top_label_->cpu_data()[i]
  Actual: 0
Expected: i
Which is: 1
/home/amax/ardiya/dncaffe/src/caffe/test/test_data_layer.cpp:229: Failure
Value of: blob_top_label_->cpu_data()[i]
  Actual: 4
Expected: i
Which is: 2
/home/amax/ardiya/dncaffe/src/caffe/test/test_data_layer.cpp:229: Failure
Value of: blob_top_label_->cpu_data()[i]
  Actual: 2
Expected: i
Which is: 4
[  FAILED  ] DataLayerTest/0.TestReadCropTestLevelDB, where TypeParam = caPUDevice<float> (282 ms)
[ RUN      ] DataLayerTest/0.TestSkipLevelDB
/home/amax/ardiya/dncaffe/src/caffe/test/test_data_layer.cpp:125: Failure
Value of: blob_top_label_->cpu_data()[i]
  Actual: 1
Expected: label % batch_size
Which is: 3
/home/amax/ardiya/dncaffe/src/caffe/test/test_data_layer.cpp:125: Failure
Value of: blob_top_label_->cpu_data()[i]
  Actual: 0
Expected: label % batch_size
Which is: 1
/home/amax/ardiya/dncaffe/src/caffe/test/test_data_layer.cpp:125: Failure
Value of: blob_top_label_->cpu_data()[i]
  Actual: 4
Expected: label % batch_size
Which is: 3
/home/amax/ardiya/dncaffe/src/caffe/test/test_data_layer.cpp:125: Failure
Value of: blob_top_label_->cpu_data()[i]
  Actual: 3
Expected: label % batch_size
Which is: 1
...........(omitted)
[  FAILED  ] DataLayerTest/0.TestReadCropTestLevelDB, where TypeParam = caPUDevice<float> (282 ms)
@ardiya
Copy link
Author

ardiya commented Jun 2, 2017

I tried to train it on imagenet, and the data layer failed to load lmdb. Any idea?

@ardiya
Copy link
Author

ardiya commented Jun 3, 2017

lmdb test fixed by changing random_order in caffe.proto to optional bool random_order = 11 [default = false]; still not sure about the backpropagation error in denseblock layer

@Tongcheng
Copy link
Owner

Hello @ardiya , for the first question which is failure of unit test about DenseBlock, one strange part is there is the numbers on both sides seem to be large integers, so what is the order of instructions runned? And what is your environment? Thanks.

@ardiya
Copy link
Author

ardiya commented Jun 3, 2017 •
edited
Loading

Hi @Tongcheng, thanks for replying.

one strange part is there is the numbers on both sides seem to be large integers

It is a float number, but there might be some problem when I copied it.
Here is the pastebin link for a better viewing https://pastebin.com/Jh8iQeNT

so what is the order of instructions runned?

I clone it, removeCmakeDep.sh, mkdir build, cd build, cmake .., make -j8 and make runtest -j8

what is your environment?

I am using Ubuntu 14.04.1 and this is the result of cmake .. https://pastebin.com/yujC0Ry5
Please tell me if you need more info, because I really have no idea what should I write here.

@zhangleiedu
Copy link

I encounter same problem when i run make runtest. Can anyone help me?
thanks
`
[----------] Global test environment tear-down
[==========] 1957 tests from 268 test cases ran. (313258 ms total)
[ PASSED ] 1945 tests.
[ FAILED ] 12 tests, listed below:
[ FAILED ] DenseBlockLayerTest/0.TestDenseBlockBwd, where TypeParam = caffe::CPUDevice
[ FAILED ] DenseBlockLayerTest/1.TestDenseBlockBwd, where TypeParam = caffe::CPUDevice
[ FAILED ] DenseBlockLayerTest/2.TestDenseBlockBwd, where TypeParam = caffe::GPUDevice
[ FAILED ] DenseBlockLayerTest/3.TestDenseBlockBwd, where TypeParam = caffe::GPUDevice
[ FAILED ] NesterovSolverTest/0.TestLeastSquaresUpdateWithMomentumMultiIter, where TypeParam = caffe::CPUDevice
[ FAILED ] NesterovSolverTest/0.TestNesterovLeastSquaresUpdateWithEverythingShare, where TypeParam = caffe::CPUDevice
[ FAILED ] NesterovSolverTest/0.TestNesterovLeastSquaresUpdateWithMomentum, where TypeParam = caffe::CPUDevice
[ FAILED ] NesterovSolverTest/0.TestNesterovLeastSquaresUpdateWithEverything, where TypeParam = caffe::CPUDevice
[ FAILED ] NesterovSolverTest/1.TestNesterovLeastSquaresUpdateWithMomentum, where TypeParam = caffe::CPUDevice
[ FAILED ] NesterovSolverTest/1.TestNesterovLeastSquaresUpdateWithEverythingShare, where TypeParam = caffe::CPUDevice
[ FAILED ] NesterovSolverTest/1.TestNesterovLeastSquaresUpdateWithEverything, where TypeParam = caffe::CPUDevice
[ FAILED ] NesterovSolverTest/1.TestLeastSquaresUpdateWithMomentumMultiIter, where TypeParam = caffe::CPUDevice

12 FAILED TESTS
make[3]: *** [src/caffe/test/CMakeFiles/runtest] 错误 1
make[2]: *** [src/caffe/test/CMakeFiles/runtest.dir/all] 错误 2
make[1]: *** [src/caffe/test/CMakeFiles/runtest.dir/rule] 错误 2
make: *** [runtest] 错误 2
`

@ToruHironaka
Copy link

I got the same problem. I think I did the same procedure as ardiya did. I cloned https://github.com/Tongcheng/caffe.git. Then, run removeCmakeDep.sh, mkdir build, cd build, cmake .., make -j4 and make runtest -j4. I am running Ubuntu 16.04, CUDA 8.0, cuDnn v5.1.

@zhimengfan1990
Copy link

Hi, @ardiya I suffer from similar problem, have you solved this? thanks.

@DanChen001
Copy link

Hi, @Tongcheng @ardiya @zhimengfan1990 I meet the same problem while running 'make runtest', have you solved this? thanks.

%%
......
/home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure
Value of: blob_top_label_->cpu_data()[i]
Actual: 4
Expected: label % batch_size
Which is: 3
/home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure
Value of: blob_top_label_->cpu_data()[i]
Actual: 3
Expected: label % batch_size
Which is: 1
/home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure
Value of: blob_top_label_->cpu_data()[i]
Actual: 0
Expected: label % batch_size
Which is: 4
/home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure
Value of: blob_top_label_->cpu_data()[i]
Actual: 3
Expected: label % batch_size
Which is: 2
/home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure
Value of: blob_top_label_->cpu_data()[i]
Actual: 2
Expected: label % batch_size
Which is: 0
/home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure
Value of: blob_top_label_->cpu_data()[i]
Actual: 0
Expected: label % batch_size
Which is: 3
/home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure
Value of: blob_top_label_->cpu_data()[i]
Actual: 0
Expected: label % batch_size
Which is: 1
/home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure
Value of: blob_top_label_->cpu_data()[i]
Actual: 3
Expected: label % batch_size
Which is: 2
/home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure
Value of: blob_top_label_->cpu_data()[i]
Actual: 3
Expected: label % batch_size
Which is: 0
/home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure
Value of: blob_top_label_->cpu_data()[i]
Actual: 4
Expected: label % batch_size
Which is: 0
/home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure
Value of: blob_top_label_->cpu_data()[i]
Actual: 1
Expected: label % batch_size
Which is: 3
/home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure
Value of: blob_top_label_->cpu_data()[i]
Actual: 3
Expected: label % batch_size
Which is: 1
/home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure
Value of: blob_top_label_->cpu_data()[i]
Actual: 1
Expected: label % batch_size
Which is: 4
/home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure
Value of: blob_top_label_->cpu_data()[i]
Actual: 3
Expected: label % batch_size
Which is: 2
/home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure
Value of: blob_top_label_->cpu_data()[i]
Actual: 1
Expected: label % batch_size
Which is: 0
/home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure
Value of: blob_top_label_->cpu_data()[i]
Actual: 0
Expected: label % batch_size
Which is: 3
/home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure
Value of: blob_top_label_->cpu_data()[i]
Actual: 2
Expected: label % batch_size
Which is: 1
/home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure
Value of: blob_top_label_->cpu_data()[i]
Actual: 3
Expected: label % batch_size
Which is: 4
/home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure
Value of: blob_top_label_->cpu_data()[i]
Actual: 3
Expected: label % batch_size
Which is: 2
/home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure
Value of: blob_top_label_->cpu_data()[i]
Actual: 1
Expected: label % batch_size
Which is: 0
/home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure
Value of: blob_top_label_->cpu_data()[i]
Actual: 0
Expected: label % batch_size
Which is: 3
/home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure
Value of: blob_top_label_->cpu_data()[i]
Actual: 4
Expected: label % batch_size
Which is: 1
/home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure
Value of: blob_top_label_->cpu_data()[i]
Actual: 2
Expected: label % batch_size
Which is: 4
/home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure
Value of: blob_top_label_->cpu_data()[i]
Actual: 4
Expected: label % batch_size
Which is: 2
/home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure
Value of: blob_top_label_->cpu_data()[i]
Actual: 1
Expected: label % batch_size
Which is: 3
/home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure
Value of: blob_top_label_->cpu_data()[i]
Actual: 4
Expected: label % batch_size
Which is: 1
/home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure
Value of: blob_top_label_->cpu_data()[i]
Actual: 3
Expected: label % batch_size
Which is: 2
/home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure
Value of: blob_top_label_->cpu_data()[i]
Actual: 1
Expected: label % batch_size
Which is: 0
/home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure
Value of: blob_top_label_->cpu_data()[i]
Actual: 3
Expected: label % batch_size
Which is: 1
/home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure
Value of: blob_top_label_->cpu_data()[i]
Actual: 3
Expected: label % batch_size
Which is: 4
/home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure
Value of: blob_top_label_->cpu_data()[i]
Actual: 2
Expected: label % batch_size
Which is: 3
/home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure
Value of: blob_top_label_->cpu_data()[i]
Actual: 4
Expected: label % batch_size
Which is: 1
/home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure
Value of: blob_top_label_->cpu_data()[i]
Actual: 1
Expected: label % batch_size
Which is: 4
/home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure
Value of: blob_top_label_->cpu_data()[i]
Actual: 3
Expected: label % batch_size
Which is: 2
/home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure
Value of: blob_top_label_->cpu_data()[i]
Actual: 4
Expected: label % batch_size
Which is: 0
/home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure
Value of: blob_top_label_->cpu_data()[i]
Actual: 1
Expected: label % batch_size
Which is: 3
/home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure
Value of: blob_top_label_->cpu_data()[i]
Actual: 1
Expected: label % batch_size
Which is: 4
/home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure
Value of: blob_top_label_->cpu_data()[i]
Actual: 0
Expected: label % batch_size
Which is: 2
/home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure
Value of: blob_top_label_->cpu_data()[i]
Actual: 1
Expected: label % batch_size
Which is: 0
/home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure
Value of: blob_top_label_->cpu_data()[i]
Actual: 4
Expected: label % batch_size
Which is: 3
/home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure
Value of: blob_top_label_->cpu_data()[i]
Actual: 3
Expected: label % batch_size
Which is: 1
/home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure
Value of: blob_top_label_->cpu_data()[i]
Actual: 0
Expected: label % batch_size
Which is: 4
/home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure
Value of: blob_top_label_->cpu_data()[i]
Actual: 0
Expected: label % batch_size
Which is: 2
/home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure
Value of: blob_top_label_->cpu_data()[i]
Actual: 3
Expected: label % batch_size
Which is: 0
/home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure
Value of: blob_top_label_->cpu_data()[i]
Actual: 4
Expected: label % batch_size
Which is: 3
/home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure
Value of: blob_top_label_->cpu_data()[i]
Actual: 0
Expected: label % batch_size
Which is: 1
/home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure
Value of: blob_top_label_->cpu_data()[i]
Actual: 3
Expected: label % batch_size
Which is: 4
/home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure
Value of: blob_top_label_->cpu_data()[i]
Actual: 3
Expected: label % batch_size
Which is: 2
/home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure
Value of: blob_top_label_->cpu_data()[i]
Actual: 1
Expected: label % batch_size
Which is: 3
/home/NAME/桌面/caffe-master-densenet/caffe/src/caffe/test/test_data_layer.cpp:125: Failure
Value of: blob_top_label_->cpu_data()[i]
Actual: 4
Expected: label % batch_size
Which is: 1
.....

%%

@DanChen001
Copy link

Hi, @zhangleiedu, I meet the same problem? have you solve this problem? thanks!
%%
[ FAILED ] 12 tests, listed below:
[ FAILED ] NesterovSolverTest/0.TestNesterovLeastSquaresUpdateWithMomentum, where TypeParam = caffe::CPUDevice
[ FAILED ] NesterovSolverTest/0.TestNesterovLeastSquaresUpdateWithEverythingShare, where TypeParam = caffe::CPUDevice
[ FAILED ] NesterovSolverTest/0.TestLeastSquaresUpdateWithMomentumMultiIter, where TypeParam = caffe::CPUDevice
[ FAILED ] NesterovSolverTest/0.TestNesterovLeastSquaresUpdateWithEverything, where TypeParam = caffe::CPUDevice
[ FAILED ] NesterovSolverTest/1.TestNesterovLeastSquaresUpdateWithEverythingShare, where TypeParam = caffe::CPUDevice
[ FAILED ] NesterovSolverTest/1.TestLeastSquaresUpdateWithMomentumMultiIter, where TypeParam = caffe::CPUDevice
[ FAILED ] NesterovSolverTest/1.TestNesterovLeastSquaresUpdateWithMomentum, where TypeParam = caffe::CPUDevice
[ FAILED ] NesterovSolverTest/1.TestNesterovLeastSquaresUpdateWithEverything, where TypeParam = caffe::CPUDevice
[ FAILED ] DenseBlockLayerTest/0.TestDenseBlockBwd, where TypeParam = caffe::CPUDevice
[ FAILED ] DenseBlockLayerTest/1.TestDenseBlockBwd, where TypeParam = caffe::CPUDevice
[ FAILED ] DenseBlockLayerTest/2.TestDenseBlockBwd, where TypeParam = caffe::GPUDevice
[ FAILED ] DenseBlockLayerTest/3.TestDenseBlockBwd, where TypeParam = caffe::GPUDevice

12 FAILED TESTS
make[3]: *** [src/caffe/test/CMakeFiles/runtest] 错误 1
make[2]: *** [src/caffe/test/CMakeFiles/runtest.dir/all] 错误 2
make[1]: *** [src/caffe/test/CMakeFiles/runtest.dir/rule] 错误 2
make: *** [runtest] 错误 2
%%

@sytow
Copy link

sytow commented May 4, 2018

hello @ardiya @Tongcheng I meet exactly the same problem, have you solved it? Despite the unsettled problem, I still successfully trained my own model based on Dense_Block, but when I tested the model using python interface, it occured that:

WARNING: Logging before InitGoogleLogging() is written to STDERR
W0504 11:01:52.860939 8205 _caffe.cpp:139] DEPRECATION WARNING - deprecated use of Python interface
W0504 11:01:52.860967 8205 _caffe.cpp:140] Use this instead (with the named "weights" parameter):
W0504 11:01:52.860972 8205 _caffe.cpp:142] Net('/home/sunyu/caffe-master/data/mydata/deploy.prototxt', 1, weights='/home/sunyu/caffe-master/data/mydata/Dense_EXP_1/my_regression_iter_35000.caffemodel')
[libprotobuf ERROR google/protobuf/text_format.cc:274] Error parsing text-format caffe.NetParameter: 56:20: Message type "caffe.LayerParameter" has no field named "denseblock_param".
F0504 11:01:52.862316 8205 upgrade_proto.cpp:90] Check failed: ReadProtoFromTextFile(param_file, param) Failed to parse NetParameter file: /home/sunyu/caffe-master/data/mydata/deploy.prototxt

I just cannot figure out that why "caffe.LayerParameter" has no field named "denseblock_param" only during the test phase while trainning is ok?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
7 participants
@ardiya @Tongcheng @ToruHironaka @zhimengfan1990 @zhangleiedu @sytow @DanChen001