Removing resizing image step

[priyanka-chaudhary]

From the HED paper I understood that we don't need to resize images as the network doesn't have any fully connected layers. So for my own dataset I wanted to change your code to remove this step so it take any size image and also produces edge map of same size as image.

But just removing these lines

im = im.resize((self.cfgs['training']['image_width'], self.cfgs['training']['image_height']))
em = em.resize((self.cfgs['training']['image_width'], self.cfgs['training']['image_height']))
is giving an error.

Is it possible to do this with your code?
Any hint or pointer would be appreciated. Thank you.

[sandhawalia]

Hi, Could you please copy paste the error you are getting here ?

Yes indeed HED doesnt fixed size image for training. Image resizing is done for faster training in this case since batch processing comes in handy with resized images. One can ofcourse train with resized images and edgemaps and test with any resolution (single image at a time).

[priyanka-chaudhary]

I commented these lines in data_parser.py

im = im.resize((self.cfgs['training']['image_width'], self.cfgs['training']['image_height']))
em = em.resize((self.cfgs['training']['image_width'], self.cfgs['training']['image_height']))

And try to train I get the following error:

pchaudha@costa:~/hed$ python run-hed.py --train --gpu-limit=0.7 --config-file='/home/pchaudha/hed/hed/configs/hed.yaml'
2017-12-21 16:44:16.193934: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
2017-12-21 16:44:16.193956: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
2017-12-21 16:44:16.193961: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
2017-12-21 16:44:16.193964: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
2017-12-21 16:44:16.193968: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.
2017-12-21 16:44:16.425315: I tensorflow/core/common_runtime/gpu/gpu_device.cc:955] Found device 0 with properties:
name: GeForce GTX TITAN X
major: 5 minor: 2 memoryClockRate (GHz) 1.076
pciBusID 0000:01:00.0
Total memory: 11.91GiB
Free memory: 11.41GiB
2017-12-21 16:44:16.425345: I tensorflow/core/common_runtime/gpu/gpu_device.cc:976] DMA: 0
2017-12-21 16:44:16.425350: I tensorflow/core/common_runtime/gpu/gpu_device.cc:986] 0: Y
2017-12-21 16:44:16.425361: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1045] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX TITAN X, pci bus id: 0000:01:00.0)
[21 Dec 2017 16h44m16s][INFO] Model weights loaded from vgg16.npy
[21 Dec 2017 16h44m16s][INFO] Added CONV-BLOCK-1+SIDE-1
[21 Dec 2017 16h44m16s][INFO] Added CONV-BLOCK-2+SIDE-2
[21 Dec 2017 16h44m16s][INFO] Added CONV-BLOCK-3+SIDE-3
[21 Dec 2017 16h44m16s][INFO] Added CONV-BLOCK-4+SIDE-4
[21 Dec 2017 16h44m16s][INFO] Added CONV-BLOCK-5+SIDE-5
[21 Dec 2017 16h44m16s][INFO] Added FUSE layer
[21 Dec 2017 16h44m16s][INFO] Build model finished: 0.1302s
[21 Dec 2017 16h44m16s][INFO] Done initializing VGG-16 model
[21 Dec 2017 16h44m16s][INFO] Training data set-up from /home/pchaudha/hed/hed-data/HED-BSDS/train_pair.lst
[21 Dec 2017 16h44m16s][INFO] Training samples 23040
[21 Dec 2017 16h44m16s][INFO] Validation samples 5760
[21 Dec 2017 16h44m16s][WARNING] Deep supervision application set to True
Traceback (most recent call last):
File "run-hed.py", line 64, in
main(args)
File "run-hed.py", line 38, in main
trainer.run(session)
File "/home/pchaudha/hed/hed/train.py", line 69, in run
run_metadata=run_metadata)
File "/home/pchaudha/.local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 895, in run
run_metadata_ptr)
File "/home/pchaudha/.local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1093, in _run
np_val = np.asarray(subfeed_val, dtype=subfeed_dtype)
File "/home/pchaudha/.local/lib/python2.7/site-packages/numpy/core/numeric.py", line 531, in asarray
return array(a, dtype, copy=False, order=order)
ValueError: setting an array element with a sequence.

For this part of your comment: " One can ofcourse train with resized images and edgemaps and test with any resolution (single image at a time)."
What should be written for testing image_width and image_height in hed.yaml file then?

[sandhawalia]

For this part of your comment: " One can ofcourse train with resized images and edgemaps and test with any resolution (single image at a time)."
What should be written for testing image_width and image_height in hed.yaml file then?

You can set it to large value like 3200. This would work only if the test images are in a batch size of 1 though.

File "/home/pchaudha/.local/lib/python2.7/site-packages/numpy/core/numeric.py", line 531, in asarray
return array(a, dtype, copy=False, order=order)
ValueError: setting an array element with a sequence.

This part is due to the fact that all training image are of different sizes and cant be processed a as batch on GPU. A way around would be reduce batch size to 1 and disabling image width and height ( Disclaimer : i haven't tried it myself)

[priyanka-chaudhary]

"You can set it to large value like 3200. This would work only if the test images are in a batch size of 1 though."
I tried this and it works with 3200. But when I try another value it doesn't. Can you please give some insight why? Like when I tried with 1200 it gives error even with batch size 1. I know it sounds stupid but I am not able to figure it out.

"This part is due to the fact that all training image are of different sizes and cant be processed a as batch on GPU. A way around would be reduce batch size to 1 and disabling image width and height ( Disclaimer : i haven't tried it myself)"
Also I wanted to try this so how should I go about it?
Like changing batch_size_train to 1 I got.
What should I put for image_width and image_height?

Thanks a lot for the support. Really appreciate it.

[yfnn]

I also have the same question. Now your code give outputs of fixed size (480*320). But I want to get flexible output size. I hope the output size is the same with the input size. So, I change the some code. I change image placeholder's size into [None,None,None,3], and edgemap placeholder's size into [None,None,None,1]. Then, in test.py, I ignore the image resize line. But, I get this error. Could you give me some help?

2018-06-05 03:47:51.917737: W tensorflow/core/framework/op_kernel.cc:1152] Invalid argument: ConcatOp : Dimensions of inputs should match: shape[0] = [1,904,962,1] vs. shape[2] = [1,904,964,1]
[[Node: concat = ConcatV2[N=5, T=DT_FLOAT, Tidx=DT_INT32, _device="/job:localhost/replica:0/task:0/gpu:0"](side_1/conv2d_transpose, side_2/conv2d_transpose, side_3/conv2d_transpose, side_4/conv2d_transpose, side_5/conv2d_transpose, concat/axis)]]
Traceback (most recent call last):
File "run-hed.py", line 55, in
parser = argparse.ArgumentParser(description='Utility for Training/Testing DL models(Concepts/Captions) using theano/keras')
File "run-hed.py", line 44, in main
tester.run(session)
File "/home/fayan/holy-edge/hed/test.py", line 68, in run
edgemap = session.run(self.model.predictions, feed_dict={self.model.images: [im]})
File "/home/fayan/anaconda2/envs/python27/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 778, in run
run_metadata_ptr)
File "/home/fayan/anaconda2/envs/python27/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 983, in _run
feed_dict_string, options, run_metadata)
File "/home/fayan/anaconda2/envs/python27/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1033, in _do_run
target_list, options, run_metadata)
File "/home/fayan/anaconda2/envs/python27/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1053, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: ConcatOp : Dimensions of inputs should match: shape[0] = [1,904,962,1] vs. shape[2] = [1,904,964,1]
[[Node: concat = ConcatV2[N=5, T=DT_FLOAT, Tidx=DT_INT32, _device="/job:localhost/replica:0/task:0/gpu:0"](side_1/conv2d_transpose, side_2/conv2d_transpose, side_3/conv2d_transpose, side_4/conv2d_transpose, side_5/conv2d_transpose, concat/axis)]]
[[Node: output_5/_45 = _Recvclient_terminated=false, recv_device="/job:localhost/replica:0/task:0/cpu:0", send_device="/job:localhost/replica:0/task:0/gpu:0", send_device_incarnation=1, tensor_name="edge_23_output_5", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/cpu:0"]]

Caused by op u'concat', defined at:
File "run-hed.py", line 55, in
parser = argparse.ArgumentParser(description='Utility for Training/Testing DL models(Concepts/Captions) using theano/keras')
File "run-hed.py", line 43, in main
tester.setup(session)
File "/home/fayan/holy-edge/hed/test.py", line 37, in setup
self.model = Vgg16(self.cfgs, run='testing')
File "/home/fayan/holy-edge/hed/models/vgg16.py", line 33, in init
self.define_model()
File "/home/fayan/holy-edge/hed/models/vgg16.py", line 84, in define_model
self.fuse = self.conv_layer(tf.concat(self.side_outputs, axis=3),
File "/home/fayan/anaconda2/envs/python27/lib/python2.7/site-packages/tensorflow/python/ops/array_ops.py", line 1034, in concat
name=name)
File "/home/fayan/anaconda2/envs/python27/lib/python2.7/site-packages/tensorflow/python/ops/gen_array_ops.py", line 519, in _concat_v2
name=name)
File "/home/fayan/anaconda2/envs/python27/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 768, in apply_op
op_def=op_def)
File "/home/fayan/anaconda2/envs/python27/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2336, in create_op
original_op=self._default_original_op, op_def=op_def)
File "/home/fayan/anaconda2/envs/python27/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1228, in init
self._traceback = _extract_stack()

InvalidArgumentError (see above for traceback): ConcatOp : Dimensions of inputs should match: shape[0] = [1,904,962,1] vs. shape[2] = [1,904,964,1]
[[Node: concat = ConcatV2[N=5, T=DT_FLOAT, Tidx=DT_INT32, _device="/job:localhost/replica:0/task:0/gpu:0"](side_1/conv2d_transpose, side_2/conv2d_transpose, side_3/conv2d_transpose, side_4/conv2d_transpose, side_5/conv2d_transpose, concat/axis)]]
[[Node: output_5/_45 = _Recvclient_terminated=false, recv_device="/job:localhost/replica:0/task:0/cpu:0", send_device="/job:localhost/replica:0/task:0/gpu:0", send_device_incarnation=1, tensor_name="edge_23_output_5", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/cpu:0"]]