using affinity = "full" breaks the code

[yadav-suraj]

Hi,

Thanks for the paper and the code.

In the networks.py and costs.py you have made an arrangement to deal with case where user wants to calculate full affinity matrix and not just knn affinity matrix but if I set "affinity : full" in run.py, it produces an error related to the cholesky factorization of the output of the feed forward NN.

/home/suraj/anaconda3/envs/suraj-tf/lib/python3.6/site-packages/sklearn/externals/joblib/externals/cloudpickle/cloudpickle.py:47: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import imp
Using TensorFlow backend.
/home/suraj/anaconda3/envs/suraj-tf/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:458: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
/home/suraj/anaconda3/envs/suraj-tf/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:459: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/home/suraj/anaconda3/envs/suraj-tf/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:460: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
/home/suraj/anaconda3/envs/suraj-tf/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:461: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/home/suraj/anaconda3/envs/suraj-tf/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:462: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint32 = np.dtype([("qint32", np.int32, 1)])
/home/suraj/anaconda3/envs/suraj-tf/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:465: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  np_resource = np.dtype([("resource", np.ubyte, 1)])
2020-12-20 14:18:42.250842: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
2020-12-20 14:18:42.250902: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
2020-12-20 14:18:42.250910: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
2020-12-20 14:18:42.250917: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
2020-12-20 14:18:42.250923: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX512F instructions, but these are available on your machine and could speed up CPU computations.
2020-12-20 14:18:42.250929: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.
Epoch: 0, loss=184521.364868, val_loss=12510.053711
Traceback (most recent call last):
  File "/home/suraj/anaconda3/envs/suraj-tf/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1327, in _do_call
    return fn(*args)
  File "/home/suraj/anaconda3/envs/suraj-tf/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1306, in _run_fn
    status, run_metadata)
  File "/home/suraj/anaconda3/envs/suraj-tf/lib/python3.6/contextlib.py", line 88, in __exit__
    next(self.gen)
  File "/home/suraj/anaconda3/envs/suraj-tf/lib/python3.6/site-packages/tensorflow/python/framework/errors_impl.py", line 466, in raise_exception_on_not_ok_status
    pywrap_tensorflow.TF_GetCode(status))
tensorflow.python.framework.errors_impl.InvalidArgumentError: Cholesky decomposition was not successful. The input might not be valid.
	 [[Node: Cholesky = Cholesky[T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/cpu:0"](add)]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "run.py", line 174, in <module>
    x_spectralnet, y_spectralnet = run_net(data, params)
  File "/home/suraj/Suraj/newfolder/SpectralNet/src/applications/spectralnet.py", line 94, in run_net
    params['spec_ne'])
  File "/home/suraj/Suraj/newfolder/SpectralNet/src/core/networks.py", line 166, in train
    batches_per_epoch=100)[0]
  File "/home/suraj/Suraj/newfolder/SpectralNet/src/core/train.py", line 92, in train_step
    return_vars_ += np.asarray(K.get_session().run(all_vars, feed_dict=feed_dict)[:len(return_var)])
  File "/home/suraj/anaconda3/envs/suraj-tf/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 895, in run
    run_metadata_ptr)
  File "/home/suraj/anaconda3/envs/suraj-tf/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1124, in _run
    feed_dict_tensor, options, run_metadata)
  File "/home/suraj/anaconda3/envs/suraj-tf/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1321, in _do_run
    options, run_metadata)
  File "/home/suraj/anaconda3/envs/suraj-tf/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1340, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Cholesky decomposition was not successful. The input might not be valid.
	 [[Node: Cholesky = Cholesky[T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/cpu:0"](add)]]

Caused by op 'Cholesky', defined at:
  File "run.py", line 174, in <module>
    x_spectralnet, y_spectralnet = run_net(data, params)
  File "/home/suraj/Suraj/newfolder/SpectralNet/src/applications/spectralnet.py", line 89, in run_net
    params['n_nbrs'], batch_sizes, siamese_net, x_train, len(x_train_labeled))
  File "/home/suraj/Suraj/newfolder/SpectralNet/src/core/networks.py", line 92, in __init__
    self.outputs = stack_layers(self.inputs, self.layers)
  File "/home/suraj/Suraj/newfolder/SpectralNet/src/core/layer.py", line 104, in stack_layers
    l = Orthonorm(outputs['Orthonorm'], name=layer.get('name'));
  File "/home/suraj/Suraj/newfolder/SpectralNet/src/core/layer.py", line 39, in Orthonorm
    ortho_weights = orthonorm_op(x)
  File "/home/suraj/Suraj/newfolder/SpectralNet/src/core/layer.py", line 22, in orthonorm_op
    L = tf.cholesky(x_2)
  File "/home/suraj/anaconda3/envs/suraj-tf/lib/python3.6/site-packages/tensorflow/python/ops/gen_linalg_ops.py", line 234, in cholesky
    result = _op_def_lib.apply_op("Cholesky", input=input, name=name)
  File "/home/suraj/anaconda3/envs/suraj-tf/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 767, in apply_op
    op_def=op_def)
  File "/home/suraj/anaconda3/envs/suraj-tf/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 2630, in create_op
    original_op=self._default_original_op, op_def=op_def)
  File "/home/suraj/anaconda3/envs/suraj-tf/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1204, in __init__
    self._traceback = self._graph._extract_stack()  # pylint: disable=protected-access

InvalidArgumentError (see above for traceback): Cholesky decomposition was not successful. The input might not be valid.
	 [[Node: Cholesky = Cholesky[T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/cpu:0"](add)]]

I am very confused here since cholesky factorization is done on the output (Y^T Y) of the NN and that no way related to the affinity matrix, yet it shows an error that cholesky decomposition was not successful. Ideally, changing "affinity" option should not make any changes to output Y of the NN. Any clarification would be very much helpful!