Tensor subnetwork canonicalization.

[bimalgaudel]

Tensor subnetwork canonicalization is useful to identify common sub-expressions.
As of now, the subnetwork canonicalization for tensors without proto-indices seems to at least not throw (have not been used in common sub-expression elimination to test out the robustness). However, the canonicalization of subnetworks involving tensors with proto-indices throws.

Examples are in the linked branch under test_tensor_network.cpp.

[Krzmbrzl]

This should probably only be tackled once my canonicalizer revamp has been merged (if it is still a problem).

Also, I kinda think the way to implement CSE would be via a cleverly designed hash function rather than via explicit canonicalization of all pairs of tensors in a network.

[bimalgaudel]

@Krzmbrzl A cleverly designed hash function is currently used to use common sub-expression elimination during evaluation. We want to use graph canonicalization to make the common sub-expressions detection at symbolic level rather than evaluation node level so that the CSE becomes even more robust. I do not expect this feature to improve the runtime significantly, however, with this implemented we will have a uniform way of finding common sub-expressions for evaluation nodes as well as symbolic refactoring.
Additionally, for tensors with proto-indices it is easy to mess up in devising the clever hash function, so the graph canonicalization is desirable.

Also, looking forward to your canonicalizer revamp!

[Krzmbrzl]

I actually meant to have a hash function on the symbolic level. Maybe we just put this on the list for our next meeting in order to make sure everyone is speaking about the same thing?