Attached is a screenshot of the issue I see. This PR should be able to handle transposes with branches.

Thanks for the PR @jicampos ! It seems you have identified a bug, but the solution isn't quite correct. When we have a pattern that looks like this, where the first transpose is branching and has a opposing transpose in the following branch:

...we actually need to skip removing those two transposes in RemoveConsecutiveChanFirstAndChanLastTrafos because there is now no tensor that can correctly feed the branch that used to fork from the output of the first transpose. The correct (and simpler) solution is to skip RemoveConsecutiveChanFirstAndChanLastTrafos when the first transpose has a forking output, and instead let the MoveTransposePastFork make copies of it on both branches later on. Afterwards, we get opposing transposes with no branching in between, and can remove them safely. I'll open a separate PR that implements this simpler solution, and add your testcase to the test suite. I'm closing this PR since the proposed solution is unfortunately not valid.