Open
Description
There're several problems with inplace option in layers:
- some layers has inplace option while others not.
- within the layers that has inplace option, some modifies input and change it back in backward (e.g. AddConstant), while some just modify the input wihthout changing it back (e.g. Dropout).
- layer's input changed in place may cause unwanted results if used carelessly
- if inplace change is used as the first layer, user's direct input will be changed.
- in branch case (i.e. one layer has two output branches), if the last layer's output is changed by the first branch, the second branch may get wrong input.
- Also, input option should not be visible to end user, as it's implementation-level optimization, we should analyze it in graph level and reuse buffers when we can.