[DON'T MERGE] Normalize GAIL encoder inputs

[ervteng]

Proposed change(s)

Normalize the GAIL encoder inputs using incoming trajectories; this was done in TF but not in Torch. This helps stabilize training and improves performance on Crawler with GAIL.

Types of change(s)

• Bug fix
• New feature
• Code refactor
• Breaking change
• Documentation update
• Other (please describe)

Checklist

• Added tests that prove my fix is effective or that my feature works
• Updated the changelog (if applicable)
• Updated the documentation (if applicable)
• Updated the migration guide (if applicable)

Other comments

[vincentpierre]

I do think it makes more sense to compute the gradient with respect to the normalized observations rather than the raw ones. Not sure at all how to do this without having to take the normalization out of the NetworkBody.

[ervteng]

I think without doing this it's actually broken and doesn't train.

We could do the ugly thing of reaching into the NetworkBody and grabbing the normalized obs, or grabbing the normalizer and applying it to the observation.

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.

---

Ervin Teng seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}