[FLAVA]Separate the pretraining loss from the pretraininig model

[ankitade]

Stack from ghstack (oldest at bottom):

• -> [FLAVA]Separate the pretraining loss from the pretraininig model #278
• [FLAVA] Move masked prediction head to flava_for_pretraining #195
• [FLAVA]Move itm head to flava model for pretraining #132
• change order of itm loss init #131

[codecov-commenter]

Codecov Report

❗ No coverage uploaded for pull request base (gh/ankitade/12/base@42f218c). Click here to learn what that means.
The diff coverage is n/a.

@@                  Coverage Diff                   @@
##             gh/ankitade/12/base     #278   +/-   ##
======================================================
  Coverage                       ?   92.26%           
======================================================
  Files                          ?       53           
  Lines                          ?     3193           
  Branches                       ?        0           
======================================================
  Hits                           ?     2946           
  Misses                         ?      247           
  Partials                       ?        0           

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.