Breaking change due to multiprocessing.Process when loading pytorch_model.bin-based model

[tomaarsen]

Bug Overview

• Loading any transformers models fails if:
  • the model only has a pytorch_model.bin, and
  • you're not inside of __name__ == "__main__"

(Taken from #34966 (comment))

Details

In short: multiprocessing.Process never works when not inside of __name__ == "__main__". I recognize that most programs should be using that line, but I'd rather not force it on my users.

If one of my users loads any model that only has a pytorch_model.bin, then it'll fail, e.g.:

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("embaas/sentence-transformers-gte-base")

or

from sentence_transformers import CrossEncoder

model = CrossEncoder("cross-encoder/ms-marco-MiniLM-L-6-v2")

which internally call

from transformers import AutoModel

model = AutoModel.from_pretrained("embaas/sentence-transformers-gte-base")

or

from transformers import AutoModelForSequenceClassification

model = AutoModelForSequenceClassification.from_pretrained("cross-encoder/ms-marco-MiniLM-L-6-v2")

All of these get:

RuntimeError:
        An attempt has been made to start a new process before the
        current process has finished its bootstrapping phase.

        This probably means that you are not using fork to start your
        child processes and you have forgotten to use the proper idiom
        in the main module:

            if __name__ == '__main__':
                freeze_support()
                ...

        The "freeze_support()" line can be omitted if the program
        is not going to be frozen to produce an executable.

        To fix this issue, refer to the "Safe importing of main module"
        section in https://docs.python.org/3/library/multiprocessing.html

Edit: To prevent people experiencing errors, I've updated all cross-encoder models to safetensors. So you can't reproduce those anymore without specifying the revision.

• Tom Aarsen

[ydshieh]

On it :-) Thanks for reporting