Remove attn mask patching

[baskrahmer]

What does this PR do?

Removes attention mask patching for specific models when doing an ONNX export.

Picked this up but I am not sure about:

1. Whether to log a warning or raise an error upon exporting with sequence_length=1 and also not what the exact scope is for such an action. Right now it raises a warning for any models that have tasks prefixed with text-generation. Maybe this should be more specific.
2. Whether to add a warning/error for previously exported models with such a configuration.

Fixes #1461

[fxmarty]

Not sure if this context is given anywhere in the code base, but anyway:

That's great @baskrahmer, thank you for the simplification! For context, @echarlaix introduced a simplification for the ONNX export of decoder-only models in #1257, where a single ONNX without subgraphs can be used, handling both prefill and decode steps (contrary to the previous decoder_model_merged.onnx) that handled both with subgraphs.

However, to do that, the traced model during the ONNX export needs to encompass the causal mask generation. Unfortunately, some architectures as llama https://github.com/huggingface/transformers/blob/fc142bd775ae4639f80a8b0085a5df33bd2853ce/src/transformers/models/llama/modeling_llama.py#L139-L147. So to export models with the new structure, we either need to patch the models to remove this controlflow (what was done), or simply use sequence_length>1 to go into the controlflow during tracing. That is what I was suggesting in #1461 for simplification purpose.

[baskrahmer]

@fxmarty thanks for the context. Sounds sensible :)

[fxmarty]

I'll test a bit more later!

[fxmarty]

Hi @baskrahmer, following huggingface/transformers#27086 quite a few _make_causal, _prepare_attention_mask functions were removed and moved elsewhere. So this PR should fix the issue. I believe you can also remove the patching of _prepare_attn_mask and _make_causal_mask for Falcon (that don't exist anymore).

[fxmarty]

see #1495

[baskrahmer]

I believe you can also remove the patching of _prepare_attn_mask and _make_causal_mask for Falcon (that don't exist anymore).

Not sure if I follow - you mean also removing this?

[fxmarty]

Hi @baskrahmer, sorry for the late reply. I meant this:

optimum/optimum/exporters/onnx/model_patcher.py

Lines 407 to 411 in 1aee8ff

	# In order to use a single decoder, we need to patch the _prepare_attn_mask function to behave independently of the sequence length.
	if isinstance(self._model, FalconModel):
	self._model._prepare_attn_mask = _falcon_prepare_attn_mask
	else:
	self._model.transformer._prepare_attn_mask = _falcon_prepare_attn_mask

EDIT: Nevermind, you already removed it!

I'm preparing a release for today in sync with Transformers release and we'll need this PR in, for the interest of time I'll be pushing to your branch to get this PR merged, apology in advance about that!

[baskrahmer]

@fxmarty thanks for the reply. All good, you are definitely more in the details here so feel free to change anything :)

[fxmarty]

@baskrahmer #1509 is merged based off your branch (I could not push to your branch), sorry for the hurry and thank you for your contribution!