🧹 Remove max_batch_tokens, num_blocks and block_size from generation kwargs

[qgallouedec]

These parameters are now automatically computed, see huggingface/transformers#40426

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[remi-or]

As a reminder, while max_batch_tokens and num_blocks are infered, block_size isn't and defaults to 32. You might want to test out how the change from 128 to 32 will impact you 🙂
Not really familiar with trl so I can't really review this, sorry!

[qgallouedec]

thanks for the feedback @remi-or!

change from 128 to 32 will impact you 🙂

will do!

[albertvillanova]

LGTM if the test of the change in block_size does not have a negative impact. Otherwise, we should keep the original 128 value.

[qgallouedec]

Tried with 0.5B and 1.7B, no difference between block_size=32 and 128👍