Input size

[maryamag85]

what is the max input size that distilled model can handle? if the original model does not have a limit on the input size, does it also preserve by distillation?

[stephantul]

Hello @maryamag85 ,

I'm assuming you are talking about sequence length. The sequence length of our models is unbounded. The default, however, is 512, and can be controlled by setting max_length when encoding. Set it None to always encode the full sequence.

Stéphan

[shreyaskarnik]

When using StaticModelForClassification how do I set max_length ?

[stephantul]

Hello @shreyaskarnik

It is a parameter of the predict function.