ImportError: cannot import name 'TOKENIZERS' from 'sacrebleu.tokenizers'

[ALiang36]

❓ Questions and Help

Before asking:

1. search the issues.
2. search the docs.

What is your question?

I'm using the whisper-flanigo model and I'm having some issues calculating WER.

Code

Traceback (most recent call last):
File "whisper_decode_video.py", line 166, in
scorer = WerScorer(
File "/root/whisper-flamingo/av_hubert/fairseq/fairseq/scoring/wer.py", line 37, in init
self.tokenizer = EvaluationTokenizer(
File "/root/whisper-flamingo/av_hubert/fairseq/fairseq/scoring/tokenizer.py", line 36, in init
from sacrebleu.tokenizers import TOKENIZERS
ImportError: cannot import name 'TOKENIZERS' from 'sacrebleu.tokenizers'

What have you tried?

What's your environment?

• fairseq Version:1.0.0a0
• sacrebleu:2.4.3