Problems for the training process

[AndyFrancesco29]

Hi, when I try to reproduce the training process, I have some problems.

I only download the goalstep benchmark from Ego4D and try to run the script scripts/ego4d/narration/live1.sh to see the evaluation process. Here is the step I have done:

1. Download the goalstep benchmark of Ego4D and sample and encode the videos into features
2. change the benchmarks_with_keys in dataloader Ego4DNarrationStream to goalstep only:
   benchmarks_with_keys = { 'goalstep': 'videos' }
3. change train.py to evaluate.py in the script.
4. use the annotation from the Ego4D dataset: goalstep_val.json (not the goalstep_livechat_trainval_filtered_21k.json) as used for narration.

However, I got errors like this:

    data = fetcher.fetch(index)  
  File "/home/jingfeix/miniconda3/envs/videollm/lib/python3.10/site-packages/torch/utils/data/_utils/fetch.py", line 54, in fetch  
    return self.collate_fn(data)  
  File "/home/jingfeix/miniconda3/envs/videollm/lib/python3.10/site-packages/transformers/trainer_utils.py", line 841, in __call__  
    return self.data_collator(features)  
  File "/aiot-nvme-15T-x2-hk01/jingfeix/LLM/videollm-online/data/data_collator.py", line 17, in data_collator  
    stop = torch.nonzero(offset_mapping[:,0] == learn_r.stop).item()  
RuntimeError: a Tensor with 0 elements cannot be converted to Scalar```



It seems there is something wrong with data collate function. Did I use any data wrong? Thank you!

[AndyFrancesco29]

I have checked the code in the data/stream.py and data/data_collator.py, the learn_ranges from the StreamMixIn cannot match the offset mapping from the tokenizer in the collator. Is there anything wrong with that?