Error with work.sh on CRF model

[andreasvc]

Hi there,
Running work.sh on a tfm model works fine, but with crf I get the following error:

Load checkpoint ./bert-crf-bookreviews-finetune/checkpoint-1200/pytorch_model.bin...
cached_features_file: ./data/bookreviews/cached_test_bert-base-cased_128_bookreviews
***** Running prediction *****
Evaluating:   0%|▏                                                                                                                                             | 1/800 [00:00<01:51,  7.17it/s]
Traceback (most recent call last):
  File "work.py", line 216, in <module>
    main()
  File "work.py", line 125, in main
    predict(args, model, tokenizer)
  File "work.py", line 200, in predict
    total_preds = np.append(total_preds, preds, axis=0)
  File "<__array_function__ internals>", line 6, in append
  File "/home/p286012/.local/lib/python3.7/site-packages/numpy/lib/function_base.py", line 4745, in append
    return concatenate((arr, values), axis=axis)
  File "<__array_function__ internals>", line 6, in concatenate
ValueError: all the input array dimensions for the concatenation axis must match exactly, but along dimension 1, the array at index 0 has size 12 and the array at index 1 has size 19

Any idea what's wrong? Training did work.

[lixin4ever]

I will try to reproduce this issue first in my spare time, stay tuned

[lixin4ever]

@andreasvc hello, I have fixed this issue by removing the metric computation part. You can check the latest commit for more details.