Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bug] In the U-RaNER model, during the English task experiment, all numerical values in the evaluation except accuracy are zero. #50

Open
6 tasks done
MingDemon411 opened this issue Dec 5, 2024 · 0 comments
Open
6 tasks done

[Bug] In the U-RaNER model, during the English task experiment, all numerical values in the evaluation except accuracy are zero. #50

MingDemon411 opened this issue Dec 5, 2024 · 0 comments
Labels
bug Something isn't working

Comments

@MingDemon411
Copy link

Checklist before your report.

  • I have verified that the issue exists against the master branch of AdaSeq.
  • I have read the relevant section in the contribution guide on reporting bugs.
  • I have checked the issues list for similar or identical bug reports.
  • I have checked the pull requests list for existing proposed fixes.
  • I have checked the commit log to find out if the bug was already fixed in the master branch.

What happened?

As shown in the output of the metrics.json file, in the experiment results obtained using orig/en.yaml and wiki128/en.yaml, the values in the evaluation section are all 0 except for the accuracy value.
{"mode": "train", "epoch": 1, "iter": 50, "memory": 10690, "lr": 2e-05, "data_load_time": 0.15898, "loss": 31.84879, "iter_time": 0.32237}
{"mode": "train", "epoch": 1, "iter": 100, "memory": 10690, "lr": 2e-05, "data_load_time": 0.16546, "loss": 17.20031, "iter_time": 0.32364}
{"mode": "train", "epoch": 1, "iter": 150, "memory": 10690, "lr": 2e-05, "data_load_time": 0.16508, "loss": 15.24405, "iter_time": 0.32274}
{"mode": "train", "epoch": 1, "iter": 200, "memory": 10690, "lr": 2e-05, "data_load_time": 0.16543, "loss": 14.71369, "iter_time": 0.32328}
{"mode": "train", "epoch": 1, "iter": 250, "memory": 10690, "lr": 2e-05, "data_load_time": 0.16525, "loss": 14.26432, "iter_time": 0.32362}
{"mode": "train", "epoch": 1, "iter": 300, "memory": 10690, "lr": 2e-05, "data_load_time": 0.16528, "loss": 14.40944, "iter_time": 0.32363}
{"mode": "train", "epoch": 1, "iter": 350, "memory": 10690, "lr": 2e-05, "data_load_time": 0.16561, "loss": 13.50247, "iter_time": 0.32366}
{"mode": "train", "epoch": 1, "iter": 400, "memory": 10690, "lr": 2e-05, "data_load_time": 0.16434, "loss": 13.7237, "iter_time": 0.32354}
{"mode": "train", "epoch": 1, "iter": 450, "memory": 10690, "lr": 2e-05, "data_load_time": 0.16508, "loss": 13.41787, "iter_time": 0.32365}
{"mode": "train", "epoch": 1, "iter": 500, "memory": 10690, "lr": 2e-05, "data_load_time": 0.16564, "loss": 13.5958, "iter_time": 0.32401}
{"mode": "train", "epoch": 1, "iter": 550, "memory": 10690, "lr": 2e-05, "data_load_time": 0.16498, "loss": 13.58528, "iter_time": 0.32321}
{"mode": "train", "epoch": 1, "iter": 600, "memory": 10690, "lr": 2e-05, "data_load_time": 0.16509, "loss": 14.23441, "iter_time": 0.32296}
{"mode": "train", "epoch": 1, "iter": 650, "memory": 10690, "lr": 2e-05, "data_load_time": 0.1651, "loss": 13.92852, "iter_time": 0.32299}
{"mode": "train", "epoch": 1, "iter": 700, "memory": 10690, "lr": 2e-05, "data_load_time": 0.1651, "loss": 13.54878, "iter_time": 0.32281}
{"mode": "train", "epoch": 1, "iter": 750, "memory": 10690, "lr": 2e-05, "data_load_time": 0.16467, "loss": 13.52761, "iter_time": 0.3228}
{"mode": "train", "epoch": 1, "iter": 800, "memory": 10690, "lr": 2e-05, "data_load_time": 0.16494, "loss": 13.79905, "iter_time": 0.3231}
{"mode": "train", "epoch": 1, "iter": 850, "memory": 10690, "lr": 2e-05, "data_load_time": 0.16518, "loss": 13.30897, "iter_time": 0.32339}
{"mode": "train", "epoch": 1, "iter": 900, "memory": 10690, "lr": 2e-05, "data_load_time": 0.16534, "loss": 13.41005, "iter_time": 0.32317}
{"mode": "train", "epoch": 1, "iter": 950, "memory": 10690, "lr": 2e-05, "data_load_time": 0.16469, "loss": 13.39628, "iter_time": 0.32238}
{"mode": "train", "epoch": 1, "iter": 1000, "memory": 10690, "lr": 2e-05, "data_load_time": 0.16478, "loss": 13.47804, "iter_time": 0.32273}
{"mode": "eval", "epoch": 1, "iter": 28, "memory": 10690, "lr": 2e-05, "evaluation/precision": 0.0, "evaluation/recall": 0.0, "evaluation/f1": 0.0, "evaluation/accuracy": 0.95031, "evaluation/macro-f1": 0.0}
{"mode": "train", "epoch": 2, "iter": 50, "memory": 10690, "lr": 1.9333333333333333e-05, "data_load_time": 0.16135, "loss": 13.26254, "iter_time": 0.32012}
{"mode": "train", "epoch": 2, "iter": 100, "memory": 10690, "lr": 1.9333333333333333e-05, "data_load_time": 0.16502, "loss": 21.83714, "iter_time": 0.32395}
{"mode": "train", "epoch": 2, "iter": 150, "memory": 10690, "lr": 1.9333333333333333e-05, "data_load_time": 0.16488, "loss": 13.51608, "iter_time": 0.3242}
{"mode": "train", "epoch": 2, "iter": 200, "memory": 10690, "lr": 1.9333333333333333e-05, "data_load_time": 0.16517, "loss": 13.49161, "iter_time": 0.32432}
{"mode": "train", "epoch": 2, "iter": 250, "memory": 10690, "lr": 1.9333333333333333e-05, "data_load_time": 0.16505, "loss": 13.33935, "iter_time": 0.32441}
{"mode": "train", "epoch": 2, "iter": 300, "memory": 10690, "lr": 1.9333333333333333e-05, "data_load_time": 0.16515, "loss": 13.52852, "iter_time": 0.32432}
{"mode": "train", "epoch": 2, "iter": 350, "memory": 10690, "lr": 1.9333333333333333e-05, "data_load_time": 0.16492, "loss": 13.47702, "iter_time": 0.32384}
{"mode": "train", "epoch": 2, "iter": 400, "memory": 10690, "lr": 1.9333333333333333e-05, "data_load_time": 0.16502, "loss": 13.65968, "iter_time": 0.32422}
{"mode": "train", "epoch": 2, "iter": 450, "memory": 10690, "lr": 1.9333333333333333e-05, "data_load_time": 0.1652, "loss": 13.37312, "iter_time": 0.32449}
{"mode": "train", "epoch": 2, "iter": 500, "memory": 10690, "lr": 1.9333333333333333e-05, "data_load_time": 0.1655, "loss": 13.19569, "iter_time": 0.32456}
{"mode": "train", "epoch": 2, "iter": 550, "memory": 10690, "lr": 1.9333333333333333e-05, "data_load_time": 0.16538, "loss": 13.10487, "iter_time": 0.32477}
{"mode": "train", "epoch": 2, "iter": 600, "memory": 10690, "lr": 1.9333333333333333e-05, "data_load_time": 0.16498, "loss": 13.36017, "iter_time": 0.32496}
{"mode": "train", "epoch": 2, "iter": 650, "memory": 10690, "lr": 1.9333333333333333e-05, "data_load_time": 0.16583, "loss": 13.02584, "iter_time": 0.32549}
{"mode": "train", "epoch": 2, "iter": 700, "memory": 10690, "lr": 1.9333333333333333e-05, "data_load_time": 0.16528, "loss": 13.26082, "iter_time": 0.32615}
{"mode": "train", "epoch": 2, "iter": 750, "memory": 10690, "lr": 1.9333333333333333e-05, "data_load_time": 0.16469, "loss": 13.25028, "iter_time": 0.32475}
{"mode": "train", "epoch": 2, "iter": 800, "memory": 10690, "lr": 1.9333333333333333e-05, "data_load_time": 0.16558, "loss": 12.9614, "iter_time": 0.32518}
{"mode": "train", "epoch": 2, "iter": 850, "memory": 10690, "lr": 1.9333333333333333e-05, "data_load_time": 0.16517, "loss": 13.15059, "iter_time": 0.3249}
{"mode": "train", "epoch": 2, "iter": 900, "memory": 10690, "lr": 1.9333333333333333e-05, "data_load_time": 0.16537, "loss": 13.3023, "iter_time": 0.32468}
{"mode": "train", "epoch": 2, "iter": 950, "memory": 10690, "lr": 1.9333333333333333e-05, "data_load_time": 0.16513, "loss": 13.32816, "iter_time": 0.32409}
{"mode": "train", "epoch": 2, "iter": 1000, "memory": 10690, "lr": 1.9333333333333333e-05, "data_load_time": 0.16521, "loss": 13.29799, "iter_time": 0.3247}
{"mode": "eval", "epoch": 2, "iter": 28, "memory": 10690, "lr": 1.9333333333333333e-05, "evaluation/precision": 0.0, "evaluation/recall": 0.0, "evaluation/f1": 0.0, "evaluation/accuracy": 0.95031, "evaluation/macro-f1": 0.0}
{"mode": "train", "epoch": 3, "iter": 50, "memory": 10690, "lr": 1.866666666666667e-05, "data_load_time": 0.1618, "loss": 13.33689, "iter_time": 0.32133}
{"mode": "train", "epoch": 3, "iter": 100, "memory": 10690, "lr": 1.866666666666667e-05, "data_load_time": 0.1651, "loss": 13.3437, "iter_time": 0.32432}
{"mode": "train", "epoch": 3, "iter": 150, "memory": 10690, "lr": 1.866666666666667e-05, "data_load_time": 0.16519, "loss": 13.60922, "iter_time": 0.32457}
{"mode": "train", "epoch": 3, "iter": 200, "memory": 10690, "lr": 1.866666666666667e-05, "data_load_time": 0.16493, "loss": 12.89893, "iter_time": 0.32454}
{"mode": "train", "epoch": 3, "iter": 250, "memory": 10690, "lr": 1.866666666666667e-05, "data_load_time": 0.16562, "loss": 13.43225, "iter_time": 0.32478}
{"mode": "train", "epoch": 3, "iter": 300, "memory": 10690, "lr": 1.866666666666667e-05, "data_load_time": 0.1647, "loss": 13.53928, "iter_time": 0.32507}
{"mode": "train", "epoch": 3, "iter": 350, "memory": 10690, "lr": 1.866666666666667e-05, "data_load_time": 0.16536, "loss": 13.28629, "iter_time": 0.32473}
{"mode": "train", "epoch": 3, "iter": 400, "memory": 10690, "lr": 1.866666666666667e-05, "data_load_time": 0.16487, "loss": 13.26797, "iter_time": 0.32435}
{"mode": "train", "epoch": 3, "iter": 450, "memory": 10690, "lr": 1.866666666666667e-05, "data_load_time": 0.16466, "loss": 15.50956, "iter_time": 0.32408}
{"mode": "train", "epoch": 3, "iter": 500, "memory": 10690, "lr": 1.866666666666667e-05, "data_load_time": 0.16523, "loss": 13.55456, "iter_time": 0.32403}
{"mode": "train", "epoch": 3, "iter": 550, "memory": 10690, "lr": 1.866666666666667e-05, "data_load_time": 0.16503, "loss": 13.24142, "iter_time": 0.32415}
{"mode": "train", "epoch": 3, "iter": 600, "memory": 10690, "lr": 1.866666666666667e-05, "data_load_time": 0.1653, "loss": 13.08147, "iter_time": 0.32432}
{"mode": "train", "epoch": 3, "iter": 650, "memory": 10690, "lr": 1.866666666666667e-05, "data_load_time": 0.16478, "loss": 13.11922, "iter_time": 0.32419}
{"mode": "train", "epoch": 3, "iter": 700, "memory": 10690, "lr": 1.866666666666667e-05, "data_load_time": 0.1652, "loss": 13.1907, "iter_time": 0.32473}
{"mode": "train", "epoch": 3, "iter": 750, "memory": 10690, "lr": 1.866666666666667e-05, "data_load_time": 0.16507, "loss": 12.87156, "iter_time": 0.32461}
{"mode": "train", "epoch": 3, "iter": 800, "memory": 10690, "lr": 1.866666666666667e-05, "data_load_time": 0.16511, "loss": 13.3374, "iter_time": 0.3252}
{"mode": "train", "epoch": 3, "iter": 850, "memory": 10690, "lr": 1.866666666666667e-05, "data_load_time": 0.16524, "loss": 13.27545, "iter_time": 0.32464}
{"mode": "train", "epoch": 3, "iter": 900, "memory": 10690, "lr": 1.866666666666667e-05, "data_load_time": 0.16482, "loss": 13.15263, "iter_time": 0.32428}
{"mode": "train", "epoch": 3, "iter": 950, "memory": 10690, "lr": 1.866666666666667e-05, "data_load_time": 0.16561, "loss": 13.11274, "iter_time": 0.32474}
{"mode": "train", "epoch": 3, "iter": 1000, "memory": 10690, "lr": 1.866666666666667e-05, "data_load_time": 0.16513, "loss": 13.25312, "iter_time": 0.32444}
{"mode": "eval", "epoch": 3, "iter": 28, "memory": 10690, "lr": 1.866666666666667e-05, "evaluation/precision": 0.0, "evaluation/recall": 0.0, "evaluation/f1": 0.0, "evaluation/accuracy": 0.95031, "evaluation/macro-f1": 0.0}
{"mode": "train", "epoch": 4, "iter": 50, "memory": 10690, "lr": 1.8000000000000004e-05, "data_load_time": 0.16198, "loss": 13.36875, "iter_time": 0.32116}
{"mode": "train", "epoch": 4, "iter": 100, "memory": 10690, "lr": 1.8000000000000004e-05, "data_load_time": 0.16533, "loss": 13.49711, "iter_time": 0.32458}
{"mode": "train", "epoch": 4, "iter": 150, "memory": 10690, "lr": 1.8000000000000004e-05, "data_load_time": 0.16478, "loss": 12.96431, "iter_time": 0.32413}
{"mode": "train", "epoch": 4, "iter": 200, "memory": 10690, "lr": 1.8000000000000004e-05, "data_load_time": 0.16538, "loss": 12.81225, "iter_time": 0.3251}
{"mode": "train", "epoch": 4, "iter": 250, "memory": 10690, "lr": 1.8000000000000004e-05, "data_load_time": 0.1653, "loss": 13.39281, "iter_time": 0.32448}
{"mode": "train", "epoch": 4, "iter": 300, "memory": 10690, "lr": 1.8000000000000004e-05, "data_load_time": 0.16477, "loss": 12.78714, "iter_time": 0.32427}
{"mode": "train", "epoch": 4, "iter": 350, "memory": 10690, "lr": 1.8000000000000004e-05, "data_load_time": 0.16572, "loss": 13.27104, "iter_time": 0.32551}
{"mode": "train", "epoch": 4, "iter": 400, "memory": 10690, "lr": 1.8000000000000004e-05, "data_load_time": 0.1649, "loss": 13.23851, "iter_time": 0.32486}
{"mode": "train", "epoch": 4, "iter": 450, "memory": 10690, "lr": 1.8000000000000004e-05, "data_load_time": 0.16554, "loss": 13.2158, "iter_time": 0.32479}
{"mode": "train", "epoch": 4, "iter": 500, "memory": 10690, "lr": 1.8000000000000004e-05, "data_load_time": 0.1652, "loss": 13.42676, "iter_time": 0.3249}
{"mode": "train", "epoch": 4, "iter": 550, "memory": 10692, "lr": 1.8000000000000004e-05, "data_load_time": 0.16462, "loss": 13.35568, "iter_time": 0.32357}
{"mode": "train", "epoch": 4, "iter": 600, "memory": 10692, "lr": 1.8000000000000004e-05, "data_load_time": 0.16524, "loss": 13.07017, "iter_time": 0.32451}
{"mode": "train", "epoch": 4, "iter": 650, "memory": 10692, "lr": 1.8000000000000004e-05, "data_load_time": 0.16434, "loss": 12.89934, "iter_time": 0.3249}
{"mode": "train", "epoch": 4, "iter": 700, "memory": 10692, "lr": 1.8000000000000004e-05, "data_load_time": 0.16579, "loss": 13.22372, "iter_time": 0.3252}
{"mode": "train", "epoch": 4, "iter": 750, "memory": 10692, "lr": 1.8000000000000004e-05, "data_load_time": 0.16559, "loss": 13.31236, "iter_time": 0.3251}
{"mode": "train", "epoch": 4, "iter": 800, "memory": 10692, "lr": 1.8000000000000004e-05, "data_load_time": 0.16547, "loss": 13.47201, "iter_time": 0.32492}
{"mode": "train", "epoch": 4, "iter": 850, "memory": 10692, "lr": 1.8000000000000004e-05, "data_load_time": 0.16525, "loss": 13.14963, "iter_time": 0.32459}
{"mode": "train", "epoch": 4, "iter": 900, "memory": 10692, "lr": 1.8000000000000004e-05, "data_load_time": 0.16559, "loss": 13.28933, "iter_time": 0.32485}
{"mode": "train", "epoch": 4, "iter": 950, "memory": 10692, "lr": 1.8000000000000004e-05, "data_load_time": 0.1651, "loss": 13.4793, "iter_time": 0.32459}
{"mode": "train", "epoch": 4, "iter": 1000, "memory": 10692, "lr": 1.8000000000000004e-05, "data_load_time": 0.16534, "loss": 13.30867, "iter_time": 0.3246}
{"mode": "eval", "epoch": 4, "iter": 28, "memory": 10692, "lr": 1.8000000000000004e-05, "evaluation/precision": 0.0, "evaluation/recall": 0.0, "evaluation/f1": 0.0, "evaluation/accuracy": 0.95031, "evaluation/macro-f1": 0.0}
{"mode": "train", "epoch": 5, "iter": 50, "memory": 10692, "lr": 1.733333333333334e-05, "data_load_time": 0.16183, "loss": 13.13687, "iter_time": 0.32122}
{"mode": "train", "epoch": 5, "iter": 100, "memory": 10692, "lr": 1.733333333333334e-05, "data_load_time": 0.16519, "loss": 13.21697, "iter_time": 0.32498}
{"mode": "train", "epoch": 5, "iter": 150, "memory": 10692, "lr": 1.733333333333334e-05, "data_load_time": 0.16565, "loss": 13.46907, "iter_time": 0.32502}
{"mode": "train", "epoch": 5, "iter": 200, "memory": 10692, "lr": 1.733333333333334e-05, "data_load_time": 0.16547, "loss": 13.33826, "iter_time": 0.32461}
{"mode": "train", "epoch": 5, "iter": 250, "memory": 10692, "lr": 1.733333333333334e-05, "data_load_time": 0.1643, "loss": 12.97871, "iter_time": 0.32509}
{"mode": "train", "epoch": 5, "iter": 300, "memory": 10692, "lr": 1.733333333333334e-05, "data_load_time": 0.16539, "loss": 13.39593, "iter_time": 0.32482}

Python traceback

show/hide

Operating system

Windows10 22H2;Ubuntu-20.04

Python version

3.8.19

Output of pip freeze

show/hide 

addict==2.4.0
aiohappyeyeballs==2.3.5
aiohttp==3.10.3
aiosignal==1.3.1
aliyun-python-sdk-core==2.15.1
aliyun-python-sdk-kms==2.16.3
async-timeout==4.0.3
attrs==24.2.0
certifi==2024.7.4
cffi==1.17.0
charset-normalizer==3.3.2
colorama==0.4.6
crcmod==1.7
cryptography==43.0.0
datasets==2.20.0
dill==0.3.8
filelock==3.15.4
frozenlist==1.4.1
fsspec==2024.5.0
huggingface-hub==0.24.5
idna==3.7
Jinja2==3.1.4
jmespath==0.10.0
joblib==1.4.2
MarkupSafe==2.1.5
modelscope==1.17.1
mpmath==1.3.0
multidict==6.0.5
multiprocess==0.70.16
networkx==3.1
numpy==1.24.4
oss2==2.18.6
packaging==24.1
pandas==2.0.3
pillow==10.4.0
pyarrow==17.0.0
pyarrow-hotfix==0.6
pycparser==2.22
pycryptodome==3.20.0
python-dateutil==2.9.0.post0
pytz==2024.1
PyYAML==6.0.2
regex==2024.7.24
requests==2.32.3
safetensors==0.4.4
scikit-learn==1.3.2
scipy==1.10.1
seqeval==1.2.2
simplejson==3.19.2
six==1.16.0
sortedcontainers==2.4.0
sympy==1.13.2
threadpoolctl==3.5.0
tokenizers==0.19.1
torch==2.4.0+cu124
torchaudio==2.4.0+cu124
torchvision==0.19.0+cu124
tqdm==4.66.5
transformers==4.44.0
typing_extensions==4.12.2
tzdata==2024.1
urllib3==2.2.2
xxhash==3.4.1
yarl==1.9.4

How to reproduce

show/hide 

python .\scripts\train.py -c examples/SemEval2023_MultiCoNER_II/configs/orig/en.yaml
python .\scripts\train.py -c examples/SemEval2023_MultiCoNER_II/configs/wiki128/en.yaml

Code of Conduct

  • I agree to follow this project's Code of Conduct
@MingDemon411 MingDemon411 added the bug Something isn't working label Dec 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@MingDemon411