Skip to content

Issues: open-compass/opencompass

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

226 Open 347 Closed
226 Open 347 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

[Bug] torch.OutOfMemoryError
#1773 opened Dec 23, 2024 by QingChengLineOne
2 tasks done
@bittersweet1999
[Bug] 推理完成了，但是因为插件版本，在做测评的时候报错了，但是预测的哪些文件都在，怎么重新继续做预测？还有就是运行的时候默认好像是多有的数据集都会测试
#1761 opened Dec 14, 2024 by Lelege0
2 tasks done
@tonysy
1
[Bug] commonsenseqa_gen_c946f2中的commonsenseqa_datasets数据集报错 TypeError: 'ListWrapper' object is not iterable
#1760 opened Dec 14, 2024 by hanbaoergogo
2 tasks done
@tonysy
1
[Bug] cannot use GPU when conduct evaluation
#1759 opened Dec 12, 2024 by takagi97
2 tasks done
@acylam
[Bug] 提示新写的Dataset未注册
#1756 opened Dec 11, 2024 by ezeli
2 tasks done
@MaiziXiao
[Bug] 无法多卡并行评测数据
#1755 opened Dec 11, 2024 by luhairong11
2 tasks done
@MaiziXiao
1
[Bug] 自定义数据集，模型进行评测结果为0分
#1752 opened Dec 11, 2024 by lomoonmoonbird
2 tasks done
@MaiziXiao
4
[Feature] Evaluate multiple metrics simultaneously
#1748 opened Dec 9, 2024 by ezeli
1 task
@bittersweet1999
2
[Bug] NPU 运行报错 core dump
#1747 opened Dec 9, 2024 by SefaZeng
2 tasks done
@bittersweet1999
1
[Bug] 自定义模型评测配置
#1746 opened Dec 9, 2024 by ucassqs
2 tasks done
@bittersweet1999
HumanEvalPlusEvaluator使用的evalplus版本是多少的
#1743 opened Dec 9, 2024 by wanggei
1 task
@bittersweet1999
[Bug] Unsupported operand types in OpenAISDK
#1736 opened Dec 4, 2024 by maanasharma5
2 tasks done
@MaiziXiao
1
[Bug] 微调过的Qwen2.5-7b输出全是感叹号/The output of fine-tuning Qwen2.5-7b is '！！！！！！'
#1732 opened Dec 3, 2024 by noname342
2 tasks done
@liushz
1
[Bug] 在无gpu的机器上执行case，运行时报错数据集未注册（其实已经注册）
#1725 opened Nov 29, 2024 by Caeser-SONG
2 tasks done
@tonysy
1
[Bug] windows下数据集的位置以及检测结果为0.0
#1723 opened Nov 29, 2024 by Dbgsaoge
2 tasks done
@tonysy
您好，请问L-Eval的主观题最终得分是使用rougeLsum这个分数吗？还有就是L-Eval数据集缺少了codeU和sci_fi有相关的评测配置文件么？
#1721 opened Nov 28, 2024 by 13416157913
2 tasks done
@acylam
[Feature] 请问使用API评测如何支持自定义数据集？
#1718 opened Nov 28, 2024 by Jimmy-L99
1 task
@acylam
1
[Bug] 测评结果为空，求助
#1716 opened Nov 26, 2024 by Jimmy-L99
2 tasks done
@liushz
3
[Bug] strategyqa answer extraction error
#1715 opened Nov 25, 2024 by Linzwcs
2 tasks done
@bittersweet1999
[Bug] stop_at_stop_token 删除了生成的方法体导致没有评估结果
#1710 opened Nov 23, 2024 by 886gb
2 tasks done
@tonysy
[Bug] DO NOT Use relative import
#1703 opened Nov 20, 2024 by jinmingyi1998
2 tasks done
@MaiziXiao
2
[Feature] HumanEvalX use Chat Mode as humaneval_openai_sample_evals_gen_159614
#1687 opened Nov 13, 2024 by tonysy
1 task done
@MaiziXiao
[Bug] The evaluation of configurations requiring a TopKRetriever, such as for flores_datasets, failed with a KeyError: 'metadata'.
#1686 opened Nov 13, 2024 by DespairL
2 tasks done
[Bug] TypeError: Expected a datasets.Dataset or a datasets.DatasetDict object, but got {'train': <modelscope.msdatasets.ms_dataset.MsDataset object at xxx>, 'test': <modelscope.msdatasets.ms_dataset.MsDataset object at xxx>}
#1678 opened Nov 12, 2024 by MrWiffer
2 tasks done
[Bug] v0.3.5版本评测Qwen/Qwen2.5-72B得分显著下降
#1675 opened Nov 11, 2024 by guoshengCS
2 tasks done
@MaiziXiao @bittersweet1999
10
ProTip! Exclude everything labeled bug with -label:bug.