Skip to content

Issues: opea-project/GenAIEval

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

24 Open 20 Closed
24 Open 20 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

[Feature] Supports Infinity Request Rate for Serving feature New feature or request
#214 opened Dec 13, 2024 by wangkl2
Publish CRAG benchmark results feature New feature or request
#212 opened Dec 6, 2024 by minmin-intel v1.2
[benchmark] Enhance benchmark.py
#209 opened Nov 29, 2024 by poussa
[Bug] Benchmarking script failing to capture logs in EKS
#207 opened Nov 27, 2024 by srinarayan-srikanthan
@gavinlichn
2
[Benchmark] vectordb scaling benchmark and validation dev
#202 opened Nov 12, 2024 by joshuayao v1.2
[Feature] Long context benchmark enhancement feature New feature or request
#201 opened Nov 12, 2024 by joshuayao v1.2
@lkk12014402 @XinyaoWa
[Feature] k8s conformance test feature New feature or request
#200 opened Nov 12, 2024 by joshuayao v1.2
@yongfengdu
Setup OpenSSF Scorecard (Current Score: 4.4) dev
#197 opened Nov 9, 2024 by zixuan-x
1 task
[Bug] Performance benchmarks for FaqGen / DocSum are calculated incorrectly (we see negative numbers and very high values) bug Something isn't working
#189 opened Nov 5, 2024 by amikoai
@lvliang-intel @wangkl2
milvus/other vectordb benchmark
#159 opened Oct 14, 2024 by kevinintel
@lkk12014402 @XinyaoWa
Grafana dashboard error accessing data source documentation Improvements or additions to documentation
#149 opened Oct 1, 2024 by rowenaal
Help Needed: Collect benchmark chatQnA on your favorite CSP (AWS, GCP, Azure) Hacktoberfest OPEAHack Available for Hackathon use
#130 opened Sep 17, 2024 by Padma3a
benchmark in v0.9 stopped supporting k3s Hacktoberfest OPEAHack Available for Hackathon use
#123 opened Sep 11, 2024 by ksandowi
4
No guide for using benchmark tool behind proxy documentation Improvements or additions to documentation
#101 opened Sep 4, 2024 by joshuayao
Missing dependencies in evals/benchmark/stresscli/requirements.txt Hacktoberfest OPEAHack Available for Hackathon use
#97 opened Sep 3, 2024 by joshuayao
Missing initial instructions in evals/benchmark/README (and others) documentation Improvements or additions to documentation Hacktoberfest OPEAHack Available for Hackathon use
#92 opened Aug 30, 2024 by dbkinder
2
Restructure genAI Eval to address evaluation of multiple categories of metrics feature New feature or request
#75 opened Aug 21, 2024 by Padmaapparao
1
lm-as-judge acc on chinese
#72 opened Aug 20, 2024 by kevinintel
@lkk12014402
E2E RAG acc on english dataset
#71 opened Aug 20, 2024 by kevinintel
@lkk12014402
reranking acc
#70 opened Aug 20, 2024 by kevinintel
@lkk12014402
Retrieval relevance
#67 opened Aug 19, 2024 by kevinintel
@lkk12014402
doc: cruft at top of /evals/metrics/bleu/README.md documentation Improvements or additions to documentation Hacktoberfest OPEAHack Available for Hackathon use
#63 opened Aug 14, 2024 by dbkinder
1
Dynamic tuning on Resource management through K8s
#58 opened Jul 30, 2024 by kevinintel
1
K8s Resource Management feature New feature or request
#32 opened Jun 17, 2024 by kevinintel
4
ProTip! Exclude everything labeled bug with -label:bug.