Skip to content

Issues: mlcommons/inference

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

166 Open 682 Closed
166 Open 682 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Metrics definition and settings used for LLM benchmarks
#2012 opened Jan 3, 2025 by vineel96
GPT-J Huggingface validation error
#2010 opened Jan 2, 2025 by Xi0131
1
mixtral accuracy script is failing with nltk==3.9.1
#2008 opened Jan 1, 2025 by arjunsuresh
R-GAT reference implementation is giving the below warning related to numpy array being non-writable.
#2007 opened Dec 31, 2024 by arjunsuresh
1
why did the specified data precision not work for MLPerf inference?
#2006 opened Dec 31, 2024 by Bob123Yang
4
CM script failed to run harness after docker done
#1998 opened Dec 24, 2024 by Bob123Yang
2
CM script not recognizing custom nvmitten wheel on ARM CPU + Nvidia GPU setup
#1991 opened Dec 18, 2024 by ChrisHuang96
16
Update system_requirements.yml file for all the reference models documentation
#1986 opened Dec 16, 2024 by arjunsuresh
llama3: performance_sample_count update
#1979 opened Dec 13, 2024 by viraatc
1
Error: Cannot perform a '--user' install. User site-packages are not visible in the virtual environment. CM error : portable CM script failed (name = build-mlperf-inference-server-nvidia, return code = 256)
#1972 opened Dec 11, 2024 by shyambansal17
9
[Llama3] Error when multiple GPUs are used
#1957 opened Dec 3, 2024 by pgmpablo157321
[Llama3] Docker dependencies issues
#1956 opened Dec 3, 2024 by pgmpablo157321
GNN/R-GAT Inference Submission disallows sorting of the nodes. Need a mechanism to prevent and detect sorting.
#1950 opened Nov 22, 2024 by ukurkure
6
[v4.1 inference] Detected system did not match any known systems.
#1937 opened Nov 18, 2024 by loganwuw
1
mixtral-8x7b baseline reproduction : gsm8k : 72.00
#1901 opened Nov 5, 2024 by DehuaTang
1
Add github actions to test the accuracy scripts of LLM models tests
#1898 opened Nov 4, 2024 by arjunsuresh
retinanet run harness fails 'executionContext.cpp::setOptimizationProfileInternal::1328] Error Code 3:'
#1866 opened Oct 2, 2024 by stbailey001
4
[4.1 postmortem] Unit test module for inference repo postmortem 4.1
#1865 opened Oct 1, 2024 by nvzhihanj
@pgmpablo157321
4
Clarification regarding how the accuracy.txt file is generated
#1861 opened Sep 27, 2024 by arjunsuresh
7
Enable max duration in loadgen
#1857 opened Sep 26, 2024 by arjunsuresh
1
Ensure the unique IDs assigned to each SUT in the results table are not changing when result spreadsheet is regenerated.
#1856 opened Sep 26, 2024 by arjunsuresh
@arjunsuresh
4
How to run text-to-image with multi GPUs
#1849 opened Sep 14, 2024 by surbanqq
2
Nvidia 4.1 inference code is giving segmentation fault for RTX 4090 (4.0 code works fine)
#1847 opened Sep 12, 2024 by arjunsuresh
6
sudo access needed to run cm run script "get sys-utils-cm"
#1846 opened Sep 11, 2024 by writingindy
5
Fail to build the docker with rootless user
#1844 opened Sep 10, 2024 by Bob123Yang
20
ProTip! Type g p on any issue or pull request to go back to the pull request listing page.