Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle missing data in console and csv exporter #201

Merged
merged 2 commits into from
Nov 27, 2024
Merged

Conversation

nv-hwoo
Copy link
Contributor

@nv-hwoo nv-hwoo commented Nov 27, 2024

When some statistics are missing (for any reason), handle it by displaying "N/A" instead of throwing an error to the user.

The console output will contain "N/A" if some values are missing
image

The CSV output will also contain "N/A"

Metric,avg,min,max,p99,p95,p90,p75,p50,p25
Request Latency (ms),205.80,205.40,207.50,207.37,206.84,206.17,205.76,205.58,205.48
Output Sequence Length,N/A,11.00,13.00,13.00,13.00,13.00,12.00,11.00,11.00
Input Sequence Length,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A,N/A

Metric,Value
Output Token Throughput (per sec),56.27
Request Throughput (per sec),4.85

The JSON exporter does not require any handling since it directly takes the statistics dictionary and saves it without any modification. So any missing data won't be in the final JSON file.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For some reason, pre-commit was complaining about the order of the includes so I had to change it.

@dyastremsky
Copy link
Contributor

Great work, Hyunjae! Looks good.

@nv-hwoo nv-hwoo merged commit 1391179 into main Nov 27, 2024
6 of 7 checks passed
@nv-hwoo nv-hwoo deleted the hwoo-fix-console branch November 27, 2024 21:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants