Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add torchbench exports and benchmarks. #845

Open
wants to merge 20 commits into
base: main
Choose a base branch
from
Open

Add torchbench exports and benchmarks. #845

wants to merge 20 commits into from

Conversation

monorimet
Copy link
Contributor

A README.md is included in the new directory for instructions on exporting the models. Not all are functional and the process requires a clone of the pytorch source code, currently, as well as a Linux distro for pytorch+rocm support (the torchbench modeling code relies on CUDA apis)

Once the exports are covered we can set up a simple benchmarking suite that runs the compiled .vmfbs and records benchmark results.

@monorimet monorimet changed the title (WIP) Add torchbench exports and benchmarks. Add torchbench exports and benchmarks. Sep 20, 2024
@monorimet monorimet requested a review from zjgarvey September 20, 2024 21:56
@monorimet monorimet marked this pull request as ready for review September 20, 2024 21:56
@monorimet
Copy link
Contributor Author

Notes on current state:

  • Need to separate out runtime functionality from export script
  • Start targeting torchbench integration with improvements to implementation, classing, statistics reporting, etc..
  • Integrate with SHARK-TestSuite CI
  • Work out upload/fetch of MLIR and weights so that this process can be used with containers running pytorch 2.3.0 without workarounds

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants