[benchmarks] Add docs file. #8023
Conversation
|
|
||
| **PyTorch/XLA Metrics:** (repeat-specific) the flag `--dump-pytorch-xla-metrics` creates a | ||
| new file, dumping PyTorch/XLA metrics, such as graph compiling and execution information. | ||
|
|
There was a problem hiding this comment.
Can we have a section on Nightly CI runs - TPU and GPU
There was a problem hiding this comment.
Sure. But I don't really have any information on those nightly CI runs. What should it contain?
There was a problem hiding this comment.
@zpcore can you please help Yukio with this section?
There was a problem hiding this comment.
additional: ideally, we also want to run verifier as part of our tests
|
|
||
| **PyTorch/XLA Metrics:** (repeat-specific) the flag `--dump-pytorch-xla-metrics` creates a | ||
| new file, dumping PyTorch/XLA metrics, such as graph compiling and execution information. | ||
|
|
There was a problem hiding this comment.
Can we include a troubleshooting section? this should include the pitfalls we ran into as well as approaches to debug the benchmark efficiently.
There was a problem hiding this comment.
@miladm I don't think it is appropriate to document any resolved issue here. If there are any features that can be used to debug different performance issues we should absolutely make sure they are captured.
|
This PR introduces a docs/torchbench.md file. It explains how to use the benchmarking scripts for running, troubleshooting, and debugging the models inside Torchbench.