Skip to content

Commit bc05eb2

Browse files
Venkata-Durga-Raokiriti-pendyala
authored andcommitted
[ZENTORCH INFRA] Update README.md
-- Enhance README.md with six improvements Signed-off-by: Venkata Durga Rao <[email protected]> Change-Id: Ib51a422b8683a423eb95874433e2040abd599e4e Signed-off-by: Venkata Durga Rao <[email protected]> (cherry picked from commit ab9c2ed)
1 parent fa663c2 commit bc05eb2

File tree

1 file changed

+10
-43
lines changed

1 file changed

+10
-43
lines changed

README.md

Lines changed: 10 additions & 43 deletions
Original file line numberDiff line numberDiff line change
@@ -114,7 +114,7 @@ or
114114

115115
Using the release package.
116116

117-
> Download the package from AMD developer portal.
117+
> Download the package from AMD developer portal from [here](https://www.amd.com/en/developer/zendnn.html).
118118
119119
> Run the following commands to unzip the package and install the binary.
120120
@@ -130,12 +130,11 @@ zentorch/_C.cpython-3x-x86_64-linux-gnu.so : undefined symbol: <some string>,
130130
it could be due to version differences with PyTorch. Verify that you are using PyTorch version
131131
2.1.2 only.
132132
* Dependent packages 'numpy' and 'torch' will be installed by '_zentorch_' if not already present.
133-
* Torch Version should be greater than or equal to 2.0
134133

135134
## 2.2. From Source
136135
Run the following commands:
137136
```bash
138-
git clone "ssh://gerritgit/amd/ec/ZenDNN_PyTorch_Plugin"
137+
git clone https://github.com/amd/ZenDNN-pytorch-plugin.git
139138
cd ZenDNN_PyTorch_Plugin/
140139
```
141140
>Note: Repository defaults to master branch, to build the version 4.2 checkout the branch r4.2.
@@ -145,7 +144,7 @@ git checkout r4.2
145144

146145
### 2.2.1. Preparing third party repositories
147146

148-
Build setup downloads the AOCL BLIS and ZenDNN repos into `third_party` folder. It can alternatively use local copies of ZenDNN and AOCL BLIS. This is very useful for day to day development scenarios, where developer may be interested in using recent version of repositories. Build setup will switch between local and remote copies of ZenDNN , AOCL BLIS and FBGEMM with environmental variables `ZENDNN_PT_USE_LOCAL_ZENDNN` , `ZENDNN_PT_USE_LOCAL_BLIS` and `ZENDNN_PT_USE_LOCAL_FBGEMM` respectively. To use local copies of ZenDNN , AOCL BLIS or FBGEMM, set `ZENDNN_PT_USE_LOCAL_ZENDNN` , `ZENDNN_PT_USE_LOCAL_BLIS` or `ZENDNN_PT_USE_LOCAL_FBGEMM` to 1 respectively. The source repositories should be downloaded/cloned in the directory where `ZenDNN_PyTorch_Plugin` is cloned for local setting. Folder structure may look like below.
147+
Build setup downloads the ZenDNN, AOCL BLIS and FBGEMM repos into `third_party` folder. It can alternatively use local copies of ZenDNN, AOCL BLIS and FBGEMM. This is very useful for day to day development scenarios, where developer may be interested in using recent version of repositories. Build setup will switch between local and remote copies of ZenDNN, AOCL BLIS and FBGEMM with environmental variables `ZENDNN_PT_USE_LOCAL_ZENDNN` , `ZENDNN_PT_USE_LOCAL_BLIS` and `ZENDNN_PT_USE_LOCAL_FBGEMM` respectively. To use local copies of ZenDNN , AOCL BLIS or FBGEMM, set `ZENDNN_PT_USE_LOCAL_ZENDNN` , `ZENDNN_PT_USE_LOCAL_BLIS` or `ZENDNN_PT_USE_LOCAL_FBGEMM` to 1 respectively. The source repositories should be downloaded/cloned in the directory where `ZenDNN_PyTorch_Plugin` is cloned for local setting. Folder structure may look like below.
149148

150149
```
151150
<parent folder>
@@ -217,29 +216,29 @@ For CNN models, set `dynamic=False` when calling for `torch.compile` as below:
217216
```python
218217
model = torch.compile(model, backend='zentorch', dynamic=False)
219218
with torch.no_grad():
220-
output = model(input)
219+
output = model(input)
221220
```
222221

223222
For hugging face NLP models, optimize them as below:
224223
```python
225224
model = torch.compile(model, backend='zentorch')
226225
with torch.no_grad():
227-
output = model(input)
226+
output = model(input)
228227
```
229228

230229
For hugging face LLM models, optimize them as below:
231230
1. If output is generated through a call to direct `model`, optimize it as below:
232231
```python
233232
model = torch.compile(model, backend='zentorch')
234233
with torch.no_grad():
235-
output = model(input)
234+
output = model(input)
236235
```
237236

238237
2. If output is generated through a call to `model.forward`, optimize it as below:
239238
```python
240239
model.forward = torch.compile(model.forward, backend='zentorch')
241240
with torch.no_grad():
242-
output = model.forward(input)
241+
output = model.forward(input)
243242
```
244243

245244
3. If output is generated through a call to `model.generate`, optimize it as below:
@@ -248,7 +247,7 @@ with torch.no_grad():
248247
```python
249248
model.forward = torch.compile(model.forward, backend='zentorch')
250249
with torch.no_grad():
251-
output = model.generate(input)
250+
output = model.generate(input)
252251
```
253252

254253
# 4. Logging and Debugging
@@ -275,15 +274,7 @@ The default level of logs is **WARNING** for both cpp and python sources but can
275274
276275
>INFO: Since all OPs implemented in _zentorch_ are registered with torch using the TORCH_LIBRARY() and TORCH_LIBRARY_IMPL() macros in bindings, the PyTorch profiler can be used without any modifications to measure the op level performance.
277276
278-
279-
## 4.3 Saving the graph
280-
Saving of the fx graphs before and after optimization in svg format can be enabled by setting the environment variable `ZENTORCH_SAVE_GRAPH` to 1.
281-
```bash
282-
export ZENTORCH_SAVE_GRAPH=1
283-
```
284-
The graphs will be saved by the names 'native_model.svg' and 'zen_optimized_model.svg', in the parent directory of the script in which the optimize function provided by the _zentorch_ is used.
285-
286-
## 4.4 Support for `TORCH_COMPILE_DEBUG`
277+
## 4.3 Support for `TORCH_COMPILE_DEBUG`
287278
PyTorch offers a debugging toolbox that comprises a built-in stats and trace function. This functionality facilitates the display of the time spent by each compilation phase, output code, output graph visualization, and IR dump. `TORCH_COMPILE_DEBUG` invokes this debugging tool that allows for better problem-solving while troubleshooting the internal issues of TorchDynamo and TorchInductor. This functionality works for the models optimized using _zentorch_, so it can be leveraged to debug these models as well. To enable this functionality, users can either set the environment variable `TORCH_COMPILE_DEBUG=1` or specify the environment variable with the runnable file (e.g., test.py) as input.
288279
```bash
289280
# test.py contains model optimized by torch.compile with 'zentorch' as backend
@@ -296,31 +287,7 @@ zentorch v4.2.0 is supported with ZenDNN v4.2. Please see the **Tuning Guideline
296287

297288
# 6. Additional Utilities:
298289

299-
## 6.1 Disabling Inductor:
300-
301-
This feature is intended for use whenever fx_graphs generated from torch.compile needs to be compared with and without Inductor compilation.
302-
303-
disable_inductor() API takes in a boolean input to disable Inductor. Once disabled, to re-enable inductor, pass "False" to the same API.
304-
305-
```python
306-
import torch
307-
import zentorch
308-
309-
# To disable Torch Inductor
310-
zentorch.disable_inductor(True)
311-
compiled_model = torch.compile(model, backend='zentorch')
312-
output = compiled_model(input)
313-
314-
# To re-enable Torch Inductor
315-
torch._dynamo.reset()
316-
zentorch.disable_inductor(False)
317-
compiled_model = torch.compile(model, backend='zentorch')
318-
319-
```
320-
321-
Fx graphs are sent to AOT Autograd using aot_module_simplified and thus Inductor is not used at all.
322-
323-
## 6.2 _zentorch_ attributes:
290+
## 6.1 _zentorch_ attributes:
324291
To check the version of _zentorch_ use the following command:
325292

326293
```bash

0 commit comments

Comments
 (0)