This directory contains Jupyter notebooks that demonstrate model compilation and inference using PyTorch Neuron for a variety of popular deep learning models. These samples can be run on AWS Inferentia (inf1 instances) using Amazon SageMaker or Amazon EC2.
For each sample you will also find additional information such as the model type, configuration used to compile the model, framework version, and a link to the original model implementation.
The following samples are available:
| Model Name | Model Type | Input Shape | NeuronSDK Version | Framework / Version | Original Implementation |
|---|---|---|---|---|---|
| BERT-base | NLP | max_length=128 | 1.10.1.2.2.0.0 | Pytorch 1.10.2 | link |
| BERT-large | NLP | max_length=128 | 1.10.1.2.2.0.0 | Pytorch 1.10.2 | link |
| CRAFT | CV - Text Detection | 1,3,800,800 - max_length=32 | 1.10.2.2.3.0.0 | Pytorch 1.10.2 | link |
| EfficientNet | CV - Image Classification | 1,3,224,224 | 1.10.1.2.2.0.0 | Pytorch 1.10.1 | link |
| GFL | CV - Object Detection | 1,3,800,1216 | 1.10.2.2.3.0.0 | Pytorch 1.10.2 | link |
| HRNet | CV - Pose Estimation | 1,3,384,288 | 1.10.2.2.3.0.0 | Pytorch 1.10.2 | link |
| MarianNMT | NLP | max_length=32 | 1.7.* | Pytorch 1.7 | link |
| R-CNN | CV - Image Classification, Detection, and Segmentation | 1,3,800,800 | 1.11.0.2.5.2.0 | Pytorch 1.11.0 | link |
| ResNet (18,34,50,101,152) | CV - Image Classification | 1,3,224,224 | 1.10.1.2.2.0.0 | Pytorch 1.10.1 | link |
| ResNetX | CV - Image Classification | 1,3,224,224 | 1.10.1.2.2.0.0 | Pytorch 1.10.1 | link |
| Roberta-base | NLP | max_length=128 | 1.10.1.2.2.0.0 | Pytorch 1.10.2 | link |
| SSD (SSD300-VGG16) | CV - Object detection | 1,3,300,300 | 1.10.2.2.3.0.0 | Pytorch 1.10.2 | link |
| TrOCR | CV - OCR | 1,3,384,384 | 1.10.2.2.3.0.0 | Pytorch 1.10.2 | link |
| VGG16 | CV - Image Classification | 1,3,224,224 | 1.10.1.2.2.0.0 | Pytorch 1.10.1 | link |
| ViT | CV - Image Classification | 1,3,224,224 | 1.10.2.2.3.0.0 | Pytorch 1.10.2 | link |
| YOLOv5 | CV - Object Detection | 1,3,640,640 | 1.10.1.2.2.0.0 | Pytorch 1.10.1 | link |
| YOLOv6 | CV - Object Detection | 1,3,640,640 | 1.11.0.2.3.0.0 | Pytorch 1.11.0 | link |
| YOLOv7 | CV - Object Detection+Pose Estimation | 1,3,960,960 | 1.10.1.2.2.0.0 | Pytorch 1.10.1 | link |
| YOLOF | CV - Object Detection | 1,3,300,300 | 1.10.1.2.2.0.0 | Pytorch 1.10.1 | link |
| Fairseq | NLP | max_length=32 | 1.10.1.* | Pytorch 1.10.1 | link |
In order to run the samples, you first need to set up a PyTorch Neuron development environment.