generated from mlcommons/mlperf_inference_submissions
-
Notifications
You must be signed in to change notification settings - Fork 4
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Loading status checks…
Auto-merge updates from auto-update branch
Showing
47 changed files
with
1,259 additions
and
1,454 deletions.
There are no files selected for viewing
6 changes: 3 additions & 3 deletions
6
...nts/gh_ubuntu-latest_x86-mlcommons_cpp-cpu-onnxruntime-default_config/README.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,3 +1,3 @@ | ||
| Model | Scenario | Accuracy | Throughput | Latency (in ms) | | ||
|----------|------------|------------|--------------|-------------------| | ||
| resnet50 | offline | 76 | 22.935 | - | | ||
| Model | Scenario | Accuracy | Throughput | Latency (in ms) | | ||
|-----------|------------|------------|--------------|-------------------| | ||
| retinanet | offline | () | 0.425 | - | |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
8 changes: 4 additions & 4 deletions
8
...t_x86-mlcommons_cpp-cpu-onnxruntime-default_config/retinanet/offline/accuracy_console.out
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Binary file modified
BIN
+147 Bytes
(100%)
...x86-mlcommons_cpp-cpu-onnxruntime-default_config/retinanet/offline/mlc-deps.png
Unable to render rich display
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
8 changes: 4 additions & 4 deletions
8
...86-mlcommons_cpp-cpu-onnxruntime-default_config/retinanet/offline/performance_console.out
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
2 changes: 1 addition & 1 deletion
2
...gh_ubuntu-latest_x86-reference-cpu-onnxruntime_v1.20.1-default_config/README.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,3 +1,3 @@ | ||
| Model | Scenario | Accuracy | Throughput | Latency (in ms) | | ||
|----------|------------|------------|--------------|-------------------| | ||
| resnet50 | offline | 76 | 23.163 | - | | ||
| resnet50 | offline | 76 | 23.169 | - | |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
6 changes: 3 additions & 3 deletions
6
...86-reference-cpu-onnxruntime_v1.20.1-default_config/resnet50/offline/accuracy_console.out
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,6 +1,6 @@ | ||
python3 python/main.py --profile resnet50-onnxruntime --model "/home/runner/MLC/repos/local/cache/download-file_8a5135a0/resnet50_v1.onnx" --dataset-path /home/runner/MLC/repos/local/cache/get-preprocessed-dataset-imagenet_23548357 --output "/home/runner/MLC/repos/local/cache/get-mlperf-inference-results-dir_2d2ae342/test_results/gh_ubuntu-latest_x86-reference-cpu-onnxruntime-v1.20.1-default_config/resnet50/offline/accuracy" --scenario Offline --count 500 --threads 4 --user_conf /home/runner/MLC/repos/GATEOverflow@mlperf-automations/script/generate-mlperf-inference-user-conf/tmp/d32da5e24b354e62870f8d3ca1333125.conf --accuracy --use_preprocessed_dataset --cache_dir /home/runner/MLC/repos/local/cache/get-preprocessed-dataset-imagenet_23548357 --dataset-list /home/runner/MLC/repos/local/cache/extract-file_95cf384e/val.txt | ||
INFO:main:Namespace(dataset='imagenet', dataset_path='/home/runner/MLC/repos/local/cache/get-preprocessed-dataset-imagenet_23548357', dataset_list='/home/runner/MLC/repos/local/cache/extract-file_95cf384e/val.txt', data_format=None, profile='resnet50-onnxruntime', scenario='Offline', max_batchsize=32, model='/home/runner/MLC/repos/local/cache/download-file_8a5135a0/resnet50_v1.onnx', output='/home/runner/MLC/repos/local/cache/get-mlperf-inference-results-dir_2d2ae342/test_results/gh_ubuntu-latest_x86-reference-cpu-onnxruntime-v1.20.1-default_config/resnet50/offline/accuracy', inputs=None, outputs=['ArgMax:0'], backend='onnxruntime', device=None, model_name='resnet50', threads=4, qps=None, cache=0, cache_dir='/home/runner/MLC/repos/local/cache/get-preprocessed-dataset-imagenet_23548357', preprocessed_dir=None, use_preprocessed_dataset=True, accuracy=True, find_peak_performance=False, debug=False, user_conf='/home/runner/MLC/repos/GATEOverflow@mlperf-automations/script/generate-mlperf-inference-user-conf/tmp/d32da5e24b354e62870f8d3ca1333125.conf', audit_conf='audit.config', time=None, count=500, performance_sample_count=None, max_latency=None, samples_per_query=8) | ||
python3 python/main.py --profile resnet50-onnxruntime --model "/home/runner/MLC/repos/local/cache/download-file_b776794b/resnet50_v1.onnx" --dataset-path /home/runner/MLC/repos/local/cache/get-preprocessed-dataset-imagenet_379126da --output "/home/runner/MLC/repos/local/cache/get-mlperf-inference-results-dir_d3cd6d60/test_results/gh_ubuntu-latest_x86-reference-cpu-onnxruntime-v1.20.1-default_config/resnet50/offline/accuracy" --scenario Offline --count 500 --threads 4 --user_conf /home/runner/MLC/repos/mlcommons@mlperf-automations/script/generate-mlperf-inference-user-conf/tmp/bc2ad4bb30694ce589e9b2d41e806a6a.conf --accuracy --use_preprocessed_dataset --cache_dir /home/runner/MLC/repos/local/cache/get-preprocessed-dataset-imagenet_379126da --dataset-list /home/runner/MLC/repos/local/cache/extract-file_91fed7e4/val.txt | ||
INFO:main:Namespace(dataset='imagenet', dataset_path='/home/runner/MLC/repos/local/cache/get-preprocessed-dataset-imagenet_379126da', dataset_list='/home/runner/MLC/repos/local/cache/extract-file_91fed7e4/val.txt', data_format=None, profile='resnet50-onnxruntime', scenario='Offline', max_batchsize=32, model='/home/runner/MLC/repos/local/cache/download-file_b776794b/resnet50_v1.onnx', output='/home/runner/MLC/repos/local/cache/get-mlperf-inference-results-dir_d3cd6d60/test_results/gh_ubuntu-latest_x86-reference-cpu-onnxruntime-v1.20.1-default_config/resnet50/offline/accuracy', inputs=None, outputs=['ArgMax:0'], backend='onnxruntime', device=None, model_name='resnet50', threads=4, qps=None, cache=0, cache_dir='/home/runner/MLC/repos/local/cache/get-preprocessed-dataset-imagenet_379126da', preprocessed_dir=None, use_preprocessed_dataset=True, accuracy=True, find_peak_performance=False, debug=False, user_conf='/home/runner/MLC/repos/mlcommons@mlperf-automations/script/generate-mlperf-inference-user-conf/tmp/bc2ad4bb30694ce589e9b2d41e806a6a.conf', audit_conf='audit.config', time=None, count=500, performance_sample_count=None, max_latency=None, samples_per_query=8) | ||
INFO:imagenet:Loading 500 preprocessed images using 4 threads | ||
INFO:imagenet:loaded 500 images, cache=0, already_preprocessed=True, took=0.0sec | ||
INFO:main:starting TestScenario.Offline | ||
TestScenario.Offline qps=0.74, mean=13.5786, time=21.633, acc=76.000%, queries=16, tiles=50.0:13.8288,80.0:20.9539,90.0:21.1509,95.0:21.3333,99.0:21.4312,99.9:21.4532 | ||
TestScenario.Offline qps=0.74, mean=13.5424, time=21.505, acc=76.000%, queries=16, tiles=50.0:13.8017,80.0:21.1621,90.0:21.2197,95.0:21.2636,99.0:21.3338,99.9:21.3496 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Binary file modified
BIN
+4.8 KB
(100%)
...-reference-cpu-onnxruntime_v1.20.1-default_config/resnet50/offline/mlc-deps.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
6 changes: 3 additions & 3 deletions
6
...reference-cpu-onnxruntime_v1.20.1-default_config/resnet50/offline/performance_console.out
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,7 +1,7 @@ | ||
python3 python/main.py --profile resnet50-onnxruntime --model "/home/runner/MLC/repos/local/cache/download-file_8a5135a0/resnet50_v1.onnx" --dataset-path /home/runner/MLC/repos/local/cache/get-preprocessed-dataset-imagenet_23548357 --output "/home/runner/MLC/repos/local/cache/get-mlperf-inference-results-dir_2d2ae342/test_results/gh_ubuntu-latest_x86-reference-cpu-onnxruntime-v1.20.1-default_config/resnet50/offline/performance/run_1" --scenario Offline --threads 4 --user_conf /home/runner/MLC/repos/GATEOverflow@mlperf-automations/script/generate-mlperf-inference-user-conf/tmp/537eb041324b459db918e9d37836c74c.conf --use_preprocessed_dataset --cache_dir /home/runner/MLC/repos/local/cache/get-preprocessed-dataset-imagenet_23548357 --dataset-list /home/runner/MLC/repos/local/cache/extract-file_95cf384e/val.txt | ||
INFO:main:Namespace(dataset='imagenet', dataset_path='/home/runner/MLC/repos/local/cache/get-preprocessed-dataset-imagenet_23548357', dataset_list='/home/runner/MLC/repos/local/cache/extract-file_95cf384e/val.txt', data_format=None, profile='resnet50-onnxruntime', scenario='Offline', max_batchsize=32, model='/home/runner/MLC/repos/local/cache/download-file_8a5135a0/resnet50_v1.onnx', output='/home/runner/MLC/repos/local/cache/get-mlperf-inference-results-dir_2d2ae342/test_results/gh_ubuntu-latest_x86-reference-cpu-onnxruntime-v1.20.1-default_config/resnet50/offline/performance/run_1', inputs=None, outputs=['ArgMax:0'], backend='onnxruntime', device=None, model_name='resnet50', threads=4, qps=None, cache=0, cache_dir='/home/runner/MLC/repos/local/cache/get-preprocessed-dataset-imagenet_23548357', preprocessed_dir=None, use_preprocessed_dataset=True, accuracy=False, find_peak_performance=False, debug=False, user_conf='/home/runner/MLC/repos/GATEOverflow@mlperf-automations/script/generate-mlperf-inference-user-conf/tmp/537eb041324b459db918e9d37836c74c.conf', audit_conf='audit.config', time=None, count=None, performance_sample_count=None, max_latency=None, samples_per_query=8) | ||
python3 python/main.py --profile resnet50-onnxruntime --model "/home/runner/MLC/repos/local/cache/download-file_b776794b/resnet50_v1.onnx" --dataset-path /home/runner/MLC/repos/local/cache/get-preprocessed-dataset-imagenet_379126da --output "/home/runner/MLC/repos/local/cache/get-mlperf-inference-results-dir_d3cd6d60/test_results/gh_ubuntu-latest_x86-reference-cpu-onnxruntime-v1.20.1-default_config/resnet50/offline/performance/run_1" --scenario Offline --threads 4 --user_conf /home/runner/MLC/repos/mlcommons@mlperf-automations/script/generate-mlperf-inference-user-conf/tmp/9b512a7a5ce044eb868702cd1da308f7.conf --use_preprocessed_dataset --cache_dir /home/runner/MLC/repos/local/cache/get-preprocessed-dataset-imagenet_379126da --dataset-list /home/runner/MLC/repos/local/cache/extract-file_91fed7e4/val.txt | ||
INFO:main:Namespace(dataset='imagenet', dataset_path='/home/runner/MLC/repos/local/cache/get-preprocessed-dataset-imagenet_379126da', dataset_list='/home/runner/MLC/repos/local/cache/extract-file_91fed7e4/val.txt', data_format=None, profile='resnet50-onnxruntime', scenario='Offline', max_batchsize=32, model='/home/runner/MLC/repos/local/cache/download-file_b776794b/resnet50_v1.onnx', output='/home/runner/MLC/repos/local/cache/get-mlperf-inference-results-dir_d3cd6d60/test_results/gh_ubuntu-latest_x86-reference-cpu-onnxruntime-v1.20.1-default_config/resnet50/offline/performance/run_1', inputs=None, outputs=['ArgMax:0'], backend='onnxruntime', device=None, model_name='resnet50', threads=4, qps=None, cache=0, cache_dir='/home/runner/MLC/repos/local/cache/get-preprocessed-dataset-imagenet_379126da', preprocessed_dir=None, use_preprocessed_dataset=True, accuracy=False, find_peak_performance=False, debug=False, user_conf='/home/runner/MLC/repos/mlcommons@mlperf-automations/script/generate-mlperf-inference-user-conf/tmp/9b512a7a5ce044eb868702cd1da308f7.conf', audit_conf='audit.config', time=None, count=None, performance_sample_count=None, max_latency=None, samples_per_query=8) | ||
INFO:imagenet:Loading 50000 preprocessed images using 4 threads | ||
INFO:imagenet:reduced image list, 49500 images not found | ||
INFO:imagenet:loaded 500 images, cache=0, already_preprocessed=True, took=1.1sec | ||
INFO:main:starting TestScenario.Offline | ||
TestScenario.Offline qps=0.74, mean=13.5595, time=21.632, queries=16, tiles=50.0:13.7440,80.0:20.8913,90.0:21.0453,95.0:21.2512,99.0:21.4173,99.9:21.4547 | ||
TestScenario.Offline qps=0.74, mean=13.5443, time=21.629, queries=16, tiles=50.0:13.4418,80.0:21.0139,90.0:21.1964,95.0:21.3541,99.0:21.4279,99.9:21.4445 |
2 changes: 1 addition & 1 deletion
2
...est_x86-reference-cpu-onnxruntime_v1.20.1-default_config/resnet50/offline/pip_freeze.json
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
2 changes: 1 addition & 1 deletion
2
...ents/gh_ubuntu-latest_x86-reference-cpu-pytorch_v2.4.0-default_config/README.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,3 +1,3 @@ | ||
| Model | Scenario | Accuracy | Throughput | Latency (in ms) | | ||
|---------|------------|------------|--------------|-------------------| | ||
| rgat | offline | 75.2 | 10.199 | - | | ||
| rgat | offline | 74.8 | 9.995 | - | |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
10 changes: 5 additions & 5 deletions
10
...-latest_x86-reference-cpu-pytorch_v2.4.0-default_config/rgat/offline/accuracy_console.out
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,10 +1,10 @@ | ||
INFO:main:Namespace(dataset='igbh-dgl-tiny', dataset_path='/home/runner/MLC/repos/local/cache/get-dataset-igbh_43f808e4', layout='COO', profile='debug-dgl', scenario='Offline', max_batchsize=1, threads=4, accuracy=True, find_peak_performance=False, backend='dgl', model_name='rgat', output='/home/runner/MLC/repos/local/cache/get-mlperf-inference-results-dir_ef0ec58e/test_results/gh_ubuntu-latest_x86-reference-cpu-pytorch-v2.4.0-default_config/rgat/offline/accuracy', qps=None, model_path='/home/runner/MLC/repos/local/cache/download-file_79babd4e/RGAT/RGAT.pt', dtype='fp32', device='cpu', user_conf='/home/runner/MLC/repos/anandhu-eng@mlperf-automations/script/generate-mlperf-inference-user-conf/tmp/1dc82411c6ea434d990d3edfeb3816a4.conf', audit_conf='audit.config', time=None, count=500, debug=False, performance_sample_count=5000, max_latency=None, samples_per_query=8) | ||
/home/runner/MLC/repos/local/cache/get-git-repo_2fd30ee0/inference/graph/R-GAT/dgl_utilities/feature_fetching.py:231: UserWarning: The given NumPy array is not writable, and PyTorch does not support non-writable tensors. This means writing to this tensor will result in undefined behavior. You may want to copy the array to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at ../torch/csrc/utils/tensor_numpy.cpp:206.) | ||
INFO:main:Namespace(dataset='igbh-dgl-tiny', dataset_path='/home/runner/MLC/repos/local/cache/get-dataset-igbh_92121abb', layout='COO', profile='debug-dgl', scenario='Offline', max_batchsize=1, threads=4, accuracy=True, find_peak_performance=False, backend='dgl', model_name='rgat', output='/home/runner/MLC/repos/local/cache/get-mlperf-inference-results-dir_292e922f/test_results/gh_ubuntu-latest_x86-reference-cpu-pytorch-v2.4.0-default_config/rgat/offline/accuracy', qps=None, model_path='/home/runner/MLC/repos/local/cache/download-file_1889c34a/RGAT/RGAT.pt', dtype='fp32', device='cpu', user_conf='/home/runner/MLC/repos/anandhu-eng@mlperf-automations/script/generate-mlperf-inference-user-conf/tmp/67256ec64bfa4f91a30a8a2d968dd943.conf', audit_conf='audit.config', time=None, count=500, debug=False, performance_sample_count=5000, max_latency=None, samples_per_query=8) | ||
/home/runner/MLC/repos/local/cache/get-git-repo_4248c027/inference/graph/R-GAT/dgl_utilities/feature_fetching.py:231: UserWarning: The given NumPy array is not writable, and PyTorch does not support non-writable tensors. This means writing to this tensor will result in undefined behavior. You may want to copy the array to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at ../torch/csrc/utils/tensor_numpy.cpp:206.) | ||
return edge, torch.from_numpy( | ||
/home/runner/MLC/repos/local/cache/get-git-repo_2fd30ee0/inference/graph/R-GAT/dgl_utilities/feature_fetching.py:312: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature. | ||
/home/runner/MLC/repos/local/cache/get-git-repo_4248c027/inference/graph/R-GAT/dgl_utilities/feature_fetching.py:312: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature. | ||
torch.load( | ||
/home/runner/MLC/repos/local/cache/get-git-repo_2fd30ee0/inference/graph/R-GAT/dgl_utilities/feature_fetching.py:318: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature. | ||
/home/runner/MLC/repos/local/cache/get-git-repo_4248c027/inference/graph/R-GAT/dgl_utilities/feature_fetching.py:318: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature. | ||
torch.load( | ||
/home/runner/MLC/repos/local/cache/get-git-repo_2fd30ee0/inference/graph/R-GAT/backend_dgl.py:70: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature. | ||
/home/runner/MLC/repos/local/cache/get-git-repo_4248c027/inference/graph/R-GAT/backend_dgl.py:70: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature. | ||
ckpt = torch.load(ckpt_path, map_location=self.device) | ||
INFO:main:starting TestScenario.Offline |
50 changes: 25 additions & 25 deletions
50
...h_ubuntu-latest_x86-reference-cpu-pytorch_v2.4.0-default_config/rgat/offline/mlc-deps.mmd
Large diffs are not rendered by default.
Oops, something went wrong.
Binary file modified
BIN
+181 Bytes
(100%)
...atest_x86-reference-cpu-pytorch_v2.4.0-default_config/rgat/offline/mlc-deps.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
10 changes: 5 additions & 5 deletions
10
...test_x86-reference-cpu-pytorch_v2.4.0-default_config/rgat/offline/performance_console.out
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,12 +1,12 @@ | ||
INFO:main:Namespace(dataset='igbh-dgl-tiny', dataset_path='/home/runner/MLC/repos/local/cache/get-dataset-igbh_43f808e4', layout='COO', profile='debug-dgl', scenario='Offline', max_batchsize=1, threads=4, accuracy=False, find_peak_performance=False, backend='dgl', model_name='rgat', output='/home/runner/MLC/repos/local/cache/get-mlperf-inference-results-dir_ef0ec58e/test_results/gh_ubuntu-latest_x86-reference-cpu-pytorch-v2.4.0-default_config/rgat/offline/performance/run_1', qps=None, model_path='/home/runner/MLC/repos/local/cache/download-file_79babd4e/RGAT/RGAT.pt', dtype='fp32', device='cpu', user_conf='/home/runner/MLC/repos/anandhu-eng@mlperf-automations/script/generate-mlperf-inference-user-conf/tmp/108a9ae2f84f42d5b12a130fe56d752c.conf', audit_conf='audit.config', time=None, count=None, debug=False, performance_sample_count=5000, max_latency=None, samples_per_query=8) | ||
/home/runner/MLC/repos/local/cache/get-git-repo_2fd30ee0/inference/graph/R-GAT/dgl_utilities/feature_fetching.py:231: UserWarning: The given NumPy array is not writable, and PyTorch does not support non-writable tensors. This means writing to this tensor will result in undefined behavior. You may want to copy the array to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at ../torch/csrc/utils/tensor_numpy.cpp:206.) | ||
INFO:main:Namespace(dataset='igbh-dgl-tiny', dataset_path='/home/runner/MLC/repos/local/cache/get-dataset-igbh_92121abb', layout='COO', profile='debug-dgl', scenario='Offline', max_batchsize=1, threads=4, accuracy=False, find_peak_performance=False, backend='dgl', model_name='rgat', output='/home/runner/MLC/repos/local/cache/get-mlperf-inference-results-dir_292e922f/test_results/gh_ubuntu-latest_x86-reference-cpu-pytorch-v2.4.0-default_config/rgat/offline/performance/run_1', qps=None, model_path='/home/runner/MLC/repos/local/cache/download-file_1889c34a/RGAT/RGAT.pt', dtype='fp32', device='cpu', user_conf='/home/runner/MLC/repos/anandhu-eng@mlperf-automations/script/generate-mlperf-inference-user-conf/tmp/a0d78dd04c8f47ebae79101e4d2f0e6b.conf', audit_conf='audit.config', time=None, count=None, debug=False, performance_sample_count=5000, max_latency=None, samples_per_query=8) | ||
/home/runner/MLC/repos/local/cache/get-git-repo_4248c027/inference/graph/R-GAT/dgl_utilities/feature_fetching.py:231: UserWarning: The given NumPy array is not writable, and PyTorch does not support non-writable tensors. This means writing to this tensor will result in undefined behavior. You may want to copy the array to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at ../torch/csrc/utils/tensor_numpy.cpp:206.) | ||
return edge, torch.from_numpy( | ||
/home/runner/MLC/repos/local/cache/get-git-repo_2fd30ee0/inference/graph/R-GAT/dgl_utilities/feature_fetching.py:312: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature. | ||
/home/runner/MLC/repos/local/cache/get-git-repo_4248c027/inference/graph/R-GAT/dgl_utilities/feature_fetching.py:312: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature. | ||
torch.load( | ||
/home/runner/MLC/repos/local/cache/get-git-repo_2fd30ee0/inference/graph/R-GAT/dgl_utilities/feature_fetching.py:318: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature. | ||
/home/runner/MLC/repos/local/cache/get-git-repo_4248c027/inference/graph/R-GAT/dgl_utilities/feature_fetching.py:318: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature. | ||
torch.load( | ||
DGL backend not selected or invalid. Assuming PyTorch for now. | ||
/home/runner/MLC/repos/local/cache/get-git-repo_2fd30ee0/inference/graph/R-GAT/backend_dgl.py:70: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature. | ||
/home/runner/MLC/repos/local/cache/get-git-repo_4248c027/inference/graph/R-GAT/backend_dgl.py:70: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature. | ||
ckpt = torch.load(ckpt_path, map_location=self.device) | ||
INFO:main:starting TestScenario.Offline | ||
Setting the default backend to "pytorch". You can change it in the ~/.dgl/config.json file or export the DGLBACKEND environment variable. Valid options are: pytorch, mxnet, tensorflow (all lowercase) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
142 changes: 71 additions & 71 deletions
142
...mmons_cpp-cpu-onnxruntime-default_config/retinanet/offline/accuracy/mlperf_log_detail.txt
Large diffs are not rendered by default.
Oops, something went wrong.
176 changes: 88 additions & 88 deletions
176
...-cpu-onnxruntime-default_config/retinanet/offline/performance/run_1/mlperf_log_detail.txt
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
2 changes: 1 addition & 1 deletion
2
...6-reference-cpu-onnxruntime_v1.20.1-default_config/resnet50/offline/accuracy/accuracy.txt
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,3 +1,3 @@ | ||
accuracy=76.000%, good=380, total=500 | ||
|
||
hash=85f2a8434e74092206ea2a2008fc008a37703af14d4e5b3839003d315b266495 | ||
hash=9c40e1a3c0c1624da382955cba3ed8cb6e3337427f4a2d5351f706fc15fd4bd5 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
140 changes: 70 additions & 70 deletions
140
...ce-cpu-onnxruntime_v1.20.1-default_config/resnet50/offline/accuracy/mlperf_log_detail.txt
Large diffs are not rendered by default.
Oops, something went wrong.
174 changes: 87 additions & 87 deletions
174
...nxruntime_v1.20.1-default_config/resnet50/offline/performance/run_1/mlperf_log_detail.txt
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
4 changes: 2 additions & 2 deletions
4
...latest_x86-reference-cpu-pytorch_v2.4.0-default_config/rgat/offline/accuracy/accuracy.txt
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,2 +1,2 @@ | ||
accuracy=75.200%, good=376, total=500 | ||
hash=860352bb233206fea1c48f25756460b160646446851c58b0812c9dbaa4ff54a6 | ||
accuracy=74.800%, good=374, total=500 | ||
hash=2e31342aa0323df39ac35203de793ab0df8a160e10e8f6b5d203ffff279a931e |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
140 changes: 70 additions & 70 deletions
140
...6-reference-cpu-pytorch_v2.4.0-default_config/rgat/offline/accuracy/mlperf_log_detail.txt
Large diffs are not rendered by default.
Oops, something went wrong.
174 changes: 87 additions & 87 deletions
174
...ce-cpu-pytorch_v2.4.0-default_config/rgat/offline/performance/run_1/mlperf_log_detail.txt
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters