-
Notifications
You must be signed in to change notification settings - Fork 0
/
power_perf_tp2.txt
6937 lines (6533 loc) · 207 KB
/
power_perf_tp2.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
GPU clocks set to "(gpuClkMin 800, gpuClkMax 800)" for GPU 00000001:00:00.0
Warning: persistence mode is disabled on device 00000001:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 800, gpuClkMax 800)" for GPU 00000002:00:00.0
Warning: persistence mode is disabled on device 00000002:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 800, gpuClkMax 800)" for GPU 00000003:00:00.0
Warning: persistence mode is disabled on device 00000003:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 800, gpuClkMax 800)" for GPU 00000008:00:00.0
Warning: persistence mode is disabled on device 00000008:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 800, gpuClkMax 800)" for GPU 00000009:00:00.0
Warning: persistence mode is disabled on device 00000009:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 800, gpuClkMax 800)" for GPU 0000000A:00:00.0
Warning: persistence mode is disabled on device 0000000A:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 800, gpuClkMax 800)" for GPU 0000000B:00:00.0
Warning: persistence mode is disabled on device 0000000B:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 800, gpuClkMax 800)" for GPU 0000000C:00:00.0
Warning: persistence mode is disabled on device 0000000C:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
All done.
/home/azureuser/miniconda3/envs/jovan_vllm/lib/python3.10/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
warnings.warn(
GPU clocks set to "(gpuClkMin 1000, gpuClkMax 1000)" for GPU 00000001:00:00.0
Warning: persistence mode is disabled on device 00000001:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 1000, gpuClkMax 1000)" for GPU 00000002:00:00.0
Warning: persistence mode is disabled on device 00000002:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 1000, gpuClkMax 1000)" for GPU 00000003:00:00.0
Warning: persistence mode is disabled on device 00000003:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 1000, gpuClkMax 1000)" for GPU 00000008:00:00.0
Warning: persistence mode is disabled on device 00000008:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 1000, gpuClkMax 1000)" for GPU 00000009:00:00.0
Warning: persistence mode is disabled on device 00000009:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 1000, gpuClkMax 1000)" for GPU 0000000A:00:00.0
Warning: persistence mode is disabled on device 0000000A:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 1000, gpuClkMax 1000)" for GPU 0000000B:00:00.0
Warning: persistence mode is disabled on device 0000000B:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 1000, gpuClkMax 1000)" for GPU 0000000C:00:00.0
Warning: persistence mode is disabled on device 0000000C:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
All done.
GPU clocks set to "(gpuClkMin 1200, gpuClkMax 1200)" for GPU 00000001:00:00.0
Warning: persistence mode is disabled on device 00000001:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 1200, gpuClkMax 1200)" for GPU 00000002:00:00.0
Warning: persistence mode is disabled on device 00000002:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 1200, gpuClkMax 1200)" for GPU 00000003:00:00.0
Warning: persistence mode is disabled on device 00000003:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 1200, gpuClkMax 1200)" for GPU 00000008:00:00.0
Warning: persistence mode is disabled on device 00000008:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 1200, gpuClkMax 1200)" for GPU 00000009:00:00.0
Warning: persistence mode is disabled on device 00000009:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 1200, gpuClkMax 1200)" for GPU 0000000A:00:00.0
Warning: persistence mode is disabled on device 0000000A:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 1200, gpuClkMax 1200)" for GPU 0000000B:00:00.0
Warning: persistence mode is disabled on device 0000000B:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 1200, gpuClkMax 1200)" for GPU 0000000C:00:00.0
Warning: persistence mode is disabled on device 0000000C:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
All done.
Open ttft file
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.22655663887659708
P50 ttft = 0.17856788635253906
P99 ttft = 0.40079737901687623
Average tbt = 0.05533905824025472
P50 tbt = 0.05535469055175782
P99 tbt = 0.055789117813110356
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.1733060223715646
P50 ttft = 0.16004300117492676
P99 ttft = 0.27827820777893064
Average tbt = 0.05680506342933291
P50 tbt = 0.05683774948120117
P99 tbt = 0.0571907901763916
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.1948878424508231
P50 ttft = 0.18489813804626465
P99 ttft = 0.34912899494171123
Average tbt = 0.0611346530914307
P50 tbt = 0.058236122131347656
P99 tbt = 0.11469740867614792
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.21434460035184535
P50 ttft = 0.18851685523986816
P99 ttft = 0.3596044540405274
Average tbt = 0.0933785171043583
P50 tbt = 0.06611161231994629
P99 tbt = 0.19424486160278326
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.22513980388641358
P50 ttft = 0.18703985214233398
P99 ttft = 0.44621349096298196
Average tbt = 0.14989850139617936
P50 tbt = 0.19092037677764895
P99 tbt = 0.20124845933914184
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.24823496118187904
P50 ttft = 0.21554148197174072
P99 ttft = 0.5250015521049499
Average tbt = 0.1624095171689987
P50 tbt = 0.2099858283996582
P99 tbt = 0.21230566596984862
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.43962394701291435
P50 ttft = 0.2740459442138672
P99 ttft = 2.6618298721313476
Average tbt = 0.19279410839080835
P50 tbt = 0.21654157638549804
P99 tbt = 0.3044935741424569
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 1.1237804975854344
P50 ttft = 0.41536927223205566
P99 ttft = 4.450040550231933
Average tbt = 0.17071266662643644
P50 tbt = 0.21440653800964354
P99 tbt = 0.2941163797378551
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 1.996028720064366
P50 ttft = 0.593809962272644
P99 ttft = 5.611352460384369
Average tbt = 0.17455865652003194
P50 tbt = 0.21518545150756835
P99 tbt = 0.2708026957511909
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 3.092231255485898
P50 ttft = 4.100458860397339
P99 ttft = 9.976745319366453
Average tbt = 0.17822845254625597
P50 tbt = 0.21542763710021973
P99 tbt = 0.21651689720153808
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 4.030135262895514
P50 ttft = 5.507741808891296
P99 ttft = 12.004428496360777
Average tbt = 0.19810283228203143
P50 tbt = 0.2164520025253296
P99 tbt = 0.2972181100845348
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 5.346222454760255
P50 ttft = 6.73125958442688
P99 ttft = 13.368747420310974
Average tbt = 0.19733144856300702
P50 tbt = 0.21507043838500978
P99 tbt = 0.29604515743255727
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.14855831861495972
P50 ttft = 0.13056766986846924
P99 ttft = 0.2559222769737244
Average tbt = 0.0438698927561442
P50 tbt = 0.04400639533996582
P99 tbt = 0.04409557819366455
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.14558491252717518
P50 ttft = 0.1362626552581787
P99 ttft = 0.23903951644897461
Average tbt = 0.0450662612915039
P50 tbt = 0.045170021057128903
P99 tbt = 0.04537228584289551
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.1664799077170236
P50 ttft = 0.14622235298156738
P99 ttft = 0.2582477903366089
Average tbt = 0.0460541125706264
P50 tbt = 0.04600067138671875
P99 tbt = 0.046586114883422854
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.16348678891251728
P50 ttft = 0.1498100757598877
P99 ttft = 0.2635683536529541
Average tbt = 0.05183292249353924
P50 tbt = 0.04657988548278809
P99 tbt = 0.11308545112609952
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.173814058303833
P50 ttft = 0.1518855094909668
P99 ttft = 0.28097796916961665
Average tbt = 0.08906897449493441
P50 tbt = 0.08386137485504194
P99 tbt = 0.1622824320793152
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.18653341010212898
P50 ttft = 0.1642014980316162
P99 ttft = 0.31672698736190796
Average tbt = 0.12014439180493369
P50 tbt = 0.15570013523101808
P99 tbt = 0.16466137313842774
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.19545194547470301
P50 ttft = 0.16665005683898926
P99 ttft = 0.3803891563415528
Average tbt = 0.13089887018073096
P50 tbt = 0.16132440567016607
P99 tbt = 0.1734361801147461
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.21375766145177635
P50 ttft = 0.17470431327819824
P99 ttft = 0.48786224842071296
Average tbt = 0.1525264343583441
P50 tbt = 0.17401852607727053
P99 tbt = 0.23793091869354335
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.45788387034801725
P50 ttft = 0.2840902805328369
P99 ttft = 2.348403124809265
Average tbt = 0.20377645873008912
P50 tbt = 0.23843896389007657
P99 tbt = 0.28435653209686285
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 1.1914254165831066
P50 ttft = 0.8776190280914307
P99 ttft = 3.6649210071563716
Average tbt = 0.16897219498952262
P50 tbt = 0.1767263889312744
P99 tbt = 0.24073154067993255
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 1.9796141937926963
P50 ttft = 2.391965389251709
P99 ttft = 5.816887996196746
Average tbt = 0.15787632708196314
P50 tbt = 0.17734346389770508
P99 tbt = 0.24143627309799284
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 2.757201357048099
P50 ttft = 3.289769411087036
P99 ttft = 7.290604681968689
Average tbt = 0.1626316765777205
P50 tbt = 0.1772172451019287
P99 tbt = 0.24109138298034755
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.11531883478164673
P50 ttft = 0.11374044418334961
P99 ttft = 0.1765307426452637
Average tbt = 0.03751585880915323
P50 tbt = 0.037558364868164065
P99 tbt = 0.03774892663955688
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.11813880148388091
P50 ttft = 0.1080634593963623
P99 ttft = 0.1979659080505371
Average tbt = 0.03851359912327357
P50 tbt = 0.038570833206176755
P99 tbt = 0.03879303932189942
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.13307456970214843
P50 ttft = 0.12281417846679688
P99 ttft = 0.21638488292694089
Average tbt = 0.039389181137084964
P50 tbt = 0.03916974067687988
P99 tbt = 0.040061381340026854
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.13500423547698231
P50 ttft = 0.11910128593444824
P99 ttft = 0.2274948596954346
Average tbt = 0.03991061071070229
P50 tbt = 0.04006547927856445
P99 tbt = 0.040425167083740235
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.14262319564819337
P50 ttft = 0.12268316745758057
P99 ttft = 0.23964771270751944
Average tbt = 0.05077710056304936
P50 tbt = 0.0403667688369751
P99 tbt = 0.1317711262702942
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.14528969675302505
P50 ttft = 0.12396121025085449
P99 ttft = 0.2708895182609558
Average tbt = 0.08443968296051034
P50 tbt = 0.04657158851623536
P99 tbt = 0.13834078359603882
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.15874002731009706
P50 ttft = 0.136824369430542
P99 ttft = 0.27161320686340334
Average tbt = 0.10801550394868208
P50 tbt = 0.1367235660552979
P99 tbt = 0.13897422218322755
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.16427546811391072
P50 ttft = 0.1435079574584961
P99 ttft = 0.30500583648681606
Average tbt = 0.12113820616021216
P50 tbt = 0.14408879280090334
P99 tbt = 0.14655828762054446
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.16385421854384402
P50 ttft = 0.1368175745010376
P99 ttft = 0.34136536359786973
Average tbt = 0.1270974666514296
P50 tbt = 0.1471567153930664
P99 tbt = 0.15340276479721043
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.350209531329927
P50 ttft = 0.2257075309753418
P99 ttft = 1.571854496002197
Average tbt = 0.17910238265991224
P50 tbt = 0.2033367156982429
P99 tbt = 0.24188705062866211
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.741041671346735
P50 ttft = 0.3077573776245117
P99 ttft = 2.596739325523376
GPU clocks set to "(gpuClkMin 1400, gpuClkMax 1400)" for GPU 00000001:00:00.0
Warning: persistence mode is disabled on device 00000001:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 1400, gpuClkMax 1400)" for GPU 00000002:00:00.0
Warning: persistence mode is disabled on device 00000002:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 1400, gpuClkMax 1400)" for GPU 00000003:00:00.0
Warning: persistence mode is disabled on device 00000003:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 1400, gpuClkMax 1400)" for GPU 00000008:00:00.0
Warning: persistence mode is disabled on device 00000008:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 1400, gpuClkMax 1400)" for GPU 00000009:00:00.0
Warning: persistence mode is disabled on device 00000009:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 1400, gpuClkMax 1400)" for GPU 0000000A:00:00.0
Warning: persistence mode is disabled on device 0000000A:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 1400, gpuClkMax 1400)" for GPU 0000000B:00:00.0
Warning: persistence mode is disabled on device 0000000B:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 1400, gpuClkMax 1400)" for GPU 0000000C:00:00.0
Warning: persistence mode is disabled on device 0000000C:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
All done.
GPU clocks set to "(gpuClkMin 1600, gpuClkMax 1600)" for GPU 00000001:00:00.0
Warning: persistence mode is disabled on device 00000001:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 1600, gpuClkMax 1600)" for GPU 00000002:00:00.0
Warning: persistence mode is disabled on device 00000002:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 1600, gpuClkMax 1600)" for GPU 00000003:00:00.0
Warning: persistence mode is disabled on device 00000003:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 1600, gpuClkMax 1600)" for GPU 00000008:00:00.0
Warning: persistence mode is disabled on device 00000008:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 1600, gpuClkMax 1600)" for GPU 00000009:00:00.0
Warning: persistence mode is disabled on device 00000009:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 1600, gpuClkMax 1600)" for GPU 0000000A:00:00.0
Warning: persistence mode is disabled on device 0000000A:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 1600, gpuClkMax 1600)" for GPU 0000000B:00:00.0
Warning: persistence mode is disabled on device 0000000B:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 1600, gpuClkMax 1600)" for GPU 0000000C:00:00.0
Warning: persistence mode is disabled on device 0000000C:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
All done.
GPU clocks set to "(gpuClkMin 1800, gpuClkMax 1800)" for GPU 00000001:00:00.0
Warning: persistence mode is disabled on device 00000001:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 1800, gpuClkMax 1800)" for GPU 00000002:00:00.0
Warning: persistence mode is disabled on device 00000002:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 1800, gpuClkMax 1800)" for GPU 00000003:00:00.0
Warning: persistence mode is disabled on device 00000003:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 1800, gpuClkMax 1800)" for GPU 00000008:00:00.0
Warning: persistence mode is disabled on device 00000008:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 1800, gpuClkMax 1800)" for GPU 00000009:00:00.0
Warning: persistence mode is disabled on device 00000009:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 1800, gpuClkMax 1800)" for GPU 0000000A:00:00.0
Warning: persistence mode is disabled on device 0000000A:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 1800, gpuClkMax 1800)" for GPU 0000000B:00:00.0
Warning: persistence mode is disabled on device 0000000B:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 1800, gpuClkMax 1800)" for GPU 0000000C:00:00.0
Warning: persistence mode is disabled on device 0000000C:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
All done.
Average tbt = 0.15558656718995859
P50 tbt = 0.15151119232177734
P99 tbt = 0.2401814794540405
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 1.39450335903328
P50 ttft = 1.5313758850097656
P99 ttft = 3.7008480358123776
Average tbt = 0.17508716523146464
P50 tbt = 0.20489196777343824
P99 tbt = 0.24119441604614258
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.10075875123341878
P50 ttft = 0.09049296379089355
P99 ttft = 0.155402889251709
Average tbt = 0.03546257416407267
P50 tbt = 0.0354996919631958
P99 tbt = 0.03574352025985718
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.1096857502346947
P50 ttft = 0.10016107559204102
P99 ttft = 0.19431614875793457
Average tbt = 0.036307611919584724
P50 tbt = 0.03634829521179199
P99 tbt = 0.03665243148803711
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.11607174873352051
P50 ttft = 0.10621857643127441
P99 ttft = 0.1916162872314453
Average tbt = 0.03710143361772808
P50 tbt = 0.03674015998840332
P99 tbt = 0.037912359237670896
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.12311439979367139
P50 ttft = 0.10861587524414062
P99 ttft = 0.1990213394165039
Average tbt = 0.03762098638022817
P50 tbt = 0.037625789642333984
P99 tbt = 0.03827567100524902
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.12467196941375733
P50 ttft = 0.11019551753997803
P99 ttft = 0.21237019300460808
Average tbt = 0.04414343166351323
P50 tbt = 0.03831202983856201
P99 tbt = 0.10407481336593653
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.1255795620381832
P50 ttft = 0.1154930591583252
P99 ttft = 0.1995777678489685
Average tbt = 0.06979686170816436
P50 tbt = 0.0436739444732666
P99 tbt = 0.12254860019683839
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.13592410740787036
P50 ttft = 0.11536192893981934
P99 ttft = 0.2346261215209961
Average tbt = 0.09903937888472056
P50 tbt = 0.12168006896972657
P99 tbt = 0.1270213146209717
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.14122284751340566
P50 ttft = 0.12450456619262695
P99 ttft = 0.24899449348449668
Average tbt = 0.10943004998816074
P50 tbt = 0.12687253952026367
P99 tbt = 0.1285766077041626
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.14262555254266618
P50 ttft = 0.12569355964660645
P99 ttft = 0.27195709705352755
Average tbt = 0.11464141632648225
P50 tbt = 0.13305063247680665
P99 tbt = 0.13503090381622315
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.1561644168127151
P50 ttft = 0.13533353805541992
P99 ttft = 0.3037578773498535
Average tbt = 0.13663228852408274
P50 tbt = 0.13670783042907717
P99 tbt = 0.2179905471801758
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.32705635936171923
P50 ttft = 0.194580078125
P99 ttft = 1.4818120121955867
Average tbt = 0.16198418758533636
P50 tbt = 0.18566539287567208
P99 tbt = 0.2198316535949707
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.6140303812107119
P50 ttft = 0.2901952266693115
P99 ttft = 2.0997689247131346
Average tbt = 0.16339665481022425
P50 tbt = 0.18361277580261295
P99 tbt = 0.21504838800430298
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.09306478500366211
P50 ttft = 0.08262050151824951
P99 ttft = 0.14782774925231937
Average tbt = 0.03453014294306437
P50 tbt = 0.03461978435516357
P99 tbt = 0.03465930557250977
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.10349678993225098
P50 ttft = 0.09775567054748535
P99 ttft = 0.1604548931121826
Average tbt = 0.03524592717488607
P50 tbt = 0.035288238525390626
P99 tbt = 0.035442142486572265
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.11361056736537388
P50 ttft = 0.09988951683044434
P99 ttft = 0.1819802188873291
Average tbt = 0.036142738887241906
P50 tbt = 0.03587484359741211
P99 tbt = 0.037056209564208985
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.11116447099825232
P50 ttft = 0.10016918182373047
P99 ttft = 0.1760767459869385
Average tbt = 0.036708415426859035
P50 tbt = 0.03687729835510254
P99 tbt = 0.03750713348388672
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.12327091693878174
P50 ttft = 0.10930359363555908
P99 ttft = 0.20514865875244137
Average tbt = 0.041244768142700235
P50 tbt = 0.03727688789367676