-
Notifications
You must be signed in to change notification settings - Fork 0
/
tp4_perf.txt
8305 lines (7800 loc) · 254 KB
/
tp4_perf.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
GPU clocks set to "(gpuClkMin 800, gpuClkMax 800)" for GPU 00000001:00:00.0
Warning: persistence mode is disabled on device 00000001:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 800, gpuClkMax 800)" for GPU 00000002:00:00.0
Warning: persistence mode is disabled on device 00000002:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 800, gpuClkMax 800)" for GPU 00000003:00:00.0
Warning: persistence mode is disabled on device 00000003:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 800, gpuClkMax 800)" for GPU 00000008:00:00.0
Warning: persistence mode is disabled on device 00000008:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 800, gpuClkMax 800)" for GPU 00000009:00:00.0
Warning: persistence mode is disabled on device 00000009:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 800, gpuClkMax 800)" for GPU 0000000A:00:00.0
Warning: persistence mode is disabled on device 0000000A:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 800, gpuClkMax 800)" for GPU 0000000B:00:00.0
Warning: persistence mode is disabled on device 0000000B:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 800, gpuClkMax 800)" for GPU 0000000C:00:00.0
Warning: persistence mode is disabled on device 0000000C:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
All done.
GPU clocks set to "(gpuClkMin 1000, gpuClkMax 1000)" for GPU 00000001:00:00.0
Warning: persistence mode is disabled on device 00000001:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 1000, gpuClkMax 1000)" for GPU 00000002:00:00.0
Warning: persistence mode is disabled on device 00000002:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 1000, gpuClkMax 1000)" for GPU 00000003:00:00.0
Warning: persistence mode is disabled on device 00000003:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 1000, gpuClkMax 1000)" for GPU 00000008:00:00.0
Warning: persistence mode is disabled on device 00000008:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 1000, gpuClkMax 1000)" for GPU 00000009:00:00.0
Warning: persistence mode is disabled on device 00000009:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 1000, gpuClkMax 1000)" for GPU 0000000A:00:00.0
Warning: persistence mode is disabled on device 0000000A:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 1000, gpuClkMax 1000)" for GPU 0000000B:00:00.0
Warning: persistence mode is disabled on device 0000000B:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 1000, gpuClkMax 1000)" for GPU 0000000C:00:00.0
Warning: persistence mode is disabled on device 0000000C:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
All done.
GPU clocks set to "(gpuClkMin 1200, gpuClkMax 1200)" for GPU 00000001:00:00.0
Warning: persistence mode is disabled on device 00000001:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 1200, gpuClkMax 1200)" for GPU 00000002:00:00.0
Warning: persistence mode is disabled on device 00000002:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 1200, gpuClkMax 1200)" for GPU 00000003:00:00.0
Warning: persistence mode is disabled on device 00000003:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 1200, gpuClkMax 1200)" for GPU 00000008:00:00.0
Warning: persistence mode is disabled on device 00000008:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 1200, gpuClkMax 1200)" for GPU 00000009:00:00.0
Warning: persistence mode is disabled on device 00000009:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 1200, gpuClkMax 1200)" for GPU 0000000A:00:00.0
Warning: persistence mode is disabled on device 0000000A:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 1200, gpuClkMax 1200)" for GPU 0000000B:00:00.0
Warning: persistence mode is disabled on device 0000000B:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 1200, gpuClkMax 1200)" for GPU 0000000C:00:00.0
Warning: persistence mode is disabled on device 0000000C:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
All done.
Open ttft file
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.1588092843691508
P50 ttft = 0.1271916627883911
P99 ttft = 0.28654760360717774
Average tbt = 0.039826897780100505
P50 tbt = 0.04000880718231201
P99 tbt = 0.04021225881576538
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.528312603632609
P50 ttft = 0.12157845497131348
P99 ttft = 3.151105546951294
Average tbt = 0.04121562639872233
P50 tbt = 0.041838550567626955
P99 tbt = 0.04198026657104492
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 1.5115797179085868
P50 ttft = 0.12472343444824219
P99 ttft = 4.5105055236816405
Average tbt = 0.042227298191615525
P50 tbt = 0.0428718090057373
P99 tbt = 0.0430106897354126
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.766527681815915
P50 ttft = 0.2713031768798828
P99 ttft = 2.9007131576538088
Average tbt = 0.08567558614219115
P50 tbt = 0.050915670394897465
P99 tbt = 0.22001996994018558
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 1.7186776399612427
P50 ttft = 0.6095216274261475
P99 ttft = 4.759110822677612
Average tbt = 0.05359071922302246
P50 tbt = 0.05359559059143067
P99 tbt = 0.053853557109832764
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 1.010488424450159
P50 ttft = 0.2109602689743042
P99 ttft = 5.733383367061615
Average tbt = 0.08992830365896234
P50 tbt = 0.07591991424560582
P99 tbt = 0.13433375167846678
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.45553744982366695
P50 ttft = 0.21145296096801758
P99 ttft = 3.239098491668701
Average tbt = 0.07417858136843333
P50 tbt = 0.05474820137023926
P99 tbt = 0.1344197177886963
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 5.022464005343886
P50 ttft = 4.658888578414917
P99 ttft = 12.26632921218872
Average tbt = 0.07091324473001873
P50 tbt = 0.07075643539428711
P99 tbt = 0.07238321590423584
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 1.1193672849776897
P50 ttft = 0.714340090751648
P99 ttft = 3.4038302254676815
Average tbt = 0.11907507307986005
P50 tbt = 0.1139422178268438
P99 tbt = 0.23959194326400782
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 2.7717932110741024
P50 ttft = 2.116025447845459
P99 ttft = 5.959805135726929
Average tbt = 0.08925269036065978
P50 tbt = 0.07536931037902832
P99 tbt = 0.11522227668762261
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 1.7231966257095337
P50 ttft = 1.5634559392929077
P99 ttft = 4.5569410061836235
Average tbt = 0.07704044271398473
P50 tbt = 0.07469842433929444
P99 tbt = 0.12911929845809927
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 5.02832310941039
P50 ttft = 4.64072322845459
P99 ttft = 11.602546367645264
Average tbt = 0.08426607476563013
P50 tbt = 0.08417911529541015
P99 tbt = 0.0847976188659668
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 3.662322971750708
P50 ttft = 4.018183588981628
P99 ttft = 8.334538328647614
Average tbt = 0.09667095787384926
P50 tbt = 0.09669013023376466
P99 tbt = 0.09690377712249756
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 1.4665195734604546
P50 ttft = 1.3279242515563965
P99 ttft = 3.182944197654724
Average tbt = 0.09619537885638255
P50 tbt = 0.09616320133209227
P99 tbt = 0.09657735204696656
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.09427491823832194
P50 ttft = 0.08984827995300293
P99 ttft = 0.143162636756897
Average tbt = 0.031893714269002275
P50 tbt = 0.03197293281555176
P99 tbt = 0.03216144847869873
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.32730292138599215
P50 ttft = 0.09865140914916992
P99 ttft = 2.2686394691467284
Average tbt = 0.03269172395978655
P50 tbt = 0.0325502872467041
P99 tbt = 0.0330446720123291
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.17732336180550712
P50 ttft = 0.15509247779846191
P99 ttft = 0.3560395622253418
Average tbt = 0.033611146381923133
P50 tbt = 0.03298087120056153
P99 tbt = 0.03475400066375732
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.23184691987386563
P50 ttft = 0.15133452415466309
P99 ttft = 1.3648874759674094
Average tbt = 0.03525386670740639
P50 tbt = 0.03544607162475586
P99 tbt = 0.036337776184082037
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.21765196323394775
P50 ttft = 0.11504089832305908
P99 ttft = 0.9991362094879127
Average tbt = 0.04034398078918461
P50 tbt = 0.03689298629760743
P99 tbt = 0.09476928710937525
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.23283756896853447
P50 ttft = 0.1371917724609375
P99 ttft = 1.2520387601852372
Average tbt = 0.052889423072338154
P50 tbt = 0.0369318962097168
P99 tbt = 0.11096371841430665
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.3035751270921263
P50 ttft = 0.13100647926330566
P99 ttft = 1.3985197257995632
Average tbt = 0.09768755207323053
P50 tbt = 0.11759934425354004
P99 tbt = 0.11812472343444826
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.11850690267172205
P50 ttft = 0.10531854629516602
P99 ttft = 0.19754825592041003
Average tbt = 0.08342275159904768
P50 tbt = 0.10985336303710938
P99 tbt = 0.11074659156799316
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.1199982800382249
P50 ttft = 0.10510671138763428
P99 ttft = 0.21449137687683106
Average tbt = 0.09255428212754281
P50 tbt = 0.1109269142150879
P99 tbt = 0.11223955392837524
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.1255354200090681
P50 ttft = 0.10689711570739746
P99 ttft = 0.22262450218200677
Average tbt = 0.10038872446332658
P50 tbt = 0.1125211238861084
P99 tbt = 0.11595929718017581
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.12839263015323216
P50 ttft = 0.11177480220794678
P99 ttft = 0.2431162500381469
Average tbt = 0.11038386556837297
P50 tbt = 0.11730477809906006
P99 tbt = 0.15716532611846978
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.14289565246646144
P50 ttft = 0.11742448806762695
P99 ttft = 0.26526232719421383
Average tbt = 0.11494500697159968
P50 tbt = 0.1199483871459961
P99 tbt = 0.17669092845916753
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.14634755779715145
P50 ttft = 0.13460147380828857
P99 ttft = 0.2960523962974549
Average tbt = 0.12178584617726951
P50 tbt = 0.12490301132202153
P99 tbt = 0.19350086688995363
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.15852214979088824
P50 ttft = 0.15401601791381836
P99 ttft = 0.2942268848419188
Average tbt = 0.15394426325093152
P50 tbt = 0.17337598800659226
P99 tbt = 0.24313983869552677
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.08250637849171956
P50 ttft = 0.074485182762146
P99 ttft = 0.1287012577056885
Average tbt = 0.0273141622543335
P50 tbt = 0.027393102645874023
P99 tbt = 0.027573586463928223
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.08545473643711635
P50 ttft = 0.08150076866149902
P99 ttft = 0.13287925720214844
Average tbt = 0.02769116447085426
P50 tbt = 0.02769021987915039
P99 tbt = 0.027928361892700197
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.08899739810398646
P50 ttft = 0.08077383041381836
P99 ttft = 0.13433983325958251
Average tbt = 0.028671456745692662
P50 tbt = 0.02837815284729004
P99 tbt = 0.029600427627563475
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.09302173009732874
P50 ttft = 0.08481764793395996
P99 ttft = 0.1426468372344971
Average tbt = 0.029113682304940585
P50 tbt = 0.029004955291748048
P99 tbt = 0.030320043563842776
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.09820125102996827
P50 ttft = 0.08904027938842773
P99 ttft = 0.15455119848251342
Average tbt = 0.02986242294311525
P50 tbt = 0.029832410812377932
P99 tbt = 0.03046886920928955
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.09643389657139778
P50 ttft = 0.08728194236755371
P99 ttft = 0.15430644989013667
Average tbt = 0.032288828492164634
P50 tbt = 0.030399441719055176
P99 tbt = 0.06962472438812312
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.09828127247013457
P50 ttft = 0.09088587760925293
GPU clocks set to "(gpuClkMin 1400, gpuClkMax 1400)" for GPU 00000001:00:00.0
Warning: persistence mode is disabled on device 00000001:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 1400, gpuClkMax 1400)" for GPU 00000002:00:00.0
Warning: persistence mode is disabled on device 00000002:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 1400, gpuClkMax 1400)" for GPU 00000003:00:00.0
Warning: persistence mode is disabled on device 00000003:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 1400, gpuClkMax 1400)" for GPU 00000008:00:00.0
Warning: persistence mode is disabled on device 00000008:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 1400, gpuClkMax 1400)" for GPU 00000009:00:00.0
Warning: persistence mode is disabled on device 00000009:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 1400, gpuClkMax 1400)" for GPU 0000000A:00:00.0
Warning: persistence mode is disabled on device 0000000A:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 1400, gpuClkMax 1400)" for GPU 0000000B:00:00.0
Warning: persistence mode is disabled on device 0000000B:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 1400, gpuClkMax 1400)" for GPU 0000000C:00:00.0
Warning: persistence mode is disabled on device 0000000C:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
All done.
GPU clocks set to "(gpuClkMin 1600, gpuClkMax 1600)" for GPU 00000001:00:00.0
Warning: persistence mode is disabled on device 00000001:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 1600, gpuClkMax 1600)" for GPU 00000002:00:00.0
Warning: persistence mode is disabled on device 00000002:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 1600, gpuClkMax 1600)" for GPU 00000003:00:00.0
Warning: persistence mode is disabled on device 00000003:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 1600, gpuClkMax 1600)" for GPU 00000008:00:00.0
Warning: persistence mode is disabled on device 00000008:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 1600, gpuClkMax 1600)" for GPU 00000009:00:00.0
Warning: persistence mode is disabled on device 00000009:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 1600, gpuClkMax 1600)" for GPU 0000000A:00:00.0
Warning: persistence mode is disabled on device 0000000A:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 1600, gpuClkMax 1600)" for GPU 0000000B:00:00.0
Warning: persistence mode is disabled on device 0000000B:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
GPU clocks set to "(gpuClkMin 1600, gpuClkMax 1600)" for GPU 0000000C:00:00.0
Warning: persistence mode is disabled on device 0000000C:00:00.0. See the Known Issues section of the nvidia-smi(1) man page for more information. Run with [--help | -h] switch to get more information on how to enable persistence mode.
All done.
P99 ttft = 0.1545371723175049
Average tbt = 0.04599278985637512
P50 tbt = 0.031156444549560548
P99 tbt = 0.09752441215515137
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.09899060984691942
P50 ttft = 0.09105300903320312
P99 ttft = 0.16379121303558344
Average tbt = 0.06666645773922114
P50 tbt = 0.07080106735229547
P99 tbt = 0.09811556339263916
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.10348967542039587
P50 ttft = 0.09354615211486816
P99 ttft = 0.1781457591056823
Average tbt = 0.07630932584721997
P50 tbt = 0.09747264385223389
P99 tbt = 0.09892292833328246
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.10559542065575009
P50 ttft = 0.09404778480529785
P99 ttft = 0.18589090347290038
Average tbt = 0.08509944688706175
P50 tbt = 0.09887733459472657
P99 tbt = 0.09991239356994629
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.11103554125185366
P50 ttft = 0.09728348255157471
P99 ttft = 0.18814789533615112
Average tbt = 0.09317259788513187
P50 tbt = 0.09989321231842041
P99 tbt = 0.10329422330856325
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.1133615008923186
P50 ttft = 0.09874868392944336
P99 ttft = 0.21730969429016114
Average tbt = 0.09232812448709952
P50 tbt = 0.1036679744720459
P99 tbt = 0.10558046722412111
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.12818794040118947
P50 ttft = 0.10507857799530029
P99 ttft = 0.31136155128479026
Average tbt = 0.09866785336943239
P50 tbt = 0.1060746192932129
P99 tbt = 0.1732333564758301
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.12534033388331317
P50 ttft = 0.11126065254211426
P99 ttft = 0.24496361255645752
Average tbt = 0.10892492826434166
P50 tbt = 0.10788989067077637
P99 tbt = 0.17484940338134766
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.07160363594690959
P50 ttft = 0.0647895336151123
P99 ttft = 0.11178372144699099
Average tbt = 0.02483789920806885
P50 tbt = 0.024871635437011718
P99 tbt = 0.025163092613220212
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.07421331178574335
P50 ttft = 0.06838369369506836
P99 ttft = 0.11935563087463379
Average tbt = 0.025292030970255532
P50 tbt = 0.02520718574523926
P99 tbt = 0.02583240509033203
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.08015779767717635
P50 ttft = 0.07385706901550293
P99 ttft = 0.12246032714843748
Average tbt = 0.026055430003574916
P50 tbt = 0.025885534286499024
P99 tbt = 0.02683764171600342
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.07944400136063738
P50 ttft = 0.07434296607971191
P99 ttft = 0.11836442947387696
Average tbt = 0.026387520534236264
P50 tbt = 0.026171159744262696
P99 tbt = 0.02763129234313965
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.08426258563995362
P50 ttft = 0.07787740230560303
P99 ttft = 0.16095290660858141
Average tbt = 0.027204926490783682
P50 tbt = 0.027222800254821777
P99 tbt = 0.02778627157211304
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.08245114982128143
P50 ttft = 0.076629638671875
P99 ttft = 0.12835107088088987
Average tbt = 0.028084243834018707
P50 tbt = 0.027559971809387206
P99 tbt = 0.040288227558136075
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.08242540490137387
P50 ttft = 0.07765841484069824
P99 ttft = 0.13115894317626955
Average tbt = 0.036003657563092005
P50 tbt = 0.02797532081604004
P99 tbt = 0.08544711685180664
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.08919636312737522
P50 ttft = 0.08016777038574219
P99 ttft = 0.2828477430343622
Average tbt = 0.04683812899761901
P50 tbt = 0.029525899887084962
P99 tbt = 0.08626814460754395
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.08744096502344659
P50 ttft = 0.07983791828155518
P99 ttft = 0.15150212526321402
Average tbt = 0.05673802812048738
P50 tbt = 0.06293985843658495
P99 tbt = 0.0864534273147583
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.0881032943725586
P50 ttft = 0.08024907112121582
P99 ttft = 0.15917734146118154
Average tbt = 0.07089055651710154
P50 tbt = 0.08622193336486816
P99 tbt = 0.08722230720520019
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.09071194684063946
P50 ttft = 0.0848991870880127
P99 ttft = 0.1584703493118286
Average tbt = 0.0799343237170467
P50 tbt = 0.0873417615890503
P99 tbt = 0.08827130794525147
96
256
600
96
256
600
96
256
600
467 - 96
Average ttft = 0.09716472505521373
P50 ttft = 0.0826418399810791
P99 ttft = 0.21749194145202622
Average tbt = 0.0787613303721452
P50 tbt = 0.08831973075866699
P99 tbt = 0.08921475124359131
96
256
600
96
256
600
96
256
600
467 - 96