01.srt

1
00:00:00,000 --> 00:00:04,050
Subtitle：PlusV98，校对ZOMI

2
00:00:05,312 --> 00:00:06,425
哈喽大家好

3
00:00:06,425 --> 00:00:09,800
我是上班不更新更新不上班的ZOMI

4
00:00:09,800 --> 00:00:11,223
这里面的所有的视频呢

5
00:00:11,223 --> 00:00:13,600
都是我用业余的时间来去搞的

6
00:00:13,600 --> 00:00:17,100
所以我的本质工作呢不是一位讲师

7
00:00:17,100 --> 00:00:19,200
我是一名研发的工程师

8
00:00:19,200 --> 00:00:20,256
今天呢 

9
00:00:20,256 --> 00:00:22,931
来给大家去汇报AI芯片里面GPU详解

10
00:00:22,931 --> 00:00:26,061
里面最核心 或者ZOMI最关心的一部分

11
00:00:26,061 --> 00:00:28,250
就是Tensor Core的原理

12
00:00:29,875 --> 00:00:32,475
在这个个PPT里面的一共有50多页

13
00:00:32,475 --> 00:00:34,897
也是ZOMI写过最长的一个PPT了

14
00:00:34,897 --> 00:00:37,070
关于AI系统这个系列里面

15
00:00:37,070 --> 00:00:39,732
现在看到整个英伟达的GPU架构

16
00:00:39,732 --> 00:00:42,100
已经来到了最后的两个内容

17
00:00:42,100 --> 00:00:43,300
第一个就是Tensor Core

18
00:00:43,300 --> 00:00:44,600
第二个就是NVLink

19
00:00:44,600 --> 00:00:47,250
而Tensor Core也是整个英伟达GPU架构里面

20
00:00:47,250 --> 00:00:48,825
最核心的一部分

21
00:00:48,825 --> 00:00:52,550
今天会给大家展开三个内容去汇报

22
00:00:52,984 --> 00:00:54,665
第一个内容就是卷积

23
00:00:54,665 --> 00:00:56,183
跟Tensor Core之间的关系

24
00:00:56,183 --> 00:00:59,959
把AI里面的卷积跟Tensor Core硬件结合起来

25
00:00:59,959 --> 00:01:02,733
接着去看看Tensor Core的基本原理

26
00:01:02,733 --> 00:01:05,101
它是怎么去组成怎么去运作的

27
00:01:05,101 --> 00:01:08,365
最后来看看Tensor Core的架构的演进

28
00:01:08,365 --> 00:01:09,553
从Volta架构的Tensor Core

29
00:01:09,553 --> 00:01:11,250
到Hopper架构的Tensor Core

30
00:01:11,250 --> 00:01:12,425
它到底有什么不一样

31
00:01:12,425 --> 00:01:14,225
你以为到这里面就结束了

32
00:01:14,225 --> 00:01:16,275
不是

33
00:01:16,275 --> 00:01:28,767
未知错误几秒钟...

34
00:01:29,425 --> 00:01:33,132
现在呢一起来回顾一下英伟达GPU的整个架构的发展

35
00:01:33,132 --> 00:01:36,525
从2010年的Fermi到2022年的Hopper

36
00:01:36,525 --> 00:01:37,525
经历了十二年

37
00:01:37,525 --> 00:01:39,500
出现了九代的架构

38
00:01:39,500 --> 00:01:41,400
里面呢 从2017年的福特架构开始

39
00:01:41,400 --> 00:01:43,900
就出现了第一代的Tensor Core

40
00:01:43,900 --> 00:01:46,500
从Volta第一代的Tensor Core开始

41
00:01:46,500 --> 00:01:50,252
NV每一个架构都会对Tensor Core进行更新

42
00:01:50,252 --> 00:01:52,852
今天来看看具体这些更新有什么不一样

43
00:01:53,300 --> 00:01:56,325
现在来到了第一个内容

44
00:01:56,325 --> 00:01:58,025
卷积计算

45
00:01:58,025 --> 00:01:59,200
其实在之前

46
00:01:59,200 --> 00:01:59,325
ZOMI的一系列的视频里面

47
00:01:59,325 --> 00:02:01,488
ZOMI的一系列的视频里面

48
00:02:01,488 --> 00:02:03,075
就讲过Kernel的优化

49
00:02:03,075 --> 00:02:03,088
Kernel的优化就去给大家去讲讲

50
00:02:03,088 --> 00:02:05,120
Kernel的优化就去给大家去讲讲

51
00:02:05,120 --> 00:02:07,663
实际上对于卷积神经网络的计算

52
00:02:07,663 --> 00:02:10,916
并不是通过滑窗的方式进行滑动计算

53
00:02:10,916 --> 00:02:13,375
也就是上面的这个图

54
00:02:13,375 --> 00:02:15,390
而真正的计算是通过

55
00:02:15,390 --> 00:02:18,089
是通过CNN或者Image2Col的方式

56
00:02:18,089 --> 00:02:19,889
进行矩阵相乘计算的

57
00:02:19,889 --> 00:02:21,289
有兴趣的朋友可以看回之前

58
00:02:21,289 --> 00:02:23,100
给大家分享的视频

59
00:02:24,350 --> 00:02:25,113
好了好了

60
00:02:25,113 --> 00:02:27,100
现在来到第二个内容

61
00:02:27,100 --> 00:02:29,700
也是正式进入到本节的内容

62
00:02:29,700 --> 00:02:31,600
Tensor Core的基本原理

63
00:02:31,600 --> 00:02:32,100
首先呢

64
00:02:32,100 --> 00:02:35,900
我要提一个问题就是什么是混合精度

65
00:02:35,900 --> 00:02:38,025
ZOMI老师你好啊

66
00:02:38,025 --> 00:02:40,841
混合精度不是指网络模型里面

67
00:02:40,841 --> 00:02:43,017
又有FP16又有FP32吗

68
00:02:44,067 --> 00:02:45,800
这个答案是错的

69
00:02:46,042 --> 00:02:49,358
实际上这里面指的混合精度呢

70
00:02:49,358 --> 00:02:52,399
是指在底层硬件算子层面呢

71
00:02:52,399 --> 00:02:54,799
使用FP32作为输入输出

72
00:02:54,799 --> 00:02:55,899
就input output

73
00:02:55,899 --> 00:02:56,599
然后呢

74
00:02:56,599 --> 00:02:59,625
使用FP32作为中间结果进行存储

75
00:02:59,625 --> 00:03:03,100
从而使得训练过程精度不降低

76
00:03:03,100 --> 00:03:04,866
而这个底层硬件的实现呢

77
00:03:04,866 --> 00:03:06,364
主要就是Tensor Core

78
00:03:06,364 --> 00:03:08,925
可以看到FP16对比FP32

79
00:03:08,925 --> 00:03:11,250
不管是从整数位还是小数位来看

80
00:03:11,250 --> 00:03:13,331
它所表示的范围呢要小很多

81
00:03:14,300 --> 00:03:18,175
现在来到了Volta架构的第一代Tensor Core

82
00:03:18,175 --> 00:03:19,414
可以看到里面SM里面呢

83
00:03:19,414 --> 00:03:21,714
就有了很多个Tensor Core

84
00:03:21,714 --> 00:03:23,914
左边对应的就是CUDA Core

85
00:03:23,914 --> 00:03:26,795
而CUDA Core以前去计算FMA

86
00:03:26,795 --> 00:03:27,773
也就是Fused MatMul

87
00:03:27,773 --> 00:03:28,800
Accumulation呢 底层硬件

88
00:03:28,800 --> 00:03:31,715
需要来回的去把数据搬运

89
00:03:31,715 --> 00:03:33,000
进行一个乘加的时候呢

90
00:03:33,000 --> 00:03:33,079
就要请寄存器

91
00:03:33,079 --> 00:03:34,503
就要请寄存器

92
00:03:34,503 --> 00:03:37,700
到ALU进行层 到寄存器 到ALU进行加

93
00:03:37,700 --> 00:03:39,000
然后到存回ALU

94
00:03:39,000 --> 00:03:40,495
整体来说

95
00:03:40,495 --> 00:03:41,500
要来回的去搬运数据

96
00:03:41,500 --> 00:03:42,000
但是呢

97
00:03:42,000 --> 00:03:44,100
引入了Tensor Core之后呢

98
00:03:44,100 --> 00:03:46,000
提供了可编程的矩阵乘法

99
00:03:46,000 --> 00:03:48,500
和累加独立的单元

100
00:03:48,500 --> 00:03:51,188
专门为AI训练进行加速

101
00:03:51,188 --> 00:03:53,188
每一个SM呢有8组Tensor Core

102
00:03:53,188 --> 00:03:54,575
每一个Tensor Core呢

103
00:03:54,575 --> 00:03:55,875
在每个时钟周期内呢

104
00:03:55,875 --> 00:03:57,716
能执行4×4×4的GEMM

105
00:03:57,716 --> 00:03:59,700
也就是64个FMA

106
00:03:59,700 --> 00:04:00,880
那整体来说呢

107
00:04:00,880 --> 00:04:04,880
就执行A×B加C等于D

108
00:04:04,880 --> 00:04:06,087
这么一个运算

109
00:04:06,087 --> 00:04:06,400
其中呢

110
00:04:06,400 --> 00:04:08,200
ABC和D呢

111
00:04:08,200 --> 00:04:09,800
都是一个4×4的矩阵

112
00:04:09,800 --> 00:04:11,200
矩阵乘法里面的数呢

113
00:04:11,200 --> 00:04:13,400
A和B可以是Fp16

114
00:04:13,400 --> 00:04:16,542
而累加矩阵C和得到的结果D呢

115
00:04:16,542 --> 00:04:18,600
可以是Fp16或者Fp32

116
00:04:18,600 --> 00:04:20,541
因此称底层硬件Tensor Core呢

117
00:04:20,541 --> 00:04:22,141
它是一个混合精度的计算

118
00:04:24,331 --> 00:04:25,949
再打开细一层

119
00:04:25,949 --> 00:04:27,863
其实呢 每个Tensor Core

120
00:04:27,863 --> 00:04:29,819
可以执行64个FMA的

121
00:04:29,819 --> 00:04:30,819
混合精度计算

122
00:04:30,819 --> 00:04:32,032
那SM里面呢

123
00:04:32,032 --> 00:04:33,106
一共有8个Tensor Core

124
00:04:33,106 --> 00:04:34,941
所以每个时钟周期内呢

125
00:04:34,941 --> 00:04:38,341
一共可以执行512个浮点运算的

126
00:04:38,341 --> 00:04:40,497
具体大家可以自己算一算

127
00:04:40,497 --> 00:04:42,309
因此呢 新一代Volta的GPU呢

128
00:04:42,309 --> 00:04:43,019
吞吐量

129
00:04:43,019 --> 00:04:45,489
对于AI的计算矩阵乘和累加呢

130
00:04:45,489 --> 00:04:47,674
会比PASCAL架构的GPU

131
00:04:47,674 --> 00:04:49,438
每个SM的AI吞吐量呢

132
00:04:49,438 --> 00:04:50,438
增加了8倍

133
00:04:50,438 --> 00:04:51,742
总共增加12倍

134
00:04:51,742 --> 00:04:53,642
因为SM会更多嘛

135
00:04:53,642 --> 00:04:55,074
对于矩阵乘的数呢

136
00:04:55,074 --> 00:04:57,126
是Fp16两个矩阵进行相乘

137
00:04:57,126 --> 00:04:59,300
然后进行Fp32的累加

138
00:04:59,300 --> 00:05:00,526
最后存储的时候呢

139
00:05:00,526 --> 00:05:02,826
是进行Fp32的存储方式

140
00:05:03,800 --> 00:05:05,724
但神经网络里面

141
00:05:05,724 --> 00:05:08,124
不仅仅只有一个矩阵乘这么简单

142
00:05:08,124 --> 00:05:10,202
在训练的过程当中呢

143
00:05:10,202 --> 00:05:12,970
就会遇到卷积跟激活进行相乘

144
00:05:12,970 --> 00:05:14,770
得到另外一个新的feature map

145
00:05:14,770 --> 00:05:16,270
另外的话 很重要的一点

146
00:05:16,270 --> 00:05:18,170
看看箭头是逆向过来的

147
00:05:18,897 --> 00:05:20,853
上面两种就是反向传播的时候

148
00:05:20,853 --> 00:05:22,853
权重还有激活层呢

149
00:05:22,853 --> 00:05:24,453
需要进行反向的计算

150
00:05:24,453 --> 00:05:26,997
与卷积的正向呢是刚好相反的

151
00:05:26,997 --> 00:05:29,297
另外呢 还有一种专门针对激活函数

152
00:05:29,297 --> 00:05:30,797
还有激活的梯度呢

153
00:05:30,797 --> 00:05:32,173
进行反向的计算

154
00:05:32,173 --> 00:05:34,187
那整体在计算的过程当中呢

155
00:05:34,187 --> 00:05:36,487
这里面有大量的绿色都是Fp16的

156
00:05:36,487 --> 00:05:37,787
真正存储的时候呢

157
00:05:37,787 --> 00:05:39,416
是使用Fp32进行存储的

158
00:05:39,416 --> 00:05:42,416
这种就是非常明确的混合精度训练

159
00:05:45,200 --> 00:05:47,000
现在来到了第三个内容

160
00:05:47,000 --> 00:05:48,500
也就是我插播的一个内容

161
00:05:48,500 --> 00:05:50,700
Tensor Core跟CUDA Programming

162
00:05:50,700 --> 00:05:52,100
之间的一个关系

163
00:05:53,069 --> 00:05:56,750
在CUDA里面 其实并不是控制每一条弯弯的线呢

164
00:05:56,750 --> 00:05:57,950
进行线程控制的

165
00:05:57,950 --> 00:05:59,350
而是通过控制一个Warp

166
00:05:59,350 --> 00:06:02,016
一个Warp呢就包含很多个线程

167
00:06:02,016 --> 00:06:04,430
同一时间并行并发的去执行

168
00:06:04,430 --> 00:06:06,030
那在真正执行的时候呢

169
00:06:06,030 --> 00:06:08,078
会做一个Warp同步的操作

170
00:06:08,078 --> 00:06:09,778
把所有的线程呢都进行同步

171
00:06:09,778 --> 00:06:11,178
然后获取同样的数据

172
00:06:11,589 --> 00:06:12,301
接着呢

173
00:06:12,301 --> 00:06:15,189
进行一个16x16的矩阵相乘和矩阵计算

174
00:06:15,189 --> 00:06:15,659
最后呢

175
00:06:15,659 --> 00:06:15,689
再把结果呢存储在不同的Warp上面

176
00:06:15,689 --> 00:06:18,029
再把结果呢存储在不同的Warp上面

177
00:06:18,364 --> 00:06:18,914
Warp呢

178
00:06:18,914 --> 00:06:22,014
就是在软件上面做一个大的线程的概念

179
00:06:22,997 --> 00:06:25,044
在CUDA程序执行的过程当中呢

180
00:06:25,044 --> 00:06:27,244
可以通过线程的Warp来去调度

181
00:06:27,244 --> 00:06:28,144
Tensor Core

182
00:06:28,144 --> 00:06:29,544
在一个Warp线程里面呢

183
00:06:29,544 --> 00:06:32,978
通过Tensor Core来提供一个16x16x16的

184
00:06:32,978 --> 00:06:34,178
矩阵运算

185
00:06:34,178 --> 00:06:34,678
哎

186
00:06:34,678 --> 00:06:36,762
刚才的Tensor Core不是4x4x4吗

187
00:06:36,762 --> 00:06:39,462
现在怎么变成16x16x16了

188
00:06:39,648 --> 00:06:40,305
不着急

189
00:06:40,305 --> 00:06:41,605
后面会展开的

190
00:06:42,200 --> 00:06:45,333
在真正CUDA通过Tensor Core进行编程呢

191
00:06:45,333 --> 00:06:47,696
通过Warp来提供CUDA C++

192
00:06:47,696 --> 00:06:50,859
WMMA的API对外提供给开发者

193
00:06:50,859 --> 00:06:52,159
这里面的WMMA呢

194
00:06:52,159 --> 00:06:54,559
主要是专门针对Tensor Core

195
00:06:54,559 --> 00:06:56,259
进行矩阵的加载了

196
00:06:56,259 --> 00:06:56,859
存储了

197
00:06:56,859 --> 00:06:58,155
还有具体的计算

198
00:06:58,155 --> 00:06:59,255
那MMA sync呢

199
00:06:59,255 --> 00:07:00,555
这个就是具体的计算

200
00:07:00,555 --> 00:07:01,455
后面有个sync呢

201
00:07:01,455 --> 00:07:02,994
就是刚才提到的

202
00:07:02,994 --> 00:07:03,794
所有的Warp呢

203
00:07:03,794 --> 00:07:05,294
之间需要进行同步的

204
00:07:05,986 --> 00:07:07,653
其实呢整体看一下

205
00:07:07,653 --> 00:07:09,753
其实刚才提到的Tensor Core是一个

206
00:07:09,753 --> 00:07:11,356
4x4的Tensor Core的核

207
00:07:11,356 --> 00:07:12,305
但实际上呢

208
00:07:12,305 --> 00:07:14,335
一个SM里面有多个Tensor Core

209
00:07:14,335 --> 00:07:16,213
不可能最细粒度的去控制

210
00:07:16,213 --> 00:07:16,900
每一个Tensor Core

211
00:07:16,900 --> 00:07:17,615
这样的效率会很低

212
00:07:17,615 --> 00:07:18,130
于是呢

213
00:07:18,130 --> 00:07:18,630
一个Warp呢

214
00:07:18,630 --> 00:07:21,072
就帮把好几个Tensor Core包装起来

215
00:07:21,182 --> 00:07:25,682
对外提供一个16x16x16的一个Warp level的卷积的指令

216
00:07:26,145 --> 00:07:26,960
那这个指令呢

217
00:07:26,960 --> 00:07:28,335
最后通过MMA sync呢

218
00:07:28,335 --> 00:07:30,300
这个API进行计算

219
00:07:30,797 --> 00:07:32,797
看看具体的CUDA代码

220
00:07:36,455 --> 00:07:38,812
在头上includeMMA以后

221
00:07:38,812 --> 00:07:39,457
namespace nvCUDA

222
00:07:39,457 --> 00:07:40,734
在nvCUDA里面呢

223
00:07:40,734 --> 00:07:41,877
首先要声明有些Fragment

224
00:07:41,877 --> 00:07:43,319
Fragment就是片段

225
00:07:43,319 --> 00:07:44,800
或者存储的数据

226
00:07:45,113 --> 00:07:45,713
这里面呢

227
00:07:45,713 --> 00:07:48,171
就是对外呈现的16x16的一个Warp level

228
00:07:48,171 --> 00:07:50,336
初始化一个输出矩阵

229
00:07:50,336 --> 00:07:50,936
然后呢

230
00:07:50,936 --> 00:07:53,236
从内存里面加载A和B两个矩阵

231
00:07:53,236 --> 00:07:53,936
然后

232
00:07:53,936 --> 00:07:56,036
真正执行WMMA的计算

233
00:07:56,036 --> 00:07:56,836
计算完之后呢

234
00:07:56,836 --> 00:07:58,471
就把结果存储回来

235
00:07:58,471 --> 00:08:00,071
存储到C里面

236
00:08:00,071 --> 00:08:02,097
整个CUDA的底层计算呢

237
00:08:02,097 --> 00:08:03,788
就是这么简单

238
00:08:05,512 --> 00:08:06,868
ZOMI老师你好啊

239
00:08:06,868 --> 00:08:07,716
我现在有个问题啊

240
00:08:07,716 --> 00:08:11,031
就你讲了Tensor Core跟CUDA之间的映射

241
00:08:11,031 --> 00:08:13,077
我大概简单懂了一个16x16的

242
00:08:13,077 --> 00:08:15,477
跟4x4之间是怎么映射

243
00:08:15,477 --> 00:08:16,177
但是呢

244
00:08:16,623 --> 00:08:19,705
像Tensor Core一次提供4x4这么小的一个Kernel

245
00:08:19,705 --> 00:08:22,411
或者16x16这么小的一个Kernel

246
00:08:22,411 --> 00:08:25,111
怎么去处理像……这种input image

247
00:08:25,111 --> 00:08:27,211
就是输入的图像要24x24

248
00:08:27,211 --> 00:08:29,611
Kernel等于7x7的GEMM呢

249
00:08:29,611 --> 00:08:32,711
或者在现在的大模型transformer结构里面呢

250
00:08:32,711 --> 00:08:35,711
一个input embedded就2x2048

251
00:08:35,711 --> 00:08:37,722
hidden size是1024x1024

252
00:08:37,800 --> 00:08:39,589
这么大的一个GEMM

253
00:08:39,589 --> 00:08:42,000
怎么放在英伟达GPU这么小的一个

254
00:08:42,000 --> 00:08:43,600
Tensor Core里面去处理呢

255
00:08:45,230 --> 00:08:45,530
哎

256
00:08:46,204 --> 00:08:48,304
你前面提到的这个问题呢非常有意思

257
00:08:48,304 --> 00:08:49,504
现在看一看了

258
00:08:49,609 --> 00:08:51,275
实际上 刚才提到的所有的卷积计算呢

259
00:08:51,275 --> 00:08:53,275
会变成GEMM矩阵层的方式

260
00:08:53,275 --> 00:08:53,975
那矩阵层呢

261
00:08:53,975 --> 00:08:55,463
现在有一个蓝色的矩阵

262
00:08:55,463 --> 00:08:56,763
和一个黄色的矩阵

263
00:08:56,763 --> 00:08:58,500
两个相乘得到绿色的矩阵

264
00:08:58,500 --> 00:08:59,500
但实际计算的时候呢

265
00:08:59,500 --> 00:09:01,000
会取片段的数据

266
00:09:01,000 --> 00:09:01,689
也就是Fragment

267
00:09:01,689 --> 00:09:03,575
取到了Fragment这条长长的

268
00:09:03,575 --> 00:09:04,475
和横横的

269
00:09:04,682 --> 00:09:06,455
就变成Fragment box

270
00:09:06,455 --> 00:09:07,671
就是线程块

271
00:09:07,671 --> 00:09:09,171
在具体硬件变成线程块

272
00:09:09,171 --> 00:09:10,615
在真正线程块执行的时候呢

273
00:09:10,615 --> 00:09:12,087
就会把这里面的

274
00:09:12,087 --> 00:09:13,487
其中一部分数据

275
00:09:13,487 --> 00:09:14,583
再提取出来

276
00:09:14,583 --> 00:09:16,268
变成Warp level的计算

277
00:09:16,268 --> 00:09:17,997
Warp level的计算呢

278
00:09:17,997 --> 00:09:19,197
其实还是很大了

279
00:09:19,197 --> 00:09:21,005
在真正Fragment执行的时候呢

280
00:09:21,005 --> 00:09:22,343
又会把它变成

281
00:09:22,343 --> 00:09:24,761
满足CUDA和矩阵输入的计算了

282
00:09:24,761 --> 00:09:26,655
因此总结一句话就是

283
00:09:26,655 --> 00:09:28,482
就是从简单的看到矩阵层

284
00:09:28,482 --> 00:09:29,982
到实际上硬件执行的时候呢

285
00:09:29,982 --> 00:09:30,982
会把它变成

286
00:09:30,982 --> 00:09:32,447
会把数据的一部分呢

287
00:09:32,447 --> 00:09:33,947
根据硬件的多级缓存

288
00:09:33,947 --> 00:09:35,747
放在box wrap Thread里面

289
00:09:35,747 --> 00:09:37,447
最终通过线程的box呢

290
00:09:37,447 --> 00:09:39,647
提供Tensor Core的核心的计算

291
00:09:41,800 --> 00:09:43,400
时间来不及了

292
00:09:43,400 --> 00:09:45,000
现在 或者后面的 

293
00:09:45,000 --> 00:09:46,500
Tensor Core的历代的发展

294
00:09:46,500 --> 00:09:48,300
也就是从p100 v100

295
00:09:48,300 --> 00:09:52,384
到Turing到H100里面的Tensor Core的变化

296
00:09:52,384 --> 00:09:53,784
在下个视频

297
00:09:53,784 --> 00:09:55,084
再给大家去分享

298
00:09:55,084 --> 00:09:56,184
今天的内容呢

299
00:09:56,220 --> 00:09:57,020
就到这里为止

300
00:09:57,020 --> 00:09:58,420
简单的回顾一下

301
00:09:58,600 --> 00:10:02,100
在硬件底层里面 所谓的混合进度计算呢

302
00:10:02,100 --> 00:10:04,010
就是指具体计算的时候呢

303
00:10:04,010 --> 00:10:04,933
是用fp16去计算

304
00:10:04,933 --> 00:10:05,933
但是存储的时候呢

305
00:10:05,933 --> 00:10:07,585
是用fp32进行存储

306
00:10:07,585 --> 00:10:09,785
有了Tensor Core的硬件之后呢

307
00:10:09,900 --> 00:10:12,007
在真正CUDA编程的时候呢

308
00:10:12,157 --> 00:10:14,122
会通过Warp把多个Tensor Core里面的线程呢

309
00:10:14,122 --> 00:10:14,866
聚集起来进行计算

310
00:10:14,866 --> 00:10:15,566
那最终呢

311
00:10:15,566 --> 00:10:18,566
对外提供一个16x16x16的 比如mma的api

312
00:10:18,566 --> 00:10:19,766
给到CUDA

313
00:10:20,700 --> 00:10:23,828
因此最终用户看到的是一个16x16的

314
00:10:23,828 --> 00:10:26,728
Warp WMMA的API

315
00:10:26,800 --> 00:10:28,330
针对真正用户的场景

316
00:10:28,330 --> 00:10:29,546
GEMM会非常大

317
00:10:29,546 --> 00:10:30,288
因此呢

318
00:10:30,288 --> 00:10:32,000
会通过多级的缓存

319
00:10:32,000 --> 00:10:34,792
利用数据的局部性 拆分成block

320
00:10:34,792 --> 00:10:35,300
Warp

321
00:10:35,300 --> 00:10:35,392
还有Thread

322
00:10:35,392 --> 00:10:36,200
还有Thread

323
00:10:36,200 --> 00:10:40,227
最终通过Thread呢去提供实际的Tensor Core的运算