06.srt

1
00:00:02,867 --> 00:00:04,867
字幕生成：慎独     字母校对：lim

2
00:00:06,295 --> 00:00:07,191
哈喽大家好 

3
00:00:07,191 --> 00:00:07,746
我是那个

4
00:00:07,746 --> 00:00:11,000
正义都能迟到为什么我上班不能迟到的ZOMI

5
00:00:11,000 --> 00:00:13,688
今天呢我要给大家分享的一个内容呢

6
00:00:13,688 --> 00:00:16,000
还是在AI计算体系里面

7
00:00:16,000 --> 00:00:19,766
今天讲的比较特别是bitwidth

8
00:00:19,766 --> 00:00:22,275
比特位或者叫做比特位宽

9
00:00:22,275 --> 00:00:24,132
其实呢我们现在呢

10
00:00:24,132 --> 00:00:27,275
还是在AI计算体系这个内容里面

11
00:00:27,275 --> 00:00:28,962
现在呢主要是在

12
00:00:28,962 --> 00:00:32,073
计算体系和矩阵运算这个大内容

13
00:00:32,073 --> 00:00:33,975
那这里面呢我们分开了四个内容

14
00:00:33,975 --> 00:00:37,295
之前我们讲了AI芯片的关键指标

15
00:00:37,295 --> 00:00:39,791
然后给大家分享了AI计算体系里面

16
00:00:39,791 --> 00:00:42,575
具体的计算的最简单的单元

17
00:00:42,575 --> 00:00:43,894
矩阵的运算

18
00:00:43,894 --> 00:00:47,275
现在呢我们来看看比特的位宽

19
00:00:48,107 --> 00:00:50,475
其实呢我这里面的有点意思就是

20
00:00:50,475 --> 00:00:52,893
四年前的我是这样的

21
00:00:52,893 --> 00:00:56,838
首先呢我不理解硬件为什么不提供int8的指令

22
00:00:57,113 --> 00:00:58,850
同时我又提出另外一个问题

23
00:00:58,850 --> 00:01:02,850
为什么硬件不支持int8的比特的位数

24
00:01:02,850 --> 00:01:06,387
这个时候呢我就去提出了一个更大的疑问了

25
00:01:06,850 --> 00:01:08,850
我现在搞的量化算法怎么落地啊

26
00:01:08,850 --> 00:01:09,850
怎么去加速呀

27
00:01:09,850 --> 00:01:11,850
你硬件都不支持我的算法

28
00:01:11,850 --> 00:01:13,850
那你硬件是不是得改了

29
00:01:13,850 --> 00:01:15,850
于是呢我就开始找硬件的同事呢

30
00:01:15,850 --> 00:01:19,850
催他们你们赶紧把int4 int8都支持起来

31
00:01:19,850 --> 00:01:21,514
我要混合比特的加速

32
00:01:21,514 --> 00:01:23,850
你的芯片才能够让我更好的利用起来

33
00:01:23,850 --> 00:01:25,850
那四年前的我呢是这样的

34
00:01:27,225 --> 00:01:28,225
好了

35
00:01:28,225 --> 00:01:28,998
讲完那么多

36
00:01:28,998 --> 00:01:30,225
我们来到第二个内容

37
00:01:30,225 --> 00:01:32,225
就是比特的位宽

38
00:01:32,225 --> 00:01:33,225
比特的width

39
00:01:33,850 --> 00:01:35,850
到底硬件遇到什么问题

40
00:01:35,850 --> 00:01:39,524
到底硬件它是怎么去支持比特位宽的

41
00:01:40,525 --> 00:01:42,525
深度学习的计算模式里面呢

42
00:01:42,525 --> 00:01:44,525
给大家讲过的就是量化的原理

43
00:01:44,525 --> 00:01:46,796
我们希望把一些很宽的一些数据

44
00:01:46,796 --> 00:01:48,521
或者一些浮点的数据

45
00:01:48,525 --> 00:01:50,908
约束到我们-127到127

46
00:01:50,908 --> 00:01:54,572
那这个时候我们就可以把很多不同样的数据

47
00:01:54,572 --> 00:01:55,975
都约束到这个范围

48
00:01:55,975 --> 00:01:58,975
同样我们可以把超出的一些数据呢

49
00:01:58,975 --> 00:01:59,975
直接把它丢掉

50
00:01:59,975 --> 00:02:01,975
这也是一种量化的方式

51
00:02:02,525 --> 00:02:05,044
如果大家对量化的技术原理呢

52
00:02:05,044 --> 00:02:06,025
非常感兴趣

53
00:02:06,025 --> 00:02:07,025
也可以去看一下

54
00:02:07,025 --> 00:02:09,025
我之前给大家的一个分享

55
00:02:09,025 --> 00:02:12,025
就是在推理系统里面的模型压缩

56
00:02:12,025 --> 00:02:15,025
特意的去讲到了低比特量化的一些原理

57
00:02:15,025 --> 00:02:17,025
还有在真正训练的时候

58
00:02:17,025 --> 00:02:19,025
我们怎么去做感知量化训练的

59
00:02:19,025 --> 00:02:20,025
那训练完之后呢

60
00:02:20,025 --> 00:02:23,025
我们去看一下怎么去把我们量化后的模型

61
00:02:23,025 --> 00:02:26,025
量化后的数据进行一个真正的部署起来

62
00:02:26,025 --> 00:02:29,025
欢迎大家去回看一下之前的内容

63
00:02:29,025 --> 00:02:30,365
如果都知道了

64
00:02:30,365 --> 00:02:31,281
那没关系

65
00:02:31,281 --> 00:02:32,025
我们下面呢

66
00:02:32,025 --> 00:02:34,240
就其实量化里面有很多种方法 

67
00:02:34,240 --> 00:02:35,350
总结起来呢

68
00:02:35,350 --> 00:02:36,400
主要有三种

69
00:02:36,400 --> 00:02:37,936
第一种是量化训练 

70
00:02:37,936 --> 00:02:38,936
就训练的时候呢

71
00:02:38,936 --> 00:02:40,186
进行量化

72
00:02:40,186 --> 00:02:41,186
第二种呢

73
00:02:41,186 --> 00:02:43,186
就是静态离线量化

74
00:02:43,186 --> 00:02:44,954
我们叫做PTQ static

75
00:02:44,954 --> 00:02:46,067
通过少量的数据

76
00:02:46,067 --> 00:02:47,186
在推理之前

77
00:02:47,186 --> 00:02:48,468
离线转换的时候呢

78
00:02:48,468 --> 00:02:50,186
进行一个简单的量化

79
00:02:50,186 --> 00:02:54,186
找到我们需要进行量化的scale和offset

80
00:02:54,186 --> 00:02:54,890
最后一种呢

81
00:02:54,890 --> 00:02:56,442
就是动态离线量化

82
00:02:56,442 --> 00:02:58,051
我们叫做dynamic PTQ

83
00:02:58,051 --> 00:03:00,186
或者ptq dynamic都行

84
00:03:00,186 --> 00:03:01,186
这种方式呢

85
00:03:01,186 --> 00:03:02,532
用的还是比较少的

86
00:03:02,532 --> 00:03:04,186
大家用的更多的是第一种

87
00:03:04,186 --> 00:03:05,876
还有第二种

88
00:03:06,211 --> 00:03:08,436
讲了量化的简单的原理

89
00:03:08,436 --> 00:03:09,140
不过呢

90
00:03:09,140 --> 00:03:10,436
我们现在想去看看

91
00:03:10,436 --> 00:03:13,436
什么决定比特位宽

92
00:03:13,436 --> 00:03:14,815
到底是什么决定的

93
00:03:14,815 --> 00:03:16,436
所以我们需要去看看

94
00:03:16,436 --> 00:03:18,436
整个在训练的流程

95
00:03:18,436 --> 00:03:19,436
在推理的流程

96
00:03:19,436 --> 00:03:21,436
有哪些数据

97
00:03:21,436 --> 00:03:22,436
这些数据

98
00:03:22,436 --> 00:03:24,436
哪些能够转换成为低比特

99
00:03:24,436 --> 00:03:27,436
哪些不能够转换成为低比特

100
00:03:27,436 --> 00:03:29,184
这个时候对系统的挑战

101
00:03:29,184 --> 00:03:30,436
对我们对AI

102
00:03:30,436 --> 00:03:31,436
对我们对算法的了解

103
00:03:31,436 --> 00:03:33,436
就有要求了

104
00:03:33,436 --> 00:03:34,436
首先呢

105
00:03:34,436 --> 00:03:35,929
我们先要了解一下

106
00:03:35,929 --> 00:03:39,436
比特位宽bit width是怎么定义的

107
00:03:39,436 --> 00:03:40,820
下面有一个图啊

108
00:03:40,820 --> 00:03:41,436
我们可以看到

109
00:03:41,436 --> 00:03:42,436
这个图画了很久啊

110
00:03:42,436 --> 00:03:47,436
我们看到FP32、FP16 到最近比较特殊的TF32

111
00:03:47,436 --> 00:03:49,436
还有比较特殊的BF16

112
00:03:49,436 --> 00:03:50,436
到传统的int32

113
00:03:50,436 --> 00:03:51,436
int16

114
00:03:51,436 --> 00:03:52,762
int8

115
00:03:52,762 --> 00:03:55,436
基本上我们可以看到总比特数呢

116
00:03:55,436 --> 00:03:57,436
是用X加1

117
00:03:57,436 --> 00:03:59,100
再加M

118
00:03:59,100 --> 00:04:00,084
而X呢

119
00:04:00,084 --> 00:04:02,436
具体的指的就是符号位

120
00:04:02,436 --> 00:04:04,794
到底是正的还是负的

121
00:04:05,050 --> 00:04:09,948
而E呢代表是我们子数的一个动态的范围

122
00:04:09,948 --> 00:04:12,948
也就是二的一次方

123
00:04:12,948 --> 00:04:13,948
这种方式

124
00:04:13,948 --> 00:04:14,844
而M呢

125
00:04:14,844 --> 00:04:16,316
就是小数位了

126
00:04:16,316 --> 00:04:19,452
能够代表我们具体的浮点的精度

127
00:04:19,452 --> 00:04:20,712
通过这种方式

128
00:04:20,712 --> 00:04:21,948
S加1加M

129
00:04:21,948 --> 00:04:25,948
然后得到比特位的位宽

130
00:04:25,948 --> 00:04:26,948
可是

131
00:04:26,948 --> 00:04:28,329
有一点很重要的

132
00:04:28,329 --> 00:04:29,948
就是我们刚才谈到了

133
00:04:29,948 --> 00:04:33,498
我们还是希望能够降低比特的位宽

134
00:04:33,498 --> 00:04:34,217
但是呢

135
00:04:34,217 --> 00:04:36,948
不影响网络模型的训练

136
00:04:36,948 --> 00:04:39,396
不影响我们网络模型的精度

137
00:04:39,396 --> 00:04:40,396
那这个时候呢

138
00:04:40,396 --> 00:04:42,847
我们对硬件就有要求了

139
00:04:42,847 --> 00:04:44,847
我们希望对于MAC

140
00:04:44,847 --> 00:04:47,847
就是累积乘加的操作呢

141
00:04:47,847 --> 00:04:48,847
它的输入输出

142
00:04:48,847 --> 00:04:51,847
能够有效地减少数据的搬运和存储

143
00:04:51,847 --> 00:04:54,847
考虑到使用降低比特位宽

144
00:04:54,847 --> 00:04:56,847
另外第二点就是我们希望 

145
00:04:56,847 --> 00:05:00,167
能够减少MAC的计算和开销的代价

146
00:05:00,167 --> 00:05:01,360
例如int8 x int8 呢

147
00:05:01,360 --> 00:05:02,847
其实我们只要存一个int16

148
00:05:02,847 --> 00:05:04,419
就可以完全解决问题了

149
00:05:04,419 --> 00:05:05,847
但是FP16 x FP16

150
00:05:05,847 --> 00:05:07,847
我们使用FP32来存

151
00:05:07,847 --> 00:05:09,649
这个时候我们整个位宽呢

152
00:05:09,649 --> 00:05:10,649
是大了很多

153
00:05:10,649 --> 00:05:12,649
整个计算的形态也会大很多

154
00:05:12,649 --> 00:05:14,434
整个需要的额外的电路呢

155
00:05:14,434 --> 00:05:15,649
也会大很多

156
00:05:16,445 --> 00:05:19,649
接下来我们看一个非常有意思的表格

157
00:05:19,649 --> 00:05:21,035
降低位宽

158
00:05:21,035 --> 00:05:22,649
有什么好处

159
00:05:22,649 --> 00:05:24,649
那其实好处非常的多

160
00:05:24,649 --> 00:05:25,184
第一个呢

161
00:05:25,184 --> 00:05:27,379
就是我们下面左边的这个图

162
00:05:27,379 --> 00:05:28,275
我们看一下这里面呢

163
00:05:28,275 --> 00:05:31,175
说了是energy的一个cost

164
00:05:31,495 --> 00:05:33,649
功耗的节省

165
00:05:33,649 --> 00:05:34,649
可以看到这里面

166
00:05:34,649 --> 00:05:38,139
大部分都是一些小单位的计算

167
00:05:38,139 --> 00:05:39,649
小单位计算可以看到

168
00:05:39,649 --> 00:05:40,969
最耗时的是什么

169
00:05:40,969 --> 00:05:43,649
最耗时的是IO的读写

170
00:05:43,649 --> 00:05:45,649
对DRAM和SRAM的一个读写

171
00:05:45,649 --> 00:05:47,649
占的时间特别的多

172
00:05:47,649 --> 00:05:49,649
但是我8bit 16bit

173
00:05:49,649 --> 00:05:51,649
这些低比特的

174
00:05:51,649 --> 00:05:54,649
基本上energy的消耗是非常少的

175
00:05:54,649 --> 00:05:57,097
当比特位数更多的时候

176
00:05:57,097 --> 00:06:00,199
一些功耗会越大

177
00:06:00,199 --> 00:06:01,223
就越耗电

178
00:06:01,543 --> 00:06:02,649
这是第一点

179
00:06:02,649 --> 00:06:03,649
第二点呢

180
00:06:03,649 --> 00:06:05,649
就是芯片的面积

181
00:06:05,649 --> 00:06:08,145
或者叫做晶体管的面积

182
00:06:08,145 --> 00:06:10,974
基本上比特位数越少

183
00:06:10,974 --> 00:06:12,974
晶体管的占用面积也就越少

184
00:06:12,974 --> 00:06:14,974
到32Flop的时候呢

185
00:06:14,974 --> 00:06:17,974
晶片的晶体管的面积就会急剧的上升

186
00:06:17,974 --> 00:06:20,412
所以可能单一块芯片里面呢

187
00:06:20,412 --> 00:06:22,460
我用同样的一些制程

188
00:06:22,460 --> 00:06:24,860
我大部分都是塞8bit的时候

189
00:06:24,860 --> 00:06:26,860
可能我就可以塞非常多的计算单元

190
00:06:26,860 --> 00:06:28,433
在某个时钟周期内呢

191
00:06:28,433 --> 00:06:29,860
我可以计算更多的数据

192
00:06:29,860 --> 00:06:32,860
而且还更能够节省电源

193
00:06:32,860 --> 00:06:33,860
更能够省电

194
00:06:33,860 --> 00:06:35,860
基于这个前提之下呢

195
00:06:35,860 --> 00:06:36,860
现在市面上呢

196
00:06:36,860 --> 00:06:38,860
已经推出了很多

197
00:06:38,860 --> 00:06:39,860
假设我在推理的时候呢

198
00:06:39,860 --> 00:06:40,860
使用8bit

199
00:06:40,860 --> 00:06:42,385
但是呢我在训练的时候呢

200
00:06:42,385 --> 00:06:44,860
使用16bit的一些产品

201
00:06:44,860 --> 00:06:45,860
那这个产品呢

202
00:06:45,860 --> 00:06:47,860
就有华为升腾910

203
00:06:47,860 --> 00:06:49,500
确实在推理的时候呢

204
00:06:49,500 --> 00:06:50,500
和训练的时候呢

205
00:06:50,500 --> 00:06:53,837
我们都可以采用很低比特的去训练

206
00:06:53,837 --> 00:06:55,418
让训练呢跑得更快

207
00:06:55,418 --> 00:06:56,418
另外的话

208
00:06:56,418 --> 00:06:57,418
英伟达的A100呢

209
00:06:57,418 --> 00:06:59,418
确实也采用了这种方式

210
00:06:59,418 --> 00:07:01,418
而它就推出了一个Tensor Core

211
00:07:01,418 --> 00:07:04,418
里面有了一个TF32

212
00:07:04,418 --> 00:07:05,787
那实际上TF32呢

213
00:07:05,787 --> 00:07:08,027
它只有19个位宽

214
00:07:08,027 --> 00:07:09,659
并不是32个位宽呢

215
00:07:09,659 --> 00:07:10,659
大家值得注意

216
00:07:12,775 --> 00:07:14,775
在最后的一个过程当中

217
00:07:14,775 --> 00:07:16,775
我们在对AI芯片的设计呢

218
00:07:16,775 --> 00:07:17,775
需要进行一个思考的

219
00:07:17,775 --> 00:07:18,775
第一个

220
00:07:18,775 --> 00:07:20,384
我们降低位宽的时候

221
00:07:20,384 --> 00:07:21,384
一般降低什么呢

222
00:07:21,384 --> 00:07:23,384
我们会降低精度

223
00:07:23,384 --> 00:07:24,384
precision

224
00:07:24,384 --> 00:07:25,384
在网络模型

225
00:07:25,384 --> 00:07:26,384
或者在神经网络里面呢

226
00:07:26,384 --> 00:07:28,384
我们其实在经过训练的过程当中

227
00:07:28,384 --> 00:07:29,384
就发现

228
00:07:29,384 --> 00:07:30,384
最后面的精度

229
00:07:30,384 --> 00:07:32,059
其实我们可以降下来的

230
00:07:32,059 --> 00:07:33,384
因为神经网络

231
00:07:33,384 --> 00:07:35,384
它是有非常大的容错率的

232
00:07:35,384 --> 00:07:36,384
第二个呢

233
00:07:36,384 --> 00:07:39,704
就是降低一个动态范围E

234
00:07:39,704 --> 00:07:41,440
而动态范围降低了

235
00:07:41,440 --> 00:07:43,440
其实我们数据的表达能力

236
00:07:43,440 --> 00:07:44,440
会是降低的

237
00:07:44,440 --> 00:07:45,440
所以这个时候

238
00:07:45,440 --> 00:07:46,440
怎么去设计一款

239
00:07:46,440 --> 00:07:48,440
有用的数据的格式

240
00:07:48,440 --> 00:07:51,988
像英伟达就设计出了自己的TF32

241
00:07:51,988 --> 00:07:54,440
就是对Mbit和Ebit

242
00:07:54,440 --> 00:07:56,440
做了一个很好的权衡

243
00:07:56,440 --> 00:07:57,144
刚才我们说了

244
00:07:57,144 --> 00:07:57,912
这个加起来呢

245
00:07:57,912 --> 00:07:59,064
位宽只有19

246
00:07:59,064 --> 00:08:01,060
1加8加10

247
00:08:01,060 --> 00:08:01,892
很有意思

248
00:08:01,892 --> 00:08:03,084
为什么这么设计呢

249
00:08:03,084 --> 00:08:04,084
我们在这里面呢

250
00:08:04,084 --> 00:08:06,400
就简单的给大家去讲一讲

251
00:08:06,400 --> 00:08:08,784
我的动态范围表示呢

252
00:08:08,784 --> 00:08:11,414
是跟FP32相同的

253
00:08:11,414 --> 00:08:14,784
但是我能够表达的精度范围呢

254
00:08:14,784 --> 00:08:19,784
反倒是跟FP16保持相同的

255
00:08:19,784 --> 00:08:20,784
那这个时候呢

256
00:08:20,784 --> 00:08:22,784
它就能够保持比较好的

257
00:08:22,784 --> 00:08:24,784
动态的范围的位宽

258
00:08:24,784 --> 00:08:27,425
然后也保证了比较好的精度

259
00:08:27,425 --> 00:08:29,281
所以说英伟达呢

260
00:08:29,281 --> 00:08:30,784
就针对A100设计出了

261
00:08:30,784 --> 00:08:34,784
独立设计出FP32的这种位宽

262
00:08:35,109 --> 00:08:36,109
在最后呢

263
00:08:36,109 --> 00:08:38,109
就是对AI芯片设计的

264
00:08:38,109 --> 00:08:40,109
一些个人的思考

265
00:08:40,109 --> 00:08:41,109
首先呢

266
00:08:41,109 --> 00:08:42,109
我们需要考虑到

267
00:08:42,109 --> 00:08:43,399
低的位宽

268
00:08:43,399 --> 00:08:45,447
会不会对网络模型

269
00:08:45,447 --> 00:08:46,447
进行影响

270
00:08:46,447 --> 00:08:47,447
因为呢

271
00:08:47,447 --> 00:08:48,215
不同的网络

272
00:08:48,215 --> 00:08:48,919
不同的数据集

273
00:08:48,919 --> 00:08:50,447
不同的任务

274
00:08:50,447 --> 00:08:51,447
不同的位宽

275
00:08:51,447 --> 00:08:53,215
可能还是有差异的

276
00:08:53,215 --> 00:08:54,447
我举个具体的例子

277
00:08:54,447 --> 00:08:56,447
就是在分类的时候

278
00:08:56,447 --> 00:08:57,447
其实

279
00:08:57,447 --> 00:08:59,447
位宽int8的训练也好

280
00:08:59,447 --> 00:09:00,943
BF16也好

281
00:09:00,943 --> 00:09:03,447
它没有像检测网络模型

282
00:09:03,447 --> 00:09:04,447
这么敏感

283
00:09:04,447 --> 00:09:05,447
这个时候精度的差异

284
00:09:05,447 --> 00:09:06,447
其实是不大的

285
00:09:06,447 --> 00:09:08,447
所以说对精度的影响

286
00:09:08,447 --> 00:09:10,447
可能还跟不同的任务相关

287
00:09:10,447 --> 00:09:13,447
那第二点就是训练和推理的位宽 

288
00:09:13,447 --> 00:09:14,447
是完全不一样的

289
00:09:14,447 --> 00:09:16,447
其实32bit呢

290
00:09:16,447 --> 00:09:18,447
我们只是做一个弱基线

291
00:09:18,447 --> 00:09:19,447
很多时候啊

292
00:09:19,447 --> 00:09:21,447
并不会使用32bit去训练

293
00:09:21,447 --> 00:09:23,447
因为实在是太慢了

294
00:09:23,447 --> 00:09:24,447
而且很耗电

295
00:09:24,447 --> 00:09:25,831
还耗时

296
00:09:25,831 --> 00:09:26,831
那另外呢

297
00:09:26,831 --> 00:09:27,831
我们对于训练呢

298
00:09:27,831 --> 00:09:29,447
可以使用IP16啦

299
00:09:29,447 --> 00:09:30,447
BF16啦

300
00:09:30,447 --> 00:09:31,447
还有TF16

301
00:09:31,447 --> 00:09:32,703
不同的格式

302
00:09:32,703 --> 00:09:34,447
而在推理的时候

303
00:09:34,447 --> 00:09:35,447
我们CV呢

304
00:09:35,447 --> 00:09:37,447
一般是以int8作为推理的

305
00:09:37,447 --> 00:09:38,447
而NLP里面呢

306
00:09:38,447 --> 00:09:39,447
很少以int8

307
00:09:39,447 --> 00:09:41,447
作为主要的推理的位宽

308
00:09:41,447 --> 00:09:42,447
而是以FP16

309
00:09:42,447 --> 00:09:44,790
或者以int8到FP16

310
00:09:44,790 --> 00:09:46,832
这种混合的模式

311
00:09:46,832 --> 00:09:49,447
就是在做大模型过程当中呢

312
00:09:49,447 --> 00:09:51,447
就发现int8跟FP16的混合

313
00:09:51,447 --> 00:09:53,447
其实对大模型的推理

314
00:09:53,447 --> 00:09:55,198
并不是说非常的敏感

315
00:09:55,198 --> 00:09:56,568
而且精度基本上没怎么降

316
00:09:56,568 --> 00:09:57,659
那最后一点呢

317
00:09:57,659 --> 00:09:58,659
也就是第三点

318
00:09:58,659 --> 00:09:59,675
我们还是要权衡

319
00:09:59,675 --> 00:10:02,100
硬件的成本的开销

320
00:10:02,100 --> 00:10:05,484
因为支持额外的数据的位宽呢

321
00:10:05,484 --> 00:10:07,579
我们需要引入更多的电路

322
00:10:07,579 --> 00:10:08,579
这是真的

323
00:10:09,100 --> 00:10:10,100
我们等一下有个图啊

324
00:10:10,100 --> 00:10:11,794
看一下英伟达的图就知道了

325
00:10:11,794 --> 00:10:12,794
那另外一个问题呢

326
00:10:12,794 --> 00:10:15,794
就是新增多少个额外的数据的位宽

327
00:10:15,794 --> 00:10:16,794
比较合适呢

328
00:10:16,794 --> 00:10:19,794
我们经常在一些硬件的指标看到呢

329
00:10:19,794 --> 00:10:21,794
FP16支持多少TFlops

330
00:10:21,794 --> 00:10:23,794
int8支持多少TFlops

331
00:10:23,794 --> 00:10:24,794
FP32呢

332
00:10:24,794 --> 00:10:26,501
支持多少TFlops

333
00:10:26,501 --> 00:10:29,554
就是因为我们需要去增加额外的数据位宽

334
00:10:29,554 --> 00:10:31,602
就会引入更多的电路

335
00:10:31,602 --> 00:10:34,602
我们现在来看看英伟达A100的

336
00:10:34,602 --> 00:10:36,602
这个整体的架构图

337
00:10:36,602 --> 00:10:37,602
然后呢

338
00:10:37,602 --> 00:10:40,858
拿出里面其中一个XM出来

339
00:10:40,858 --> 00:10:42,602
进行一个剖析

340
00:10:42,602 --> 00:10:44,602
可以看到这里面的一个SM啊

341
00:10:44,602 --> 00:10:47,198
就是我们刚才看到的一扎SM里面

342
00:10:47,198 --> 00:10:48,309
我们这里面呢

343
00:10:48,309 --> 00:10:49,602
就有很多int32

344
00:10:49,602 --> 00:10:50,602
FP32

345
00:10:50,602 --> 00:10:51,602
FP64

346
00:10:51,602 --> 00:10:55,602
还有Tensor Core里面的TF32的

347
00:10:55,602 --> 00:10:57,109
不同的计算的格式

348
00:10:57,109 --> 00:10:58,240
里面都是支持的

349
00:10:58,240 --> 00:11:00,525
当然了它里面还可能还是有一些int8

350
00:11:00,525 --> 00:11:01,525
或者其他数据格式

351
00:11:01,525 --> 00:11:03,232
而这里面的int32呢

352
00:11:03,232 --> 00:11:05,845
可能还会拆分成为两个int16

353
00:11:05,845 --> 00:11:08,232
或者四个int8的这种形式

354
00:11:09,270 --> 00:11:13,293
简单地总结一下就是我们在芯片设计之前

355
00:11:13,293 --> 00:11:16,293
我会给大家去讲讲一些很无聊的工作

356
00:11:16,293 --> 00:11:17,884
就是AI的计算体系

357
00:11:17,884 --> 00:11:19,293
希望让大家去了解

358
00:11:19,293 --> 00:11:22,293
去知道整个深度学习的计算模式

359
00:11:22,293 --> 00:11:23,293
是怎么样的

360
00:11:23,293 --> 00:11:24,957
计算体系

361
00:11:24,957 --> 00:11:26,549
又应该长什么样子

362
00:11:26,549 --> 00:11:27,293
于是呢

363
00:11:27,293 --> 00:11:30,293
就引起AI的最重要的计算的方式

364
00:11:30,293 --> 00:11:31,556
卷积

365
00:11:31,556 --> 00:11:34,874
其实可以化成矩阵运算MM

366
00:11:34,874 --> 00:11:36,293
而MM的计算呢

367
00:11:36,293 --> 00:11:38,998
我们可以引入更低比特的位宽

368
00:11:38,998 --> 00:11:40,293
去进行计算

369
00:11:40,293 --> 00:11:43,293
也不会太影响我们训练和推理的精度

370
00:11:43,293 --> 00:11:44,946
那在下一节内容里面呢

371
00:11:44,946 --> 00:11:47,293
我们会去讲讲专用的硬件

372
00:11:47,293 --> 00:11:48,594
那所谓专用硬件呢

373
00:11:48,594 --> 00:11:52,613
就是指我们DOMAIN SPECIFIC ARCHITECTURE

374
00:11:52,613 --> 00:11:54,661
AI芯片的内容呢

375
00:11:54,661 --> 00:11:56,517
慢慢地深入起来了

376
00:11:56,517 --> 00:11:58,677
好了 今天的内容呢 就到这里为止 

377
00:11:58,677 --> 00:11:59,677
谢谢各位

378
00:12:02,900 --> 00:12:03,900
回过头来

379
00:12:03,900 --> 00:12:04,900
四年前的我呢

380
00:12:04,900 --> 00:12:07,495
我们刚才提出了很多一些

381
00:12:07,495 --> 00:12:08,495
argue的情况

382
00:12:08,495 --> 00:12:10,495
或者不理解硬件为什么设计的情况

383
00:12:10,495 --> 00:12:11,495
现在的我呢

384
00:12:11,495 --> 00:12:12,495
是这么想的

385
00:12:12,495 --> 00:12:13,495
其实呢

386
00:12:13,495 --> 00:12:14,495
一切呢

387
00:12:14,495 --> 00:12:16,495
都没有我们想象中的这么简单

388
00:12:16,495 --> 00:12:19,039
在比特位宽的缩减和增加呢

389
00:12:19,039 --> 00:12:20,495
它是一个系统性的工程

390
00:12:20,495 --> 00:12:21,495
我们需要考虑到

391
00:12:21,495 --> 00:12:23,173
AI的整个计算体系

392
00:12:23,173 --> 00:12:24,173
硬件的架构

393
00:12:24,173 --> 00:12:24,998
硬件的成本

394
00:12:24,998 --> 00:12:26,429
还有系统的综合成本

395
00:12:26,429 --> 00:12:28,154
等非常多的问题

396
00:12:28,154 --> 00:12:30,056
所以说一切没有那么简单

397
00:12:30,056 --> 00:12:30,784
现在呢

398
00:12:30,784 --> 00:12:32,495
今天的内容就到这里为止了

399
00:12:32,495 --> 00:12:33,599
谢谢各位