bert_t5_gpt
-
重点看下，decoder 部分的 Multi-head attention 其实是 Masked 的，见图中最右侧的下三角矩阵
- 这也是 GPT（decoder-only）的模型架构所采用的方式
-
post vs. pre LayerNorm
-
Notifications
You must be signed in to change notification settings - Fork 10
License
chunhuizhang/bert_t5_gpt
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
About
No description, website, or topics provided.
Resources
License
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published