This is a module build based on Albert with tensorflow2.0. I use Albert to train a Tinybert. I convert albert ckpt from tensorflow1.0 to tensorflow2.0 which adapt to this projcet. The albert and tinybert co-traing code in file run_pretraining.py. This projcet is done by 2020.3, if someone need the code-use-guide, please leave the issue, I'd like to write a document.
-
Notifications
You must be signed in to change notification settings - Fork 0
License
ahzz1207/ALBERT-TINYBERT
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
About
No description, website, or topics provided.
Resources
License
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published