From 75c6a6065523a09ab0b0f4f351fe2f6d37b9db50 Mon Sep 17 00:00:00 2001 From: Quentin Meeus Date: Mon, 21 Sep 2020 15:30:57 +0200 Subject: [PATCH] Update the number of parameters As calculated by: ```python from transformers import AutoModel model = AutoModel.from_pretrained("bert-base-multilingual-cased") print(f"{sum(p.numel() for p in model.parameters()):,}") ``` --- multilingual.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/multilingual.md b/multilingual.md index 13842579f..392b21f41 100644 --- a/multilingual.md +++ b/multilingual.md @@ -5,11 +5,11 @@ more single-language models, but we may release `BERT-Large` versions of these two in the future: * **[`BERT-Base, Multilingual Cased (New, recommended)`](https://storage.googleapis.com/bert_models/2018_11_23/multi_cased_L-12_H-768_A-12.zip)**: - 104 languages, 12-layer, 768-hidden, 12-heads, 110M parameters + 104 languages, 12-layer, 768-hidden, 12-heads, 178M parameters * **[`BERT-Base, Multilingual Uncased (Orig, not recommended)`](https://storage.googleapis.com/bert_models/2018_11_03/multilingual_L-12_H-768_A-12.zip)**: - 102 languages, 12-layer, 768-hidden, 12-heads, 110M parameters + 102 languages, 12-layer, 768-hidden, 12-heads, 178M parameters * **[`BERT-Base, Chinese`](https://storage.googleapis.com/bert_models/2018_11_03/chinese_L-12_H-768_A-12.zip)**: - Chinese Simplified and Traditional, 12-layer, 768-hidden, 12-heads, 110M + Chinese Simplified and Traditional, 12-layer, 768-hidden, 12-heads, 178M parameters **The `Multilingual Cased (New)` model also fixes normalization issues in many