FEAT: support sparse vector for bge-m3 #2540
Conversation
|
For For bge-m3, it's too large for CI, we can test it manually. |
OK |
…o main FEAT: add test_convert_ids_to_tokens for convert_ids_to_tokens
|
About reloading in |
…nvert_ids_to_tokens. Added _encode_bgem3 method for bge-m3 model encoding
…nvert_ids_to_tokens. Added _encode_bgem3 method for bge-m3 model encoding
|
您好， 输出结果 如何生成稀疏向量呢？是在部署时添加配置么？ |
参考bge-m3在huggingface中的官方示例，当需要返回稀疏向量时，需指定 |
|
感谢回复，官方样例用的是FlagEmbedd from FlagEmbedding import BGEM3FlagModel
model = BGEM3FlagModel('BAAI/bge-m3', use_fp16=True) # Setting use_fp16 to True speeds up computation with a slight performance degradation
sentences_1 = ["What is BGE M3?", "Defination of BM25"]
sentences_2 = ["BGE M3 is an embedding model supporting dense retrieval, lexical matching and multi-vector interaction.",
"BM25 is a bag-of-words retrieval function that ranks a set of documents based on the query terms appearing in each document"]
output_1 = model.encode(sentences_1, return_dense=True, return_sparse=True, return_colbert_vecs=False)
output_2 = model.encode(sentences_2, return_dense=True, return_sparse=True, return_colbert_vecs=False)
# you can see the weight for each token:
print(model.convert_id_to_token(output_1['lexical_weights']))
# [{'What': 0.08356, 'is': 0.0814, 'B': 0.1296, 'GE': 0.252, 'M': 0.1702, '3': 0.2695, '?': 0.04092},
# {'De': 0.05005, 'fin': 0.1368, 'ation': 0.04498, 'of': 0.0633, 'BM': 0.2515, '25': 0.3335}]
# compute the scores via lexical mathcing
lexical_scores = model.compute_lexical_matching_score(output_1['lexical_weights'][0], output_2['lexical_weights'][0])
print(lexical_scores)
# 0.19554901123046875
print(model.compute_lexical_matching_score(output_1['lexical_weights'][0], output_1['lexical_weights'][1]))
# 0.0请问有没有请求的样例呢？ |
|
加载 bge-m3 选择 hybrid_mode=True，界面是额外选项，命令行是 --hybrid_mode True。 然后请求时加 return_sparse=True。 @lmolhw5252 试验下是不是可以，ok 的话我们加个文档。 |
这样不行
|
|
没有这个参数，页面内加载hybrid_mode=True会报错
| |
李宏伟
|
|
***@***.***
|
…---- Replied Message ----
| From | Xuye ***@***.***> |
| Date | 12/5/2024 12:14 |
| To | ***@***.***> |
| Cc | ***@***.***> ,
***@***.***> |
| Subject | Re: [xorbitsai/inference] FEAT: support sparse vector for bge-m3 (PR #2540) |
加载 bge-m3 选择 hybrid_mode=True，界面是额外选项，命令行是 --hybrid_mode True。
然后请求时加 return_sparse=True。
@lmolhw5252 试验下是不是可以，ok 的话我们加个文档。
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you were mentioned.Message ID: ***@***.***>
|
|
是 1.0.1 吗？ |
|
|
通过restful请求，如何生成sparse向量？在body中封装了return_sparse:true 不生效 |
|
加载 bge-m3 的时候需要指定 hybrid_mode，值为 True。 |
|
在可视化页面里面可以设置吗？
| |
李宏伟
|
|
***@***.***
|
…---- Replied Message ----
| From | Xuye ***@***.***> |
| Date | 12/16/2024 18:29 |
| To | ***@***.***> |
| Cc | ***@***.***> ,
***@***.***> |
| Subject | Re: [xorbitsai/inference] FEAT: support sparse vector for bge-m3 (PR #2540) |
加载 bge-m3 的时候需要指定 hybrid_mode，值为 True。
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you were mentioned.Message ID: ***@***.***>
|
|
可以，additional 选项里设置。 |
|
报错
| |
李宏伟
|
|
***@***.***
|
…---- Replied Message ----
| From | Xuye ***@***.***> |
| Date | 12/16/2024 18:31 |
| To | ***@***.***> |
| Cc | ***@***.***> ,
***@***.***> |
| Subject | Re: [xorbitsai/inference] FEAT: support sparse vector for bge-m3 (PR #2540) |
可以，additional 选项里设置。
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you were mentioned.Message ID: ***@***.***>
|
|
升级 1.1.0 了吗？有报错贴报错， |
|
1.0.1版本，我更新下试试，谢谢
| |
李宏伟
|
|
***@***.***
|
…---- Replied Message ----
| From | Xuye ***@***.***> |
| Date | 12/16/2024 18:42 |
| To | ***@***.***> |
| Cc | ***@***.***> ,
***@***.***> |
| Subject | Re: [xorbitsai/inference] FEAT: support sparse vector for bge-m3 (PR #2540) |
升级 1.1.0 了吗？有报错贴报错，
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you were mentioned.Message ID: ***@***.***>
|
|
能在一个请求中同时返回 sparse和dense 吗？ |
不行。目前只能分两个请求。 |
支持bge-m3模型的稀疏向量生成功能，
调用方式为：
model.create_embedding(text, return_sparse=True)新增convert_ids_to_tokens方法
该方法可将token_id转换为人类可读文字，调用方式为
该方法返回类型为List[str]，如传入List[str]，将按顺序返回值
Fixes #2527 .