Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

构建ngram字典 #26

Open
testcross-01 opened this issue Apr 25, 2023 · 8 comments
Open

构建ngram字典 #26

testcross-01 opened this issue Apr 25, 2023 · 8 comments

Comments

@testcross-01
Copy link

您好，我想问一下ZEN模型在构建ngram字典是使用了什么工具？我想在自己的领域的文本上构建一个ngram字典，但不知道如何构建比较好。
@GuiminChen
Copy link
Collaborator

GuiminChen commented Apr 25, 2023 via email

@fany198
Copy link

fany198 commented Jun 13, 2023

您好，我也想知道ZEN模型在构建ngram字典是使用了什么工具？我想在自己的领域的文本上构建一个ngram字典，麻烦您了，我的邮箱是[email protected]

@GuiminChen
Copy link
Collaborator

GuiminChen commented Jun 13, 2023 via email

@shizhediao
Copy link

shizhediao commented Sep 20, 2023 •
edited
Loading

您好，
感谢关注我们的工作，有多种不同的方法可以构建ngram字典。其中一种可以参考这篇文章里用到的PMI方法 (Section 3.1)，相关代码也已开源 https://aclanthology.org/2021.acl-long.259.pdf

@GuiminChen
Copy link
Collaborator

GuiminChen commented Sep 20, 2023 via email

@whsbq
Copy link

whsbq commented Dec 29, 2023

您好，我也想问一下ZEN模型在构建ngram字典是使用了什么工具？我想在自己的领域的文本上构建一个ngram字典，麻烦了

@GuiminChen
Copy link
Collaborator

GuiminChen commented Dec 29, 2023 via email

@shizhediao
Copy link

shizhediao commented Dec 29, 2023 •
edited
Loading

您好，
感谢关注我们的工作，有多种不同的方法可以构建ngram字典。其中一种可以参考这篇文章里用到的PMI方法 (Section 3.1) https://aclanthology.org/2021.acl-long.259.pdf
代码实现在这里：https://github.com/shizhediao/T-DNA/blob/main/TDNA/pmi_ngram.py

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@shizhediao @GuiminChen @testcross-01 @fany198 @whsbq