-
Notifications
You must be signed in to change notification settings - Fork 0
/
DESCRIPTION
51 lines (51 loc) · 1.42 KB
/
DESCRIPTION
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
Package: morphemepiece
Type: Package
Title: Morpheme Tokenization
Version: 1.2.3
Authors@R: c(
person(given = "Jonathan",
family = "Bratt",
role = c("aut", "cre"),
email = "[email protected]",
comment = c(ORCID = "0000-0003-2859-0076")),
person(given = "Jon",
family = "Harmon",
role = c("aut"),
email = "[email protected]",
comment = c(ORCID = "0000-0003-4781-4346")),
person(given = "Bedford Freeman & Worth Pub Grp LLC DBA Macmillan Learning",
role = c("cph"))
)
Description: Tokenize text into morphemes. The morphemepiece algorithm uses a
lookup table to determine the morpheme breakdown of words, and falls back on a
modified wordpiece tokenization algorithm for words not found in the lookup
table.
URL: https://github.com/macmillancontentscience/morphemepiece
BugReports: https://github.com/macmillancontentscience/morphemepiece/issues
License: Apache License (>= 2)
Encoding: UTF-8
RoxygenNote: 7.1.2
Roxygen: list(markdown = TRUE)
Imports:
dlr (>= 1.0.0),
fastmatch,
magrittr,
memoise (>= 2.0.0),
morphemepiece.data,
piecemaker (>= 1.0.0),
purrr (>= 0.3.4),
readr,
rlang,
stringr (>= 1.4.0)
Suggests:
dplyr,
fs,
ggplot2,
here,
knitr,
remotes,
rmarkdown,
testthat (>= 3.0.0),
utils
VignetteBuilder: knitr
Config/testthat/edition: 3