Disambiguation of the glosses by tones and part of the speech finalized #48
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…rity for the learning of the tone which in its last modification, necessitated data structures more adapted than those we inherited the source code of NLTK. In the present state, we verify that learning the part of discourse, and its disambiguation has not been altered following the introduction of the beginning of this architectural change.
…because the compound tokens of syllables are sent individually to the label, this had led to poor accuracy.
…form in differential_tone_coding.py and an adjustment for reordering lists.
…al to the code separator '_' , the encoder will detect a error its code checker, for the moment we exclude this character from the encoding process which generates the learning data in order not to annoy the coder.
… can now be coded as a character to be deleted or inserted thanks to the implementation of a split2 function.
… 4, using the zipFile module.
…tone learning and prediction based on a subset of caracters (only on diacritic caracters or only on non-diacritic caracters).
… added options. In the previous commit, we learn still on all the caracters even with option --diacritic_only or --non_diacritic_only.
…on 1% of total corpora.
|
In addition to some bugfix, the current pulling carry out two options allowing to precise two partially tone modelings (the one treating of the subset of all diacritics characters, and another one focusing on all non diacritics characters). |
…alue is positive, which specify a syllabification when the value is negative. Otherwise, the segmentation is bypassed.
…it's like a corpora cleaning function)
…on the specified (with --evalsize) portion of corpora
…nor filter neither edit operation decomposition
… post-processing more esay
…list(list([token_non_marked, token_marked]))
…trée en list(list([token_non_marked, token_marked]))" This reverts commit e3c523e.
…djustement" This reverts commit 38a9570.
…isplay adjustement"" This reverts commit 7d0299b.
This reverts commit 6d1d5c8.
This reverts commit 5efe5f4.
This reverts commit e3f45b3.
|
Minors bugfix |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The disambiguation of the glosses by tones and part of the speech is finalized after some adjustments during the test.