Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Frequent OCR issue: cut words #10019

Open
CharlesNepote opened this issue Mar 25, 2024 · 2 comments
Open

Frequent OCR issue: cut words #10019

CharlesNepote opened this issue Mar 25, 2024 · 2 comments
Labels
🧽 Data quality https://wiki.openfoodfacts.org/Quality ingredients Spellcheck Spellchecking ingredient list and product name to increase ingredient recognition.

Comments

@CharlesNepote
Copy link
Member

CharlesNepote commented Mar 25, 2024 •
edited
Loading

Several ingredients' issues are due to bad OCR results with cut words: leci -thin, concen - trate, konzen - trat, Emul - gator, émul - sifiant, conser - vateur, Ascorbin - säure, Sonnenblu - menöl, natür - liches, etc. Eg. https://world.openfoodfacts.org/product/28389064/euka-menthol?rev=17

For example, if we search for "leci - thine" and its derivatives, we found 35 products with this issue.

I think there are thousands of cases like this. This request allows to detect many errors.

Spell checkers like Language tool detect these issues but do not provide suggestions to fix it.
@CharlesNepote CharlesNepote added 🧽 Data quality https://wiki.openfoodfacts.org/Quality Spellcheck Spellchecking ingredient list and product name to increase ingredient recognition. ingredients labels Mar 25, 2024
@teolemon teolemon moved this to To discuss and validate in 🍊 Open Food Facts Server issues Apr 23, 2024
@github-throwaway
Copy link
Contributor

Maybe this is worth checking out? https://github.com/wolfgarbe/SymSpell
It has ports for many different languages available :)

@github-throwaway
Copy link
Contributor

@CharlesNepote what about a find and replace for " - "? 😄

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
🧽 Data quality https://wiki.openfoodfacts.org/Quality ingredients Spellcheck Spellchecking ingredient list and product name to increase ingredient recognition.
Projects
🍊 Open Food Facts Server issues
Status: To discuss and validate
🧽 Ensuring Data Quality
Status: To do
Milestone
No milestone
Development

No branches or pull requests
2 participants
@CharlesNepote @github-throwaway