Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use Robotoff to prefill some nutrition fields #74

Open
Tracked by #70
alexfauquette opened this issue Jul 26, 2022 · 3 comments
Open
Tracked by #70

Use Robotoff to prefill some nutrition fields #74

alexfauquette opened this issue Jul 26, 2022 · 3 comments

Comments

@alexfauquette
Copy link
Member

alexfauquette commented Jul 26, 2022

What

  • The ML server robotoff can extract some information from nutrition texts (not a lot)

Here is the PR that introduced it first time. All the useful information can be found here

Part of

@teolemon teolemon changed the title Use Robotof to refill some inputs Use Robotoff to fill some nutrition inputs Sep 28, 2022
@teolemon teolemon changed the title Use Robotoff to fill some nutrition inputs Use Robotoff to prefill some nutrition fields Sep 28, 2022
@raphael0202
Copy link
Contributor

raphael0202 commented Oct 25, 2024

The nutrient extraction model was deployed and integrated to Robotoff. For every new image, we run the model on it and generate a prediction. An insight is generated if in the extracted nutrient values, at least one value is not present in the current nutrients.

To get the insights:

GET https://robotoff.openfoodfacts.org/api/v1/insights?insight_types=nutrient_extraction&order_by=popularity

There are currently no ordering on this endpoint, but I plan to add a order_by={random,popularity} option. Added and deployed

Nutrient values are in insight.data. It contains:

  • entities: a subset of the extracted entities at different processing steps (raw, aggregated, postprocessed). We only have postprocessed here, it's more useful as debug information. We can ignore this field here.
  • nutrients: a dictionary mapping nutrient name to a dict containing:
    • value: the value to add, without the unit
    • unit: the unit (can be g, mg, µg or null). If it's null, it's because we couldn't extract it from the image (either it's missing, the model was wrong or the OCR result was not good enough). In such case I think we can safely use the "default" unit, which depends on the nutrient (as it's done on Product Opener).
    • score: the entity score. Maybe not really relevant here, as this score is not calibrated (most values are > 0.98).
    • char_start, char_end: start and end character offsets in the original text
    • start, end: start and end word offset in the original text

@teolemon
Copy link
Member

@alexfauquette @raphael0202 since we now have the Nutrition prediction route, I'm bumping this one to P0 (let's be a bit dramatic ;)

@teolemon
Copy link
Member

Also, here's another route pour single barcode insights: openfoodfacts/openfoodfacts-server#10950

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: To Discuss & Validate
Development

No branches or pull requests

4 participants