You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The nutrient extraction model was deployed and integrated to Robotoff. For every new image, we run the model on it and generate a prediction. An insight is generated if in the extracted nutrient values, at least one value is not present in the current nutrients.
To get the insights:
GET https://robotoff.openfoodfacts.org/api/v1/insights?insight_types=nutrient_extraction?barcode={BARCODE}
Nutrient values are in insight.data. It contains:
entities: a subset of the extracted entities at different processing steps (raw, aggregated, postprocessed). We only have postprocessed here, it's more useful as debug information. We can ignore this field here.
nutrients: a dictionary mapping nutrient name to a dict containing:
value: the value to add, without the unit
unit: the unit (can be g, mg, µg or null). If it's null, it's because we couldn't extract it from the image (either it's missing, the model was wrong or the OCR result was not good enough). In such case I think we can safely use the "default" unit, which depends on the nutrient (as it's done on Product Opener).
score: the entity score. Maybe not really relevant here, as this score is not calibrated (most values are > 0.98).
char_start, char_end: start and end character offsets in the original text
start, end: start and end word offset in the original text
We would go for an "Extract Nutrition Facts" button, very much like ingredients, only enabled if we have a precomputer nutrition insight already available (otherwise greyed out). That's a way to keep the user in charge, and avoid potential backlash on "unwanted/unhelpful help" from Robotoff.
Additional values provided by the model would be orange, unchanged values (also provided by the model) would stay normal.
In the future, We might nudge the user for a new photo if we deem it too old, but the result won't be instant (photo, bg task, inference, reloading). We might use animations to provide hints that new values arrived.
Problem
GET https://robotoff.openfoodfacts.org/api/v1/insights?insight_types=nutrient_extraction?barcode={BARCODE}
Nutrient values are in
insight.data
. It contains:entities
: a subset of the extracted entities at different processing steps (raw
,aggregated
,postprocessed
). We only havepostprocessed
here, it's more useful as debug information. We can ignore this field here.nutrients
: a dictionary mapping nutrient name to a dict containing:value
: the value to add, without the unitunit
: the unit (can beg
,mg
,µg
ornull
). If it'snull
, it's because we couldn't extract it from the image (either it's missing, the model was wrong or the OCR result was not good enough). In such case I think we can safely use the "default" unit, which depends on the nutrient (as it's done on Product Opener).score
: the entity score. Maybe not really relevant here, as this score is not calibrated (most values are > 0.98).char_start
,char_end
: start and end character offsets in the original textstart
,end
: start and end word offset in the original textWe would go for an "Extract Nutrition Facts" button, very much like ingredients, only enabled if we have a precomputer nutrition insight already available (otherwise greyed out). That's a way to keep the user in charge, and avoid potential backlash on "unwanted/unhelpful help" from Robotoff.
Additional values provided by the model would be orange, unchanged values (also provided by the model) would stay normal.
In the future, We might nudge the user for a new photo if we deem it too old, but the result won't be instant (photo, bg task, inference, reloading). We might use animations to provide hints that new values arrived.
(part of this bug courtesy of @raphael0202 )
Part of
The text was updated successfully, but these errors were encountered: