Support Robotoff multi-project mode

[raphael0202]

There are discrepancies between O*F flavors (OFF, OBF, OFF-PRO,...) and environments (prod/preprod) that prevent a real multi-project support from Robotoff.

• (most urgent) it looks like we don't perform OCR on all images on the pro platform (ex: https://fr.pro.openfoodfacts.org/images/products/org-lea-nature/345/630/001/6208/3.json does not exist). It looks like it's the case on OBF, I haven't checked the other platforms. We should perform OCR on images of all platforms.
• If possible, we should have JSONL dump available on all platforms, and on preprod (currently we don't dump the database as a JSONL file on preprod even for OFF)
• [after Robotoff deployment] Call Robotoff webhook during product update or new image upload for all platforms (off-pro, obf, opf,...)

Part of

• 🏭 Producers platform (Tracker issue) #5521
• 🧴 Open Beauty Facts (tracker) #5505
• 🐾 Open Pet Food Facts (Tracker) #5536
• 📸 Open Products Facts (Tracker) #5555

[raphael0202]

@manon-corneille @stephanegigandet I ping you here following our conversation at the last weekly meeting.

In order to add Robotoff nutrition image detection capabilities on the pro platform, we need OCR files for the pro platform. From what I see from the server logs, we do generate OCR for the pro platform, but not for "raw" images (1.jpg, 2.jpg,...). Robotoff only uses raw images as input.

JSONL dump shouldn't be difficult to set up, I can take care of it (now that I start to contribute to openfoodfacts-server ;))

On Robotoff side, what needs to be done based on the differences between the pro and public platform (@stephanegigandet tell me if I'm missing something):

• images are stored in a different folder (with a prefix folder named after the organization ID)
• the _id field in mongo contains organization ID + barcode, so we need to provide both when querying mongoDB. Does it actually matter by the way? Are there some duplicated barcodes between organizations?
• does the API work the same way as on the public platform?

[stephanegigandet]

There are duplicated barcodes in the pro platform, because different entities can provide data for the same products. e.g. we can get data from the producer, from some labels (they may send us a list of barcodes of products with their label from many different producers), or from other databases like codeonline or usda. That's why the id is org + barcode. We need to keep this combination of org + barcode on the robotoff side as well.

Note that there may be also experimental data on the pro platform (tests from producers etc.), so that's one more reason to make sure we keep the org in the id of the product.

I don't think that I added API support to the pro platform yet. It might be working if you use a pro platform moderator account and pass some extra parameters to identify the org, but no guarantees. We should add proper support + integration tests etc.

[github-actions]

This issue has been open 90 days with no activity. Can you give it a little love by linking it to a parent issue, adding relevant labels and projets, creating a mockup if applicable, adding code pointers from https://github.com/openfoodfacts/openfoodfacts-server/blob/main/.github/labeler.yml, giving it a priority, editing the original issue to have a more comprehensive description… Thank you very much for your contribution to 🍊 Open Food Facts

[alexgarel]

(most urgent) it looks like we don't perform OCR on all images on the pro platform (ex: https://fr.pro.openfoodfacts.org/images/products/org-lea-nature/345/630/001/6208/3.json does not exist). It looks like it's the case on OBF, I haven't checked the other platforms. We should perform OCR on images of all platforms.

@raphael0202 with the new install, images are now systematically sent to OCR (but it means old photos do not have it yet)