Squashing modifications : ecobalyse codes extraction + tacking issues on

unit tests Tackling recipes.t and attributes.t issues Tackling recipes.t issues Tackling recipes.t + attributes issues : correcting unit tests Tackling ingredients.t issues : correcting unit test Tackling ingredients.t issues : correcting unit test v2 Tackling unit tests for nutrsicore.t/ingredients.t refactor: merge the products and product images directories for OFF, OBF, OPF and OPFF (#10959) In the last few weeks, we normalized product barcodes and deduplicated products that existed in multiple flavors (Open Food Facts, Open Beauty Facts, Open Product Facts and Open Pet Food Facts). We are now going to merge the directories that contain product and product revision data (the .sto files) and the product images. We will keep separate MongoDB databases for each flavor. This will: - make it much easier to move products from one product type to another - remove the possibility of having duplicate products on multiple flavors - make it much easier to have read and write APIs that can be used to retrieve / update products of any type. Deployment plan: - stop OBF, OPF, OPFF for the duration of the migration (a couple of hours) - update OFF code - move products and product images dirs from OBF, OPF, OPFF to the OFF directory structure - change the links to products and products of images on OBF, OPF, OPFF to use the OFF dirs - restart OBF, OPF, OPFF with the new code In this PR: - code updates to check product_type when reading / editing a product for both the website and API. Redirect to the right flavor if the product type is different than the one of the server the request was made to. - changed the product edit form to allow moderators to change the product_type - keep support for moderators to put "obf" etc. in the change code field - tests for the above - small refactor to put into Config.pm things that are the same for all flavors and that are currently duplicated in Config_off.pm etc. Will solve: - #9221 - #7539 - #1174 - #497 - #320 Tackling check perl test taxonomy: Italian for bacon (#11027) Fix Italian for bacon fix: apidoc openApi ecoscore mapping (#11009) - [x] PR title is prefixed by one of the following: feat, fix, docs, style, refactor, test, build, ci, chore, revert, l10n, taxonomy - [x] Code is well documented - [x] Include unit tests for new functionality - [x] Code passes GitHub workflow checks in your branch - [x] If you have multiple commits please combine them into one commit by squashing them. - [x] Read and understood the contribution guidelines This PR introduces several improvements to the API documentation and schema organization: 1. **New Schema File for Country Codes** - Created a dedicated `country-code.yaml` file to centralize country code definitions - Introduced reusable `CountryCode` enum with all supported country codes - Added `CountryValues` schema to standardize country-to-number mappings - Commented out pattern properties in favor of generator-friendly alternatives 2. **Product Ecoscore Schema Improvements** - Refactored country-specific properties to use the new `CountryValues` schema - Added proper type handling for nullable integer fields (`transportation_score`) - Removed redundant pattern properties in favor of schema references - Enhanced type definitions for scores and values 3. **Packaging Schema Enhancements** - Added comprehensive examples for packaging properties - Added missing fields with proper examples: - `number_of_units` - `quantity_per_unit` and related fields - `recycling` - `weight_measured` 4. **Agribalyse Schema Updates** - Added `agribalyse_proxy_food_code` field - Added `agribalyse_food_code` field - Added `co2_agriculture` field The changes improve code organization, reduce duplication, and enhance compatibility with OpenAPI code generators while maintaining backward compatibility with existing data structures. - Fixes #[ISSUE NUMBER] This refactoring improves schema maintainability and provides better type safety through proper enum definitions and standardized patterns for country-specific data. feat: openApi refactor fields parameter to support multiple values with enum validation (getProductByBarCode/search) (#11012) This PR improves the OpenAPI specification for the `fields` parameter in the product and search endpoint. The changes make the API specification more accurate and provide better validation and documentation for the fields parameter, which is critical for controlling response payload size and structure. - Improves API documentation accuracy and completeness - Makes the fields parameter behavior more predictable and better documented - Helps API consumers understand available field options 1. Created new file `parameters/product_available-fields.yaml` to house fields parameter definition 2. Updated main API spec to reference the new parameter definition 3. Removed redundant fields parameter definition from individual endpoints 4. Added comprehensive enum of available field values 5. Added validation rules and examples - Verified that the OpenAPI spec validates correctly - Checked that the parameter definition matches actual API behavior - Confirmed that examples are valid and helpful Note: This change is purely to the API specification and does not affect the actual API implementation. chore: week47 (#11025) Mainly pine nuts clean up --------- Co-authored-by: Arnaud Leene <[email protected]> Tackling integration test docs: openapi for requested product type (#11028) Trying to upate the OpenAPI spec for reading products: - add new product_type parameter - add possible response codes (302 and 404) --------- Co-authored-by: Alex Garel <[email protected]>
openfoodfacts · Nov 20, 2024 · 00dd1db · 00dd1db
1 parent ab65eff
commit 00dd1db
Show file tree

Hide file tree

Showing 89 changed files with 3,349 additions and 535 deletions.
diff --git a/cgi/product_jqm_multilingual.pl b/cgi/product_jqm_multilingual.pl
@@ -56,7 +56,7 @@ =head1 DESCRIPTION
 use ProductOpener::Packaging qw/:all/;
 use ProductOpener::ForestFootprint qw/:all/;
 use ProductOpener::Text qw/remove_tags_and_quote/;
-use ProductOpener::API qw/get_initialized_response/;
+use ProductOpener::API qw/get_initialized_response check_user_permission/;
 use ProductOpener::APIProductWrite qw/skip_protected_field/;
 
 use Apache2::RequestRec ();
@@ -114,6 +114,15 @@ =head1 DESCRIPTION
 		$product_ref = init_product($User_id, $Org_id, $code, $country);
 		$product_ref->{interface_version_created} = $interface_version;
 	}
+	else {
+		# There is an existing product
+		# If the product has a product_type and it is not the product_type of the server, redirect to the correct server
+
+		if ((defined $product_ref->{product_type}) and ($product_ref->{product_type} ne $options{product_type})) {
+			redirect_to_url($request_ref, 307,
+				format_subdomain($subdomain, $product_ref->{product_type}) . '/cgi/product_jqm.pl?code=' . $code);
+		}
+	}
 
 	# Process edit rules
 
@@ -219,25 +228,46 @@ =head1 DESCRIPTION
 		}
 	}
 
-	# 26/01/2017 - disallow barcode changes until we fix bug #677
-	if ($User{moderator} and (defined single_param('new_code'))) {
+	# Change code or product type
 
-		change_product_server_or_code($product_ref, single_param('new_code'), \@errors);
-		$code = $product_ref->{code};
+	if (defined single_param('new_code')) {
 
-		if ($#errors >= 0) {
-			$response{status} = 0;
-			$response{status_verbose} = 'new code is invalid';
+		if (check_user_permission($request_ref, $response_ref, "product_change_code")) {
 
-			my $data = encode_json(\%response);
+			push @errors, change_product_code($product_ref, single_param('new_code'));
+			$code = $product_ref->{code};
+		}
+		else {
+			push @errors, "No permission: product_change_code";
+		}
+	}
 
-			write_cors_headers();
-			print header(-type => 'application/json', -charset => 'utf-8') . $data;
+	if (    (defined single_param("product_type"))
+		and ($product_ref->{product_type} ne single_param("product_type")))
+	{
 
-			exit(0);
+		if (check_user_permission($request_ref, $response_ref, "product_change_product_type")) {
+
+			push @errors, change_product_type($product_ref, single_param("product_type"));
+		}
+		else {
+			push @errors, "No permission: product_change_product_type";
 		}
 	}
 
+	# Display an error message and exit if we have a fatal error (no permission to change barcode or product type, or invalid barcode or product type)
+	if ($#errors >= 0) {
+		$response{status} = 0;
+		$response{status_verbose} = join(",", @errors);
+
+		my $data = encode_json(\%response);
+
+		write_cors_headers();
+		print header(-type => 'application/json', -charset => 'utf-8') . $data;
+
+		exit(0);
+	}
+
 	#my @app_fields = qw(product_name brands quantity);
 	my @app_fields
 		= qw(product_name generic_name quantity packaging brands categories labels origins manufacturing_places emb_codes link expiration_date purchase_places stores countries  );

diff --git a/cgi/product_multilingual.pl b/cgi/product_multilingual.pl
@@ -342,6 +342,15 @@ ($product_ref)
 		if (not defined $product_ref) {
 			display_error_and_exit($request_ref, sprintf(lang("no_product_for_barcode"), $code), 404);
 		}
+		else {
+			# There is an existing product
+			# If the product has a product_type and it is not the product_type of the server, redirect to the correct server
+			# We use a 302 redirect so that browsers issue a GET request to display the form (even if we received a POST request)
+			if ((defined $product_ref->{product_type}) and ($product_ref->{product_type} ne $options{product_type})) {
+				redirect_to_url($request_ref, 302,
+					format_subdomain($subdomain, $product_ref->{product_type}) . '/cgi/product.pl?code=' . $code);
+			}
+		}
 	}
 }
 
@@ -399,11 +408,17 @@ ($product_ref)
 
 	exists $product_ref->{new_server} and delete $product_ref->{new_server};
 
-	# 26/01/2017 - disallow barcode changes until we fix bug #677
-	if ($User{moderator} and (defined single_param("new_code")) and (single_param("new_code") ne "")) {
-
-		change_product_server_or_code($product_ref, single_param("new_code"), \@errors);
-		$code = $product_ref->{code};
+	if ($request_ref->{admin} or $User{moderator}) {
+		if ((defined single_param("new_code")) and (single_param("new_code") ne "")) {
+			change_product_code($product_ref, single_param("new_code"));
+			$code = $product_ref->{code};
+		}
+		if (    (defined single_param("product_type"))
+			and (single_param("product_type") ne "")
+			and ($product_ref->{product_type} ne single_param("product_type")))
+		{
+			change_product_type($product_ref, single_param("product_type"));
+		}
 	}
 
 	my @param_fields = ();
@@ -848,14 +863,12 @@ ($product_ref, $field, $language, $request_ref)
 
 	my $label_new_code = $Lang{new_code}{$lc};
 
-	# 26/01/2017 - disallow barcode changes until we fix bug #677
-	if ($User{moderator}) {
-	}
-
 	$template_data_ref_display->{org_id} = $Org_id;
 	$template_data_ref_display->{label_new_code} = $label_new_code;
 	$template_data_ref_display->{owner_id} = $Owner_id;
 
+	$template_data_ref_display->{product_types} = $options{product_types};
+
 	# obsolete products: restrict to admin on public site
 	# authorize owners on producers platform
 	if ($User{moderator} or $Owner_id) {

diff --git a/docs/api/ref/api-v3.yml b/docs/api/ref/api-v3.yml
@@ -38,6 +38,7 @@ paths:
           schema:
             type: string
             example: "3017620422003"
+        - $ref: "./parameters/requested_product_type.yaml#/components/parameters/RequestedProductType"
         - $ref: "#/components/parameters/cc"
         - $ref: "#/components/parameters/lc"
         - schema:
@@ -84,6 +85,18 @@ paths:
                 allOf:
                   - $ref: ./responses/response-status/response_status.yaml
                   - $ref: ./schemas/product.yaml
+        "302":
+          description: Redirect to the correct server for the product type of the requested product
+          headers:
+            Location:
+              schema:
+                type: string
+                description: URL to the correct server
+              examples:
+                redirect:
+                  value: "https://world.openbeautyfacts.org/api/v3/product/8710447445990"
+        "404":
+          description: Product not found
       description: |-
         Retrieve information for a product with a specific barcode.
 

diff --git a/docs/api/ref/api.yml b/docs/api/ref/api.yml
@@ -47,14 +47,10 @@ paths:
           explode: false
           schema:
             type: string
-            example: "3017620422003"
-        - name: fields
-          in: query
-          description: Specific fields to return. Use 'knowledge_panels' for Knowledge Panels only.
-          required: false
-          schema:
-            type: string
-            enum: [knowledge_panels]
+            examples:
+              - "3017620422003"
+        - $ref: "./parameters/requested_product_type.yaml#/components/parameters/RequestedProductType"
+        - $ref: "#/components/parameters/fields"
       responses:
         "200":
           description: OK
@@ -76,7 +72,18 @@ paths:
                   $ref: "./examples/get_product_by_barcode_spread.yaml"
                 knowledge-panels:
                   $ref: "./examples/get_product_by_barcode_knowledge_panels.yaml" # You'll need to create this example
-
+        "302":
+          description: Redirect to the correct server for the product type of the requested product
+          headers:
+            Location:
+              schema:
+                type: string
+                description: URL to the correct server
+              examples:
+                redirect:
+                  value: "https://world.openbeautyfacts.org/api/v2/product/8710447445990"
+        "404":
+          description: Product not found
   /cgi/product_image_upload.pl:
     post:
       tags:
@@ -645,14 +652,7 @@ components:
         * `created_t`, `last_modified_t`, are about creation and modification dates
         * `nothing`, tells not to sort at all (because if you do not provide the sort_by argument we default to sorting on popularity (for food) or last modification date)
     fields:
-      schema:
-        type: string
-        example: "code,product_name"
-      in: query
-      name: fields
-      description: |
-        The fields to be returned from the product object can also be limited.
-        If not specified, it returns the entire product object response.
+      $ref: ./parameters/product_available_fields.yaml#/components/parameters/ProductAvailableFields
     knowledge_panels_included:
       schema:
         type: string

diff --git a/docs/api/ref/parameters/product_available_fields.yaml b/docs/api/ref/parameters/product_available_fields.yaml
@@ -0,0 +1,17 @@
+components:
+  parameters:
+    ProductAvailableFields:
+      name: fields
+      in: query
+      description: Specific fields to return. Use 'knowledge_panels' for Knowledge Panels only.
+      required: false
+      schema:
+        type: string
+        pattern: "^[a-zA-Z0-9_.-]+(,[a-zA-Z0-9_.-]+)*$"
+        description: Custom comma-separated list of product field names
+        examples:
+          - "knowledge_panels"
+          - "knowledge_panels,product_name,nutriments"
+          - "categories_tags,packaging_tags"
+          - "allergens_tags,nutriscores"
+          - "images,brands_tags,categories_tags,countries_tags,languages_tags,ecoscore_tags"
diff --git a/docs/api/ref/parameters/requested_product_type.yaml b/docs/api/ref/parameters/requested_product_type.yaml
@@ -0,0 +1,24 @@
+components:
+  parameters:
+    RequestedProductType:
+      name: product_type
+      in: query
+      description: >
+        Used for READ queries for one product. Expected product type of the requested product. Defaults to the product type of the server the query is sent to
+        (e.g. 'food' for Open Food Facts, 'beauty' for Open Beauty Facts, etc.). 'all' matches all product types.
+        If the product exists on a different server that matches the requested product type, the API will return a 302 redirect to the correct server.
+        Otherwise, the API will return a 404 error.
+        It is possible that new product types will be added in the future.
+      required: false
+      schema:
+        type: string
+        enum:
+          - "all"
+          - "beauty"
+          - "food"
+          - "petfood"
+          - "product"
+        description: Expected product type of the requested product
+        examples:
+          - "all"
+          - "food"
diff --git a/docs/api/ref/schemas/agribalyse.yaml b/docs/api/ref/schemas/agribalyse.yaml
@@ -1,5 +1,7 @@
 type: object
 properties:
+  agribalyse_proxy_food_code:
+    type: string
   agribalyse_food_code:
     type: string
   co2_agriculture:

diff --git a/docs/api/ref/schemas/ecoscore-country-code.yaml b/docs/api/ref/schemas/ecoscore-country-code.yaml
@@ -0,0 +1,81 @@
+components:
+  schemas:
+    EcoscoreCountryCode:
+      type: string
+      enum:
+        [
+          "ad",
+          "al",
+          "at",
+          "ax",
+          "ba",
+          "be",
+          "bg",
+          "ch",
+          "cy",
+          "cz",
+          "de",
+          "dk",
+          "dz",
+          "ee",
+          "eg",
+          "es",
+          "fi",
+          "fo",
+          "fr",
+          "gg",
+          "gi",
+          "gr",
+          "hr",
+          "hu",
+          "ie",
+          "il",
+          "im",
+          "is",
+          "it",
+          "je",
+          "lb",
+          "li",
+          "lt",
+          "lu",
+          "lv",
+          "ly",
+          "ma",
+          "mc",
+          "md",
+          "me",
+          "mk",
+          "mt",
+          "nl",
+          "no",
+          "pl",
+          "ps",
+          "pt",
+          "ro",
+          "rs",
+          "se",
+          "si",
+          "sj",
+          "sk",
+          "sm",
+          "sy",
+          "tn",
+          "tr",
+          "ua",
+          "uk",
+          "us",
+          "va",
+          "world",
+          "xk",
+        ]
+      # patternProperties not supported by generators
+      #patternProperties:
+      #  (?<country_code>\w\w):
+      #    type: integer
+    EcoscoreCountryValues:
+      type: object
+      propertyNames:
+        $ref: "#/components/schemas/EcoscoreCountryCode"
+      additionalProperties:
+        type: number
+        default: 0