feat: use wordpress for static pages

.env

@@ -67,10 +67,13 @@ ELASTICSEARCH_HOSTS=
 LOG_LEVEL_ROOT=TRACE
 LOG_LEVEL_MONGODB=TRACE
 LOG_LEVEL_RATE_LIMITER=TRACE
+# those are for producer platform, to connect to odoo
 ODOO_CRM_URL=
 ODOO_CRM_DB=
 ODOO_CRM_USER=
 ODOO_CRM_PASSWORD=
+# contents taken from wordpress
+WORDPRESS_URL=https://off:[email protected]
 
 # No need to have it running, it's just to compose a URL
 # you might also use the .net service

.github/labeler.yml

@@ -1516,11 +1516,19 @@ Averages by categories:
   - any-glob-to-any-file: 'scripts/create_small_categories_nutriments_per_country_for_testing.pl'
 
 # This lives at https://github.com/openfoodfacts/openfoodfacts-web. 
-# We'd like to use a CMS like wordpress instead. 
-# Make sure to clone it if you want the content page to work on your local instance.
+# We're migrating pages progressively to our Wordpress-based CMS. 
+# Make sure to clone openfoodfacts-web it if you want the content page to work on your local instance.
+# TODO: explain how you should do for Wordpress
 Static content:
 - changed-files:
   - any-glob-to-any-file: 'scripts/copy_text_files.pl'
+  - any-glob-to-any-file: 'docs/dev/explain-website-contents.md'
+  - any-glob-to-any-file: 'docs/dev/how-to-setup-wordpress.md'
+  - any-glob-to-any-file: 'html/css/wp.css'
+  - any-glob-to-any-file: 'lib/ProductOpener/CMS.pm'
+  - any-glob-to-any-file: 'templates/web/pages/content/menu.tt.html'
+  - any-glob-to-any-file: 'templates/web/pages/content/wordpress_content.tt.html'
+  - any-glob-to-any-file: 'tests/unit/cms.t'
 
 # Matomo is our analytics system. We need to fetch matomo logs from the mobile app to generate scan statistics. We used to do that with server side logs.
 # We also want to add Matomo events on the key part of the Web experience (see tracking issue for details)

.stylelintrc.json

@@ -7,5 +7,8 @@
     "no-descending-specificity": null,
     "scss/at-extend-no-missing-placeholder": null,
     "no-invalid-position-at-import-rule": null
-  }
+  },
+  "ignoreFiles": [
+    "html/css/wp.css"
+  ]
 }

Dockerfile

@@ -79,7 +79,13 @@ RUN --mount=type=cache,id=apt-cache,target=/var/cache/apt set -x && \
         liblog-any-adapter-log4perl-perl \
         # NB: not available in ubuntu 1804 LTS:
         libgeoip2-perl \
-        libemail-valid-perl
+        libemail-valid-perl \
+        # locale
+        locales
+# set locale
+RUN sed -i '/en_US.UTF-8/s/^# //g' /etc/locale.gen && \
+    locale-gen
+ENV LANG=en_US.UTF-8 LANGUAGE=en_US:en LC_ALL=en_US.UTF-8
 RUN --mount=type=cache,id=apt-cache,target=/var/cache/apt set -x && \
     apt install -y \
         #

cgi/display.pl

@@ -195,6 +195,9 @@
 elsif (defined $request_ref->{properties}) {
 	display_properties($request_ref);
 }
+elsif (defined $request_ref->{content}) {
+	display_content($request_ref);
+}
 elsif (defined $request_ref->{text}) {
 	if ($request_ref->{text} =~ /^products_stats/) {
 		display_stats($request_ref);

conf/apache-2.4/modperl.conf

@@ -31,4 +31,5 @@ PerlPassEnv RATE_LIMITER_BLOCKING_ENABLED
 PerlPassEnv ODOO_CRM_URL
 PerlPassEnv ODOO_CRM_DB
 PerlPassEnv ODOO_CRM_USER
-PerlPassEnv ODOO_CRM_PASSWORD
+PerlPassEnv ODOO_CRM_PASSWORD
+PerlPassEnv WORDPRESS_URL

docker-compose.yml

@@ -36,6 +36,7 @@ x-backend-conf: &backend-conf
     - ODOO_CRM_DB
     - ODOO_CRM_USER
     - ODOO_CRM_PASSWORD
+    - WORDPRESS_URL
   depends_on:
     - memcached
   volumes:

docs/dev/explain-website-contents.md

@@ -0,0 +1,58 @@
+# Explain Website Contents
+
+The Open Food Facts site has content pages that explain the project and its goals,
+or give explanations on important topics.
+
+Those contents comes from two sources:
+
+* the openfoodfacts-web repository
+* an external CMS, which is an instance WordPress
+
+## CMS Pages
+
+The pages from the CMS are the new way of creating content.
+They offer an easy to use interface to create and edit content, for non technical users.
+
+Note: We don't use pages but posts, because they have tags
+which can be used to filter the pages according to the Product Opener instance.
+
+We use the WPML plugin to enable internationalization and translation of pages.
+WPML is connected to crowdin so that we can have our community translate the contents.
+
+We talk to the WordPress API to get the content of the pages.
+At startup, `load_data`, calls `load_cms_data`
+which will use the API to get the known pages with some metadata.
+
+A special URL, available to administrators, `/content/refresh`
+can be used to call `load_cms_data` again, and refresh the pages list.
+
+
+### Crowdin / WPML integration
+
+See [How to setup wordpress](./how-to-setup-wordpress.md)
+
+### How to create a page
+
+See https://wiki.openfoodfacts.org/Open_Food_Facts_Contents
+
+## openfoodfacts-web Pages
+
+This is the historic way of creating contents, and is still used for some pages.
+As contents are reworked, we should try to use the CMS pages.
+
+The contents are html files in the `openfoodfacts-web` repository, under the `lang/xx/texts` folders.
+
+This repository also contains translations for emails, or additives and will remain valid for those specific contents.
+
+Because it uses plain html files, it is complicated for non technical users to edit the contents.
+
+The integration with crowdin is done through the usual github / crowdin workflow.
+
+### Connecting pages to the website
+
+See [How to use pages from openfoodfacts-web](./how-to-use-pages-from-openfoodfacts-web.md)
+
+
+## How to migrate from openfoodfacts-web to the CMS
+
+See https://wiki.openfoodfacts.org/Open_Food_Facts_Contents#How_to:_replace_an_Openfoodfacts-web_page_by_a_Wordpress_page_.28Not_yet_in_production.29

docs/dev/how-to-setup-wordpress.md

@@ -0,0 +1,89 @@
+# How to setup wordpress for contents
+
+We use a WordPress instance to manage the contents of the site.
+See [Explain Website contents](./explain-website-contents.md)
+
+## Install WPGraphQL extension
+
+Install https://wordpress.org/plugins/wp-graphql/
+for Product Opener will use the GraphQL API to get the contents.
+
+## Install WPML extension
+
+This is a plugin to manage translations of contents.
+It is not free, but has the advantage to be integrated with crowdin,
+and to have more advanced features than PolyLang.
+
+Install WPML media translations and WPML Multilingual CMS extensions.
+
+We also install the WPML GraphQL extension.
+
+## WPML configuration
+
+In *WPML* menu, *settings submenu*, *Taxonomies Translations*, tell that Tags and Categories are not translatable.
+
+## Connect WPML to crowdin
+
+We asked Crowdin for an account as an Open Source Project to be able to have more than one project.
+
+This is needed because WPML does not use same id for languages as Crowdin,
+and it's quite impossible to change the code on WPML side,
+so we will need to map the languages in a specific way in the Crowdin project.
+
+In crowdin, I used the *openfoodfacts* account (linked to *tech* email address).
+
+I created the *openfoodfacts-contents* project, copying languages from the *openfoodfacts* project. And using advanced tab, I also used the same glossary and memory translation.
+
+Then go to the store, and install the [WPML app](https://store.crowdin.com/wpml-app).
+
+I installed it to the *openfoodfacts-contents* project.
+
+Go to *openfoodfacts-contents* project, in *integrations* tab, click on *create token*, and copy the token value.
+
+In the configuration of WPML, go to the *Translation management* menu, *Translators* tab and activate Crowdin.
+In the Crowdin box, click on *authenticate* button and paste the token value, then submit.
+
+Also use the "Refresh Language pairs" button.
+
+## Setting up languages
+
+In Wordpress, go to the *WPML, Languages* sub menu, and add languages.
+I added all but Macedonian which is not supported by crowdin out of the box.
+
+
+We have some differences between default WPML langage codes, that we have to setup by ourselves.
+I saw that by trying to send my first batch of translations to crowdin ([see below](#test-sending-translations))
+
+We log into Crowdin, go in the *openfoodfacts-contents* project, *settings*,
+*languages* sub menu, then click on *Add custom language codes*
+and change the *locale* for:
+- Spanish -> es
+- Irish -> ga
+- Armenian -> hy
+- Punjab -> pa
+- Portugese -> pt-pt
+- Portugese, Brazilian -> pt-br
+- Swedish -> sv
+- Chinese Simplified -> zh-hans
+- Chinese Traditional -> zh-hant
+- Nepali -> ne
+- Urdu (India) -> ur
+
+## Test sending translations
+
+Go to *WPML* menu, *Translation Management* sub menu, *Dashboard* tab.
+Select an article, and all languages.
+
+Then chose on the *Assign to translator*, and click *Add to translation Basket*.
+
+Then go to the *Translation Basket* tab, set a batch name and timeline,
+verify all contents are set to *Crowdin* as translator, and click on "Send all items for translation".
+
+If you got problems, you may look at the *communication log* (link at the bottom of the page).
+
+```
+11/27/2024 10:59:59 am - error - (27) Crowdin doesn't support the following language pair: en -> es,ga,hy,mk,ne,pa,pt-pt,pt-br,sv,ur,zh-hans,zh-hant.<br />Please <a href='https://crowdin.com/'>contact Crowdin</a> and ask them to add mapping for en to es,ga,hy,mk,ne,pa,pt-pt,pt-br,sv,ur,zh-hans,zh-hant, so they can receive this job. You can send them this email:<br /><i>Dear Crowdin,<br />I wanted to let you know that you're currently not supporting en to es,ga,hy,mk,ne,pa,pt-pt,pt-br,sv,ur,zh-hans,zh-hant which I am hoping to use on my website.<br />Could you update the configuration to ensure it works for me in the future?<br />Regards</i>
+```
+
+After searching a bit if I could do change the code on WPML side, which was not possible,
+I fixed it by changing language mappings in Crowdin projects (and that's why it's a separate project from the main one).