fix: use a specific jail for manual ban

also add support for ranges

docs/how-to-fail2ban-ban-bots.md

@@ -8,16 +8,23 @@ sudo apt install fail2ban
 
 On debian 11, also follow [How to install fail2ban on debian 11+](./how-to-fail2ban-debian-11+.md)
 
+## Configuring some filters
 
-## Configuring nginx-botsearch.conf
+We normally install those filters, with standard configurations:
 
-We can use nginx-botsearch jail to ban bots to use our web services.
+`nginx-botsearch` (banning bots that blindly search for old software install)
+and `nginx-http-auth` (banning bots making too much failed auth attempts)
 
-In practice, we will use manual ban, with a permanent bantime.
+## Configuring a jail for manual ban
 
-Enable fail2ban nginx-botsearch with our specific configuration:
+We can create a new jail to ban bots from using our web services.
+
+In practice, we will use the nginx-botsearch filter on a fake log file,
+and add ips manually to the jail with a permanent bantime.
+
+Enable fail2ban nginx-manual-ban jail with our specific configuration:
 ```bash
-ln -s /opt/openfoodfacts-infrastructure/confs/common/fail2ban-nftables/jail.d/nginx-botsearch.local /etc/fail2ban/jail.d/
+ln -s /opt/openfoodfacts-infrastructure/confs/common/fail2ban-nftables/jail.d/nginx-manual-ban.local /etc/fail2ban/jail.d/
 systemctl restart fail2ban
 ```
 
@@ -29,17 +36,20 @@ systemctl restart fail2ban
 ### See banned ips
 
 ```bash
-sudo fail2ban-client status nginx-botsearch
+sudo fail2ban-client status nginx-manual-ban
 ```
 
 ### Ban an ip
 
 ```bash
-sudo fail2ban-client set nginx-botsearch banip <IP>
+sudo fail2ban-client set nginx-manual-ban banip <IP>
 ```
-Note that it support ip ranges, like `123.456.789.1/24`
+
+Note that it supports ip ranges, like `123.456.789.1/24`
 
 ### Unban an ip
 ```bash
-sudo fail2ban-client set nginx-botsearch unbanip <IP>
-```
+sudo fail2ban-client set nginx-manual-ban unbanip <IP>
+```
+
+If ip is part of a range, the whole range must be unbanned.

docs/how-to-fail2ban-debian-11+.md

@@ -18,9 +18,12 @@ ln -s /opt/openfoodfacts-infrastructure/confs/common/fail2ban-nftables/nftables.
 systemctl restart nftables
 ```
 
+
 Configure fail2ban to use nftables rule, and fail2ban table:
 ```bash
 ln -s /opt/openfoodfacts-infrastructure/confs/common/fail2ban-nftables/action.d/nftables-common.local /etc/fail2ban/action.d/
+# also override the fail2ban nftables action so that it handles ip interval like 10.0.0.0/24
+ln -s /opt/openfoodfacts-infrastructure/confs/common/fail2ban-nftables/action.d/nftables.local /etc/fail2ban/action.d/
 ln -s /opt/openfoodfacts-infrastructure/confs/common/fail2ban-nftables/jail.d/use-nftable.local /etc/fail2ban/jail.d/
 ```
 
@@ -31,3 +34,22 @@ ln -s /opt/openfoodfacts-infrastructure/confs/common/fail2ban-nftables/systemd/f
 systemctl daemon-reload
 systemctl restart fail2ban
 ```
+
+## Seeing it in action
+
+If you have ip in a jail
+```bash
+fail2ban-client status
+fail2ban-client status <jail-name>
+```
+
+You should see it in the corresponding addr_set elements (`addr-set-<jainame>`):
+
+```bash
+# all rules
+nftable list ruleset
+# more precise: fail2ban table
+nft list table inet f2b-table
+# more precise: fail2ban addr_set
+nft list set inet f2b-table addr-set-<jail-name>
+```

docs/reports/2023-07-off2-off-reinstall.md

@@ -1758,14 +1758,17 @@ To test my installation I added this to `/etc/hosts` on my computer:
 - **FIXME** fix all scripts (eg. split_gs1_codeonline_json.pl) which use /srv/codeonline/imports as input and /srv2/off/codeonline/imports as output !
 - **FIXME** are we writting to lang/ ?
   * Missions.pm does --> we don't use it anymore ? however change the code to be sure
-  * gen_sucres.pl and gen_sugar.pl do --> move it to another folder (files or data)
+  * (done) added a fixme to gen_sucres.pl and gen_sugar.pl
   * gen_top_tags_per_country does --> move it to another folder (data/stats) and change nginx config
 - **FIXME** bug bandeau off --> update dist files
-- **FIXME** have a well identified secrets directory for various secrets used by sh scripts (for those of perl, use Config2)
+
+- **DOING** have a well identified secrets directory for various secrets used by sh scripts (for those of perl, use Config2) --> see [Copying secrets](#copying-secrets)
+  - ftp secrets (.netrc)
 
 - **FIXME** schedule gen feeds smartly
 
-- **FIXME** add systemd timer to launch export_producers_platform_data_to_public_database.sh on off-pro
+- (done) add export_producers_platform_data_to_public_database.sh to producers import task on off-pro (instead of a specific cron)
+
 - **FIXME** imports (to run on off-pro side):
   - auto: carrefour
     - modify to run on off-pro side
@@ -1990,4 +1993,11 @@ I look at the mongodb but did not find any foodbattle database.
 
 ### TODO after off1 re-install
 
-- add replications
+- add replications
+
+### TODO to have more things working
+- add prometheus exporters to all machines:
+  - for nginx on reverse proxy
+  - for memcached on memcached container
+  - for postgres on postgresql container
+  - for apache on off/obf/opf/opff/off-pro