From d8ae0d529cbc94e93867d58f10b0a8027f4d2601 Mon Sep 17 00:00:00 2001 From: kx1t Date: Tue, 13 Feb 2024 16:42:38 -0500 Subject: [PATCH 1/5] add `IPMAPS` feature --- README.md | 79 ++++++++++++------- .../create-ipmaps/dependencies.d/10-ssl | 0 .../create-ipmaps/dependencies.d/20-webproxy | 0 .../create-ipmaps/dependencies.d/30-blockbot | 0 .../create-ipmaps/dependencies.d/40-geoip | 0 .../dependencies.d/50-check-ipv6 | 0 .../create-ipmaps/dependencies.d/60-block-ips | 0 .../dependencies.d/70-basic-authentication | 0 .../create-ipmaps/dependencies.d/80-set-cors | 0 .../dependencies.d/99-test-webproxy | 0 .../create-ipmaps/dependencies.d/webproxy | 0 .../etc/s6-overlay/s6-rc.d/create-ipmaps/run | 2 + .../etc/s6-overlay/s6-rc.d/create-ipmaps/type | 1 + .../s6-rc.d/user/contents.d/create-ipmaps | 0 rootfs/etc/s6-overlay/scripts/20-webproxy | 2 +- rootfs/etc/s6-overlay/scripts/create-ipmaps | 55 +++++++++++++ 16 files changed, 110 insertions(+), 29 deletions(-) create mode 100644 rootfs/etc/s6-overlay/s6-rc.d/create-ipmaps/dependencies.d/10-ssl create mode 100644 rootfs/etc/s6-overlay/s6-rc.d/create-ipmaps/dependencies.d/20-webproxy create mode 100644 rootfs/etc/s6-overlay/s6-rc.d/create-ipmaps/dependencies.d/30-blockbot create mode 100644 rootfs/etc/s6-overlay/s6-rc.d/create-ipmaps/dependencies.d/40-geoip create mode 100644 rootfs/etc/s6-overlay/s6-rc.d/create-ipmaps/dependencies.d/50-check-ipv6 create mode 100644 rootfs/etc/s6-overlay/s6-rc.d/create-ipmaps/dependencies.d/60-block-ips create mode 100644 rootfs/etc/s6-overlay/s6-rc.d/create-ipmaps/dependencies.d/70-basic-authentication create mode 100644 rootfs/etc/s6-overlay/s6-rc.d/create-ipmaps/dependencies.d/80-set-cors create mode 100644 rootfs/etc/s6-overlay/s6-rc.d/create-ipmaps/dependencies.d/99-test-webproxy create mode 100644 rootfs/etc/s6-overlay/s6-rc.d/create-ipmaps/dependencies.d/webproxy create mode 100755 rootfs/etc/s6-overlay/s6-rc.d/create-ipmaps/run create mode 100644 rootfs/etc/s6-overlay/s6-rc.d/create-ipmaps/type create mode 100644 rootfs/etc/s6-overlay/s6-rc.d/user/contents.d/create-ipmaps create mode 100755 rootfs/etc/s6-overlay/scripts/create-ipmaps diff --git a/README.md b/README.md index 90b7c2a..54b06aa 100644 --- a/README.md +++ b/README.md @@ -9,7 +9,7 @@ - [What is it?](#what-is-it) - [How do I get it?](#how-do-i-get-it) - [How do I configure it?](#how-do-i-configure-it) - - [General parameters:](#general-parameters) + - [General parameters](#general-parameters) - [Configuration of the Webproxy](#configuration-of-the-webproxy) - [Configuration of SSL](#configuration-of-ssl) - [GeoIP Filtering](#geoip-filtering) @@ -18,6 +18,7 @@ - [Basic Authentication](#basic-authentication) - [Advanced Setup](#advanced-setup) - [Host your own web pages](#host-your-own-web-pages) + - [Automatic creation of web pages with geographic map of visitors](#automatic-creation-of-web-pages-with-geographic-map-of-visitors) - [Extras](#extras) - [Troubleshooting](#troubleshooting) - [Acknowledgements](#acknowledgements) @@ -35,14 +36,14 @@ The following example highlights this: | Web Service | Original address | New Address with Webproxy | | ----------- | ----------------------------- | ------------------------- | -| readsb | http://10.0.0.191:8080 | http://myip/readsb | -| piaware | http://10.0.0.191:8081 | http://myip/piaware | -| tar1090 | http://10.0.0.191:8082 | http://myip/tar1090 | -| planefence | http://10.0.0.191:8083 | http://myip/planefence | -| planefinder | http://10.0.0.191:8086 | http://myip/planefinder | -| graphs | http://10.0.0.191:8080/graphs | http://myip/graphs | -| radar | http://10.0.0.191:8080/radar | http://myip/radar | -| acarshub | http://10.0.0.188 | http://myip/acarshub | +| readsb | | | +| piaware | | | +| tar1090 | | | +| planefence | | | +| planefinder | | | +| graphs | | | +| radar | | | +| acarshub | | | ## How do I get it? @@ -51,7 +52,7 @@ This is less than 5 minutes of work -- use [this script](https://github.com/sdr- Once this is done, create a working directory and download the `docker-compose.yml` file: -``` +```bash sudo mkdir -p -m 777 /opt/webproxy && cd /opt/webproxy wget https://raw.githubusercontent.com/kx1t/docker-reversewebproxy/main/docker-compose.yml ``` @@ -67,13 +68,13 @@ With that, you are ready to run the proxy! The Webproxy can be entirely configured in the `docker-compose.yml`, or, optionally, you can create a more advanced setup manually. You can also start with the `docker-compose.yml` configuration and then add to this manually in the future. Here's how: -### General parameters: +### General parameters A "_" means that this is the default value | Parameter | Values | Description | |-----------|--------|-------------| -| `AUTOGENERATE` | `ON`_, `OFF` | Determines if the system will use the `REVPROXY` and `REDIRECT` settings of the `docker-compose.yml` file (`ON`), or a manually generated `locations.conf` file (`OFF`). | -| `VERBOSELOG` | `ON`_, `OFF` | Determines if the internal web service Access and Error logs will be written to the Docker log (accessible with `docker logs webproxy`) (`ON`), or that logging will be switched `OFF`. +| `AUTOGENERATE` | `ON`, `OFF` | Determines if the system will use the `REVPROXY` and `REDIRECT` settings of the `docker-compose.yml` file (`ON`), or a manually generated `locations.conf` file (`OFF`). | +| `VERBOSELOG` | `ON`, `OFF` | Determines if the internal web service Access and Error logs will be written to the Docker log (accessible with `docker logs webproxy`) (`ON`), or that logging will be switched `OFF` | | `CORSHOSTS` | list of hosts | Comma separated list of host/DNS names of CORS exceptions. These are needed if a website calls into an external API, for example when adding the RainViewer overlay to VRS. Most browsers block this unless the external API target is added to this variable. Default is empty. Example value you can use to add the RainViewer API: `api.rainviewer.com`. Adding `"_"` will disable CORS protection for all hostnames; this is not recommended! | You may have to adjust your `port:` and your `volumes:` mapping to your liking, especially if you are not running on the Raspberry Pi standard `pi` account. @@ -85,7 +86,7 @@ If `AUTOGENERATE=ON`, the system will build a Webproxy based on the `REVPROXY` a `REVPROXY` defines the proxy-pairs to serve the `destination` target when the user browses to `urltarget`. The user's browser will never be redirected to an internal IP address for service, all web pages are being served from the Webproxy. As such, the process of going to the correct website/port to get the web page is completely hidden from the user. `REVPROXY` has the following format: `urltarget|destination[|user1|pass1[|user2|pass2[|...|...]]]` -For example, for REVPROXY=readsb|http://10.0.0.191:8080, a user browsing to http://mydomain/readsb will be proxied to a service located at http://10.0.0.191:8080. The user's browser will _never_ see the internal IP address. +For example, for REVPROXY=readsb|, a user browsing to will be proxied to a service located at . The user's browser will _never_ see the internal IP address. Note - both the `urltarget` and the `destination` must be URLs or directories, and cannot be a file name. You can provide a comma separated list of `urltarget|destination` pairs, similar to the example in the default `docker-compose.yml`. The optional `|user1|pass1|user2|pass2|...|...` addons define the allowed username/password combination for this specific revproxy. @@ -109,14 +110,14 @@ A "_" means that this is the default value | `SSL` | `DISABLED`_, `ENABLED` | Enable the installation of SSL certificates | | `SSL_EMAIL` | your email address | A valid email address is needed to get a certificate | | `SSL_DOMAIN` | A list of web domains | We will enabled SSL for these. Note - they must be reachable domains at this container for the SSL certificate to be successfully installed! | -| `SSL_TOS` | `REJECT`_, `ACCEPT` | Indicates your acceptance of the T&S's for the SSL certificateset forth at https://letsencrypt.org/repository/#let-s-encrypt-subscriber-agreement | -| ` SSL_REDIRECT` | `DISABLED`, `ENABLED`_ | When set to ENABLED, all incoming non-SSL traffic is redirected to use SSL +| `SSL_TOS` | `REJECT`_, `ACCEPT` | Indicates your acceptance of the T&S's for the SSL certificateset forth at | +| `SSL_REDIRECT` | `DISABLED`, `ENABLED` | When set to ENABLED, all incoming non-SSL traffic is redirected to use SSL | Note: your SSL certificates are valid for 90 days. The container will check daily if they need renewing, and will do so of there's less than a month before the expiration date. **LetsEncrypt will start sending you emails about the pending expiration about 45 days before the deadline. Sometimes, the expiration date in this email doesn't correspond to the real expiration date of the certificates. You can safely ignore these emails as long as your container is running.** If you want to check the official expiration date of your certificates, this command will show you: -``` +```bash docker exec -it webproxy certbot certificates ``` @@ -128,7 +129,7 @@ The Reverse Webproxy can filter incoming requests by originating IP. It uses an | -------------------- | -------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `GEOIP_DEFAULT` | \\*, `ALLOW`, `DISALLOW` | Empty: GeoIP filtering is disabled; `ALLOW`: only those countries listed in the `GEOIP_COUNTRIES` parameter are permitted; `DISALLOW`: the countries listed in `GEOIP_COUNTRIES` are filtered. | | `GEOIP_COUNTRIES` | | Comma-separated list of 2-letter country abbreviations, for example `RU,CN,BY,RS` (which means Russia, China, Bielorus, Serbia). | -| `GEOIP_RESPONSECODE` | 3-digit HTTP response code | Default if omitted: `403` ("Forbidden"). Other codes that may be useful: `402` (payment required), `404` (doesn't exist), `418` (I am a teapot - used to tell requesters to go away), `410` (Gone), `500` (Internal Server Error), `503` (service unavailable). See https://developer.mozilla.org/en-US/docs/Web/HTTP/Status | +| `GEOIP_RESPONSECODE` | 3-digit HTTP response code | Default if omitted: `403` ("Forbidden"). Other codes that may be useful: `402` (payment required), `404` (doesn't exist), `418` (I am a teapot - used to tell requesters to go away), `410` (Gone), `500` (Internal Server Error), `503` (service unavailable). See | ### BlockBot Filtering @@ -137,7 +138,7 @@ The BlockBot feature filters out HTTP requests based on a fuzzy match of the HTT | Parameter | Values | Description | | ----------------------- | ------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `BLOCKBOT` | string snippets of User Agent fields | Comma-separated strings, for example `google,bing,yandex,msnbot`. If this parameter is empty, the BlockBot functionality is disabled. | -| `BLOCKBOT_RESPONSECODE` | 3-digit HTTP response code | Default if omitted: `403` ("Forbidden"). Other codes that may be useful: `402` (payment required), `404` (doesn't exist), `418` (I am a teapot - used to tell requesters to go away), `410` (Gone), `500` (Internal Server Error), `503` (service unavailable). See https://developer.mozilla.org/en-US/docs/Web/HTTP/Status | +| `BLOCKBOT_RESPONSECODE` | 3-digit HTTP response code | Default if omitted: `403` ("Forbidden"). Other codes that may be useful: `402` (payment required), `404` (doesn't exist), `418` (I am a teapot - used to tell requesters to go away), `410` (Gone), `500` (Internal Server Error), `503` (service unavailable). See | ### `iptables` blocking @@ -146,7 +147,7 @@ As an option, the system can use `iptables` to block any IP match of GeoIp or Bl The system will scan the logs for any BlockBot or GeoIP filtered request, and adds any IP address for which a return value of `$BLOCKBOT_RESPONSECODE` or `$GEOIP_RESPONSECODE` to the `iptables` blocked list, unless the IP is part of a value or range specified in the `ip-allowlist` (see below). The `iptables` blocker is updated in batches every 60 seconds. To enable this behavior, set `IPTABLES_BLOCK` to `ENABLED` or `ON`. You can also specify the time an IP address should stay on the `iptables` block list with the `IPTABLES_JAILTIME` parameter. Additionally, you must add the `NET_ADMIN` capacity to the container; see the [`docker-compose.yml`](docker-compose.yml) for an example. -``` +```yaml cap_add: - NET_ADMIN ``` @@ -157,11 +158,11 @@ As long as the `/run/nginx` volume is mapped (see example in [`docker-compose.ym If you want to remove IP addresses from the blocked list, you can do so manually by removing them with a text editor from the file `ip-blocklist` in the mapped volume. Alternatively, you can use a simple utility to do this while running the container: -``` +```bash docker exec -it webproxy manage_ipblock ``` -Note that the `IPTABLES_BLOCK` feature enables logging to disk (specifically, `/var/log/nginx/access.log`). You may want to map this directory to a `tmpfs` volume (see example in [`docker-compose.yml'](docker-compose.yml)). Log rotation keeps 24 files of 1 hour each around; the 1 hour log rotation intervals and number of retained backups are configurable with the `LOGROTATE_INTERVAL` and `LOGROTATE_MAXBACKUPS` docker environment variable. +Note that the `IPTABLES_BLOCK` feature enables logging to disk (specifically, `/var/log/nginx/access.log`). You may want to map this directory to a `tmpfs` volume \(see example in [`docker-compose.yml'](docker-compose.yml)\). Log rotation keeps 24 files of 1 hour each around; the 1 hour log rotation intervals and number of retained backups are configurable with the`LOGROTATE_INTERVAL` and `LOGROTATE_MAXBACKUPS` docker environment variable. | Parameter | Values | Description | | ---------------------- | ------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | @@ -182,7 +183,7 @@ The container supports basic authentication for the local web page through the ` | `LOCAL_CREDS` | | A list of credentials in the format `username1 | password1,username2 | password2,...` | | `REVPROXY` | | A comma separated list in this format: | -``` +```yaml REVPROXY=origin1|http://destination1|username1|password1|username2|password2, origin2|http://destination2|username3|password3|username4|password4|username5|password5, origin3|http://destination3, @@ -205,11 +206,33 @@ Note -- the web server inside the container does NOT run as `root`, so you must Feel free to create additional subdirectories if needed for your project. Also note -- the website may not be reachable if you redirected or proxied `/` to some other service. +#### Automatic creation of web pages with geographic map of visitors + +If you set `IPMAPS=true`, the container will try to automatically create IP maps of the visitors to your website. This includes any visit that goes to a URL that is handled by the WebProxy, regardless if it's rendering a local page, being sent to a reverse proxy address, or being redirected somewhere else. The website will automatically generate the following pages that are updated every 15 minutes. You can change the defaults by defining the parameters below. + +The system uses to create these maps from the webserver's access logs. + +If `IPMAPS` is not enabled, the pages will not exist. Any previously generated map redirects will be deleted. + +| URL | Map Type | +| --- | -------- | +| `/ipmaps-all.html` | Redirection to a map with **all** visitors | +| `/ipmaps-filtered.html` | Redirection to a map with only visitors who were denied access due to Geo-IP block or BotBlock | +| `/ipmaps-accepted.html` | Redirection to a map with only those visitors that passed the filtering and that were allowed to browse to the resource they tried to access | + +The following related parameters can be set: + +| Parameter | Values | Description | +| ------------- | --------------------- | ---------------------------------------------------------------------------------------------------------------------- | ------------------- | -------------- | +| `IPMAPS` | `on`/`enabled`/`true`/`1` or
`off`/`disabled`/`false`/`0`
or empty | If enabled, IPMAPS will be generated as described above. If disabled or empty (default), maps aren't generated | +| `IPMAPS_INTERVAL` | value in secs or empty | Interval of generation of the IP Maps. Default if omitted is `900` seconds | +| `IPMAPS_BASENAME` | partial file name | Base file name of the map URL. Default value is `ipmaps-`, which would correspond to `http://ip_addr/ipmaps-all.html` / `http://ip_addr/ipmaps-filtered.html` / `http://ip_addr/ipmaps-allowed.html` | + ### Extras - Get a URL to a geographic map of all IPs that hit your WebProxy by typing: -``` +```bash docker exec -it webproxy ipmap ``` @@ -235,7 +258,7 @@ docker exec -it webproxy ipmap - Issue: Planefinder doesn't work correctly - Solution: make sure that you have added the following to the `REVPROXY` variable (replace ip address and port with whatever is appropriate for your system): - ``` + ```yaml planefinder|http://10.0.0.191:8086, ajax|http://10.0.0.191:8086/ajax, assets|http://10.0.0.191:8086/assets, @@ -243,7 +266,7 @@ docker exec -it webproxy ipmap - Issue: The docker logs show an error like this on start up: -``` +```text nginx: [emerg] socket() [::]:80 failed (97: Address family not supported) nginx: configuration file /etc/nginx/nginx.conf test failed ``` @@ -260,7 +283,7 @@ nginx: configuration file /etc/nginx/nginx.conf test failed - Solution: This is caused by certificates that have been added to `webproxy` at different points in time. To fix it, back up any web pages that are directly served by the container, and recreate the entire setup. Please note that doing this more than 5 times in a week will lock you out and prevent you from recreating existing certificates for up to a week, so USE THIS SOLUTION SPARINGLY. The solution assumes that the container name is `webproxy` and that its working directory is `~/.webproxy` . If this is different, you may have to adapt the commands accordingly. It's preferable to feed the script line by line rather than all at once, so you can monitor the outcome. -``` +```bash cd ~ # go to the home directory docker stop webproxy # stop the webproxy container @@ -301,4 +324,4 @@ without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program. -If not, see https://www.gnu.org/licenses/. +If not, see . diff --git a/rootfs/etc/s6-overlay/s6-rc.d/create-ipmaps/dependencies.d/10-ssl b/rootfs/etc/s6-overlay/s6-rc.d/create-ipmaps/dependencies.d/10-ssl new file mode 100644 index 0000000..e69de29 diff --git a/rootfs/etc/s6-overlay/s6-rc.d/create-ipmaps/dependencies.d/20-webproxy b/rootfs/etc/s6-overlay/s6-rc.d/create-ipmaps/dependencies.d/20-webproxy new file mode 100644 index 0000000..e69de29 diff --git a/rootfs/etc/s6-overlay/s6-rc.d/create-ipmaps/dependencies.d/30-blockbot b/rootfs/etc/s6-overlay/s6-rc.d/create-ipmaps/dependencies.d/30-blockbot new file mode 100644 index 0000000..e69de29 diff --git a/rootfs/etc/s6-overlay/s6-rc.d/create-ipmaps/dependencies.d/40-geoip b/rootfs/etc/s6-overlay/s6-rc.d/create-ipmaps/dependencies.d/40-geoip new file mode 100644 index 0000000..e69de29 diff --git a/rootfs/etc/s6-overlay/s6-rc.d/create-ipmaps/dependencies.d/50-check-ipv6 b/rootfs/etc/s6-overlay/s6-rc.d/create-ipmaps/dependencies.d/50-check-ipv6 new file mode 100644 index 0000000..e69de29 diff --git a/rootfs/etc/s6-overlay/s6-rc.d/create-ipmaps/dependencies.d/60-block-ips b/rootfs/etc/s6-overlay/s6-rc.d/create-ipmaps/dependencies.d/60-block-ips new file mode 100644 index 0000000..e69de29 diff --git a/rootfs/etc/s6-overlay/s6-rc.d/create-ipmaps/dependencies.d/70-basic-authentication b/rootfs/etc/s6-overlay/s6-rc.d/create-ipmaps/dependencies.d/70-basic-authentication new file mode 100644 index 0000000..e69de29 diff --git a/rootfs/etc/s6-overlay/s6-rc.d/create-ipmaps/dependencies.d/80-set-cors b/rootfs/etc/s6-overlay/s6-rc.d/create-ipmaps/dependencies.d/80-set-cors new file mode 100644 index 0000000..e69de29 diff --git a/rootfs/etc/s6-overlay/s6-rc.d/create-ipmaps/dependencies.d/99-test-webproxy b/rootfs/etc/s6-overlay/s6-rc.d/create-ipmaps/dependencies.d/99-test-webproxy new file mode 100644 index 0000000..e69de29 diff --git a/rootfs/etc/s6-overlay/s6-rc.d/create-ipmaps/dependencies.d/webproxy b/rootfs/etc/s6-overlay/s6-rc.d/create-ipmaps/dependencies.d/webproxy new file mode 100644 index 0000000..e69de29 diff --git a/rootfs/etc/s6-overlay/s6-rc.d/create-ipmaps/run b/rootfs/etc/s6-overlay/s6-rc.d/create-ipmaps/run new file mode 100755 index 0000000..a73341f --- /dev/null +++ b/rootfs/etc/s6-overlay/s6-rc.d/create-ipmaps/run @@ -0,0 +1,2 @@ +#!/bin/sh +exec /etc/s6-overlay/scripts/create-ipmaps diff --git a/rootfs/etc/s6-overlay/s6-rc.d/create-ipmaps/type b/rootfs/etc/s6-overlay/s6-rc.d/create-ipmaps/type new file mode 100644 index 0000000..5883cff --- /dev/null +++ b/rootfs/etc/s6-overlay/s6-rc.d/create-ipmaps/type @@ -0,0 +1 @@ +longrun diff --git a/rootfs/etc/s6-overlay/s6-rc.d/user/contents.d/create-ipmaps b/rootfs/etc/s6-overlay/s6-rc.d/user/contents.d/create-ipmaps new file mode 100644 index 0000000..e69de29 diff --git a/rootfs/etc/s6-overlay/scripts/20-webproxy b/rootfs/etc/s6-overlay/scripts/20-webproxy index 7407e57..98f6220 100755 --- a/rootfs/etc/s6-overlay/scripts/20-webproxy +++ b/rootfs/etc/s6-overlay/scripts/20-webproxy @@ -160,7 +160,7 @@ fi ln -sf /etc/nginx/sites-available/default /etc/nginx/sites-enabled/default # set logging to what VERBOSELOG says. Default is off, "ON" means to stdout, "FILE" means to access.log/error.log file in external volume -if [[ "${VERBOSELOG,,}" == "on" ]] || [[ "${VERBOSELOG,,}" == "true" ]] +if chk_enabled "${VERBOSELOG}" then "${s6wrap[@]}" echo "Enabled HTTP logging to container logs" sed -i 's|\(^\s*access_log \).*|\1/dev/stdout ;|' /etc/nginx/nginx.conf diff --git a/rootfs/etc/s6-overlay/scripts/create-ipmaps b/rootfs/etc/s6-overlay/scripts/create-ipmaps new file mode 100755 index 0000000..bf51afc --- /dev/null +++ b/rootfs/etc/s6-overlay/scripts/create-ipmaps @@ -0,0 +1,55 @@ +#!/command/with-contenv bash +#shellcheck shell=bash disable=SC2154,SC2089 + +# shellcheck disable=SC1091 +source /scripts/common + +# Advise to keep IPMAPS_INTERVAL < about 900 secs - this corresponds to 2920 requests +# per month, well below the maximum of 4000 requests from a single IP address for the +# free API of https://ipinfo.io/ + +IPMAPS_INTERVAL="${IPMAPS_INTERVAL:-900}" +IPMAPS_BASENAME="${IPMAPS_BASENAME:-ipmap-}" +HTMLDIR="/run/nginx/html" + +notavail_template='

Map Currently Unavailable

This map is currently not available; please try again later. Last update:##TIME##' +redir_template='' + +if ! chk_enabled "$IPMAPS"; then + rm -f "${HTMLDIR}/${IPMAPS_BASENAME}"*.html + exec sleep infinity +fi + +while :; do + + ipmap_all="$(/usr/local/bin/ipmap -l 2>/dev/null| tail -1)" || true + ipmap_filtered="$(/usr/local/bin/ipmap -f 2>/dev/null| tail -1)" || true + ipmap_accepted="$(/usr/local/bin/ipmap -v 2>/dev/null| tail -1)" || true + + "${s6wrap[@]}" echo "Updating ${IPMAPS_BASENAME}all.html --> $ipmap_all" + { if [[ "${ipmap_all:0:4}" == "http" ]]; then + echo "${redir_template//##REDIRURL##/$ipmap_all}" + else + echo "${notavail_template//##TIME##/$(date)}" + fi + } > "${HTMLDIR}/${IPMAPS_BASENAME}all.html" + + "${s6wrap[@]}" echo "Updating ${IPMAPS_BASENAME}filtered.html --> $ipmap_filtered" + { if [[ "${ipmap_filtered:0:4}" == "http" ]]; then + echo "${redir_template//##REDIRURL##/$ipmap_filtered}" + else + echo "${notavail_template//##TIME##/$(date)}" + fi + } > "${HTMLDIR}/${IPMAPS_BASENAME}filtered.html" + + "${s6wrap[@]}" echo "Updating ${IPMAPS_BASENAME}accepted.html --> $ipmap_accepted" + { if [[ "${ipmap_accepted:0:4}" == "http" ]]; then + echo "${redir_template//##REDIRURL##/$ipmap_accepted}" + else + echo "${notavail_template//##TIME##/$(date)}" + fi + } > "${HTMLDIR}/${IPMAPS_BASENAME}accepted.html" + + sleep "$IPMAPS_INTERVAL" + +done \ No newline at end of file From 109a9a7809cda568886d97ed3d9b17ca42295f60 Mon Sep 17 00:00:00 2001 From: kx1t Date: Tue, 13 Feb 2024 17:04:10 -0500 Subject: [PATCH 2/5] debugging --- buildnow.sh | 19 +++++++------------ rootfs/etc/s6-overlay/scripts/20-webproxy | 6 +++--- rootfs/etc/s6-overlay/scripts/80-set-cors | 3 ++- 3 files changed, 12 insertions(+), 16 deletions(-) diff --git a/buildnow.sh b/buildnow.sh index caeb5cd..41422d6 100755 --- a/buildnow.sh +++ b/buildnow.sh @@ -4,26 +4,21 @@ set -x [[ "$1" != "" ]] && BRANCH="$1" || BRANCH=main [[ "$BRANCH" == "main" ]] && TAG="latest" || TAG="$BRANCH" -[[ "$ARCHS" == "" ]] && ARCHS="linux/armhf,linux/arm64,linux/amd64,linux/i386" +[[ "$ARCHS" == "" ]] && ARCHS="linux/armhf,linux/arm64,linux/amd64" BASETARGET1=ghcr.io/sdr-enthusiasts -BASETARGET2=kx1t +#BASETARGET2=kx1t -IMAGE1="$BASETARGET1/$(pwd | sed -n 's|.*/docker-\(.*\)|\1|p'):$TAG" -IMAGE2="$BASETARGET2/$(pwd | sed -n 's|.*/docker-\(.*\)|\1|p'):$TAG" +IMAGE1="$BASETARGET1/docker-reversewebproxy:$TAG" +#IMAGE2="$BASETARGET2/$(pwd | sed -n 's|.*/docker-\(.*\)|\1|p'):$TAG" -#shellcheck disable=SC2077,SC2015 -[[ "$IMAGE1" == "$BASETARGET1/reversewebproxy:$TAG" ]] && IMAGE1="$BASETARGET1/webproxy:$TAG" || true -#shellcheck disable=SC2077,SC2015 -[[ "$IMAGE2" == "$BASETARGET2/reversewebproxy:$TAG" ]] && IMAGE2="$BASETARGET2/webproxy:$TAG" || true - -echo "press enter to start building $IMAGE1 and $IMAGE2 from $BRANCH" +echo "press enter to start building $IMAGE1 from $BRANCH" read -r starttime="$(date +%s)" # rebuild the container git checkout "$BRANCH" || exit 2 git pull -a -docker buildx build --compress --push "$2" --platform "$ARCHS" --tag "$IMAGE1" . -docker buildx build --compress --push "$2" --platform "$ARCHS" --tag "$IMAGE2" . +docker buildx build --compress --push --platform "$ARCHS" --tag "$IMAGE1" . +#docker buildx build --compress --push "$2" --platform "$ARCHS" --tag "$IMAGE2" . echo "Total build time: $(( $(date +%s) - starttime )) seconds" diff --git a/rootfs/etc/s6-overlay/scripts/20-webproxy b/rootfs/etc/s6-overlay/scripts/20-webproxy index 98f6220..aae9dec 100755 --- a/rootfs/etc/s6-overlay/scripts/20-webproxy +++ b/rootfs/etc/s6-overlay/scripts/20-webproxy @@ -7,7 +7,7 @@ source /scripts/common # create run directory. This is where the locations.conf file will be located. # Create the correct ownership and permissions for the /var/log/nginx directory: -adduser -U www-data >/dev/null 2>&1 +adduser -U www-data >/dev/null 2>&1 || true mkdir -p /var/log/nginx mkdir -p /run/nginx/html mkdir -p /run/nginx/.htaccess @@ -34,11 +34,11 @@ if chk_enabled "$AUTOGENERATE"; then [[ -f /run/nginx/locations.conf ]] && mv -f /run/nginx/locations.conf "/run/nginx/locations.conf.bkup-$(date +%y%m%d%H%M%S)" # keep only the newest 10 backups: - org=$(shopt -p nullglob) # save current nullglob setting + org=$(shopt -p nullglob) || true # save current nullglob setting shopt -s nullglob # set nullglob to avoid ls errors when no backups are present # shellcheck disable=SC2010 ls -tp /run/nginx/locations.conf.bkup-* | grep -v '/$' | tail -n +11 | xargs -d '\n' -r rm -- # remove any backups older than the newest 10 - $org # restore the original shopt nullglob setting + if [[ -n "$org" ]]; then $org; fi # restore the original shopt nullglob setting # configure the locations.conf file: # first read the REVPROXY and REDIRECT environment variables into an array: diff --git a/rootfs/etc/s6-overlay/scripts/80-set-cors b/rootfs/etc/s6-overlay/scripts/80-set-cors index a73f10e..01cad68 100755 --- a/rootfs/etc/s6-overlay/scripts/80-set-cors +++ b/rootfs/etc/s6-overlay/scripts/80-set-cors @@ -25,7 +25,8 @@ for cors in "${cors_array[@]}" do if [[ -n "${cors}" ]]; then printf "add_header Access-Control-Allow-Origin %s;\n" "${cors}" >> /run/nginx/cors_headers.conf - s6wrap --quiet --timestamps --prepend="${APPNAME}" --args echo "Added CORS exception for ${cors}" + # shellcheck disable=SC2154 + "${s6wrap}" echo "Added CORS exception for ${cors}" fi done exit 0 From d0fbfc648f5cc9a0d45e660b28f20c28546fa399 Mon Sep 17 00:00:00 2001 From: kx1t Date: Tue, 13 Feb 2024 17:06:34 -0500 Subject: [PATCH 3/5] bug fix --- rootfs/etc/s6-overlay/scripts/80-set-cors | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rootfs/etc/s6-overlay/scripts/80-set-cors b/rootfs/etc/s6-overlay/scripts/80-set-cors index 01cad68..dc3540a 100755 --- a/rootfs/etc/s6-overlay/scripts/80-set-cors +++ b/rootfs/etc/s6-overlay/scripts/80-set-cors @@ -26,7 +26,7 @@ do if [[ -n "${cors}" ]]; then printf "add_header Access-Control-Allow-Origin %s;\n" "${cors}" >> /run/nginx/cors_headers.conf # shellcheck disable=SC2154 - "${s6wrap}" echo "Added CORS exception for ${cors}" + "${s6wrap[@]}" echo "Added CORS exception for ${cors}" fi done exit 0 From e3da184147c024698cff775dbb2997cc43ac9e51 Mon Sep 17 00:00:00 2001 From: kx1t Date: Tue, 13 Feb 2024 17:12:06 -0500 Subject: [PATCH 4/5] beautification --- README.md | 8 ++++---- rootfs/etc/s6-overlay/scripts/create-ipmaps | 2 +- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index 54b06aa..14652dd 100644 --- a/README.md +++ b/README.md @@ -216,9 +216,9 @@ If `IPMAPS` is not enabled, the pages will not exist. Any previously generated m | URL | Map Type | | --- | -------- | -| `/ipmaps-all.html` | Redirection to a map with **all** visitors | -| `/ipmaps-filtered.html` | Redirection to a map with only visitors who were denied access due to Geo-IP block or BotBlock | -| `/ipmaps-accepted.html` | Redirection to a map with only those visitors that passed the filtering and that were allowed to browse to the resource they tried to access | +| `/ipmap-all.html` | Redirection to a map with **all** visitors | +| `/ipmap-filtered.html` | Redirection to a map with only visitors who were denied access due to Geo-IP block or BotBlock | +| `/ipmap-accepted.html` | Redirection to a map with only those visitors that passed the filtering and that were allowed to browse to the resource they tried to access | The following related parameters can be set: @@ -226,7 +226,7 @@ The following related parameters can be set: | ------------- | --------------------- | ---------------------------------------------------------------------------------------------------------------------- | ------------------- | -------------- | | `IPMAPS` | `on`/`enabled`/`true`/`1` or
`off`/`disabled`/`false`/`0`
or empty | If enabled, IPMAPS will be generated as described above. If disabled or empty (default), maps aren't generated | | `IPMAPS_INTERVAL` | value in secs or empty | Interval of generation of the IP Maps. Default if omitted is `900` seconds | -| `IPMAPS_BASENAME` | partial file name | Base file name of the map URL. Default value is `ipmaps-`, which would correspond to `http://ip_addr/ipmaps-all.html` / `http://ip_addr/ipmaps-filtered.html` / `http://ip_addr/ipmaps-allowed.html` | +| `IPMAPS_BASENAME` | partial file name | Base file name of the map URL. Default value is `ipmap-`, which would correspond to `http://ip_addr/ipmap-all.html` / `http://ip_addr/ipmap-filtered.html` / `http://ip_addr/ipmap-allowed.html` | ### Extras diff --git a/rootfs/etc/s6-overlay/scripts/create-ipmaps b/rootfs/etc/s6-overlay/scripts/create-ipmaps index bf51afc..ddf90bf 100755 --- a/rootfs/etc/s6-overlay/scripts/create-ipmaps +++ b/rootfs/etc/s6-overlay/scripts/create-ipmaps @@ -12,7 +12,7 @@ IPMAPS_INTERVAL="${IPMAPS_INTERVAL:-900}" IPMAPS_BASENAME="${IPMAPS_BASENAME:-ipmap-}" HTMLDIR="/run/nginx/html" -notavail_template='

Map Currently Unavailable

This map is currently not available; please try again later. Last update:##TIME##' +notavail_template='

Map Currently Unavailable

This map is currently not available; please try again later. Last update: ##TIME##' redir_template='' if ! chk_enabled "$IPMAPS"; then From 9fe3737e53722c50f4a2bfcbf59d5517e6436a94 Mon Sep 17 00:00:00 2001 From: kx1t Date: Tue, 13 Feb 2024 17:19:09 -0500 Subject: [PATCH 5/5] update readme --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 14652dd..d8566fe 100644 --- a/README.md +++ b/README.md @@ -222,8 +222,8 @@ If `IPMAPS` is not enabled, the pages will not exist. Any previously generated m The following related parameters can be set: -| Parameter | Values | Description | -| ------------- | --------------------- | ---------------------------------------------------------------------------------------------------------------------- | ------------------- | -------------- | +| Parameter | Values | Description | +| --------- | ------ | ---------------- | | `IPMAPS` | `on`/`enabled`/`true`/`1` or
`off`/`disabled`/`false`/`0`
or empty | If enabled, IPMAPS will be generated as described above. If disabled or empty (default), maps aren't generated | | `IPMAPS_INTERVAL` | value in secs or empty | Interval of generation of the IP Maps. Default if omitted is `900` seconds | | `IPMAPS_BASENAME` | partial file name | Base file name of the map URL. Default value is `ipmap-`, which would correspond to `http://ip_addr/ipmap-all.html` / `http://ip_addr/ipmap-filtered.html` / `http://ip_addr/ipmap-allowed.html` |