diff --git a/README.md b/README.md index 5395c81d..58efcad7 100644 --- a/README.md +++ b/README.md @@ -31,6 +31,7 @@ Builds and runs on `amd64`, `arm64`, `arm/v7`, `arm/v6` and `386` architectures. - [ACARS](#acars) - [VDLM2](#vdlm2) - [HFDL](#hfdl) + - [Inmarsat L-Band](#inmarsat-l-band) - [Viewing the messages](#viewing-the-messages) - [Which frequencies should you monitor](#which-frequencies-should-you-monitor) - [A note about data sources used for the web site](#a-note-about-data-sources-used-for-the-web-site) @@ -76,14 +77,15 @@ I am missing a boat load of people who have provided feed back as this project h ## Getting valid ACARS/VDLM2 data -External to ACARS Hub you need to be running an ACARS and/or VDLM2 decoder for ACARS Hub, and have that decoder connect to ACARS Hub to send over the messages for processing. +External to ACARS Hub you need to be running an ACARS, VDLM2, HFDL, and/or Inmarsat L-Band decoder for ACARS Hub, and have that decoder connect to ACARS Hub to send over the messages for processing. The following decoders are supported: - [acarsdec](https://github.com/TLeconte/acarsdec) or one of the forks of acarsdec. I suggest [the airframes fork](https://github.com/airframesio/acarsdec). Run the decoder with the option `-j youracarshubip:5550`, ensuring that port `5550` is mapped to the container if the source is external to your docker network. - [dumpvdl2](https://github.com/szpajder/dumpvdl2). Run the decoder with the option `--output decoded:json:udp:address=,port=5555`, ensuring that port `5555` is mapped to the container if your source is external to the docker network. - [vdlm2dec](https://github.com/TLeconte/vdlm2dec). Run the decoder with the option `-j youracarshubip:5555`, ensuring that port `5555` is mapped to the container if the source is external to the docker network. -- [dumphfdl](https://github.com/szpajder/dumphfdl). Run the decoder with the option `--output decoded:json:udp:address=,port=5556`, ensuring that port `5556` is mapped to the container if the source is external to the docker network.. +- [dumphfdl](https://github.com/szpajder/dumphfdl). Run the decoder with the option `--output decoded:json:udp:address=,port=5556`, ensuring that port `5556` is mapped to the container if the source is external to the docker network. +- [satdump](https://github.com/SatDump/SatDump). Run the decoder with the Inmarsat.json options for `udp_sinks` set to `"address": "127.0.0.1"` and `"port": "5557"` , ensuring that port `5557` is mapped to the container.. For VDLM decoding `dumpvdl2` is preferred as the decoder provides richer data and is more modern than `vdlm2dec`. @@ -93,6 +95,7 @@ For ease of use I have provided docker images set up to work with ACARS Hub. Thi - [docker-dumpvdl2](https://github.com/sdr-enthusiasts/docker-dumpvdl2) for VDLM decoding. This is the preferred decoder. - [docker-vdlm2dec](https://github.com/sdr-enthusiasts/docker-vdlm2dec) as an alternative for VDLM decoding. This decoder is far less feature-rich compared to `dumpvdl2` and is provided only as an alternative if you have a strong preference for using this over `dumpvdl2`. - [docker-dumphfdl](https://github.com/sdr-enthusiasts/docker-dumphfdl) for HFDL decoding. +- [docker-satdump](https://github.com/rpatel3001/docker-satdump) for Inmarsat L-Band decoding. - [acars_router](https://github.com/sdr-enthusiasts/acars_router) for routing ACARS messages from one source to another. This is useful if you have a decoder that can only send messages to one destination, but you want to send messages to multiple destinations. This is the preferred way to get data in to ACARS Hub. ## Up-and-Running @@ -101,15 +104,17 @@ The document below covers a lot of configuration options, however, most of them ## Ports -| Port | Description | -| ---------- | ---------------------------------------- | -| `80` | Port used for the web interface | -| `5550/udp` | Port used for pushing ACARS JSON data to | -| `5555/udp` | Port used for pushing VDLM2 JSON data to | -| `5556/udp` | Port used for pushing HFDL JSON data to | -| `15550` | Port used for exposing JSON ACARS data | -| `15555` | Port used for exposing JSON VDLM2 data | -| `15556` | Port used for exposing JSON HFDL data | +| Port | Description | +| ---------- | -------------------------------------------------- | +| `80` | Port used for the web interface | +| `5550/udp` | Port used for pushing ACARS JSON data to | +| `5555/udp` | Port used for pushing VDLM2 JSON data to | +| `5556/udp` | Port used for pushing HFDL JSON data to | +| `5557/udp` | Port used for pushing Inmarsat L-Band JSON data to | +| `15550` | Port used for exposing JSON ACARS data | +| `15555` | Port used for exposing JSON VDLM2 data | +| `15556` | Port used for exposing JSON HFDL data | +| `15557` | Port used for exposing JSON Inmarsat L-Band data | ## Volumes / Database @@ -197,6 +202,12 @@ In the configuration options for tar1090. Setting this will include additional a | ------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------- | ------- | | `ENABLE_HFDL` | Toggle HFDL decoding on. If set to `external` this will enable HFDL processing in the container. Push valid `HFDL` data to UDP port 5556 (needs port mapping 5556:5556/udp). | No | `false` | +### Inmarsat L-Band + +| Variable | Description | Required | Default | +| ------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------- | ------- | +| `ENABLE_IMSL` | Toggle Inmarsat L-Band decoding on. If set to `external` this will enable IMSL processing in the container. Push valid `IMSL` data to UDP port 5557 (needs port mapping 5557:5557/udp). | No | `false` | + ## Viewing the messages The container implements a basic web interface, listening on port `80`, which will show messages as they are received. @@ -253,6 +264,8 @@ If there are airlines you notice that are wrong because the data used is wrong ( If you wish to access the JSON data that the decoders `acarsdec` and `dumpvdl2` generate with an external program expose the following ports in your docker-compose configuration: - Port 80 for the web site +- Port 15557 for UDP Inmarsat L-Band JSON +- Port 15556 for UDP HFDL JSON - Port 15555 for UDP VDLM2 JSON - Port 15550 for UDP ACARS JSON @@ -263,8 +276,12 @@ ports: - 80:80 - 5550:5550/udp - 5555:5555/udp + - 5556:5556/udp + - 5557:5557/udp - 15550:15550 - 15555:15555 + - 15556:15556 + - 15557:15557 ``` And then you will be able to connect to `yourpisipaddress:15555` or `yourpisipaddress:15550` respectively, in whatever program can decode ACARS/VDLM JSON. diff --git a/acarshub-typescript/package.json b/acarshub-typescript/package.json index 56f5bcb0..286c7927 100644 --- a/acarshub-typescript/package.json +++ b/acarshub-typescript/package.json @@ -1,6 +1,6 @@ { "name": "acarshub-typescript", - "version": "3.4.0", + "version": "3.5.0", "description": "Web front end for the docker-acarshub project", "main": "index.js", "repository": "https://github.com/sdr-enthusiasts/docker-acarshub", diff --git a/acarshub-typescript/src/helpers/menu.ts b/acarshub-typescript/src/helpers/menu.ts index 1fec596d..14faf868 100644 --- a/acarshub-typescript/src/helpers/menu.ts +++ b/acarshub-typescript/src/helpers/menu.ts @@ -61,19 +61,19 @@ export let menu = { generate_stat_submenu: function ( acars: boolean = false, vdlm: boolean = false, - hfdl: boolean = false + hfdl: boolean = false, + imsl: boolean = false ): void { let text: string = ""; - const acars_prefix = acars && vdlm ? "'acars'" : "''"; - const vdlm_prefix = vdlm && acars ? "'vdlm'" : "''"; - const hfdl_prefix = hfdl ? "'hfdl'" : "''"; - let show_combined = false; + let ennum = [acars, vdlm, hfdl, imsl].filter(x => x).length; + // if any two of acars, vdlm, hfdl, imsl are true, set show_combined to true + let show_combined = ennum > 1; - // if any two of acars, vdlm, hfdl are true, set show_combined to true - if (acars && vdlm) show_combined = true; - if (acars && hfdl) show_combined = true; - if (vdlm && hfdl) show_combined = true; + const acars_prefix = show_combined ? "'acars'" : "''"; + const vdlm_prefix = show_combined ? "'vdlm'" : "''"; + const hfdl_prefix = show_combined ? "'hfdl'" : "''"; + const imsl_prefix = show_combined ? "'imsl'" : "''"; if (show_combined) { text = @@ -82,22 +82,28 @@ export let menu = { if (acars) { text += `${ - acars && vdlm ? " | " : "" + show_combined ? " | " : "" }ACARS Graphs`; } if (vdlm) { text += `${ - acars && vdlm ? " | " : "" + show_combined ? " | " : "" }VDLM Graphs`; } if (hfdl) { text += `${ - acars && vdlm ? " | " : "" + show_combined ? " | " : "" }HFDL Graphs`; } + if (imsl) { + text += `${ + show_combined ? " | " : "" + }IMSL Graphs`; + } + text += ' | Message Error Graphs'; $("#stat_menu").html(text); diff --git a/acarshub-typescript/src/index.ts b/acarshub-typescript/src/index.ts index d1c3abb2..d9134d36 100644 --- a/acarshub-typescript/src/index.ts +++ b/acarshub-typescript/src/index.ts @@ -630,9 +630,10 @@ export function sound_alert(): void { export function generate_stat_submenu( acars: boolean = false, vdlm: boolean = false, - hfdl: boolean = false + hfdl: boolean = false, + imsl: boolean = false ): void { - menu.generate_stat_submenu(acars, vdlm, hfdl); + menu.generate_stat_submenu(acars, vdlm, hfdl, imsl); } export function find_matches(): plane_data { diff --git a/acarshub-typescript/src/interfaces.ts b/acarshub-typescript/src/interfaces.ts index 2537d884..1ec5ac71 100644 --- a/acarshub-typescript/src/interfaces.ts +++ b/acarshub-typescript/src/interfaces.ts @@ -70,6 +70,7 @@ export interface decoders { acars: boolean; vdlm: boolean; hfdl: boolean; + imsl: boolean; arch: string; allow_remote_updates: boolean; adsb: { diff --git a/acarshub-typescript/src/pages/stats.ts b/acarshub-typescript/src/pages/stats.ts index 19abddf6..b6f6debf 100644 --- a/acarshub-typescript/src/pages/stats.ts +++ b/acarshub-typescript/src/pages/stats.ts @@ -44,6 +44,7 @@ export let stats_page = { chart_frequency_data_acars: (null) as Chart, chart_frequency_data_vdlm: (null) as Chart, chart_frequency_data_hfdl: (null) as Chart, + chart_frequency_data_imsl: (null) as Chart, chart_message_counts_data: (null) as Chart, chart_message_counts_empty: (null) as Chart, @@ -55,6 +56,7 @@ export let stats_page = { acars_on: false as boolean, vdlm_on: false as boolean, hfdl_on: false as boolean, + imsl_on: false as boolean, width: 1000 as number, tol: new palette("tol", 12, 0, "").map(function (hex: any) { @@ -204,34 +206,42 @@ export let stats_page = { let freq_data_acars: number[] = []; let freq_data_vdlm: number[] = []; let freq_data_hfdl: number[] = []; + let freq_data_imsl: number[] = []; let freq_labels_acars: string[] = []; let freq_labels_vdlm: string[] = []; let freq_labels_hfdl: string[] = []; + let freq_labels_imsl: string[] = []; let freq_labels_acars_positions: string[] = []; let freq_labels_vdlm_positions: string[] = []; let freq_labels_hfdl_positions: string[] = []; + let freq_labels_imsl_positions: string[] = []; let freq_labels_acars_offset: number[] = []; let freq_labels_vdlm_offset: number[] = []; let freq_labels_hfdl_offset: number[] = []; + let freq_labels_imsl_offset: number[] = []; let total_count_acars: number = 0; let total_count_vdlm: number = 0; let total_count_hfdl: number = 0; + let total_count_imsl: number = 0; let acars_offset: number = 5; let vdlm_offset: number = 5; let hfdl_offset: number = 5; + let imsl_offset: number = 5; Object.entries(this.freqs_data.freqs).forEach(([key, value]) => { if (value.freq_type === "ACARS") { total_count_acars += value.count; - } else if (value.freq_type === "HFDL") { - total_count_hfdl += value.count; } else if (value.freq_type === "VDL-M2") { total_count_vdlm += value.count; + } else if (value.freq_type === "HFDL") { + total_count_hfdl += value.count; + } else if (value.freq_type === "IMS-L") { + total_count_imsl += value.count; } else { console.error("Unknown freq type: " + value.freq_type); } @@ -274,6 +284,18 @@ export let stats_page = { freq_labels_hfdl_offset.push(hfdl_offset); hfdl_offset += 60; } + } else if (value.freq_type === "IMS-L") { + freq_data_imsl.push(value.count); + freq_labels_imsl.push(value.freq); + + if (value.count / total_count_imsl > 0.2) { + freq_labels_imsl_positions.push("center"); + freq_labels_imsl_offset.push(0); + } else { + freq_labels_imsl_positions.push("end"); + freq_labels_imsl_offset.push(imsl_offset); + imsl_offset += 60; + } } else { console.error("Unknown freq type: " + value.freq_type); } @@ -291,6 +313,10 @@ export let stats_page = { this.chart_frequency_data_hfdl.destroy(); } + if (this.chart_frequency_data_imsl !== null) { + this.chart_frequency_data_imsl.destroy(); + } + if (freq_data_acars.length > 0) { this.render_freq_graph( "ACARS", @@ -323,6 +349,17 @@ export let stats_page = { "#hfdl_freq_graph" ); } + + if (freq_data_imsl.length > 0) { + this.render_freq_graph( + "IMSL", + freq_labels_imsl, + freq_data_imsl, + total_count_imsl, + "frequencies_imsl", + "#imsl_freq_graph" + ); + } } }, @@ -420,6 +457,8 @@ export let stats_page = { this.chart_frequency_data_vdlm = temp_chart; } else if (label === "HFDL") { this.chart_frequency_data_hfdl = temp_chart; + } else if (label === "IMSL") { + this.chart_frequency_data_imsl = temp_chart; } // clamp the height of the parent container to the height of the chart based on the number of elements // this is a hack to get the chart to display properly @@ -433,21 +472,15 @@ export let stats_page = { typeof this.count_data !== "undefined" && typeof this.count_data.count !== "undefined" ) { - const total: number = - this.count_data.count.non_empty_total + - this.count_data.count.empty_total + - this.count_data.count.non_empty_errors; - const total_non_empty: number = - this.count_data.count.non_empty_total + - this.count_data.count.non_empty_errors; - const error: number = this.count_data.count.non_empty_errors; - const good_msg: number = this.count_data.count.non_empty_total - error; + const data_error: number = this.count_data.count.non_empty_errors; + const data_good: number = this.count_data.count.non_empty_total; + const data_total: number = data_error + data_good; const empty_error: number = this.count_data.count.empty_errors; const empty_good: number = this.count_data.count.empty_total; const empty_total: number = empty_error + empty_good; - const counts_data: number[] = [good_msg, error]; + const counts_data: number[] = [data_good, data_error]; const counts_empty: number[] = [empty_good, empty_error]; const count_labels: string[] = [ @@ -500,7 +533,7 @@ export let stats_page = { }, title: { display: true, - text: `Non-Empty Messages (${total_non_empty.toLocaleString()})`, + text: `Non-Empty Messages (${data_total.toLocaleString()})`, }, datalabels: { backgroundColor: function (context: any) { @@ -518,7 +551,7 @@ export let stats_page = { " (" + // count_labels[context.dataIndex] + // "\n" + - ((value / total_non_empty) * 100) + ((value / data_total) * 100) .toFixed(2) .toLocaleString() + "%) " @@ -607,9 +640,10 @@ export let stats_page = { this.acars_on = msg.acars; this.vdlm_on = msg.vdlm; this.hfdl_on = msg.hfdl; + this.imsl_on = msg.imsl; if (this.stats_page_active) - generate_stat_submenu(this.acars_on, this.vdlm_on, this.hfdl_on); + generate_stat_submenu(this.acars_on, this.vdlm_on, this.hfdl_on, this.imsl_on); }, signals: function (msg: signal): void { @@ -733,6 +767,11 @@ export let stats_page = { ? '
 
' : "" } + ${ + this.imsl_on + ? '
 
' + : "" + }
 
 
@@ -782,7 +821,7 @@ export let stats_page = { Chart.register(...registerables); // page is active this.set_html(); - generate_stat_submenu(this.acars_on, this.vdlm_on, this.hfdl_on); + generate_stat_submenu(this.acars_on, this.vdlm_on, this.hfdl_on, this.imsl_on); this.show_signal_chart(); this.show_alert_chart(); this.show_count(); diff --git a/rootfs/etc/s6-overlay/s6-rc.d/imsl_server/dependencies.d/01-acarshub b/rootfs/etc/s6-overlay/s6-rc.d/imsl_server/dependencies.d/01-acarshub new file mode 100644 index 00000000..e69de29b diff --git a/rootfs/etc/s6-overlay/s6-rc.d/imsl_server/dependencies.d/02-database b/rootfs/etc/s6-overlay/s6-rc.d/imsl_server/dependencies.d/02-database new file mode 100644 index 00000000..e69de29b diff --git a/rootfs/etc/s6-overlay/s6-rc.d/imsl_server/dependencies.d/03-nginx b/rootfs/etc/s6-overlay/s6-rc.d/imsl_server/dependencies.d/03-nginx new file mode 100644 index 00000000..e69de29b diff --git a/rootfs/etc/s6-overlay/s6-rc.d/imsl_server/dependencies.d/04-rrdvalidator b/rootfs/etc/s6-overlay/s6-rc.d/imsl_server/dependencies.d/04-rrdvalidator new file mode 100644 index 00000000..e69de29b diff --git a/rootfs/etc/s6-overlay/s6-rc.d/imsl_server/run b/rootfs/etc/s6-overlay/s6-rc.d/imsl_server/run new file mode 100755 index 00000000..6ae7b02d --- /dev/null +++ b/rootfs/etc/s6-overlay/s6-rc.d/imsl_server/run @@ -0,0 +1,2 @@ +#!/bin/sh +exec /etc/s6-overlay/scripts/imsl_server diff --git a/rootfs/etc/s6-overlay/s6-rc.d/imsl_server/type b/rootfs/etc/s6-overlay/s6-rc.d/imsl_server/type new file mode 100644 index 00000000..5883cff0 --- /dev/null +++ b/rootfs/etc/s6-overlay/s6-rc.d/imsl_server/type @@ -0,0 +1 @@ +longrun diff --git a/rootfs/etc/s6-overlay/s6-rc.d/imsl_stats/dependencies.d/01-acarshub b/rootfs/etc/s6-overlay/s6-rc.d/imsl_stats/dependencies.d/01-acarshub new file mode 100644 index 00000000..e69de29b diff --git a/rootfs/etc/s6-overlay/s6-rc.d/imsl_stats/dependencies.d/02-database b/rootfs/etc/s6-overlay/s6-rc.d/imsl_stats/dependencies.d/02-database new file mode 100644 index 00000000..e69de29b diff --git a/rootfs/etc/s6-overlay/s6-rc.d/imsl_stats/dependencies.d/03-nginx b/rootfs/etc/s6-overlay/s6-rc.d/imsl_stats/dependencies.d/03-nginx new file mode 100644 index 00000000..e69de29b diff --git a/rootfs/etc/s6-overlay/s6-rc.d/imsl_stats/dependencies.d/04-rrdvalidator b/rootfs/etc/s6-overlay/s6-rc.d/imsl_stats/dependencies.d/04-rrdvalidator new file mode 100644 index 00000000..e69de29b diff --git a/rootfs/etc/s6-overlay/s6-rc.d/imsl_stats/run b/rootfs/etc/s6-overlay/s6-rc.d/imsl_stats/run new file mode 100755 index 00000000..2e253a81 --- /dev/null +++ b/rootfs/etc/s6-overlay/s6-rc.d/imsl_stats/run @@ -0,0 +1,2 @@ +#!/bin/sh +exec /etc/s6-overlay/scripts/imsl_stats diff --git a/rootfs/etc/s6-overlay/s6-rc.d/imsl_stats/type b/rootfs/etc/s6-overlay/s6-rc.d/imsl_stats/type new file mode 100644 index 00000000..5883cff0 --- /dev/null +++ b/rootfs/etc/s6-overlay/s6-rc.d/imsl_stats/type @@ -0,0 +1 @@ +longrun diff --git a/rootfs/etc/s6-overlay/s6-rc.d/user/contents.d/imsl_server b/rootfs/etc/s6-overlay/s6-rc.d/user/contents.d/imsl_server new file mode 100644 index 00000000..e69de29b diff --git a/rootfs/etc/s6-overlay/s6-rc.d/user/contents.d/imsl_stats b/rootfs/etc/s6-overlay/s6-rc.d/user/contents.d/imsl_stats new file mode 100644 index 00000000..e69de29b diff --git a/rootfs/etc/s6-overlay/scripts/04-rrdvalidator b/rootfs/etc/s6-overlay/scripts/04-rrdvalidator index 2ea9270f..d1d03be4 100755 --- a/rootfs/etc/s6-overlay/scripts/04-rrdvalidator +++ b/rootfs/etc/s6-overlay/scripts/04-rrdvalidator @@ -16,9 +16,13 @@ if [[ ! -e "${RRD_FILE}" ]]; then fi # check and see if the grep output of rrdinfo containers 'HFDL'. If it does, exit 0 -if rrdtool info "${RRD_FILE}" | grep -q 'HFDL'; then - exit 0 -else +if ! rrdtool info "${RRD_FILE}" | grep -q 'HFDL'; then rrdtool tune "${RRD_FILE}" DS:HFDL:GAUGE:120:U:U echo "[04-rrdvalidator] Added HFDL to ${RRD_FILE}" fi + +# check and see if the grep output of rrdinfo containers 'IMSL'. If it does, exit 0 +if ! rrdtool info "${RRD_FILE}" | grep -q 'IMSL'; then + rrdtool tune "${RRD_FILE}" DS:IMSL:GAUGE:120:U:U + echo "[04-rrdvalidator] Added IMSL to ${RRD_FILE}" +fi diff --git a/rootfs/etc/s6-overlay/scripts/imsl_server b/rootfs/etc/s6-overlay/scripts/imsl_server new file mode 100755 index 00000000..13951cee --- /dev/null +++ b/rootfs/etc/s6-overlay/scripts/imsl_server @@ -0,0 +1,23 @@ +#!/command/with-contenv bash +#shellcheck shell=bash + +if [[ ${ENABLE_IMSL,,} =~ external ]]; then + + set -o pipefail + + if true || [[ $((MIN_LOG_LEVEL)) -ge 4 ]]; then + # shellcheck disable=SC2016 + echo "Starting service" | stdbuf -oL awk '{print "[imsl_server] " strftime("%Y/%m/%d %H:%M:%S", systime()) " " $0}' + fi + + set -e + + # Listen for the output of acars_router / imsl, and make it available for multiple processes at TCP port 15557 + # shellcheck disable=SC2016 + { + socat -T 60 -u udp-listen:5557,fork,reuseaddr stdout | { cat; kill -s INT 0; } \ + | ncat -4 --keep-open --listen 0.0.0.0 15557 | { cat; kill -s INT 0; } + } 2>&1 | stdbuf -oL awk '{print "[imsl_server] " strftime("%Y/%m/%d %H:%M:%S", systime()) " " $0}' +else + sleep 86400 +fi diff --git a/rootfs/etc/s6-overlay/scripts/imsl_stats b/rootfs/etc/s6-overlay/scripts/imsl_stats new file mode 100755 index 00000000..847ef11d --- /dev/null +++ b/rootfs/etc/s6-overlay/scripts/imsl_stats @@ -0,0 +1,43 @@ +#!/command/with-contenv bash +#shellcheck shell=bash + +if [[ ${ENABLE_IMSL,,} =~ external ]]; then + + set -o pipefail + + # Require that imsl_server is running + if ! netstat -an | grep -P '^\s*tcp\s+\d+\s+\d+\s+0\.0\.0\.0:15557\s+(?>\d{1,3}\.{0,1}){4}:\*\s+LISTEN\s*$' > /dev/null; then + if [[ $((MIN_LOG_LEVEL)) -ge 4 ]]; then + # shellcheck disable=SC2016 + echo "Waiting for imsl_server" | stdbuf -oL awk '{print "[imsl_stats ] " strftime("%Y/%m/%d %H:%M:%S", systime()) " " $0}' + fi + sleep 1 + exit + fi + if [[ $((MIN_LOG_LEVEL)) -ge 4 ]]; then + # shellcheck disable=SC2016 + echo "imsl_server ready, starting service" | stdbuf -oL awk '{print "[imsl_stats ] " strftime("%Y/%m/%d %H:%M:%S", systime()) " " $0}' + fi + + # Start our stats loop + while true; do + + # capture 5 mins of flows + timeout --foreground 300s socat -u TCP:127.0.0.1:15557 CREATE:/database/imsl.past5min.json + + # shellcheck disable=SC2016 + echo "$(sed 's/}{/}\n{/g' /database/imsl.past5min.json | wc -l) IMSL messages received in last 5 mins" | stdbuf -oL awk '{print "[imsl_stats ] " strftime("%Y/%m/%d %H:%M:%S", systime()) " " $0}' + + # rotate files keeping last 2 hours + for i in {24..1}; do + mv "/database/imsl.$((i-1)).json" "/database/imsl.$i.json" > /dev/null 2>&1 || true + done + mv "/database/imsl.past5min.json" "/database/imsl.0.json" > /dev/null 2>&1 || true + + done + +else + + # If here then IMSL is not enabled, sleep forever + sleep 86400 +fi diff --git a/rootfs/etc/s6-overlay/scripts/rrd-images b/rootfs/etc/s6-overlay/scripts/rrd-images index da1ccfc4..8ff2afa9 100755 --- a/rootfs/etc/s6-overlay/scripts/rrd-images +++ b/rootfs/etc/s6-overlay/scripts/rrd-images @@ -12,6 +12,7 @@ if [[ ${ENABLE_WEB,,} =~ true ]]; then ARGS_VDLM=("-a" "PNG" "-w" "1000" "-h" "200" "--vertical-label" "Messages" "--slope-mode") ARGS_ACARS=("-a" "PNG" "-w" "1000" "-h" "200" "--vertical-label" "Messages" "--slope-mode") ARGS_HFDL=("-a" "PNG" "-w" "1000" "-h" "200" "--vertical-label" "Messages" "--slope-mode") + ARGS_IMSL=("-a" "PNG" "-w" "1000" "-h" "200" "--vertical-label" "Messages" "--slope-mode") total_enabled=0 @@ -33,7 +34,13 @@ if [[ ${ENABLE_WEB,,} =~ true ]]; then total_enabled=$((total_enabled + 1)) fi - # if any two or more of ENABLE_ACARS, ENABLE_VDLM, ENABLE_HFDL is external then we need a totals graph + if [[ ${ENABLE_IMSL,,} =~ external ]]; then + ARGS_ALL+=("DEF:messages-imsl=/run/acars/acarshub.rrd:IMSL:AVERAGE" "LINE1:messages-imsl#DA70D6:IMSL") + ARGS_IMSL+=("DEF:messages-imsl=/run/acars/acarshub.rrd:IMSL:AVERAGE" "LINE1:messages-imsl#DA70D6:IMSL") + total_enabled=$((total_enabled + 1)) + fi + + # if any two or more of ENABLE_ACARS, ENABLE_VDLM, ENABLE_HFDL, ENABLE_IMSL is external then we need a totals graph if [[ ${total_enabled} -ge 2 ]]; then ARGS_ALL+=("DEF:messages-total=/run/acars/acarshub.rrd:TOTAL:AVERAGE" "LINE1:messages-total#00FF00:Total") fi @@ -116,6 +123,21 @@ if [[ ${ENABLE_WEB,,} =~ true ]]; then rrdtool graph "${OUTPUT_DIR}/hfdl1year.png" --title "1 Year" --start "-1y" "${ARGS_HFDL[@]}" >/dev/null 2>&1 fi fi + + if [[ ${ENABLE_IMSL,,} =~ external ]]; then + # IMSL Graphs + + rrdtool graph "${OUTPUT_DIR}/imsl1hour.png" --title "1 Hour" --start "-1h" "${ARGS_IMSL[@]}" >/dev/null 2>&1 + rrdtool graph "${OUTPUT_DIR}/imsl6hour.png" --title "6 Hours" --start "-6h" "${ARGS_IMSL[@]}" >/dev/null 2>&1 + rrdtool graph "${OUTPUT_DIR}/imsl12hour.png" --title "12 Hours" --start "-12h" "${ARGS_IMSL[@]}" >/dev/null 2>&1 + rrdtool graph "${OUTPUT_DIR}/imsl24hours.png" --title "1 Day" --start "-1d" "${ARGS_IMSL[@]}" >/dev/null 2>&1 + rrdtool graph "${OUTPUT_DIR}/imsl1week.png" --title "1 Week" --start "-7d" "${ARGS_IMSL[@]}" >/dev/null 2>&1 + if [[ "${STARTING_UP}" == "TRUE" || $(date +%M) == "00" ]]; then + rrdtool graph "${OUTPUT_DIR}/imsl30days.png" --title "1 Month" --start "-1mon" "${ARGS_IMSL[@]}" >/dev/null 2>&1 + rrdtool graph "${OUTPUT_DIR}/imsl6months.png" --title "6 Months" --start "-6mon" "${ARGS_IMSL[@]}" >/dev/null 2>&1 + rrdtool graph "${OUTPUT_DIR}/imsl1year.png" --title "1 Year" --start "-1y" "${ARGS_IMSL[@]}" >/dev/null 2>&1 + fi + fi fi if [[ $((MIN_LOG_LEVEL)) -ge 4 ]]; then diff --git a/rootfs/scripts/healthcheck.sh b/rootfs/scripts/healthcheck.sh index 1edd2f6f..68deeb33 100755 --- a/rootfs/scripts/healthcheck.sh +++ b/rootfs/scripts/healthcheck.sh @@ -50,6 +50,114 @@ EXITCODE=0 # } +# ===== Check imsl_server, imsl_feeder, imsl_stats processes ===== + +if [[ ${ENABLE_IMSL,,} =~ external ]]; then + + echo "==== Checking imsl_server =====" + + # Check imsl_server is listening for TCP on 127.0.0.1:15557 + imsl_pidof_imsl_tcp_server=$(pgrep -f 'ncat -4 --keep-open --listen 0.0.0.0 15557') + if ! check_tcp4_socket_listening_for_pid "0.0.0.0" "15557" "${imsl_pidof_imsl_tcp_server}"; then + echo "imsl_server TCP not listening on port 15557 (pid $imsl_pidof_imsl_tcp_server): UNHEALTHY" + EXITCODE=1 + else + echo "imsl_server TCP listening on port 15557 (pid $imsl_pidof_imsl_tcp_server): HEALTHY" + fi + + if [[ ${ENABLE_WEB,,} =~ true ]]; then + if ! netstat -anp | grep -P "tcp\s+\d+\s+\d+\s+127.0.0.1:[0-9]+\s+127.0.0.1:15557\s+ESTABLISHED\s+[0-9]+/python3" > /dev/null 2>&1; then + echo "TCP4 connection between 127.0.0.1:ANY and 127.0.0.1:15557 for python3 established: FAIL" + echo "imsl_server TCP connected to python server on port 15557 (pid $imsl_pidof_imsl_tcp_server): UNHEALTHY" + EXITCODE=1 + else + echo "TCP4 connection between 127.0.0.1:ANY and 127.0.0.1:15557 for python3 established: PASS" + echo "imsl_server TCP connected to python server on port 15557: HEALTHY" + fi + fi + + echo "==== Checking imsl_stats =====" + + # Check imsl_stats: + imsl_pidof_imsl_stats=$(pgrep -fx 'socat -u TCP:127.0.0.1:15557 CREATE:/database/imsl.past5min.json') + + # Ensure TCP connection to imsl_server at 127.0.0.1:15557 + if ! check_tcp4_connection_established_for_pid "127.0.0.1" "ANY" "127.0.0.1" "15557" "${imsl_pidof_imsl_stats}"; then + echo "imsl_stats (pid $imsl_pidof_imsl_stats) not connected to imsl_server (pid $imsl_pidof_imsl_tcp_server) at 127.0.0.1:15557: UNHEALTHY" + EXITCODE=1 + else + echo "imsl_stats (pid $imsl_pidof_imsl_stats) connected to imsl_server (pid $imsl_pidof_imsl_tcp_server) at 127.0.0.1:15557: HEALTHY" + fi + + echo "==== Check for IMSL activity =====" + + # Check for activity + # read .json files, ensure messages received in past hour + + imsl_num_msgs_past_hour=$(find /database -type f -name 'imsl.*.json' -cmin -60 -exec cat {} \; | sed -e 's/}{/}\n{/g' | wc -l) + if [[ "$imsl_num_msgs_past_hour" -gt 0 ]]; then + echo "$imsl_num_msgs_past_hour IMSL messages received in past hour: HEALTHY" + else + echo "$imsl_num_msgs_past_hour IMSL messages received in past hour: UNHEALTHY" + EXITCODE=1 + fi + +fi + +# ===== Check hfdl_server, hfdl_feeder, hfdl_stats processes ===== + +if [[ ${ENABLE_HFDL,,} =~ external ]]; then + + echo "==== Checking hfdl_server =====" + + # Check hfdl_server is listening for TCP on 127.0.0.1:15556 + hfdl_pidof_hfdl_tcp_server=$(pgrep -f 'ncat -4 --keep-open --listen 0.0.0.0 15556') + if ! check_tcp4_socket_listening_for_pid "0.0.0.0" "15556" "${hfdl_pidof_hfdl_tcp_server}"; then + echo "hfdl_server TCP not listening on port 15556 (pid $hfdl_pidof_hfdl_tcp_server): UNHEALTHY" + EXITCODE=1 + else + echo "hfdl_server TCP listening on port 15556 (pid $hfdl_pidof_hfdl_tcp_server): HEALTHY" + fi + + if [[ ${ENABLE_WEB,,} =~ true ]]; then + if ! netstat -anp | grep -P "tcp\s+\d+\s+\d+\s+127.0.0.1:[0-9]+\s+127.0.0.1:15556\s+ESTABLISHED\s+[0-9]+/python3" > /dev/null 2>&1; then + echo "TCP4 connection between 127.0.0.1:ANY and 127.0.0.1:15556 for python3 established: FAIL" + echo "hfdl_server TCP connected to python server on port 15556 (pid $hfdl_pidof_hfdl_tcp_server): UNHEALTHY" + EXITCODE=1 + else + echo "TCP4 connection between 127.0.0.1:ANY and 127.0.0.1:15556 for python3 established: PASS" + echo "hfdl_server TCP connected to python server on port 15556: HEALTHY" + fi + fi + + echo "==== Checking hfdl_stats =====" + + # Check hfdl_stats: + hfdl_pidof_hfdl_stats=$(pgrep -fx 'socat -u TCP:127.0.0.1:15556 CREATE:/database/hfdl.past5min.json') + + # Ensure TCP connection to hfdl_server at 127.0.0.1:15556 + if ! check_tcp4_connection_established_for_pid "127.0.0.1" "ANY" "127.0.0.1" "15556" "${hfdl_pidof_hfdl_stats}"; then + echo "hfdl_stats (pid $hfdl_pidof_hfdl_stats) not connected to hfdl_server (pid $hfdl_pidof_hfdl_tcp_server) at 127.0.0.1:15556: UNHEALTHY" + EXITCODE=1 + else + echo "hfdl_stats (pid $hfdl_pidof_hfdl_stats) connected to hfdl_server (pid $hfdl_pidof_hfdl_tcp_server) at 127.0.0.1:15556: HEALTHY" + fi + + echo "==== Check for HFDL activity =====" + + # Check for activity + # read .json files, ensure messages received in past hour + + hfdl_num_msgs_past_hour=$(find /database -type f -name 'hfdl.*.json' -cmin -60 -exec cat {} \; | sed -e 's/}{/}\n{/g' | wc -l) + if [[ "$hfdl_num_msgs_past_hour" -gt 0 ]]; then + echo "$hfdl_num_msgs_past_hour HFDL messages received in past hour: HEALTHY" + else + echo "$hfdl_num_msgs_past_hour HFDL messages received in past hour: UNHEALTHY" + EXITCODE=1 + fi + +fi + # ===== Check vdlm2_server, vdlm2_feeder, vdlm2_stats processes ===== if [[ ${ENABLE_VDLM,,} =~ external ]]; then @@ -83,10 +191,10 @@ if [[ ${ENABLE_VDLM,,} =~ external ]]; then # Ensure TCP connection to vdlm2_server at 127.0.0.1:15555 if ! check_tcp4_connection_established_for_pid "127.0.0.1" "ANY" "127.0.0.1" "15555" "${vdlm2_pidof_vdlm2_stats}"; then - echo "vdlm2_stats (pid $vdlm2_pidof_vdlm2_stats) not connected to acars_server (pid $vdlm2_pidof_vdlm2_tcp_server) at 127.0.0.1:15555: UNHEALTHY" + echo "vdlm2_stats (pid $vdlm2_pidof_vdlm2_stats) not connected to vdlm2_server (pid $vdlm2_pidof_vdlm2_tcp_server) at 127.0.0.1:15555: UNHEALTHY" EXITCODE=1 else - echo "vdlm2_stats (pid $vdlm2_pidof_vdlm2_stats) connected to acars_server (pid $vdlm2_pidof_vdlm2_tcp_server) at 127.0.0.1:15555: HEALTHY" + echo "vdlm2_stats (pid $vdlm2_pidof_vdlm2_stats) connected to vdlm2_server (pid $vdlm2_pidof_vdlm2_tcp_server) at 127.0.0.1:15555: HEALTHY" fi echo "==== Check for VDLM2 activity =====" @@ -157,8 +265,8 @@ if [[ ${ENABLE_ACARS,,} =~ external ]]; then fi -# If either ENABLE_VDLM or ENABLE_ACARS is set: -if [[ ${ENABLE_ACARS,,} =~ external ]] || [[ ${ENABLE_VDLM,,} =~ external ]]; then +# If ENABLE_VDLM or ENABLE_ACARS or ENABLE_HFDL is set: +if [[ ${ENABLE_ACARS,,} =~ external ]] || [[ ${ENABLE_VDLM,,} =~ external ]] || [[ ${ENABLE_HFDL,,} =~ external ]]; then echo "==== Check webapp =====" diff --git a/rootfs/webapp/acars_formatter.py b/rootfs/webapp/acars_formatter.py index cf87b3a6..76c9d735 100755 --- a/rootfs/webapp/acars_formatter.py +++ b/rootfs/webapp/acars_formatter.py @@ -26,9 +26,140 @@ def format_acars_message(acars_message): if "hfdl" in acars_message: return format_hfdl_message(acars_message) + if acars_message.get("source", {}).get("app", {}).get("name") == "SatDump": + if acars_message.get("msg_name") == "ACARS": + return format_satdump_imsl_message(acars_message) + else: + return None + + if acars_message.get("app", {}).get("name") == "JAERO": + return format_jaero_imsl_message(acars_message) + return acars_message +def count_errors(unformatted_message): + total_errors = 0 + for key, value in unformatted_message.items(): + if type(value) is dict: + total_errors += count_errors(value) + else: + if key == "err" and value: + total_errors += 1 + return total_errors + + +def format_jaero_imsl_message(unformatted_message): + imsl_message = dict() + + imsl_message["error"] = count_errors(unformatted_message) + + if t := unformatted_message.get("t"): + if sec := t.get("sec"): + imsl_message["timestamp"] = sec + + if station := unformatted_message.get("station"): + imsl_message["station_id"] = station + + if isu := unformatted_message.get("isu"): + if acars := isu.get("acars"): + if msg_text := acars.get("msg_text"): + imsl_message["text"] = msg_text + + if arinc622 := acars.get("arinc622"): + imsl_message["libacars"] = json.dumps(arinc622) + if gs_addr := arinc622.get("gs_addr"): + imsl_message["fromaddr_decoded"] = gs_addr + + if ack := acars.get("ack"): + imsl_message["ack"] = ack + + if blk_id := acars.get("blk_id"): + imsl_message["block_id"] = blk_id + + if label := acars.get("label"): + imsl_message["label"] = label + + if mode := acars.get("mode"): + imsl_message["mode"] = mode + + if reg := acars.get("reg"): + imsl_message["tail"] = reg + + if dst := isu.get("dst"): + if addr := dst.get("addr"): + imsl_message["toaddr"] = int(addr, 16) + imsl_message["icao"] = int(addr, 16) + + if src := isu.get("src"): + if addr := src.get("addr"): + imsl_message["fromaddr"] = addr + + if refno := isu.get("refno"): + imsl_message["msgno"] = refno + + return imsl_message + + +def format_satdump_imsl_message(unformatted_message): + imsl_message = dict() + + if timestamp := unformatted_message.get("timestamp"): + imsl_message["timestamp"] = timestamp + + if station_id := unformatted_message.get("source", {}).get("station_id"): + imsl_message["station_id"] = station_id + + if freq := unformatted_message.get("freq"): + imsl_message["freq"] = freq + + if level := unformatted_message.get("level"): + imsl_message["level"] = level + + imsl_message["error"] = count_errors(unformatted_message) + + if mode := unformatted_message.get("mode"): + imsl_message["mode"] = mode + + if label := unformatted_message.get("label"): + imsl_message["label"] = label.replace("\x7f", "d") + + if bi := unformatted_message.get("bi"): + imsl_message["block_id"] = bi + + if message := unformatted_message.get("message"): + imsl_message["text"] = message + + if more_to_come := unformatted_message.get("more_to_come"): + imsl_message["end"] = not more_to_come + + if plane_reg := unformatted_message.get("plane_reg"): + imsl_message["tail"] = plane_reg.replace(".", "") + + if tak := unformatted_message.get("tak"): + imsl_message["ack"] = chr(tak).replace(chr(0x15), "!") + + if libacars := unformatted_message.get("libacars"): + imsl_message["libacars"] = json.dumps(libacars) + + if flight := unformatted_message.get("flight"): + imsl_message["flight"] = flight + + if fromaddr_decoded := unformatted_message.get("fromaddr_decoded"): + imsl_message["fromaddr_decoded"] = fromaddr_decoded + + if sigunit := unformatted_message.get("signal_unit"): + if aes_id := sigunit.get("aes_id"): + imsl_message["toaddr"] = aes_id + imsl_message["icao"] = aes_id + if ges_id := sigunit.get("ges_id"): + imsl_message["fromaddr"] = ges_id + if ref_no := sigunit.get("ref_no"): + imsl_message["msgno"] = ref_no + + return imsl_message + + def format_hfdl_freq(unformatted_freq): # input is in Hz # output is in MHz @@ -43,17 +174,6 @@ def format_hfdl_freq(unformatted_freq): return truncated -def count_hfdl_errors(unformatted_message): - total_errors = 0 - for key, value in unformatted_message.items(): - if type(value) is dict: - total_errors += count_hfdl_errors(value) - else: - if key == "err" and value: - total_errors += 1 - return total_errors - - def format_hfdl_message(unformatted_message): hfdl_message = dict() libacars = dict() @@ -66,7 +186,7 @@ def format_hfdl_message(unformatted_message): # error # walk the entire message and look for err fields. Count the total of trues - hfdl_message["error"] = count_hfdl_errors(unformatted_message["hfdl"]) + hfdl_message["error"] = count_errors(unformatted_message["hfdl"]) # freq if "freq" in unformatted_message["hfdl"]: diff --git a/rootfs/webapp/acarshub.py b/rootfs/webapp/acarshub.py index 811d4a8f..a3b1ca13 100755 --- a/rootfs/webapp/acarshub.py +++ b/rootfs/webapp/acarshub.py @@ -66,6 +66,7 @@ thread_acars_listener = Thread() thread_vdlm2_listener = Thread() thread_hfdl_listener = Thread() +thread_imsl_listener = Thread() thread_message_listener_stop_event = Event() # db thread @@ -90,6 +91,7 @@ vdlm_messages_last_minute = 0 acars_messages_last_minute = 0 hfdl_messages_last_minute = 0 +imsl_messages_last_minute = 0 error_messages_last_minute = 0 # all namespaces @@ -118,17 +120,20 @@ def update_rrd_db(): global acars_messages_last_minute global error_messages_last_minute global hfdl_messages_last_minute + global imsl_messages_last_minute acarshub_rrd_database.update_db( vdlm=vdlm_messages_last_minute, acars=acars_messages_last_minute, error=error_messages_last_minute, hfdl=hfdl_messages_last_minute, + imsl=imsl_messages_last_minute, ) vdlm_messages_last_minute = 0 acars_messages_last_minute = 0 error_messages_last_minute = 0 hfdl_messages_last_minute = 0 + imsl_messages_last_minute = 0 def generateClientMessage(message_type, json_message): @@ -153,6 +158,8 @@ def getQueType(message_type): return "ACARS" elif message_type == "HFDL": return "HFDL" + elif message_type == "IMSL": + return "IMS-L" elif message_type is not None: return str(message_type) else: @@ -245,6 +252,8 @@ def message_listener(message_type=None, ip="127.0.0.1", port=None): global acars_messages_last_minute elif message_type == "HFDL": global hfdl_messages_last_minute + elif message_type == "IMSL": + global imsl_messages_last_minute disconnected = True @@ -344,14 +353,14 @@ def message_listener(message_type=None, ip="127.0.0.1", port=None): acarshub_logging.log( "Reassembly successful, message not skipped after all!", f"{message_type.lower()}Generator", - 1, + level=LOG_LEVEL["DEBUG"] ) except Exception as e: # reassembly didn't work, don't do anything but print an error when debug is enabled acarshub_logging.log( f"Reassembly failed {e}: {combined}", f"{message_type.lower()}Generator", - level=LOG_LEVEL["DEBUG"], + level=LOG_LEVEL["WARNING"] ) # forget the partial message, it can't be useful anymore @@ -366,16 +375,12 @@ def message_listener(message_type=None, ip="127.0.0.1", port=None): msg = None try: msg = json.loads(part) - except ValueError as e: + except ValueError: if part == split_json[-1]: # last element in the list, could be a partial json object partial_message = part - acarshub_logging.log( - f"JSON Error: {e}", f"{message_type.lower()}Generator", 1 - ) - acarshub_logging.log( - f"Skipping Message: {part}", f"{message_type.lower()}Generator", 1 + f"Skipping Message: {part}", f"{message_type.lower()}Generator", LOG_LEVEL["DEBUG"] ) continue except Exception as e: @@ -389,34 +394,39 @@ def message_listener(message_type=None, ip="127.0.0.1", port=None): que_type = getQueType(message_type) - if message_type == "VDLM2": - vdlm_messages_last_minute += 1 - elif message_type == "ACARS": - acars_messages_last_minute += 1 - elif message_type == "HFDL": - hfdl_messages_last_minute += 1 + formatted_message = acars_formatter.format_acars_message(msg) - if "error" in msg: - if msg["error"] > 0: - error_messages_last_minute += msg["error"] + if formatted_message: + if message_type == "VDLM2": + vdlm_messages_last_minute += 1 + elif message_type == "ACARS": + acars_messages_last_minute += 1 + elif message_type == "HFDL": + hfdl_messages_last_minute += 1 + elif message_type == "IMSL": + imsl_messages_last_minute += 1 - que_messages.append((que_type, acars_formatter.format_acars_message(msg))) - que_database.append((que_type, acars_formatter.format_acars_message(msg))) + if "error" in msg: + if msg["error"] > 0: + error_messages_last_minute += msg["error"] - if ( - len(list_of_recent_messages) >= list_of_recent_messages_max - ): # Keep the que size down - del list_of_recent_messages[0] + que_messages.append((que_type, formatted_message)) + que_database.append((que_type, formatted_message)) - if not acarshub_configuration.QUIET_MESSAGES: - print(f"MESSAGE:{message_type.lower()}Generator: {msg}") + if ( + len(list_of_recent_messages) >= list_of_recent_messages_max + ): # Keep the que size down + del list_of_recent_messages[0] - client_message = generateClientMessage( - que_type, acars_formatter.format_acars_message(msg) - ) + if not acarshub_configuration.QUIET_MESSAGES: + print(f"MESSAGE:{message_type.lower()}Generator: {msg}") - # add to recent message que for anyone fresh loading the page - list_of_recent_messages.append(client_message) + client_message = generateClientMessage( + que_type, formatted_message + ) + + # add to recent message que for anyone fresh loading the page + list_of_recent_messages.append(client_message) def init_listeners(special_message=""): @@ -425,6 +435,7 @@ def init_listeners(special_message=""): global thread_acars_listener global thread_vdlm2_listener global thread_hfdl_listener + global thread_imsl_listener global thread_database global thread_scheduler global thread_html_generator @@ -516,6 +527,22 @@ def init_listeners(special_message=""): ) thread_hfdl_listener.start() + if not thread_imsl_listener.is_alive() and acarshub_configuration.ENABLE_IMSL: + acarshub_logging.log( + f"{special_message}Starting IMSL listener", + "init", + level=LOG_LEVEL["INFO"] if special_message == "" else LOG_LEVEL["ERROR"], + ) + thread_imsl_listener = Thread( + target=message_listener, + args=( + "IMSL", + acarshub_configuration.LIVE_DATA_SOURCE, + acarshub_configuration.IMSL_SOURCE_PORT, + ), + ) + thread_imsl_listener.start() + status = acarshub_helpers.get_service_status() # grab system status # emit to all namespaces @@ -652,6 +679,7 @@ def main_connect(): "vdlm": acarshub_configuration.ENABLE_VDLM, "acars": acarshub_configuration.ENABLE_ACARS, "hfdl": acarshub_configuration.ENABLE_HFDL, + "imsl": acarshub_configuration.ENABLE_IMSL, "arch": acarshub_configuration.ARCH, "allow_remote_updates": acarshub_configuration.ALLOW_REMOTE_UPDATES, "adsb": { diff --git a/rootfs/webapp/acarshub_configuration.py b/rootfs/webapp/acarshub_configuration.py index 264f211c..a5f1f1a8 100755 --- a/rootfs/webapp/acarshub_configuration.py +++ b/rootfs/webapp/acarshub_configuration.py @@ -31,6 +31,7 @@ ENABLE_ACARS = False ENABLE_VDLM = False ENABLE_HFDL = False +ENABLE_IMSL = False DB_SAVEALL = False ACARSHUB_DB = "" IATA_OVERRIDE = "" @@ -45,6 +46,7 @@ ACARS_SOURCE_PORT = 15550 VDLM_SOURCE_PORT = 15555 HFDL_SOURCE_PORT = 15556 +IMSL_SOURCE_PORT = 15557 LIVE_DATA_SOURCE = "127.0.0.1" # This is to switch from localhost for ACARS/VDLM to connecting to a remote data source ACARSHUB_VERSION = "0" ACARSHUB_BUILD = "0" @@ -102,6 +104,12 @@ ): ENABLE_HFDL = True +if ( + os.getenv("ENABLE_IMSL", default=False) + and str(os.getenv("ENABLE_IMSL")).upper() == "EXTERNAL" +): + ENABLE_IMSL = True + if ( os.getenv("DB_SAVEALL", default=False) and str(os.getenv("DB_SAVEALL")).upper() == "TRUE" @@ -117,6 +125,9 @@ if os.getenv("HFDL_SOURCE_PORT", default=False): HFDL_SOURCE_PORT = int(os.getenv("HFDL_SOURCE_PORT")) +if os.getenv("IMSL_SOURCE_PORT", default=False): + IMSL_SOURCE_PORT = int(os.getenv("IMSL_SOURCE_PORT")) + # Application Settings if os.getenv("ACARSHUB_DB", default=False): diff --git a/rootfs/webapp/acarshub_database.py b/rootfs/webapp/acarshub_database.py index 30561644..ab8a2fa9 100755 --- a/rootfs/webapp/acarshub_database.py +++ b/rootfs/webapp/acarshub_database.py @@ -360,7 +360,7 @@ def is_message_not_empty(json_message): ] for field in fields: - if field in json_message: + if json_message.get(field): return True return False diff --git a/rootfs/webapp/acarshub_helpers.py b/rootfs/webapp/acarshub_helpers.py index c10a2b1a..84409209 100755 --- a/rootfs/webapp/acarshub_helpers.py +++ b/rootfs/webapp/acarshub_helpers.py @@ -324,7 +324,7 @@ def service_check(): continue - match = re.search("^(?:acars|vdlm2)_server", line) + match = re.search("^(?:acars|vdlm2|hfdl|imsl)_server", line) if match: if match.group(0) not in servers: @@ -349,7 +349,7 @@ def service_check(): continue - match = re.search("\\d+\\s+(?:ACARS|VDLM2) messages", line) + match = re.search("\\d+\\s+(?:ACARS|VDLM2|HFDL|IMSL) messages", line) if match: if line.find("ACARS") != -1 and "ACARS" not in receivers: @@ -380,10 +380,38 @@ def service_check(): else: system_error = True receivers["VDLM2"]["Status"] = "Unknown" + if line.find("HFDL") != -1 and "HFDL" not in receivers: + receivers["HFDL"] = dict() + receivers["HFDL"]["Count"] = line.split(" ")[0] + if line.endswith("UNHEALTHY"): + if time.time() - start_time > 300.0: + system_error = True + receivers["HFDL"]["Status"] = "Bad" + else: + receivers["HFDL"]["Status"] = "Waiting for first message" + elif line.endswith("HEALTHY"): + receivers["HFDL"]["Status"] = "Ok" + else: + system_error = True + receivers["HFDL"]["Status"] = "Unknown" + if line.find("IMSL") != -1 and "IMSL" not in receivers: + receivers["IMSL"] = dict() + receivers["IMSL"]["Count"] = line.split(" ")[0] + if line.endswith("UNHEALTHY"): + if time.time() - start_time > 300.0: + system_error = True + receivers["IMSL"]["Status"] = "Bad" + else: + receivers["IMSL"]["Status"] = "Waiting for first message" + elif line.endswith("HEALTHY"): + receivers["IMSL"]["Status"] = "Ok" + else: + system_error = True + receivers["IMSL"]["Status"] = "Unknown" continue - match = re.search("^(acars|vdlm2)_stats", line) + match = re.search("^(acars|vdlm2|hfdl|imsl)_stats", line) if match: if match.group(0) not in stats: diff --git a/rootfs/webapp/acarshub_rrd_database.py b/rootfs/webapp/acarshub_rrd_database.py index 99c2298b..2a948257 100755 --- a/rootfs/webapp/acarshub_rrd_database.py +++ b/rootfs/webapp/acarshub_rrd_database.py @@ -22,18 +22,18 @@ import os -def update_db(vdlm=0, acars=0, error=0, hfdl=0): - total = vdlm + acars + hfdl +def update_db(vdlm=0, acars=0, error=0, hfdl=0, imsl=0): + total = vdlm + acars + hfdl + imsl args = ( "--template", - "ACARS:VDLM:TOTAL:ERROR:HFDL", - f"N:{acars}:{vdlm}:{total}:{error}:{hfdl}", + "ACARS:VDLM:TOTAL:ERROR:HFDL:IMSL", + f"N:{acars}:{vdlm}:{total}:{error}:{hfdl}:{imsl}", ) try: rrdtool.update("/run/acars/acarshub.rrd", *args) acarshub_logging.log( - f"rrdtool.update: N:{acars}:{vdlm}:{total}:{error}:{hfdl}", + f"rrdtool.update: N:{acars}:{vdlm}:{total}:{error}:{hfdl}:{imsl}", "rrdtool", level=LOG_LEVEL["DEBUG"], ) @@ -56,6 +56,7 @@ def create_db(): "DS:TOTAL:GAUGE:120:U:U", "DS:ERROR:GAUGE:120:U:U", "DS:HFDL:GAUGE:120:U:U", + "DS:IMSL:GAUGE:120:U:U", "RRA:AVERAGE:0.5:1:1500", # 25 hours at 1 minute reso "RRA:AVERAGE:0.5:5:8640", # 1 month at 5 minute reso "RRA:AVERAGE:0.5:60:4320", # 6 months at 1 hour reso diff --git a/version-nextgen b/version-nextgen index c219f723..c0c4025d 100644 --- a/version-nextgen +++ b/version-nextgen @@ -1 +1 @@ -v3.4.0 +v3.5.0