WIP - Not ready for Merge - Consolidate spread Pi.Alert work

[fosspill]

This PR adds a variety of changes.
Since the development of features for Pi.Alert has been so spread I've tried my best to consolidate it all into one single package.

• Add support for ntfy.sh and discord/slack webhook
• nmap support (https://github.com/cjd/Pi.Alert) (Closes nmap support Docker-Image-for-Pi.Alert#6)
• Reorganize .config file
• Fix vendor database updates (Vendor database no longer updates pucherot/Pi.Alert#150)
• Fix small typo (typo fix pucherot/Pi.Alert#143)
• Fix duplication for "Domotic" and "Other" (Fix query so that "Domotic" and "Other" do not appear twice in the device list pucherot/Pi.Alert#118)
• Docs update (Update INSTALL.md pucherot/Pi.Alert#82)
• lighttpd matching update (match all domains ending in pi.alert in lighttpd pucherot/Pi.Alert#65)
• All changes from https://github.com/jokob-sk/Pi.Alert

The dockerfile will likely need some work for both nmap support and for curl to properly pull the device lists

Remaining:

• Some cleanup
• Install script updates
• Ensuring new config file options can be migrated properly
• Ensuring proper credit to all authors
• Dockerfile changes and tests

[fosspill]

@jokob-sk I'd gladly split this across multiple PRs when it's ready for merging

[jokob-sk]

Thanks a lot for the work on this! Happy to merge it once you think it's ready to do so.

[jokob-sk]

I've tried to build a docker image with your fork (https://github.com/fosspill/Pi.Alert). I had to update the dockerfile a bit (see below). But afterwards, it's building fine however, I'm getting an error for the NMAP scan option:

    arpscan_devices = execute_nmap (NMAP_OPTIONS, arpscan_devices)                                                                 
  File "/home/pi/pialert/back/pialert.py", line 526, in execute_nmap                                                               
    nmap_output = subprocess.check_output (nmap_args, universal_newlines=True)                                                     
  File "/usr/lib/python2.7/subprocess.py", line 216, in check_output                                                               
    process = Popen(stdout=PIPE, *popenargs, **kwargs)                                                                             
  File "/usr/lib/python2.7/subprocess.py", line 394, in __init__                                                                   
    errread, errwrite)                                                                                                             
  File "/usr/lib/python2.7/subprocess.py", line 1047, in _execute_child                                                            
    raise child_exception                                                                                                          
OSError: [Errno 2] No such file or directory 

I've also pushed the containers into a separate image if you'd like to check/test them yourself: https://hub.docker.com/r/jokobsk/pialert_merge_test

Updated Docker file:

(download this and overwrite the content of the docker file with the below)

FROM debian:buster-slim

ARG myinstall="install --no-install-recommends"

#Update and reduce image size
RUN apt update \
    && apt $myinstall -y \
    && apt $myinstall apt-utils -y \
    && apt $myinstall cron -y \
    && apt install git -y \
    && apt $myinstall sudo -y

#add the pi user
RUN useradd -ms /bin/bash pi 
WORKDIR /home/pi
# Lighttpd & PHP
RUN apt $myinstall lighttpd -y \
    && mv /var/www/html/index.lighttpd.html /var/www/html/index.lighttpd.html.old \
    && ln -s ~/pialert/install/index.html /var/www/html/index.html \
    && apt $myinstall php php-cgi php-fpm php-sqlite3 -y \
    && lighttpd-enable-mod fastcgi-php \
    && apt $myinstall sqlite3 -y

# arp-scan, Python, ip tools
RUN apt install arp-scan -y \
    && apt $myinstall dnsutils net-tools -y \
    && apt $myinstall python -y 
RUN  apt install python-pip -y \
    && pip install --upgrade pip  \
    && pip install requests \
    && apt $myinstall iproute2 -y

# Pi.Alert
RUN apt clean \
    # && git clone https://github.com/jokob-sk/Pi.Alert.git pialert     \ 
    && git clone https://github.com/fosspill/Pi.Alert pialert     \ 
    # delete .git specific files to make the image smaller
    && rm -r /home/pi/pialert/.git \
    && ln -s /home/pi/pialert/front /var/www/html/pialert  \
    && python /home/pi/pialert/back/pialert.py update_vendors \    
    && (crontab -l 2>/dev/null; cat /home/pi/pialert/install/pialert.cron) | crontab - \
    && chgrp -R www-data /home/pi/pialert/db \
    && chmod -R 770 /home/pi/pialert/db \
    # changing the default port number 80 to something random, here 20211
    && sed -ie 's/= 80/= 20211/g' /etc/lighttpd/lighttpd.conf \
    && service lighttpd restart 

# Expose the below port
EXPOSE 20211

# Set up startup script to run two commands, cron and the lighttpd server
ADD start.sh /home/pi
RUN chmod +x /home/pi/start.sh

CMD ["/home/pi/start.sh"]

[ghost]

I’ve forked cjd’s branch and fixed the errors which were present during the building of the Docker image.

There are some other commits which fix minor issues but mainly they modify the webui.

https://github.com/0x09F911029D74E35BD84156C5635688C0/docker.pi-alert/commits/develop

[jokob-sk]

Hi @0x09F911029D74E35BD84156C5635688C0 !

Thanks for that! However, this doesn't run properly either. Getting this error:

Pi.Alert 3.02 (2021-04-24)
---------------------------------------------------------
Scan Devices
    ScanCycle: 1
    Timestamp: 2022-04-25 19:51:00

Scanning...
    arp-scan Method...
    nmap Method...
Traceback (most recent call last):
  File "pialert.py", line 1548, in <module>
    sys.exit(main())
  File "pialert.py", line 86, in main
    res = scan_network()
  File "pialert.py", line 371, in scan_network
    arpscan_devices = execute_nmap (NMAP_OPTIONS, arpscan_devices)
  File "pialert.py", line 523, in execute_nmap
    nmap_output = subprocess.check_output (nmap_args, universal_newlines=True)
  File "/usr/lib/python2.7/subprocess.py", line 216, in check_output
    process = Popen(stdout=PIPE, *popenargs, **kwargs)
  File "/usr/lib/python2.7/subprocess.py", line 394, in __init__
    errread, errwrite)
  File "/usr/lib/python2.7/subprocess.py", line 1047, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

Also, your build seems to have some issues with the crontab file. I'd recommend starting form my fork (https://github.com/jokob-sk/Pi.Alert) and trying to merge the remaining forks.

Feel free to check the image I've built for you: https://registry.hub.docker.com/r/jokobsk/pialert_merge_test

The docker file is the same as in my above comment, only replaced this line to point to your fork:

&& git clone https://github.com/fosspill/Pi.Alert pialert

Thanks in advance for helping!

[ghost]

Apologies, I was working on merging fosspill:main into the branch, which introduced the error.

The current branch is now experimental