- Release: v2.5.5
- Author: Corrado Gerbaldo - IU1BOW
- Mail: [email protected]
- Licensing: Gpl V3.0 see LICENSE file.
- Languages: This application is written in Python 3.12/flask,Javascript and HTML
DXSpider is a great DX Cluster software that has useful telnet interface. I wrote this application in order to add a web user interface to DXSpider and show the spots collected. The user could see 50 spots at time and filter them by band, spotter continent and spotted continent.
For this application, I've used:
- Bootstrap for stylesheet CSS
- Apache ECharts for managing charts
- qrz.com For each callsing, found you can click on lens and you'll see him on qrz.com
- flag-icon-css https://github.com/lipis/flag-icon-css I used it to show the country flags
- ng3k.com ng3k.com I used to get information about "Announced Dx Operations". Thanks to Bill/NG3K !!!
- silso sidc.be/silso used to show propagation trend in "Chart & stats" section
- hamqsl www.hamqsl.com/solar.html used to show solar and band conditions
- kc2g prop.kc2g.com used to show MUF map
You can find my web site at https://www.iu1bow.it
see file "CHANGELOG.md"
1) DXSpider First of all you have to installed DXspider and connected with some other cluster nodes.
2) MariaDB / MySQL
Then you have to install MariaDB on your web server, on the same server where DXSpider is running and configure DXSpider to use it: in your spider folder edit local/DXVars.pm adding these lines:
# the SQL database DBI dsn
$dsn = "dbi:mysql:dxcluster:localhost:3306";
$dbuser = "your-user";
$dbpass = "your-password"; If you would change some MariaDB parameters, then you can find them in /etc/mysql/my.cnf or /etc/my.cnf, depending on your distro.
If the database will not be created automatically, please see "DB_ISSUES.md"
Is it possible that have also to install mariadb libraries. For Ubuntu:
foo@bar:~$ sudo apt-get install -y libmariadb-dev 3) Python modules
You could install python modules using automatic or manual way.
3.1) Automatic modules install
after downloaded spiderweb move in the main folder and install using requirements.txt file
foo@bar:~$ cd spiderweb
foo@bar:~$ pip install -r requirements.txt3.2) Manual modules install
First of all you have to install the python3 pip installer
foo@bar:~$ sudo apt install python3-pipThis application is based on Flask To install Flask:
foo@bar:~$ pip install flask
foo@bar:~$ pip install Flask-minify
foo@bar:~$ pip install flask_wtf
foo@bar:~$ pip install pandasIn the path spiderweb/cfg/ rename config.json.template in config.json:
foo@bar:~$ mv config.json.template config.jsonthen edit it and set the user and password of your database, the menu items, and other stuffs (callsign, mail address...). There is also a specific parameter, named "enable_cq_filter" used to enable the CQ Zone filtering.
Othewhise, if you prefer, you could use a utility to edit your configuration and menu. Go in "script" folder and run ./config.sh
foo@bar:~$ cd scripts
foo@bar:~$ ./config.sh
*** DxSpider configuration ***
Configuration file loaded from: ../cfg/config.json
h: help
vc: view config.
ec: edit config.
vm: view menu
em: edit menu
s: save
t: load config. from template
x: exit
Make your choice:
Starting from version 2.4, since all activities are managed by the application, you don't need to schedule anything.
Now, you can run your web application with the following command:
foo@bar:~$ python3 webapp.pyThe flask default port is 5000, so you can see your web app, typing http://localhost:5000 in your web browser.
Keep in mind that the flask web server, usually is used as a test server.
There are some ways to use it in production.
My configuration is: Cloudflare + Nginx + Bjoern
Bjoern is a lightweight WSGI for python.
for installing it:
foo@bar:~$ sudo apt install libev-dev libevdev2
foo@bar:~$ pip3 install bjoernIf you want you can make it as a daemon service. Create and edit a file named for example spiderweb.service (in the systemd folder)
foo@bar:~$ sudo nano /etc/systemd/system/spiderweb.serviceBelow an example of configuration, keep in mind that it is just an example (you have to put your data for your application here a little guide ):
[Unit]
Description=bjoern instance spiderweb
After=network.target
After=multi-user.target
[Service]
User=web
Group=www-data
Type=simple
WorkingDirectory=/home/web/spiderweb
Environment="PATH=/home/web/spiderweb"
ExecStart=/usr/bin/python3 /home/web/spiderweb/wsgi.py
[Install]
WantedBy=multi-user.targetThen you can install and start the daemon:
foo@bar:~$ sudo systemctl enable spiderweb.service
foo@bar:~$ sudo systemctl start spiderweb.service
foo@bar:~$ sudo systemctl status spiderweb.service
● spiderweb.service - bjoern instance spiderweb
Loaded: loaded (/etc/systemd/system/spiderweb.service; enabled; vendor preset: enabled)
Active: active (running) since Sun 2020-10-25 09:56:35 UTC; 8h ago
Main PID: 6518 (python3)
Tasks: 1 (limit: 420)
CGroup: /system.slice/spiderweb.service
└─6518 /usr/bin/python3 /home/web/spiderweb/wsgi.py
Oct 25 09:56:35 dxcluster01 systemd[1]: Started bjoern instance spiderweb.Now you can install and configure NGINX
Install with:
foo@bar:~$ sudo apt install nginxConfigure:
sudo nano /etc/nginx/sites-available/myappserver {
listen 80;
server_name iu1bow.it www.iu1bow.it;
location ^~ /.well-known/ {
alias /home/web/verify/.well-known/;
}
location / {
ssi off;
include proxy_params;
proxy_pass http://localhost:8080/;
proxy_set_header Host $host;
}
}
For SSL, I'm using Cloudflare. This is a free service that allow you to use https and a proxy cache.
Search engine indexing: when you are on-line, if you would to index your website on search engines, you have to generate a file named sitemap.xml and put it in /static/ folder. There are many tools to generate sitemap.xml, for example www.xml-sitemaps.com
Index on MySQL: if you would to increase speed on callsign search, you could define some index on the table 'spot'. You can see more details on "create_mysql_index.sql"
This application is designed for desktop and mobile phone. It is a PWA, so it could be installed and used like an app on mobile.
Spot list
You can retrieve last spots calling "/spotlist"; For example www.iu1bow.it/spotlist
Country of a callsign
You can retrive some information about a callsign with callsign; for example: www.iu1bow.it/callsign?c=IU1BOW
Directory structure
/ . main application files
├── cfg . configuration files (put here your config.json with your setting)
├── docs . documentation
├── lib . python libs used for the application
├── log . application log
├── scripts . utility scripts for testing, build etc.
├── static . static files css, js, data, html, images etc.
│ ├── css .
│ │ ├── dev . development css not minifyed/uglifyed
│ │ └── rel . release css minifyed/uglifyed (do not change these files)
│ ├── data . application data (world.json)
│ ├── html .
│ │ └── dev . html templates used for build release static html (for offline)
│ │ └── rel . release static html (for offline)
│ ├── images . static images
│ │ └── icons . static icons
│ └── js .
│ ├── dev . development js not minifyed/uglifyed
│ └── rel . release js minifyed/uglifyed (do not change these files)
└── templates . html templates used by python flask for serve the web pages
Application description
The main server application webapp.py is in the root folder. In this application there are routing to html dynamic templates and serves also back-end API. This is wrapped by wsgi.py for using with bjoern server.
Static files (css, js...) are in static directory: here there are subdirectories:
devwhere you can edit and modify sourcesrelhere there are release files created with the building process and used in production
Lint
For lint javascript I use ESLint. You can install with npm init @eslint/config
pylint pip install pylint
Building process
Prerequisites:
| Component | Description | Install command |
|---|---|---|
| npm | a packet manager for javascript | depend on your os. See official page |
| uglify-js | npm component used to minify and uglify javascript | npm install uglify-js -g |
| css-minify | npm component used to minify css | npm install css-minify -g |
| staticjinja | python module used to create static page starting from a html template | pip install staticjinja |
You can build the software for test (dev), or for production (release) environments.
In scripts directory launch:
-
./build.sh -dfor dev environment -
./build.sh -rfor release environment
desktop
mobile
