Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

build and action: add timescaledb #36

Merged
merged 42 commits into from
Sep 24, 2024
Merged

build and action: add timescaledb #36

merged 42 commits into from
Sep 24, 2024

Conversation

gsanchietti
Copy link
Member

@gsanchietti gsanchietti commented Aug 21, 2024 •
edited
Loading

This PR implements the infrastructure for historical monitoring.

Changes:

  • add Timescale database to store report data:
    • the database dump has been added to the module backup (additional testing is needed)
    • the database is accessed in write mode only from the API server
    • the database is access in read only mode from Grafana
  • add MaxMind license for the API server
  • apply retention also to Timescale database
  • update Grafana to 11.2.0 (required to make work the render plugin on a subpath)

Relates issues:

Related PRs:

@gsanchietti gsanchietti force-pushed the report branch 3 times, most recently from 99ae329 to 6e505ae Compare August 21, 2024 14:31
@gsanchietti gsanchietti force-pushed the report branch 8 times, most recently from c7f7afa to 6d0d89d Compare September 11, 2024 16:03
@Tbaile Tbaile mentioned this pull request Sep 12, 2024
Merged
gsanchietti and others added 17 commits September 23, 2024 17:18
@gsanchietti
build and config: add timescaledb
f58e454 
Timescale database is used to store report data
@gsanchietti
systemd: fix webssh dependency
c56eb8c
@gsanchietti
grafana: add new monitoring dashboards
9ccd05a
@gsanchietti
build: bump grafana version
982c5a0 
Required to make it work the image render plugin on a subpath
@gsanchietti
grafana: provision timescale datasource
e9f9479
@gsanchietti
configure: add maxmind_license
75733c0
@gsanchietti
README: add report info
bd11415
@gsanchietti
add vpn dashboard
c1161e1
@Tbaile @gsanchietti
fix: expanding tailscale workers to 100
3c68ab7
@Tbaile @gsanchietti
feat: adding backup of timescale
0977b90
@Tbaile @gsanchietti
fix: solving backup issues
a947942
@Tbaile @gsanchietti
fix: hammering down restore process
ce55c2d
@Tbaile @gsanchietti
fix: dumping only data and restoring using psql
6b4f8f1
@gsanchietti
grafana: add VPN network traffic chart
1cf67d1 
The char has been added to the Metrics dashboard:
it is not possible to add it inside the reporting dashboard
because Prometheus uses the uuid to select the unit,
while Timescale uses an integer id
@Tbaile @gsanchietti
fix: final touchup of database restore
8108257
@Tbaile @gsanchietti
feat: added grafana user to postgresql
b3a7f83
@Tbaile @gsanchietti
feat: added configurable retention period
2c3d44c
gsanchietti and others added 20 commits September 23, 2024 17:18
@gsanchietti
fix: api, set GIN_MODE
35379fc
@gsanchietti
grafana: rename malware dashboard to security
6d24dd5
@gsanchietti
grafana: improve traffic_by_client
cee4ad2
@gsanchietti
grafana: traffic dashboard, add host link
e3147aa
@Tbaile @gsanchietti
feat(ui): added maxmind token field and tooltip for metrics
aaa632b
@gsanchietti
grafana: add connectivity dashboard
4e0260d
@gsanchietti
grafana: update dashboard queries
15ba2ac 
Use uuid instead unit_id to identify the units.
@gsanchietti
grafana: fix VPN in unit dashboard
49523a5
@gsanchietti
grafana: vpn, add chart from prometheus
89e8404
@gsanchietti
grafana: add promethus chart to connectivity
a4b6c29
@gsanchietti
grafana: connectivity, add total traffic chart
cd04ba5
@Tbaile @gsanchietti
fix(grafana): using best effort method to format dates in dashboards
bad7c4f
@Tbaile @gsanchietti
feat(grafana): added time range links for network traffic
3db9d51
@gsanchietti
grafana: add links to all dashboards
c57b874
@gsanchietti
grafana: fix network_traffic
44fe6ab
@gsanchietti
grafana: dashboard multiple fixes
279a276
@gsanchietti
grafana: make dashboard read-only
e6d42d9
@gsanchietti
grafana: connectivity, add wan staus
9923951
@gsanchietti
grafana: dashboards minor fixes
488f45d
@gsanchietti
build: update controller to 1.1.0
fa4ad12
@gsanchietti gsanchietti marked this pull request as ready for review September 23, 2024 15:21
@gsanchietti gsanchietti requested a review from Tbaile September 23, 2024 15:21
Tbaile
Tbaile requested changes Sep 23, 2024
View reviewed changes
build-images.sh Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
@gsanchietti gsanchietti requested a review from Tbaile September 23, 2024 15:42
@gsanchietti gsanchietti merged commit b943e9a into main Sep 24, 2024
1 check passed
@gsanchietti gsanchietti deleted the report branch September 24, 2024 08:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Tbaile Tbaile Tbaile approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@gsanchietti @Tbaile