-
Notifications
You must be signed in to change notification settings - Fork 9
/
CHANGELOG
1276 lines (1106 loc) · 61.1 KB
/
CHANGELOG
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
# Changelog
All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [Unreleased]
## [2.15.5] - 2024-06-28
### Added
- [API_PARSER] [SIGNALSCIENCES_NGWAF] Add site_name key in logs
- [API_PARSER] [GATEWATCHER_ALERTS] New collector
- [API_PARSER] [CISCO_UMBRELLA] New collector
### Fixed
- [API_PARSER] [VECTRA] Correctly allow to test the Collector before saving it
- [API_PARSER] [HARFANGLAB] Increase delay to 10 minutes to be sure to get all logs
- [AUTHENTICATION] [GUI] LDAP Users can now authenticate once activated
## [2.15.4] - 2024-05-27
### Changed
- [API_PARSER] [CYBEREASON] Update malops fetching
- [DEPENDENCIES] Python dependencies upgrade
- gunicorn 21.2.0 -> 22.0.0
- [CLUSTER] [CONFIG] Reload all Rsyslog/Filebeat Listeners when updating the Redis cluster password
### Fixed
- [FRONTEND] Missing redis fields in template infos
- [API_PARSER] [HARFANGLAB] Increase the limit of logs in request and reduce time range of requests
- [API_PARSER] [CSC_DOMAINMANAGER] Split logs with several records
- [API_PARSER] Correctly select custom parser certificate when set
- [REPUTATION_CTX] Ensure there are no duplicates left for GeoLite files in Mongodb
- [API_PARSER] [HARFANGLAB] Add threshold check to avoid infinite loop while getting more than >=10k logs on single execution
- [API_PARSER] [TRENDMICRO_VISIONONE] Avoid sporadic errors on collector's GUI test
## [2.15.3] - 2024-04-24
### Added
- [API_PARSER] [RAPID7_IDR] Add priority field in the logs from rapid7_idr api
- [API_PARSER] [APEX] Use the page_token parameter to prevent duplication of logs
### Fixed
- [WORKFLOW] Silent error when saving a new worflow
- [FRONTEND] [API] Correctly return Frontend information when Reputation Contexts are set
- [API_PARSER] [RETARUS] Fix bad json formatting leading to escaped string sent over rsyslog
## [2.15.2] - 2024-04-02
### Changed
- [CRON] Only update Reputation Context DB on Wednesdays and Saturdays at 2am
- [CRON] Only update security audits every night at 11pm
## [2.15.1] - 2024-03-22
### Fixed
- [FRONTEND] TCP session were not received properly when LOG Frontend was set to TCP+UDP
## [2.15.0] - 2024-03-04
### Added
- [FRONTEND] Redis stream support for input log
- [FRONTEND] Allow to simply use local Redis without worrying about parameters when creating a Redis Listener
- [REDIS] Password support for local redis
- [REDIS] New api function set_password and set_replica_of
### Changed
- [CLUSTER] Evolve daemon's code to allow correctly handling tasks during service shutdowns
- [API] Increase API key length for new clusters from 16 to 32 characters
- [INSTALL] Improve and stabilize the Redis bootstrapping process
- [CONFIG] [API] Remove the object ID from the API: there is only one configuration object!
- [API_PARSER] [NOZOMI] Add missing 'Content-Type' and 'Accept' headers in requests
### Fixed
- [FRONTEND] Allow skipping unresponsive nodes during HAProxy test_conf
- [REPUTATION] Changed predator references to barricade
- [REPUTATION_CTX] Ensure internally defined DBs cannot be modified
- [SYSTEM] [PKI] Fix wrong ca.key after cluster_join
- [API_PARSER] Connect automatically to redis matser
- [API_PARSER] [TRENDMICRO_VISIONONE] Correctly handle errors in log fetches
- [API_PARSER] [TRENDMICRO_VISIONONE] Avoid 429 errors by increasing default pagination
- [API_PARSER] [TRENDMICRO_VISIONONE] Delay log fetching by 5 minutes to be sure to get all logs
- [API_PARSER] [NOZOMI] Set a custom User-Agent to prevent request blacklisting from technology
- [HAPROXY] [LISTENERS] Put back a correct 60s server timeout for Rsyslog/Filebeat communication
- [HAPROXY] [CONFIGURATION] Prevent configuration failures when using packaged lua scripts
## [2.14.3] - 2024-02-20
### Fixed
- [REPUTATION] Changed predator references to barricade
## [2.14.2] - 2024-02-19
### Changed
- [API_PARSER] [VECTRA] Store current access token to avoid re-generating it every time
- [DEPENDENCIES] Python dependencies upgrade
- aiohttp 3.9.1 -> 3.9.3
- attrs 23.1.0 -> 23.2.0
- beautifulsoup4 4.12.2 -> 4.12.3
- boto3 1.33.2 -> 1.34.32
- botocore 1.33.2 -> 1.34.32
- bs4 0.0.1 -> removed
- cryptography 41.0.7 -> 42.0.2
- django 4.2.7 -> 4.2.9
- djongo 1.3.6_2 -> 1.3.6.post3
- frozenlist 1.4.0 -> 1.4.1
- google-api-core 2.14.0 -> 2.16.1
- google-api-python-client 2.108.0 -> 2.116.0
- google-auth 2.23.4 -> 2.27.0
- google-auth-httplib2 0.1.1 -> 0.2.0
- googleapis-common-protos 1.61.0 -> 1.62.0
- jinja2 3.1.2 -> 3.1.3
- markupsafe 2.1.3 -> 2.1.4
- maxminddb 2.5.1 -> 2.5.2
- meraki 1.39.0 -> 1.42.0
- netaddr 0.9.0 -> 0.10.1
- pillow 10.1.0 -> 10.2.0
- protobuf 4.25.1 -> 4.25.2
- pyopenssl 23.3.0 -> 24.0.0
- s3transfer 0.8.1 -> 0.10.0
- typing-extensions 4.8.0 -> 4.9.0
- yarl 1.9.3 -> 1.9.4
- setuptools 69.0.2 -> 69.0.3
### Fixed
- [API_PARSER] [CSC_DOMAINMANAGER] Correct missing import...
- [API_PARSER] [RETARUS] Fix update lock to prevent multiprocess
- [API_PARSER] [SENTINEL_ONE_MOBILE] Fix try..except raises
- [API_PARSER] [FORCEPOINT] Fix update lock to prevent multiprocess
- [API_PARSER] [FORCEPOINT] Use a safer alternative to XML decoding
- [WORKFLOW] Trigger build_conf on various changes
- [GUI] [NODES] Wrong state of mongo replica when a node is down
- [GUI] [NODES] Correctly print result when stepping down a Mongodb node
- [LOG_FORWARDER] Reload related frontends of type "file", "redis" or "kafka" without any node selected
- [CLUSTER_CREATE] Correctly reset Admin user during cluster creation
- [HAPROXY] Wrong backend timeout caused by the new intermediate backend
- [WORKFLOW] Correctly update Haproxy's old Frontend configuration when a different one is choosen on a Workflow
## [2.14.0] - 2024-01-10
### Added
- [API_PARSER] [SIGNALSCIENCES_NGWAF] New collector
- [HAPROXY] [CONF] JWT verification through workflow
- [IDP] [LDAP] Allow to map custom IDP fields to LDAP attributes
### Changed
- [DEPENDENCIES] Python dependencies upgrade
- aiohttp 3.8.6 -> 3.9.1
- boto3 1.28.68 -> 1.33.2
- botocore 1.31.68 -> 1.33.2
- cachetools 5.3.1 -> 5.3.2
- certifi 2023.7.22 -> 2023.11.17
- charset-normalizer 3.3.1 -> 3.3.2
- cryptography 41.0.4 -> 41.0.7
- django 4.2.6 -> 4.2.7
- google-api-core 2.12.0 -> 2.14.0
- google-api-python-client 2.104.0 -> 2.108.0
- google-auth 2.23.3 -> 2.23.4
- idna 3.4 -> 3.6
- maxminddb 2.4.0 -> 2.5.1
- meraki 1.38.0 -> 1.39.0
- protobuf 4.24.4 -> 4.25.1
- pyasn1 0.5.0 -> 0.5.1
- pyopenssl 23.2.0 -> 23.3.0
- python-ldap 3.4.3 -> 3.4.4
- s3transfer 0.7.0 -> 0.8.1
- urllib3 1.26.18 -> 1.26.18
- werkzeug 3.0.0 -> 3.0.1
- yarl 1.9.2 -> 1.9.3
- setuptools added -> 69.0.2
- [FRONTEND] Deploy a listener on every node when listening_node is empty
### Fixed
- [API_PARSER] [FORCEPOINT] Avoid duplicate processes when input takes long to process
- [CSC_DOMAINMANAGER] Fix missing json.dumps on logs list
- [PORTAL] [SELF_SERVICE] Correct wrong password change/logout links in self-service page
- [GUI] [WORKFLOW] Correctly print items' list when editing a Workflow
- [GUI] [FRONTEND] Correctly select a good ruleset for API collector but let user override the value
- [CONFIG] [SYSTEM] Correct issue preventing some certificates to be deleted from system
- [OPENID] [MODELS] Get correct URLs for OpenID IDPs
- [LDAP] Correctly get boolean values from LDAP replies
- [REPUTATION_CONTEXT] Add missing fixture for default GeoLite MMDBs download
## [2.13.0] - 2023-11-29
### Added
- [FRONTEND] New last_update_time field
- [GUI] [WORKFLOW] CORS policy settings
- [HAPROXY] [CONF] CORS policy into config generation
- [API_PARSER] [HARFANGLAB] Fetch new endpoint for Threats logs
- [API_PARSER] [APEX] New parser
- [FRONTEND] [TCP] Add advanced option custom_tl_frame_delimiter for Rsyslog tcp frontends
### Removed
- [FRONTEND] [TIMEOUTS] Incoherent "timeout connect" in Frontend attributes
### Changed
- [HAPROXY] [CONF] Split Frontend and Workflow config to improve performance
- [HAPROXY] [CONF] Change test config names by uuids
- [PERFORMANCES] Avoid high overhead when loading Workflows by simplifying string representation
### Fixed
- [GUI] [FRONTEND] Bug allowing the creation of an http or tcp frontend without a listener
- [LOG_FORWARDER] Rsyslog config generation error when '-' found in LOGOM name
- [LOG_FORWARDER] Frontends configs aren't refreshed when LOGOM used as parse failure
- [WORKFLOW] Re-allow multiple workflows on the same FQDN with sub-paths
- [HAPROXY] Ensure X-Forwarded-proto header is correctly set on Frontend configuration
- [API_PARSER] [NOZOMI] Set correct timestamp format for query limits
## [2.12.1] - 2023-11-16
### Fixed
- [API] [BACKEND] Broken API URL
- [API] [WORKFLOW] Do not error on Workflow creation through API
- [SYSTEM] [USERS] Change password of an existing user
## [2.12.0] - 2023-11-15
### Added
- [API_PARSER] [VECTRA] New parser Vectra
- [MANAGE] [COMMANDS] New command get_api_key
- [OPENID] [CONNECTOR] New JWT validation parameters
- [OAUTH2] [USERINFO] Allow to validate JWT in /userinfo with new OpenID connector's parameters
### Changed
- [API_PARSER] [MS_SENTINEL] timeGenerated -> processingEndTime and incident.id -> incident.name
- [DEPENDENCIES] Python dependencies upgrade
- aiohttp 3.8.5 -> 3.8.6
- boto3 1.28.52 -> 1.28.68
- botocore 1.31.52 -> 1.31.68
- cffi 1.15.1 -> 1.16.0
- charset-normalizer 3.2.0 -> 3.3.1
- django 3.2.22 -> 4.2.6
- djongo pypi -> VultureProject
- google-api-core 2.11.1 -> 2.12.0
- google-api-python-client 2.100.0 -> 2.104.0
- google-auth 2.23.0 -> 2.23.3
- googleapis-common-protos 1.60.0 -> 1.61.0
- meraki 1.37.2 -> 1.38.0
- packaging 23.1 -> 23.2
- pillow 10.0.1 -> 10.1.0
- protobuf 4.24.3 -> 4.24.4
- pytz deleted
- s3transfer 0.6.2 -> 0.7.0
- sqlparse 0.2.4 -> 0.4.4
- urllib3 1.26.17 -> 1.26.18
- werkzeug 2.3.7 -> 3.0.0
- [API] [PF] Changed old PF whitelist/blacklist from config to new API entrypoint 'api/v1/system/pf/'
- [INTERNAL] Adapt model relationships to Django 4.2
### Fixed
- [PORTAL] [IDP] Refresh token validation with multiple IDP
- [FRONTEND] Disable log condition if log forwarder disabled
- [REPUTATION_CTX] Invalid headers templating
- [PORTAL] [SSO] Avoid autologon password retrieval failures
- [API_PARSER] [TRENDMICRO_VISIONONE] Fix Alerts endpoint params
- [API] [CLUSTER_CONFIG] Add missing allowed fields for API PATCHing
- [SETTINGS] Update CSRF_TRUSTED_ORIGINS format for Django 4
- [URLS] Simplification of different paths
- [DEPRECATION] Change request.META to request.headers
- [DEPRECATION] get_random_string now requires length argument
- [DEPRECATION] Keyword "trans" replaced by "translate" in templates
- [DEPRECATION] make_aware function changed to django aware datetime
- [API] [OPENID] Correctly handle form errors during API modifications
## [2.11.1] - 2023-10-13
### Added
- [GUI] [FRONTEND] New field for custom proxy to use with an api parser
- [LOGOM] Add queue control options
- Max Workers
- New Worker start condition
- Worker shutdown inactivity condition
- Queue shutdown timeout
### Changed
- [DEPENDENCIES] Python dependencies upgrade
- aiohttp 3.8.4 -> 3.8.5
- async-timeout 4.0.2 -> 4.0.3
- boto3 1.26.147 -> 1.28.52
- botocore 1.29.147 -> 1.31.52
- captcha 0.4 -> 0.5.0
- certifi 2023.5.7 -> 2023.7.22
- charset-normalizer 3.1.0 -> 3.2.0
- configparser 5.3.0 -> 6.0.0
- cryptography 41.0.2 -> 41.0.4
- django 3.2.20 -> 3.2.22
- decorator 5.1.1 -> removed
- frozenlist 1.3.3 -> 1.4.0
- google-api-core 2.11.0 -> 2.11.1
- google-api-python-client 2.88.0 -> 2.100.0
- google-auth 2.19.1 -> 2.23.0
- google-auth-httplib2 0.1.0 -> 0.1.1
- googleapis-common-protos 1.59.0 -> 1.60.0
- gunicorn 20.1.0 -> 21.2.0
- maxminddb 2.3.0 -> 2.4.0
- meraki 1.33.0 -> 1.37.2
- mock 5.0.2 -> 5.1.0
- netaddr 0.8.0 -> 0.9.0
- pillow 9.5.0 -> 10.0.1
- protobuf 4.23.2 -> 4.24.3
- pyjwt 2.7.0 -> 2.8.0
- pyotp 2.8.0 -> 2.9.0
- pyparsing 3.0.9 -> 3.1.1
- pytz 2023.3 -> 2023.3.post1
- redis 4.5.5 -> 4.6.0
- s3transfer 0.6.1 -> 0.6.2
- soupsieve 2.4.1 -> 2.5
- typing-extensions 4.6.3 -> 4.8.0
- urllib3 1.26.16 -> 1.26.17
- validators 0.20.0 -> 0.22.0
- werkzeug 2.3.4 -> 2.3.7
- setuptools 67.8.0 -> removed
- [TLS] Update default parameters for accepted TLS versions
- [LOGOM] Put generic advanced parameters in an Advanced drop-down section
### Fixed
- [API_PARSER] [CSC_DOMAINMANAGER] Correctly handle log stagnation and query interval
- [API_PARSER] [VADESECURE] Ensure correct user ID is used depending on API
- [PKG_UPGRADE] Avoid trying to launch upgrade scripts when node is not bootstrapped
- [GUI] [ACLS] Improve 'Used by' presentation of Wokrflows using ACLs
- [GUI] [FRONTEND] Prevent mode modification if Frontend is linked to a Workflow/IDP
- [GUI] [BACKEND] Prevent mode modification if Backend is linked to a Workflow
- [CODE] [CLEAN] Minor corrections
- [SSO_CLIENT] [COOKIES] Fix cookies' banning when receiving application's response
- [SSO_CLIENT] [COOKIES] Correctly check the 'HttpOnly' attribute set from application's response
- [IDP] [COOKIES] Ensure 'HttpOnly' attribute is always set on IDP session cookies
- [UPGRADE] [SCRIPTS] Avoid updating Frontend's Redis internal IP when a custom port is used
- [CSS] Incorrect color conditions for custom form errors
- [API_PARSER] [CISCO_DUO] Unwanted error log on empty response
- [LOG_FORWARDER] [KAFKA] Ensure kafka custom configurations are correctly handled and validated
- [LOG_FORWARDER] [REDIS] Non-working LogForwarder when no password was set
- [LOG_FORWARDER] [FILE] Wrong configuration of the spool directory when disk queuing is enabled
- [BACKEND] Avoid salt errors on 'tcp_health_check_expect_match' field
## [2.10.2] - 2023-09-25
### Fixed
- [API_PARSER] [CROWDSTRIKE] Delay query time of 2 minutes - to avoid missing events
- [FRONTEND] [FILEBEAT] Missing Rsyslog garbage templates in generated configurations
- [NODE] [STATE] Avoid errors on initial heartbeat monitoring
### Changed
- [FRONTEND] Set new default redis server to '127.0.0.5'
- [FRONTEND] [GUI] Show redis parameters when editing a Filebeat Listener to be able to select a different Redis cache
- [FRONTEND] [CONFIGURATION] Allow configured redis_password to be used for Filebeat Listener's cache
## [2.10.1] - 2023-09-12
### Fixed
- [LDAP_REPO] Avoid formatting errors for fields inserted to LDAP when creating/updating users
- [LDAP_REPO] Correct missing default user/group objectclasses configuration on existing repositories
- [RSYSLOG] Correct wrong rsyslog input configuration generation
- [NODE] [STATUS] Don't update Node state when put into MAINTENANCE
- [API_PARSER] Avoid problems while getting parameters by defining custom values
- [API_PARSER] [STATUS] Correctly set and update each node status for API parsers
- [API_PARSER] [SENTINEL_ONE] Fix account_type="user service" authentication routine
## [2.10.0] - 2023-08-22
### Removed
- [PREDATOR] Remove predator references
### Added
- [GUI] [FRONTEND] Selector of a custom certificate to use with an api parser
- [LOG_FORWARDER] [HIREDIS] Ability to specify that the destination key is a dynamic template
- [LOG_FORWARDER] New send_as_raw feature for LogForwarders in gui
- [API_PARSER] New collector **Retarus**
- [BACKEND] [GUI] HTTP/2 Health check
- [BACKEND] [GUI] TCP Health check
- [CLUSTER] Node state and heartbeat
- [GUI] API collector node runner selector
- [LDAP] Allow to define custom ObjectClasses for Users and Groups when creating them through Vulture
- [LOG_FORWARDER] [KAFKA] New Kafka log forwarder
### Changed
- [LOG_FORWARDER] [HIREDIS] Use updated templates <name>_json
- [FILEBEAT] [CONFIGURATION] Use internal Redis proxyfication with Haproxy instead of direct connections
### Fixed
- [FRONTEND] [CLONE] Resolve GUI loading problems when cloning an existing Frontend
- [FRONTEND] [FILEBEAT] Avoid overriding default module config for all Frontends when changed in one
- [FRONTEND] Disable logforwarder if logging disabled
- [FRONTEND] Reload services only on involved nodes
- [FRONTEND] Remove old conf when changing frontend mode to avoid duplicate ruleset
- [MANAGE] Allow force toggle into maintenance
- [CLUSTER_JOIN] Correctly reload Haproxy internal configurations on every node
- [SYSTEM] [API] Add missing trailing slash to the /api/system/pki/get_ca/ endpoint
- [RSYSLOG] [CONFIGURATION] Improve inputs' configuration generation on clustered nodes
- [FRONTEND] [CONFIGURATION] Improve generation/deletion of obsolete configuration files on clustered nodes
## [2.9.1] - 2023-08-16
### Fixed
- [API_PARSER] [SENTINEL_ONE_MOBILE] Fix query params and API uri
- [FRONTEND] timeouts on save and wrong endpoints
## [2.9.0] - 2023-07-20
### Removed
- [LOG_FORWARDER] [ELASTICSEARCH] RateLimit settings
### Added
- [API_PARSER] New API parser SentinelOne Mobile
- [API_PARSER] New API parser CSC DomainManager
- [LOG_FORWARDER] Disk-assisted queues for reliability
- [PORTAL] [IDP] Refresh token support through OAuth2 flow
### Changed
- [VOCABULARY] Replace 'slave' word uses with 'replica'
- [NETWORK] Improve the way Node IPs are updated on disk
- [NETIF] [API] Only return from API call when refresh has been finished
- [DEPENDENCIES] Python dependencies upgrade
- django 3.2.19 -> 3.2.20
- cryptography 41.0.1 -> 41.0.2
### Fixed
- [PORTAL] [OPENID] Improve error returns and code quality
- [SYSTEM] [CERTIFICATES] Protect used certificates when attempted to be deleted
- [SYSTEM] [CERTIFICATES] Reload HAProxy on certificate change
- [ACLS] Rebuild HAProxy conf on ACL change
- [LOGFORWARDER] [ELASTICSEARCH] Enable ELS>=8 compatibility
- [LOGFORWARDER] [ELASTICSEARCH] Ensure index patterns are kept to lowercase
- [LOGFORWARDER] [ELASTICSEARCH] Avoid newline problems in generated configurations
- [CLUSTER_JOIN] New node couldn't resolve all other nodes' name and IP
- [CLUSTER_JOIN] New node missed an initial Rsyslog configuration file
- [NODE] [CONFIGURATION] Correctly update rc.conf when editing a Node on GUI/API
- [REFRESH_NIC] Correctly get DHCP-enabled interfaces' IP
- [API_PARSER] [SENTINEL_ONE] Fix authorization token type for service user accounts
- [API_PARSER] [CYBEREASON] FIx timestamp for `malware` logs
- [API_PARSER] [IMPERVA] Fix duplicates files downloading
- [API_PARSER] [IMPERVA] Fix crash if file does not exist
- [API_PARSER] [SAFENET] Increment last_api_call if no logs
- [API_PARSER] [CISCO_DUO] Fix update last_api_call if next_offset key doesn't exist
- [PORTAL] [SELF_SERVICE] Correctly list available applications for users connected through local repositories
- [PORTAL] [SSO] Avoid SSO errors when using local repositories
- [CLUSTER_CREATE] Some Node IPs were missing from Node's DB and on-disk configurations after cluster creation
## [2.8.2] - 2023-07-05
### Fixed
- [SYSTEM] [MIGRATIONS] Resolve manual migrtions problems with existing CARP interfaces
## [2.8.1] - 2023-07-05
### Added
- [NETWORK] Allow possibility to create LAGG interfaces
- [NET2MONGO] Can now import system configurations for interfaces, aliases, VLANs and LAGG interfaces
### Changed
- [NETWORK] [GUI] VLAN configuration rework: ability to create/configure VLAN virtual interfaces directly from GUI
- [NETWORK] Internal rework of network interfaces parameters
- [SYSTEM] Improved capabilities to modify system/network configurations
- [API] [NETWORK] Allow getting a list of NetworkAddress objects filtering by name, ip, type and NICs
### Fixed
- [PF] [CONFIG] Configuration generation could fail with some network interfaces configured
## [2.8.0] - 2023-06-28
### Added
- [API_PARSER] [CYBEREASON] Add support for 'malware' URI
- [FRONTEND] [GUI] Allow to configure the maximum number of workers for LOG Listeners
- [FRONTEND] [GUI] Allow to configure/activate new mmdblookup cache size parameter for LOG Listeners
- [FRONTEND] [GUI] Allow to configure redis LOG input dequeue batch size
### Removed
- [API_PARSER] [ELASTICSEARCH] Last code traces related to the parser
### Changed
- [PORTAL] [IDP] Define a new custom cookie name for IDP portal sessions
- [SETTINGS] [LOGGING] Add correct logging for some code shared between GUI and Portal
- [GUI] [HAPROXY_TEST_CONF] Remove nbproc (deprecation) and use nbthread instead
- [FRONTEND] [GUI] Put most of advanced LOG parameters in the 'Advanced' drop-down
### Fixed
- [PORTAL] [VIEWS] Correctly print error messages on self-service page
- [PORTAL] [SELF_SERVICE] Repair and improve self-service page
- [PORTAL] [AUTH_FILTERS] Authentication filters could not check conditions properly
- [AUTH] [LDAP] Encode binary fields in hexadecimal before handling them as claims
- [PORTAL] [HAPROXY_CONF] Correctly check CORS condition for IDPs
- [NET2MONGO] Fix vlan interfaces detection
- [HAPROXY] [CONF] IDPs wouldn't work on Frontends without any Workflow assigned
- [API_PARSER] [SENTINELONE] Avoid query date stagnation
- [NETWORK] [GUI] Don't change network interface type when saving it
- [LOGFORWARDER] [ELASTICSEARCH] remove deprecated option for ELS>=7 compatibility
## [2.7.1] - 2023-06-07
### Fixed
- [FRONTEND] resolve a bug that prevented users to modify a Listener existing before 2.7.0
- [API_PARSER] Remove useless default values for Cisco DUO fields
## [2.7.0] - 2023-05-24
### Added
- [API_PARSER] New API parser CISCO DUO
### Changed
- [DEPENDENCIES] Remove constraint on cryptography lib, upgrade cryptography to v40.0.2 and pyopenssl to v23.1.1
### Fixed
- [LOGGING] [PORTAL] Don't log exceptions when a user connects to a Portal for the first time!
- [PKI] [FORM] Use a safer SSL context for fields' validation
- [API_PARSER] [GSUITE_ALERTCENTER] prevent last_api_call stagnation
- [API_PARSER] [IMPERVA] Don't log exceptions on 404 errors
- [API] Correctly search objects by name for
- LDAP repositories
- OTP Repositories
- Portals
- ACLs
## [2.6.1] - 2023-05-22
### Fixed
- [FRONTEND] [MODELS] Wrong fields' type when trying to use API parsers due to inappropriate conversions
## [2.6.0] - 2023-05-05
### Removed
- [PYTHON] [IMPORTS] useless 'yaml' import in frontend's models.py
- [PYTHON] [REQUIREMENTS] replace python-oauth2's uuid generation with built-in uuid lib
- [PYTHON] [REQUIREMENTS] Completely remove sqlalchemy from requirements and code
### Added
- [REQUIREMENTS] New files requirements.in (application requirements) and requirements.txt (fixed version requirements)
### Fixed
- [API] [FRONTEND] Correctly search Frontend by name when it contains spaces
- [FRONTEND] Correctly serialize object's data (for APIs and GUI)
- [NETWORK] Avoid rc.conf parsing errors on interface entries when values are enclosed in single quotes
- [API_PARSER] [WAF_CLOUD_PROTECTOR] Gargantually increase performance by decompressing in memory
- [API_PARSER] [TRENDMICRO_VISIONONE] Fix logs formatting
- [API_PARSER] [REACHFIVE] Fix logger
- [API_PARSER] [PROOFPOINT_TAP] Fix logger
### Changed
- [SETTINGS] Adapt pymongo database settings to v3.13
- [API_PARSER] [HARFANGLAB] Add a 2-minute delay on techno fetches
- [API_PARSER] [SENTINEL_ONE] Add a 5-minute delay on techno fetches
- [GUI] [WORKFLOW] Allow to search/filter Workflows on their name OR FQDN
## [2.5.3] - 2023-04-03
### Fixed
- [API_PARSER] [TRENDMICRO_VISIONONE] fix format of logs and update of timestamp
- [API_PARSER] [CYBEREASON] Delay request at the API by 5 minutes
## [2.5.2] - 2023-03-24
### Fixed
- [PORTAL] Include URL's query parameters during portal redirections
## [2.5.1] - 2023-03-17
### Fixed
- [FRONTEND] [FORM] Add missing attribute 'trendmicro_visionone_token' for Frontend edition
## [2.5.0] - 2023-03-16
### Added
- [API_PARSER] New parser Trendmicro Visionone
- [API_PARSER] [MS_SENTINEL] Add 'entities' to the logs
### Fixed
- [API_PARSER] [NETSKOPE] Fix infinite loop when there is no log
- [SYSTEM] [RSYSLOG] Only restart Rsyslog when everything has been (re)built
## [2.4.2] - 2023-03-03
### Fixed
- [PORTAL] Allow tokens obtained from an application portal to pass the portal when token is supplied through headers
## [2.4.1] - 2023-03-01
### Added
- [API_PARSER] [LOGGING] Add Process PID to parsers' logs
### Fixed
- [API_PARSER] Regularly scan processes to join to avoid accumulating zombies
- [CONFIG] [PF] Update PF configurations when whitelist/blacklist entries are changed on GUI
- [GUI/API] [NETWORK] API configuration didn't work as expected
- [GUI/API] [NETWORK] Small corrections to internal functions and data types
- [CONFIG] [FILEBEAT] Correctly remove filebeat configuration files when removing a filebeat Frontend
- [RSYSLOG] Only restart service once after modifying configuration files
### Changed
- [LOGGING] [PF] Don't send PF logs to mongodb anymore, log to /var/log/pf/parsed.log instead
- [UPGRADE] [PKG] Better filtering of update scripts' folders during upgrades
- [UPGRADE] [SCRIPTS] Only launch some post-upgrade scripts when node was bootstrapped and is able to receive them
- [PORTAL] Return a 401 instead of a 403 when refusing a header-authenticated request
- [NETIF] [API] Await cluster results when refreshing interfaces list before returning from the API call
## [2.4.0] - 2023-02-16
### Removed
- [FILES] Obsolete files (owned and used on vulture-base)
- [HAPROXY] [CONFIG] Obsolete Frontend defender policy SPOE
- [APACHE] Obsolete remaining functions relative to Apache
### Added
- [HAPROXY] [CONFIG] Added Redis main node proxying
- [API_PARSER] [CYBEREASON] New flattened fields (ips, hostnames, domains...)
### Fixed
- [API_PARSER] [CLOUDPROTECTOR] Prevent concurent executions of parser on system
### Changed
- [IDP] [API] Return a generic 400 when the requested portal/repo/user doesn't exist
- [HAPROXY] [TESTS] Improve validation tests by using a mock global file
- [PORTAL] [SESSION_VALIDATION] Complete rework of session validation and simplification of underlying code
- [PORTAL] [CONFIG] Improve URL rewriting logic on authenticated workflows when user is not connected
- [SYSTEM] [RC] rework/simplify rc scripts to get/set/remove values from rc.conf
- [DJANGO] [API] Override error responses to ensure API endpoints ('/api/*') return 'application/json' content
- [SYSTEM] [NETWORK] Set/get IP/routing system configurations from /etc/rc.conf instead of /etc/rc.conf.d/network
- [API_PARSER] [CYBEREASON] Rework of parser for new API version
## [2.3.0] - 2023-02-02
### Fixed
- [HAPROXY] [PORTAL] redirect request to portal whenever authentication validation couldn't be done
- [IDP] [API] Correct return code 400 when User without TOTP has a request for TOTP reset
- [SYSTEM] [BLACKLIST/WHITELIST] Reload pf configuration when adding/removing whitelist/blacklist IPs to cluster
- [API_PARSER] [CLOUDFLARE] Fix ram explosion + improve resilience by downloading by bulks
- [API_PARSER] [AKAMAI] Fix HBSD13 SemLock error when loading GUI code
### Added
- [IDP] [API] When locking/unlocking a user, also disable/enable its access tokens
- [API_PARSER] [WAF_CLOUD_PTORECTOR] New API parser WAF Cloud Protector
### Changed
- [HAPROXY] [PORTAL] Deny requests with Authorization header, instead of redirecting them to the portal
## [2.2.4] - 2023-01-18
### Fixed
- [API_PARSER] [IMPERVA] Prevent downloading same files
- [CRONTABS] [SECURITY_UPDATE] Script could fail to save new Geolite MMDBs due to filename inconsistency in DB
### Changed
- [RSYSLOG] [PSTATS-SEC] Disable rsyslog stat generation every second
- [API_PARSER] [AKAMAI] Improve parser's performances
- [API_PARSER] [PROOFPOINT_TRAP] Add treatment delay of 15 minutes to have the alerts after their update
- [API_PARSER] [PROOFPOINT_TRAP] Get more incident information, including incident severity
## [2.2.3] - 2023-01-04
### Fixed
- [API_PARSER] [NETSKOPE] last_api_call was not correctly updated + prevent stagnation
### Removed
- [API_PARSER] [PROOFPOINT_TRAP] Removed filtering on 'open' alerts
## [2.2.2] - 2022-12-21
### Fixed
- [TLS] When a TLS profile is updated, all Listeners and Applications using it are also reloaded and underlying configurations are reloaded
- [FRONTEND] [GUI] When selecting plain text for a frontend, it is correctly handled as http
- [FRONTEND] [CONFIGURATION] Correctly list related Nodes of a modified Frontend (some types were not handled correctly)
- [TENANTS] [GUI] Prevent whitespaces in tenants' name
- [API_PARSER] [BLACKBERRY_CYLANCE] Prevent last_api_call from getting stuck more thant 24h in the past
- [API_PARSER] [THREADS] Don't handle signals when code is not run in main thread (GUI w/ gunicorn case)
- [API_PARSER] [NETSKOPE] Correctly handle non-2xx status codes returned by API
- [REPUTATION_CTX] [GUI] Correctly show POST body parameter when selecting POST method
### Added
- [API_PARSER] [PROOFPOINT_TRAP] New API parser Proofpoint TRAP
### Changed
- [GUI] [TLS] It's now impossible to delete a TLS profile used in a frontend or a backend (will list where it is used)
- [GUI] [SETTINGS] Improved logging configuration for GUI
- [PORTAL] [SETTINGS] Improved logging configuration for portal
## [2.2.1] - 2022-12-13
### Fixed
- [GUNICORN] [CONF] Timeout increased to 60 seconds
- [APACHE] [GUNICORN] Set 4 workers w/ 4 threads
- [API_PARSER][DEFENDER] GUI 'test' button where always returning "Success" even if credentials where wrong
### Changed
- [PORTAL] [GUNICORN] Set 4 workers w/ 4 threads
- [NGINX] [CONF] Proxy timeout increased to 5 seconds
## [2.2.0] - 2022-12-09
### Fixed
- [API_PARSER] [CROWDSTRIKE] last_api_call was not properly updated
- [SYSTEM] [CERTIFICATES] Ensure correct rights and ownerships to imported node's certificate files
- [CLUSTER] [CLUSTER_JOIN] Numerous improvements to cluster_join script, for both new node and called master
- [SYSTEM] [NET2MONGO] Allow to get network configuration from /etc/rc.conf AND /etc/rc.conf.d/network, and accept more ifconfig formats
### Added
- [OAUTH] Add 'Access-Control-Allow-Origin' header to oauth2 endpoints for API requests through SPAs
- [GUI] [SYSTEM] Add nginx in the apache jail to work as proxy and serve static files for gunicorn
- [API_PARSER] [PROOFPOINT_CASB] New api parser Proofpoint CASB
### Changed
- [GUI] [SYSTEM] Replace apache's WSGI with gunicorn to serve GUI
- [PORTAL] [SYSTEM] Run portal as non-root
### Removed
- [APACHE] Completely remove apache package and logic in code (service, handling, etc...)
## [2.1.4] - 2022-11-18
### Fixed
- [IDP] [API] Don't fail on token updates with explicit expire_at
- [API_PARSER] [IMPERVA] Use requests' auth object instead of manually created Authorization header
- [API_PARSER] [IMPERVA] Typo in logger made the process crash
- [API_PARSER] [SYMANTEC] Correctly extracting token from downloaded archive
- [SSO_FORWARD] Missing variable in logger
- [PORTAL] Various logger levels and message corrections
### Removed
- [API_PARSER] [ELASTICSEARCH] Obsolete/non-working parser
## [2.1.3] - 2022-11-09
### Fixed
- [API_PARSER] [IMPERVA] Prevent infinite download loop when a file cannot be decoded
- [API_PARSER] [SYMANTEC] Fix retrieve of token from downloaded archive tail
### Added
- [API_PARSER] [CROWDSTRIKE] Add api_host to 'url' event log field
## [2.1.2] - 2022-10-27
### Changed
- [INTERNAL] [REDIS_SESSIONS] Allow key's expiration to be set using a datetime object
- [IDP] [API] token's APIs now take an 'expire_at' parameter instead of a 'timeout' parameter
### Fixed
- [API_PARSER] [SOPHOS_CLOUD] Api return 400 http_code if trying to fetch more than 24h
- [API_PARSER] [IMPERVA] Download multiple files at once, to prevent delay increasing
- [API_PARSER] [SYMANTEC] Don't use token if it's older than 24h, seems to be expired
- [API_PARSER] [CROWDSTRIKE] Prevent last_api_call stagnation if no events where retrieved during the last 24h
## [2.1.1] - 2022-10-19
### Fixed
- [PORTAL] [SSO] SSO could fail when portal have no oauth2 activated
- [PORTAL] Correctly refresh session timeout only when necessary
- [APACHE] [CONFIGURATION] Update Apache WSGI's configuration to allow code to read 'Authorization' header
- [API_PARSER] [SENTINEL_ONE] Avoid parser getting stuck after not having received any logs for more than 24h
- [API_PARSER] [GSUITE] Parser failure caused by wrong 'since' variable type
- [API_PARSER] [CROWDSTRIKE] Parser failure caused by wrong 'since' variable type
- [USER_PORTAL] [API] Could not get all portals through API when no ID/name was specified
- [PORTAL] Validate session Cookies, then Vulture Headers, then 'Authorization' Header during session validation
- Allows protected applications to use 'Authorization' Headers when a user browses it with a web session
### Added
- [UPGRADE] Added bash script to uninstall python 3.8 and install generic python in apache and portal jail
- [IDP_USERS] Support for new API paths
- Added paths for uniformization with new /tokens endpoints
- Ability to use portal/repo names in queries
### Changed
- [PORTAL] [LOGGING] Improved logging and details in case of errors
- [DEPENDENCY] Changed uses of python 3.8 to generic python3 to use the default python version for the system (ie python 3.8 for HBSD12 and 3.9 for HBSD13)
### Removed
- [PKG_UPGRADE] Don't rebuild python env using `virtualenv`, binaries are no longer symlinked
## [2.1.0] - 2022-09-27
### Fixed
- [LDAP_CLIENT] Use escape_dn_chars() instead of escape_filter_chars() in search_user() for sanitation
- [OAUTH] Ensure 'sub' is present in oauth2 tokens
- [BOOTSTRAP] [CONFIGURE.SH] wrong order of imports in script
- [API_PARSER] [SYMANTEC] Handle cases where the token is not in archive
- [OUAUTH] [TOKEN] exp and iat from /token are correctly set when returning an oauth token
- [ACLS] Avoid multiple declarations of the same set of rules in generated Haproxy configurations
### Changed
- [IDP_API] Improved logging to include systematic portal, repo, user (when available) and action
### Added
- [IDP] [API] New APIs to manage oauth2 tokens creation
- Create a new token for a specific user, on a specific portal and repository
- User must authenticate themselves with a valid oauth token
- Claims will be created using internal repository data and IDP scope filtering
- Expiration time can be overridden during POST/PATCH
- Tokens can be deleted by user, they will be removed if user is removed
- [LDAP] [TOOLS] Utility function to get a user by its full DN
- [REDIS] [TOKENS] Utility function to check if a key exists in Redis
- [REDISOAUTH2SESSION] Utility function to delete an existing key and its subordinate(s)
- [REDIS] [BASE] New scan_all() utility function to get all keys with possible matching pattern and type
- [API_PARSER] [GSUITE] Add ipv4/ipv6 proxy support for api parser
## [2.0.0] - 2022-09-15
### Fixed
- [VULTURED] [MONITOR] Avoid crashes when a new node joins a cluster
- [STRONGSWAN] [CLEANUP] Remove unnecessary exception condition
- [STATICS] Fixed static dirs definition in settings
- [CTI] Use database relation to link tenant to a feed instead of its filename
- [BACKEND] [CONSTRAINTS] Old Server target:port unicity has been manually and completely removed
- [NET2MONGO] Find available external IPs more efficiently by searching different syntaxes in configuration files
- [API_PARSER] [GUI] [SYMANTEC] errors not showing correctly when testing
- [LOG_FORWARDER] [PF] Add missing NAT rules for log forwarders in some Frontend LOG modes (Redis, Kafka)
- [LOG_FORWARDER] [CONF] Reload associated Frontends' configuration when changing a LogForwarder
- [FRONTEND] [GUI] Return an error when a Frontend uses an already existing Listener (error on duplicates)
- [BACKEND] [GUI] Avoid errors on backend copy
- [SYSTEM] [APACHE] Change wrong apache config reloads to apache service reloads
- [RSYSLOG] [TEMPLATES] Formatting errors on `generic_json` output template
- [FRONTEND][API_COLLECTOR][HARFANGLAB] Prevent duplicated collected alerts by using advanced API parameters
- [HAPROXY][CONF] Don't bind Listeners' ports configured for other nodes
- [ERROR_TEMPLATE] Reload haproxy every time a template is rewritten (not only when related Frontends' conf has changed)
- [FRONTEND][API_COLLECTOR][SENTINELONE] Prevent duplicated events by modifying last_api_call variable
### Removed
- [DEFENDER] Removed defender policies and rules
- [BHYVE/VM] Removed VM functionalities
- [GUI] [LOGVIEWER] Removed LogViewer
- [YARA] Removed yara policies/rules
- [GUI] [ZFS] Removed ZFS administration
- [NETWORK] Removed vm-public/tap0 interfaces
- [FRONTEND] [IMPCAP] Removed support for impcap Listeners (also remove existing listeners)
- [PF] [BLACKLISTING] Removed blacklisting from default predator databases
- [PF] [NAT] Removed useless RDR/NAT for DNS, netdata and dashboard
- [DARWIN] [CLUSTER] Removed log <-> alert recontextualisation from Vulture code
- [LOG_FORWARDERS] Removed 'Internal Dashboard' from available forwarders
- [DOCUMENTATION] Removed in-GUI documentation
- [RSS] Removed unused Predator news system
- [CLEANUP] Removed obsolete/deprecated files
### Changed
- [JAIL] Differentiate 'apache' and 'portal' jails' secadm rules and gunicorn configuration
- [PREDATOR] Disabled most Predaotr support : only some generic databases are downloaded (no tenant-specific DB)
- [PF] [CONFIG] Only reload pf config when required
- [SYSTEM] Check if the node is bootstrapped through database (and not through system files)
- [API_PARSER] [SYMANTEC] make use of external API's 'token' information to get logs continuously while avoiding duplicates
- [MODELS] Added fields option for filtering to ErrorTemplate's to_dict function
- [MODELS] Changed to_dict to include frontends related to the Error Template in to_dict function
- [PF] [CONF] Avoid creating duplicate NAT rules for log forwarders
- [WORKFLOW] [GUI] Optimized how resources are gotten from APIs during workflow edition/creation
- [BACKEND] [GUI] Optimized how http_health_check_headers are provided in forms
- [SYSTEM] [DEPRECATED] use of management.ip file in favor of rc conf in /etc/rc.conf.d/network
- [API_PARSER] [BASE] handle SIGINT/SIGTERM signals to ask parsers to stop
- [API_PARSER][AKAMAI] Stop when a SIGINT/SIGTERM is received
- [API_PARSER][BLACKBERRY_CYLANCE] process at most 24h of logs during an execution
- [API_PARSER][CROWDSTRIKE] process at most 24h of logs during an execution
- [API_PARSER][DEFENDER] process at most 24h of logs during an execution
- [API_PARSER][GSUITE] process at most 24h of logs during an execution
- [API_PARSER][NETSKOPE] process at most 24h of logs during an execution
- [API_PARSER][RAPID7_IDR] process at most 24h of logs during an execution
- [API_PARSER][SENTINEL_ONE] process at most 24h of logs during an execution
- [API_PARSER][SYMANTEC] Don't wait for next execution when being rate-limited by the API, stop and let next execution handle the next fetch
### Added
- [DJANGO] [SETTINGS] Added 'serverSelectionTimeoutMS' for database(s) connection timeout
- [UTILS] [MONGODB] New connect_with_retries() function to wait for a valid database connection with retries before returning
- [MIGRATIONS] New model migrations for deletions and updates
- [UPDATE_SCRIPTS] New migration scripts to remove obsoleted objects and configuration
- [API_PARSER] New parser Trendmicro Worryfree
- [API_PARSER] [MODEL] [SYMANTEC] new internal field 'token'
- [API_PARSER] [SYMANTEC] support for multiple archives in file downloaded from API
- [API] GET, POST, PUT, DELETE API methods for adding, modifying and deleting http messages
- [MODELS] [TO_DICT] Ability to filter fields from objects with an optional `fields` parameter (related objects are still fully serialized)
- [API] Ability to query object(s) and select the fields to return using the new `fields` optional parameter in the `to_dict()` function of models
- [API] [FRONTEND] Ability to query frontends on their `mode` and/or `enabled` status
- [API] [BACKEND] Ability to query backends on their `mode` and/or `enabled` status
- [CLUSTER][MESSAGE_QUEUE] Ability to wait for message result in code
- [UTILS] [RC] Toolkit functions to get/set rc variables in main or sub-files
- [CLUSTER] [MESSAGE_QUEUE] message instance(s) returned from api_request() calls (to be able to wait on them and/or get result of execution
- [RSYSLOG] [TEMPLATES] Missing elastic templates in built-in log types' output templates
- [FRONTEND][LOG] add (advanced) parameters for rate-limiting on TCP log listeners
- [LOG_FORWARDER][SYSLOG] add (advanced) parameters for output rate-limiting on syslog forwarder
- [LOG_FORWARDER][ELASTICSEARCH] add (advanced) parameters for output rate-limiting on elasticsearch forwarder
- [CHANGELOG] Missing file for past known releases
- [API_PARSER] New safenet parser
## [1.18.0] - 2022-08-05
### Fixed
- [HAPROXY][CONFIGURATION] Split spoe configuration directives to avoid configuration parse limits
- [HAPROXY][CONFIG_GENERATION] Correctly order regeneration of Haproxy files when changing objects related to Authentication
- [HAPROXY][CLEANUP] Silence most warnings, resolved minor problems and clarified frontends configuration files
- [PORTAL] Credentials decoding error for BASIC authentication
- [OPENID] Use system's proxy and `verify_certificate` option while fetching a token on an external IDP
- [OPENID] [GUI] testing already configured OpenID returned an error
- [API_PARSER] [SENTINEL_ONE] timestamp inconsistencies could cause holes or duplicates while fetching logs
- [API_PARSER] [WAF_CLOUDFLARE] use timezone-aware timestamps, and correctly update last_api_call
### Added
- [OPENID] [GUI] Prevented suggestion of previous OpenID client secrets
- [GITIGNORE] Python environments
- [PORTAL] Allow authentication through Oauth2's PKCE validation, allowing direct authentication with single-page apps
- [PORTAL] Allow authentication through `Authorization` with a Bearer token
- [PORTAL] Allow authentication through `X-Vlt-Token` with a Bearer token
- [UPDATE] [SCRIPT] New script to update haproxy configuration files
## [1.17.0] - 2022-08-02
### Added
- [API PARSER] WAF Cloudflare proxy support
- [API PARSER] Office365 proxy support
- [API PARSER] Gsuite proxy support
- [API PARSER] Sophos proxy support
- [API PARSER] ProofpointPod proxy support
## [1.16.1] - 2022-07-18
### Fixed
- [FRONTEND] Fix syntax error in fields list, causing 500 error when creating/editing new listener
## [1.16.0] - 2022-07-15
### Added
- [FRONTEND][API_COLLECTOR] Add support for fetching SophosCloud events
## [1.15.0] - 2022-07-07
### Added
- [FRONTEND][API_COLLECTOR] Add support for fetching GSuite alerts
### Fixed
- [FRONTEND][API_COLLECTOR][HARFANGLAB] Fix duplicated alerts collected
## [1.14.0] - 2022-07-01
### Fixed
- [API][FRONTEND] Avoid errors while trying to create new header's objects on a Frontend
- [FRONTEND][HEADERS] Handle headers' values with spaces, simple quotes and double quotes
### Added
- [API][PARSER] New api parser WAF Cloudflare
## [1.13.0] - 2022-06-24
### Added
- [SSOForward] add configurable request timeout on SSOForward requests
- [HTTPD CONF] Servers' timeouts are now in a separate timeouts.conf file
- [PARSER] [API] new parser Proofpoint PoD
- [GITIGNORE] python env files and folders
- [PARSER] [UTILS] internal function to get an object in arbitrary-length leaf, or return default otherwise
### Fixed
- [PARSER] [API] messages weren't sent completely to Rsyslog when messages exceeded a certain size
- [API_PARSER] [HARFANG] get correct field for timestamp extraction
- [API_PARSER] [AKAMAI] Properly acquire data_lock during internal last_log_time updates
## [1.12.1] - 2022-05-17
### Fixed
- [PORTAL] Create a new portal session value from safe generator if value taken from client cookies is not formatted correctly or is not present in current cache
### Added
- [UTILS][HASHES] validation function for hash digest's format (length and characters used)
## [1.11.0] - 2022-03-31
### Fixed
- [HAPROXY][FRONTEND] ACLs generated were wrong on some conditions
- [HAPROXY][BACKEND] ACLs generated were wrong on some conditions
- [ACLS] patterns with spaces given on GUI weren't quoted in configuration
- [CLUSTER INSTALLATION] wrong variable utilization in cluster_join()
- [PORTAL] correctly set 'iat' field in token
- [SELF_SERVICE] avoid errors during password change when user does not exist
- [API_LOGGER][SENTINELONE] Use correct timestamp from 'activity' alerts to update last_api_call
- [API_LOGGER][SENTINELONE] add a 'to' argument to fix queried intervals and avoid gaps between executions
### Changed
- [GUI][ACL] Print the correct boolean syntax for haproxy on the edition preview
- [SERVICE] [HAPROXY] prefer reloading service instead of restarting (start a new process that cleanly takes over the old one)
- [IDP] set reset password URL validity to 72 hours
### Added
- [PORTAL] ensure the session stays valid for at least 5 minutes when requesting double-authentication on a portal
- [PORTAL] centralize portals haproxy load-balancing and add healthchecks to avoid errors while nodes/portals are unavailable
- [API_LOGGER][SENTINELONE] better logging of parser execution
## [1.10.0] - 2022-03-18
### Added
- [API_PARSER] Blackberry Cylance
## [1.9.0] - 2022-03-07
### Added
- [FRONTEND][API-PARSER] Add Nozomi (probe) alerts collect support
## [1.8.0] - 2022-02-23
### Added
- [PARSER][API] New API parser Vadesecure for Microsoft O365
## [1.7.1] - 2022-02-17
### Fixed
- [PORTAL] [GUNICORN CONF] service didn't listen on jail's IPv6 address (no connection from IPv6 networks)
### Added
- [API][URLS] uniformize URLs to work with and without the trailing '/'
## [1.7.0] - 2022-02-03
### Fixed
- [HAProxy][Frontend] Fix acl syntax according to last haproxy version
### Changed
- [PORTAL] Use **[gunicorn](https://docs.gunicorn.org/en/stable/)** instead of **[apache wsgi](https://pypi.org/project/mod-wsgi/)** to host portal engine
## [1.6.1] - 2022-02-03
### Added
- [FRONTEND]
- Add 'validate' boolean parameter to disable config testing
- Improve performances by reloading frontend config asynchronously
- Refactor jinja template haproxy_frontend for better performances
- [API][CLUSTER] Add supervision api endpoint to get tasks (*/api/v1/system/cluster/tasks/*)
- [API][CLUSTER] Add supervision api endpoint to get cluster/node status (pending tasks, frontend statuses) (*/api/v1/system/cluster/tasks/*)
- [IDP][API] Deny username changes on an existing user entry
- [DARWIN][API] Now possible to get a Policy by its name
- [CLUSTER][JOIN] Sync master's secret key between joining nodes to avoid encrypion/decryption problems between nodes of a same cluster
### Fixed
- [VULTURED] Update status of task if an error occurred (status was let 'running' forever)
- [API-PARSER][ALL] Fix status update of frontend when an error occurs - "error" was never set
- [UPGRADE] Fix package upgrade failure due to conflicted versions of python package in environment
- [PORTAL] URL link for 'lost password'
- [PORTAL] Error while retrieving user from mail while loading 'lost password' URL
- [PORTAL][SELF_SERVICE] Password reset email missed a '/' in the reset URL
- [PORTAL] Avoid strange redirections during authentication (caused by previous queries)
- [UTILS][AES] Padding errors
- [TOTP] Various errors preventing TOTP to work properly
- [PORTAL] Only allow user's session at the end of the whole process, permitting SSOForward to work as expected when Learning values are asked to the user
- [SSOForward] Use response.text to avoid errors when getting and parsing forms
- [SYNTAX] Code cleanup and syntax corrections
- [TOTP] Avoid errors during disconnexion when portal has no OTP configured
- [LDAP] Correctly create group in LDAP when IDP portal must add a new user to it
- [LDAP] Don't apply sanitation on group enumerations in code
- [PORTAL] Redirect user to workflow's public_dir instead of previous page in case of (re)authentication failure
- [PORTAL] Captcha field now works as intended
- [PORTAL] [SELF_SERVICE] reset password key is now hidden in 'password change' form
- [LDAP] Avoid decoding errors when formatting data from user search (could result in 500)
- [PORTAL] Correctly redirect client on successful authentication on a portal without SSOForward
- [CLUSTER][JOIN] Correctly sync Redis between joining nodes of a same cluster
### Removed
- [SECADM] Obsolete 'node' rule
### Changed
- [PORTAL] [SELF_SERVICE] Don't make a difference when returning for a password reset request between a valid and an invalid email (prevent user enumeration via email)
- [PORTAL] [SELF_SERVICE] Improve errors reporting during password reset request
- [PORTAL] [SELF_SERVICE] Password reset URLs are now available for 3 days
- [TOOLS] [LDAP] Cleaner errors
- [PORTAL] [SELF_SERVICE] Refuse empty password when changing them
- [OPENID] OpenID repositories can no longer have the same 'client_id' (duplicates will be deleted, and replaced with the remaining object when used)
## [1.5.2] - 2022-01-05
### Fixed
- [API_PARSER][SENTINEL_ONE] Update last_api_call for all kinds of events
## [1.5.1] - 2021-12-29
### Fixed
- [API_PARSER][HARFANGLAB] Isoformat already setting timezone - prevent invalid timestamp format
## [1.5.0] - 2021-12-08
### Added
- [API PARSER] CROWDSTRIKE API parser
## [1.4.1] - 2021-12-03
### Fixed
- [API_PARSER] [DEFENDER] Use system proxy when available