Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extend Suricata support #8372

Closed
wants to merge 139 commits into from
Closed

Extend Suricata support #8372

wants to merge 139 commits into from

Conversation

0xThiebaut
Copy link
Contributor

@0xThiebaut 0xThiebaut commented Feb 22, 2025

Extend Suricata support in OPNsense to increase the monitoring capabilities (e.g., integrate with Malcolm)

Features:

  • Extend support for EVE log, supporting all available protocols (RDP, DNS, ...).
  • Add support for PCAP logs.

Subsequent changes:

  • Update configuration template to align with Suricata 7.0.8 (e.g., add new fields such as ja4).
  • Disable stats.log (stats can be enabled in EVE).
  • Add migration logic.

fichtner and others added 30 commits January 21, 2025 11:25
(cherry picked from commit 2862db6)
While here simplify the unlock code and correct the comment.

(cherry picked from commit 814e983)
…sense#8228)

The NetworkValidator fails on IPv4-mapped addresses[1], for example ::ffff:198.51.100.0/120.  Inferring IPv4 from the presence of a period (.) fails as these IPv6 address forms contain periods. Inferring IPv6 from the presence of a colon (:) should be more robust.

1. https://www.rfc-editor.org/rfc/rfc4291#section-2.2

(cherry picked from commit c375cc5)
By ignoring invalid UTF-8 we can get the bulk of the data.  It's
not likely we are ever going to get unencoded binary data through
JSON, especially in pluginctl.

PR: https://forum.opnsense.org/index.php?topic=45194.0

(cherry picked from commit 60f297f)
(cherry picked from commit dcc457b)
- split up the logic into class collection and status collection
so that out of scope objects don't need to check their status.
- with the previous, status dismissal doesn't need a status check
either anymore
- remove the UI delay from head.inc as well
- scale up the disk space status thresholds a bit for systems with
lower assigned disk space
- non-persistent status objects without a location had their
pointer-events removed, making it non-dismissable

(cherry picked from commit cfdd274)
(cherry picked from commit 788680a)
(cherry picked from commit 2bb0301)
https://github.com/hagezi/dns-blocklists

closes opnsense#8224

Deprecated (but can still be downloaded for compatibility reasons):

* NoCoin List
* All Porn List
* PornTop1M List
* Simple Ad List
* Simple Tracker List
* WindowsSpyBlocker (*)

(cherry picked from commit 4f12007)
PR: opnsense/installer#20

(cherry picked from commit c8871da)
(cherry picked from commit c38dbab)
(cherry picked from commit 7335db7)
…8198

It's probably still best to prevent editing associated rules as we can not guarantee settings stick, for unassociated rules, we should be able to edit them.
Fixes missing type in the process here (on firewall rules).

(cherry picked from commit e4b6ec1)
…me requires a token lifetime.". closes opnsense#8203

(cherry picked from commit 526d747)
* aliases: add optional authorization for URL type aliases

* aliases: missing break statement

* aliases: update helptext and link to proper id

* aliases: merge token and password fields and shorten code

* Firewall: Aliases - minor modifications for opnsense#8238

o re-add selectpicker, title on select killed the default bevaior
o change token to password as the default use-case is username+password, bearer is less common / functional

* aliases: validation should be adjusted as such

---------

Co-authored-by: Ad Schellevis <[email protected]>
(cherry picked from commit 8d26447)
…arses json payloads and extracts addresses, closes opnsense#8107

While here, also fix a minor issue in opnsense#8238 to calculate a proper alias has value when auth properties are specified.

(cherry picked from commit 03a8812)
Monviech and others added 10 commits March 10, 2025 08:47
(cherry picked from commit d68d48f)
…y_button (opnsense#8253)

(cherry picked from commit 855bd7a)
(cherry picked from commit 24a6142)
(cherry picked from commit 8a9fe28)
(cherry picked from commit bac5dfe)
(cherry picked from commit 8591b00)
(cherry picked from commit 83723fd)
(cherry picked from commit d7d45dc)
AdSchellevis and others added 11 commits March 14, 2025 12:02
…e stats instead of dumping them.

This commit changes PF.list_tables() to yield both the name of the aliases as well as (limited) stats, in places where we only check for totals, these are faster to collect than counting them in python.

There should be no functional impact.

(cherry picked from commit 81ec980)
… to unbound plugin in stead of the existing prefiltering option. closes opnsense#8415

The previous handling "skimmed" the blocklist using regular expressions, but when these lists include wildcards, you need to filter the exact item to exclude it (e.g. *.org.domain in a blocklist will still block a.org.domain in a passlist).

By moving the evaluation to the place where requests are evaluated, we can pass the likely intended domains by their provided regex.
Although there is a performance penalty, it should be limited since we only compile the regex once.

(cherry picked from commit aa2cff3)
(cherry picked from commit 8c63dad)
(cherry picked from commit 50ed8ae)
…nd add unit test

(cherry picked from commit c93e0ac)
(cherry picked from commit b514aaf)
(cherry picked from commit 8496404)
…n "(de)select all" button on the same row.

(cherry picked from commit 676dbb9)
… and move current json static_aliases and interface networks into their own classes.

(cherry picked from commit fdded45)
(cherry picked from commit 655c742)
(cherry picked from commit c3994d1)
… empty node, just changes how these are being created for the first time.

(cherry picked from commit f569ead)
…roper channel and extend them with relevant event information. (ControllerRoot)

(cherry picked from commit 2eb75b2)
(cherry picked from commit 0ee71a7)
@0xThiebaut 0xThiebaut marked this pull request as draft March 15, 2025 21:27
@0xThiebaut
Copy link
Contributor Author

Will create a new PR once this has been fixed.

@0xThiebaut 0xThiebaut closed this Mar 15, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.