RFC: Firewall use case

[njlavigne]

An "RFC" with a collection of use cases, feature ideas, and opportunities related to using Suricata in a firewall role.

I don't expect to merge this, but look forward to the discussion.

[victorjulien]

First of all, thanks a lot for this write up Jamie. It's great. Side note: we've been debating RFCs as a tool for a while, and I think this a great example of how it can be used (or at least started :) )

Some incomplete initial thoughts:

I agree a different type of rule-set should be made possible. In this new set type we should use a different approach: explicit everything, no automagic behind the scenes changing behavior (e.g. IP-only), no complex and hard to follow conditions leading to vastly different outcomes (e.g. pass pkt vs pass flow).

I think our first step should be to define a scheme for this, then worry about how to get there is a reasonable backwards compatible way later.

I've often thought about allowing users to annotate rule files with specific config directives, but rule managers (including suricata-update) generally create a single rule file out of the many, so it may not be helpful to mark individual rule files as different types. So instead, perhaps Suricata can just load files differently.

E.g. something explicit like

default-rule-path: /var/lib/suricata/rules
rule-files:
  - suricata.rules
firewall-rule-path: /var/lib/suricata/fwrules
firewall-rule-files:
  - firewall.rules

This could then impose a strict set of conditions on the rules in that file, as well as the individual rules:

• the rules in the file would never change ordering.
• rules would be force to only pick explicit detection hooks and explicit actions
  e.g. drop:pkt tcp:pre-stream any any -> any any (tcp.flags:R+; dsize:>0; msg:"DROP RST with data before the stream engine";)
• Explicit actions would require clear documentation obviously
• Hooks need to be clearly defined
• Optimizations can never change the behavior.

An example of something we do today, but should not do in this idea, is the current IP-only logic. Currently rules are evaluated after parsing to see if they meet the "IP-only" conditions. If so, they automagically change their behavior in significant ways:

1. inspection happens only for first packet per flow dir
2. actions apply to the flow
3. they are evaluated before other rules, so violating any ordering logic

In my pseudo rule lang example above, we could get to the same result using something like:

pass:flow ip:flow_start_bidir 1.2.3.4 any -> any any ();

The engine could then still optimize it behind the scenes to use a radix tree or similar, as long as the other conditions are not violated.

I guess a fundamental decision that this would require is: how would these rules interact with "common" IDS/IPS rules?

Here an analogy with a common IPS setup may be helpful.

When building a firewall on linux, a common setup is:

1. an (nf|ip)tables rule-set for managing an IP level default-DENY policy
2. for any traffic that would be ACCEPTed, instead use NFQUEUE
3. Suricata will thus only receive (over NFQUEUE) the traffic within the constraints of the iptables rule-set, and can apply it's IPS signatures to drop the bad traffic and pass the rest.

I wonder if that is useful as a goal to replicate. Like a 2 stage process.

1. enforce rule based policy (based on IP/TCP but also app-layer things like SNI)
2. inspect what matches the policy in (1) against the more threat detection oriented signature set

I guess this would be expressed more like

policy-rule-path: /var/lib/suricata/policy-rules
policy-rule-files:
  - policy.rules # e.g. bob can talk to alice
signature-rule-path: /var/lib/suricata/signatures
signatures-rule-files:
  - signatures.rules # e.g. ET open rules-set for detecting various attacks

Rules vs signatures?

In the suricata docs, code and general terminology we use "signatures" and "rules" interchangeably. However this feels like a mistake. The current engine is very much a signature engine, where we're looking for the needle in a haystack and we can reorder and interpret things to a large extend to do so in an optimized way. The firewall use-case wants to see a policy engine, with clear, explicit and predictable rules.

I don't know if it is too late for us, but we can try to separate these terms to reflect these different meanings. Alternatively, we need something new. E.g. a "policy rule" or something.

[njlavigne]

First of all, thanks a lot for this write up Jamie. It's great. Side note: we've been debating RFCs as a tool for a while, and I think this a great example of how it can be used (or at least started :) )

Happy to help contribute to this - it was Juliana that suggested an RFC format when I came looking for an approach that would work for this kind of brainstorming.

I've often thought about allowing users to annotate rule files with specific config directives, but rule managers (including suricata-update) generally create a single rule file out of the many, so it may not be helpful to mark individual rule files as different types. So instead, perhaps Suricata can just load files differently.

I like the idea of a separate rule or file type. You didn't say it directly but if Suricata were to introduce a new "firewall" rule type then I think it's important that these rules cannot be confused for IDS rules - so an IDS rule could not be loaded as a firewall rule and vice versa. If the firewall rules were to require the explicit actions like drop:pkt that would be enough to distinguish them and allow the rest of the structure to remain familiar enough that they would make sense to users.

All of my concerns around backward compatibility in this case would only apply to the new firewall policy rules, and the IDS part of the engine can retain the freedom to change interpretations of IDS signatures where it makes sense to do so.

I wonder if that is useful as a goal to replicate. Like a 2 stage process.

1. enforce rule based policy (based on IP/TCP but also app-layer things like SNI)
2. inspect what matches the policy in (1) against the more threat detection oriented signature set

Yes, absolutely - I didn't mention it here but I have separately been looking for a way to do this. Separating the two rulesets is ideal for this because I think a very common case will be for users to author their own firewall policy, while continuing to use a set of threat detection signatures sourced from an external provider.

I don't know if it is too late for us, but we can try to separate these terms to reflect these different meanings. Alternatively, we need something new. E.g. a "policy rule" or something.

From my experience I would expect that updating docs/code would be doable, but getting a large existing userbase to adopt the change consistently will be much harder. So if it's one or the other then introducing some new terminology for a new feature would certainly be easier. Unfortunately in the firewall world "rules" are an already well-established term too - "firewall rules" could be an option but there will be a tendency to shorten it to just "rules", which is ambiguous.

However if the rule types are distinct and not interchangeable (as above) then this may not be a real problem in practice? You could say "firewall rule" or "IDS rule" when it's necessary to disambiguate but otherwise it should be clear from the context.

[jasonish]

In the suricata docs, code and general terminology we use "signatures" and "rules" interchangeably. However this feels like a mistake. The current engine is very much a signature engine, where we're looking for the needle in a haystack and we can reorder and interpret things to a large extend to do so in an optimized way. The firewall use-case wants to see a policy engine, with clear, explicit and predictable rules.

I mostly agree, but the naming could still be confusing? How much of a signature could a firewall rule contain? You mention TLS SNI, would we maybe restrict what could go in these rules?

[njlavigne]

I mostly agree, but the naming could still be confusing? How much of a signature could a firewall rule contain? You mention TLS SNI, would we maybe restrict what could go in these rules?

I think that would be the way I would approach it - starting with a restricted smaller set of supported features such that the behavior could be very well specified & tested, and expand it over time.

Would it help if I were to provide a list of what I see as the important core features as a starting point?

[victorjulien]

I mostly agree, but the naming could still be confusing? How much of a signature could a firewall rule contain? You mention TLS SNI, would we maybe restrict what could go in these rules?

I think that would be the way I would approach it - starting with a restricted smaller set of supported features such that the behavior could be very well specified & tested, and expand it over time.

Would it help if I were to provide a list of what I see as the important core features as a starting point?

Yes absolutely!

[jufajardini]

Really appreciate the effort put into sharing your experience and suggestions here, Jamie. It helps me have a better view of many things.

As you know, we're working on documenting the rule types - and thanks for your feedback there, btw. The ticket for documenting the --engine-analysis output is something I would also like to tackle soon.

Rules vs signatures?

Agreeing with Jamie here that new terminology could be better.

I wonder if that is useful as a goal to replicate. Like a 2 stage process.

+ 1 on this, especially with the explicit rule language

[jufajardini]

Wonder if it would be helpful to highlight this RFC on discord, listing main points for discussion, to see if we get more input.

[njlavigne]

I think that would be a good idea and useful to see if e.g. the opnsense team has thought much about this possibility.

[jufajardini]

Good point. Pondering how/ where to document this.

I think the place where we touch this indirectly is when discussing the Exception Policies, but as that's not the goal there, it's not a full description.

Should at least add a ticket to track this.

[jufajardini]

Well, there's one, already https://redmine.openinfosecfoundation.org/issues/5465

[victorjulien]

https://redmine.openinfosecfoundation.org/issues/7481

[njlavigne]

My first attempt at putting some rough ideas together on what a firewall-specific rule language might look like. Some of the ideas may still be half baked or a bit too dreamy to be realistic, but I think it's still useful as a starting point to get something out there that we could start poking holes in and refining.

Baseline assumptions:

Firewall rulesets will always feature these properties:

• Strict ordering: rules are evaluated in the order written, without reordering based on action or other properties prior to evaluation.
• All rules are "terminating" - matching against a rule means that the decision ends there, and no later rules will be evaluated.
• Midstream policy and async-oneside features are used to enforce that all connections are symmetrically routed. Connections that are not observed by Suricata in both directions should fail as early in the connection as possible.
• App-layer and memcap policies are assumed to fail closed (at least for the firewall rule processing part, IDS could be different)
• Default deny - anything not specifically allowed would be subject to a default action (drop or reject). It would be possible to configure a permissive "pass" as the last rule, but this wouldn't be the default.

Core theme: explicit intent with inference

We have spoken about explicit intent as a desirable feature for deterministic and predictable behavior. I've been thinking about that a little bit in the context of how newer languages like rust blend that with an ability to infer things like type information that are well-specified from context but without the extreme verbosity. To piggyback on your recent example from PR 12422:

pass tls:client_hello_done any any -> any any (tls.sni; content:"www.google.com"; sid:21; alert;)

The PR proposes an optional event hook (the client_hello_done) that lets the rule evaluate deterministically at a well-specified point in the life of the connection, which is a great thing. I'd be interested in your thoughts around retaining that determinism (for the firewall rule language at least, separate from the IDS one) but with the explicit event as optional. The idea being that even without the :client_hello_done specifier, the engine should be able to know from the rule options that client_hello_done is the earliest event in the lifetime of the connection where it's safe/possible to evaluate the tls.sni condition and choose that. With this idea it would still be allowed to specify the event hook, but if it were omitted then the engine would choose the earliest possible point, meaning that in practice I would expect the explicit event hook specifiers to be used rarely since the engine can usually choose the right option by itself.

The same idea extends to the lower-layer rules that are necessary today for application-layer rules to work with default deny. Consider again the tls protocol example rule above - used with a default-deny policy, it would normally be necessary to pair that with a TCP-layer rule to allow the connection to become established and reach the point where the upper layer protocol emerges, similar to

# Something like this to allow TCP & TLS handshaking to pass the default deny
pass:pkt tcp any any -> any any (flow:not_established; sid:20;)

# Application-layer TLS rule
pass:flow tls any any -> any any (tls.sni; content:"www.google.com"; sid:21; alert;)

# All others dropped by default

The same idea applies here - the pass:pkt is a rule that explicitly matches at the packet level and mirrors some parts of the tls layer rule, but can be seen by users as a kind of boilerplate that shouldn't be necessary, because the engine should have enough information to know that a rule like that is always needed in this case. I think the reality may be a bit more complicated than a static rule to handle all cases like protocol upgrades that happen later, but taking some kind of a tradeoff that makes the common case easy at the expense of more complexity expressing rules for connections that upgrade later via things like STARTTLS (which is rare) seems like an attractive idea, and fits well with the "simple things should be easy and advanced things should be possible" mantra that often leads to good designs. Similar to the one above, this would mean that in practice the packet-type actions like pass:pkt would be valid but uncommon in real world use.

Example ruleset

An example ruleset for illustration:

# Example of an IP layer rule blocking on IP attributes.  This rule is first because regardless
# of the higher-layer rules below, we don't want to allow any communication with certain geos.
drop:pkt ip any any -> any any (geoip:any,AB,CD; msg:"Block certain geos"; sid:100)

# Allow outbound ICMP echo requests.  This is a stateful flow rule (pass:flow) that is smart
# enough to allow the responses to come back too.  I don't believe this is possible to express
# today in suricata - stateful ICMP flows work, but not with ICMP type filtering, since the
# responses use a different type: https://redmine.openinfosecfoundation.org/issues/6268
# The port number "any" is required by syntax, despite the protocol not having ports
pass:flow icmp $HOME_NET any -> $EXTERNAL_NET any (icmp_type:echo_request; msg:"Ping!"; alert; sid: 101)

# A TLS layer drop rule matching on JA3.  This rule implicitly uses the client_hello_done event hook,
# because Suricata knows that is the earliest point in the connection when the data necessary to
# evaluate JA3 is available.  Similarly a "pass:pkt tcp $HOME_NET any -> 172.16.0.0/12 any not_established"
# type rule that would normally be necessary to pass the handshake is omitted because Suricata knows those
# are necessary to reach the client_hello_done point.
drop:flow tls $HOME_NET any -> 172.16.0.0/12 any (ja3.hash; content:"e7eca2baf4458d095b7f45da28c16c34"; \
    msg:"Drop naughty JA3"; sid:102;)

# Disallow TLS v1.0 to some destinations.  This rule uses the explicit :server_hello_done event hook,
# but it is also possible to omit that because Suricata is aware that it is the earliest point where
# the negotiated version is available.  Similar to the rule above, this one does not explicitly specify
# a "pass:pkt tcp $HOME_NET any -> $EXTERNAL_NET any" rule necessary to allow the lower-layer handshake.
drop:flow tls:server_hello_done $HOME_NET any -> $EXTERNAL_NET any (tls.version:1.0; \
    msg:"TLS 1.0 not allowed"; sid:103;)

# Finally, allow TLS connections by SNI that are not blocked by any of the rules above.  Since the match
# condition is on SNI, this would normally be evaluated at the :client_hello_done hook, but due to the
# context it may be necessary to delay this evaluation until :server_hello_done - for connections that
# match both this rule and the one above (103), the action of rule 103 should be applied because it is
# ordered first.
pass:flow tls $HOME_NET any -> $EXTERNAL_NET any (tls.sni; content:"www.google.com"; sid:104; alert;)

# Implicit drop all else