Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

wolfSentry update for release 1.6. #113

Merged
merged 1 commit into from
Oct 24, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
106 changes: 106 additions & 0 deletions wolfSentry/src/ChangeLog.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,112 @@

<br>

# wolfSentry Release 1.6.0 (October 24, 2023)

Release 1.6.0 of the wolfSentry embedded firewall/IDPS has enhancements,
additions, and improvements including:

## New Features

This release adds native support for the CAN bus address family, and for
bitmask-based address matching. CAN addresses and bitmasks are now handled in
configuration JSON, as numbers in decimal, octal, or hexadecimal, supporting
both 11 bit (part A) and 29 bit (part B) identifiers.


## Noteworthy Changes and Additions

`wolfsentry/wolfsentry.h`:

* Add `WOLFSENTRY_ROUTE_FLAG_REMOTE_ADDR_BITMASK` and `WOLFSENTRY_ROUTE_FLAG_LOCAL_ADDR_BITMASK` to `wolfsentry_route_flags_t`.
* Add `WOLFSENTRY_ACTION_RES_USER0`-`WOLFSENTRY_ACTION_RES_USER6` to `wolfsentry_action_res_t` `enum`, add `WOLFSENTRY_ACTION_RES_USER7` macro, and refactor `WOLFSENTRY_ACTION_RES_USER_BASE` as a macro aliased to `WOLFSENTRY_ACTION_RES_USER0`.
* Remove !`WOLFSENTRY_NO_STDIO` gate around `wolfsentry_kv_render_value()`.

`wolfsentry/wolfsentry_settings.h`:

* Rename `WOLFSENTRY_NO_STDIO` to `WOLFSENTRY_NO_STDIO_STREAMS`.
* Rename `WOLFSENTRY_HAVE_NONGNU_ATOMICS` to `WOLFSENTRY_NO_GNU_ATOMICS`.
* Added handling for `WOLFSENTRY_NO_SEM_BUILTIN`, `WOLFSENTRY_NO_ADDR_BITMASK_MATCHING`, and `WOLFSENTRY_NO_IPV6`.
* Gate inclusion of `stdio.h` on !`WOLFSENTRY_NO_STDIO_H`, formerly !`WOLFSENTRY_NO_STDIO`.
* Added `WOLFSENTRY_CONFIG_FLAG_ADDR_BITMASKS`, and rename `WOLFSENTRY_CONFIG_FLAG_NO_STDIO` to `WOLFSENTRY_CONFIG_FLAG_NO_STDIO_STREAMS`.

`src/addr_families.c` and `wolfsentry/wolfsentry_af.h`: Split `WOLFSENTRY_AF_LINK` into `WOLFSENTRY_AF_LINK48` and `WOLFSENTRY_AF_LINK64`, with `WOLFSENTRY_AF_LINK` aliased to `WOLFSENTRY_AF_LINK48`.

`src/kv.c`: remove !`WOLFSENTRY_NO_STDIO` gate around `wolfsentry_kv_render_value()`.

`src/json/load_config.c`: In `convert_sockaddr_address()`, add separate handling for `WOLFSENTRY_AF_LINK48` and `WOLFSENTRY_AF_LINK64`.

`Makefile`:

* Refactor `NO_STDIO`, `NO_JSON`, `NO_JSON_DOM`, `SINGLETHREADED`, `STATIC`, and `STRIPPED` to pivot on definedness, not oneness.
* Add feature flags `NO_ADDR_BITMASK_MATCHING` and `NO_IPV6`.
* Rename feature flag `NO_STDIO` to `NO_STDIO_STREAMS`.


## Performance Improvements

`src/routes.c`: Added AF-mismatch optimization to `wolfsentry_route_lookup_0()`.


## Documentation

Add inline documentation for `WOLFSENTRY_NO_GETPROTOBY`, `WOLFSENTRY_SEMAPHORE_INCLUDE`, `WOLFSENTRY_THREAD_INCLUDE`, `WOLFSENTRY_THREAD_ID_T`, and `WOLFSENTRY_THREAD_GET_ID_HANDLER`.

`doc/json_configuration.md`: add documentation and ABNF grammar for `"bitmask"` node in route endpoints.


## Bug Fixes and Cleanups

Fixes for user settings file handling:

* Don't `#include <wolfsentry/wolfsentry_options.h>` if `defined(WOLFSENTRY_USER_SETTINGS_FILE)`.
* Generate and install `wolfsentry/wolfsentry_options.h` only if `USER_SETTINGS_FILE` is undefined, and if `USER_SETTINGS_FILE` is defined, depend on it where previously the dependency was unconditionally on `wolfsentry/wolfsentry_options.h`.
* If `USER_SETTINGS_FILE` is set search it to derive JSON build settings.

`Makefile`: Don't add `-pthread` to `LDFLAGS` if `RUNTIME` is `FreeRTOS-lwIP`.

`wolfsentry/wolfsentry_settings.h`:

* Eliminate inclusion of `errno.h` -- now included only in source files that need it.
* Fix handling for `WOLFSENTRY_SEMAPHORE_INCLUDE` to give it effect in all code paths (previously ignored in POSIX and FreeRTOS paths).

`src/routes.c`:

* in `wolfsentry_route_event_dispatch_0()`, move update of `meta.purge_after` inside the mutex.
* in `wolfsentry_route_get_metadata()`, conditionalize use of 64 bit `WOLFSENTRY_ATOMIC_LOAD()` on pointer size, to avoid dependency on library implementation of `__atomic_load_8()`.

`src/wolfsentry_internal.c`: fix use-after-free bug in `wolfsentry_table_free_ents()`, using new `table->coupled_ent_fn` mechanism.

`src/json/load_config.c`: In `convert_sockaddr_address()`, handle `sa->addr_len` consistently -- don't overwrite nonzero values.

`src/json/{centijson_dom.c,centijson_sax.c,centijson_value.c}`: eliminate direct calls to heap allocator functions in `WOLFSENTRY` code paths, i.e. use only `wolfsentry_allocator`.

`src/json/centijson_value.c`: fix uninited-variable defect on `cmp` in `json_value_dict_get_or_add_()`.


## Self-Test Enhancements

Makefile.analyzers new and enhanced test targets:

* `user-settings-build-test`: construct a user settings file, then build and self-test using it.
* `library-dependency-singlethreaded-build-test` and `library-dependency-multithreaded-build-test`: comprehensive check for unexpected unresolved symbols in the library.
* `no-addr-bitmask-matching-test`, `no-ipv6-test`, `linux-lwip-test-no-ipv6`: tests for new feature gates.
* `freertos-arm32-build-test`: newly refactored to perform a final link of `test_lwip` kernel using lwIP and FreeRTOS kernel files and newlib-nano, followed by a check on the size of the kernel.

Added `wolfsentry/wolfssl_test.h`, containing self-test and example logic relocated from `wolfssl/wolfssl/test.h` verbatim.

`tests/test-config*.json`: added several bitmask-matched routes, added several diagnostic events (`"set-user-0"` through `"set-user-4"`), and added no-bitmasks and no-ipv6 variants. Also removed AF-wildcard route from `tests/test-config-numeric.json` to increase test coverage.

`tests/unittests.c`:

* Additional tweaks for portability to 32 bit FreeRTOS
* Add FreeRTOS-specific implementations of `test_lwip()` and `main()`.
* In `test_json()`, add `wolfsentry_addr_family_handler_install(...,"my_AF2",...)`.
* In `test_json()`, add bitmask tests.
* Added stub implementations for various FreeRTOS/newlib dependencies to support final link in `freertos-arm32-build-test` target.

<br>

# wolfSentry Release 1.5.0 (September 13, 2023)

Release 1.5.0 of the wolfSentry embedded firewall/IDPS has enhancements,
Expand Down
85 changes: 42 additions & 43 deletions wolfSentry/src/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,19 +46,18 @@ be generated from the top of the wolfSentry source tree with `make doc-html`.
This, and the source code itself, are the recommended API references.

The PDF version of the API reference manual is pregenerated and included with source
distributions in the `doc/` subdirectory. The latest version is always
distributions in the `doc/` subdirectory at `doc/wolfSentry_refman.pdf`. The latest version is always
available [on GitHub](https://raw.githubusercontent.com/wolfSSL/wolfsentry/master/doc/wolfSentry_refman.pdf).



## Dependencies

In its default build, wolfSentry depends on a POSIX runtime, specifically the
heap allocator, clock_gettime, stdio, semaphore, pthreads, and string APIs.
However, these dependencies can be avoided with various build-time options. The recipe
<br>
```
make STATIC=1 SINGLETHREADED=1 NO_STDIO=1 EXTRA_CFLAGS='-DWOLFSENTRY_NO_CLOCK_BUILTIN -DWOLFSENTRY_NO_MALLOC_BUILTIN'
```

`make STATIC=1 SINGLETHREADED=1 NO_STDIO=1 EXTRA_CFLAGS="-DWOLFSENTRY_NO_CLOCK_BUILTIN -DWOLFSENTRY_NO_MALLOC_BUILTIN"`

builds a libwolfsentry.a that depends on only a handful of basic string
functions and the `inet_ntop()` library function (from POSIX.1-2001, and also
Expand Down Expand Up @@ -102,43 +101,47 @@ topic.

| `make` Option | Macro Option | Description |
| -------------- | ------------ | ----------- |
| `V` || Verbose `make` output <br> e.g. `make V=1 -j test` |
| `USER_MAKE_CONF` || User-defined make clauses to include at the top of the main Makefile <br> e.g. `make -j USER_MAKE_CONF=Makefile.settings` |
| `EXTRA_CFLAGS` || Additional arguments to be passed verbatim to the compiler |
| `EXTRA_LDFLAGS` || Additional arguments to be passed verbatim to the linker |
| `SRC_TOP` || The source code top level directory (default `pwd -P`) |
| `BUILD_TOP` || Build with artifacts in an alternate location (outside or in a subdirectory of the source tree) <br> e.g. `make BUILD_TOP=./build -j test`|
| `DEBUG` || Compiler debugging flag to use (default `-ggdb`) |
| `OPTIM` || The optimizer flag to use (default `-O3`) |
| `HOST` || The target host tuple, for cross-compilation (default unset, i.e. native targeting) |
| `RUNTIME` || The target runtime ecosystem -- default unset, `FreeRTOS-lwIP` and `Linux-lwIP` are recognized |
| `C_WARNFLAGS` || The warning flags to use (overriding the generally applicable defaults) |
| `STATIC` || Build statically linked unit tests |
| `STRIPPED` || Strip binaries of debugging symbols |
| `BUILD_DYNAMIC` || Build dynamically linked library |
| `VERY_QUIET` || Inhibit all non-error output during build |
| `TAR` || Path to GNU tar binary for `make dist`, should be set to `gtar` for macOS |
| `VERSION` || The version to package for `make dist` |
| `V` | | Verbose `make` output <br> e.g. `make V=1 -j test` |
| `USER_MAKE_CONF` | | User-defined make clauses to include at the top of the main Makefile <br> e.g. `make -j USER_MAKE_CONF=Makefile.settings` |
| `EXTRA_CFLAGS` | | Additional arguments to be passed verbatim to the compiler |
| `EXTRA_LDFLAGS` | | Additional arguments to be passed verbatim to the linker |
| `SRC_TOP` | | The source code top level directory (default `pwd -P`) |
| `BUILD_TOP` | | Build with artifacts in an alternate location (outside or in a subdirectory of the source tree) <br> e.g. `make BUILD_TOP=./build -j test`|
| `DEBUG` | | Compiler debugging flag to use (default `-ggdb`) |
| `OPTIM` | | The optimizer flag to use (default `-O3`) |
| `HOST` | | The target host tuple, for cross-compilation (default unset, i.e. native targeting) |
| `RUNTIME` | | The target runtime ecosystem -- default unset, `FreeRTOS-lwIP` and `Linux-lwIP` are recognized |
| `C_WARNFLAGS` | | The warning flags to use (overriding the generally applicable defaults) |
| `STATIC` | | Build statically linked unit tests |
| `STRIPPED` | | Strip binaries of debugging symbols |
| `BUILD_DYNAMIC` | | Build dynamically linked library |
| `VERY_QUIET` | | Inhibit all non-error output during build |
| `TAR` | | Path to GNU tar binary for `make dist`, should be set to `gtar` for macOS |
| `VERSION` | | The version to package for `make dist` |
| `LWIP` | `WOLFSENTRY_LWIP` | True/false -- Activates appropriate build settings for lwIP |
| `NO_STDIO` | `WOLFSENTRY_NO_STDIO` | Define to omit functionality that depends on `stdio` |
| `NO_STDIO_STREAMS` | `WOLFSENTRY_NO_STDIO_STREAMS` | Define to omit functionality that depends on `stdio` stream I/O |
| | `WOLFSENTRY_NO_STDIO_H` | Define to inhibit inclusion of `stdio.h` |
| `NO_ADDR_BITMASK_MATCHING` | `WOLFSENTRY_NO_ADDR_BITMASK_MATCHING` | Define to omit support for bitmask matching of addresses, i.e. support only prefix matching. |
| `NO_IPV6` | `WOLFSENTRY_NO_IPV6` | Define to omit support for the IPv6 address family. |
| `NO_JSON` | `WOLFSENTRY_NO_JSON` | Define to omit JSON configuration support |
| `NO_JSON_DOM` | `WOLFSENTRY_NO_JSON_DOM` | Define to omit JSON DOM API |
| `CALL_TRACE` | `WOLFSENTRY_DEBUG_CALL_TRACE` | Define to activate runtime call stack logging (profusely verbose) |
| `USER_SETTINGS_FILE` | `WOLFSENTRY_USER_SETTINGS_FILE` | A substitute settings file, replacing autogenerated `wolfsentry_settings.h` |
| `SINGLETHREADED` | `WOLFSENTRY_SINGLETHREADED` | Define to omit thread safety logic, and replace thread safety functions and macros with no-op macros. |
|| `WOLFSENTRY_NO_PROTOCOL_NAMES` | If defined, omit APIs for rendering error codes and source code files in human readable form. They will be rendered numerically. |
|| `WOLFSENTRY_NO_GETPROTOBY` | Define to disable lookup and rendering of protocols and services by name. |
|| `WOLFSENTRY_NO_ERROR_STRINGS` | If defined, omit APIs for rendering error codes and source code files in human readable form. They will be rendered numerically. |
|| `WOLFSENTRY_NO_MALLOC_BUILTINS` | If defined, omit built-in heap allocator primitives; the `wolfsentry_host_platform_interface` supplied to wolfSentry APIs must include implementations of all functions in `struct wolfsentry_allocator`. |
|| `WOLFSENTRY_HAVE_NONGNU_ATOMICS` | Define if gnu-style atomic intrinsics are not available. `WOLFSENTRY_ATOMIC_*()` macro definitions for intrinsics will need to be supplied in `WOLFSENTRY_USER_SETTINGS_FILE` (see `wolfsentry_util.h`). |
|| `WOLFSENTRY_NO_CLOCK_BUILTIN` | If defined, omit built-in time primitives; the `wolfsentry_host_platform_interface` supplied to wolfSentry APIs must include implementations of all functions in `struct wolfsentry_timecbs`. |
|| `WOLFSENTRY_USE_NONPOSIX_SEMAPHORES` | Define if POSIX semaphore API is not available. If no non-POSIX builtin implementation is present in `wolfsentry_util.c`, then the `wolfsentry_host_platform_interface` supplied to wolfSentry APIs must include a full semaphore implementation (shim set) in its `wolfsentry_semcbs` slot. |
|| `WOLFSENTRY_SEMAPHORE_INCLUDE` | Define to the path of a header file declaring a semaphore API. |
|| `WOLFSENTRY_USE_NONPOSIX_THREADS` | Define if POSIX thread API is not available. `WOLFSENTRY_THREAD_INCLUDE`, `WOLFSENTRY_THREAD_ID_T`, and `WOLFSENTRY_THREAD_GET_ID_HANDLER` will need to be defined. |
|| `WOLFSENTRY_THREAD_INCLUDE` | Define to the path of a header file declaring a threading API. |
|| `WOLFSENTRY_THREAD_ID_T` | Define to the appropriate type analogous to POSIX `pthread_t`. |
|| `WOLFSENTRY_THREAD_GET_ID_HANDLER` | Define to the name of a void function analogous to POSIX `pthread_self`, returning a value of type `WOLFSENTRY_THREAD_ID_T`. |
|| `FREERTOS` | Build for FreeRTOS |
| | `WOLFSENTRY_NO_PROTOCOL_NAMES` | If defined, omit APIs for rendering error codes and source code files in human readable form. They will be rendered numerically. |
| | `WOLFSENTRY_NO_GETPROTOBY` | Define to disable lookup and rendering of protocols and services by name. |
| | `WOLFSENTRY_NO_ERROR_STRINGS` | If defined, omit APIs for rendering error codes and source code files in human readable form. They will be rendered numerically. |
| | `WOLFSENTRY_NO_MALLOC_BUILTINS` | If defined, omit built-in heap allocator primitives; the `wolfsentry_host_platform_interface` supplied to wolfSentry APIs must include implementations of all functions in `struct wolfsentry_allocator`. |
| | `WOLFSENTRY_HAVE_NONGNU_ATOMICS` | Define if gnu-style atomic intrinsics are not available. `WOLFSENTRY_ATOMIC_*()` macro definitions for intrinsics will need to be supplied in `WOLFSENTRY_USER_SETTINGS_FILE` (see `wolfsentry_util.h`). |
| | `WOLFSENTRY_NO_CLOCK_BUILTIN` | If defined, omit built-in time primitives; the `wolfsentry_host_platform_interface` supplied to wolfSentry APIs must include implementations of all functions in `struct wolfsentry_timecbs`. |
| | `WOLFSENTRY_NO_SEM_BUILTIN` | If defined, omit built-in semaphore primitives; the `wolfsentry_host_platform_interface` supplied to wolfSentry APIs must include implementations of all functions in `struct wolfsentry_semcbs`. |
| | `WOLFSENTRY_USE_NONPOSIX_SEMAPHORES` | Define if POSIX semaphore API is not available. If no non-POSIX builtin implementation is present in `wolfsentry_util.c`, then #WOLFSENTRY_NO_SEM_BUILTIN must be set, and the `wolfsentry_host_platform_interface` supplied to wolfSentry APIs must include a full semaphore implementation (shim set) in its `wolfsentry_semcbs` slot. |
| | `WOLFSENTRY_SEMAPHORE_INCLUDE` | Define to the path of a header file declaring a semaphore API. |
| | `WOLFSENTRY_USE_NONPOSIX_THREADS` | Define if POSIX thread API is not available. `WOLFSENTRY_THREAD_INCLUDE`, `WOLFSENTRY_THREAD_ID_T`, and `WOLFSENTRY_THREAD_GET_ID_HANDLER` will need to be defined. |
| | `WOLFSENTRY_THREAD_INCLUDE` | Define to the path of a header file declaring a threading API. |
| | `WOLFSENTRY_THREAD_ID_T` | Define to the appropriate type analogous to POSIX `pthread_t`. |
| | `WOLFSENTRY_THREAD_GET_ID_HANDLER` | Define to the name of a void function analogous to POSIX `pthread_self`, returning a value of type `WOLFSENTRY_THREAD_ID_T`. |
| | `FREERTOS` | Build for FreeRTOS |

### Build and Self-Test Examples

Expand All @@ -158,8 +161,7 @@ Install from an alternate build location to a non-standard destination:

`make BUILD_TOP=./build INSTALL_DIR=/usr INSTALL_LIBDIR=/usr/lib64 install`

Build libwolfsentry.a and test it under various analyzers (memory and thread
testing under full battery of valgrind and sanitizer tests):
Build libwolfsentry.a and test it in various configurations:

`make -j check`

Expand All @@ -180,15 +182,15 @@ elaborate makefile code including additional rules and dependency mechanisms.)

Build the smallest simplest possible library:

`make -j SINGLETHREADED=1 NO_STDIO=1 DEBUG= OPTIM=-Os EXTRA_CFLAGS='-DWOLFSENTRY_NO_CLOCK_BUILTIN -DWOLFSENTRY_NO_MALLOC_BUILTIN -DWOLFSENTRY_NO_ERROR_STRINGS -Wno-error=inline -Wno-inline'`
`make -j SINGLETHREADED=1 NO_STDIO=1 DEBUG= OPTIM=-Os EXTRA_CFLAGS="-DWOLFSENTRY_NO_CLOCK_BUILTIN -DWOLFSENTRY_NO_MALLOC_BUILTIN -DWOLFSENTRY_NO_ERROR_STRINGS -Wno-error=inline -Wno-inline"`

Build and test with user settings:

`make -j USER_SETTINGS_FILE=user_settings.h test`

Build for FreeRTOS on ARM32, assuming FreeRTOS and lwIP source trees are located as shown:

`make -j HOST=arm-none-eabi RUNTIME=FreeRTOS-lwIP FREERTOS_TOP=../third/FreeRTOSv202212.00 LWIP_TOP=../third/lwip EXTRA_CFLAGS='-mcpu=cortex-m7'`
`make -j HOST=arm-none-eabi RUNTIME=FreeRTOS-lwIP FREERTOS_TOP=../third/FreeRTOSv202212.00 LWIP_TOP=../third/lwip EXTRA_CFLAGS=-mcpu=cortex-m7`


## Project Examples
Expand All @@ -211,6 +213,3 @@ build with wolfSentry integration, and use `--with-wolfsentry=/the/install/path`
if wolfSentry is installed in a nonstandard location. The wolfSSL test
client/server can be loaded with user-supplied wolfSentry JSON configurations
from the command line, using `--wolfsentry-config <file>`.



9 changes: 7 additions & 2 deletions wolfSentry/src/json_configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -168,12 +168,14 @@ allowed values are as in the ABNF formal syntax later in this document.
"interface" : uint8,
"address" : route_address,
"prefix-bits" : uint16,
"bitmask" : route_address,
"port" : endpoint_port
},
"local" : {
"interface" : uint8,
"address" : route_address,
"prefix-bits" : uint16,
"bitmask" : route_address,
"port" : endpoint_port
}
}
Expand Down Expand Up @@ -271,7 +273,8 @@ Each route is composed of the following elements, all of which are optional.
* <b>`remote`</b> -- The attributes to match for the remote endpoint of the traffic.
* <b>`interface`</b> -- Network interface ID, as an arbitrary integer chosen and used consistently by the caller or IP stack integration.
* <b>`address`</b> -- The network address, in idiomatic form. IPv4, IPv6, and MAC addresses shall enumerate all octets. See `route_address` definition in the ABNF grammar below for permissible values.
* <b>`prefix-bits`</b> -- The number of bits in the <b>`address`</b> that traffic must match.
* <b>`prefix-bits`</b> -- The number of bits in the <b>`address`</b> that traffic must match (mutually exclusive with <b>`bitmask`</b>).
* <b>`bitmask`</b> -- A bitmask to be applied to the traffic address before matching with the route <b>`address`</b> (mutually exclusive with <b>`prefix-bits`</b>).
* <b>`port`</b> -- The port number that traffic must match.

* <b>`local`</b> -- The attributes to match for the local endpoint of the traffic. The same nodes are available as for <b>`remote`</b>.
Expand Down Expand Up @@ -405,7 +408,7 @@ route_local_endpoint_clause = DQUOTE %s"local" DQUOTE ":" route_endpoint
route_endpoint = "{"
[ route_interface_clause "," ]
[ route_address_clause ","
[ route_address_prefix_bits_clause "," ]
[ (route_address_prefix_bits_clause / route_address_bitmask_clause) "," ]
]
[ route_port_clause "," ]
-","
Expand All @@ -415,6 +418,8 @@ route_interface_clause = DQUOTE %s"interface" DQUOTE ":" uint8

route_address_clause = DQUOTE %s"address" DQUOTE ":" route_address

route_address_bitmask_clause = DQUOTE %s"bitmask" DQUOTE ":" route_address

route_address = DQUOTE (route_address_ipv4 / route_address_ipv6 / route_address_mac / route_address_user) DQUOTE

route_address_ipv4 = uint8 3*3("." uint8)
Expand Down
Loading