fix(container): update public.ecr.aws/emqx/emqx ( 5.8.0 → 5.8.3 )

[robottoms-up]

This PR contains the following updates:

Package	Update	Change
public.ecr.aws/emqx/emqx (source)	patch	5.8.0 -> 5.8.3

---

Release Notes

emqx/emqx (public.ecr.aws/emqx/emqx)

v5.8.3: EMQX v5.8.3

Compare Source

v5.8.3

Make sure to check the breaking changes and known issues before upgrading to EMQX 5.8.3.

Enhancements

Core MQTT Functionalities

• #​14219 Enhanced Connection Rate Limiter for Improved System Resilience.

  • Improved system stability and responsiveness under high connection rates: Previously, when the connection rate limit was exceeded, listener acceptors would ignore new connection attempts, potentially resulting in an unrecoverable state if a large number of clients connected or reconnected frequently within a short period. Listeners now accept pending connections but immediately close them if the rate limit is reached. This reduces resource strain and improves system resilience during peak loads.

  • New listener option nolinger introduced: When set to true, a TCP-RST is sent immediately upon socket closure, helping to mitigate SYN flood attacks and further enhancing connection-handling efficiency.

  • max_connection configuration for MQTT listeners now capped by system limits: The max_connection value for MQTT listeners is now constrained by the system's limits (e.g., ulimit from the OS and node.process_limit). If configured to infinity or a value greater than the system limit, it will automatically be adjusted to match the system's maximum limit.

  • SSL listeners' ssl_options now validated before changes: Previously, invalid SSL options (such as unsupported TLS versions) could be accepted, causing client connection failures after a listener reconfiguration. With this update:

    • The node will fail to boot if a listener is configured with invalid SSL options.
    • Requests to apply invalid SSL options via the Dashboard or config API will now fail with a 400 status code.

Configuration

• #​14195 Added support for client ID override.

  EMQX now provides greater flexibility by allowing custom client ID overrides using the mqtt.clientid_override={Expression} configuration. This introduces a more dynamic approach to client ID management. As part of this update, the use_userid_as_clientid and peer_cert_as_clientid options are deprecated, though they will remain available for compatibility until version 6.0.

MQTT over QUIC

• #​14283 Improved QUIC transport, upgrade quicer to 0.1.9.
  • Early release of remote stream resources in the abnormal scenarios.
  • Added more troubleshooting APIs. For more details, see: emqx/quic@0.1.6...0.1.9.

Bug Fixes

Core MQTT Functionalities

• #​14201 Prevent check_gc warning from appearing when a WebSocket connection encounters a rate limit.
• #​14215 Fixed an issue where calls to the retainer (via REST or CLI) would throw an exception if it was disabled.
• #​14223 Ensured the WebSocket close reason is returned as an atom to avoid crashes, specifically preventing the error: error: {{case_clause,#{invalid_property_code => 51}},[{cowboy_websocket...}}.
• #​14260 Resolved a rare race condition that could cause the connection process to crash if the CONNECT packet was not fully received before the idle timeout (default 15 seconds) expired.
• #​14268 Fixed another rare race condition that could cause the WebSocket connection process to crash when the CONNECT packet was not fully received before the idle timeout expired.
• #​14266 Updated emqtt from version 1.13.0 to 1.13.5. For more details, please refer to the emqtt changelog.

Durable Sessions

• #​14160 Ensured that topic matching rules for durable session subscriptions are properly applied to topics starting with the $ symbol, in accordance with the MQTT specification.

REST API

• #​14117 Fixed an issue in the REST API documentation where the Users endpoint was incorrectly listed as supporting Basic Authentication.

Data Integration

• #​14172 Resolved a potential race condition where testing a connector using the HTTP API could leave lingering resources if the HTTP request timed out.

• #​14178 Fixed an issue where configuration synchronization could become stuck on a particular node due to simultaneous deletion of rules across different nodes in the cluster.

• #​14226 Mitigated a scenario where, under high load, a node could lose track of resource metrics (e.g., action/source) and fail to recover without a restart. Now, when restarting a resource or resetting its metrics, the system attempts to recreate the lost metrics.

  Additionally, warning logs related to metric failures, such as those for "hot-path" metrics like matched, are now throttled to prevent excessive log flooding. Example of throttled log:

  2024-11-14T13:56:44.134289+00:00 [warning] tag: RESOURCE, clientid: clientid, msg: handle_resource_metrics_failed, peername: 172.100.239.1:33896, reason: {badkey,matched}, stacktrace: [{erlang,map_get,[matched,#{}],[{error_info,#{module => erl_erts_errors}}]},{emqx_metrics_worker,idx_metric,4,[{file,"src/emqx_metrics_worker.erl"},{line,560}]},...
  
  2024-11-14T13:57:12.490503+00:00 [warning] msg: log_events_throttled_during_last_period, period: 1 minutes, 0 seconds, dropped: #{handle_resource_metrics_failed => 2294}
  

• #​14265 Fixed an issue where a badkey error would occur when stopping a connector if the MQTT Source action failed to subscribe successfully.

• #​14296 Prevented ecpool_sup from being blocked by a slow-starting ecpool_worker.

Configuration

• #​14180 Fixed an issue with variform expressions returning 'undefined' when a variable is bound to the value undefined or null. Now, an empty string is returned instead.

• #​14289 Resolved a log file path issue when importing configurations from a different environment. The EMQX_LOG_DIR environment variable is set to /opt/emqx/log in Docker but /var/log/emqx/ when installed via RPM/DEB packages. Prior to this fix, log file paths (default file handler and audit handler) are environment-variable interpolated when being exported. This could cause crashes when importing configs into a different environment where the directory didn’t exist.

  With this fix, log file paths are no longer environment-variable interpolated during export. Additionally, absolute log directory paths from older versions are now converted back to environment variables if the path doesn’t exist in the new environment.

Extension

• #​14243 Fixed an issue where the client.connect hook was not being triggered for some gateways.

MQTT over QUIC

• #​14258 Reduced the QUIC connection shutdown timeout. Previously, QUIC connections had a 5-second timeout for graceful shutdown. If the client was unresponsive, EMQX would log warnings like:

  [warning] msg: session_stepdown_request_timeout, action: discard,
  

  or potentially cause a timeout on the Dashboard when attempting to disconnect the client. The timeout has now been reduced to 1 second for "kick" actions and 3 seconds for other scenarios.

Breaking Changes

• #​14305 Removed support of md4, md5 and ripemd160 in authentication as they are not compliant with NIST Secure Hash Standard.

v5.8.2: EMQX v5.8.2

Compare Source

5.8.2

Release Date: 2024-11-12

Enhancements

Core MQTT Functionalities

• #​14059 Added a new configuration option for the retainer to cap message expiry intervals for retained messages. This enables garbage collection to remove messages sooner if storage is running low.

• #​14072 Updated the virtual machine to use Unicode for its printable range. This improvement enhances the readability of certain binary data in messages. For instance, a binary previously displayed as <<116,101,115,116,228,184,173,230,150,135>> will now be formatted as <<"test中文"/utf8>>, providing clearer representation.

MQTT Durable Sessions

• #​14130 Reduced CPU usage for idle durable sessions.

  Previously, idle durable sessions periodically woke up to refresh the list of DS streams. With this change, stream discovery is now event-based, significantly lowering CPU consumption during idle periods. Additionally, the update reduces the delay in notifying sessions of new streams, effectively eliminating the long-tail latency in end-to-end processing.

REST API

• #​13889 Enhanced the performance of the /api/v5/monitor_current and /api/v5/metrics APIs.

  Previously, these APIs queried clustered nodes sequentially in a loop. Now, the queries are sent in parallel, reducing response time. The latency is now primarily dependent on the slowest node in the cluster.

  Additionally, a node parameter was added to the /api/v5/monitor_current API, allowing targeted queries to a single node instead of the entire cluster. For instance, using ?aggregate=false&[email protected] will return data exclusively for the specified node.

EMQX Clustering

• #​13903 Added logs to inform the user when a replicant node cannot find a core node with the same release version as its own.

Security

• #​13923 Added zone support in authentication, authorization, and mountpoint templates.

  Previously, to reference a client's zone in authentication or authorization rules, users needed to access it through client_attrs. Now, the ${zone} placeholder can be used directly in these templates, simplifying rule creation and enabling zone-specific configurations.

  For example, the following ACL rule uses ${zone} to dynamically apply permissions based on a client’s assigned zone: {allow, all, all, ["${zone}/${username}/#"]}.

• #​14102 Added support for SSL private key passphrase from a secret file.

  EMQX can now read the passphrase from a secret file if password is configured as ...ssl_options.password = "file://{path-to-secret-file}".

Data Integration

• #​14065 Added a new queuing_bytes metric for data integration. This metric shows the RAM and/or disk resources consumed by buffering for a specific action.

Observability

• #​14096 Exposed emqx_conf_sync_txid as a Prometheus metric, allowing for monitoring the configuration file synchronization status of each node in the cluster.

MQTT over QUIC

• #​13814 Connection Scope Keepalive for MQTT over QUIC Multi-Stream:

  Introduced a new feature to keep MQTT connections alive when data streams remain active, even if the control stream is idle.

  Previously, clients were required to send MQTT.PINGREQ on idle control streams to keep the connection alive. Now, a shared state tracks activity across all streams for each connection. This shared state is used to determine if the connection is still alive, reducing the risk of keepalive timeouts due to Head-of-Line (HOL) blocking.

• #​14112 Added support ssl_options.hibernate_after in QUIC listener to reduce memory footprint of QUIC transport.

Bug Fixes

Core MQTT Functionality

• #​13931 Updated the gen_rpc library to version 3.4.1, which includes a fix to prevent client socket initialization errors from escalating to the node level on the server side.

• #​13969 Optimized the periodic cleanup of expired retained messages to ensure efficient resource usage, particularly in cases with a large volume of expired messages.

• #​14068 Added the handle_frame_error/2 callback to all gateway implementation modules to handle message parsing errors.

• #​14037 Improved the internal database bootstrap process to better tolerate temporary unavailability of peer nodes, particularly when a new node joins an existing cluster.

• #​14116 Fixed an issue where the default configuration for the retainer was generated incorrectly after joining a cluster.

MQTT Durable Sessions

• #​14042 Fix crash in the durable session after updates to subscription parameters (such as QoS, no_local, upgrade_qos, ...).

• #​14052 Corrected memory usage reporting from cgroups when in use.

• #​14055 Updated the /clients_v2 API to properly respect all filtering arguments when querying offline clients with durable sessions. Previously, only the username filter was applied, while other filtering arguments were ignored.

• #​14151 Fixed handling of the conn_state filter in the /clients_v2 API for offline clients with durable sessions. Previously, these clients could be incorrectly selected with conn_state=connected.

• #​14057 Resolved a compatibility issue that prevented the Messages DS database from starting due to a slightly different database configuration schema. This issue occurred when upgrading EMQX from version 5.7.x with session durability enabled.

REST API

• #​14023 Fixed an issue with the GET /monitor HTTP API where returned values could appear higher than actual values, depending on the requested time window. For data points within a 1-hour window, this distortion is only visual on the Dashboard. However, for data points older than 1 hour, the data distortion is permanent.

  The affected metrics include:

  • disconnected_durable_sessions
  • subscriptions_durable
  • subscriptions
  • topics
  • connections
  • live_connections

EMQX Clustering

• #​13996 Fixed an intermittent crash occurring when using emqx conf fix to resolve configuration discrepancies, particularly if a configuration key was missing on one of the nodes.

Security

• #​13922 Updated the CRL (Certificate Revocation List) cache to use the full Distribution Point (DP) URL as the cache key. Previously, only the path part of the URL was used, causing conflicts when multiple DPs shared the same path.

• #​13924 Fixed an issue where JWK keys could leak into debug logs upon JWT authentication failure.

Data Integration

• #​13916 Fixed an issue where the parent metric failed was not incremented when a rule’s failed.no_result or failed.exception metrics were updated.

• #​14001 Resolved a race condition where a resource (such as a connector, action, source, authentication, or authorization) could falsely report a connected, healthy channel after a brief disconnection. This issue could result in excessive action_not_found log entries when the race condition occurred.

• #​13913 Fixed an issue with the actions and source HTTP APIs where a 500 status code would be returned if a timeout occurred while attempting to update or delete a resource.

• #​14101 Resolved an issue where deleting a resource would fail if a source and an action were both created with the same name.

Observability

• #​13909 Fixed log formatting for cases where the payload cannot be displayed as readable UTF-8 Unicode characters.

• #​14061 Improved log information when emqx_cm:request_stepdown/3 fails.

  In scenarios where a client channel needs to terminate another channel with the same ClientID, a race condition may occur if the target channel has already been closed or terminated. In such cases, error logs and stack traces that provide no useful information will no longer be generated.

• #​14070 Removed the connector's state from error and warning logs due to its potential length. For issue analysis, the connector's state can now be accessed through emqx_resource:list_instances_verbose/0. Below is an example of a log entry before this change:

  pid: <0.43914.0>, connector: connector:sqlserver:connector-05a2e105, reason: [Microsoft][ODBC Driver 17 for SQL Server][SQL Server]Argument data type varchar is invalid for argument 2 of dateadd function. SQLSTATE IS: 42000, state: {"resource_opts":{"start_timeout":5000,"start_after_created":true,"health_check_interval":15000},"pool_name":"connector:sqlserver:connector-05a2e105","installed_channels":{"action:sqlserver:action-4b033621:connector:sqlserver:connector-05a2e105":{"sql_templates":{"batch_insert_temp":{"send_message":{"batch_insert_tks":["{str,<<\" ( \">>}","{var,[<<\"messageId\">>]}","{str,<<\", \">>}","{var,[<<\"measurement\">>]}","{str,<<\", \">>}","{var,[<<\"Analog_IN_Fault_1\">>]}","{str,<<\", \">>}","{var,[<<\"Analog_IN_Fault_2\">>]}","{str,<<\", \">>}","{var,[<<\"Analog_IN_Fault_3\">>]}","{str,<<\", \">>}","{var,[<<\"Analog_IN_Fault_4\">>]}","{str,<<\", \">>}","{var,[<<\"Analog_IN_PV_1\">>]}","{str,<<\", \">>}","{var,[<<\"Analog_IN_PV_2\">>]}","{str,<<\", \">>}","{var,[<<\"Analog_IN_PV_3\">>]}","{str,<<\", \">>}","{var,[<<\"Analog_IN_PV_4\">>]}","{str,<<\", DATEADD(MS, \">>}","{var,[<<\"ms_shift\">>]}","{str,<<\", DATEADD(S, \">>}","{var,[<<\"s_shift\">>]}","{str,<<\", '19700101 00:00:00:000') ))\">>}"],"batch_insert_part":"insert into TransactionLog(MessageId, Measurement, Fault1, Fault2, Fault3, Fault4, Value1, Value2, Value3, Value4, DateStamp) \r\n"}}}}}},msg: invalid_request
  

• #​14099 Removed an error-level log entry that was triggered when validation of UTF-8 strings in MQTT messages failed.

  Example of the removed log entry:

  {"time":"2024-10-11T06:05:07.610048+00:00","level":"error","msg":"supervisor: {esockd_connection_sup,0.53591191.0}, errorContext: connection_shutdown, reason: #{cause => invalid_topic,reason => malformed_utf8_string_length}, offender: [{pid,0.53591191.0},...]", ..., "error_logger":{"type":"supervisor_report","tag":"error_report"}}
  

• #​14091 Implemented a fix to remove function_clause from log messages when users provide unsupported write syntax.

  Example of unsupported syntax:

  weather,location=us-midwest,season=summer temperature=82 ${timestamp}u 

  Before this fix, the error log would contain the function_clause error, as shown:

  pid: <0.558392.0>, info: {"stacktrace":["{emqx_bridge_influxdb_connector,parse_timestamp,[[1719350482910000000,<<\"u\">>]],[{file,\"emqx_bridge_influxdb_connector.erl\"},{line,692}]}", ...], ..., "error":"{error,function_clause}"}, tag: ERROR, msg: resource_exception
  

  This change improves log clarity by omitting function_clause in cases of syntax errors.

v5.8.1: EMQX v5.8.1

Compare Source

5.8.1

Release Date: 2024-10-14

Make sure to check the breaking changes and known issues before upgrading to EMQX 5.8.1.

Important Changes

• #​13956 Updated the gen_rpc library to version 3.4.1, which includes a node crash issue.
  Previously, if a node is force shutdown down while RPC channels are being established, it may cause a cluster peer node to crash.

Enhancements

Core MQTT Functionalities

• #​13525 Added new configuration item shared_subscription_initial_sticky_pick to specify the strategy for making the initial pick when shared_subscription_strategy is set to sticky.

• #​13942 The HTTP client now automatically reconnects if no activity is detected for 10 seconds after the latest request has expired.
  Previously, it would wait indefinitely for a server response, causing timeouts if the server dropped requests.

  This change impacts below components.

  • HTTP authentication
  • HTTP authorization
  • Webhook (HTTP connector)

Authentication and Authorization

• #​13863 EMQX now supports ${cert_common_name} placeholder in topic name templates for raw ACL rules.

• #​13792 The banned-clients API GET /banned supports querying the rules using filters in the query string.

  The available filters are:

  • clientid
  • username
  • peerhost
  • like_clientid
  • like_username
  • like_peerhost
  • like_peerhost_net

  When adding a new banned client entry, the default expiration time for entries without the until parameter specified has been changed from 1 year to infinite.

Rule Engine

• #​13773 Disabled rule actions now do not trigger out_of_service warnings.

  Previously, if an action is disabled, there would be a warning log with msg: out_of_service,
  and the actions.failed counter was incremented for the rule.

  After this enhancement, disabled action will result in a debug level log with msg: discarded,
  and the newly introduced counter actions.discarded will be incremented.

MQTT over QUIC

• #​13814 Connection Scope Keepalive for MQTT over QUIC Multi-Stream:

  This update introduces a new feature to maintain MQTT connections over QUIC multi-streams, even when the control stream is idle but other data streams are active.

  Previously, clients had to send MQTT.PINGREQ on idle control streams to keep the connection alive. Now, a shared state is maintained for each connection, monitoring activity across all streams. This shared state helps determine if the connection is still active, reducing the risk of keepalive timeouts caused by Head-of-Line (HOL) blocking and improving overall connection stability.

Bug Fixes

Core MQTT Functions

• #​13702 Clean up the corresponding exclusive subscriptions when a node goes down.

• #​13708 Fixed an issue which may cause shared subscription 'sticky' strategy to degrade to 'random'.

• #​13733 Made cacertfile optional when configuring https listener from emqx ctl conf load command.

• #​13742 Fixed when subscribing with + as the first level, or # as a wildcard, retained messages with topics starting with $ are incorrectly received.

• #​13754 Fixed an issue when websocket connection would break consistently on its own.

• #​13756 Introduced more randomness to broker assigned client IDs.

• #​13790 The default heartbeat interval for the MQTT connector has been reduced from 300 seconds to 160 seconds.

  This change helps maintain the underlying TCP connection by preventing timeouts due to the idle limits
  imposed by load balancers or firewalls, which typically range from 3 to 5 minutes depending on the cloud provider.

• #​13832 Fixed that the Publish endpoint would have a 500 error when persistent session were enabled.

• #​13842 Fixed a UTF-8 string validation exception.

Upgrade and Migration

• #​13731 Resolved an issue that prevented clusters running on EMQX 5.4.0 from upgrading to EMQX 5.8.0. This fix introduces a migration procedure to update specific internal database tables created in version 5.4.0 to align with the new schema.

Breaking Changes

• #​13792 The default expiration time for a banned item that is created without an until value is now infinity (previsouly capped at 1 year limit).

• #​13742 Fixed an issue when a client would receive retained messages for a topic starting with $ when it subscribed to topic # or +.

  This fix satisfies the requirement of MQTT-4.7.2-1.

---

Configuration

📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about these updates again.

---

• If you want to rebase/retry this PR, check this box

---

This PR has been generated by Renovate Bot.