Merge pull request #142 from pixxelhq/master

v5.10.3-Pixxel.SNAPSHOT-3.0.0

Changelog

@@ -1,3 +1,13 @@
+Version 5.10.3 released 7-Oct-2024
+- The alarm work done in 5.10.2 enabled by mistake the EventAlarmServer which used to be disabled by default. That causes Yamcs not to start in many configurations which used to work (including the Yamcs quickstart) due to missing the event_alarms_realtime. 
+  This version restores back the old behaviour - namely the EventAlarmServer is disabled by default. 
+
+Version 5.10.2 released 6-Oct-2024
+- Improvement in command stacks: they are called simply "stacks" and allow interleaving commands with comments and steps allowing the operator to check the live value of defined parameters.
+- Added event filtering.
+- Alarm reloading - when Yamcs restarts it will reload the non-cleared alarms from the database such that they are active again.
+- Smaller tweaks and fixes.
+
 Version 5.10.1 released 1-Sept-2024
 - Activities: Execute known script files with (configurable) external program.
 - Small tweaks and fixes related to Parameter Archive and the web plugin.

README.md

@@ -23,7 +23,7 @@ For commercial licensing please contact [Space Applications Services](https://ww
 
 ## Development Setup
 
-To work on the core components of Yamcs you need JDK11, Maven and npm.
+To work on the core components of Yamcs you need JDK17+, Maven and npm.
 
 Build Java jars:
 

distribution/pom.xml

@@ -3,7 +3,7 @@
   <parent>
     <groupId>org.yamcs</groupId>
     <artifactId>yamcs</artifactId>
-    <version>5.10.1-Pixxel.SNAPSHOT-3.0.0</version>
+    <version>5.10.3-Pixxel.SNAPSHOT-3.0.0</version>
   </parent>
 
   <artifactId>distribution</artifactId>

docs/http-api/.gitignore

@@ -1,4 +1,4 @@
+/_build/
 .autogen
-*.rst
-!/overview.rst
-!/websocket.rst
+/*/**/*.rst
+/index.rst

docs/http-api/conf.py

@@ -61,4 +61,9 @@
 )
 yamcs_api_destdir = "."
 yamcs_api_title = "Yamcs HTTP API"
-yamcs_api_additional_docs = ["overview.rst", "websocket.rst"]
+yamcs_api_additional_docs = [
+    "overview.rst",
+    "filtering.rst",
+    "partial-responses.rst",
+    "websocket.rst",
+]

docs/http-api/filtering.rst

@@ -0,0 +1,263 @@
+Filtering
+=========
+
+Some list methods provide a ``filter`` option. This option can be use to provide a query expression to filter based on the fields of each list item. Methods providing this option allow to use ``POST`` in addition to ``GET``, to avoid encoding of lengthy queries in the query parameter.
+
+The filter syntax allows for two kinds of search: text search, and field search.
+
+
+Text Search
+-----------
+
+A single word is matched against the full resource. It is up to the specific resource implementation to determine which fields are considered for this comparison, usually all textual fields. The search is case-insensitive, exact, and may be partial.
+
+For example, search resources that match the text `wombat`:
+
+.. code-block:: text
+   
+   wombat
+
+To find resources that match both the text `icy` and the text `wombat` (at the same time), provide them both separated by whitespace:
+
+.. code-block:: text
+   :caption: Separated by space
+
+   icy wombat
+
+.. code-block:: text
+   :caption: Separated by newline
+
+   icy
+   wombat
+
+Search terms may be enclosed in double quotes, which allows the search to include special characters.
+
+The previous example is identical to:
+
+.. code-block:: text
+
+   "icy"
+   "wombat"
+
+If you would rather search for the exact sequence `icy wombat`, use double quotes around the full search term:
+
+.. code-block:: text
+
+   "icy wombat"
+
+To search resources that `do not` match the text `wombat`, negate the term by prefixing with the minus sign:
+
+.. code-block:: text
+
+   -wombat
+
+
+Logical Operators
+^^^^^^^^^^^^^^^^^
+
+Logical operators ``AND``, ``OR`` and ``NOT`` can be used to form more complicated queries. These operators must be specified in uppercase, else they are considered to be search terms.
+
+For example:
+
+.. code-block:: text
+
+   wombat OR hippo
+
+.. code-block:: text
+
+   NOT hippo OR (icy AND wombat)
+
+NOT has highest precedence, followed by OR, then AND. Where needed, use parenthesis to avoid any confusion.
+
+Use of the `AND` operator is optional, as this is the default behavior when multiple terms are provided.
+
+
+Line Comments
+^^^^^^^^^^^^^
+
+Queries can span any number of lines. Lines can be commented out using the `--` prefix.
+
+The following example searches for resources that textually match with both `wombat` and `gorilla`.
+
+.. code-block:: text
+
+   wombat
+   --hippo
+   gorilla
+
+
+Field Search
+------------
+
+Each filterable resource defines a number of fields that can be used to filter directly. This allows for better targeting than with text search, and will generally perform better.
+
+The available filterable fields vary from one resource to another, and are documented on their respective pages (for example, see: :doc:`events/list-events`).
+
+Field search requires a comparison query of the form: `FIELD OPERATOR VALUE`. For example, to filter resources that have the field `foo` set to `wombat`, use any of the following:
+
+.. code-block:: text
+
+   foo=wombat
+   foo = wombat
+   foo = "wombat"
+
+Each field has a specified type: string, number, boolean, binary or enum. The following sections describe the operators for each of these types.
+
+
+String Field Comparison Operators
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The following operators can be used in string field comparisons.
+
+.. list-table::
+   :widths: 25 25 50
+
+   * - =
+     - foo = "wombat"
+     - The field `foo` equals `wombat`
+   * - !=
+     - foo != "wombat"
+     - The field `foo` does not equal `wombat`
+   * - <
+     - foo < "wombat"
+     - The field `foo` is alphabetically before `wombat`
+   * - <=
+     - foo <= "wombat"
+     - The field `foo` equals `wombat`, or is alphabetically before `wombat`
+   * - >
+     - foo > "wombat"
+     - The field `foo` is alphabetically after `wombat`
+   * - >=
+     - foo >= "wombat"
+     - The field `foo` equals `wombat`, or is alphabetically after `wombat`
+   * - :
+     - foo:"wombat"
+     - The field `foo` contains the substring `wombat`
+   * - =~
+     - foo =~ "bat$"
+     - The field `foo` ends with the substring `bat`
+   * - !~
+     - foo !~ "bat$"
+     - The field `foo` does not end with the substring `bat`
+
+The operators `=~` and `!~` allow to match the field against the provided regular expression. The match is unanchored, so use the prefix `^` and the suffix `$` when you want to match the full field value.
+
+Regular expressions are case-sensitive. To enable case-insensitive matching, you can use an embedded flag expression:
+
+.. code-block::
+
+   foo =~ "(?i)bat$"
+
+Regular expressions must be double-quoted. For the other operators, double quotes are optional, unless you want to match special characters.
+
+
+Number Field Comparison Operators
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The following operators can be used in number field comparisons:
+
+.. list-table::
+   :widths: 25 25 50
+
+   * - =
+     - foo = 123.45
+     - The field `foo` equals `123.45`
+   * - !=
+     - foo != 123.45
+     - The field `foo` does not equal `123.45`
+   * - <
+     - foo < 123.45
+     - The field `foo` is smaller than `123.45`
+   * - <=
+     - foo <= 123.45
+     - The field `foo` equals `123.45`, or is smaller than `123.45`
+   * - >
+     - foo > 123.45
+     - The field `foo` is greater than `123.45`
+   * - >=
+     - foo >= 123.45
+     - The field `foo` equals `123.45`, or is greater than `123.45`
+
+The comparison value may be double-quoted.
+
+
+Boolean Field Comparison Operators
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The following operators can be used in boolean field comparisons:
+
+.. list-table::
+   :widths: 25 25 50
+
+   * - =
+     - foo = true
+     - The field `foo` is `true`
+   * - !=
+     - foo != true
+     - The field `foo` is not `true` (so, null or false)
+
+The comparison values `true` and `false` are case-insensitive, and may be double-quoted.
+
+
+Binary Field Comparison Operators
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The following operators can be used in binary field comparisons:
+
+.. list-table::
+   :widths: 25 25 50
+
+   * - =
+     - foo = aabb
+     - The field `foo` is two bytes long, `0xAA` and `0xBB`
+   * - !=
+     - foo != aabb
+     - The field `foo` does not match `0xAABB`
+   * - :
+     - foo:aabb
+     - The field `foo` contains the binary `0xAABB`
+
+The provided hexstring is case-insensitive, and may be double-quoted.
+
+
+Enum Field Comparison Operators
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Assume a field `foo` of the following enum type:
+
+.. code-block:: java
+
+   enum Severity {
+     INFO,
+     WATCH,
+     WARNING,
+     DISTRESS,
+     CRITICAL,
+     SEVERE;
+   }
+
+The following operators can be used in enum field comparisons.
+
+.. list-table::
+   :widths: 25 25 50
+
+   * - =
+     - foo = INFO
+     - The field `foo` equals `INFO`
+   * - !=
+     - foo != INFO
+     - The field `foo` does not equal `INFO`
+   * - <
+     - foo < WATCH
+     - The field `foo` is before `WATCH`, using enum order
+   * - <=
+     - foo <= WATCH
+     - The field `foo` is `WATCH`, or before `WATCH`, using enum order
+   * - >
+     - foo > WATCH
+     - The field `foo` is after `WATCH`, using enum order
+   * - >=
+     - foo >= WATCH
+     - The field `foo` is `WATCH`, or after `WATCH`, using enum order
+
+The provided enum constant is case-insensitive, and may be double-quoted.

docs/http-api/overview.rst

@@ -54,27 +54,6 @@ Cross-origin Resource Sharing (CORS) allows access to the Yamcs HTTP API from a
 CORS is off by default on Yamcs Server, but available through configuration.
 
 
-.. rubric:: Response Filtering
-
-Responses can be filtered using the query parameter ``fields``, or alternatively by setting the HTTP header ``X-Yamcs-Fields``. This is usually not needed, because when unspecified all fields are returned. Some clients may anyway want to be explicit, for optimizing the message sizes.
-
-Field names are applicable to the top-level response message, and multiple fields can be separated by commas. Methods that return a list of messages apply the mask to each of the listed resources. Field paths can be of arbitrary depth separated by dots. Only the last part can refer to a repeated field.
-
-Some examples:
-
-Return information on the `simulator` instance, but include only the ``name`` and ``state`` fields:
-
-.. code-block::
-
-    curl 'localhost:8090/api/instances/simulator?fields=name,state'
-
-Return a list of all instances, but include only the ``name`` and ``state`` fields:
-
-.. code-block::
-
-    curl 'localhost:8090/api/instances?fields=name,state'
-
-
 .. rubric:: JSON
 
 All API methods are designed for JSON-over-HTTP. Matching type definitions in this documentation are written in TypeScript syntax because of its high readability. Note however that we do not currently mark parameters as optional (``?``).

docs/http-api/partial-responses.rst

@@ -0,0 +1,20 @@
+Partial Responses
+=================
+
+To reduce the size of an HTTP response message, it is possible to restrict returned fields by providing the query parameter ``fields``, or alternatively by setting the HTTP header ``X-Yamcs-Fields``. This is also called a `field mask`.
+
+Field names are applicable to the top-level response message, and multiple fields can be separated by commas. Methods that return a list of messages apply the mask to each of the listed resources. Field paths can be of arbitrary depth separated by dots. Only the last part can refer to a repeated field.
+
+Some examples:
+
+Return information on the `simulator` instance, but include only the ``name`` and ``state`` fields:
+
+.. code-block::
+
+    curl 'localhost:8090/api/instances/simulator?fields=name,state'
+
+Return a list of all instances, but include only the ``name`` and ``state`` fields:
+
+.. code-block::
+
+    curl 'localhost:8090/api/instances?fields=name,state'

docs/http-api/websocket.rst

@@ -192,7 +192,7 @@ To confirm your request, Yamcs will first send you a reply that looks somewhat l
 As the client, we note that the server has assigned the call identifier ``3`` to this subscription.
 
 .. note::
-    The property ``@type`` is an artifact generated by Yamcs JSON backend. It specifies the equivalent Protobuf message type of the ``data`` object (Yamcs generates JSON based on Protobuf definitions). You should be able to ignore this property because we enforce each message ``type`` to be using only a single ``data`` message.
+    The property ``@type`` is an artifact generated by Yamcs JSON backend. It specifies the equivalent Protobuf message type of the ``data`` object (Yamcs generates JSON based on Protobuf definitions). You may ignore this property because each message ``type`` uses only a single ``data`` message.
 
 
 Next we receive continued time updates, each in a WebSocket frame: