Update const generation scripts based on zwave-js 13

.github/workflows/check-generated-modules.yml

@@ -11,16 +11,27 @@ jobs:
       matrix:
         python-version:
           - "3.12"
+        node-version:
+          - "20"
 
     steps:
       - uses: actions/[email protected]
         with:
           fetch-depth: 2
+      - name: Set up Node.js ${{ matrix.node-version }}
+        uses: actions/setup-node@v4
+        with:
+          node-version: ${{ matrix.node-version }}
+          cache: 'npm'
+      - name: Generate registry JSON files
+        run: |
+          npm install
+          npm run generate
       - name: Set up Python ${{ matrix.python-version }}
         uses: actions/[email protected]
         with:
           python-version: ${{ matrix.python-version }}
-      - name: Install dependencies
+      - name: Install Python scripts dependencies
         run: |
           python -m pip install --upgrade pip
           pip install -r requirements_scripts.txt

.gitignore

@@ -1,3 +1,9 @@
+node_modules
+*.lock
+package-lock.json
+notifications.json
+sensors.json
+
 # Hide sublime text stuff
 *.sublime-project
 *.sublime-workspace

package.json

@@ -0,0 +1,13 @@
+{
+  "scripts": {
+    "generate": "node scripts/generate_zwave_js_registry_json.js"
+  },
+  "type": "module",
+  "devDependencies": {
+    "@tsconfig/node20": "^20.1.4",
+    "@types/node": "^22.7.4",
+    "ts-node": "^10.9.2",
+    "typescript": "^5.3.3",
+    "zwave-js": "^13.4.0"
+  }
+}

requirements_scripts.txt

@@ -1,7 +1,5 @@
 colorlog==6.8.2
-requests==2.32.3
 python-slugify==8.0.4
 types-python-slugify==8.0.2.20240310
-types-requests==2.32.0.20240914
 -r requirements.txt
 -r requirements_lint.txt

scripts/.gitignore

@@ -0,0 +1,6 @@
+node_modules
+*.lock
+package-lock.json
+*.json
+!package.json
+!tsconfig.json

scripts/README.md

@@ -2,19 +2,25 @@
 
 This directory contains scripts that are used to help maintain this library.
 
+## Python scripts
+
 To run these scripts, you will need to install `zwave-js-server-python` library's [requirements_scripts.txt](../../requirements_scripts.txt) into your environment.
 
 These scripts have to be run manually, and any changes that result from running the scripts have to be submitted as a PR to be included in the project.
 
-## `generate_multilevel_sensor_constants.py`
+### `generate_multilevel_sensor_constants.py`
+
+This script is used to download the latest multilevel sensor types and scales registries from the [zwave-js](https://github.com/zwave-js/zwave-js-server) repository and generate constants for the multilevel sensor command class. The generated constants can be found [here](../../zwave_js_server/const/command_class/multilevel_sensor.py).
 
-This script is used to download the latest multilevel sensor types and scales JSON files from the [zwave-js](https://github.com/zwave-js/zwave-js-server) repository and generate constants for the multilevel sensor command class. The generated constants can be found [here](../../zwave_js_server/const/command_class/multilevel_sensor.py).
+### `generate_notification_constants.py`
 
-## `run_mock_server.py`
+This script is used to download the latest notification registry from the [zwave-js](https://github.com/zwave-js/zwave-js-server) repository and generate constants for the notification command class. The generated constants can be found [here](../../zwave_js_server/const/command_class/notification.py).
+
+### `run_mock_server.py`
 
 This script allows you to run a mock Z-Wave JS Server instance using a network state dump from the `zwave-js-server-python` library. While the functionality is limited for now and is intended to be expanded in the future, the mock server also supports manipulating the state of the network by replaying events on it and emulating a responsive network by setting up mocked responses to commands. The main purpose of this mock server is to allow developers to test, build, and troubleshoot applications that use the `zwave-js-server-python` library to integrate with [zwave-js](https://github.com/zwave-js/node-zwave-js) (e.g. Home Assistant).
 
-### Usage
+#### Usage
 
 At a minimum, the mock server instance needs the file path to a network state dump (which can be retrieved from an existing network using the library's [dump](../../zwave_js_server/dump.py) module). All other inputs are optional.
 
@@ -45,13 +51,13 @@ optional arguments:
                         received.
 ```
 
-### Inputs/File Formats
+#### Inputs/File Formats
 
-#### Network State Dump (required)
+##### Network State Dump (required)
 
 The network state dump tells the server what the initial state of the network should be for the driver, the controller, and all of the network's nodes. The output of the library's [dump](../../zwave_js_server/dump.py) module can be used directly as the input to the server.
 
-##### File Format
+###### File Format
 
 ```json
 [
@@ -90,13 +96,13 @@ The network state dump tells the server what the initial state of the network sh
 ]
 ```
 
-#### Events to Replay (optional)
+##### Events to Replay (optional)
 
 `zwave-js` events can be replayed on the server once a client has connected to the server instance and started listening to emulate things happening on the network. These events can be provided initially at runtime via a JSON file, but the server also has a [`/replay` endpoint](#replay-endpoint) that can be POSTed to in order to add events to the replay queue.
 
 Command results can be recorded from a live network using the library's Client class (see the [Recording section](#recording-events-and-commandscommand-responses) for more details)
 
-##### Limitations
+###### Limitations
 
 - The queue currently only gets played immediately after a new client starts listening to the server
 - The events will fire sequentially without any delays between the events
@@ -105,7 +111,7 @@ Command results can be recorded from a live network using the library's Client c
 - There is currently no way to control the timing of the events
 - There is currently no way to reorder events in the queue
 
-##### File Format
+###### File Format
 
 ```json
 [
@@ -120,20 +126,20 @@ Command results can be recorded from a live network using the library's Client c
 ]
 ```
 
-#### Command Results (optional)
+##### Command Results (optional)
 
 The server can respond to commands from a queue of command responses. After each command, the first response for that command is removed from the queue and sent back to the client. In this way, you can control the exact behavior of what the server would send the client, even if there are multiple calls for the same command. These command results can be provided initially at runtime via a JSON file, but the server also has a [`/replay` endpoint](#replay-endpoint) that can be POSTed to in order to add command results to the queue.
 
 Command results can be recorded from a live network using the library's Client class (see the [Recording section](#recording-events-and-commandscommand-responses) for more details)
 
-##### Limitations
+###### Limitations
 
 - Unlike events, which remain in the queue forever, command results are only returned once. To add to the queue, use the `/replay` endpoint
 - There is currently no way to clear a queue for a particular command
 - There is currently no way to clear all command responses
 - There is currently no way to reorder command results in the queue
 
-##### File Format
+###### File Format
 
 ```json
 [
@@ -152,20 +158,28 @@ Command results can be recorded from a live network using the library's Client c
 ]
 ```
 
-### Recording events and commands/command responses
+#### Recording events and commands/command responses
 
 The library's Client class can record event and command/command result messages that occur between a server instance and a client. This would allow you to e.g. troubleshoot a user's problem by having them record everything that happens, reproduce the issue, and then send you the result to feed directly into the mock server.
 
-#### Begin recording
+##### Begin recording
 
 There are two ways to enable recording of commands and messages:
 1. When creating the Client class instance, pass `record_messages=True` into the Client constructor and the class instance will begin recording all events, commands, and results of commands after the client starts listening to updates from the server.
 2. Call `Client.begin_recording_messages()` at any point to begin recording all events, commands, and results of commands.
 
-#### End recording
+##### End recording
 
 You can end recording by calling `Client.end_recording_messages()`. This call will return a list which can be directly passed into the `--combined-replay-dump-path` option once serialized to a JSON file.
 
-### `/replay` endpoint
+#### `/replay` endpoint
 
 The `replay` endpoint accepts an HTTP POST request with either a single event or command/command response or a list of them. They will be added to the end of their respective queue.
+
+## Typescript scripts
+
+To run these scripts, go to the `scripts` folder and run `yarn install`.
+
+### `serialize_zwave_js_registries.ts`
+
+This script imports helper functions from Z-Wave JS to retrieve the Z-Wave JS registries used in the Python scripts above.

scripts/const.py

@@ -18,3 +18,5 @@
     "# ----------------------------------------------------------------------------------- #",
     "",
 ]
+GITHUB_PROJECT = "zwave-js/node-zwave-js"
+BRANCH_NAME = "master"

scripts/generate_multilevel_sensor_constants.py

@@ -1,27 +1,22 @@
 #!/usr/bin/env python3
 """Script to generate Multilevel Sensor CC constants."""
+
 from __future__ import annotations
 
 from collections import defaultdict
 from collections.abc import Callable, Mapping
-import json
 import pathlib
 
 from const import AUTO_GEN_POST, AUTO_GEN_PRE
 from helpers import (
     enum_name_format,
     format_for_class_name,
+    get_json,
     get_manually_written_code,
-    normalize_name,
-    remove_comments,
+    get_registry_location,
     run_black,
+    separate_camel_case,
 )
-import requests
-
-GITHUB_PROJECT = "zwave-js/node-zwave-js"
-BRANCH_NAME = "master"
-SENSOR_TYPES_FILE_PATH = "packages/config/config/sensorTypes.json"
-DEFAULT_SCALES_FILE_PATH = "packages/config/config/scales.json"
 
 CONST_FILE_PATH = (
     pathlib.Path(__file__).parent.parent
@@ -33,57 +28,28 @@ def normalize_scale_definition(scale_definitions: dict[str, dict]) -> dict[str,
     """Convert a scales definition dictionary into a normalized dictionary."""
     scale_def_ = {}
     for scale_id, scale_props in scale_definitions.items():
-        _scale_id = int(scale_id, 16)
         scale_name_ = enum_name_format(scale_props["label"], True)
-        scale_def_[scale_name_] = _scale_id
+        scale_def_[scale_name_] = scale_id
 
     return dict(sorted(scale_def_.items(), key=lambda kv: kv[0]))
 
 
-sensor_types = json.loads(
-    remove_comments(
-        requests.get(
-            (
-                f"https://raw.githubusercontent.com/{GITHUB_PROJECT}/{BRANCH_NAME}/"
-                f"{SENSOR_TYPES_FILE_PATH}"
-            ),
-            timeout=10,
-        ).text
-    )
-)
-default_scales = json.loads(
-    remove_comments(
-        requests.get(
-            (
-                f"https://raw.githubusercontent.com/{GITHUB_PROJECT}/{BRANCH_NAME}/"
-                f"{DEFAULT_SCALES_FILE_PATH}"
-            ),
-            timeout=10,
-        ).text
-    )
-)
-
-scales = {
-    normalize_name(scale_name): normalize_scale_definition(scale_def)
-    for scale_name, scale_def in default_scales.items()
-}
-
+scales = {}
 sensors = {}
-for sensor_id, sensor_props in sensor_types.items():
-    sensor_id = int(sensor_id, 16)
+
+for sensor_props in get_json("sensors.json"):
+    sensor_id = sensor_props["key"]
     scale_def = sensor_props["scales"]
     remove_parenthesis_ = True
     if sensor_id in (87, 88):
         remove_parenthesis_ = False
     sensor_name = enum_name_format(sensor_props["label"], remove_parenthesis_)
     sensors[sensor_name] = {"id": sensor_id, "label": sensor_props["label"]}
-    if isinstance(scale_def, str):
-        sensors[sensor_name]["scale"] = normalize_name(
-            scale_def.replace("$SCALES:", "")
-        )
-    else:
-        scales[sensor_name] = normalize_scale_definition(scale_def)
-        sensors[sensor_name]["scale"] = normalize_name(sensor_name)
+    scale_name = separate_camel_case(sensor_props.get("scaleGroupName", ""))
+    if not (scale_name := enum_name_format(scale_name, remove_parenthesis_)):
+        scale_name = sensor_name
+    scales.setdefault(scale_name, {}).update(normalize_scale_definition(scale_def))
+    sensors[sensor_name]["scale"] = scale_name
 
 scales = dict(sorted(scales.items(), key=lambda kv: kv[0]))
 sensors = dict(sorted(sensors.items(), key=lambda kv: kv[0]))
@@ -123,9 +89,7 @@ def generate_int_enum_base_class(class_name: str, docstring: str) -> list[str]:
     return class_def
 
 
-SENSOR_TYPE_URL = (
-    f"https://github.com/{GITHUB_PROJECT}/blob/{BRANCH_NAME}/{SENSOR_TYPES_FILE_PATH}"
-)
+SENSOR_TYPE_URL = get_registry_location("SensorTypes.ts")
 
 lines = [
     '"""Constants for the Multilevel Sensor CC."""',