[WIP] [UI] Add voice dialog

[GiviMAD]

This is a WIP PR for the issue #2275

The PR uses these audio WebSocket which is still in review openhab/openhab-core#4032 so you need to use the org.openhab.core.io.websocket and org.openhab.core.voice bundles from there if you want to test it.

It is built on top of the webpack 5 upgrade PR #2267, because on webpack 4 you need to use loaders and I didn't want to generate more work to do in a future upgrade. The actual PR changes are in the last commit.

Changes added:

I have added that panel at the end of the "help & about" page (it should only be displayed if the necessary APIs are available), it contains the following options:

• "Enable Voice Dialog": When enabled the dialog execution will start after the first user event on the page (needed to start the microphone stream).
• "Connection ID": And auto generated code that is used to register the audio components. Ex: the id of the registered sink will look like "pcm::ui-52-68::sink".
• "Selected Keyword": Rustpotter config, choose rustpotter model file. At the moment the file is selected using an input type file and persisted using an IndexedDB instance.
• "Keyword Threshold": Rustpotter config, threshold in range 0 - 1.
• "Keyword Min Detections": Rustpotter config, min number of positive scores to emit the detection.
• "Listening Item": Dialog configuration, Item to toggle while listening the command.
• "Location Item": Dialog configuration, used by the Standard language processor to improve the Item selection.

Modifying any of the options triggers a page reload.

The dialog will use the voice services configured at "Settings / Voice". Once the dialog is running it can be triggered using the voice if the rustpotter model was provided, or using the listenAndAnswer action. For example from the oh console by running voice listenandanswer --source pcm::ui-52-68::source --sink pcm::ui-52-68::sink.

For now I have placed some console warning in the events emitted about the dialog state.

I wanted to open the PR in case you can give me some feedback about the integration of the feature or in case someone wants to dig into the implementation.

Best regards.

[relativeci]

Job #1392: Bundle Size — 11.86MiB (+7.89%).

355c5fc(current) vs 1c03c60 main#1391(baseline)

Warning

Bundle contains 19 duplicate packages – View duplicate packages

Warning

Bundle introduced 4 new packages: reentrant-lock, rustpotter-worklet, rustpotter-web-slim and one more – View changed packages

Bundle metrics   8 changes 4 regressions

	Current Job #1392	Baseline Job #1391
Initial JS	1.84MiB(+0.11%)	1.84MiB
Initial CSS	607.73KiB	607.73KiB
Cache Invalidation	22.37%	95.87%
Chunks	223(+1.36%)	220
Assets	250(+3.31%)	242
Modules	3098(+0.78%)	3074
Duplicate Modules	162(+1.89%)	159
Duplicate Code	1.82%(+4.6%)	1.74%
Packages	154(+2.67%)	150
Duplicate Packages	18	18

Bundle size by type   3 changes 3 regressions

	Current Job #1392	Baseline Job #1391
JS	9.33MiB (+1.54%)	9.19MiB
CSS	888.93KiB (~+0.01%)	888.86KiB
Other	744.53KiB (+87431.8%)	871B
Fonts	526.1KiB	526.1KiB
Media	295.6KiB	295.6KiB
IMG	140.74KiB	140.74KiB
HTML	1.24KiB	1.24KiB

---

View job #1392 report View GiviMAD:feature/voice_dialog branch activity View project dashboard