Skip to content

Commit

Permalink
UI3-226:
Browse files Browse the repository at this point in the history 
* Added MQTT topics `ui3/instance_id/playaudio` and `ui3/instance_id/playtts` so that UI3 can be remotely commanded to make noise.
* Extended the MQTT section of the help file to cover the new topics.
  • Loading branch information
@bp2008
bp2008 committed Nov 29, 2022
1 parent f99260a commit c1c2069
Show file tree
Hide file tree
Showing 3 changed files with 91 additions and 12 deletions.
2 changes: 1 addition & 1 deletion ui3.htm
View file
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@
};
</script>
<script type="text/javascript">
var ui_version = "225";
var ui_version = "226";
var bi_version = "%%VERSION%%";
var appPath_raw = "%%VIRTDIR%%";
var local_bi_session = "%%SESSION%%";
Expand Down
26 changes: 22 additions & 4 deletions ui3/help/help.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -564,11 +564,11 @@ <h1><img mysrc="img/MainMenuIcon.png" /> Main Menu</h1>
<h1>MQTT Remote Control</h1>
<p>Since UI3-216, MQTT can be configured as a communication channel to remotely control UI3 instances. MQTT is a lightweight publish/subscribe messaging system commonly used in home automation.</p>
<h3>Prerequisite</h3>
<p>To use MQTT in UI3, you must first have access to an MQTT broker with WebSocket support, such as <a href="https://mosquitto.org/">Eclipse Mosquitto&#8482;</a> software. Mosquitto is commonly installed as an addon to the popular Home Assistant OS running on a raspberry pi or a virtual machine or docker container, but the Mosquitto server can also be installed on Windows if desired. Installing and configuring an MQTT broker is outside the scope of this document.</p>
<p>To use MQTT in UI3, you must first have access to an MQTT broker with WebSocket support, such as <a href="https://mosquitto.org/">Eclipse Mosquitto&#8482;</a> software. Mosquitto is commonly installed as an addon to the popular Home Assistant OS running on a Raspberry Pi or a virtual machine or docker container, but the Mosquitto server can also be installed on Windows if desired. Installing and configuring an MQTT broker is outside the scope of this document.</p>
<h3>UI3 Configuration</h3>
<p>Go to UI3's Main Menu > UI Settings > MQTT Remote Control section and fill in the WebSocket URL, User Name, and Password fields. Adjust the Instance ID if you desire (it is randomly generated, but can be edited). For basic users, the MQTT Advanced options should be left at their default values. When ready, enable the MQTT Client via the toggle button labelled "Enable MQTT Client". Toast messages should quickly appear indicating success or failure of the connection to the MQTT broker.</p>
<p class="bg-info padded-info"><b>What are WebSocket URLs?</b><br /><br />WebSocket URLs are very similar to regular web URLs, except instead of beginning with <code>http://</code> or <code>https://</code>, they begin with <code>ws://</code> for unencrypted connections, or <code>wss://</code> for connections that will be secured with TLS.<br /><br />An example WebSocket URL is <code>ws://homeassistant.local:1884/</code> which indicates that the connection should use an unencrypted connection to the host named <code>homeassistant.local</code> on TCP port <code>1884</code>.</p>
<h3>Remote Functions</h3>
<h3>Remotely Control UI3's State</h3>
<p>You can use UI3's MQTT Remote Control capabilities to change the current live group or camera, maximize or unmaximize the video player, and change the volume of audio streaming.</p>
<div class="panel panel-info">
<div class="panel-heading">List of MQTT topics used by UI3</div>
Expand All @@ -586,6 +586,24 @@ <h3>Remote Functions</h3>
<p>Note the <code>instance_id</code> text in the MQTT topic list above. <code>instance_id</code> is placeholder text which must be replaced by the Instance ID as configured in UI Settings. It is this Instance ID which enables you to individually target specific UI3 instances when multiple UI3 instances are connected to the same MQTT broker. The Instance ID is randomly generated by each web browser you run UI3 in, but it can be modified manually if desired. It is saved in the browser's Local Storage along with all other UI settings.</p>
<p>UI3 subscribes to these topics with its own Instance ID, and reacts when other MQTT clients publish messages to them.</p>
<p>UI3 also publishes messages to these topics when the local state changes due to user-interaction, so that other MQTT clients can subscribe to them to learn the current state of the UI3 instance. This behavior coincidentally allows a UI3 instance to remotely control another UI3 instance if they are both connected to the same MQTT broker, and both have the same Instance ID assigned.</p>
<h3>Audio and Text-to-Speech</h3>
<p>Enable <b>"Audio/TTS Events"</b> within the advanced MQTT configuration, and you'll be able to command UI3 to play audio files and use text-to-speech to speak arbitrary text strings. When this option is enabled, UI3 will test if audio playback is allowed upon UI startup, and if the browser refuses, UI3 will require user input to finish loading. Some web browsers allow you to explicitly allow audio without user input on certain web sites, and this is recommended if you will be using UI3's audio features.</p>
<div class="panel panel-info">
<div class="panel-heading">Audio/TTS Topics</div>
<div class="panel-body">
<dl class="dl-vertical">
<dt>ui3/instance_id/playaudio</dt>
<dd>Publish an audio file name to this topic, and UI3 will play the sound. For example, publish <code>doorbell.mp3</code> to <code>ui3/instance_id/playaudio</code>. If a previous audio file is still playing, it will be immediately stopped and the new audio file will play. Sound volume is controlled by UI Settings > Event-Triggered Sounds > Sound Effect Volume.</dd>
<dt>ui3/instance_id/playaudio/{player_id}</dt>
<dd>This alternate topic lets you provide an arbitrary Player ID string for the purpose of allowing multiple sounds to be played concurrently. More about that below.*</dd>
<dt>ui3/instance_id/playtts</dt>
<dd>Publish some text to this topic, and the web browser's text-to-speech engine will speak the text aloud. The voice can be selected via <b></b>UI Settings > UI Status Sounds > Speech Voice.</dd>
</dl>
</div><!--/.panel-body -->
</div><!--/.panel -->
<p>Note the <code>instance_id</code> text in the MQTT topic list above. <code>instance_id</code> is placeholder text which must be replaced by the Instance ID as configured in UI Settings.</p>
<p>For these commands, if you use the text "<code>global</code>" as the Instance ID, then all connected UI3 instances will receive the message. For example, publish <code>doorbell.mp3</code> to <code>ui3/global/playaudio</code>.</p>
<p><b>*{player_id}</b>. Normally, UI3 will only allow one audio file to be played at the same time via MQTT message. To work around this, you can optionally provide any unique string in place of <code>{player_id}</code> to cause UI3 to use a different player object identified by that string. For example, you could publish <code>airhorn.mp3</code> to <code>ui3/global/playaudio/A</code> and publish <code>doorbell.mp3</code> to <code>ui3/global/playaudio/B</code> and both audio files would play concurrently.</p>
<h3>Online Status</h3>
<p>MQTT clients can track whether an Instance ID has any connected UI3 instances by looking at children of the <code>ui3/instance_id/online</code> topic as shown in the following screenshot from <a href="http://mqtt-explorer.com/">MQTT Explorer</a>:</p>
<p>
Expand All @@ -597,11 +615,11 @@ <h3>Tips</h3>
<p>If you have trouble understanding how MQTT works, I suggest using <a href="http://mqtt-explorer.com/">MQTT Explorer</a> to view the activity on your MQTT broker in real-time. This will enable you to easily see what UI3 is doing with MQTT topics. The tool can also publish messages so you can use it to test UI3's remote control capability.</p>
<p>The "All cameras" group is not actually named <code>All cameras</code>. This group is internally named <code>Index</code> and must be referred to as such for remote control purposes.</p>
<p>The <code>vid</code> topic is only used for live viewing at this time, but may be extended in the future to support timeline playback or clips/alerts.</p>
<p>It is recommended when publishing MQTT messages to UI3, that you always use the "retain" flag. This provides the most consistent and predictable behavior.</p>
<p>It is recommended when publishing state changes to UI3, that you always use the "retain" flag. This provides the most consistent and predictable behavior. However when publishing <code>playaudio</code> or <code>playtts</code> messages, do NOT use the "retain" flag unless you want the audio to play every time UI3 loads.</p>
<h3>Advanced Configuration</h3>
<p>UI3 offers several advanced options for its MQTT client.</p>
<p>By default, UI3 performs two-way communication with the MQTT broker, both publishing its own state changes and subscribing to receive state changes from the broker. You can individually toggle the publish and subscribe functions in order to have the remote control work only one-way.</p>
<p>MQTT supports 3 different "QOS" values (0, 1, 2) for subscribing and for publishing. Non-zero QOS values attempt to offer additional guarantees of message delivery, but for most users, QOS 0 is sufficient and it is the fastest option. Advanced users may choose different QOS values individually for publishing and subscribing.</p>
<p>MQTT supports 3 different "QOS" values (0, 1, 2) for subscribing and for publishing. Non-zero QOS values attempt to offer additional guarantees of message delivery, but for most users, QOS 0 is sufficient and it is the fastest option. Advanced users may choose different QOS values individually for publishing and subscribing. QOS 0 ensures that messages are delivered at most one time. QOS 1 ensures that messages are delivered at least one time. QOS 2 ensures that messages are delivered exactly one time.</p>
<p>By default, UI3 publishes its state messages with the "Retain" flag set in order to help persist the UI state between page loads. If desired, you can turn this off.</p>
</div><!--/.tab-pane -->
<div class="tab-pane" id="export" role="tabpanel" aria-labelledby="nav-export">
Expand Down
75 changes: 68 additions & 7 deletions ui3/ui3.js
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2638,8 +2638,8 @@ var defaultSettings =
key: "ui3_mqttSubscribeEnabled"
, value: "1"
, inputType: "checkbox"
, label: 'Subscribe to /state/ Topics'
, hint: "Disabling this will disable inbound state synchronization with the MQTT broker."
, label: 'Subscribe to Topics'
, hint: "Disabling this will disable inbound state synchronization with the MQTT broker. MQTT-triggered audio commands will not work."
, category: "MQTT Remote Control"
}
, {
Expand All @@ -2649,7 +2649,7 @@ var defaultSettings =
, minValue: 0
, maxValue: 2
, label: 'Subscribe QOS'
, hint: "The MQTT QOS value (0,1,2) that is sent when subscribing to the /state/ topics."
, hint: "The MQTT QOS value (0,1,2) that is sent when subscribing to topics."
, category: "MQTT Remote Control"
}
, {
Expand Down Expand Up @@ -2678,6 +2678,14 @@ var defaultSettings =
, hint: "If enabled, UI3 will set the Retain flag when publishing to the /state/ topics."
, category: "MQTT Remote Control"
}
, {
key: "ui3_mqttAudioEvents"
, value: "0"
, inputType: "checkbox"
, label: 'Audio/TTS Events'
, hint: "If enabled, UI3 will subscribe to playaudio and playtts topics and may require user interaction to finish loading UI3."
, category: "MQTT Remote Control"
}
, {
key: "ui3_fullscreen_videoonly"
, value: "1"
Expand Down Expand Up @@ -19461,6 +19469,16 @@ var biSoundPlayer = new (function ()
var file = settings.getItem("ui3_sound_" + event);
player.Play(file, settings.ui3_eventSoundVolume);
}
this.PlaySoundFile = function (audioPlayerId, file)
{
if (developerMode)
console.log('PlaySoundFile("' + audioPlayerId + '", "' + file + '")');
var player = playerCache["custom_" + audioPlayerId];
if (!player)
player = playerCache["custom_" + audioPlayerId] = new BISoundEffect();

player.Play(file, settings.ui3_eventSoundVolume);
}
this.AdjustVolume = function ()
{
for (var event in playerCache)
Expand All @@ -19473,7 +19491,8 @@ var biSoundPlayer = new (function ()
{
var eventSoundPlayerEnabled = settings.ui3_eventSoundVolume > 0 && (settings.ui3_sound_motion !== "None" || settings.ui3_sound_trigger !== "None" || settings.ui3_sound_alert !== "None");
var videoStatusSoundsEnabled = settings.ui3_uiStatusSounds === "1" || settings.ui3_uiStatusSpeech === "1";
if (eventSoundPlayerEnabled || videoStatusSoundsEnabled)
var mqttSoundsEnabled = settings.ui3_mqttClientEnabled === "1" && settings.ui3_mqttAudioEvents === "1";
if (eventSoundPlayerEnabled || videoStatusSoundsEnabled || mqttSoundsEnabled)
{
try
{
Expand All @@ -19490,10 +19509,16 @@ var biSoundPlayer = new (function ()
reason += "event-triggered sound player";
if (videoStatusSoundsEnabled)
{
if (eventSoundPlayerEnabled)
if (reason)
reason += " and ";
reason += "video status sound player";
}
if (mqttSoundsEnabled)
{
if (reason)
reason += " and ";
reason += "MQTT-triggered sound player";
}
inputRequiredOverlay.Show(reason);
}
});
Expand Down Expand Up @@ -31708,7 +31733,8 @@ function MqttClient()
return;
try
{
toaster.Success("UI3 is ready to be remotely controlled.", 3000);
//if (settings.ui3_mqttStatusToasts === "1")
// toaster.Success("UI3 is ready to be remotely controlled.", 3000);
}
catch (ex)
{
Expand All @@ -31729,7 +31755,8 @@ function MqttClient()
}
}
};
client.subscribe("ui3/" + instance_id + "/state/#", subscribeOptions);
client.subscribe("ui3/" + instance_id + "/#", subscribeOptions);
client.subscribe("ui3/global/#", subscribeOptions);
}

function onMessageArrived(message)
Expand Down Expand Up @@ -31777,6 +31804,20 @@ function MqttClient()
statusBars.setProgress("volume", newVolume, "");
}
}
else if (parts[0] === "ui3" && (parts[1] === instance_id || parts[1] === "global") && parts[2] === "playaudio")
{
handlePlayAudio(parts[3], message.payloadString);
}
}
else if (parts.length === 3)
{
if (parts[0] === "ui3" && (parts[1] === instance_id || parts[1] === "global"))
{
if (parts[2] === "playtts")
handlePlayTTS(message.payloadString);
else if (parts[2] === "playaudio")
handlePlayAudio(null, message.payloadString);
}
}
}
catch (ex)
Expand Down Expand Up @@ -31868,6 +31909,26 @@ function MqttClient()
}
}

function handlePlayAudio(audioPlayerId, audioFileName)
{
if (settings.ui3_mqttAudioEvents !== "1")
{
console.log("MQTT Audio Events are not enabled. Ignoring request:", audioFileName);
return;
}
if (audioFileName.indexOf('..') > -1 || audioFileName.indexOf(':') > -1)
{
console.log("Rejected audio file name:", audioFileName);
return;
}
biSoundPlayer.PlaySoundFile(audioPlayerId, audioFileName);
}

function handlePlayTTS(message)
{
programmaticSoundPlayer.Speak(message, true);
}

connect();
}
function MqttTopicState()
Expand Down

0 comments on commit c1c2069

Please sign in to comment.