Import WearML-Engine

The latest version of the WearML engine can be downloaded from the RealWear HTML GitHub Repository, or by installing via NPM.

npm install wearml

After downloading, include the javascript file to your web page. The script will only execute on Android based devices with a screen size of 854×480.

Defining Speech Commands

Once you have added the javascript file to your web page you can start adding speech commands to your HTML elements. There are two methods for associating a voice command with its intended application functionality:

Adding DOM Attributes to the HTML

For simple controls like input fields or buttons that already have predefined onClick functionality, the easiest way to add a voice command is by setting the ‘data-wml-speech-command’ attribute to the desired phrase.

<input data-wml-speech-command="Enter Username" class="form-control" type="text" placeholder="Username"/>

Adding a Callback Command

Sometimes there may need to be a voice command that is not associated with any particular UI element, in which case you can add a Callback Command to run arbitrary JS code when the voice command is recognized.

wearML.addCallbackCommand("Refresh", null, function(){ 

//Refresh the UI... 

 }); 

..... 

var refreshFunc = function() { 

//Refresh the UI... 

 }; 

var commandSets = 'main|loginpage'; 

wearML.addCallbackCommand("Refresh", commandSets, refreshFunc);

You can also remove a Callback Command individually:

wearML.removeCallbackCommand("Refresh")

In addition, the voiceCommandsCallback method is called every time a command is successfully recognized. You can use this to selectively perform additional actions upon successful command recognition.

Updating Voice Commands

In order to update voice commands the wearML engine will use a MutationObserver to detect the adding and removing of html dom elements. If you are finding that speech-commands and overlays are not correctly updating when changing your html content you can call the public method:

wearML.getCommands()

This will force the WearMLEngine to reload grammar and overlays.

Using CommandSets

WearHF interprets the entire DOM tree at once to build its list of commands, which sometimes can lead to a mismatch between the commands WearHF detects and the commands the user expects to be able to interact with. For example, if a menu drawer is overlaid over the main UI pane, the user expects to only be able to interact with the menu items, not the the UI elements underneath, which are obscured to the user but are still present on the DOM. This issue can be circumvented by associating one or more ‘Command Sets’ with each command and programatically switching between them.

In the DOM

A CommandSet can be defined for a DOM element by setting the DOM attribute ‘data-wml-commandsets’ to a string value referencing one or more CommandSets, separated by a pipe character.

<a href="#" data-wml-speech-command="Open Menu" data-wml-style=".wmlMenuBtn" data-wml-commandsets="main|favorites" 
 onclick="wearML.setCommandSet('menu')" class="link icon-only open-panel"><i class="fa fa-bars"></i></a>

This element definition will tell WearHF to register the command ‘Open Menu’ when the active CommandSet is set to ‘main’ or ‘favorites’.

With a Callback Command

The CommandSets for a Callback Command is defined using the second argument of the wearML.addCallbackCommand function.

wearML.addCallbackCommand("Open Menu", "main|favorites", function(){ menu.open(); });

Switching the Active CommandSet

The wearML object contains the variable wearML.commandSet, which contains a single String defining the active CommandSet. This variable defaults to null, in which case all commands detected by WearHF will be registered upon detection, regardless of their CommandSet. This variable can be manipulated using the wearML.setCommandSet() function at any time, after which it will change the active CommandSet and subsequently refresh its list of commands.

<script type="text/javascript"> 
     document.addEventListener("DOMContentLoaded", function(event) { 

          wearML.setCommandSet('main'); //Set Active CommandSet to 'main' as soon as the document loads 
          wearML.addCallbackCommand("Refresh", "main|favorites", refreshFunc); 
     }); 
 </script>

WearML overlays

WearML Overlays are the hints provided by WearHF to help the user with navigating the current screen. WearML overlay’s have many attributes and options to choose from in order to provided helpful hints to the user. There are many features contained in the wearML overlays and the API can be found below for reference.

HTML WearML Overlay

In order to customize the wearML overlay first create a CSS style and add custom css attributes using “–” and then appended the WearML Style attribute you wish to use.

<style> 
     .nativeSpeechRightAligned{ 
     --overlay_show_number:false; 
     --overlay_show_dot:true; 
     --overlay_persists:true; 
     --overlay_anchor_hv:"110,50"; 
     } 
</style>

Add the style to your dom element:

<input data-wml-style=".nativeSpeechRightAligned" data-wml-speech-command="Enter Username" 
 class="form-control" type="text" placeholder="Username"/

Updating the ‘Show Help’ Menu

For commands that have no obvious visual indication for what to say, the best practice is to update the ‘Show Help’ menu with hints so the user can figure out what to do.

WearHF will automatically add any command to the ‘Show Help’ Menu if the CSS attribute ‘–include_help’ is a part of its style.

... 
<style> 
          .wmlSearch{ 
                --overlay_show_number:false; 
                --overlay_show_text:true; 
                --overlay_persists:true; 
                --include_help:true; 

     }
</style> 
<body> 
     <a href="#" data-wml-speech-command="Search" data-wml-commandsets="menu" data-wml-style=".wmlSearch" class="search-link item-content list-panel-all close-panel item-link"> 
</body>
...

Callback Commands are automatically added to the Show Help menu upon being registered.

API Definitions

DOM Attributes

Attribute Data Type Description
data-wml-speech-command String text/content_description/no/xxxx Optional: Defines the source for the speech command. text will take the text attribute from the component. content_description will use that attribute from the component. no will turn the voice command off on the component all together. xxxx You are also able to provide a custom voice command here, e.g. xxxx
data-wml-style String Optional: References to a CSS style using the classname.
data-wml-commandsets String Defines which CommandSets this command will be available to. Can be one or more string values, separated by a pipe character ( | ).

CSS Attributes

Attribute Data Type Description Notes
root Boolean All elements below this will inherit the attributes provided to this dom tag unless otherwise specified.
overlay_show_text Boolean Optional: Turns a text label on or off. Text on the label will be taken from the speech_command that is set. (default = no)
overlay_persists Boolean Optional: Number and/or overlay won’t fade away. (default = no, fades away)
overlay_orientation String left,right,top,bottom Optional: Text overlay direction (default = right)
overlay_background_color String Optional: Changes the background color of the element String is represented as HEX
overlay_text_color String Optional: Changes the background color of the element String is represented as HEX
overlay_border_color String Optional: Changes the background color of the element String is represented as HEX
overlay_anchor_hv String as “H,V” Optional: Sets the anchor point horizontally and vertically, specified as a percentage. 0 means anchor to left,top edge of element. 100 means anchor to right,bottom edge of element. 50 means anchor to middle of element.
overlay_show_dot Boolean Optional: Turns purple dot icon on or off for the element. Off by default
overlay_show_icon Boolean Optional: Turns microphone icon on or off for the element. On by default if there is a text overlay
hf_scroll String “None” = switches off headtracker

“Horizontal” = Headtracker only works horizontally

“Vertical” = Headtracker only works vertically

text_field String For text field elements, this will indicate what keyboard should be opened.

“dictation” = keyboard will open in Dictation mode 

“barcode” = keyboard will open in ‘Scan Code’ mode

“keyboard” = keyboard will open in default text entry mode

barcode String any, qr , code128, up, cean Optional: Will define which type of barcode is being scanned. Ignored if the text_field isn’t set to barcde. (default = any)
global_commands Boolean Optional: Disables all global commands, doesn’t matter which component it is applied to.
include_help Boolean Optional: If true, the command associated with this element will be automatically added to the ‘Show Help’ Menu when detected by WearHF and part of the active CommandSet.
hide_help Boolean Optional: Hides the Show Help Menu, doesn’t matter which component it is applied to.