A very important element of a Voice UI is the attention system. To start, you need to be able to tell people when the assistant is listening for commands, processing a command, and giving an answer. And it likely needs to be done for multimodal contexts: visual LEDs on a device, visual graphics on a screen, audio, and haptic. Also, you need to consider if your user is doing something important, like driving, where too much stimuli can distract from that critical task.
Another very important feature of a VUI's attention system is the schema for how interruptions are handled. For a general example, how do you prioritize notifications while Siri or Alexa is delivering or managing output while you're driving? Should a new text message stop playback of your Spotify song, and, if yes, should it immediately read the message or ask if you want to hear it read back? Is asking to hear the read-out too much of a distraction for a driver?
So I worked on interruptions for Merlyn's attention system, and created a mapping for how up to four modes (device LED, monitor display, a Merlyn avatar, and device audio) would behave for a number of different interruption scenarios. For a small example, should a paused video automatically resume when a teacher interrupts playback with a question for Merlyn? And are there situations where the answer is different for that same scenario?
The unique challenge with Merlyn was that there are situations where a teacher wants to give Merlyn commands without interrupting an output the system is currently managing. So the mappings and animations shown below were an essential tool used to help the product team make decisions about how Merlyn handles interruptions.