Sonos Home Theater
5 years after launching the number one selling Playbar, Sonos has evolved the home theater again with a powerful yet affordable voice enabled soundbar.
Project Overview
As the Lead and sole UX designer on the Beam project I worked with PM, ID, Dev and Engineering to deliver an exceptional user experience in not only the home theater but in the hub of our home, the living room.
I spent over a year researching, developing, designing, testing and launching the Sonos Beam.
Hardware UX
Voice control in the home theater
Modular speaker configuration
HDMI, CEC and ARC connectivity
Touch controls
Wall and surface mounted scenarios
3.0, 3.1, 5.0 and 5.1 home theater configurations
Research
Development and oversight of all research
Moderation of hallway and in office user testing
Oversight of Beta testing
50 unboxing hallway tests
25 in home setup and configuration tests
1000+ Beta users
Software UX
App based setup and configuration
Voice Partner Setup
Visual and audio interfaces
On screen display
Settings and speaker control
Upgrade to HDMI
great technology becomes a great experience
One of the many advancements Beam brought to the Sonos’ home theater offering was HDMI connectivity. Before we could start designing interfaces we had to figure out how the technology was going to work. I spent my first few months at Sonos researching HDMI protocol, HDMI standards and how different TVs interacted with HDMI. HDMI is a superior a/v/connectivity technology that brought many advantages beyond Optical which was Sonos’ current solution.
Optical (old)
One way communication
Audio transfer only
Compressed surround audio
HDMI (new)
Two way communication
Audio, Video, Control and ID data transfer
Uncompressed surround audio
HDMI Immersion
Because Beam needed to work across all modern television brands I spent 2 days with a product Manager and a Product Engineer in a windowless room surrounded by televisions in a facility in Oregon to understand exactly how HDMI worked across devices.
Although HDMI has many standards and specifications it is implemented quite differently across devices. To fully understand HDMI and Beam would interact with different TVs I had to immerse myself in HDMI and understand how connectivity and interfaces worked across Brands and Models of Televisions.
Over 2 days I tested and documented over 50 televisions from over 10 brands.
Understanding leads to Innovation
Once we understood what HDMI was and how it behaved we could really get into making magic for our users.
While HDMI was being used to deliver sound to the user our UX team was much more concerned with the connectivity and control advantages. We identified three primary areas of innovation for our users Setup and Voice Interaction. The following will dig into each of these innovations.
Setup
The internet of things is amazing because it just works or at least thats what we want our users to think. To make things “just work” it take an immense amount of design and testing.
The problem
To begin I worked with a team of engineers and designers to understand how the setup flow worked. I began by documenting everything that needed to happen so we could figure out how to get to what we wanted to happen. In order to better communicate with those involved in the company we blew out every step of the setup flow. After confirming that this was in fact the current state, we rolled up our sleeves and found areas where we could remove steps and leverage HDMI to make the User’s experience as seamless as possible.
Sonos is an excellent system and one of the foundational products from the IoT sector but being such a complex system there are many steps to setting up a single speaker, even if everything goes according to plan. While it may not be the sexiest problem it is one that every IoT user faces. The connectivity solutions that HDMI offers meant that there was potential to improve setup experience of Sonos’ home theater offerings. It was our goal to make setup as easy as possible for our users.
setup solutions
Our HDMI research allowed us to create Instructions that we presented to our users based on the Model of their TV which could be read from the HDMI signal. Most TVs can be configured in a single menu while some need the user to access two menus. While this could be complicated our system told the user which menus to access, which option to select and could then check whether or not the connection was successful. We developed a flow that gave users exactly as much instruction as they needed.
Working with what we’ve got
Single Menu
Higher end TVs had the audio controls and the CEC controls in the same menu.
Dual Menu
Some TVs had a separate menus, one for audio controls and one for CEC controls.
Null
We knew that there will always be badly behaving TVs that our system will not recognize. We wanted to be prepared for those TVs as well.
One size does not fit all
One of the primary goals behind using HDMI was to identify a way to make setup for the user easier and while the above was far more setup support than any other competitive product we saw, we found that if we did a check before entering setup we would see that 75% of TVs could be configured by our app in the back ground without any user interaction. The best UX is invisible.
While we were happy about the 75%, 25% of a million users is still a large number of users that would need some amount of help. We created custom instructions based on their TV brand. While the user was setting up their TV, beam would ping the TV after each pivotal step and as soon as the TV was configured properly the user would bypass any remaining instructions.
Validation
As a part of twice monthly testing cycles we were able to test and validate these flows and designs multiple times and made sure to be constantly iterating.
Nothing to see here
We were able to reduce the minimum home theater flow from 13 screens to 1 screen.
“Setup is one of the most complained about issues with Sonos customers. In addition to creating a delightful setup experience for Home Theater we were able to eliminate it altogether for 75% of users.”
This work proved to be invaluable to our company. My product manager and I were named as inventors for patent US20190373313A1
A rising tide Lifts all boats
As a byproduct of my work I led the translation of our Sonos Specification document from a static PDF document to a dynamic spec using Sketch, Zeppelin and Confluence.
Voice
HDMI was not the only innovative new technology brought to Beam and the Sonos ecosystem. Beam would be one of the first Home Theater products in the world to ship with a built in voice assistant. Beam launched with Alexa enabled and Google Assistant quickly followed. Voice was an exciting new frontier at the time and no companies (even Amazon) had explored Voice in the living room.
Ears and mouths
In our early investigations into Alexa on Sonos it became apparent that users, and brands alike viewed “Smart Speakers” as a single product when in actuality a smart speaker is the combination of a microphone and a speaker. This may seem like a simple concept but it gets extremely complicated when multiple speakers around the house are connected a potentially different amount of microphones and all of the microphones are connected to one maybe two different voice assistants.
We have a problem
If you have used a smart speaker you know that they can be unresponsive, unreliable and sometimes altogether unnecessary, this is all highly amplified in a Home Theater Experience. The reigning model of Alexa interaction is that feedback is given immediately after the speaker hears the Wake Word. This creates a behavior of a user pausing every time they utter a wake wake word (Utterance). While watching a movie the chime not only interrupts the the audio but “ducking” (volume reduction) reduces the the audio to an inaudible level so the system can hear the command better. We wanted to find a way to interact with a voice assistant while not disturbing the Home Theater experience.
A priority shift
At the beginning of this project voice interaction had only been considered during audio playback. This was fine until Home theater is added to the mix.
When watching a Movie or a TV show the focus is on the content. Which completely changes the context of the voice assistant.
Trusting voice
To understand the problem a bit more I worked with the Voice research team to understand how users use voice systems in the home and at the time it was becoming evident that many advanced users disabled the chime on their system as they felt confident with Alexa’s ability to hear and respond to their commands.
“Voice feedback is intrusive and unnecessary how can we fix it?”
A Solution
We had a theory that a simple tweak to the voice system could greatly improve the experience of the user experience. We identified that removing the chime altogether and only ducking when an audio response was necessary could almost eliminate the identified problem.
Validation
In order to take our theory beyond speculation we understood that we needed to test this new proposed feature. To do so we set up a series of internal and external tests.
Compounding interest
Before we could test our solution we needed to experience it ourselves so we worked with a development team to hack a voice prototype together that could control the ducking and chime behavior of our voice enabled speakers. Sonos also offers a standalone voice enabled speaker which can be paired with a home theater speaker to act a surround system. To add to the situations the standalone team asked us to throw a stereo pair into the mix.
User Testing
We were able to identify 30 Sonos users with varying levels of Voice Assistant experience to test multiple versions of our prototype to tell us many things about our new voice enabled Home Theater System.
From which speaker is voice feedback most preferred (The front)
Is the current voice interaction audio feedback distracting/frustrating (Sometimes)
Do you prefer our solution (mostly)
Small changes, big results
Because voice technology was so new it was clear that any sweeping change could frustrate a large amount of users. In order to deliver an amazing experience and allow users to configure their system to their preferences we shipped the experience as tested, but created a robust menu that allowed users to adjust their system to their liking. The two major features that shipped were:
While the solution was preferable and even coveted by many of our users, the decision was not unanimous. Many users were used to the Alexa system and did not want it augmented. Some users had trouble telling if the system had heard them. Some users assumed the system heard them and were frustrated when their ask returned nothing.
Speaking to the right speaker
The one clear result of our research is that multiple voice enabled speakers in a home environment can be chaotic and confusing. Based on the system we devised different speakers would have their microphones automatically deactivated based on the configuration in which they were setup.
This work and the implications for the future were included in Patents US20190179611a1 and WO2019152722A1
Voice Options
Before this work started there were no voice settings for individual speakers they were either listening or not.
The work done for home theater voice interaction made it clear that there was not one voice configuration that would work for all of our users. In order to meet our users needs the voice option work we devised for Home Theater was then implemented across all voice enabled Sonos speakers.
This solution and research was integral to the 3 patents that we successfully filed due to this initiative.
On Screen Display
A way forward
HDMI compliance meant that Beam was required to have some sort of Graphics processor. After further investigation it was found that it was able to output 1080p resolution. As a side project I decided to explore this a bit by asking the question “what does a Sonos On Screen experience look like?
Enhanced Setup
Having a direct connection to the Televisions through HDMI means that we can enhance the setup experience by making use of the largest display in the home. Each step of setup can be displayed and supplemental information can be provided.
Intro to Sonos
An ongoing challenge for Sonos is communicating the concept of the product ecosystem to new users. By showing a welcome video there is a moment of excitement to celebrate the user’s purchase and also explain a bit more about the capabilities of the Sonos system.
Visual Voice
Being able to talk to a speaker is nice but sometimes you can get lost in the conversation. The television would be an excellent way to display not only the conversation that is happening but also the content that is being requested.
Connected A/V Experience
At the time of this project Spotify had just launched the integration of music videos into their product. Being able to display these videos on the screen would be an excellent way to connect users to the content.
Content Switching
Having control over the TV means that voice commands could allow a user to switch to different inputs on their TV.
Not always meant to be
This was purely a side project based on innovations that I thought could push Sonos into a new unique direction. While all of this work was 99.9% out of scope I did work with a developer to bring the What’s Playing screen to life during an internal hack-a-thon.
Beam did ship with a small On Screen experience which is essentially a screen saver to connect users to the platform a bit more when they are listening to music with the TV on. May ultimate goal was to get a foot in the door with On Screen Display and I am happy to see it every time I turn my TV on.
