Welcome to the OpenHome team spotlight series, where we highlight the people building the next era of human-computer interaction.
We’re thrilled to welcome Lisa Falkson, former Senior UX Design Manager at Amazon, to the OpenHome team!
Lisa brings over twenty years of industry and research experience, specializing in the design, development, and deployment of voice-first multimodal interfaces. Most notably, Lisa led a team of Conversation (CxD) and UX Designers building Amazon Alexa’s core experiences.
Before Amazon, Lisa worked on speech-enabled IVR applications at Nuance, an early leader in speech recognition that was acquired by Microsoft in 2021. Her experience also includes next-generation voice user interfaces at NIO and CloudCar, as well as Amazon’s Fire Phone and early iOS interfaces at Volio.
We recently sat down with Lisa to explore the evolution of voice, from the early days of speech recognition and lessons of building Alexa to the recent breakthroughs that are making voice feel more natural than ever. We unpack why great voice experiences have always been as much about understanding people as understanding technology and what the future of voice holds as AI moves beyond the screen and into the world around us.
Let’s dive in!
You’ve been working in the voice technology space for a long time, starting with your work at Nuance. What first drew you to voice and what did those early experiences teach you about what it takes to make voice tech actually work for people?
I happened upon voice technology by sheer luck! When I applied to graduate school, I was accepted into UCLA’s Speech and Auditory Perception Laboratory and was lucky enough to learn the fundamentals of speech recognition and TTS (text-to-speech) from my brilliant advisor, Abeer Alwan.
At Nuance, I learned how important the voice user experience was in making the technology work. When engaging with enterprise customers, we needed to sell solutions to customer problems, not just sell software. The key point was focusing on how customers would enjoy using the systems and have higher success rates.
You started on the more technical side, but came to realize that many of the biggest problems were actually design. What led you down that path?
Yes, I started out as a “Speech Technology Specialist,” which meant I wrote grammars (the rules for what the speech recognition engine would understand) and tuned speech recognition parameters (confidence scores, etc.) to improve performance. In that role, I spent a lot of time reviewing data, and it was through this process that I discovered that most of the issues that customers experienced were due to the conversation design vs. the speech technology.
For example, customers were confused by the wording of a question, or the questions should’ve been asked in a different order. In many cases, the biggest improvements that could be made to an existing system would involve changing the conversation design itself. That’s how I ended up being what was then called a “Dialog Designer,” later “VUI (Voice User Interface) Designer,” and now “Conversation Designer” or CxD.
Over the course of your career, you’ve had a front-row seat to several major chapters in the evolution of voice, from Nuance to working on Alexa. What has changed most about how we design voice interfaces?
When I first started out designing voice user interfaces, the technology was so limited that you had to explicitly define what responses customers could make to the system! This was extremely rigid. For example, we relied on prompt tables and flow charts to show the fixed nature of the conversation. Later, the responses understood were widened as the recognition technology improved, but we were still limited to fixed TTS responses.
Now, the LLM generates the TTS and it’s a free-for-all! For LLM-based systems, our documentation is more fluid. It consists of sample dialogs and guidelines for the technology, such as prompt engineering. However, both processes are highly iterative and collaborative with product and engineering. Most importantly, the fundamentals behind voice design, such as Grice’s Maxims, have always been the same.
This new era we’re entering with LLMs opens up a whole fresh design space in terms of what people can do with voice. What new design challenges arise with this shift?
LLM-based systems have certain risks that deterministic systems do not. Without proper guardrails and certain behaviors anticipated ahead of time, the system can state racist and sexist beliefs, or assert that certain historical events did not occur. It’s important to do exhaustive red-teaming on LLM-based systems before deployment for this reason.
Looking ahead, what do you see as the next frontier for voice technology? What are the opportunities that you’re most excited to tackle?
I think the next frontier is a device that is truly personalized to you: the voice, the personality, the utility. How many times have you used a device that misunderstood you and did something totally different that you don’t even need it for? It happens to me regularly. I would also love a different sound and feel from the basic Alexa.
The other missing piece, I believe, has been emotion recognition and nonverbal audio recognition; the device should truly understand you and meet you where you are, and it should be aware of ambient noises like a baby crying, glass breaking, etc. In other words, spatial intelligence.
After spending so much time in your career building with some of the largest companies in tech, what excites you about OpenHome?
A lot of things drew me towards OpenHome. The first was probably the focus on the developer ecosystem. They really care about getting that right! After that, I was intrigued by the flexibility of the platform to use the best-in-class LLM and TTS to create custom voice experiences. I love the idea of creating a custom agent that I designed just for me, and then bringing that out of the computer and into the real world on an OpenHome speaker.
Certainly, the product is compelling, but I was also incredibly impressed with the founding team and their vision and passion for the product and space. After working for years at Amazon on Alexa, it was refreshing to meet new people and fresh ideas coming up, versus a team of people simply working on improving an existing product.
Join us in building the future of AI
At OpenHome, we’re building a future where AI is freed from chat to become an ambient intelligence layer woven into everyday life. OpenHome provides the hardware, platform, and community that enable developers to create voice-first experiences for the physical spaces people work and live in.
Through our open-source Voice SDK, local models, sensor layer, and DevKit, OpenHome turns a home device into an intelligent, conversational interface that can listen, sense, understand, and act in the real world.
To learn more, join our Discord, read our blog, and check out our developer docs.