I wanted to explore how increased autonomy and connectivity in cars could be leveraged to create better music entertainment experiences. One of my assumptions before the project began was that people would primarily want an agent that could help them curate the right music based on their destination. What the project showed me was that the power of a social assistant in the car extends much farther and can actually help people become more in touch with their music tastes over time.
Independent Design Project
Jan 2021 - May 2021
Visual design
Conversational design
Prototyping
Figma
WoZ prototype
Bailey, is a social conversational agent in the car that facilitates a new way for people to experience music on the go by understanding a person's relationship to what they listen to, curating music for them, and helping them better distinguish the nuances of their own musical tastes.
The final design shows a step by step wizard that takes users through the most important information needed to provide an accurate analysis. With the amount of data needed and the potential for users to have missing information, the tool uses intelligent dynamic defaults to estimate numbers where users may have gaps.
The focus here is to help build trust with the user by showing them that there are significant algorithmic processes being undertaken to provide accurate analyses and show what those actual processes are.
The final screen of the tool displays the results of the analysis showing the most important data points with opportunities to get more detailed analyses delivered electronically by providing personal contact information. Users can also manipulate important real-time variables that do not require a full analysis re-run.
Using the left collapsible menu, users can see their current inputs and easily modify one or more and run a new analysis without having to go back through the entire wizard.
The final design shows a step by step wizard that takes users through the most important information needed to provide an accurate analysis. With the amount of data needed and the potential for users to have missing information, the tool uses intelligent dynamic defaults to estimate numbers where users may have gaps.
The focus here is to help build trust with the user by showing them that there are significant algorithmic processes being undertaken to provide accurate analyses and show what those actual processes are.
The final screen of the tool displays the results of the analysis showing the most important data points with opportunities to get more detailed analyses delivered electronically by providing personal contact information. Users can also manipulate important real-time variables that do not require a full analysis re-run.
Using the left collapsible menu, users can see their current inputs and easily modify one or more and run a new analysis without having to go back through the entire wizard.
Playing music in cars today requires you to connect your phone, choose what you want to listen to, and manage playing the right music while engaging in a high cognitive task. Discovering new music on the go is almost impossible. This current experience diminishes a person's ability to appropriately curate and connect to their music experience and we wanted to change that.
As autonomous vehicles enter the consumer market, connectivity will continue to increase and enrich in-vehicle experiences. We wanted to explore how a connected agent in the car could engage in a multi-sensory live interaction with the passenger to create an immersive entertainment experience.
People don't just listen to music. They talk about it, share it, and discover it. We wanted to create a music experience that allowed people to do more than just listen while riding alone in an autonomous vehicle.
We had participants record and reflect on their daily music listening by having them fill out a questionnaire every time they had a music experiences for 3 days. The questions asked participants about what they listened to, if they were engaged in another activity or actively listening, devices they were using, and how they discovered the music. The diary study ended in with an interview where we had participants walk through their entries and talk about their music listening habits and motivations.
People have a better music listening experience when they have a personal connection to the music.
The right music can help people reach an aspirational state (mood or goals for another simultaneous activity).
People want an easy way to find music they know they will like.
People want to have control over their music experience to customize it in any way they want to.
I helped us define the level of proactivity by recommending we vary it based on the estimated confidence the voice assistant had in the actions it was taking (a lower level of proactivity is assumed to use less personal data and deliver less overall value). In all scenarios, the assistant engages the participants.
In a low proactive scenario, we had the agent ask the participant what they wanted to listen to and confirm all actions before taking them. The assistant would not make any music recommendations.
In a medium proactive scenario, we had the agent recommend songs, playlists, volume levels and, changing speakers (based on prior knowledge and contextual factors) to the participant but, still confirm all actions before taking them.
In a high proactive scenario, we had the agent just communicate that they are changing the music, volume level, or speaker (based on prior knowledge and contextual factors) without confirming with the participant. We also had the agent explicitly explain their actions.
We created four sets of storyboards to validate the insights we derived and explore ideas of how they could be addressed by a re-imagined in-vehicle music experience with a connected agent. Each set contained 3 storyboards with increasing levels of riskiness in the interaction between the agent and the person.
People want an agent that can curate a music experience for them when they don't know what to play.
People want control over the personal information that the agent knows and infers to inform how it curates the music experience.
People want the agent to facilitate a music experience that is appropriate for their current activity in the car.
People are open to engaging with social agent who converses with them about their music preferences.
I helped design a 1 hour in-person experience prototype to simulate being in an autonomous vehicle. We gave people a hypothetical roundtrip journey and used a multi-modal embodied agent to facilitate a conversational music experience during the ride.
We experimented with an embodied agent to see if people would be more comfortable having a personal music-related conversation with an agent that had an appearance compared to the formless one we had tested thus far. We chose not to give the agent a human embodiment to avoid people perceiving it as capable of interacting as a real human would but, gave it eyes because our foundational research had shown that eye-gaze is a powerful tool when building a trusting relationship between a human and agent.
People want the agent to interact with them in ways that helped them understand their music tastes at a deeper level.
People are willing to give the agent control over their music experience if they feel that it appropriately understands their tastes and needs.
Curating a focused in-person environment for the experience was more immersive compared to our previous remote setup where participants were in their own space.
People view the agent as a "friend in the car" but, differ in how much they want to interact with it based on their current mood and personality.
How can we best use Bailey's conversational nature and music knowledge to educate people about their own music listening tastes to become better at discovering and curating music experiences themselves?
People highly value Bailey's ability to recommend the right music but, how do we provide immediately value as Bailey learns and improves over time and continue to build trust in the agent when it fails. How can we use other sources of data to train Bailey faster?
How can we use Bailey to curate immersive entertainment experiences beyond music listening?
Designing a social agent
How do we design an agent that can learn people's music preferences through conversation and curate the best possible music experience?
Testing Sessions
Methodology
We designed a virtual test drive experience and simulated a conversational agent that people could engage with over a Zoom call. To simulate a "learning agent" we scheduled 3 30-minute sessions with each participant schedule over a 5 day span.
Iterating on the fly
While conducting the studies we had meetings as a team to discuss our findings and improve our agent/setup on the fly to try to create a better experience. Some of the major changes that we made were to only engage participants in-between songs and display confirmations to participant requests via text when we found that they did not like the agent interrupting the music.
Key insights
People perceived the agent as an "intelligent friend in the car" who made the music listening experience more enjoyable by sharing its own perspectives.
People are only willing to disclose highly personal information to the agent if they clearly understand the value it provides to their music experiences.
People need to clearly understand what the agent is capable of and its communication patterns in order to feel comfortable engaging with it.
An agent that engages a person in a conversation in a car can identify music recommendations more efficiently than a person can with the tools currently available to them.