The challenge
The solution
My motivation
Experiencing music alone
Exploring new ways to listen
Designing a social agent
Increasing immersiveness
Future work

The before and after

I wanted to explore how increased autonomy and connectivity in cars could be leveraged to create better music entertainment experiences. One of my assumptions before the project began was that people would primarily want an agent that could help them curate the right music based on their destination. What the project showed me was that the power of a social assistant in the car extends much farther and can actually help people become more in touch with their music tastes over time.

Organization

Independent Design Project
Jan 2021 - May 2021

My hats

Visual design
Conversational design
Prototyping

Tools

Figma
WoZ prototype

Project overview

The challenge

How might we utilize the affordances of autonomous vehicles and conversational technology to curate highly satisfying single-rider music-listening experiences?

The solution

Meet Bailey

Bailey, is a social conversational agent in the car that facilitates a new way for people to experience music on the go by understanding a person's relationship to what they listen to, curating music for them, and helping them better distinguish the nuances of their own musical tastes.

Concept video

Process

Solution

01. Wizard with intelligence

The final design shows a step by step wizard that takes users through the most important information needed to provide an accurate analysis. With the amount of data needed and the potential for users to have missing information, the tool uses intelligent dynamic defaults to estimate numbers where users may have gaps.

02. building trust with transparency

The focus here is to help build trust with the user by showing them that there are significant algorithmic processes being undertaken to provide accurate analyses and show what those actual processes are.

03. actionable insights

The final screen of the tool displays the results of the analysis showing the most important data points with opportunities to get more detailed analyses delivered electronically by providing personal contact information. Users can also manipulate important real-time variables that do not require a full analysis re-run.

04. Flexibility to run multiple analyses

Using the left collapsible menu, users can see their current inputs and easily modify one or more and run a new analysis without having to go back through the entire wizard.

01. wizard with intelligence

The final design shows a step by step wizard that takes users through the most important information needed to provide an accurate analysis. With the amount of data needed and the potential for users to have missing information, the tool uses intelligent dynamic defaults to estimate numbers where users may have gaps.

02. building trust with transparency

The focus here is to help build trust with the user by showing them that there are significant algorithmic processes being undertaken to provide accurate analyses and show what those actual processes are.

03. actionable insights

The final screen of the tool displays the results of the analysis showing the most important data points with opportunities to get more detailed analyses delivered electronically by providing personal contact information. Users can also manipulate important real-time variables that do not require a full analysis re-run.

04. Flexibility to run multiple analyses

Using the left collapsible menu, users can see their current inputs and easily modify one or more and run a new analysis without having to go back through the entire wizard.

Why autonomous vehicles, conversational agents, and music?

Autonomous vehicles present new frontiers for active entertainment experiences on the go

Playing music in cars today requires you to connect your phone, choose what you want to listen to, and manage playing the right music while engaging in a high cognitive task. Discovering new music on the go is almost impossible. This current experience diminishes a person's ability to appropriately curate and connect to their music experience and we wanted to change that.

Autonomous vehicles will be connected vehicles

As autonomous vehicles enter the consumer market, connectivity will continue to increase and enrich in-vehicle experiences. We wanted to explore how a connected agent in the car could engage in a multi-sensory live interaction with the passenger to create an immersive entertainment experience.

Music experiences in cars have significant room to grow

People don't just listen to music. They talk about it, share it, and discover it. We wanted to create a music experience that allowed people to do more than just listen while riding alone in an autonomous vehicle.

How do people experience music alone?

11
Diary Studies
+ Interviews

Methodology

We had participants record and reflect on their daily music listening by having them fill out a questionnaire every time they had a music experiences for 3 days. The questions asked participants about what they listened to, if they were engaged in another activity or actively listening, devices they were using, and how they discovered the music. The diary study ended in with an interview where we had participants walk through their entries and talk about their music listening habits and motivations.

Key insights

People have a better music listening experience when they have a personal connection to the music.

The right music can help people reach an aspirational state (mood or goals for another simultaneous activity).

People want an easy way to find music they know they will like.

People want to have control over their music experience to customize it in any way they want to.

Levels of proactivity

I helped us define the level of proactivity by recommending we vary it based on the estimated confidence the voice assistant had in the actions it was taking (a lower level of proactivity is assumed to use less personal data and deliver less overall value). In all scenarios, the assistant engages the participants.

Low proactivity

In a low proactive scenario, we had the agent ask the participant what they wanted to listen to and confirm all actions before taking them. The assistant would not make any music recommendations.

Medium proactivity

In a medium proactive scenario, we had the agent recommend songs, playlists, volume levels and, changing speakers (based on prior knowledge and contextual factors) to the participant but, still confirm all actions before taking them.

High proactivity

In a high proactive scenario, we had the agent just communicate that they are changing the music, volume level, or speaker (based on prior knowledge and contextual factors) without confirming with the participant. We also had the agent explicitly explain their actions.

Exploring a better in-car music experience

How can we create a connected autonomous vehicle experience that enables people's music listening goals?

10
Speed Dates

Methodology

We created four sets of storyboards to validate the insights we derived and explore ideas of how they could be addressed by a re-imagined in-vehicle music experience with a connected agent. Each set contained 3 storyboards with increasing levels of riskiness in the interaction between the agent and the person.

Set 1: Having a personal connection to the music
Set 2: Reaching an aspirational state or desired mood
Set 3: Easily discovering the right music
Set 4: Control over customizing the music experience

Key insights

People want an agent that can curate a music experience for them when they don't know what to play.

People want control over the personal information that the agent knows and infers to inform how it curates the music experience.

People want the agent to facilitate a music experience that is appropriate for their current activity in the car.

People are open to engaging with social agent who converses with them about their music preferences.

Designing a social agent

How do we design an agent that can learn people's music preferences through conversation and curate the best possible music experience?

27
Wizard of Oz
Testing Sessions

Methodology

We designed a virtual test drive experience and simulated a conversational agent that people could engage with over a Zoom call. To simulate a "learning agent" we scheduled 3 30-minute sessions with each participant schedule over a 5 day span.

The first iteration of our setup
The second iteration of our setup with text confirmations

Iterating on the fly

While conducting the studies we had meetings as a team to discuss our findings and improve our agent/setup on the fly to try to create a better experience. Some of the major changes that we made were to only engage participants in-between songs and display confirmations to participant requests via text when we found that they did not like the agent interrupting the music.

Key insights

People perceived the agent as an "intelligent friend in the car" who made the music listening experience more enjoyable by sharing its own perspectives.

People are only willing to disclose highly personal information to the agent if they clearly understand the value it provides to their music experiences.

People need to clearly understand what the agent is capable of and its communication patterns in order to feel comfortable engaging with it.

An agent that engages a person in a conversation in a car can identify music recommendations more efficiently than a person can with the tools currently available to them.

Designing a more immersive experience

How can we create a more immersive autonomous vehicle, agent experience that is safe during COVID?

5
Experience Prototype Tests

Methodology

I helped design a 1 hour in-person experience prototype to simulate being in an autonomous vehicle. We gave people a hypothetical roundtrip journey and used a multi-modal embodied agent to facilitate a conversational music experience during the ride.

Our testing setup
The on-boarding flow for our experience prototype

Why embodied?

We experimented with an embodied agent to see if people would be more comfortable having a personal music-related conversation with an agent that had an appearance compared to the formless one we had tested thus far. We chose not to give the agent a human embodiment to avoid people perceiving it as capable of interacting as a real human would but, gave it eyes because our foundational research had shown that eye-gaze is a powerful tool when building a trusting relationship between a human and agent.

Key insights

People want the agent to interact with them in ways that helped them understand their music tastes at a deeper level.

People are willing to give the agent control over their music experience if they feel that it appropriately understands their tastes and needs.

Curating a focused in-person environment for the experience was more immersive compared to our previous remote setup where participants were in their own space.

People view the agent as a "friend in the car" but, differ in how much they want to interact with it based on their current mood and personality.

Future work

How might we contiue to develop Bailey in the future?

The personal music trainer

How can we best use Bailey's conversational nature and music knowledge to educate people about their own music listening tastes to become better at discovering and curating music experiences themselves?

Creating immediate value for the user as Bailey improves over time

People highly value Bailey's ability to recommend the right music but, how do we provide immediately value as Bailey learns and improves over time and continue to build trust in the agent when it fails. How can we use other sources of data to train Bailey faster?

The entertainment assistant

How can we use Bailey to curate immersive entertainment experiences beyond music listening?

Conclusion

Reflecting on the project

This was my first full end to end product design project starting with research all the way through to the implementation of the beta version of the product for both consumers to interact with and to be presented to potential customers for licensing. It was an incredible experience and challenge learning how to create a truly value-generating product with a great user experience under a variety of time, technical, and user constraints. I was unfortunately unable to continue the project beyond the beta release, however, I helped to define the scope and direction for future iterations of the product. These next steps included conducting extensive user testing, dynamically changing the required inputs based on a user's specific fleet management role and desired analyses, and the creation of a dashboard for clients such as electric utilities who want to license the tool to collect and manage user data.

Back to projects