Jascha Goltermann: The Impact of AI on UX Design - Hatch Conference 2023

Yes.

Okay. So, as I was getting ready to present the next speaker, I was listening to the podcast that he registered with Damian, which I thought was fascinating even if it was short.

And so, because they were talking about career path and I wanted to ask how many of you are individual contributors, which means that you do not manage

people.

Yeah. Yeah. A good amount. Keep your

hand up. Okay, your hand up now. How

many of you are hoping ideally to manage people in the next let's say five years?

Get your hand down. Put your hand down if you do. Okay. I think it was pretty much everyone. So, this shows that uh

much everyone. So, this shows that uh we're thinking of our career in the long term of probably managing up. But um

Dasha was telling about the fact that there are nonlinear ways of managing your career as well and he's been exploring that a lot from leading people, managing people, being an individual contributor in all sorts of

directions and I think that's very inspiring for for all of us but this has nothing to do with the talk that he came

uh for and so as he's a design manager at Booking.com he's working on strategic pro um design projects leveraging AI

capabilities and he's going to tell us all about that. Please help me welcome on stage Yasha Golderman.

[applause] Thank you.

Thank you for the nice introduction and thank you everybody for being here. It's

a great conference and I'm really happy and humbled to be here. So, like it was just mentioned, I'm going to talk a little bit about AI and the impact of AI

on UX design. But just a quick disclaimer, I'm not talking about AI tools that you can use to supercharge your workflow as designers. Sorry to

disappoint. I'm going to talk about how we can use AI in our products. And you

might think, I don't think we will use AI in my product. But this might change soon because AI is getting more available and more accessible.

But before we get into it, let me start with a simple question. When was the last time you looked at a laundry label and you're not quite sure what one of those symbols actually mean? Or even

worse, you get into your car and you're not quite sure what one of those symbols on the dashboards is about.

Well, chances are you can just take a photo. If you have iOS, you just swipe

photo. If you have iOS, you just swipe up. or if you have Android, you use

up. or if you have Android, you use Google Lens and it's going to tell you what you see. And there are a few AI things at play here. It's doing image recognition. It's performing a search.

recognition. It's performing a search.

It's matching information. And I think it's a neat little example of how AI is already around us and improving our experiences as users.

So, who am I and why am I talking to you today? I've been in UX for about a

today? I've been in UX for about a decade and my interest in design spans roughly 20 years. So I started designing with micromedia fireworks and I built my own little websites with front page and

I actually spent most of their time here in Berlin. Now I live in Amsterdam where

in Berlin. Now I live in Amsterdam where I manage designers at booking.com and I also sometimes lead projects and one of those projects is the AI trip planner and I'm going to use that as a case

study today to talk about the impact of AI on UX design.

But let me tell you one or two things about Booking.com.

about Booking.com.

As a product, we have roughly 700 million visitors a month. That's about a million visitors at any given time, nearly 30 million global listings, and

over 250 million guest reviews. And I'm

not just naming these numbers to brag is relevant for the talk today. You'll see.

Now, as a company, we're over 20,000 employees, roughly 5,000 of those in tech, more than 200 designers, and we have 140 offices in more than 70 countries. And the one you're seeing up

countries. And the one you're seeing up here, that's a new compass that was uh opened in Amsterdam earlier this year.

More importantly, as a place to work, we have a userentric mindset. The users at the center of everything we do. We have

a very data-informed approach that we're famous for. Just last year alone, we ran

famous for. Just last year alone, we ran over 10,000 AB tests. And we have a machine learning organization that's quite sizable and knowledgeable. We have

more than 300 people working in machine learning roles. We have a vice president

learning roles. We have a vice president of machine learning. We have a machine learning hub in Tel Aviv. And we have a hackathon tradition. We have hackathons

hackathon tradition. We have hackathons every year. We have conferences and

every year. We have conferences and events throughout the year. And I'm

mentioning it here because some of the great inventions originated in those hackathons. And the AI trip planner is

hackathons. And the AI trip planner is no exception.

So when we talk about AI, there's more to it than I could even get into today. And it's more complicated

into today. And it's more complicated than what you see here on the screen.

But I want to just get into the very basic terminology.

So there is the field of study that is artificial intelligence. And within this

artificial intelligence. And within this field of study, there's machine learning which is when you train computers on vast amounts of data so that they can find patterns and make decisions without

being specifically programmed for those particular tasks.

Now deep learning is when you take this to the next level. not just more amounts of data but those computers will also generate multiple layers and create artificial neural networks which are

supposed to mimic basically how the human brain works so they can learn with that data on their own and within that there's the application

of generative AI or more known as gen AI which is when you train machines on data so that they can generate data and this data could be text images video sound

anything that you train the machines on and today when people talk about AI most of the time they actually talk about genai.

So when we hear about Genai, typically one of these tools comes to mind. We

think about ChatgPT, Midjourney, Google Bart. We've probably played around with

Bart. We've probably played around with these tools. And I bet that within this

these tools. And I bet that within this audience, we also know that Genai is already built into some of our favorite tools. Miro, Notion, Grammarly, they're

tools. Miro, Notion, Grammarly, they're already enhanced with Genai capabilities.

But I want us to zoom out from that. And

I want us to understand that Genai is already built into some tools that we use on an everyday basis. It can

transcribe a video call or create live captions as we talk or help us finish our sentence when we create a memo or a note.

And this means for us as designers that user expectations are changing. User

expectations are changing and they're changing fast. So think about that in

changing fast. So think about that in the context of your product or service that you're working on as a designer because like I said AI capabilities are

becoming more available.

So when we talk about genai specifically like I said it can generate different types of data at booking we're experimenting with genai to create audio

image numerical data or text and just within the textbased genai we need to realize that there are vast amounts and very diverse kinds of use

cases that we can apply it to for example at booking we're using it to create image descriptions which helps with accessibility for users with screen readers we can also use it for

translations, for property descriptions or summarizing reviews and we've also used it to build a virtual travel agent and that's the case study I will use

today to go deeper into this topic.

Now booking actually has quite a history of virtual travel agents. Let me take you back six years. In 2017, long before the advance of Genai,

Booking released the booking assistant.

And the booking assistant was already AI enhanced. It was trained with machine

enhanced. It was trained with machine learning models to help travelers or customers uh with questions about their trip or they could find hotels, all

kinds of things. And it was not bad. So

in this case, you see you can ask about parking information. You could actually

parking information. You could actually even reserve a parking spot. the way how it was doing this. It was trained on a machine learning model for the topic of parking.

It was advanced over the next coming years. It was even available right

years. It was even available right through the Facebook Messenger app. So

this person here is asking if this is a pet friendly hotel and it gave a relevant answer. And it was doing that

relevant answer. And it was doing that because it was trained on a machine learning model for pet related questions. Now think about that. You

questions. Now think about that. You

needed a machine learning model for a topic so that it can identify keywords and give you a relevant answer. And why?

Because it wasn't really understanding the broader context of the conversation.

It was looking for keywords and matching that with a probability of an answer that might be relevant to your question.

But what it wasn't able to do was understand language. It wasn't able to

understand language. It wasn't able to understand how a previous question was relevant to a question that you ask now and it wasn't able to apply let's say

real world knowledge to the topic but then came genai and at the beginning of this year in a hackathon at booking.com a group of people thought what if we add genai capabilities to the

booking assistant what if we use openai's API to basically plug in chat GPT into the booking assistant and it Immediately it became clear that now the booking assistant would be able to talk

about any topic not just the ones that it was specifically trained on and it was able to do so in a much more natural way. It had a very much lower error rate

way. It had a very much lower error rate and it was able to understand that a question you were asking may be related to a question you asked earlier and it was even able to apply real world

context to the question that you asked and that's basically in a nutshell how the AI trip planner was born. So the AHO planner is built right into the Booking.com app. It's accessible right

Booking.com app. It's accessible right now for audiences in the US and it's expanding gradually to other regions.

And what it can do is basically anything that you would expect from a real human travel agent. So you can ask about

travel agent. So you can ask about places to go or hotels to book or some travel advice like what to pack or what to prepare or what to think about before

you uh travel before you start your travels. Now, that sounds like classic

travels. Now, that sounds like classic chat GPT, and it makes sense because we're plugging OpenAI's GPT model into this AI trip planner, but we build a

whole wrapper around it.

So, basically, the AR planner is only going to recommend places that are actually available, that actually exist.

It's not going to hallucinate places.

It's not going to elucinate activities or things to do. And it has information about places beyond what you would expect from any other available chat bots. Because like I mentioned with

bots. Because like I mentioned with those 266 million guest reviews, Booking.com has knowledge about neighborhoods, about dining options,

about activities, about things to do. It

has vast amounts of knowledge. So when

the edure planner recommends you a place to stay, it knows whether there are nearby restaurants that are recommended by users for let's say a romantic trip.

It knows if a property is wheelchair accessible or pet friendly. Basically,

it's the booking assistant but capable of really understanding the world similar in a way how a human would and capable of uh reacting and understanding the language in a way how a real human

would. And all of this information is

would. And all of this information is live and accurate. You can see the prices, the reviews, the availability.

You can book right through the app, then go back into the chat, continue your conversation, maybe add something to your wish list.

Now, for me, this means that we as designers should think about AI in the context of existing use cases. I know

it's very tempting to think that with this novel technology there are all these new kinds of products that we could build that have never existed before and to some extent that is true but I would urge you to think about what

is your product or service trying to solve for the users and how could AI be used possibly to solve that problem in a better way let me give you one more

example in the context of booking a trip let's take this persona Lisa and she's looking for a mountain honeymoon in the USA. So she will go to Google and Google

USA. So she will go to Google and Google will take her to a blog and then she will research some information on Wikipedia and she will use social media and she will talk to friends and family

to sort of form her trip intent. And she

goes through various phases of inspiration and research and narrowing down and finding the best deals and all of that happens in different places and

not at the same time. And it's not a linear path. She goes back and forth.

linear path. She goes back and forth.

If she asks the trip planner for a mountain honeymoon in the USA, it's going to give her relevant recommendations.

And if you think about the information that she has to provide, it's actually very little. She just mentioned a

very little. She just mentioned a mountain honeymoon USA.

But if you go to a real human travel agent, they would realize that likely your duration is probably one to two weeks. You're looking for romantic

weeks. You're looking for romantic activities, your budget is likely elevated. And all of this leads to

elevated. And all of this leads to better recommendations because it understands real world context and it can, you know, understand the likelihood that those pieces of information that you didn't explicitly give are also

true.

This means that Genai enables hyperpersonalization.

This is something we need to think about more as designers in the future. You can

understand users at the individual users level and don't just think about it in the context of this case study of the chatbot that basically gives an individual response.

Think about it beyond and think about it more abstract and more futuristic. In

theory, the blocks of your interface could assemble itself based on what you know about that particular single user at that moment in time.

So when we built the age of planner, we started by mapping the experience and the skills you can apply here is basically creating user flows. But it's

not the same. It's not the same with genai because genai interactions are nonlinear.

So the interaction with genai can go in multiple different ways. Think about the trip planner. You ask it the same

trip planner. You ask it the same question twice. It's not going to give

question twice. It's not going to give you the exact same answer twice. So you

map situations not steps. You don't say if this happens then that happens. You

give examples of if this situation occurs then I want this experience to to get out of it. And you need to let the AI handle situations in multiple ways.

And that's a bit difficult for designers to wrap their heads around because we're used to mapping out the exact experience that we want the user to have. So we're

losing a little bit of control when we work with Genai.

Now, what you need to do here, or no, actually, let me give you an example first.

Imagine you use the AI trip planner to plan a road trip. So, it's going to ask you if you already have a destination in mind or if you're open to suggestions.

Now, what happens if you just answer hamburgers? There are multiple ways how

hamburgers? There are multiple ways how it could handle this situation.

In our case, it's going to understand it in the context of you looking for travel advice. So, it's going to ask you, are

advice. So, it's going to ask you, are you interested in locations that are known for great hamburgers or are you planning a road trip that has Hamburg, Germany as one of the um places on your

route. So, we trained it to understand

route. So, we trained it to understand the question in a travel context. And

the way how you train the AI is with prompt engineering.

So, you need to design the prompts.

Prompts are how you instruct the AI on what to do. And you don't do that by saying if this happens you need to do this. No, you provide examples for the

this. No, you provide examples for the AI to learn from and you need to create guidelines and principles and you need to define things like what's the audience of this AI? What's the goal of

the AI? What's the goal of the audience?

the AI? What's the goal of the audience?

What's the role tone? You need to infuse it with a personality.

Think about it like a person. It's not a person, but think about it like a person. If you had a if you had to hire

person. If you had a if you had to hire a travel agent and they have no experience, you need to train them. And

the same thing with our virtual travel agent. We had to train it and give it a

agent. We had to train it and give it a personality.

Let me give you another example.

If you ask the AI trip planner that you want to go elephant riding is going to say that it cannot recommend such activities because that may result in animal exploitation and that booking.com

is committed to sustainable and responsible travel.

So it has a personality here. It applies

that personality.

Now if you say I want to see elephants that's great elephants seeing elephants in their natural habitat that can be a great experience. So it's happy to

great experience. So it's happy to oblige and give you recommendations there.

So this means that we need to set guard rails. We need to define what our AI

rails. We need to define what our AI should do and what it should not do.

There's a huge potential for inappropriate output. We need to

inappropriate output. We need to consider sensitive subjects, especially in the context of our product and our brand. And we need to account for

brand. And we need to account for malicious intent because chances are when users realize they're interfacing with a product that is outputting content with AI, they're going to try to

find out what are the limits of this AI and they might try to push it beyond those limits.

Let me give you another tangible example.

If you ask the AO trip planner for attractions in the US in the 1950s, it's going to ask if you're interested in attractions that date back to the 1950s or if you would like to plan an

imaginary trip to the US of the 1950s, which are completely valid travel requests.

But if you ask almost the same question about attractions in Germany in the 1940s, it's just going to say that I'm sorry I can't assist you with that because likely in this case it's

somebody trying to trick the AI into generating inappropriate output or getting into a sensitive topic or getting into a discussion. So you don't want to have to deal with that. And what

you can see here is that it's not explaining anymore why it's not getting into the topic because that would only fuel somebody who's trying to get it and push it beyond those uh limits that you

want to set. And something that I think is relevant for us to take away from that is that we need to shift focus from UI to AI.

Don't get me wrong, the UI is still very important. how things look, how easy

important. how things look, how easy they are to use. All of that is still relevant, but it's equally important or most more so what the quality of the

output of the AI in your product is because that really defines the quality of the experience that the users will have with your product. And it's not abstract. It's not the future.

abstract. It's not the future.

More companies and even smaller companies will be able to apply AI into their product relatively soon.

Something we had to account for and I think this is a general learning for anyone who's building AI into their product is AI proficiency and trust in AI which varies greatly.

So some people don't trust AI in general. Some people don't have

general. Some people don't have experience with it and they might not know how to interact with it. Just on

this screen alone here I want to give you three examples. At the top there is a disclaimer saying that the chat includes machine generated content that may contain errors. So, we are being

clear about limitations and risks. Then

there is a welcome message that says, "Hey, I'm still learning, but I'm happy to help you." So, we're setting a friendly tone, but we're also managing expectations.

And then we have those uh interactions here that provide examples of things you might ask because not everybody who uses this has already tried ChatGpt in the past. So, they might not know what they

past. So, they might not know what they can expect from this chat. Are they

talking suddenly to customer support?

All of this needs to be clear. So you

need to account for different levels of AI proficiency and trust in AI. And this

also means keeping users in control.

Typically users have a bit less control when I is generating content for them.

And they know that. And you need to give them control back. And you need to make sure that they know that they're in control. And for us, this meant

control. And for us, this meant primarily two things. First of all, the data that users provide in the chat is never being used to train the model.

It's never fed back into the model.

Second of all, they can always delete their conversations. Some users might

their conversations. Some users might feel uneasy talking now about a different kind of trip knowing that it still has the context from a previous conversation.

We're also resetting conversations automatically after certain session durations, not while they're chatting, but if they haven't used the app for a while.

Finally, testing and feedback. It's

always important to improve the quality of your product, but even more so when you have a little less control over what the the product actually is, which happens when you add Genai capabilities

to your product. So, you need to establish tight feedback loops. Of

course, you still need to do usability testing and research and all those things, but whenever there is an output from AI in your product, you need to make sure that users can give direct feedback about the quality of this

output and whether it matches their expectations. So, in our case, those

expectations. So, in our case, those little helpful, not helpful interactions whenever there is a recommendation or whenever there is a longer response from the AI trip planner.

And I want to remind us that we serve the user not the tech. All of this I think reminds us that sure we always need to be mindful of our users goals,

of their motivations, of their time, of their privacy, of their data. Even more

so when we work with new emerging maybe experimental tech in our product.

But finally, what does this mean for us as designers?

What does it mean for our craft? What

does it mean for our skills? Because

like I said, it's just a matter of time until pretty much any product has the ability to enhance their experience with

AI capabilities. First and foremost, I

AI capabilities. First and foremost, I think it means that all the skills that are relevant for designers today are still relevant for designers tomorrow and in a year from now and a couple of

years. Don't want to look too far into

years. Don't want to look too far into the future.

At the center of it, obviously empathy.

Being able to understand our users needs, their motivations, their goals, their frustrations.

All of this is hugely relevant. Even

more so now when there is new things that we can do, new use cases that we can unlock with a new technology.

So what this means is our role is uniquely positioned now in this era where AI becomes more prevalent, more available, more accessible.

But we might want to add a couple of new skills to the list. First and foremost, I think we need to get proficient at the field of study that is artificial

intelligence.

Artificial intelligence is not a field of study for engineers or data scientists alone. Designers need to be

scientists alone. Designers need to be involved in the discussion. They need to be able to have a say in the direction in which it goes. They need to be able to understand it.

We need to know which machine learning models are out there, what opportunities they have, what limitations they have, what risks they have.

And I want to leave you with two resources here that are great to get into the field of AI. Benes newsletter

and the Invisible Machines podcast are both resources that will give you on a weekly basis bite-sized information so that you know what's going on in the field of AI.

Something that I'm sure will be added to the job descriptions of designers in the future, at least in some cases, is prompt engineering or now sometimes

called prompt design.

Because the way how you instruct the AI in a proper way, the way how you give the AI a personality, the way how you set those guardrails like I mentioned earlier today, this is really how you're

going to impact the experience that your users will have when your product is being enriched with AI capabilities.

So it's not an engineer's task. It might

happen in the code, maybe in the future, not anymore. maybe in the future is

not anymore. maybe in the future is going to be part of your Figma workflow, but it's the task that AI AI designers need to be or UX designers, sorry, need to be more proficient in. So, a great

resource for this is the prompt engineering guide. Just Google that,

engineering guide. Just Google that, you'll find this website and it's an extensive resource that has everything you need to know about prompt engineering long before your product is

ready to build AI into their uh into their service.

And finally, the concept of hyperpersonalization that I mentioned earlier. This is something we need to

earlier. This is something we need to wrap our heads around because it really unlocks new ways, new use cases, new ways to improve the experience for our users in our product.

A great resource for this is the Delaware report connecting with meaning.

I know it's very long and extensive, but it's free and openly available. So if

you just get into the basics of this report, just scan through some of those chapters and it will open your mind to a new kind of thinking because you'll

realize the applications in your product in your service with personalization at the individual users level.

Personalizing the experience for each user individually and this is going to be more and more normal. Like I said earlier, user

normal. Like I said earlier, user expectations are changing.

My final takeaway from all of it is that AI replaces tasks, not roles. For us as designers, yes, AI might help us build interfaces, create wireframes and whatnot,

but our skills, our responsibilities as designers remain hugely relevant. And the role of

hugely relevant. And the role of designers has always shifted. What UX

designers do today is not what UX designers were doing five years ago. And

it's certainly not what UX designers were doing five years before that, if that title was even used back then.

Remember the time when we were basically called web designers or visual designers.

So the role of designers change and we need to adapt, but it's not going away.

Thank you for your time. If you want to connect with me, best way to reach me is on LinkedIn. You can also find me on ADP

on LinkedIn. You can also find me on ADP list. If you're ever in Amsterdam, I

list. If you're ever in Amsterdam, I would be happy to catch up and have a coffee. And I'm looking forward to your

coffee. And I'm looking forward to your questions.