AI Meets Sleep Health: Inside ResMed's AI powered health companion | AWS She Builds Tech Skills

Hello and welcome to another episode of AWS She Builds Tech Skills. My name is Tamar Gagerti.

I'm a senior solutions architect at AWS and your host for today's show.

We will be talking about the power of artificial intelligence, machine learning, large language models in the medical field. If you want to learn more about how these technologies are being used to improve people's lives, stay with us. for that I have a very special guest today joining us from Rasmet.

Um welcome to the show Phila.

It's great to have you today.

I'm so excited about the episode that we will be sharing with everybody. Um we always like to have our guests introduce themselves.

If you can introduce yourself, share more about your current role at ResMed and what was your career path to get to that role.

>> Yeah, sure. Thank you so much Tamara for having me here today. I'm really excited to have this chat with you. Yeah.

So throughout my career I've had a few um a few points.

So uh I started out really uh at Stanford combining quantum mechanics with machine learning to accelerate the discovery of chemical compounds and materials. So here we develop new methods to represent materials at the atomic and subatomic level that is compatible in a way that machine learning models can understand.

And then I moved on to Bluma where I was head of AI. I was leading the development of an AI platform to automate commercial real estate lending.

Um and here we use privacy preserving AI systems to fully automate the information extraction from lending documents.

And now at ResMet, I drive strategic initiatives that leverage machine learning, AI, and generative AI to deliver patient centric innovation, clinical impact, and operational efficiencies.

And I help shape our data and AI vision in a way that aligns closely with our business priorities and regulatory environment.

And yeah to execute on this I have a cross functional team of data scientists, machine learning engineers, um technical product managers and UX research uh working closely with product engineering and executive leadership to transform the experience and the health of our users.

So yeah, my responsibilities are uh focused on the development of these predictive and generative AI applications.

Um but the team is also an important partner for the governance um of responsible and ethical AI across the enterprise.

So yeah, we ensure that data science isn't just a technical function but really a catalyst for product differentiation and long-term innovation in digital health.

>> Excellent. Thanks for the introduction and amazing work that that you're doing.

I'm very excited to jump deeper into predictive and generative AI models further into the show. Um but we talked about ResMed and I wanted to uh share more about if if you can share with us who Resmet is, who the customers are.

I know that we also talk about patients.

So um for those that are new or haven't heard about ResMed, who is Resmet?

>> Yeah, of course. Um, ResNet is a global leader in digital health and sleep technology and our core business is really to develop cloudconnected medical devices and software solutions to treat sleep apneoa and other chronic respiratory conditions, but also other sleep issues like insomnia. So, our core users are basically people in need of better sleep.

>> Thanks. Um, I know we all like to get a great night's sleep. Sounds like RestMet is is helping a lot of people achieve that.

And to add more context on the work that ResMed is doing, uh, let's maybe double click on sleep apnoa and the importance of sleep related to health.

>> Yeah, absolutely. So maybe um you know if you know a little bit about sleep apnoa uh but really what it also does is it disrupts oxygen flow increasing the risk of serious conditions including cardiovascular disease, type 2 diabetes and fatty liver disease. And also these frequent sleep interruptions can reduce the life quality of our patients causing daytime fatigue, poor concentration, mood issues and also a higher chance of accidents.

So that all kind of affects the life expectancy as well.

So untreated sleep apnoa can raise the risk of stroke, heart failure, early death.

Um but thankfully treatments like CPAP can significantly improve longevity and overall health of our patients >> and thanks for sharing this insightful stats and the importance of sleep.

At AWS also we like to work with our customers and understand the pain points from the more business perspective and um also their customers in this case the patients that are requiring this new technologies.

Let's add even more context as to why uh the work that ResMed is doing is impactful.

What can you tell me more about the awareness and diagnostics piece?

Yeah. So globally around 940 million adults have obstructive sleep apnoa and up to 90% of moderate to severe cases go undiagnosed.

So that really shows just how big the awareness gap is that we are dealing with here. Um and even in the US uh out of the 30 to 40 million adults who have or are likely to have sleep apnoa 20 million remain undiagnosed.

Right? So this really highlights how inefficient and fragmented the pathway for users is from awareness to diagnosis and onto therapy. So yeah and then even after diagnosis 30 to 40% of patients either don't start therapy or they stop it often due to discomfort or lack of education or doubts about its effectiveness.

Um so awareness alone isn't just enough. it must be paired with ongoing support throughout the journey and yeah that's kind of in it of itself a real business challenge for us and for >> yeah now that we've been t we will be talking a little more on that personalization and that journey of uh following with the with the patient itself really caught me by my attention when you mentioned that 90% of that moderate to sever severe cases go undiagnosed uh this is a very large percent of of people that could be benefiting from the work that you're doing.

I know as the director of data science a IML your team has been working on a number of projects and initiatives to improve the patient care and most recently you led uh the development of an AI power health assistance using large language models to provide personalization uh for patient interaction.

For those that are new to sleep apnnea space, can you provide a little more context on what personalization or personalized p uh patient care looks like and what is needed to achieve such personalization.

>> Yeah, absolutely. So, first of all, big shout out for the team. All of my team members work hard and really take ownership of the quality of what they build.

Um, so, you know, if I would be able to have them all here today, I would.

Um but yeah now the team has really looked at a couple of related business challenges just to give a bit more context.

The first one is more around coordinate the users journey from awareness to diagnosis to therapy and the second one is more around how to optimize the experience and the outcomes for people who are already on therapy.

So granted they are both related um but the needs of the users change over the course of their journey. So we believe that personalized patient care starts with meeting users where they are and then reduce logistical, emotional and knowledge barriers.

So how do we do that?

In this implementation, we use the individual users behaviors and questions as they interact with our Genai companion to then personalize the conversation.

So, based on what we know about the user, the companion offers encouragement, follow-up suggestions, which include things like workflow automations, and also localized resources that are specific to the country or location that they're in.

Um, to really help users access the support more seamlessly.

And yeah, behind the scenes, we fine-tuned a large language model to identify what stage of the patient journey the user is in and then dynamically match them with the most relevant action, whether that's education, troubleshooting, or getting started with therapy. Um, thank you very much for sharing. It was great to see also the the demo. uh jumping into a more technical lens because here we have a very technical audience that wants to know more on what are those details.

Could you walk us through the business challenges you were trying to solve and also specifically how uh you looked at personal personalized therapy through a technical lens.

>> Yeah. So let's talk a little bit more more about how that personalization looks like for users who are already on therapy.

Right. So um when they already on therapy here one core business challenge is data fragmentation right so our users generate a lot of data from wearables medical devices and but it's really connected into a cohesive actionable experience right and in a way we're all looking for that holistic coaching right so we're working on solving this through real time data integration and contextaware insights that are available right when users wake up.

Right? So from a technical lens, we transform complex data streams into precomputed metrics and then these metrics help us spot trends and identify issues or friction points that then trigger proactive nudges from our companion before the issue escalates.

Right? So um we're implementing this using a large language model agent powered system that analyzes the data ask context risk rich follow-up questions and then creates meaningful explanations and coaching.

So this allows patients to not only understand their data but also act on it to improve their therapy experience and health outcomes.

>> Thanks. and data is always at the core of everything.

So it's it's great to understand how ResNet went through that data integration, the interpretation and um I wanted to also look at how um ResMed is establishing that deep medical pattern recognition for hyperpersonalized patient experience.

for that. I know that you mentioned your team leverage AWS's language model capabilities to analyze the sleep apnnea um the sleep and wearable data for personalized patient care. How did your team achieve this level of personalization?

What really happens behind the scene and what technologies are you leveraging for this?

Yeah. So basically it all starts with our UX research right which found that users want a proactive actionable and engaging experience in addition to summary analytics.

So to address this, we are leveraging therapy device data to generalize generate personalized insights and then initiate this proactive datadriven conversations uh with our users through this geni companion that I mentioned before.

And our data warehouse and MLOps platform are hosted on AWS infrastructure and act as an input to the AI system.

And we also look at the conversational history and context of the patient.

So behind the scene, the large language model is continuously analyzing both structured and unstructured data to deliver this kind of personalized proactive coaching and help patients troubleshoot their therapy in real time. So yeah, a chatbot doesn't have to be just about answering questions.

It can be about anticipating needs and solving issues.

>> Thanks again, Phila. It's it's good to start with the business perspective.

So now we're talking more on the technical aspect and once you have that connection between we talked about the patient data we also talked about the device data and you selected a large language model of choice.

What comes next? What do you harness the power of how do you harness the power of of those large language models?

Yeah. So, in the future, we're envisioning joining more and more data sources to achieve a more cross-sectional view of the user, which would allow a more holistic analysis and better coaching.

Um, and here we're aiming to do a few different things.

One is to detect issues with the therapy such as mask leak and other issues.

Um, we also want to screen for co-orbidities and other health issues uh to really kind of address the person as a whole and other things that might be going on.

And then third uh is to personalize the therapy itself.

Right?

Everyone might need a slightly different tweak to the p uh to the therapy. So we want to help here.

And as part of this, we're looking into large models um that can deeply understand the signal data from wearable and uh medical devices um like our CPAP machines.

And we believe that combining the power of large language models on the one hand with the insights of signal foundation models on the other hand um can really create a next generation AI system that delivers this intelligent proactive and personalized health interventions.

Right? So going from coaching to kind of a more intervention um enabled system.

Yeah, >> that's the full end to end.

That's very interesting. And how do you approach this problem on even more technical lens?

>> Yeah, so one step would be to really deep dive into the hidden patterns that are in this complex cross-sectional longitudinal signals.

Um so wearable and medical devices device data um are sequence data like time series data just like language.

So we can use the transformer architecture to apply it to this multimodel health um signal data.

And there are some research papers out there that show that these large models can outperform traditional machine learning models in certain tasks.

And there's several reasons for this.

Um but I want to maybe call out four major reasons uh why that is. The first one is the unified multimodal representation.

So this means that the model can take and ingest um and learn from multiple data types at once capturing interactions across the data types um so across modalities uh that some traditional machine learning model miss and then a second reason is uh the automated feature learning.

So instead of relying on handcrafted features large models learn rich representations directly from raw or minimally processed data.

um reducing information loss.

So that's that's a big one. And then the third one is um the self-supervised pre-training is pretty related, but um you know by leveraging this vast amount of unlabeled data, the models can build these strong generalizable foundations and understanding of the data um requiring fewer labeled examples later for fine-tuning compared to traditional machine learning models. And then last but not least, the fourth reason will be the advanced sequence modeling.

So transformer architectures excel at capturing long range dependencies in time series data enabling a deeper pattern recognition in wearable and device data because you know we we do have data for longer times of our patients.

So we want to make sure we you know we capitalize on that.

And yeah, for all of these good reasons, we're exploring these models um to and apply them to our users health data.

>> Thanks for sharing and and I think this also illustrates the what we're sharing right now on the screen. um really the power of those large language models like you mentioned going from traditional to now harnessing the powerful benefits of those large language models and uh we covered a lot from the business the technical even diving deeper now into large language models.

I know that ResMed has been working on quickly going through the motion of ideiation. Um that would be like business pain points then prototyping.

We walk through that too.

And then when we're building a PC thinking about if you want to put those PC's pro proof of concepts into production if they're successful um a lot of customers come to us and say how is that possible? How can you achieve that?

So I want to hear from you what helped you and your team, your organization come together to really drive forward towards a northstar when it comes to a IML and generative AI.

Yeah, we um really wanted to accelerate the adoption of AI and especially Gen AI as you know it kind of the transformer models came out a couple years ago.

Um and to spearhead this we founded the generative AI center of excellence right so we call it Genai CEO um and we kind of followed a fivepillar strategy one being governance standards and guidelines processes um platforms products and workforce adaptation and education.

So all of these five pillars are really important in addressing the overall adoption of a new technology and yeah executing on this strategy we developed the first risk assessment framework for genai applications together with a framework for guard rails and platform support.

These are three really important things um to make it kind of scalable across the organization.

Um and yeah so it makes sure that we identify the priority risk categories we address them with the guardrails in a way that is scalable and we are also able to quantify the results with the right platform capabilities.

So this last point is really interesting because when it comes to guard rails, we want to be able to test their efficacy.

Um and you know there are platforms available to do this and that really accelerates um the time to market for Genai products and yeah I think we are actually one of the first met companies that had a Genai companion.

So we're really proud of that and yeah to further um accelerate adoption of GI we collected use cases across the organization and help launch proof of concepts um to get teams up and running.

Um but at the same time we are also developing these architectural standards and reusable code to reduce that time of market and help with standardization of how we do genai across the organization.

So not every team needs to reinvent the wheel.

Thanks. This is an amazing blueprint that a lot of of others in the same journey of going through Genai how to scale it across organizations can leverage and congratulations on going live and in production with the application.

Um usually we like to close our sessions by uh having our guests share a little more on their technical career development tips. Uh if there's any resources or career advancement tips that you would like to share with the audience, all of them are welcome.

Sophie Lumina, I'll give you the floor to share a little more on that end.

>> Yeah, happy to. Um yeah, if you're really early in your career, um I would definitely um you know, think about if you want to do a technical career or if you're kind of more interested in the product of things. But if you are interested in a technical career in AI, meaning you want to kind of become a data scientist, a data science manager and kind of grow uh in that technical role, I would seriously consider getting a PhD.

Um so a you know doctorate um these research methods that you learn are really invaluable and open doors later on.

Um if you don't do a PhD that's probably fine if you can get research experience in some other way, right?

um it doesn't have to be maybe the whole five years but maybe you know some research um uh projects here and there are really really helpful to kind of get that scientific thinking going and then yeah if you're looking into machine learning engineering and AI engineering that is really a growing field especially now with Genai and all these agents out there um this is a great opportunity for software engineers who want to transition into the AI space and um I would say it's probably relatively easy to skill up on those skills and kind of you know um expand into that field more. Yeah, maybe as a last tip I try to stay engaged in the AI community at universities and online and it's just a great way to stay up to date on share business ideas but also learn about cutting edge art architectures and yeah just connect with your friends.

Thank you Filima uh for all those tips for sharing so much about what's happening in the medical space at restnet.

We learn also about large language models and all the technology that it's powering to provide uh better health to those patients and so many people getting better sleep.

So just wanted to uh say thank you very much for joining us and also thanks to the audience for staying with us.

Hope you learned a lot. Today we have other sessions coming up. So stay tuned on our channel.

And yeah, thank you very much everybody.