The Lazy Loading Pattern: How to Make Python Programs Feel Instant

I have a script here that loads data from a pretty large CSV file. As you can see, there's like millions of records in here. Now, I'm doing this in a pretty

here. Now, I'm doing this in a pretty naive way. I'm simply loading it here.

naive way. I'm simply loading it here.

And then I have some functions to analyze sales, count the number of sales, and there's a simple textbased user interface.

This is what happens when I run this code. So, as you can see, it loads all

code. So, as you can see, it loads all that CSV data, and that actually takes a pretty long time. So, the program is not frozen. It's working, but it's loading

frozen. It's working, but it's loading these millions of rows before doing actually anything useful. This is a classic example of eager loading. But

today, I'm going to fix this by using the lazy loading pattern and show you a couple of things that you can do.

Actually, the concept of lazy loading dates back to already 1960s when operating systems began loading memory pages only when accessed. Martin Fowler

later popularized it in patterns of enterprise application architecture, describing how OMS like Django delay database queries until the data is

actually used. And you probably also use

actually used. And you probably also use this daily yourselves by using uh Django, SQL Alchemy, TensorFlow and even Pandas rely on it to handle large data

more efficiently. And there's many other

more efficiently. And there's many other real life examples of lazy loading as well in uh websites that already show the UI before the data is actually there or operating systems that show you the

login screen while it's still in the background loading all sorts of stuff.

So let's see how you can use that same principle in Python code. Now before we start, if you want to learn more about how to design a piece of software from scratch, I have a free guide for you at

iron.codes/design

iron.codes/design guide. This walks you through the seven

guide. This walks you through the seven steps I take whenever I'm designing a new system. So at the start of the

new system. So at the start of the video, I already showed you the eager, naive approach to this. And as you can see, we now finally have the user interface. So now I can do things with

interface. So now I can do things with the data like I can analyze the sales data which as you can see is a huge number of dollars or I can count the

total number of sales records as well using that same data and of course now the interaction is pretty fast because we already loaded the data. Quitting is

also really fast which is great. Now of

course you don't want your user to wait 10 seconds just to get to the menu. It's

not great UX. So let's make this more responsive by deferring that loading of all the data until it's actually needed.

So if I go back to the main file, as you can see, what happens here is that at the start of the main function, I actually load the sales data. So a very simple way that you could do lazy

loading is by basically taking this and moving it to the point where you actually made a choice like so. And now when I run the main

like so. And now when I run the main file again, I get the user interface immediately. Now, of course, as soon as

immediately. Now, of course, as soon as I then pick one of these options like analyzing the sales data, well, then we're back to loading all of that data again, which simply takes a lot of time.

And in fact, we made it kind of worse because now if I pick the second option, counting the total sales record, well, it's going to load the CSV data again.

So it seems like we traded that initial loading time that we had when starting the application for something that's arguably even worse because now the user always has to reload all that data every

time and that's also not really what you want, right? At least quitting is still

want, right? At least quitting is still fast. That's something one thing you can

fast. That's something one thing you can do to avoid having to load the data all the time is to combine the lazy loading pattern with caching. And that's

actually pretty easy to set up in Python because font tools built-in standard library already has caching possibilities. So from font tools I'm

possibilities. So from font tools I'm going to import the cache decorator. And

if you want to cache loading sales data then it's actually really easy. We can

simply write cache on top of this load sales function. That's all that it

sales function. That's all that it takes. We don't have to change anything

takes. We don't have to change anything else here. So let me show you how that

else here. So let me show you how that actually works. So I'm going to start my

actually works. So I'm going to start my main file again. Now of course I have to uh pick an option again. So I'm going to analyze the sales data. So at this point it's going to load all that data from

the file which is what we want, right?

We need the data in order to do this.

And then it computes the total sales.

When I do this again, it has cached the value. So it's way faster. And then if I do count total

faster. And then if I do count total sales record, it's also fast because it doesn't need to load the data that has been cached. So as you can see, lazy

been cached. So as you can see, lazy loading combined with caching is actually a really powerful feature and you can implement it in Python in a way that doesn't really break the design too

much by relying on the cache decorator for example. So let me quit again.

for example. So let me quit again.

Quitting is still fast, although it's not that fast actually. I'm not sure what it's actually doing. Ah, there we are. So quitting is now actually slower

are. So quitting is now actually slower and that's probably because Python needs to do some clean up, clean up the memory before actually quitting the application. So as you can see, lazy

application. So as you can see, lazy loading caching together works really well. But there are some cases where you

well. But there are some cases where you do want to lazy load things, but actually caching doesn't really work all that well. For example, let's say we

that well. For example, let's say we have a function that gets conversion rates between different currencies and it uses some external API

for that.

So this is going to give us a dictionary of strings to float values and we're going to return just a few random conversions. Let's say the base

random conversions. Let's say the base currency is US dollar. So there the conversion is one. We have euro which is 1.1. Not sure what the current

1.1. Not sure what the current conversion rate is. And let's say we also have the Japanese yen and that's I

don't know something like this but going to print some info here.

So normally this would be an API call, right?

And let's simulate that.

by sleeping for a couple of seconds.

This is a very slow currency conversion API that we're using. And then what we're going to do in analyze sales is that we're going to convert the total to the currency that we want. We can

actually also rewrite that using some smarter Python syntax. So, we're going to take the sum of a float conversion of

the amount field in the data for S in the list of sales like so. And

then we don't need all of this. But what

we can do now is we get the rate and that's going to be get conversion rates for some currency. And let's also

pass that as an argument like so.

And if it's not there, we're simply going to return 1.0. So no conversion at all. And then we're going to return

all. And then we're going to return total times the rate. So now our sales analysis allows us to modify the currency and it uses this external

service to get actually the conversion rate. Of course, we also have to change

rate. Of course, we also have to change the UI for that. So if we want to analyze the sales data, then we're going to get the currency

as an input like so. And let's make this slightly

like so. And let's make this slightly more robust by automatically uppercasing it. And if there's no response, then

it. And if there's no response, then we're simply going to assume that the currency is going to be US dollar. And

then analyze sales.

gets the currency as an argument and then let's also write here total sales in currency like so there we go let's run this see

what it does so first the option is to analyze sales data we enter the currency and now it starts loading the CSV data just like before which unfortunately

still takes a while but in a minute I'm going to show you something you do about that? So it fetches the conversion rate

that? So it fetches the conversion rate which also took a little while and then it computes the total sales in that currency. So if we do that again then of

currency. So if we do that again then of course the data is cached but the conversion rate is not cached.

So you could do that. I could decide to also cache the conversion rates and basically do this right. The problem is that conversion rates change over time.

So if we simply cache them, it means that now basically until we restart the application, it's always going to use those same conversion rates. And well

here, of course, in the particular example, I'm kind of faking it. But if

it's a remote service, then caching the result may actually lead to all sorts of problems. But there we can perhaps be smarter about how we are going to cach this. And what you can do is actually

this. And what you can do is actually assign timing to this. So here's how you can actually do this. I already created a function for this and it's called TTL cache. This basically puts a time limit

cache. This basically puts a time limit on your cache. So it's a decorator just like the regular fun tools cache decorator. But this decorator gets a

decorator. But this decorator gets a number of seconds and after that it's no longer going to cache the result. So

basically how it works is it has cache data and has the cache time. Then it

checks whether the cache result is still valid. If so it returns this and

valid. If so it returns this and otherwise it recomputes this and it stores this. And by the way, I didn't

stores this. And by the way, I didn't pay any attention here to make this thread safe or anything. It's really

just an example of how you could in a simple way set this up. And now what we can do in case of the conversion rate is make this a TTL cache. And we can give

it like I don't know, let's say we want to refresh our conversions every 60 seconds. Then this is how you can do it.

seconds. Then this is how you can do it.

So by setting it up this way, you have lazy loading. You only load things when

lazy loading. You only load things when you actually need them. you cache

results and when you work with a remote service you can also cache things and then only reload them if you actually need them. So in this case we're okay

need them. So in this case we're okay with being at most 1 minute behind in terms of conversion rates. So when you run this again

let's analyze the sales data one more time fetches conversion rates and then it gives you this total number of sales if we do this again.

So now you see it was way faster. So it

basically didn't need to load the data that was already in the cache and also it cached the conversion rate. But then

if we continue this after a minute the conversion rate cache clears and it's going to reload this. Now there is one more way you can approach this and for that I'm going to remove the cache

operator here and that is that sometimes you may not actually use all the data.

So for example, in case of loading the sales data, it's possible that you only want to know sales numbers of the first 10,000 records or something. And then of course, you don't need to load in all

those 10 million records to actually do the computation. So what you can do in

the computation. So what you can do in that case is not let load sales return a list, but actually use the generator mechanism in Python. And that gives you

way more control over when you actually need to load that data. So, we're going to return a generator and this is what

we'll import from typing like so. And of

course, uh the generator is a bit of a complicated type to specify, but this is basically what we're going to return.

And there's some other options here as well. I did a video covering generators

well. I did a video covering generators more in detail. I also talk about how this type annotation actually works.

Won't spend too much time on it here. Uh

let's remove this because that's no longer true. But uh what you can do now

longer true. But uh what you can do now is actually instead of returning all of this data is to actually put the for loop here

like so.

And then we're going to yield the row.

So now we're not loading everything here, but we supply a generator which gives us way more control. For example,

what you can do now in analyze sales, let's take this loading option here and move it to analyze sales. We're going to refactor this in a minute. And we won't

use this as an argument.

We'll also remove it here.

And actually, if we want to do this cleanly, we should probably supply a path here and not put it hardcoded here.

Like so. So now I can use the generator version of load sales to actually only load the data that I actually need. So

let's say we start with the total is zero and then we're going to use the enumerate function and what is sle

sale enumerate over load sales let me supply the path

and we start at one and then total plus equals the floatingoint conversion value of the

sales amount. So we delete this. We also

sales amount. So we delete this. We also

delete this. Then if

I is let's say larger than 10,000, let's assume we just want the first 10,000 records.

Then we're going to break. And then we still get the conversion rate and we return the total times the rate. So now

we're not only lazy loading, we also only load the data that we actually want.

Let's run this one more time. We analyze

the sales data, US dollar. And as you can see, loading is now much faster because actually we only load the data that we need. So there's various strategies that you've seen today. We

can use caching to make sure we load things only once. we can make sure to not load everything at the start of our application so that we can already interact with the application without

having to uh wait for 10 seconds before we can actually start. Um we've seen an example of loading only the data that you need by using generators. So there's

a lot of techniques you can uh use to make your program your code feel more responsive um while still actually loading the data that you need when you need it. And that is the essence of the

need it. And that is the essence of the lazy loading patterns. It's not really a design pattern per se like strategy or abstract factory. It's more of a way of

abstract factory. It's more of a way of thinking of how you're going to set up your code so that it runs in the most efficient way possible. Now there's even more that you can do and that's al

something that you see used a lot in operating systems and websites which is that you kind of become smart about the data that you actually load but not lazy

loading but preloading actually loading things already in advance before you actually know that you're going to need them. And what you could do here is

them. And what you could do here is basically as soon as you show the user interface and while the user is still thinking about which option they want and they're typing in whatever they need, you can actually already start

loading data in the background. And

here's an example of how you could do that. So I already wrote a version for

that. So I already wrote a version for this because I don't want to code all of this from scratch. It will make the video way too long. But Python of course has support for threading and um

threading combined with uh the UI works actually really well in Python. So what

I did here, I still have all these things that are cached. It's almost the same as uh what you've seen before, but I have a function that's called preload the sales data. So I'm not using a

generator here. I'm simply preloading

generator here. I'm simply preloading it. And within that I define a function

it. And within that I define a function that actually uh calls computing the total sales and the sales count the thing that we did before. And then it starts a thread to actually do this. And

in my main function I call that function so that it starts the thread. So when

you run this then here's what happens. Now it's coded a bit dirty because it prints this stuff as part of the user interface. It didn't

really do a good job there. But as you can see, as it already shows the UI, it does this in the background. So while

I'm thinking about what I want, it already has done it. So I can now analyze my sales data. Let's say I want to convert to euro. It still fetches those conversion rates, but it has

already loaded the data. So that means this version has lazy loading. So it

shows the UI immediately, but it's also smart in that it preloads things and then it caches those results. So there's

a lot of different concepts you use together here. So when to use lazy

together here. So when to use lazy loading? Well, especially when you're

loading? Well, especially when you're loading a large amount of data, obviously if startup speed is important to you if you have some operations that

are uh rarely used. So preloading

everything would be wasteful. But

there's also some disadvantages to lazy loading, which is that it may add some extra complexity and perhaps you don't really care. It makes in some case the

really care. It makes in some case the behavior of the system a bit harder to predict, especially if you preload your data in a separate thread like I shown you just now. And by adding stuff like multi-threading, you may actually

introduce other problems because now you also have to make sure that your code is thread safe. So it's definitely not for

thread safe. So it's definitely not for everything, but know that this is a possibility uh that you can do. Other

problems you may encounter with this pattern is if you cache uh lists or uh dictionaries that change later on, you may get inconsistent results. And in

terms of caching, you have to be careful to not cache functions that depend on external states like APIs. Now, I've

shown you one way you can address that by using a time limit on your on your cache, but this is then something that you need to think about. Final thing to be mindful of with the lazy loading is that it can actually hide performance

issues if you're not careful because a property access might suddenly trigger a big query or uh loading some big file that you didn't expect. So use this

sparingly where the performance gain is really important and not just because you can. Like I said, lazy loading is

you can. Like I said, lazy loading is not really a design pattern, but it's more of a principle to delay uh cost until it's absolutely necessary. And

when you combine this with caching, it's a really powerful way to optimize performance without over complicating your architecture.

But I'd love to hear from you. Do you

use lazy loading or caching in your projects? Maybe in Django, Pandas, or

projects? Maybe in Django, Pandas, or your own APIs? Let me know in the comments. And if you enjoyed this video,

comments. And if you enjoyed this video, don't be lazy and actually click that like and subscribe button because that really helps my channel. Now, if you want to explore more software design

videos like this one, check out my design patterns playlist right here.

Thanks for watching and see you next