DECI : Distributed Eventually Consistent In‑Memory Cache

Hey. Hi everyone. I am H and today in this video we will be discussing about what is DESI, what problem does it solve

and how to build and run it. So DESI is distributed eventually consistent inmemory cache.

So uh what does it mean?

Okay. So I have already prepared some diagrams uh to save some time.

Uh okay. So we all must be familiar with this kind of architecture where uh our

application makes a network call to get the data from radius uh and the we use the result of it to either make some

database call or return to user or whatever. So uh in this architecture we

whatever. So uh in this architecture we have multiple instances of our application. The radius is deployed on

application. The radius is deployed on another server or is provided by some service by uh the cloud providers like

AWS, GCP, Azure or a service like elastic cache. So our application is

elastic cache. So our application is making a network call to that server. It

returns the data and based on the data we take further action.

Okay. So uh radius the best thing about radius is that uh it is an in-memory cache.

Okay. But here if we see in this architecture we are making a network call from our application to the address instance.

So that completely uh erases the benefit of having an in-memory cache uh since we are making the network call.

So uh let's say uh let's consider on an average uh the latency to get the data from the memory

is around 1 microcond or 10 microsconds and to get it from the uh hard disk it is around 1 to 10 millisecond.

Yeah, there is a order of magnitude difference. But since we have introduced

difference. But since we have introduced a network call uh in between to access the cache, the whole benefit of having it in memory is lost.

Uh okay. So how does DESI approach this problem? So we are desi tries to solve

problem? So we are desi tries to solve this uh ele tries to eliminate this network call uh and

uh by introducing a local cache process.

So the same thing is uh written on this uh desi homepage. So uh it says here it keeps the cache directly alongside the

application process on the same machine.

Okay. So how does it uh do it?

uh so uh let's consider this is our server and uh where our application is deployed

and now there has two components one is called LCP another one is GCP the LCP is here and GCP is here so LCP

is local cache process and it is global cache process GCP uh so let's get Come back to our uh previous diagram. So this is our server.

previous diagram. So this is our server.

Our application is deployed here. Now

the whole cache is maintained in memory uh using local cache process and uh using the IPC interprocess communication

via a unique socket our application access that inmemory cache and uh use it however it wants. So we

have completely eliminated the network call uh the network call that we were making the here to access the cache.

Yeah. So now uh you must be having one question like uh okay so that is in memory for a for a single instance. What

about instance two or instance three? Uh

in this architecture it is the source of uh uh the whole cache. uh every uh any change or update made to this uh cache

uh is visible to all of the instances of the application and they can all access the cache from this single instance.

Uh this is where the GCP part of the DCP comes in picture. Uh okay. So GCP stands for global cache process and uh it has

two functions. one is to maintain a

two functions. one is to maintain a global cache and another is to act as a synchronizer between the LCPs. So how it

does that? So let's come back to this uh

does that? So let's come back to this uh another diagram. So we have again

another diagram. So we have again multiple instances of our application.

Each has their own LCP uh deployed alongside it uh as a process uh as alongside your application

process. So if there is a change made

process. So if there is a change made from application instance into the LCP, LCP communicates it to the GCP and GCP

fans out pushes out that change to other LCPs to other peer LCPs of the same application.

Okay. So each LCP maintains a pool of connections pool of connection with the GCP. All of

these are TCP connection. And whenever

application makes some changes uh in uh the local cache uh any create update or delete change which changes the state of

the cache it propagates this change to the GCP and then GCP again fans out these changes to the other LCPs of the

same application.

That's the synchronizer part of GCP.

Also GCP maintains a global cache uh internally which can be accessed uh by all the LCPs irrespective of which application they belong to. So this

again acts like uh this single source of cache. So let's say if we if the data is

cache. So let's say if we if the data is huge or if there is some shared data that needs to be uh accessed by different microservices

uh in our u uh in our system uh then uh that can be stored in this GCP and can be accessed

uh via network called like we are we are used to access it uh in this kind of architecture.

Okay. Uh so uh that's what uh LCP GCP is. And this uh diagram here shows uh

is. And this uh diagram here shows uh like uh so uh let's say this is our server one, this is server two and this is

another server which hosts global cache process.

Now GCP uh whenever a application connects with GCP it connects using a group let's say uh this application and

this application is uh an instance of a user service.

So it creates a group user service group and uh communicates synchronizes uh through this group. So groups are the

name spaces in the global cache process uh using which multiple microservices uh can have their own separate space

uh like payment service can have another group or there is some group three for another service or something like that.

Okay.

So uh let's uh uh so you can go through this uh whole uh information on this uh homepage. So

that's uh you can access the whole implementation and the design at this uh uh repo and there are docker images for

both GCP and LCP.

Uh also the client libraries are there.

Currently it is only for NodeJS. Uh for

other languages the uh development is in progress. Uh so you can use it to access

progress. Uh so you can use it to access uh both the LCP and GCP. We will be going through it uh soon. Uh so this is

how you can run them as containers. And

if you want to build it from source or include it in to your pipeline uh without using the container then you can do so from uh using these steps. Okay,

these are just the configuration files.

Uh okay then uh let's uh move on to some code.

Uh okay. Uh first I'll be opening the uh Docker uh Docker Hub pages for both

GCP and LCP. So this is for GCP and uh if we go to the next one, it's for LCP.

Uh okay. So currently uh let me switch to the terminal and let's see here. So currently we don't have any uh container running.

I'll be you running both the GCP and LCP on the single machine u as containers.

Okay fine. Uh you can find more the information uh all the informations here which directory is what uh where the configuration file is wall file is and

uh the log file. Okay. So I'll be using uh this command. Uh this will mount the log directory uh so that we can have

persistence persistent log and also the wall file that is write ahead log.

Okay. So let me first uh run it. So what

it does is uh docker run name is GCP detached mode and the mounts are uh it mounts the GCP um in temp uh to the to to this location

also the wall file and the port exposed is 7480 and uh it goes to uh that is the image name. Okay, let me hit it. So I have

name. Okay, let me hit it. So I have already pulled the image that's why it just ran it. So if we see here now it

says this is running and let me uh uh show the logs also A69.

Okay. So our GCP is started. Now let me also start uh LCP.

So we'll go to this page and we'll again uh use this command. So we'll be mounting the logs directory. Uh okay let

me just do that.

Okay so let's hit it and our uh LCP is also started and it communicated with GCP. So you just saw it through the

GCP. So you just saw it through the logs.

Uh now I won't be going through the architecture for how it works internally the LCP as well as GCP. uh we'll just go

u uh tell we will just show like uh how does it work through the code okay now uh I'll switch to the NodeJS

library the client library so I have already pulled it and uh written some basic CRUD applic uh CRUD operations like

uh get set delete exists uh so uh DESI uses a subset of rest protocol that is radius uh

synchronization protocol.

Uh so uh it is easier to uh understand and uh integrate in your existing applications. Okay. So here I'm just

applications. Okay. So here I'm just importing the desi and uh making a connection. Uh so we

have just uh so let me just change it to okay u LCP and uh uh I'll explain it why this is

stem/lcp.sock

stem/lcp.sock and uh we'll uh get to know more about it. So we have just est here we will be

it. So we have just est here we will be establishing a pool of 10 connections to this socket

and uh let's will see about more about it. So there are some test cases that I

it. So there are some test cases that I have written already. So this file just exposes a single uh function that is test which uh uh which which we'll be uh

using to uh understand more about it. So let's

see. So first let's go to the CRU test.

So first we are making a okay before this before this let me show you what uh what does it actually mean.

So this is uh this is the client library for desi.

Uh yeah it is uh just published. So uh

that's fine. Leave it. Uh so you do this with the import dye connect to this and then these are the CRUD. So there are two kind of queries LCP queries and GCP

queries. Uh this will be accessing the

queries. Uh this will be accessing the global cache and this will be accessing the local cache. So uh this is set set the name

value as amigo get the name delete the name and if name exists or not. uh same

kind of queries are on the global cache and uh we'll go through this uh a little later.

Okay fine. Uh now uh let's go back to our uh code. So now we are we have already made a connection here. Then we

will be executing this function. What it

will be doing first is uh it will be fetching the name from the cache which it will not found. So it should be null.

Then it will be setting it and we will be getting uh the response as one. Then

it will again fetch the name. We should

be getting the name here. And then it checks if the name exists or not. Then

it deletes it. And here it checks again.

So we should be getting the zero because we have already deleted it. And this is something we are setting another uh key

test and we'll see the use of it. uh a

little later. Okay. So this function is called CR test and let me execute it now.

So I'll be just doing node index.js and here we go. So what uh it has

established these 10 connections and uh for the get name we got the null set one and since we have already set the name

we got the name here hers. Then does it exist? Yes. And now we have deleted it.

exist? Yes. And now we have deleted it.

The response of delete is one. Then we

again check whether this does it exist again no. And this is the set timer. We

again no. And this is the set timer. We

will know more about the set time test. Sorry. Uh so these are the logs

test. Sorry. Uh so these are the logs that we just got here and DCP maintains a right ahead log and it runs in a separate thread. Uh we'll go through the

separate thread. Uh we'll go through the internals of both GCP and LCP in a separate video or it will be too long.

So yeah so it will maintain it here and we can okay let's uh for the completion let's just check it. So I am in the temp directory and I'll go to the GCP. So it

has to be mounted log and wall files.

We'll go to the wall one. And here we have created this desi.wall. Uh I have created uh previously desi.2. Let me

remove it.

Uh yes.

Okay. So

um let me just do this.

So we um so the last command we ran was test and we stored some time stamp in it. So

that's what it is showing. Let me

increase the number of lines here. So

before that we deleted uh the name and that is showing it here. Okay. So that's

the writer log file. Let's leave it as it is.

Uh okay. So now that's how we are accessing our LCP. So we are making the operations in LCP using the client

library and uh it is uh making the CR operations like u setting the name, getting it, deleting it or checking if

it is exists or not.

Okay. So now uh let's go and uh create another uh LCP

uh to simulate this kind of uh scenario uh which one uh this one. So multiple application

instances synchronizing via GCP and are able to access the GCP.

this one or this one both are the same images. It just shows a lot of

images. It just shows a lot of information.

Okay, let's go back. Uh but first uh now we need to go through what is the configuration file for LCP and u how to run it. Since we are doing it on a

run it. Since we are doing it on a single machine, we'll have to make some uh changes.

So let's uh so where are this is for GCP? Let's check it here. copy this

GCP? Let's check it here. copy this

command and uh paste it here. Now before

running it, so let me just zoom it. Uh

before running it, we need to make some changes because we can't be using the same folder for running another LCP. So

here I will be going first to temp and making another folder LCP2. Let's go to this folder and uh

uh okay. Now

uh okay. Now let's LCP docon configuration file.

Let's go back to that. Copy this whole or not let's not copy it from here. Copy

this LCP.con file and uh paste it here. So what is it? We are

connecting to the group. The group is desi here. And what does group represent

desi here. And what does group represent is it represents uh this the group here. So if I change the group then uh this another LCP will

be registered here. But uh we are u checking for the same group the synchronization and everything. So we'll

be registering it here.

Okay. So that is the group. How many

connections? We will be making 100 connections. LCP name it is. What is the

connections. LCP name it is. What is the socket? So we have already used temp/

socket? So we have already used temp/ LCB sock. If you go to here and come

LCB sock. If you go to here and come here, so we have already used it because that's the default uh uh configuration and we have not changed it when we first

ran LCP uh LCP1. So let me change it to LCP2. It will be easy. Uh same thing

LCP2. It will be easy. Uh same thing here though it does not matter as of for now. Uh then these are some

now. Uh then these are some configurations that are used internally.

So total sync messages uh how many connections does it make with GCP? So

100 it maintains a connection pool of 100 connections. What 100 connection

100 connections. What 100 connection pool this kind this connection pool.

So you can configure it from here.

That's the ser GCP server IP and what port it is running on and uh the maximum GCP connections. Okay, let's

just uh close it for now.

Okay, now we'll go back to our the docker command and first of all we will change the name to LCP2

and we are still mounting the temp directory. Now we'll be making this LCP2

directory. Now we'll be making this LCP2 and I have uh copied the wrong command. So you have

to copy this one. So since we will be using the configuration file as well uh so we can okay let me just copy it again

and let's do it here.

Okay so we'll be changing the name to two and uh all this configuration and the user and the group ID uh we can we

will be discussing uh soon. So let's

change it to LCP2 and we have created a LCP.com file in temp/lcp2.

Uh we can check it here. So printing

directory is that and uh what we have here is this. Okay. So that's what I am mounting it here templ2cp.com to the location where it is present in

the docker container. And that's all. We

should be good to go or yes. So it connected with GCP and uh maintained a connection of

created a connection of uh uh connection pool of 100 uh TCP connections

and uh it uh uh so we just saw if I go back to the GCP one and uh go to the wall file. So we have the wall here and

wall file. So we have the wall here and uh what just happened is it requested the wall file from GCP. GCP read that

wall file transmitted it to LCP2.

So whatever the changes we made in uh LCP1 let's say we are scaling our application. So it will uh reach that uh

application. So it will uh reach that uh uh reach that cache state uh by uh requesting the wall file from GCP and

will be at the same state.

Uh we can check that actually. So let's

uh connect to LCP2 and uh instead of so now instead of running the CRU test again we will be changing it a little bit. So if you

remember we just uh we did all the CRUD operation on name. So we tried to fetch it first then we set it again fetch it.

So we got the name here. We checked if it regressed we deleted it. And here we got the value of zero because we just deleted it here.

Uh and at the very end of our CR test we set some test key here.

uh and uh added a time stamp to it. So

first of all, let me uh first check it with the LCP1 that we just ran and instead of going with the uh CR test, we

will just fetch the uh test key. So we

we must be getting some value here. So

let me run it. So we got uh this 678.

And if we uh check our uh wall file, so uh this is also 678. So that matches, but uh we

also 678. So that matches, but uh we just checked it in the uh the same LCP that we made the change. Now this change

should also be present in the uh LCP2 that we just started because of the wall synchronization.

So we what we just did is changed con changed the connection socket to the socket connection of LCP2 and we'll be uh running our script again and yes

indeed it is present here.

So that's how the synchronization works between uh uh the LCPs of the same group

choosing the GCP. Now let's uh do a more uh involved uh uh test.

Okay. So,

what should we do?

Uh, okay. So, let's first uh make some changes here.

Let me comment it out.

Okay, before uh going there uh let's just uh let's also check about the u global cache. So here there is another

global cache. So here there is another function called lip test. What it does is again make some uh LCP queries like sets the name to amigo. Get it, delete

it, check if it is exists. So we will be getting one here.

We will be getting amigo here. we will

be getting one here and we should be getting uh zero here.

Okay. And now here we are making the call to the GCP accessing the global cache. Every command that exists with G

cache. Every command that exists with G uh G set, GET, G exist or GD delete uh comes from is for the global cache. So

you can check it here.

So these are the LCP queries. These are

the GCP queries. Uh G set G get G del G exist and for LCP there is no G.

Uh okay. So uh we should again be getting one here and uh global amigo should be here and uh we check exist

then it should be one here.

So we will be doing these operations on our LCP2.

Okay, let's uh do this and let me run it. So yeah, here here it goes. It is established again a pool of

goes. It is established again a pool of connections. Then the data is for one um

connections. Then the data is for one um what one amigo 1 and zero. Let's make it will make more sense something like

this. So we got one amigo 1 0 1 amigo 1

this. So we got one amigo 1 0 1 amigo 1 0 then one then it should be global amigo and then again one. So yeah it uh

accesses both uh the cache from our local process and also from the global cache. So here it uh does that.

cache. So here it uh does that.

Now let's uh do one more test where we will be making some uh we will be setting some uh key uh key value in our

LCP2 and it should be propagated to LCP1 through uh GCP.

Okay. Uh so that should happen in real time instead of uh uh reading it from a wall file. uh and uh these

wall file. uh and uh these uh these TCP connections uh will be used uh in this synchronization.

Okay. Uh so the process is application process will make a change in LCP. It

will propagate that change to GCP. GCP

will uh push that change to all the peer LCPs. So currently we only have two

LCPs. So currently we only have two LCPs. Let's say if there are four or

LCPs. Let's say if there are four or five LCPs something like that then it will push that change to all other LCPs

it internally maintains multiple cues to handle that. So all LCPs can push uh

handle that. So all LCPs can push uh simultaneously push changes to uh each other. So GCP handles that kind of

other. So GCP handles that kind of synchronization.

Okay. So first we will be connecting again to LCP2 and let's make some change

to it and let's write some command here.

So await dy dot uh set and it has

it takes key value and sync. Oh let me

make it here something like this. Okay.

So key. So let's say key name is uh uh ABC. Not a very creative name but it

uh ABC. Not a very creative name but it is what it is. So value is uh uh let's say xyz.

Okay. Let's just keep it simple. So now

we will make this change and the output should be uh let's con output is this and console.log log

the out.

Okay.

Okay. So, we got output as one. So, I'm

not exiting the process. Uh that's why it's uh hanging there. But, uh that's fine.

Uh okay. So, now if we check the logs here. So, this is LCP2. This is LCP1.

here. So, this is LCP2. This is LCP1.

This is GCP in uh all in LC. So, there are also multiple threads. uh used in both LCP

multiple threads. uh used in both LCP internally and GCV which maintains separate uh pool of connections. So LCP2

used something called global cache operations thread and it took that whatever we sent to this. So we sent ABC

XY Z pushed it using these uh this thread and some connect GCP connection to GCP

uh it received it here. So set ABC XY Z and uh it again uh looped the uh so it

uh push pushing for LCPQ. Okay.

Uh so first of all it wrote it to the wall file and also pushed it to other peer LCPs. So sending sync operation to

peer LCPs. So sending sync operation to other LCPs for producer LCP. This each

LCP has its own unique ID. uh so uh that way uh GCP can identify which one to push uh uh synchronization operations to

and there is some in if you look at the LCP1 so there is some cache uh synchronization thread uh it uh GCP

pushed the data to this and uh it uh must have received it somewhere here so this is not this is previous one so let

me scroll a little bit here yeah set A B C XY Z and uh uh it now it must have updated its cache as well. So let's

check that.

So what we will be doing here is we will be making another call. Let me

just write it a little comp in a compact way. Uh so it will be like await des.get

way. Uh so it will be like await des.get

get and we will be getting the key and what was the key name that was ABC.

Okay. So let's first get it from the LCP2 to see what its value is. So yeah

it's XY Z. No surprise is there. Let's

get it from LCP as well because it should be synchronized. So yeah and there it is. Yeah. So that works good.

Uh so that's how the whole operation and the components of TESI works. Now let's

take a step forward and u um let's uh let's what

okay so let's create another group uh let's call okay and uh we will be creating one okay okay before that before that so we have we are already

running LCP1 and LCP2 here let me create another terminal here and

Uh, make it a little bigger. Okay, fine.

So, let's run another LCP in the same group and check if it has all these uh changes that we just

did. So, it must have ABC and it must

did. So, it must have ABC and it must have the test. So, what test? this test

key which which we set when we first ran the okay

thread operations. So this 678.

thread operations. So this 678.

Okay. So now how to run it? So let me go to the GCP or sorry LCP copy this command

and uh go to the go run and let me zoom it. Okay

it. Okay now before running it will again make some changes. So I'll just say it LCP3.

some changes. So I'll just say it LCP3.

Okay. Then temp file logs will be here and take the configuration file from here. And yes, but before hitting enter,

here. And yes, but before hitting enter, we should create this as well. So I'll

copy it from LCP2 to LCP3.

Go to LCP 3.

And we here I should delete that actually.

Okay.

So I'll change it to LCP3.

And why we are making these changes because we are running on the same machine and if both different processes will point to the same token then that's

not will not end in a good way.

Okay. All other operation u all other um values here we have already discussed.

Now let me come close it

and go back to our uh docker command and let's hit run. Okay, so it established a

connection should have read the uh synchroniz wall file. So this is these are all the

wall file. So this is these are all the connections it made. So now let's go to our uh

uh NodeJS library and make a connection to this third one and let's call it something like this. So yes, so it

indeed has the test available and let's also see if it has the uh ABC uh available as well. So okay let's

do that and yes indeed and let's take it one uh step further.

So we will be making so now we have three LCPs running 1 2 3. This is the GCP. We will be making some change in

GCP. We will be making some change in LCP 3 and it should be in propagated to

LCP 1 and LCP 2 via GCP uh that we know from here. Okay. So,

let's do that.

Uh, so I'll just use this one. Okay.

this one. Okay.

And instead of this, let me use TEF and uh XY Z again. Okay. How about that?

again. Okay. How about that?

So, we will be making this to LCP3.

That's fine. Let's do this call. And

yes, we got response as one.

Okay. Uh now let's fetch this value.

First let's fetch it from LCP3 only. Yes, we got XYZ again. Now

LCP3 only. Yes, we got XYZ again. Now

let's fetch it from LCP1.

And here it is. Good, good, good. Now

fetch it from LCP2. And yes, we do get it.

I guess I'll stop here because it's already more than around 38 minutes.

Okay. So that's how the u whole um LCP GCP works in DC.

So yes that's how you can run it and uh I hope you find it useful and yes there are uh much more work to be done here

like client library there are some uh things to do in uh desi itself. So this

is just the version one multi more more work is needed uh something like u uh

stateful incremental parser for this uh protocol that uh desi uses and uh uh

detect if if due to some uh slow network uh GC uh LCP is lagging behind uh then to detect the synchronization lag and

again request the all file to uh get up to date.

Uh so yeah those thing uh will be coming soon. So till then I hope you find it

soon. So till then I hope you find it useful and would really love to hear your thoughts about it and u

uh thank you for uh listening to uh me I guess. So thank you and

let me know uh and uh reach out to me uh there. So you can reach out to me I

there. So you can reach out to me I guess. Uh so this is me toxen

guess. Uh so this is me toxen uh hush. This is uh just my name uh here

uh hush. This is uh just my name uh here in this repo. So yeah you can reach out to me at toxengmail.com.

Uh thank you.