Ollama Course – Build AI Apps Locally

olama is an open-source tool that simplifies running large language models locally on your personal computer in this course you'll learn how to set up

and use olama this Hands-On course covers polling and customizing models rest apis python Integrations and real world projects like a grocer lless

organizer rag system and an AI recruiter agency Paulo created this course hello and welcome to this mini course I'm very excited that you're here because I know

you want to get right in and start learning about AMA and building AI based Solutions using AMA first let me go ahead and introduce myself again which

is always awkward okay my name is Paulo deson I am a software Ai and Cloud engineer but most importantly I am an

online instructor and I have taught over 280,000 students all over the world in about 175 countries uh skills ranging from

building mobile applications Android to learning Ai and Cloud Technologies so this is what I do and I love what I do what is this minicourse about I want to

show you how to leverage olama and its many many models so that you can build AI Solutions a applications locally for

free usually the structure that I like to follow and has Pro proven to work is in the beginning or the first part of the courses I start with the basics the

fundamentals as well as Theory Concepts because I think it's important for you to also understand the theory before you go into Hands-On and then the second

part is usually Hands-On but sometimes I like to mix those two together so that it's more exciting and you don't get bored okay so I know you're very excited to get started and I'm excited that

you're here okay let's go ahead and get started well this course is going to be about ol that's the main thing so we're going to learn how to build local large

language model llm applications using AMA so the idea is that we're going to be able to use AMA to customize our models meaning that we are able to use

different flavors of models so we can test them around and all of that is actually going to be free which is very exciting and also we're going to be able to build rag system so retrieval

augmented generation systems that are powered solely by AMA models keep in mind all of this is going to be for free you're not going to have to pay for

anything okay and we are going to learn about tools and function calling when we are using these AMA models so we can Aid

our large language models that are going to be using with more things that they can do okay I'm going to look at all of that and ultimately I'm going to show

you how to use AMA models to build full-fledged large language model applications using these models okay keep in mind all of these models are

going to be for free we don't have to pay a dime to use these models now let's talk about who is this course for now this course is for developers AI

Engineers open-minded Learners machine learning engineers and so forth as well as data scientists so if you are

somebody who is willing to put in the work and wants to learn about AMA and build local llm applications then this

course is for you now let's talk about course prerequisites in this course I assume that you know programming this is a very um important skill obviously and

I expect you to at least know the basics you'll be seeing a lot of python in this course so understand understanding how python Works how to write python code is

obviously very important you also need to understand very basics of AI machine learning large language model you don't have to be an expert but just have some general

understanding keep in mind also that this is not a programming course because I'm not going to be teaching you how to program and most importantly I always tell my Learners is that you have to be

willing to learn and if there things that you need to go and brush on just go and do that and come back to the course but most likely you won't need to go

anywhere but continue in this course and learn as much as you can so most of my courses I have this mixture of two things I have Theory so this is where we

talk about the fundamental concepts the lingo and so forth and I have Hands-On because it's all about actually doing things that way you actually understand

and know how to get things done that's the whole point and so the thing is I'd like to do a mixture of both and sometimes we have more handson and

sometimes we we may have more Theory but I try to balance it all out where we have mixture both because I know I believe at least that is important for you to have both in your learning

process but in this course you are mainly going to be focusing on Hands-On although you will still have some Theory fundamentals as well all right so let's

look at the development environment setup so in this case you know that this is all P about python which means you'll have to have python installed and also

you have to have some sort of a code editor for me I'm going to be using VSS code so you don't have to use that if you don't like it but that's what I'm going to be using you can use whatever

you want okay now I'm not going to go through the process of installing python or vs code or anything like that because you should be able to have that if not you should be able to do that on your

own so one place that I always recommend people go is to this link here kin.com knowledgebase install python there are many places online where you can learn

or they will walk you through the installation of python if that's your case uh but this seems to be one of the best my favorite okay so if you don't

have python installed please go ahead and get that set up so that we can continue in the next next videos now we have everything set up on our machine and we're ready to go so we're going to

do a deep dive into AMA so you understand exactly what is how it works and what problem um AMA solves all right so let's go ahead and get started now

it's time for us to do the Deep dive in ama so we understand exactly what is AMA what is the motivation behind and the advantages that ama brings for us

developers okay so first let's look at AMA what is it ama is an open-source tool that is designed to essentially

simplify the process of running large language models locally meaning on your own Hardware the idea here is very

simple as you know right now if you want to run large language models or if you want to use a model large language model in this case just a model most likely

you'll have to use open Ai chbt and so forth and many others out there that are paid and the thing is with a Lama you don't have to pay for anything it's free

and that's the beauty so the idea is that ama sits at the center and allows us developers to pick different large language models depending on the

situation depending on what we want to do at its core AMA uses what we call the CLI which is command line interface what

this does is it manages is all the things in the back end so the installation and also the execution of different models but all of that again

keep in mind it's all locally you will see as we go through that ama abstracts away the technicality so the technical complexity that are involved when we

want to set up these models which makes Advanced language processing accessible to a broader audience such as developers

researchers and hobbyists in nutshell AMA provides a straightforward way to download run and interact with various

models or LMS without us having to rely on cloud-based services or even dealing with complex setup procedures rather what is the problem that ama solves now

when we talk about large language models we are usually talking about this thing called rag system retrieval augmented generation the idea is very simple is

that we have documents those documents are chopped into smaller chunks right and then we pass those through a model that is responsible for creating

embeddings which is the vector representation of these chunks those embeddings are saved somewhere and then a query comes in goes through the same

process of embedding right and then once we have this embedding that is indeed what's actually used to query do similar search internally in a factor database

so that we can then pass all that information right through a large language model a model and of course get the response now again usually what happen is that all these models that we

need to run a rag system for instance we may need to pay such as when we use openingi trpt and all different models that they provide okay now the other

thing also behind all of that is that when we are using AMA models these are ours meaning that we download them internally in our local machine and we

have a lot of control okay so that is the beauty of or in this case that is the problem that they solved but that's not the only problem that been solved

the other one is that privacy so in this case here when we run our own models locally uh we are making sure that our

data doesn't need to be sent to external servers because remember when we passing data through a large language model using chbt in this case or openi models

and so forth we literally are passing through or sending our private data to different servers that we don't know what will happen out there so this way

we have some security issues but with AMA that goes away because we have this enhanced privacy and security this is very important because imagine that you

have an application that needs to be uh dealing with sensitive information then it's important to have your own models

your own llm so that you have this contained enir environment where you're not sending out that sensitive information when you want to set up large language models it is cumbersome

which means is technically challenging and often requiring knowledge of machine learning Frameworks and also Hardware configurations but as you will see with

AMA all of that is simplified so this whole process is simplified by handling the heavy lifting for you also most importantly cost efficiency now we're

going to be eliminating the need for cloud-based Services which means you're going to avoid this ongoing costs associated with API calls or server

usage because everything remember is going to be local goal so once you have everything set up you can run models without additional expenses the other

one is lency reduction what that means is that local execution obviously reduces the latency inherent in network communications so when we are

communicating with something that is somewhere else through the network of course there is always a delay even if the network is really fast there's

always this latency the delay okay but since every everything is local we don't have that problem okay because we're not dealing with remote servers which

results in Faster response times for interactive applications and also most importantly we are able to customize our own models because when we running these models

locally it allows for greater flexibility in customizing or even fine-tuning those models to better suit specific needs without having this

limitations imposed by thirdparty services so these are some of the advantages we have by using AMA models so I know that we are eager to go

ahead and start doing the Hands-On but let's continue here to talk about some key features of AMA okay so the first one is what we call the Model Management

this is a big one which means that we are able to easily download and if we want to able to switch between different models or between different large

language models and that is the main point about AMA is that the idea is that we have this Center place called Lama as a open source as a framework as a tool

that essentially allows us to manage all of the different models that we can use interchangeably okay and next one of course is we have this unified interface

what does that mean well that means that we're able to interact with various models using one consistent set of commands as you will see and of course we also have what we call extensibility

what that means well it that means we have this support for adding custom models and extensions if we need to obviously we have what we call also

performance optimizations which means you're able to utilize your own Hardware effectively including GPU acceleration if available because remember all AMA

allows us to do is to manage all this large language models that we can download locally that is the key word key Point here and then we can do all sort of things with that we have this

performance that is optimized because everything is internal it's locally as well extensibility because we can add different models and extensions as we

see fit and of course we have this unified interface because we're able to interact with different models uh but still using a very consistent set of commands as you will see but the main

point here of of course is that we have this Model Management in one place we're able to easily download and switch between different large language models let's look at some use cases well as you

can see you can imagine we have a lot of different use cases uh for one of the first one is development and testing so you can imagine that as a

developer you are able to look for different large language models and switch them to test them out to see which one is is going to perform better

depending on what you want done so that's huge right so this is a deal of course for developers who are looking for ways to test applications that

integrate large language models without having to set up different environments and all that complexity so also for education and resource what that means

is that makes it really easy as a platform for Learning and experimentation without the barriers of Entry that are asso ated with cloud services so it's really easy just

download certain kinds of large language models and you can go ahead and do testing and do some research and so forth and we talked about this secure

applications the beauty of having AMA is that it provides this managed or this management platform of all of these different large language models which

means it's suitable for Industries like healthc care or Finance where data privacy is very critical because everything is your own you download the

large language models on your own machines so in conclusion AMA addresses these challenges of accessibility privacy and of course as we talked about

cost in the realm of large language models so by enabling local execution which is exactly what we do with AMA it

democratizes the use of advanced AI Technologies make making them more accessible and practical for a wide range of applications which is exactly

what you would want okay so now it's time for us to look into setting up AMA locally um and what we'll do we're going to do go through the installation process and do the setup before we do

setup there's some things that I need to let you know most of you should be okay there are some system requirements we need to be aware of first of all uh AMA

supports Mac Linux and windows and I'm sure other operating systems there as well but these are the main ones and also make sure that we have at

least close to 10 GB of free storage on your machine this is very important because as you will see some of these models require a lot of space and also

processor as long as you have a modern CPU processor you should be fine and most of you should have that in place okay so just something to keep in mind all right let's go ahead and get started

with setting installing and setting up AMA so if you go to ama.com O lam.com as such

this is what you are going to be able to see says here get up and running with large L models now because this is a moving Target in technology things may

change a little bit or dramatically by the time you watch this video hopefully not but the most important thing here to remember is that the IDE is still the same you will be able to download this

tool and use that way independent on how it looks okay so should look the same but you never know and you can see here at the recording of this video says here

Lama 3.2 this is the latest Lama 3.2 model that they provide we need to download AMA how do we do this well it's very simple all you do click here

download Once you click that this will take you to download AMA page now you can see we have three different flavors we have Mac OS so the browser was able

to pick up that I am on Mac and so offered Mac if you're on Linux of course you'll have to click and look at this so you'll have to install by taking this

command and running on your terminal if your windows of course is also just like mac you have to download as an application and run it okay whichever

way you have to download the AMA zip file so I'm on Mac I'm going to click download and then you can see that it is downloading here so I'm going to wait for a

second all right so it was downloaded and let's go ahead and click and open real quick so this is what you should see a zip file at least on Mac all right

so I'm going to go ahead and double click and it's going to and zip and then it's going to look like this as an application okay so what I will do I just double click again and it will

install again as an application on my back so I'm going to say open that's fine and then I'm going to move to applications and there we go so once we've done that I'm going to show you

real quick here this is what I see so I have now this window on Mac that allows us to go through the process I'm going to say

next and install the command line I'm going to say go ahead and say install and I'm going to pass here my credentials say okay so run your first

model so we're going to run this AMA run Lama 3.2 now again this number here may be different from when you're watching this doesn't matter just follow the instructions so I'm going to say finish

so what happens now is going to go ahead in the back and run the command and everything is good so it's going to install the Llama 3.2 which is a model

that's a large language model that's going to be installed with our installation so we can get started and you can see once that happens that the Llama is running because now now we have

this icon at the top that is showing there this is Lama icon and you can quit from here as well okay next what we can do is let's go back to models this page

this is where all the different models that ama has this is where they are aggregated and we can look into them and learn about them you can see at the top

we have Lama 3.2 which is the latest one which is the one that was installed already locally in our machine well come to this later and see more now let's go

ahead and click on this one so I can show you something real quick here okay the beauty here is that when you click on any of these large language models

you will see essentially the same things okay we have these 3B we'll talk about this later and um 1B latest and so forth

okay and then most importantly we also have all of these things that we don't care about at this point we have a readme we have the sizes it tell us

exactly what they mean what they good at this 3B default for instance and how to run it so this is essentially what you have at the top here so we're going to

cop this and we're going to open a command line or a terminal I have one open and I'm going to paste that so what this does well it say Lama run Lama 3.2 so what we saying here we're going to

use a llama as the tool the framework right to run llama 3.2 which incidentally it's already installed on our machine right I'm going to just

enter and what we'll do is going to spin up the Lama 3.2 model for us and then open this shell where we can start interacting

with the model just like that now I need you to think to go back to what we talked about the beauty or the advantages of using ol is that ama has

different models that we can use and install them internally locally on our machine and we can easily then through AMA as this manager of this Model start

interacting with different models right now we are interacting we're able to interacting through the shell here with Lama 3.2 model which means I can ask question

like what how old is the universe look at that and it gives me an answer the thing to keep in mind is that

Lama 3.2 tends to be a little bit verose and as you can see uh it is quite veros but that's okay I can say in short tell me how old is the universe so you can

actually direct it to tell you exactly what you need right kind of kind of like prompting it a little bit so the universe is approximately 13.8 billion

years all right so I can also say for instance clear see what's going to happen glad you could convey that information clearly I guess I have to say something like

this clear it cleared the session okay now I can continue here by saying for instance for help I can say forward

slash I can put the exclamation point or I can say help like this so this will give me it's actually exactly other

commands that I can use we have the set show load save and clear session as we saw as well as Buy to exit the shell so

let's go ahead and use this show here okay I'm going to say forward slash show what we want what is that I want to show I want to show info so this will give us

information about the model that we are looking at which is the Lama 3.2 says here architecture is llama the parameters is 3.2b we'll talk about that

and the context length is that and embedding length and so forth and some other licensing information so just like that ladies and gentlemen we're able to

install AMA and of course because AMA itself is just a manager we need to have the actual large language model and that's easy to do by installing in this

case llama 3.2 we could have installed any other model of course but we start with that one when we're installing

AMA all right so now I can say another question tell me a short joke okay here's one why couldn't the

bicycle stand up by itself because it was too tired very funny all right and if I want to get out of this shell I can say for a slash buy like this and you

can see we're no longer in our Lama 3 shell so I can clear and we're done okay very good so I hope this is a good introduction there is a lot that coming

um and I hope this excites you because the power that we have right now is at all this is locally we've imported we have downloaded the AMA tool in this

case manager along with one model which is Lama 3.2 okay make sure that you're able to do this and I'll see you next now to keep in mind that the whole idea

of a Lama is that we have this place where we have many different models we can use interchangeably we can just pick one and then drop it and pick another one and test until we find something

that works so it is important to always look at these models page here so we can look at what's available and most importantly as you can see here you can also go ahead

and search or actually filter by featured most popular and uh or newest and so forth okay so newest here seems

to be neotron so keep in mind again that these will change depending on when you're watching this video okay so maybe you're not going to see this as the newest but the important thing is this

is the place for you to come and look at what's going on so as far as we're concerned right now updated 3 weeks ago we have the Lama 3.2 it is also important for us to have

more information about those large language models because the idea is for us to be able to choose what works for what we trying to accomplish okay so

let's click on Lama 3.2 so when you click on Lama 3.2 or on any large language model that is shown here you're taken to this description page now there's a lot going on here the

important part here is that we understand at least the basics of what's Happening first of all we see that is under Tools 1B 3B which we'll talk about

but also we see that it was pull 1.1 million times okay which means this is essentially being used by many many many

people and then here we have these tags six to three tags but then we have here this drop- down list that tells us different things first one we have

different flavors of this Lama 3.2 we have this 3B which is the latest that's the one we have and then if we want a shorter smaller one is 1B one important

to keep in mind also is that depending on the flavor that we get here right you notice that also we have the sizes so

depending on the flavor that we have here we can have different sizes so this one 3B is 3 2.0 gigabytes okay so it's

not too bad but 1B is 13 GB and latest of course is 2 GB which is this one latest same thing okay and we've seen this before we have this command that

we're going to use to run in this case this particular model now if you go down here we have a read me tells a little bit more about

this model important for us to read and then we have the sizes so 3B parameters which is default this says here the 3B model

outperforms the Jamma 2 2.6 B and the 5 3.5 Min models on tasks such as following instructions summarization

prompt rewriting tool use but what does this really mean so I want you to look at this as a guide because again we go back to the whole point of Ama is that

we have different models that we can use and us as developers or as people who are wanting to use these models is that we need to find something that works for

us this is why it's important for you to always think about testing different models until you find something that works for you what works for me in a situation X Y and Z it's not obviously

going to be what's going to work for you in your own situation so that is something to keep in mind always all right so it tells us here the

comparison that for 3B parameter this size here it's really good at following instructions summarization prompt rewriting and Tool use so use tools or

function calling and so forth now when it comes to 1B parameter this is a smaller version of 3.2 it says here the model is compatitive with other 1

through 3B parameter models its use cases include personal information management multilingual knowledge retrieval rewriting tasks running

locally on edge okay so if you want to pull Lama 3.2 1B then you would run this so you can imagine if you want something

that will be good for this use case obviously you will use Lama 3.2 1B and let's go ahead I'm going to show you how to do this so we can run or Lama

run Lama 3.2 to pull down to get this Lama 3.2 1B so I'm going to copy this go back to our terminal just paste

that actually I should copy the whole thing and paste it so run Lama 32 1B so this

will download this particular subset of model this model so hit enter you can see it's pulling down manifest it's pulling everything in what may take a little bit depending on on this size

remember we're actually downloading locally so that means it will take up some space farewell so now you can see that

we have installed Lama 3 1B so if I were to go ahead and say for slash again show info you can see voila it's now we're

now using architecture Lama of course the same thing parameters is 1 2 B which is exactly different than the full-fledged Lama 3.2 so it's a little

smaller okay and context length and all that information so we can go and ask question how old is the universe like this and it's going to go

ahead and do the same thing but because we know from Reading here we know that this particular one here is really good at

writing tasks running locally on edge multilingual knowledge retrievals so I can ask things like as an example how do

you say hello I am fine in Tai let's see okay so it was really good this is

just a very simplistic uh test here so again you can see the things you can do you can pull in different large language models to see what's going on now the other thing I can do here I can say I'm

going to just say buy just to get out of here and still here I can say AMA and I can say list so this command

here is going to list all of the models that have been download downloaded locally look at this now you can see we have this Lama 3.2 1B this is the latest

one okay it says exactly when this was downloaded and of course I have a few others that I've done loaded a while ago so L 32 latest time ago I have this

embedding large as well which we'll talk about later very good so here it shows everything that has been downloaded in

terms of a list of all of the models let's look at parameters and do a quick Deep dive uh so that we understand a little bit more about the information

that comes with models so we're going to learn what are parameters as well as what is that they really mean right so when we say show info that command we

saw this we have this model architecture llama parameters of course 3.2b context length embedding length quantization and so forth so what do all

of these mean let's start with the first one so the first one is in this these llama architecture here first of all the architecture side of thing that means it

was architected it's a llama large language model meta AI that means they are the ones who created it matter Facebook and also that means that these models were designed with one thing in

mind which is efficiency they were designed to be extremely efficient which means they are strong when it comes to performing even at smaller scales

compared of course to other large models out there so they're very efficient and perform really well now when we look at then the parameters side of things it

says here 3.2b and we saw that sometimes it fluctuates to 1.2 or3 or 1 and so forth different numbers these are the

internal weights and biases that the model learns during training but what they do really is that they determined essentially how the model processes

input data and how it generates output because that's the whole point okay now when we say 32b for instance what does that mean

well that stands for 3.2 billion parameters now you can imagine even if you don't understand exactly what those means that means if you have 3.2 billion

parameters in this case these weights and biases everything neurons and nodes and everything that means you have a lot inside of this neural neural network

which means there's a lot of information that can be passed around the relationships and so forth which in terms mean that this model the the more

billions or the more parameters it has which means has more connections and more interactivity inside that means the more accurate it is when it comes to

getting the results now it's important to understand also that the number of parameters does reflect the complexity and the capacity of the model as

explained but there's also what we call a tradeoff which means that more parameters can improve performance of

course but it require more computational resources so if you have a large language model that is 3B or 5B or 7B or

8B well that means it's going to be amazing in terms of performance but also means it is going to require a large amount of computational resources so

when we have something like 32b this kind of strikes the balance between per performance and resource consumption so let's summarize this

simply when we talk about parameters talk about 3B or 2B or 10p or 7p and so forth these are numbers inside a neural

network that it adjusts to learn how to turn inputs into correct outputs so the largest number the more the better in

this case but we have drawbacks in that you also need uh a lot of computational resources to run it so next we have the

context length so this number refers to the maximum number of tokens which essentially are pieces of text that the large language model can ingest in a

single input so when we have something like 131,072 this is an exceptionally long context length what that means is that

it can handle very long documents capturing dependencies across large large spans of text which is really good that means you can take big large books

and article lengthy articles and extensive conversations and it can deal with it with no problem when we talk about the embedding length we're talking about the size of the vector

representation for each token in the input text so when we say 372 what we're saying is that we have

372 dimensions in the embedding space so the larger number again the more Dimensions we have which means the more relations you're going to have in this Vector space so what does that mean well

that means that we have what we call the semantic richness which means higher dimensional embeddings can obviously capture more nuanced meanings and

relationships between words when it comes to a large language model then that means this will reflect the model's ability to understand very complex

language patterns so the large ler we have this High dimensional the more the model is going to understand complex language patterns now there are some

implications again is that we have this competition load so larger embeddings increases of course the compettition requirements and also if we have that

that's going to improve the model's ability to generate contextually relevant and coherent responses okay and the last one we have here is quantization now there's books

and we can spend hours and hours talking about this but in general quantization essentially is a technique used to reduce the size of a neural network

model by in this case reducing the Precision of its weights this number indicates that the model's weights are quantized or

quantized to four bits so translating essentially we just saying we now have a smaller model and faster processing

and lower memory usage so now we have a more efficient um model so going back to our Lama 3.2 model page here to see more information

now you understand what 3B means 1B means and the implications and so forth and one thing you will realize also is that at the bottom we have what we call benchmarks now I don't really trust a

lot of these benchmarks because anybody can inflate B the benchmarks or deflate them to follow certain agenda but nonetheless it's nice to look at and see

The Benchmark but one thing I want to show you here is the implication of the sizes the parameters if I go back perhaps and let's go back to models here

real quick and let's look for something different let's look for Lama 3.1 you can see with Lama 3.1 you even have 8B 70b

45b let's click on 405b well let's look at this one you can see that if we go to

405b that's 229 GB of space you have to have in your hard dis that's a lot and even if you're able to have that

internally locally that that means you also have to have the computational capacity for you to be able to run these models so this is something to keep in

mind you could have the capacity of having 229 gabyt locally but will you have the capacity of the computational

resources that are needed to run this now keep in mind as you learn now this is probably one of the best models versions right because it has everything

it's large billions and billions of of nodes and neurons and everything that's something to uh keep in mind and so for things that most of

us are able to do want to do 8B even 70b should be totally fine okay just something to keep in mind and as you go through each one of these different

models they have different things okay you can see of course we have more information here 45b 7B 8B and so forth as we talked about and they tell us the

evaluation and what they're good at when should you use it and all these different informations as well as human evaluation as you see here so this is something to always go ahead and read

and perhaps go to uh some links here to read more about it now I want you to always have this inquisitive mindset whenever you're looking for models so that you know exactly what you're

getting um and contract that with what you want to accomplish that's how you need to uh go by uh when it comes to using AMA and different models here it's

not just one model fits all that's not the idea the idea is that you find something that will work for you okay so models are real fun and you can always

um go ahead and test them out now now let's go ahead and run through some commands uh with our Lama that way we are well situated and we understand how

to use some of the most important commands Okay Okay so let's go ahead and get started so we know that we always start with AMA we can say AMA list this

will list all of the installed models internally locally so you can see we have

32b 32 callon 1B here it's the size and when it was modified and we have other ones that I have here in your case you should probably most likely just have

one let's say I want to delete or remove a model I can say AMA remove also I can say AMA help which is going to give me some of the commands

that I can use we have serve have create Show run stop pull push list we've seen

before PS and RM for remove so let's start with I'm going to go back say AMA list to show the list of all of the models that we have here let's say I

want to delete the this one here so I'm going to copy that and I'm going to say clear AMA RM to remove and pass in the

model name hit enter and it's going to delete that model now if I go ahead and say AMA list again we should see that we no longer have that model very good and

if you want to delete this other one here you can also do like that now if we want to go ahead and pull in a new run we can say

AMA and I'm going to say pull and the name of the model as we've seen before so again we can go back to our models

website all right AMA Library models and let's say we want to get Jamma okay so I go back here you can see models all these models let's say I want to get

something different let's say we want this code Gemma if you click here you can see that we have different flavors let's say I want to be the smaller so

you can see we can run this one here I can either run all of this or I can just pull first this one and then run it okay so two ways to do the same thing so I can say AMA pull like that it's going to

go ahead and pull our model now remember this is 1.

gigabytes and may take a little bit okay so once that is done I can say AMA list we should see that we have code

Jamma so I can now run it right so let's go ahead say all Lama run coama so it's going to now run coama so we can interact with it go back here I can say

that Co Jamma is a collection of powerful lightweight models that can perform a variety of coding tasks like fill in the middle code completion code generation so forth so let's see we can

test it out right can see if this works so I can say for instance write me a python

function that returns hello world okay there we go you can see Define function your name return hello world your name and there we go all

right very good so you can see that we're able to of course download a new model and test it out so what I can do

again I can just say for slash buy to get out of here and then I can say AMA

list to see that and I can copy this and remove it so I'm going to say AMA RM to remove that and it's

deleted and AMA list again you can see that we no longer have Jemma okay so back to our models here I can go

ahead and look at models and imagine that we want to find a model that deals with images or that is able to read an image and tell us what's going on

there's a model called lava so lava is a novel endtoend trained large multimodel that combines a vision

encoder and funa for general purpose Visual and language understanding so this is is a good candidate for what we call a multimodel model which means is that

it's able to deal with images as well as text and so forth and other types okay so we have just this one and then we have this lava Lama 3 so depending on the situation we can read more to figure

out which one is the one we should probably look into so let's click on this lava and you can see we have 7B 13B and 34b so you know which one we're going to

be using so I'm going to go ahead head and get at least the 7B which is this one here latest or 7B doesn't matter it's about 4.7 gigabytes okay so let's go ahead and pull this one

in I'm going to copy that and let's go ahead and just paste it all in and get our lava 7B all right so after a little while you can see that we downloaded a few things

here but most importantly we have this 4.1 gab so we have now our model is running and you can see we have our shell I'm going to just say what is your

name see what's going to say I don't have a name of course it's just a large language model okay so now let's see what we can do one thing I'm going to do

actually is I'm going to move over to a code editor vs code to make this simple okay so I'm just going to say byy for now and then I'm going to open up here

to see vs code so I have this flower1 PNG we're going to use this with of course our lava model to see if it

can read this image and tell us what it's all about okay so let me go ahead and open the terminal here real quick and so what I'm going to do first let's

go ahead and say AMA I'm going to run lava 7B okay now it's running very good so

what I'm going to do now is I'm going to say what is in this image and I'm just going to

pass the image name which is flower one. PNG let's see what's going

flower one. PNG let's see what's going to happen so now it's going to think and tell us exactly what it sees the image shows a bunch of small purple white

flowers with yellow centers which appears to be a type of pansy they're growing in what seems like soil or a potting medium surrounded by what looks

like green foliage so that's pretty good right for if you go back here and look yeah looks exactly that's exactly what it's describing which is really really

awesome now let's test something here I'm going to test to see if this model has this history capability meaning it's able to save the conversation for

context I'm going to say and now write me a short poem about the about that let's see what's

going to happen ah there we go so it was able to look look at that a purple and white with yellow Hearts Define they add A Touch of Beauty to the world around

little patch and so forth so it knows now we know if we're testing that this model is able to save the context the

history of the things that we've been asking uh the context of our conversation tell me more

about those flowers let's see what it's going to say okay there we go it knows exactly what flowers we're

talking about which is what we had here okay I can ask can you tell me where they grow

best and keep in mind again this is all local large language model that we running we don't have to go and send request to a network somewhere it's all

local very good so pansies are native to the Mediterranean region and all that great stuff very good so we see that it is indeed working so we're able to pull

in a new large language model in this case lava which is a multimodel model which allows us to deal with text as well with images and so forth and we ask

it to tell us what it sees right in this image here and it was able to give us the description of the image and we ask

some follow-up questions about that image and it works so I've changed to

AMA run Lama 3.2 model right now so I can do it again you can see since it's loaded then we can see that we are able to go to the

shell so what I want to be able to do is show you that we can also do many many different things here right so because we are running on a large language model it's a model that does a lot of things I

can perhaps say I have a sentence that I'm not sure the sentiment of that sentence right I can say tell me what is the

sentiment of the following sentence I not willing to pay you back let's see

what's going to happen okay again it's very verbose which is okay but essentially says that the sentiment of the sentence I am not willing to pay you

back is negative and confrontational it implies that the speaker is unwilling or unable to repay a debt all right so this is really good now one thing we can do I

can also say can you please be less verbos okay there we go so the beauty here is that you're talking or you're interacting with a large language model

and you can actually prompt it to do what you want it to do in this case I told it to always be prompt to always be less verbos at least in this case and

you can see says the sentiment of I am not willing to pay your back is negative and confrontational it implies resentment defensiveness and obligation aversion okay now the beauty here is

also we can modify or in this case customize our model because we can actually add certain pieces of metadata to the model that cater to what we want

it to be like what I mean by that is that we can actually create a file here right click I'm going to click here and this is what we call a model file so it

has to be something like this model file no extension whatsoever and inside of this file file this is where we are going to add a few lines of text code

essentially so at the top here I'm going to say all caps say from because we are modifying it from a different from a

base model it's going to be from Lama 3.2 in this case if you have something different you add that and then here is where we're going to set a few things so we can set a lot of things but now I'm

going to show you how to set in this case the temperature what is a temperature temperature is what allows the model to be more creative or more

direct and matter of fact so the higher the number in this case from 0 to one the higher the number closer to one the

more elaborate the more um creative the model is going to be okay so I can change to 0 point three for instance if I want it to be less

creative and then I can add for instance the system message so this is just the prompt right that it needs to know of so

in this case here put that into let's say triple quotes like that to say

inside here we can put text a very smart assistant who answers questions succintly and

informatively okay something like this so this is our model file which is going to

allow us to modify or customize our model okay so we can add more parameters here more things to Aid our model but this is okay for now so how do we do

this now to make it so that our model indeed will comply with what we put together here in this model file well it's very simple all we have to do now

I'm going to let's go say byy real quick here all we have to do now is we need to use the create command from AMA to create

some sort of a new version of our model which is going to be specified by this model file very easy really so I'm going

to say AMA and then say create and I can give it a name just say James okay whatever you want and then we're going to say f to say to the

file it's going to be under the model file notice the model file is actually the same level here so I can just go ahead and that so if I hit enter what's going to happen now let's say enter okay

so says success you can see that it went ahead and transferred model data so using the layers and everything in the background and so now if I save for

instance list you can see that we have this James so it really created a replica of the main

model and now this is going to be different from the Lama 3 3.2 right because it will have a little bit more of a modified customized piece to it

very good so now I can use Mario or James I should say to ask questions so look now I can say

orama first run James look at that and we should run James how beautiful is that so look what will happen I can say

what is your name look what's going to happen my name is James and look how suc it is it

goes straight to tell to answer what needs to be answered are you smart let's see I designed to process and provide accurate information quickly and efficiently making me highly

knowledgeable on a wide range of topics and all of that comes from this because we said that your assistant that was very smart

ask answer questions suly and informatively very good very good you the power things you can do right so now I can ask questions let's see let's say

tell me about all tell me about the oceans let's see okay so this is pretty good because

it's such an open ended question it went ahead there are five oceans gives me all that information oceans play a crucial role in regulating the earth's climate weather patterns and ecosystems they

also provide all of this information here okay so it is to the point that's pretty good this was just an example to show you the things you can do you can

actually modify customize your model to be and do certain things that you wanted to do this of course was very very simplistic but you can see for the

system here we can prompt it even better with more complex information and change temperature and all these different things let's go ahead and just say byy

and because we're done let's go ahead and get rid of it so that we save some space so llama

RM James okay so Lama list we should not see James and while I'm here let me go ahead and actually get rid of some of these as well so that we have more space

okay very good so far we've been using the CLI the command line interface with AMA which is attached with AMA that governs everything in a back end but we

can also use the rest API in fact most of the things that we're going to be using later are based on the rest API which means there is an endo that we can

hit and run in this case the models so how does that work one thing to keep in mind is that in the back end as you see well you can't quite see here but at

the top my bar here I do have this well you probably see this quid AMA but there is this AMA that is running in the back end so that means in the background it's running which means we're able to do all

sort of things that we're doing right the all Lama application per se it's running but it's actually being served at an endpoint locally and all of that

is being served at Local Host 11434 what does that mean well that means then we can generate a response using the olama rest API because it has

it attached to it and it's running in the back end in the background okay so to do so it's very simple all you have to do you have access to all of this we can curl

something like this so in this case you can see it's a curl and this is the end point so Local Host 1143 for API gen

generate and then dasd and then we pass the payload here so here we're saying the model is going to be Lama 3.2 right and the prompt we passing along it why

is the sky blue so if I were to run this or hit enter what will happen is it's going to hit that end point and of course it's going to show me this now

this is not very helpful mainly because it just shows a lot of gibberish and there is this stream that's happening but if you look look closely you can see

that every time this run extremely it's going to show a certain word in response so combining all these words will be the

response so that means then when we write the actual rest API payload we need to also add something else to it to

tell it that we don't want it to stream as it is right now so that we just get result what that means now I can let me

go back down let's clear so now what we need to pass is this here that says stream false so say we don't want it to stream just give us the response if you

hit enter you can see it will take a few moments and then voila we have this model everything all the metadata about the model that we're using in this case

is say tell me a fun fact about Portugal and then says here here is one response did you know that the town of obid obid

in Portugal is often called the fairy tale Town due to its medieval architecture and picturesque streets right it gives it some other information

all right so that is a difference here if we put stream false then we get that if this is not there it assumes that stream is true that's why we get what do

you saw earlier now in this case you can see the end point is generate now we can also get the chat with the model the chat end point okay so generate the

difference here generate it just goes ahead and predicts what needs to happen as you see just gives us result but we can also pass in the chat endpoint so

let me go and clear to do so I have that you will have access to all of this anyway so no worries and there we go we do the same thing curl now the end point

here is API for SL chat and then DD and then we pass here the payload so so we pass the model the messages messages

plural that's why we have here a list okay so we pass the role it's going to be user and the content is going to say tell me a fun fact about mosm Beek let's

go ahead and run this so there we go and now this is a different endpoint it gives us here some here's one fun fact about gorang gorza Peninsula which has

one of the largest Coral in the world um I think this is actually not correct they have to remember that sometimes these large lar models are not always

correct so there's no such thing as going a peninsula I know this because I'm from there but anyway you can see that it's giving us some results large ler models and are always correct so as

always we can also pass more information or more specification to our endpoint in our payload so imagine that we want to

for instance we want to request a Json mode we can do so by requesting it by adding Json right so you can see here we

are getting the generated API and we have the same thing model prompt now here we even said in our prompt make

sure that we want to get the response as Json but also we want to say here as one of the parameters that format it has to be Json to make sure that indeed the

large language model aders to this okay let's go ahead and run and you can see doesn't look that great but we want to believe that this is actually a Json okay that is coming

in so you can see the curly Bryce is or opening their model and then llama 3 created and all that information and we should have an actual Json but you can

see response Day morning Sky color and so forth the beauty here is that if you go to this link here gith habo Lama docs

API end points or so many other end points we can go ahead and hit um using the rest API okay you can always come here and look at some of them we can do

essentially the same thing we were able to do in our s ey that's the whole point we can do that using the rest API for instance we can copy a model show model

information you've seen that before okay you can just curl and then do that so I'm not going to do that you can do that yourself and then this is the result that you get so all these things that we've done before you're going to be

able to do them as well using the endpoint API the rest API endpoints we've learned a lot so far there are a lot of things that we've learned about

the CLI we learn how to use the CLI of course which is attached to orama to do all sort of things to pull in different

models to run them to remove them to even modify the model or in this case customize the model by creating the

model file file and then use that to create some sort of a copy or a modified version customized version of that model which is very handy as you see for use

cases that you may have okay so a lot that I've thrown at you I hope this is making sense I hope you are practicing and seeing the power and most importantly here remember is that all of

that is local you're not having to do anything uh passing something to the network to get anything back in that way which means it's free you're just using

your own resources which is the power of AMA so as a quick full summary here AMA as we know is a platform that allows you to run large language models locally

which is really awesome the great thing is that it supports various models tailored for different tasks including text generation code generation and

multimodel applications so essentially AMA model support these tasks here text generation such as Lama 3 x could be 3.2

3.10 or 3.11 depends Moll and so many others uh we have also code generation one of the example was code Lama okay

and multi model application in this case text and images and we saw in this case was lava the lava model now this is just

to show you the breath of things or models that ama provides Okay so again your job is really to figure out what

models is going to give you the best results for your use case Okay so keep in mind that you always have to test figure out which one will work for you

all right so to create large language model applications using AMA we saw that first of all uh we can use large language models that come with Alama

because Alama hosts per se all these different large language models we can use now we also learned that there are different ways to interact with AMA and

its models so we saw that the main way is through the CLI the command line interface so it's easy straightforward and that's what we've been looking into

and then the second way would be in this case to use UI based interface which we're going to look into next so essentially have a user interface where

the back and we can put or apply a model AMA model and then we have a nice user interface that users or yourself can use

to interact with the model now we've just finished looking at the rest API so this is the base of essentially everything else we're going

to be doing after the next UI based interface way of interacting with the models okay so essentially we're able to use rest API curl and then hit the end

point and get information that way about generating text or um in this case deleting things deleting models or looking up everything related to our

models so essentially the same things that we're able to do using the CLI we have a way of doing that using the API the rest API and later we'll see we're

going to use AMA python library now this is where we're going to be able to then use the tools in Python in code because now we have more customization more

freedom per se to start building the actual large language models applications that we so want to do okay okay so now let's go ahead and look at

the UI based interface so if you go to m thata MST that app as you see here this is what you will see so you can read about it say Lama 3.2 Vapor mode all

this great stuff say the easiest way way to use local and online AI models so without Misty painful setup endless configurations confusing UI Docker and

all these other things with MTI one app one click setup no Docker no terminal offline and private unique and Powerful feature so that's the reason why I chose

this of course you can choose any other that you might find out there that I mentioned so this is what I'm going to be using and you can see there's a lot of logos here saying that it works with

uh meta stuff with chat open Ai and many many others so it's really good so you can see Windows Mac and Linux and they do have a user interface so you can see

exactly how it will look like so essentially it's going to be like having chat GPT uh but now we are running our own models so click here and I am on Mac

of course I'm going to go and go that way if you're on Windows you have the option to do the 64 or 64 CPU only MD or Nvidia if you're Mac you have apple

silicon or Intel chip if you're Linux of course you have these flavors as well so pick the one that works best for you so I'm on Mac I'm going to go to Apple silicon like that so it's going to go

ahead and download okay so it's now downloaded I'm go let's go ahead and open it and I'm going to double click to start the installation so I'm going to go ahead

and drop it there very good and let me go ahead and open it real quick okay so I'm going to say open all right and there you have it now once you

open it this should happen and the beauty here is that you can see how would you like to get started you can set up locally AI I remote modeles

provided and so forth but if you look down here look at this Gama get started quickly using AMA models all right and

even found it under users meama and all the models they found it locally because it detected that we have Ama installed

in our in our uh on our machine so I'm going to go ahead and say continue down here okay that's it just one click and

we are done and you can see here at the bottom here make see I can make this larger yeah I can make this larger that's very good okay at the bottom here you can click smaller you can click you can see

we have different levels or different models we have lava 7B which you remember and I have Lama 3.2 all right because those are the ones

that we have installed at least I have installed locally if you have more than one then you're going to see all of them sure here and this is beautiful because we didn't have to do anything just one

click indeed like they promised and we are there so let me choose the Lama 3.2 and the moment you do that that's it and you can just start chatting just like

that I'm going to say how old are you just silly to see what's going to say so this is being powered Again by

our own model so I was released to the public in 2023 very good can you tell me

a funny short story and just like that look how fast this is one day a man walked into a library and asked the librarian do you

have any books on puff loves dogs and scroller cats scroll danger I think that's what you say the libran replied it rings a bell but I'm not sure if it's hair or

not okay that's very cool so we have that so you can start talking you can start chatting with this long large language model and if I wanted to I can

also go and attach a document let's click here and I'm going to add one image that we've seen before drop it there

now image is attached let me go back here and I can ask tell me about the image now here's the thing

the reason why it's saying this is because the model doesn't CU we're using llama 3.2 it doesn't really know how to deal with the images however if I change

here go to Lava 7 and ask the same question tell me about the image Aha and now because lava is a mul

model multimodal model it will know how to answer questions about the image there we go now it's working the image shows two purple flowers with yellow

centers okay very good so it is describing it took a little bit of doing here perhaps it should have reset everything but it's okay you can see that now it is actually working because

we're using lava 7 and it will tell me exactly what it sees now I can change to Lama 2 or

3.1 and if I say tell me about the image it's going to give me some issues of course okay very good so it can't do

that but the other thing I can do here I can also add let me just delete this I'm going to add a document let's say for instance I'm going to add this document

that we've seen before which is the b o i PDF there we go I'm when I say can you tell

me about the PDF file okay so now I have some issues here the best way to do this is you go to attach knowledge Stacks so click here and I'm going to say add your

first knowledge stack so I'm going to say my tester and the thing is because we are

now invoking want to create a rag system we need to have an embedding model so that it can create those embeddings of the PDF file of the document that way it

can be saved and then the large language model is able to Converse that way okay so we're going to just use whatever they give us nomic uh embedding test text I'm

going to add that it's very good and at this point let's go ahead and add an actual file so I'm going go ahead and drop it again in this case here just drop it like that all right

so once you do that you have to hit compose so that it a it's able to pick it up do all the things it needs to do to actually get information from our

file okay embeddings and everything say compose so it's composing okay knowledge stack saved and composed successfully you can now use it

for chatting very good so let me get out of here and I'm going to go here so click again here and what I want to do

here is I want to be able to say click on my tester but this is my knowledge base I want to include in my chat here so make sure you click that and that

will what I will do is we'll let this know that indeed we have included that knowledge base which includes all the

information about our PDF file so say Give me a summary of the PDF

file okay let's uh says there's no PDF file mentioned in our conversation so far the text we discussed earlier was from beneficial ownership and everything so it took a

little bit uh because I'm asking PDF it doesn't really know about the PDF because all of that information was actually transformed into text and embeddings and everything for the large

language model so I just can say Give me a summary of the document okay there we go so it says the document provide instruction on how to fill out certain items including

identifying documents issues a company's images and everything and it is indeed beneficial ownership all of that so we've seen this before I can keep asking what are

the penalties for not filing and and there you go it gives me

exactly the penalty and everything that will happen if I don't file right I can just say give me the

penalties maybe that's a better question perhaps okay that's better because now it's giving me exactly what I need okay based on the text from beneficial ownership information blah blah blah and

everything is there failure to comply with beneficial ownership reporting requirements Cil penalty 10,000 fine up to 500,000 and so forth

okay what are the deadlines for filing okay so you can see that this is

working we're able to converse with our data our own document now the beauty here if you AR haven't realized yet is that all of this again is internal it's

our own large language models we can of course go and spin out a different large language model OKAY adding new provider and so forth and we can use

interchangeably as you see here and we passed in a knowledge base which means we said here are some documents in this case which is one document and use that to create a knowledge base so a rag

system essentially and so that we can chat and ask questions about that knowledge base okay and just like chat GPT we can go ahead and create a new chat we can do all sort of things as you

see here the point is not for me to go through Misty thoroughly the point is to show you what's possible to create a user interface or to use a user

interface that is guided or is being fueled by our own AMA model how cool is that so what I want you to do is to play

around with this and you can see it's just to show you how amazing this is because we have our own box that we can uh pass in sensitive documents and all

those things without worrying about prices you know having to call an API anything external put this in a cloud

base or having to pay for usage so it's all here and we can use as we see fit so we're able to see all of the ways

in which we can interact with and its models so we looked at CLI command line interface and we saw that it's really easy because it's the fastest access to

our models it's very easy but as you know it's not scalable in sense that you're not going to build a full-fledged application using that and then we went

and looked at the API the rest API which is essentially the same but now we have a different back door per se we we using rest API to pass in certain payloads and

get information so we're doing the same thing that we can do with CLI hitting the same functions per se but now we're using different end points with the rest

API and we just finished looking at the UI based interface so this is way easier for us to be able to have a UI based

interface and actual interface such as chat DBT that way we're able to ask questions uh change manually quickly

different models that we may have and start chatting with our models so it's really really easy to put uh together or to have that working using the M that

app and of course there are different flavor different tools out there that you can use that will do the same thing right that would give you this UI based

interface so I chose MTI because it's just easy as you saw to install and get started so it wasn't about how to use these tools but I was about to give you

the knowledge and tools that you can then uh use on your own and explore more okay so now we're going to go to the fun part which

is we're going to use now sort of a a backend combination of the API rest API through the python Library the AMA

python library because truth be told we want to be able to create local large language model applications using AMA

models and so for that we need a way for us to be able to use python this code in this case code or any other language but in this case going to be python to be able to take advantage of these models

that ama provides and so that's what we're going to be doing next which is we're going to get started with AMA python Library so we can use code to

interact with AMA Models All right so I have this folder here we've seen before called AMA we got a few things here let's get started here

simply by using the AMA library but actually I'm going to do the hard way first to show you that we can actually use code to get to the end point just

like what we saw with the restful API so real quick so we can see what's going on so let me go ahead and create a new file here so we're going to have access to all of these code so no worry so I'm

going to say start one. P1 or py okay and so what we'll do here first actually is I'm going to go ahead and

make sure that we have an actual virtual environment for our python project so I'm going to say Python 3 okay like that so we should have a

virtual environment there that's very good and let's go ahead and activate it so Source bnv if you're on Windows of course there's a different way to do

this activate and voila so now we have our virtual environment AC activate active I should say very good so what is the first thing

we're going to do well first of all I'm going to go and import let's say pip install I'm going to install requests real fast here so we have

that okay very good because we're going to use that to do what we are looking for it to do all right so I'm going to

go ahead and import requests so we have that and also I'm going to import Json so we have that as well okay so I'm going to create a URL here which is

going to be the URL where is the end point it's going to be HTTP for our Local Host here so it's not going to be

that it is 1 4 34 and in this case I want to go to the generate endpoint like this

okay and for that remember we need to pass some payload data when we pass along so I'm going to say data just create a little uh dictionary here and for a dictionary I'm going to

pass the model that we need it's not going to be gpt3 or gpt2 we're going to call this llama it's going to be

Lama 3. two and make sure that llama is

Lama 3. two and make sure that llama is running for this to work always and then I'm going to pass the [Music]

prompt I'm going to pass something that says tell me

a short story and make it funny like that okay so there we go this is our payload that we're going to pass along

using the rest API so I'm going to send the request so I'm going to say response pass in the request. post pass

in url and Json and what I want to do here is I want to say stream to true to say I want this to be

streamed okay as you will see in a little bit okay maybe this a little bit smaller so you can see everything all right okay so then let's go ahead and check the response first so I'm going to

say check the response status so it's going to say if response status is as you see there I already have all these code so I'm not

going to Bor you with all of the intricacies I'm going to put it all there okay so what we're doing now is that we're going to check if response code is 200 which means all is good and then we're going

to go ahead and start generating thing so we're going to go and iterate over the streaming response because we said the streaming to true as it comes the response that comes in okay and so what

we're doing here we're decoding the line and parsing everything until we actually print the generated text as we go through very good so let's go ahead and see if this works I'm going to go ahead

and run this make sure that of course the AMA icon is running is out there otherwise this is not going to work so let's go ahead and run let's run this

looks like I'm having issues right so I'm need to pass API like that okay my bad that was my

problem let's go ahead and give it a try again okay and there we go you can see that is indeed working and it's really really fast as you see here generat text

here's a short silly story for you once upon a time there was a chicken blah blah blah and there you have it all right very good so this is indeed

working we're able to use code to actually interact with the our llama 3.2

model locally right albe it we are not using the AMA uh python Library but you can see that this is actually working go ahead and play around with this and I'll

see you next all right so this was really nice we're able to use the rest API in code here the restful API should

sayama restful API in code to interact with Lama 3.2 this is fine but now I'm going to show you how to use the AMA

Library the AMA python Library so we can go straight into code we don't have to hit these we don't have to explicitly hit these end

points all right let me go ahead and create a new file here let's call this start to.py remember you will have access to

to.py remember you will have access to all this code so do not worry okay so the first thing we need to do here of

course is to make sure that we import or we get the right dependency so going to say pip install we need to install or

Lama as such very simple and there we go now once we have a Lama we can start working at right so

first let's go ahead and import AMA there we go and real fast here I'm going to show you how simple this is so the same thing we did before with the

CLI and even with the rest API we can do the same thing here using this dependency right using this SDK

perite so now I can say put inside of response and I'm going to say all Lama and I can say list so this function here

as in name imply is going to list list it should list all of the you guess it all of the things that we have in this case all of the models so let's go ahead and save and give it a

quick run run so real quick okay I ran and you can see that we have this Json that comes up it says

llama 7 oh name lava 7 and that is the model and I should have gives all that information and I should also have the

Lama 3.2 and some other things all right this is very cool very good all right

very good so you can imagine we can say AMA list we can also say AMA let's say chat chat response client

create and delete embed and all these other different things right so that's very very exciting I just want to show you that I'm going to just call comment that out so we don't have a lot of

things here all right so let's go ahead and hit the chat endpoint or in this case the chat API

well as you know it's very simple so let's go ahead and get started here all right so I'm going to say res we can put

anywhere orama chat so. chat and then I'm going to pass a

so. chat and then I'm going to pass a few things and you can see that if you have over it says it respects at least

to have the model name so the model I'm going to say llama 3.2 okay just add the one that you have and then here because it's chat I'm

going to pass messages okay so this comes as a list and I'm going to pass the role as a user and the content is going to say something I can say

something different that perhaps why is this sky blue something like this okay so now I

can actually print what comes back in this case from our chat we pass the model and the messages could be more than one message okay we

can also say uh for our role here another one that says let me just put like that I can say the context pass the context pass all different things that I

may want to pass okay but so far this is okay let's go ahead and see how this works I'm going to go ahead and just run from here okay to a little bit and you can

see we have the results so you can see the model and everything but most importantly we have the content says content is the sky appears and everything but to make it even better I

can just go ahead and get the content so let's go to result uh let's say I think I have to go to

messages and they get the content like that okay let's go ahead and run again all not messages

message okay all right and there you go so this guy appears blue because of phenomenom because of phenomenon called scattering and all of that great stuff so we have

all the information here now you noticed before when we just look at showing the entire payload it comes in with a lot information about

the model when it was created of course the content the role and all that information at the bottom we have some other pieces of data metadata so done

true total duration how long it took and low duration prompt evaluation and uh also evaluation count prompt evaluation

duration all these little things that we may want to keep track of all right so just like that ladies and gentle ladies and gentlemen we're

able to use the chat endpoint here using uh the ol Lama python Library so go ahead and play around with this and see what we can do because this is the

beginning of a very exciting thing I think now one thing you can also do you should know this is that we can pass

other things in our chat function here right see we have model messages tool stream so we can say we want this to be streamed and format as well very cool so

what you can do I'm going to give another example here for streaming let me go ahead and comment this out so we have that for streaming it's essentially the same thing notice I passing messages

we're all user now I have a different question why is the ocean so salty and I can say stream to true because I want it to be streamed when I

do that then whenever I want to print that out I have to put that through a for Loop or a loop of some sort that way we can get all these pieces that are

being streamed so we can see them being flashed out in our command line okay so let's go ahead and run real quick so you can see just run like

this takes a second and there you have it okay so there we go if you want to have the this capability of streaming

and showing those messages as they come in this is how you would do using Code all right so go ahead and play around perhap add different messages or

different questions here and see how this behaves and what you can also do you can use a different model whichever model you may have running

okay so one thing to keep in mind is that all we've seen now this AMA python Library West DEC I guess I could call

that it's all based or designed around the AMA rest API so internally when we call orama that chat internally we're

essentially doing what we did before I showed you here so it's hitting this endpoint URL and passing in the API in this case go to chat or generate or

create and so forth it's just like an AB it's just an abstracted way of hitting this end point so we don't have to explicit L do this so I made sure to put

it here so that you have that okay now we can hit now we can use another now we can also use the same now we can go and

use a different function or end point for instance like generate if want to generate something right we can do that what I mean by that is I can say for

instance AMA generate and pass that and pass that prompt I can even pass all of that I don't think I can okay I don't need to pass any of that so we can also

just generate a response instead of chatting as such so nothing new here I'm just going to leave it as is so you can see the other thing we can do is we can

say for instance we can show by saying AMA let's just print and say

AMA and then that show and we want to show what want to show anything about llama three like that that two okay

let's go ahead and see and there you go after a few moments you can see it tells everything we need to know about our model OKAY a lot of

information that's for sure okay there we go next thing we can do is we can actually use the model file as we did

before but all in code to create a separate model that has Specific Instructions so it's very simple I have

some parts of the code here so we create in this case the model file we don't have the actual model file but we can put all in line is here as you see here

and the other thing I can do is I can pass the temperature here let's go back to this one to see how you do that just

copy this and pass temperature 0.

1 for instance and then all we need to do now that we have this information is as we saw before we are going to go

ahead and create the actual model so and then we're going to pass the new name know it

all and then pass the model file which is this one here so we can make this more robust if we want for instance we

can say you are very very suce and informative something like that all right okay so now that we have created

our new know it all so we can go ahead and generate something right use the generate message or generate function so

I'm going to put into a variable so AMA generate and I'm going to pass the model that I'm going to be using yes going to be know it all this one here

same name and The Prompt there so I can just say for instance sure that's fine

okay and we can go ahead and print the result need to go into response that's how it comes as okay and there we have it so let's go

ahead and give it a run and see so it's generating creating everything and some sometime soon we should have there we go the ocean

salinity is primarily due to the Earth's water cycle very good it looks great and it

works so now if I come here and just say AMA list we should see that we do have this new know-it-all latest which

is two gabes was created about 20 seconds ago all right very good so these are are the things that we've done before using

the CLI as well as using the rest API but now we're using the AMA we're using the AMA python library osdk to do the same thing

because I just want to show you what's possible so this is more flexible as you see because now we actually use encode and the power of the rest API in the back end to create very simple

applications of course this is just to show you what's possible and soon enough we're going to be able to create those large language model applications that

are based on locally using AMA models so once we done that we can of course

delete models okay so can say all delete know it all and if I run this it's going to go ahead and run again and go ahead

and delete our friend know it all okay so I have the response again but most importantly if I go back and list we

should see that we no longer have no at all okay so go ahead and play around with this and uh I'll see you later okay so now that you have all the tools you

understand what Lama is what problem comes to solve and how to actually build a simple uh useful applications using

AMA AI based applications using AMA now it's time for us to dive in and start build building real world use cases using and let's go ahead and get started

all right so now it's time for us to start building something useful things that are actually useful so the first thing we're going to do here is we're

going to build a grocery list organizer okay or categorizer so the idea is you have for instance a grocery list file text file that has all these different

items okay and so what we want to do is we're going to use the line 3.2 the model with Allama of course and see if

we can recreate this entire grocer list and actually create a new grocery list from this but categorized into categories so for apples and all these

things could be produce and then we have snacks and so forth okay that's what we're going to do so you will have access to all this code of course but you notice that I now I have this data

folder here which will have this grocery list that here with all of these items and so let's go ahead and create a new file here called

categorizer py and this is where we're going to start playing around first thing going ahead and import

AMA and then we're going to import OS because we are going to interact with the directories and finding things and at the top we're going to create I'm going to just add this model so we can

use it so it's going to be be llama 3.2 okay so let's go ahead and set up the paths and I already have them set up

so let me go ahead and get that so essentially for input path it's going to be data grocery list so essentially I'm going to data get this grocery list and

output is going to be data and it's going to be called categorize grocery list at txt so it's going to be under data again once all is done

all right so the first thing is we're going to go ahead and check paths if the input file Exist by

using OS exist and pass this input file path like that and I'm going to print input file if it's not found right and then we're going to get out of there and

then we're going to read these ancored and then we're going to read the uncategorized grocery items from the input file

to do so we just open the file through the input file the path and read as F and then we take all those items we read

everything strip everything into our items so these items now should have all of the items from our grocery list text file and as we know to direct our large

language model we need to pass a prompt so to say this is what you need to do this is how you're going to be doing it right right so kind of prompted to give it some directions so it knows not to ve

off somewhere else so for that we need to have this prompt that I already created so there is the prompt so you are an assistant that categorizes and

sorts grocery items so we specify exactly what this model does okay here's a list of groceries so items going to be all of the items that we received there

please one categorize these items into appropriate categories such as produce Dairy meat Bakery beverage beverages Etc

and two sort the items alphabetically within each category so in each category you put every every item into make sure you sort um alphabetically and three

present the categorized list in a clear and organized manner using bullet points or numbering that's it so we can make this more complicated as we wish but

this should be fine all right so now we have the prompt we have of course now we have the file we have all of the items inside of this

variable here from the text file and we have the prompt okay we're passing in that information through the prompt so we can use that prompt then to send the

prompt and get the response okay so now what we do here put inside of a try catch essentially so we are generating

we're using AMA generate we pass in the model and the model remember is what we set up here okay and then we pass in the prompt

remember the prompt will have the items and of course the prompt to make sure that the large language models knows what to do and then we have generated a

response or text we get that information if we have something or we just get nothing okay return nothing okay so now we are writing the categorized list to

the output file because we want to get that what I can do also here is I can print real quick here the generated text so which is going to be the actual

response so we can do that here just for fun I can just say generated text or generated or let's say categorized

list and so then we write into a file and that's it and we categorize grocery list has has been saved and we show the

output file the path that's it all right okay that's that is pretty much it so now we're going to go ahead and run this so I'm going to open this different ways of running just run here or you can

write the command for running but that's okay so let's go ahead and what's what's going to happen so things are happening in the background and voila you can see

that we see categorize list has been saved data categorize list and there it is but also we wanted to show it here

look at this categorize list after categorizing and sorting gral items I present to you the following list so produce apples all of these items that

seems correct we have dairy cheese eggs milk and yogurt very nicely done meat Seafood chicken ground beef salmon shrimp and sausages Bakery bread flour

pasta rice Pantry almonds and all of these other items beverages that sounds great Frozen ice cream snacks chips chocolate and everything and now if I go

to see as I said we have here the categorized grocery list if I click it's going to be exactly the same where we saw printed out there but now it is

indeed a txt file now this is very small and very simple but I hope it's showing you the things that you can do and keep in mind that all of this is locally we

don't have to pay for chat DBT or open API or anything else this is something that is local to us and keep in mind we're using llama 3.2 but if you after

doing some testing and realize that for whatever you trying to do there's another large language model on another a model that will fit with what you're

trying to do then obviously it's easy to swap that large language model to use that that's the beauty because remember here I'm saying we're using Lama 3.1 but

I can go ahead and pull another large language model another model to use and nothing will change all will change is changing this to a different model that

is the beauty of a Lama all right so what I want you to do is perhaps to make this even better or maybe you have some ideas that you may want to implement here because you have

the large language model and you have some data maybe you have a list of uh something or a text that has something you want to summarize it you want to see what is the

sentiment of that text and so forth there's so many things that you can do all of that because we now have this capability through AMA of picking and

choose whichever model that we want to use incredible all right play around with this and I'll see you next so now it's time for us to look at something a little bit more complex something that

you will find in the real world we which is building rack systems with AMA so with AMA of course we can build more

complex large language model applications as I have mentioned so rag what is this thing rag stands for retrieval augmented generation so

essentially is a way for us to converse or to talk or to chat with our own documents our own data internally now the thing to keep in mind

is that large language models are actually very limited in terms of knowledge so they're really bombarded with a lot of different information textual audio and everything that's how

they were trained on but they don't know what they don't know meaning they only know what they were trained on and so it makes it really difficult for us to

build applications that are catered or that are customized to our own needs so rag solves that problem because now instead of the large language models to

know only what it knows we can actually use it because it's really good at predicting things at summarizing and kind of brainstorming and getting pieces

and pieces and putting together and get a response we can use that to solve this problem by injecting quote unquote injecting our own documents so that is

one problem that it solves but also we're solving the issue of hallucination because any large language models you've encountered it makes stuff up because it

knows a lot but if you ask questions sometimes just doesn't know but it doesn't say doesn't know it just says things that makes no sense that may seem true but they're false so this is what

we're call hallucination that's what rag comes to help large language models and in turn help us when building rag systems now let's go and look have a

deep dive into rag I think this is important especially for those of us who don't really know what rag is or or knew new to rag even if you already know what

rag is I will just go real fast so you understand what really happens an overview so the idea is very simple is that we have these documents right so

PDF files text files whatever and those are actually passed through this processing of parsing and pre-processing them what that means is that all of

those are chapped into small tiny chunks documents okay and those in turn are then put through an embedding model what

that does is it takes this textual information these documents chopped up and now they're going to be converted

into a vector space into Vector into a number representation of those chunked documents that's what embeddings mean essentially and once that happens the

vectorization of that that's what happens right it's a different format now of this data and that is saved into

a vector database okay so all of this is what we call indexing this whole process we call it indexing what happens then is that a user comes in with a question

query so that query say how old is is the universe for instance is also taken in and because of this whole ecosystem

now is into um is transformed into a vectorized system that also has to be embedded so the query is also embedded

into a model uses embedding model to embed that to vectorize this query and so that is indeed what it is actually either passed onto the vector database

or it's used for searching because now you can imagine if we are searching using text and this realm on this other side here it's a vector database or

vector store is saving everything mostly in Vector we need to have this correspondence right between we have the question but it has to be transformed

into the same format as what we have here so that way it's easier to do this searching and so what happens now is that these systems have what we call

retrieval um capabilities so they retrieve the pieces that are needed for answering the question meaning if the query is about how old is

universe is going to find all the pieces all the documents inside of the vector store that perhaps are close to that

question right and packages are all up and that's what's passed along with the prompt right the documents these pieces that we

retrieved we call them relevant documents and the query itself and that is indeed what is passed through the large language model the model so that

the large language model model has this contextual idea Sense on how to answer that question so it generates an answer which then we get a response so in

nutshell ladies and gentlemen that is what rag is and is very powerful for the reasons that I've just shown you here now what do rag systems need as you saw

well they need a few things first of all they need a large language model because that is essentially what drives most of the things essentially because we get all that information the documents

pieces of documents related documents for what we are trying to ask for and the prompt and everything that's what it takes and the large language model has

so much knowledge but also has these uh reasoning capabilities to know how to package it all up and answer that question coherently and also we need this document Corpus so essentially it's

the knowledge base what we passing through the document embeddings is very important for the reasons I saw there because you take these documents you

break them down into embeddings small pieces of vectors and if we need somewhere to save those vectors these are not just normal databases this what we call Vector databases these are

vector stores so there are many out there uh we have phas Pinacle uh pine cone and chroma DB and then we need to have this retrieval mechanism and most

Vector databases have in them some retrieval mechanism to retrieve retrieve what needs to be retrieved to then be passed along Downstream when we want to

build large language model applications we can Brute Force everything and do everything our own on our own but it's always nice to have tools that kind of

help us and Abstract out all the complexities of building large language model applications and so that's when we use tools such as Lang chain so it's a tool that makes it really really easy to

deal with large language models and build robust large language model applications Lang chain is a an amazing framework actually and there's a lot of things that it does but one of the

things that it does well is that it's able to load and parse documents easily we can use classes and wrappers to split documents and chunk them up um generate

embeddings and many many many other things okay so it provides this unified abstraction for working with large language models and building llm application

so tools are always welcome okay so I thought probably would be a good idea to look into Vector store and embeddings as an overview now it says Here length chain but essentially this is just in

general how things work when it comes to Vector store embeddings large language models how to build a rack system and how everything actually works in the

back end okay so just relax and let's go through this okay all right so this is the idea the main idea really as we saw before is that there's got to be some

way of loading documents so we need to load the documents that come in in various forms Sometimes they come in as URL PDF files text file databases and so

forth all of that is pulled together through as documents okay this is document loading and then goes through the process of splitting those documents as we saw and then once these are split

of course go through the process of embeddings and everything and that is put into storage right this is a vector space or factor store but it has to have

some retrieval as we saw because this is a way of getting information related to the question the query that comes in okay so the retriever goes through the

vector store finding things are related to the question and then get relevant splits relevant documents and that is all what's put together with the prompt

and large language model and pieces of documents and then large language model is able ble to give us the answer the result that's what we just saw so dis

spitting what happens we go through the process of splitting and the splitting documents or the chunks are then passed

through the embedding model so embedding model embeds creates this factorized or this number system or number format

representation of those texts or those pieces of information Okay so so it looks something like this literally the text is converted into vector embeddings

and looks something like that okay so embedding Vector keeps the content and the meaning of the text intact now this text with so the ideas

that DET Texs with similar content and meaning will have similar vectors and so if you look at this diagram here this di this diagram hopefully puts everything

into perspective we said that text with simp similar content and meaning will have similar vectors now remember that in this Vector space High dimensional space there's a lot of things going on

lot of high dimension right so this is very simplified so that means then if you look at cat and kitty in this Vector space in this High dimensional space you can see that they are really close in

meaning because a cat and a kitty they're essentially the same thing but a cat and run not the same thing right and so that is the representation but if you

look at the actual representation in the vector space vectors as vectors numbers you will see something similar like this because cat and kitty are pretty much

the same in terms of the semantic demeaning they will have some sort of similar numbers at some somewhere okay

that's STA and that is actually what is stored into a vector database these numbers these vectors okay those are embeded things now the great thing is

that Vector databases can be used in various cases now you can use all of this in searching for instance If You're Building A system that wants to find

relevant uh results to the query string recommendation systems for instance finding items with related text strings uh that are recommended and

classification to classify strings uh by most relevant and similar labels okay so this is some these are some use cases that are used when we talk about Vector

stores and Vector databases and vectors embeddings and so forth now the whole embedding process works as follows number one of course we have documents

that are split into chunks of text we saw this and then those are transformed into embeddings which then are added or

saved as embeddings into a vector database now the great thing here is that most Vector databases they not only save the embedding Vector they also

embed or save the original text split so on one side we're going to have the actual embedding Vector as you see here and on other side the corresponding original text so that's how most Vector

databases store things so this whole process is extremely important because when we then are ready to query the vector store for embeddings in this case

this is what happens first thing is we have here as you see let's put it all together we have the question this is the indexing part of thing the question goes through the embed process because

it has to be transformed into embedding so that the whole system understands right and then what happens is that we go through the process of indexing here

so it knows essentially to look for what is going to be similar to this embed which is part of this question so search and compare all of the entries in the vector store

and then it picks out the most similar entry and once those are picked those most similar entries those are the ones that are packaged up with everything

through the large language model so the large language model what it will do is going to take the most similar entry found and pass them into the to with the query the question the pieces of

documents and everything to then get the answer the result so that how it all works so I wanted to show you this so that you have an idea on how Vector

store and the large language model how everything comes together and rack systems so now that we understand rack systems we understand how embeddings large language model how this whole

thing comes together this is the simple rack system handson or at least the overview of what we are going to be putting together so the idea here is very simple is that we're going to use

Lang chain to pass through the PDF file to load that and then use l chains character text splitter to split that into small

chunks as we have discussed and then we're going to use AMA embeddings this is the important part AMA also has a model for embeddings that is going to

allow us to embed all of this information and put that into a chroma DB database and then what will happen is that we're going to pass in a query and we're going to use again Lang chains

multi-query retrieve that will be able to do all things that it needs to do to transform this query into embeddings of course and search the chroma database with all those

embeddings and then pull out the uh most similar pieces of information or data or documents and then put it all together relevant docs query and prompt and pass

that through a large language model which in this case again you know is going to be Lama 3.2 and of course generate the resp response that we so

desperately need so this is a r system now using AMA embeddings and using Lama 3.2 which is part of AMA now key Point here is

that for all these embeddings AMA embeddings models and this large L model we can swap these with whatever else that we might want to use and the whole

system will still work again I'm repeating myself I know that is the beauty of AMA is that once we have the system set up we can swap the embeddings

we can swap the large language model these models with whatever else we want and it will still work okay so this is what we're going to be putting together next all right so first things here I

have this B oi PDF we've seen this before so it should have access to all of this anyway it's just these beneficial ownership information report that I found online and that's what

we're going to be using for our RX system okay now one thing I've also added here is this requirements. txt so

this has all of the requirements all of the things the dependencies that we need for all these thing for all these project that we're going to be working

on so I kept it here so that we can run so Al that you have it here that way it's easier you don't have to watch me always go ahead and type things and

importing things and so forth so you have I may still do that but you have this work at txt with all the dependencies so the first thing I want to do of course is go ahead and run that

real quick so pip install. R

requirements txd like that so it's going to go ahead and get all the dependencies that was pretty fast that's that's very nice and we should be good all right very

good and next you can see that I have also created this PDF rag. py this is what we're going to be working in so I have a few things that we're going to do

here so I'm going to copy and paste in a few things so first we need to ingest the PDF files and then we need to extract text from that PDF file and

split into small chunks so essentially the same things that we have been talking about in our presentation and then we're going to send those chunks to embedding model saved embeddings perform

similar research retrieve and do all sort of things okay so that's the overview of what needs to be done all right so let's go ahead and get started now the first thing is I'm going

to import a few things here so this is from Lang chain Community now this is working now because I installed a Rand requirements TT which

included link chain community and other things so important that this happens all right so we getting the unstructured PDF loader and this online pdf loaded

here now we may use one over the other but you have both and next what I'm going to do here I'm going to have the doc path and the actual models so I'm

the model is going to be Lama 3.2 and the path through document just have one here is data Boi

PDF that's what I'm doing okay so let's load the PDF file okay so I use the unstructured PDF loader and pass the doc path which is

this one here and then data is going to get all of the loaded information right okay so now I can go ahead and actually

print for preview so we see that things are actually working and you know that things are actually working I'm going to print just about 100 items from our

content from data let's go ahead and run this fast so in the back end we should be able to do all the things like getting the

information and load the document and I'm going to should be able to see soon here

something okay very good so we went ahead and showed a few hundred words or 100 parts of the content that comes in

in our content data which is the data that we loaded which is our PDF file very good so we see here P beneficial ownership information and that is good

so just to test it out to make sure everything is working okay so that was the end of PDF ingestion so we're able to get that PDF next we're going to

extract the PDF and do some chunking essentially okay so for that we need a lot of help of course so I'm going to go ahead and get all of the import so what

happens here is that first of all we're getting ama ama embeddings this is a wrapper of AMA through L chain that's the beauty of Lang chain because we have

a lot of wrapper classes and methods and so forth and then we have this recursive character text splitter which will help us to recursively split our text and

then of course we have the chroma DB through Lang train all right so first of all let's go ahead and split things here keep in mind all we're doing here is

independent of large language models we're using this is across the board this is how you would use you would do anytime you want to create a rack system

all right so we splitting everything the chunk is 1200 so we're saying for each 1,200 chunks we're going to split so we're saying each chunk will have about

1,200 so we're saying each chunk size will have about 1,200 and the overlap is going to be 300 so the greater the overlap the better because that means we

are conserving the context okay very good and then chunks we pass this text splitter which should have all that

information actually this is all from the data that we loaded from here and then we use text splitter and split the document so now we have those chunks and

if you want to see we can go ahead and print things out real quick here so you can see okay so you can see that we have the number of chunks first says here

somewhere number of chunks is 42 and we have the actual chunks table of contents and all of that so this is good that means things are

working and the beauty of using lank chain is that it adds metadata to all of these pieces you can see now we have this metadata was added called source is

going to be data Boi PDF which is really good okay otherwise we will have to do all of that manually okay so we know this is working let me go ahead and comment this

out okay now it's time for us to add into our Vector database because we have those chunks and we need to add them but how do we add to Vector database well first we need to do something first we

need to create those embeddings if you go to ama if we go to ama.com and go to models you find many many different

models one of the models called embeddings let say embeddings you can see that we have this nomic embedding embed text we also have this MX Bay

embed large and many others okay we can use whatever want but this is the one we're going to be using so click here and you can see it's called it's a high performing open embedding model with a

large token context window which means that means it allows you to put a large amount of text for embeddings to for it to be able to embed okay so this is what

we're going to be using for our example here that means we'll have to pull this in code and so let's go ahead and do that first I'm going to go ahead and

import AMA and then I'm going to incode pull in this embedding now I already have this so it's just going to

go ahead and update I'm just put this in way so you have and then what we need to do is to actually create the vector database and it's very simple all

we do we say chroma from documents and pass the documents chunks remember the chunks are here and the embedding now we're using this AMA embeddings and

passing the mo the model nomic embed text to generate the embeddings so all of that is paying p through our cha from

documents function and of course we collection name we just pass name simple rag that's fine okay all right so continuing let's go

ahead and do the retrieval side of things so how do we retrieve things and so forth so first let's go ahead and pass in a few things here now we need to

some help with L chain where we can create prompt templates as you see and also way of parsing whatever comes in or goes through this rack system we'll see

in a second here so that's why I'm passing importing all of these and while we're here let's go ahead and also import a few other things such as the chat AMA because that's what we're going

to be using that model to as our large language model and then now we have this runable pass through this is part of what we're going to see

later part of chaining that happens with AMA with L chain we have this multi- quare retriever as you see it's going to help us with a lot of things because we're going to be able to retrieve

documents going to be able to give a query and use a large language model to sort of write a set of queries and pick up the best one to pass through the large language model so this is all

optimization under the hood okay let's go ahead and set the model that we're going to be using so this is how set model now this is a wrapper or Lama wrapper and we pass

the model which is Lama 3 to all right so we have the large language model already now as you know we're going to go ahead and use the simple technique to allow us again to generate multiple

questions from one single question and then retrieve documents based on those questions which allows us to get the best of both worlds so this is the query prompt that I'm going to use here so you

can see here I'm using the prompt template the reason why is because we can then pass things like input variables it's going to be the question that internally we'll know how to pass around all these pieces of data what you

can see here the templates say you are an AI language model assistant your task is to generate five different versions of the given user question to retrieve

relevant documents from a vector database by generating multiple perspectives on the user question your goal is to help the user overcome some

of the limitations of distance based similarity search provide these alternative questions separated by new lines original question and it pass the question which is going to be passed

along so this prompt template here it's going to be passed along soon through the Retriever and it will know how to deal with all of that this is beautiful

because we're using lank chain and it does everything automatically for us all right so now let's go ahead and actually create our retriever so our retriever here we're going to use the multi-query

retriever from large langage model and we pass a few things okay we have the vector database so we need of course the vector database that we created here and then we're going to transform it into a

retriever so retriever will know how to retrieve things from database it's going to return something and we need to pass the large language model because it

needs to know how to take that item all those information retrieved and how to process that and with that we also need the prompt so you remember in the diagram when it comes to once we have

everything set up we retrieve information we retrieve pieces in this case we're retrieving pieces that are similar to The query that we're passing along which is inside of a prompt and then we need

the large language model which is going to drive everything it's going to use all that information compact everything and be able to answer questions all right very good and next we're going to

use the create the rag prompt so let's go ahead and do that real quick so essentially it's another prompt that we can use this is our question that we pass along okay rag prompt this is just

another template we're going to pass says answer the question based only on the following context so we're going to pass the context and the actual questions this context and everything is going to passed along as you will see

later right internally with blank chain okay so once we have that we need to create an actual chat prompt from all of that so we use chat prompt from template and pass this template here so these

wrappers here internally will know how to pass in this context how to pull in the question and so forth cuz these come in as just variables okay and once we

done that that's when we then put all of that into a chain which is going to go through the process of pulling in all the all the pieces that we put together so that we can get the result so essentially what it means what is

happening here is that we passed the context we know the context is going to be the retriever retriever will know as you saw here how to get things from

database knows the large L metal model that's going to be driving everything it also knows the prompt which we created here which has a few input variables we're passing along so we're putting all

together and then we have the question runnable pass through let's just say just pass through don't mind much and then we have here a chain to pass in a prompt and then we need the large

language model again and then at the end once we have the result we're going to pass that through a string output parsel essentially to parse everything out to make everything clean and nice all that

is done we can just start asking questions so I'm going to comment this out let's start with the main question at the top

here say what is the document about for instance let's go ahead and run this so this is it this is our fullon rag system let's go ahead and run and see if this

works okay so done loading done splitting document done adding to Vector database very good okay it says here the question the answer

says the document appears to be a set of instruction for reporting beneficial ownership information especially the Boi filing instructions provide guidance

and blah blah blah very good that's exactly right and so now we can pass questions so that's one question let's see if I go and comment this out I want

this second on what are the main points as a business owner I should be aware of let's go ahead and run again now this is very inefficient because in a way it goes back and does

the splitting and adding everything but that's okay just to show you how things are done okay look at this as a business owner you should be aware of the following main points and gives us all

the points and let me do another one here okay based on provided text here's how to report beneficial ownership

information Boi to finsen complete filing and there we go all right so it is indeed working we

were able to put together a full rag system be it we could improve improve in terms of repetition of you know splitting into all the things that's

okay but you can see that it this works we use large language model in this case one large language model from our own in

this case ol Lama uh llama 3.2 and then we used embeddings also from all Lama just like that again we can just split

around we can change the models that we're using and leave everything as is and it should work very good very good indeed so I hope you're seeing uh the benefits here number one again remember

is that we don't pay anything we can run this as many times as we want we're not paying anything for any hits to the API in this case open AI or anywhere else

because this is local which is really really awesome if you ask me so I hope this is helpful and uh take this code and change it around make it your own

maybe you have a different PDF file that you want to uh converse with or whatever else that you need so go ahead and play around this is your code you have access

to it and so just just do it all right so we finished this PDF rag it works fine and so I decided actually to show you here I have the other piece of code

which is exactly same that's the same thing but I call this PDF rag clean so essentially it's the same thing as what we've seen it's just that it's a little bit cleaner and looks better okay so

essentially at the top here we have some configurations and blah blah blah and have this doc path we have this model name so this created constants at the

top here and Vector name as well okay so we have a function here called injest PDF it's essentially that we just pass the doc path and it does all the thing that it needs to do we have the split

documents function this is how exactly how you should do it right split everything into modules and then we have this create database and it returns a VOR DB and we create Retriever and

everything and we're passing along the templates and all of that so essentially like I said what we've seen before but now in a more cleaner way and then we have this

create chain which creates the chain for us as we saw before the code is still the same and then we have this main so now we just call for instance get the

data by calling injust PDF and then we get the chunks VOR DB instantiation the llm instantiation and Retriever and then we have to check and then we have our

question right so we can add different question here let's go ahead and run this we should see the same thing essentially but just different code

which is cleaner okay so it's doing all the things okay so there's a lot of good logging that's happening and voila

so at the end response according to instructions this is what we have so nothing out of this world the same thing but this is is way better because it's way organized so you have both and you

are set and because it's always nice to have some sort of a user interface I have another version you have access to called PDF Rag streamlit and in this

case when we run this we should be having we should be able to have a user interface using streamlet so essentially obviously it's the same code it's just

that we have at the bottom here streamlet let look see setting up the title and user input and everything else okay you can look at

this code but it's it's not uh out of this world so let's go ahead and run this so I can show you real quick okay there we go so now I can let's see so it's going to go

generate response verus running the load Vector DB and in a few moments hopefully we should see an answer okay so to file for beneficial

ownership information there we go all that information very good very good indeed and I can say what's the

penalty for not filing there we go and it gives us the information all right so there you have it now you have different versions and

this is the user interface version that you can see that we are conversing with our rack system with our document so quick break here I just want to let you

know that I thank you for being here for learning this awesome tool with me here I do have something for you as a way of me saying thank you for being here with

me and I appreciate you watching this video this mini course I have an olama pack that you can download that includes source SCE code code templates and many

other things that it's going to be a surprise and you can get that for free obviously you go in the link in the description and you can go and get this AMA starter pack all right okay let's continue all right so the next

application that we're going to be building we're going to still use AMA Lama 3.2 but you can always change the model and this application is going to

be a full-fledge application that allows us to actually use swarm framework which is is a new open AI framework that

allows us to create agents agents that will actually be able to delegate certain tasks to other agents and do different tasks so this is going to be

full-fledged application an AI solution application that you can actually use in real world most of the ones that we've been doing you can use in real world of course but this one is something that is

actually I would say that it's usable so if you want to get a job usually you have to find somebody who will have that job for you okay and usually you have to

go through recruiters and recruiters usually work in the agency and it's a slew of process a lot of people that are involved usually to get you matched with

an employer and so in this video I'm going to show you how to take this whole agency with different people working

these are recruiters how to actually create AI agents that will do exactly what the real recruiters do these recruitment agencies have a lot of

moving parts and one of the things that can be very difficult is that when they need to vet people they actually have to of course read the resumés and they are

getting many many of those resumés and have to go through each one of those and contact people and try and trying to figure out their qualifications and how would they match them to an employer

what would be a solution well you know this drum roll yes the solution would be AI agents with AI agents they're going to be able to pass along different tasks

to other AI agents to do exactly that from extracting information and then analyze the resume and maybe start figure out how to connect or how to

match to different employers or different positions and then have a full report with recommendation so that the whole agency knows what is

the process that went through to get to that conclusion where X person or X Y and Z people need to be matched to to a certain company and so that is exactly

what we're going to be doing in this video I'm going to show you how to use large language model in this case AMA Lama 3.2 to Aid Us in this on top of that

we're going to use swarm framework that allows us to create agents AI agents very quickly simply and to the point so

let's go ahead and let's go ahead and build a recruitment agency all with AI agents so here's what we're going to be building so here is the user interface

so it is indeed a UI recruiter agency so you can go ahead of course and upload things but most importantly you can see we have this about that we click it gives us some about this whole thing now

of course here is saying AMA 3.2 the framework swarm framework and all the things so it says here our system uses specialized AI agents to do what

well to extract information from rums and analyze candidate profiles and also match with suitable positions screen

candidates and then provide details recommendations so this is going to be a full-fledged agency that we're going to

be building using code and llama 3.2 okay of course the front end is going to be stream it which is what you see here so the idea is very simple and let me go

back to the upload so here what we can do is we can go ahead and upload files now I already have a resume that I'm going to go ahead and just drop it right

there so you can imagine now things are going to be happening in a back end so the analyzing of the resumé is happening well first of all different agents are doing what they need to do to get

started so the first agent is going to go ahead and and pull information from this PDF file which is a resume and then it's going to pass on to different agents that will analyze and do all sort

of things so this is going to take a little bit and at the end the main agent is going to be able to give us recommendations through analysis and there we go we can see that we have now

this analysis complete so we have recommendations so so we have analysis I can also close this tab menu there so scale analysis analysis for Janine

Nail's resume and gives us sales engineering industrial engineering project manager a management and all these great great stuff so it pulls in all the information from the resume and

at the end here you can see recommendation to enhance J Na's profile consider following add more experience develop technical expertise increase

Education and Training confidence is 85% wow it gives us the confidence that this person could potentially do well in

a certain industry and of course I'll show you that results also have been saved locally but you can see also have other things here we have this job

matches it tells us exactly this person matches well 80% in New York for this data scientist position all right and we

have the software engineer position and says it the match is 70% and is remote so this is exciting that's exactly what we're going to be building and also we

have the screening here screening results gives us a summary of everything qualifications and the percentage uh experience relevance and all of this

information all right and of course we have another tab the final recommendation they give us the recommendation and all of this ladies

and gentlemen is built using AI but most importantly using AI agents you can see how helpful how useful this tool can be by the way if you are interested in

being a part of a community where you find like-minded people to build AI tools from scratch and you will have templates we have weekly monthly calls

that we have for Q&A and many many other things in this community where people are actually building AI Solutions you'll have everything you need to build

AI Solutions using code and AI tools is a place where I'm very proud about and there's a lot of people who are enjoying this community and I would love for you to join the community where you'll be

able to dive deeper into building AI solutions that actually solve problem okay so let's go ahead and get started all right so I already have the code and

because of the nature of this project uh there's a lot of moving parts and uh there's a lot of code as well not too much and so what I will do I will just walk through the code and no worries

you're going to have access to all this code but what I want you to keep in mind is that you can take this and apply to something else so that is the beauty of using agents that's the beauty of

building these Solutions AI Solutions so I have here inside AI recruiter agency and we have this folder called agents let's focus on that real quick but at

the center of everything the way I wanted to organize this code is that because each agent will have to have access to the large language model in this case we're using AMA that's another

Point here is that in this case we're using AMA Lama 3.2 which means you don't have to pay for inference or pay for an

NPI key which is really great and it's really good because then you can build these systems build these AI Solutions locally and test them out before you can

push them to production so that is one of the great things about using AMA models but it is still the same if you want to use different models Okay so at the center of everything the way I

organize this code is a little bit more organized in a sense where you would find in a production application now there are a few things here that you probably will have to kind of work

around to make it production ready but I put the code so that it is a little bit more organized we're using classes and everything now if you don't know a lot about classes bython it's okay um I have

a lot of videos where I talk about python um but this is how you would usually want to put together a potentially production ready application at the center of everything we have this

base agent here what is this well it is indeed a class make this smaller so you can see as you can see here it's a class called Base Class that we pass in a few

things in this Constructor so when we call Base Class which is going to be that all agents are going to inherit from which means they're going to model

after um this is what we have so each agent of course has a name instructions so this is the model uh way that we do with

agents using swarm that is what we're using here and you pass in of course the opening eyes now because I have another video where I talk about how to use

swarm and opening ey but override opening eye in this case so that we can use l 3.2 or AMA any AMA model okay so this is how you override that I have

another video where I talk about that you can find somewhere here or you can just search on my channel okay so what we do here we pass a base URL and

instead of having in this case the base URL that would have been for openi we have our own base Ur So this tells us

that this is going to be locally using AMA in this case pointing to a llama and AMA model okay and then we pass API key

here just AMA it's not requ required but it needs to be passed around so at this point here when we instantiate this base agent we're going to know that we actually going to be using instead of of

open AI because we're using swarm framework we're going to be using our local AMA models okay and then we have this function here that is defined

called run so this will get of course the message which going to be a list and then we have this query or Lama so at this point we're going to pass a a

prompt as a string and then we're going to go through the process of calling our client or Lama client which was created here and then we say hit the chat in

completions that create so in this case here you notice we passing the Lama 3.2 and we have messages in this case we have system and we pass the self

instructions so these instructions will have passed along through the agent okay so this is how you structure it I know for some of you this may be a little bit Advanced but it's a good it's a good

exercise and we're going to pass temperature and tokens and all of that all right and then we return the response so whenever we say query orama this is what's going to happen we're

going to pass the model and then the messages all of that is going to be passed along also the prompt and the instructions okay okay so here is this

helper function here par Jon safely this is just going to bring get the text that is coming in as I said safely pars Json from text handling potential errors

that's all we're doing here so that is the base agent so now once we have the space agent what we need is to create the actual agent swarm opening ey agents

and if I go back here you can see that I have a bunch of Agents here so these agents we have for instance the extractor agent as name imply is going to be responsible for extracting

information from the PDF file and then we have the match agent which is going to match whatever we got from the extraction in this case the PDF and try

to match it with the company or the position that the person may be a good fit for we have the orchestrator and we have profile enhancement recommander and

Screen agent let's start with screen agent because all of these agents will essentially in terms of code codewise they will look pretty much the same now this is important look how beautiful

this is so the screen agent here we are creating a class just to make things simpler and scalable and then it's inheriting inheriting from the base

agent right because the base agent as we saw has all the properties all the functions or methods that needs for it to actually become an agent so that's what we're doing here we're initializing

our things so we are calling initializing the names to be screener and we have the instructions here to say screen candidates based on qualifications alignment experience and

everything so we're passing the actual instructor instructions for this agents we say provide comprehensive screen reports and all that and we have the run so this is just going to go print to say

okay screener conducting initial screening that's all that is this uh the Run function is going to do and then we have work workflow context here we are evaluating getting information from our

messages that we getting and then get the content so essentially we're going to get what the large language model in conjunction with the agent the results

and pass that into the query AMA and if you remember if I command click query AMA what does it do query AMA model with the given prompt so now we get that

information we got and then pass through AMA this agent here well it's a call essentially to this endpoint so that it knows what to do and then we get the

response and of course we return an object or in this case a dictionary which is going to have screening report and we're passing into date here we could make this more Dynamic but this is

okay and screening score 85 in this case right so let's go ahead to another agent let's go to extractor agent so what does it do well this extractor agent as name

apply it just extracts the information from a PDF file okay we importing a few things here but the idea is very simple again it's base agent is the base agent

that we pass right that's going to to be the model for this extractor agent and we have the name instruction extract and structure information for from reses

focus on personal information work experience education skills and certification and all that provide output in a clear structured in format so it's important that we make these

instructions very direct and very concise okay we have to run again so this case is going to show us this extractor is processing resume and we do the exact the same thing as we saw

before so like like I said earlier most of these are going to be the same in terms of the code right and we extract everything and also we um extract the

text in this case from the PDF file because we're going to pass that along because we take of course the raw text that we should have at this point and we pass along to query AMA as you know what

it does and then of course we return again the raw text the structure data in this case the exu structured I should say information and

the results so we're just returning a dictionary so we can see what's going on all right so it's pretty much the same let's go to matcher the matcher here you know what it's going to do it's going to

match whatever it was extracted from resume trying to match with certain positions that they might have so it's a matcher the same thing again that's the

beauty and the instructions here is that match candidate profiles with job positions consider skill match experience level location and everything

return matches in Json format with title match score and location Fields okay the same thing as you see here this run it's

going to print matcher finding suitable jobs and all the great stuff now here is where we just passing sample jobs so

this is where you would perhaps hit a an API that might have somewhere with a lot of different jobs or or a PDF file or a

CSV file whatever you have that would have different jobs so this case here we just have the samples jobs with with a few titles and everything so that's the

beauty you can just plug in the source where you'll have a list of different jobs with requirements locations and everything okay very good so that's it

that's what's going to be used for the matcher to be able to match it's going to look at the information that it's received from other people this case other agents and find the match okay so

match response so this is where the matching results from a Lama come in so what we do we call the query or Lama and we pass this F string here analyze the following profile and provide jobs

matching valid Json okay so we have the profile here it's going to be and passes in okay the analysis because in order to analyze in order to do the matching it

needs to have the pieces that it needs in this Cas anal analysis results that were gotten from a different agent okay and then we pass the available jobs is

this so here's return only adjacent object with this exact structure so it's very important to usually guide the large language models when it comes to

the formats that you want okay and there we go so this is the format that we have here make it smaller so you can see everything and then we can parse the

response to make sure that we get everything in this is PA Jon safely function that we saw in the um agent I

believe the base agent okay we have some fullbacks here uh for errors and everything and then we just return the par response now before we go to the

orchestrator here let's go to the en profile enhancer agent so what does it do well this is going to kind of enhance the profile as the name apply it's it's

going to go ahead and pass in the extracted information it's going to copy that information and enhance essentially the user the profile of that candidate and return that enhanced profile

essentially to extract what needs to extract and put all nicely okay and then of course uh the profile agent here that's what we pass we call agent

profile agent and pass Lama 3 and we say enhance the candidate's profile based on the extracted information so this is where we actually call this is where then we pass this function that we

created here okay and pass through the agent because you will know that is the function it needs to run very cool all right let's continue here and let's go to the

recommender agent so recommender agent as the name imply it's going to be the same thing it's going to recommend right so we have the instructions here generate final recommendations considering extract profile skills

analysis and provide all these information so again nothing new here and of course we're printing everything when we run and in this case here we do

the same thing we pass we call the query AMA and pass the workflow context so essentially is the information that we're getting from the evaluation of all

the messages come in okay and then we return the recommendation this is going to be sort of a a dictionary with a time stamp and confidence level in this case

we just say hi okay of course you can add more things to make it actually um realistic but this is pretty good okay let's continue let's go to screener agent now the screener agent I think we

looked at it well that's exactly what it does it screens to make sure everything is good and pass it along all right so that is what it does it's very simple

the same thing again and there you have it now the most important thing here again to keep in mind is that everything is base agent so that means all these

agents have a parent class that sets everything up so that is the best way for you to think about when you're building real world applications or Solutions all right okay let's go ahead

and look at the orchestrator the orchestrator in this case is the one that orchestrates everything that is the the the myestro if you will okay and so

it has a lot of things of course we have to import all of those but again it is still an agent so it has to um inherit from base agent so we give it a name we

give it instructions Now instructions are very important for agents okay says it coordinate the recruitment workflow and delegate tasks to spe uh specialized

agents ensure proper flow of information between extraction extraction analysis matching screening and recommendation phases phases maintain context uh

important maintain context and aggregate results from each stage okay so we got self that set up agents so the a function that is been called that goes

ahead initializes all the specialized agents all the ones that we' talked about look at that okay very good because as an orchestrator it has to know which agents that it has access to

to start delegating we have the run again the same thing we've seen and then we have this process application so essentially it's going to just go ahead and pass

some information here along and print out orchestrator starting application process and so forth so we extract the resume so it's going to go ahead and run

to go and extract resume update in this case the work context workflow context and then we analyze the candidate profile okay called self analyzer to

start the analysis right so this case we're calling all of those agents right you can have over can see it is indeed an

analyzer agent okay and then we app upate the workflow context here so each step of the way as we're calling these different or the orchestrator is calling

the these different agents is going to do what it needs to do okay delegate and create in this case add to the workflow context okay and does with matching jobs

screening candidates and then we generate the recommendation all right so it calls the recommander and that's exactly the same thing and we have some exception just in case things don't work

and that's pretty much it now we have other things here logs and everything but that's something that you can take a look yourself but this this is it and so

once we have that we also have of course the app that why a app that py should say so this has some code with streamlet

which allows us to look at something that is nice to look at as you saw earlier okay so at this point here let's go ahead and do a quick run so we say stream late

run app.py and what will happen is going to

app.py and what will happen is going to go ahead and open our agency AI recruiter agency we have a broken link here but that's okay we can go to about

for instance and tells us about this agency it's very very cool what it does and everything and all of this was created using streamlet and we go to

upload now we can actually browse and upload something a resume so I do have a resume that we can go ahead and upload I'm going to upload here real quick and the moment you upload that drop it and

it's going to start the process and the cool thing as this happens I have here logs that are actually being sent out and also I have what's happening in the

whole orchestration uh showing here orchestrator starting application process and is analyzing everything and it's working it will take a little bit

because there's a lot of things are happening and you can see all of that Happening Here Right tells us

orchestrator starting analyzer did that matching screener and recommender and it's happening so we're going to take a

little moment here until hopefully we get the final results okay so after a little bit you can see that then we have here analysis complete ah how cool is

that so we have different tabs here with different information right so we have skills analysis is it tells us exactly what happened key skills and expertise

level and goes over everything about this candidate two years of experience and it gives it so it's extracted everything from the resume and

qualifications um and also even some subjections right however the fact that her master degree is not explicitly mentioned in her work experience section raises question about how much of her

degree was applied in real world real world scenarios so this is these are agents uh just like you know real world agents real agents who would have to

think about um whatever they are thinking whatever they are analyzing so career progression potation red flags look at that all of this okay and the

confidence uh score is 85% and here if I go back to next tab look at this job matches data scientist

75% New York Senior software engineer about 60% all right we can look at screening so screening results 85% of uh

screening results with confidence based on the provided information I will perform a comprehens comprehensive screening of Janine Nel's resume qualification alignment it goes through

all of that so all these all these agents were able to work together to come to a certain uh conclusion okay and we can go through and all that it also

says results are under result results analysis so we also create a textual file txt file with all these analysis okay and I can go recommendation and based on this

analysis I would recommend that Janine now develop more explicit examples expand her Professional Network and all of these different things all right there you have it so just like that

ladies and gentlemen if you not excited about this I don't know why really will make you excited this is something that you can actually use it's usable right you can imagine that if you have in fact

here's what I will do if you have a friend of yours who is a recruiter show them this video and see if they could use something like this right don't make

them feel like their jobs are in Jeopardy no but just some ask them if they could use such thing how much time they will save them um daily from

vetting people and all that all right well thank you so much for going through this course and I really hope that this course was helpful and that you can see

the power that we get when use o Lama because now we're able to pick and choose different models and we can build really good AI Solutions locally without

having to spend a dime at least um you will have to have a computer of course you don't have to pay for opening ey keys and all of that so I hope this was useful I hope this was helpful to you

the thing also I've been hinting throughout this entire course is that I do have a surprise for you which is what I call Ama startup pack which will have

everything that you need to get started with AMA from code templates um and other things that are surprise so you should see a link in the descriptions if you don't see it you can you can just

leave a comment as well I'll show you the link where you can go and get all of that again thank you so much for your time and I really hope this was helpful if you have any questions at all just

leave them below and I'll do my best to um serve you to answer them thank you again till next time be well