Build and deploy LlamaAgents from scratch

Hi everyone, Cleia here from Llama Index

and today I'm going to walk you through

building a Llama agent from the grounds

up. So first of all, what is a llama

agent? A llama agent is basically uh our

brand new product which is currently in

private alpha that lets you build and

deploy workflows both as a backend and

as a fullstack service on our cloud

platform Lama cloud. more of this later

when we will actually see a llama agent

in action. So first of all u let's

understand what we want to build here.

So we want to build an anomaly detection

workflow and anomaly detection llama

agent for pest control and the scenario

is very simple. You have to imagine you

are a farmer and you have a crop and

your sword enemies are agricultural

pests which are normally insects that

eat your crop lowering the productivity

of your farm. So in order to understand

whether or not the pest can be of damage

can be detrimental for your uh

productivity you have to monitor and you

generally divide your crop into smaller

sub areas and you count you sample the

population for these sub areas counting

the individuals. If you have a low

number of individuals per area this

means it's good but if you have a high

number of individuals per area this is

very bad and it can lead to a lower

productivity because the best is eating

the crop. So how can you avoid all of

this? How can you keep the best under

control? Obviously the monitoring is

only the first step because then once

you produce the data you have to compare

this data with historical normal

population data. So normal population

data and you can do this uh via a vector

search engine for example quadrant which

is the vector search engine that we're

going to use today. Uh which stores

basically the data as vectors and you

can imagine that basically what we can

do here is the same. We can represent

the count of population samples as

matrices and as vectors and we can

compare them with the vector data stored

in our vector search engine. And so what

happens here is that we can now then

calculate the distance of the recorded

data, the recorded data and the I would

say reference normal population data.

And if the distance is too high, that's

probably an abnormal population growth,

an anomaly that has been detected. And

what we do then is basically start pest

management or put into place certain

number of techniques to reduce the pest

population. So how are we going to build

this workflow? So in order to build this

workflow we will do basically um we will

have basically I would say three steps

three main steps. The first the first

step is that we're going to take we're

going to receive from our front end and

we will see this in a second. We're

going to receive the data as a string as

a str as as a CSV string. And so we're

going to turn that CSV string into a

polar data frame. Once we have that CSV

string in a polish data frame we can do

two things. we can visualize the data.

So as a heat map, a density plot that

displays how the how the population is

distributed across the the different sub

areas of our of our crop. And we upload

this image to superbase or in general

any S3 compatible storage so that we

have this image available to pull back

into our front end when the workflow is

done. And then always um like again from

the polar data frame we can represent

the polar data frame as a vector and we

can use that for to search for similar

data points in quadrant. So obviously if

there are no similar data points because

the distance of our basically of our

data of our data generated data is uh

too high uh we will probably run into an

anomaly. So anomaly an anomaly will be

detected and so we will say okay this

data is anomalous whereas on the other

side if there are similar points this

mean that our data is probably non

anomalous. So this is how we will

perform anomaly detection and once we

will have this workflow deployed you

will see that we will have a front end

which will send which will uh allow us

to generate data and to send this data

through the whole pipeline of our

workflow which at that point will be a

lama agent a deployed llama agent that

will generate all the data and pull them

back to the front end uh with the heat

map with the heat map image and with the

anomaly detection prediction. So that is

basically what we're going to build

today. We're not going to uh dive deep

into how we implement the front end, but

we're going to see how you can build the

LM index agent with code.

So here we are and we're going to start

building our L agent with code. So first

thing that we need to do is we need to

set up our our directory, our

repository. And I am going to use VV as

my goto tool for Python project. So I

would say uv init dot and I will

initialize basically my repository. I

will delete the main file because we do

not need it for now and I will create a

directory called src anomaly detection.

So sorry src

I will then add some um some like

default files and uh this is basically

what we need to start. So we need a

Python package or Python basically

Python script that we can reference from

our PI project.mml file. And so what

we're going to do is basically we're

going to add some details as for example

to build back end which is not that much

of an interest to us and uh we're going

to to add actually the reference to our

workflow to what we will build as a

workflow uh for uh our for our

deployment. So basically we will say

that the workflow will be placed under

anomaly detection workflow main uh

workflow which is which will be in the

name that we will give to our workflow

instance and we will set basically the

name of our deployment with llama

deploy. Uh the name of the deployment is

anomaly detection workflow. It will uh

it will inherit the environment

variables from ourd file and it will be

deployed on llama cloud uh setting lac

cloud equal true. So let's now build our

workflow and let's get back to the uh

our sketch to see what we need. So first

of all we need polars then obviously we

need soup base we need quadrant and we

need obviously lama index workflows. So

we're going to add all these

dependencies

and we're going to install all of these

dependencies. As you can see, V is

extremely fast and it also created for

us avmbages

installed. So this is great. Uh this

works. So now we need what we need to do

is we need to build first of all our

third party I would say dependencies

that can help us upload to superbase

search for quadrant and obviously also

represent our data as a polar data

frame. So let's start with the first

step which is representing our data as a

polar data frame and let's create

something that's called data processing.

Let's put in a py file and let's create

also a processing

py file. Here we will define our data

processing uh logic. Then obviously we

need some S3 storage logic and so we

will create an S3 storage

and we will then create also here a doy

file and a storage file and then

obviously we need um the quadrant utils

the vector search

let's say even better vector store

create also our init.y py file and we

will create a vector

store py uh file. So we will populate

all of these files one by one. Let's

start with our processing file and now

processing our processing file we will

define a data processor class that first

of all validates the data ensuring that

they are in the format that we expect

them to be. Then it loads the data into

a polish data frame and then it plots

the data into a um heat map that is

saved into a temporary file and that

will be used to uh upload obviously to

superbase. And speaking of superbase, we

define here an S3 storage class under

the S3 storage utility I would say

package uh and module. And obviously uh

we need to connect to uh the S3. We need

to connect to superbase first of all. So

we define a bunch of a a bunch of like

connection parameters. Then we uh

basically that we basically create a um

store function that is able first of all

to upload uh the that is able to upload

our im the image to a storage to a

storage space on superbase what they

call a storage bucket and then obviously

uh we will create a uh URL for that. we

will get a URL for that image uh and we

will return that URL if nothing bad

happened during the storing operation.

So basically all of this uh is just to

have a URL of the image to display that

into the front end. Last but not least u

our anomaly detection client our anomaly

detection class anomaly detector which

has to connect to a quadrant uh cloud

instance. So we have some also here

connection details and we have to

basically get a quadrant client and then

we define anomaly by just uh creating a

vector from our data frame our polar

data frame and then querying the points

and having basically a score threshold

that is the minimum distance that is the

maximum allow distance between the

points and uh the and like basically the

query uh vector for these points to be

considered similar. And so obviously

this is an anomaly if if the length of

the points in the result is zero. But if

we do retrieve similar results obviously

this will be not anomalous data. This

will be normal data.

So now that we have all our utility

third party modules um ready we can

create our we can create our workflow

and we will create it here as we said

under the workflow module. We'll type in

an inpy also here and we will create the

uh we will create obviously three files

a main.py which will contain our main

workflow and as I like to separate the

concerns here I will have an events py

file that will have all the events in my

workflow and our resources

py file that contains all the resources

that my workflow is using. So let's fill

them up. First of all, let's fill up the

events uh the events file. And so we

will obviously define an input event

with the data uh as a string as we said

because we expect the front end to send

the data as a CSS string. Uh the first

obviously step is processing data. So we

will expect that obviously we will have

a produced heat map a produced heat map

as an image and so we will return the

path to the image. We said it's a

temporary file that we just use for

storage for like temporary storage of

the image on the local machine and then

obviously we have the image stored on

superbase. So there's no need we will

discard we will remove that image. Uh

then we will proceed by uploading the

image and so we will return the URL of

the image and um last but not least we

will output like we will provide the

output of the workflow with obviously

whether the data are anomalous or not

the URL of the image and whether there

was an error during the execution of the

work workflow because there can be an

error what wherever between the starting

step and the last step. So we have to

handle also that. So now let's move on

to resources. As resources obviously

what uh we need is basically the S3

storage. So we have our S3 storage. We

have the super superbase URL, the

superbase API key and the super

superbase bucket ID which we'll put into

a MV file and which will be available

also to our llama agent when we deploy

it. We will see it in a second. Uh and

we have the anomaly detector. So these

are the resources that we can inject

within our workflow. And then obviously

we can define the workflow state. So we

have basically an internal

representation of the workflow with

variables such as uh the data frame

which are available throughout the whole

workflow execution. So now that we're

ready with both events and resources,

let's define our workflow.

So the workflow is this one and as we

said we define it in the main uh file

and the workflow is this anomaly

detection workflow that first as first

step uh is that as a first step has the

data loading the data process loading

and processing. The second step is

uploading the image and the third step

is detecting the anomaly. As you can

see, our workflow uses the workflow

state as an internal representation as

well as our resources like the S3

storage annotated here and the anomaly

detection uh resource annotated here. So

we basically have everything in place.

Our workflow will run through the first

step with the input data with coming

from the input event. Then if everything

goes well, it will produce a data

processed event which will go to the

second step. If everything goes well and

the image is has been uploaded, it will

produce an image uploaded event with the

URL of the image and finally it will

search basically for the anomaly uh for

an anomaly and so it will produce an

output event with uh the anomaly

evaluation and with the uh URL of the

image. If anything goes wrong in the

previous step, it those steps will

produce a output event reporting an

error occurred while processing the data

or while uploading the image. Last but

not least, as we specified in our work

by project.mml, the name of the variable

that has the workflow is workflow. So we

have to specify a variable in the main

um in the main subm module uh called

workflow and assign it to anomaly

detection workflow with a timeout of

approximately of 5 minutes. So this is

our workflow and this is basically ready

to go. So we can uh from here

try to let's say preview our workflow

deployment our workflow backend

deployment using uh llama ctl we can

which we can install in our environment

u pip install llama. So now that we have

llama ctl we can take a look at the uh

different subcomands that we can use and

we can in order to preview our workflow

we can do llama ctl surf. This will kick

off basically a build of the

environment. We already have all the

packages installed. So this will start

basically our backend application here

with everything uh that we that we need.

Obviously this won't work because we

haven't yet defined our environment

variables. But you can see that you can

preview the deployments. You can preview

all the endpoints. And for example, uh

we can take a look at presumably uh like

the health of our application just by

running a simple uh curl command and

just like curl the health of the server

and you can see that the status of our

server is healthy.

So uh yeah, you can see obviously all

the things that have been uh that have

been logged here. So once we have

everything set up, we need basically we

only need basically the uh environment

variable. So what we're going to do is

we will create a MV file in which we

will basically put all our environment

variable. So here they are. We have all

the environments variable in our MV file

that we specified here as this

environment source of our LMA deploy or

actually llama CDL in this case uh tool.

And so we can now do llama cdl serve and

this will actually obviously work also

as a backend service. So we could post a

we could post basically to this run

endpoints and we will get the result of

the workflow. But we won't do that uh

right here. This is just for you to see

that you can also preview your

deployments uh into in your uh

environment. And we will now what we

will now do is that we will create a

deployment. We will create a workflow

deployment directly uh on GitHub. So we

will create a GitHub repository and we

will create a deployment.

So first of all in order to create a

GitHub repository and create a

deployment we will need obviously we

will already have the the the directory

is already a git repository because u

init also creates a git repository. So

we just need to add all our changes

and commit them.

Once we have that, we can then create a

repository on GitHub uh using ghrele

create, but you can also do it through

the browser uh user interface. Uh we can

create we can create this uh and we can

just describe it as an anomaly detection

workflow. So we'll set the visibility to

public. We'll add a remote. We'll call

this remote origin and we will push all

the local changes to the remote. So now

we have our uh repository created and so

we can deploy our llama agent uh

directly from this GitHub reference. So

first thing that we can do basically to

deploy the llama agent is do llamasl

deployment create. What this will do is

will prompt us to create a deployment.

But there will be a problem here. If you

are not authorized to create a

deployment because you are not uh um

authenticated like you're not logged in

into LANC cloud you want to first run

lacl out and like lamacl and you will

see basically you can do lamacl out

login or you can use a api token to

login to lama cloud. So I am already

logged in so we're not going to we're

not going to do that again but you will

have to do that. you will be prompted to

do it if you are not yet logged in when

you create a deployment. So let's now

create a deployment. As you can see, we

say anomaly detection workflow has the

name of our deployment. Uh we have our

GitHub repository and the GitHub

reference is our branch. Uh we don't

have any config file. We don't need

personal access token because we already

we are already actually authenticated

and we can see that llama has already

pulled in all the secrets from our

environment. So now what we just need to

do is we just need to save this and

basically create the deployment and as

you can see it'll it already gives us

the uh endpoint um on which the

deployment is going to be uh is going to

be created and as you can see the

deployment is not pending but uh the

environment uh the environment is

basically getting started. Um UV is

installing all the dependencies and now

we can start seeing the informations

from the application and you can see

that the application has actually

started and presumably in a second our

status will be will go from pending to

running. So we deployed our application.

So basically what happens here is we

deployed our application but maybe we

want also we want to change something.

Let's say I want to be more uh specific

here and I want to say I am sorry but an

error occurred while processing data and

I can copy this message and I maybe want

to push this to my repository. So I will

have get art get minus ms

and I will push.

So now obviously what you would expect

is your deployment to be up to date with

this. So in order to put your deployment

up to date, you just run lamasl

deployments

updates and you will be prompted to

select the anomaly like the um

deployment you want to you want to

update. We will select anomaly detection

workflow and as you can see we updated

from the first commit to the last

commit. As you can see you have uh the

last commit here. So uh you updated your

deployment and you want to see that the

deployment is being rolled out. So you

first you run lacl deployment sket and

as you can see our deployment is right

now rolling out its update and it won't

be available for a while uh while it's

rolling out presumably a couple of

minutes or even less if your uh if your

boot time is lower and as you can see

now this was even less than a couple of

minutes the status is now running but

how can I see how can I access this uh

from the internet from a HTTPS endpoint

So in order to say this, you just need

to go to cloud.lamicex.ai.

So to your personal account and you need

to go to agents if you have access to

the agents and as you can see here we

have our anomaly detection workflow and

you can take a look and take a pick to

whatever uh here is needed uh to um run

the workflow and you can basically use

this and also test out this endpoint

directly this endpoints directly from

here. So now that we built everything,

now that we did everything, let's look

at this in action. This workflow that we

just deployed, let's look at it in

action. And so in order to look at it in

action, I like I created this demo

hooking up the basically the this

workflow here, the anomaly detection

workflow with a very very simple front

end that we saw here which is a go web

server deployed with dolo which has a

generate data command in which you can

generate synthetic data. In this case we

generated anomalous data and then you

can go and detect anomalous just by

pasting the generated data. You can now

run evaluate data and this evaluate data

will in this case detect an anomaly and

return you the generated heat map

exactly as we expect our workflow to do.

You know the we pull the generated heat

map from superbase and we detect

anomalies using quadrant. So this is

pretty much everything like from

building basically a llama agent from

scratch. Uh if you are interested and if

you want to register to Llama agents,

you can uh input a couple of data and

obviously um you we will get back to you

uh regarding your possibility to access.

Basically, as you can see, our Llama

agents are um what we would say a secure

and self-contained environment, a cloud

environment accessible to you through

your uh Llama cloud API key that helps

you uh basically deploy uh deploy

workflows that you can use as backend

for projects, projects like this one

that I just showed you. I hope you

enjoyed this video and I hope to see you

in the next one.