Orchestrating Complex AI Workflows with AI Agents & LLMs

Artificial intelligence is said to transform IT. Yesterday I asked AI how many AI agents are being created every day? The answer was 11,000 per day based on news releases and other public sources. At that pace, we'll have over a million new AI agents deployed this

public sources. At that pace, we'll have over a million new AI agents deployed this year. While there's no way to know exactly how many AI agents are being created each day, chances

year. While there's no way to know exactly how many AI agents are being created each day, chances are that you will be asked to work on a project to develop AI agents, or to begin working with an orchestration platform to work with orchestration of complex workflows in your environment using AI

agents. There's some good news. Ah, orchestration of workflows with agents are in

agents. There's some good news. Ah, orchestration of workflows with agents are in many ways an extension of some of the established frameworks and tools that most developers have worked with before. Today we'll take a look at how agent orchestration is fitting into the

existing IT ecosystem. Okay, now the big new new kid on the block, uh, are these GPT models um that are that are allowing us to have these large language models. And what we're what we're seeing with the LLM is that we're bringing

in a, a strong language um faculty that is allowing us to open up the kind of logic that we're dealing with ah when we're automating a business task with technology. Right. So we know that the LLMs are trained on massive text datasets. They understand

technology. Right. So we know that the LLMs are trained on massive text datasets. They understand

human language. Um. And and so this is a really nice um component to to pull into our design framework when we're when we're building software. So, some of the really, um, you know, some of the really common software that we're, that we're dealing with is

broadly assistance. Okay. And,

broadly assistance. Okay. And,

agents. And the simple thing to remember about assistants and agents um are that they're actually very similar. Um. But assistants are going to, to really be driven in a, in a prompt response framework. Right? So, you know, we ask a question. It's called a prompt. And then we get, you

response framework. Right? So, you know, we ask a question. It's called a prompt. And then we get, you know, basically we get an answer uh, a response. Okay. Now, with agents. Um, agents

don't really need to be prompted in this way. Um, so with agents, what we're really doing is we're talking about defining goals. And then what we want out are outcomes. Okay. And so again, assistance, we're sort of asking

outcomes. Okay. And so again, assistance, we're sort of asking questions and looking for answers. Agents, we're defining goals and we're looking for outcomes. Now, the

the really big um, the really big difference here is down to the definition of the word agency. An

agency means that we are giving the software agency to actually take action at its discretion within the boundaries that we set. Whereas an assistant, an assistant is just gonna sit there until it's prompted. Right? And then it's gonna be at the ready to answer us when we prompt it. Okay? So, remember that when we're developing assistants and when we're developing

agents, um, there's a lot of news but new buzzwords there's sort of small language models, large language models ,constrain language models. Um, you know, with agents, there are a whole bunch of different design decisions that that we make. Um, but at the end of the day, it's it's very important to remember that this is software. Right? And so experienced software engineers should come into this space ah with

some confidence that you can bring all of your best practices, ah, and past project experience. Um.

And and all of those those same things are gonna serve you very well. So, most of the developers, um, that that I interact with have have have mentioned that ,once they really jump in and get started working with assistants and working in these agentic frameworks, that they're able to have, they're able to make progress quite quickly um, and have a little bit of fun, as well, with these new

with these new technologies. So, I was having a conversation ah recently, maybe a little bit of a debate on with a friend of mine, and we were talking about agents and orchestration layers.

And my friend said, hey, why? What is different about an orchestration layer? Is it really just robotic process automation with LLMs added into it? And ah, you know, I thought that was a that was an interesting take. Um. So interesting that, you know, I think, when we're talking about people who are

interesting take. Um. So interesting that, you know, I think, when we're talking about people who are building things and and talking about things, we often like an example. So, let's let's imagine that we have a business process here with three steps. And and, obviously, the three steps here are gonna be supported by technology. And, let's talk about

what it would look like to bring an orchestration layer into this process. And so, the the first thing we want to remember when we're talking about orchestration. So we have we have a flow like this. And, when we when we talk about orchestration and

orchestration. So we have we have a flow like this. And, when we when we talk about orchestration and agents, we just said a moment ago that we're bringing, we're bringing, um, we're bringing LLMs into place. What I'm gonna do is I'm gonna create a little triangle agent here. This is an agent. And

place. What I'm gonna do is I'm gonna create a little triangle agent here. This is an agent. And

we said that, you know, agents have LLM capabilities. And, we we stressed that that we're talking about goals and outcomes. Okay? So,

let's say let's take an example of a process where, you know, the thing that we want out is a customer quote. So an actual quote that we can send to a customer.

customer quote. So an actual quote that we can send to a customer.

And the thing that we're putting in is a goal. Okay? And the goal is to create a quote a create a quote. So we want to create a commercial quote that's good enough so that our sales team can send that quote out to a customer and not get us

in trouble with all the finance people and the product people. So, the the the way to think about orchestration versus robotic process automation.

Okay? So, you know, ultimately, they're similar in that we wanna understand our process. So, in this case, it's kind of left to right. We wanna

understand our process. And we want to boil this down into a really, really tight job story so that we understand, if we want to create a quote, what has to happen so that we get a quote on the other side that's conforming to all of our, ah, to all of our desires. So, in the old days, robotic process automation would typically have to interact with, let's say, the first step here is maybe we

have ah an application like a CRM. And this is where we kinda understand, you know, where we are in step in a sales process. And this is where we know when we need to when it's time to create a quote.

Okay. Now, the API would would probably have a difficult time um, discerning when to when it's time to make a quote, unless we had a really strong like programmatic step built in here where we click, you know, make a quote or something. It would have to be really obvious. Okay?

Because we're gonna through the API, we're gonna be able to access very specific actions, like almost like calling a so a software process. And, we're gonna be able to access like very, very specific structured data. So, usually like data tables, you know, or keyword pairs or things that

are very structured in the RPA framework. Okay? So, after we after we go in here, let's assume we can go in here and we can detect when we need to create a quote. And let's say that we can also retrieve valuable information, like the customer name and the customer address and some of the other, you know, particulars that we're gonna need along the way here to get over here ah, where we wanna

get to to our quote. So, then let's say the middle process is something like, um, you know, product or product SKUs. Um, and, and maybe and maybe product catalog. Okay. So here maybe we have a database of ah of

product catalog. Okay. So here maybe we have a database of ah of of approved product SKUs and that maybe we have a separate database that's kind of in this container that's going to give us detailed product catalog and more, more descriptions um of

those SKUs. And then lastly, you know, over here, this might be some kind of like ah sort of financial

those SKUs. And then lastly, you know, over here, this might be some kind of like ah sort of financial application or fric financial container that knows how to look at what we pull together and sort of, um, you know, correctly put prices on things. Um, and then maybe there's also sort of a legal container ah,

or legal information in this container where we kind of know our, our Ts and Cs that we wanna attach ah based on the based on the combination of SKUs that we put together. Okay. So, so this is kind of a very simple cartoon of, of a process where we're trying to, you know, when we wanna create a quote, we wanna leverage, you know, a CRM application, um, a data source that has product SKUs

and product catalog. then we wanna get into some kind of financial application that can price SKUs and and and also apply legal T's and C's based on those SKUs. Okay? Now, in the in the RPA world, you know, what we're gonna need is we're gonna need APIs um, and and kind of really, well,

well-defined data tables into each of these resources. But we're probably gonna run into some real problems when we when we try to

resources. But we're probably gonna run into some real problems when we when we try to do this use case, um, because we're gonna really need to make sure that all of our, our applications are configured in ways that are very, very highly structured and that really provide us explicit triggers and explicit structure um around this problem. So, probably not

impossible. But if any of you have tried to use RPA to do something like creating quotes for your

impossible. But if any of you have tried to use RPA to do something like creating quotes for your sales team uh, drop it in the comments and tell us how it went. I'm sure there there I'm sure there's some interesting challenges that you've run into. Now, when we go to the orchestration, we suddenly have agents. Okay? And the beautiful thing about agents is that we can have many of them, okay, that are

agents. Okay? And the beautiful thing about agents is that we can have many of them, okay, that are going to be working together. So, with the agents, we're going to we're gonna create, ah, you know, in reality, let me make sure I have my agent colors still the same here. Um. In reality, we're gonna make, you know, we're probably gonna make an army of of little agents. Right? So, agents, you you know do

really well when we keep them kind of narrowly defined and and and keep their job stories really, really tight. So, because we're gonna give these things agency, and and that means we don't want them

tight. So, because we're gonna give these things agency, and and that means we don't want them to get off the rails. We don't want them. We don't wanna hand these things LLMs and give them a big scope and, you know, and have these things doing ah a lot of things that are not useful. So, we're gonna create ,we're probably gonna have kind of a master agent for this process that the

orchestration layer is gonna u, is is gonna leverage to then delegate parts of this overall process to this little army of agents we're gonna end up with over here. So, when we're orchestrating the, okay, we wanna create a quote. So, now each of these resources like our CRM, for example, our CRM is going to need to become an MCP host. Um,

and it's gonna need to spawn an M an MCP service very similar to down here. We're in kind of a client server architecture, so not much is changing in terms of the overall framework. Ah. But

now that we have this MCP service, we can kind of spawn, we can kind of spawn our, you know, we can spawn our, our agents. This might be our master agent. And then our master agent. We're gonna get a little fancy with the artwork here. Sorry. And then our master agent, you

agent. We're gonna get a little fancy with the artwork here. Sorry. And then our master agent, you know, may spawn agents, one and two that that are really well trained on how to dive through this MCP ser ah service in this hosted area. And these two, these two agents are really

good at evaluating where we are. So this first agent might be able to evaluate, okay, and identify, hey, we're in the right step of the process to create a quote. So, this agent may say yes, we need to create a quote for this specific ah opportunity. Um. And then agent two might be a grabber

agent. So when agent one says yep, this is the this is time to generate a quote, um, a message may be sent

agent. So when agent one says yep, this is the this is time to generate a quote, um, a message may be sent to agent two. And agent two may go in and grab, you know, customer name, customer number, um, customer address, um, and then may pull documents that have been attached into the into the CRM workflow to

pull out, you know, what products and services have been discussed with the customer um and what is it?

What does it look like we need to pull together for a quote? That information then comes back and can be checkpointed in the in the orchestration layer, so that now these agents, agent one and two have now done their job and they're released and they go away. What we've done is we've cached some context data so that now we launch um, our sort of

captain, our captain agent here, um, now, now launches agents, you know, three and four.

And agents three and four. Try to keep my colors the same. Here, this

is this, this is now an MCP. We're gonna need all of these things to be MCP services. Okay. So that our agents know how to talk to them. And so, basically the agents three and

services. Okay. So that our agents know how to talk to them. And so, basically the agents three and four are gonna dive through the MCP of, ultimately, a data source. So now instead of lots of tools, we're describing really specific data sources around product SKUs and product catalog.

So, agent three maybe designed to go take a look at the data that we fetched that agent two fetched and then say, okay, I know how to take the data that agent two fetch, and I know how to go in here, and I know how to interpret the product SKUs so that now I can pull together a

list of SKUs that I believe satisfies the the requirements for this customer. And then the those may be handed off to agent four. And agent four may know may know how to come in here and navigate the product catalog. So, agent four says I've gotta go onto the product catalog, and I've gotta check

product catalog. So, agent four says I've gotta go onto the product catalog, and I've gotta check and make sure that the list of SKUs that agent three gave me all work together and that we can ship them together. So I might be checking compatibility. I might be checking, um, you know, legal constraints or commercial terms. Um. I may even be doing really higher-level logic like checking

against whether or not these SKUs um, are aligned with the current, um, product goals, sales goals, etc. So, really, this can get quite complicated and and can can become much richer than what we used to think about, you know, down here in in robotic process automation. So now the these agents three and four

come back, you know, and they add some more data into our into our story up here. So, over here we got some like customer info and and and and a list of what we thought needed to be in the quote. Over here we started adding useful stuff like we got we got SKUs that we think we need to go in the quote. And we

did a lot of checks around whether or not these SKUs go together off the product catalog and again, sales goals. And then we move forward and say, okay, now we're moving into this financial, legal Ts and Cs area. So again, our our little master agent knows where we're at. Three and four are released. And now we move. We need green again. And now we move

at. Three and four are released. And now we move. We need green again. And now we move into agents, um, five and six. So we call in our agent five and six team.

And again we dive into the MCP service, into our financial and and legal, um, terms and conditions catalog. Okay. Now our legal department, you know, may have already had has already set up this

conditions catalog. Okay. Now our legal department, you know, may have already had has already set up this data source for us and has vetted it. And our financial, uh, teams have already vetted, um, you know, our pricing, um, etc. Now, our first, uh, agent five may come in and knows to take

our, ah, SKU list and come in here. And and agent five prices our SKU list. Okay. So

agent five is is pricing. And this probably involves, you know, inferring or looking at sort of some notion of quantity. Um, and then agent six may come in and understand how to interpret our legal catalog. And so when agent six comes in and interprets the legal legal catalog, we pull

together, um, you know, the the the terms and conditions and those sorts of things. So now the now agents five and six can make up here. And we've, you know, we've added some data. I guess I didn't write the word data, but this is all data up here. So we've added some, you know, we've added some pricing and we've

added some legal Ts and Cs. And now you can kinda the story is kinda writing itself at this point. Right? So now, you know, what what can we do over here? Well, we we know the customer

point. Right? So now, you know, what what can we do over here? Well, we we know the customer name. You know we know the customer address. We know, um, you know, we

name. You know we know the customer address. We know, um, you know, we probably, uh, we probably uh know some of the details about what's been agreed between us and this customer. Like, do we have a MSA in place? Um, and so then over here in a quote, we move forward and

customer. Like, do we have a MSA in place? Um, and so then over here in a quote, we move forward and we start to have, um, number and SKUs of what we think we need to ship. We've already

done some checks against product catalog and sales goals.

And lastly, you know, we get into pricing and legal Ts and Cs. And, you know, just for fun. And um,

Cs. And, you know, just for fun. And um,

just for fun, you know, we'll say that there was an agent seven to top this off. And, you know, agent seven maybe maybe operates up here. And, you know, when we get when we get when we get our three processes completed, agent seven creates our

quote in a really nice format in a way everybody's gonna like. And that's how we meet our goal. So, looking at this example, I think what we can see is that both approaches, the the agentic orchestration and the robotic process automation, are ultimately geared to increase

productivity by automating these low-value tasks so that our teams can focus on the more high-value ah goal here of increasing revenue. so, you know, going back to the to the conversation that I had with my friend, when you really start looking at the richness of what can be done with agents and orchestration versus RPA, we really see this as a paradigm shift in what's

possible as opposed to an incremental step forward.