Agent Framework: Building Blocks for the Next Generation of AI Agents

Once we run this, we get an uh an experience that looks like this. And so

what we see is that um we see a new UI and the exact same agent that we've developed, we just defined we see here.

Hello everyone, welcome to open at Microsoft show. We have a very nice team

Microsoft show. We have a very nice team here together getting together to do like a big announcement.

Hello everyone.

Hello.

Hello.

Wow, we have all the time zones here. It

is a big day or a big night for some people that for me is a big day. And so,

welcome this new team or this team that's getting together to do to do this big announcement. Elijah, that's up to you.

announcement. Elijah, that's up to you.

Tell me what you're going to present today. What's this big announcement?

today. What's this big announcement?

Welcome.

Yes. Thanks so much for having us, George. super excited to be here. We

George. super excited to be here. We

really do have the dream team here today on on open at Microsoft. So we're

excited. So we are super um proud, excited uh all the emotions to uh announce today the uh Microsoft

Microsoft agent framework. Um this is the uh the framework for doing your multi-agent orchestrations. Um here at

multi-agent orchestrations. Um here at Microsoft, we've been working really hard on this and we're excited to bring it to the world. Um, and there we've got a lot of great features that we're going to dive into today, but we wanted to

highlight just at the top that that this works on has open standards. We have a great pipeline set up to work with Microsoft Research and it's very very much committed to open-source and

communitydriven um as well as ready to get uh into your production workflows.

And so we want to give you all a little bit of context if we uh uh about kind of how this framework came about.

Previously we had two great frameworks.

Semantic kernel which was very much designed for um production workloads was very much uh uh uh great for for all um kind of heavy duty uh uh workflows that

you may have or or doing multi- agent orchestration that way. And we also had autogen which had come out of the Microsoft research side of the house was really great for quick prototypes um was

very simple and and and uh uh could could do a lot a lot of great things uh as well. And what we've tried to do is

as well. And what we've tried to do is bring the best of both of these frameworks together into one unified Microsoft agent framework. And there's

we're going to get into a little bit more about this. And so if we want to continue, we can also then talk about uh um how the different ways that we can uh orchestrate these multi- aent systems with agent framework. One is agent

driven orchestration. And for those fans

driven orchestration. And for those fans of semantic kernel and autogen, you're familiar with things like the magentic pattern or the concurrent pattern. We

got a lot of great multi-agent patterns.

And these are allow a lot of uh agents to be able to talk to each other. We

know that individ while individual agents are great, multi- aents can uh kind of uh accomplish more complex tasks um and that you can orchestrate these through a lot of great patterns. We're

also super excited that we're going to talk about in future videos too to announce this workflow orchestration which fans of semantic kernel process framework may uh will be really excited for this one. And this allows you to

combine your traditional business uh processes and and business events with multi- aent orchestration. So here we've got a combination of agents deal working

with um b traditional business processes and you can plug in agents where it makes the most sense in your in those flows. And so I'm going to turn it over

flows. And so I'm going to turn it over to Evan who's going to talk to us a little bit more about um how this looks in comparison to what we do with semantic kernel. Evan.

semantic kernel. Evan.

Yeah, thank you so much. I'm going just to share here, you know, the the URL that's now on GitHub. Uh you can access leave like a a staff for this team

because they're working hard on this for for a while. So everybody put a hands up there for the team. Yeah. Well done. And

Evan, yeah, follow the link there, leave a star and and leave your comments as well. open your issues

well. open your issues and don't worry that they're going to work on like a migration path for SAT and uh you know for autoen as well. Evan

tell me.

Yes. Yeah. So I'm I'm really excited to to introduce uh the agent framework today and also let people know this is this is they're going to be familiar patterns. Um one

thing I'm really excited about is that in semantic kernel we had lots of agents. uh we had lots of different

agents. uh we had lots of different choices for agents and a lot of questions we got were well when do I use you know agent A or agent B or agent C

type uh for for my you know my use case and so one thing that we've done you can see here on the screen is that we have the ability to create a responses agent here on the left with semantic kernel takes a couple lines of code to create a

client and then you have to you know make sure you have your settings you give your instructions and so on underneath we have the very familiar chat completion agent So it uses the AI

services underneath. What we've done

services underneath. What we've done today is we're simplifying this and we're providing one chat agent and the ability to drop in a chat client. And so

now you can stop worrying about well which agent do I need to use. They all

have the familiar agent structure, the protocol, and then you can just drop in the chat client that you need.

And so one one other thing is Yeah. Go

ahead.

Have you noticed that where is the kernel in the new framework? Do we have kernels?

The kernel is gone. That's an extremely extremely good question. And uh we're we're simplifying things now by uh removing this this kernel that needs to

be passed around. And so tools what can now be registered, you know, at at an agent level. Tools can be passed in at a

agent level. Tools can be passed in at a run level. Um but we we've gone we've

run level. Um but we we've gone we've we've simplified it. We found that the abstractions don't necessarily need the kernel like it did uh you know in

previous versions. And so that's that's

previous versions. And so that's that's going to be a great simplification. The

one thing I do want to call out is in this uh second pattern below, you can actually just create an agent from a client. So you have two different

client. So you have two different patterns uh ones that depend, you know, on your coding style or your business needs. Um, but really easy ways to get

needs. Um, but really easy ways to get started. Minimal amount of code and

started. Minimal amount of code and probably pretty familiar. Uh, you you can create the function. Here we have get weather. You can either decorate it

get weather. You can either decorate it with the the AI tool decorator or you can just pass in the function and it'll still, you know, work as as expected,

right? I think one thing one thing I

right? I think one thing one thing I just want to call out like when you create a agent directly from client it solves a lot of issues that we encounter we create uh custom agents for each

client right because now we're doing this for you instead of you have to do it to figure out what the difference depending is of the clients definitely right so that that was a big goal in the

value ad we wanted to bring over from semantic kernel uh is is really creating you know kind of more intuitive APIs, better abstractions, uh, easier to work

with, and we know agents are still at the forefront of everything. And so

being able to get started, you know, with these easy abstractions is is at top of mind for us.

Well, Evan, I always say people watching this, they say, "Oh, that's another framework. That's a new framework." It's

framework. That's a new framework." It's

more than that. I think is bringing together the those you know two frameworks you know and bring the best of them like you said they simplified like remove the candle

remember some candle was created before uh MCP servers and all these things and now you just simplify and make sure that will be easier and definitely

if you're watching this hope that VTO has some magic to show very soon here that we're going everybody's going to love Yeah, I ju I just want to call out too

that you know we the auto from the autogen team there was there was an amazing you know SDK created and the ability now to have both minds working

on one SDK is is so much more powerful uh it's it's really now we're able to kind of agree on these abstractions that we bring forward and so these unique perspectives from both SDKs and and

having that now in one agent framework is is really exciting but that is a lot of fun the work that's a lot of fun yes so I I also want to hear now how you

know what what are the differences in in autogen between autogen and agent framework uh and so I think you know Eric can you tell us a little bit about what what

we're going to be be looking at here thanks seven um so I'm really excited so

autogen has been around since uh the of uh the AI chat GBT era. So I think it itself has gone through a couple of

evolutions and uh the interface has changed uh a few times but we eventually arrived at a pretty simple uh easy to

recognize and understand interface which is you have an agent and you have tools and you put them together and it goes to work.

And now let's see the agent framework.

uh it looks pretty much the same. So we

are keeping the simplicity in the new aging framework uh but we added a few tricks. So um first of all aging

tricks. So um first of all aging framework the new aging framework uh introduces uh more compatibility with

different uh different uh model service endpoints. Uh and the other one is it

endpoints. Uh and the other one is it has better support for hosted tools. So

in autogen this things uh was not supported at the moment and I think the most important thing is um it also

introduced the workflow API that the autogen didn't have. So if you're coming from autogen and you wonder how do I get my agents to follow a predefined order

of execution uh instead of fixing the prompts in your group chat you can actually just create a workflow and it agent will follow that

uh order deterministically. So I think that is a biggest uh update I would say and the biggest um uh good uh the best

thing that uh for autogen users. Uh so I highly encourage you to check it out.

What what if you know what I guess what if I still wanted to work on kind of some experimental uh aspects or I have an idea that I think would be good for agent framework.

Is there support for that? because I

know autogen was born from a research project and yeah is there still ability to kind of work on that?

Um yeah that's a really good question.

So yeah autogen has a lot of great contributors. Um we really thankable of

contributors. Um we really thankable of all the amazing contributions and great ideas that push autogen forward as a community and we welcome all of you guys

to join the new community uh to to work on the new framework together. So

there's two parts to it. So one is uh the new framework would have a core framework to itself. So this core framework will working towards more production ready enterprise usage but we

also have a lab component. So we call it agent framework lab. So in the agent framework lab we're going to introduce uh the amazing research that's going on

in Microsoft research. We also welcome your contribution. So you can contribute

your contribution. So you can contribute to the lab package if you have a great idea on new agent oxation or have a good ideas on let's say how do I test my

agent on this new benchmark or how do I train my agent using reinforcement learning right so all of these uh new idea we decide it to have a nice um like

a dedicated uh location for us to uh collaborate and then um to experiment so it's called Microsoft agent framework lab and it's part of that in your

framework itself.

Very nice. That's great.

Do you want to talk about some more I guess some some more of the APIs we've been working on around uh you know agent framework and maybe some improvements

even around from what autogen had. Do

you want you want to say quick few words about that Eric?

Yeah. Yeah. So on the agent abstraction set um so autogen has something called um model context right. So model context was used to store the message history

and it has another component called uh memory which is to um take uh external data and use that to augment the bottle context. So it was two separate things

context. So it was two separate things but now in the agent Microsoft agent framework uh we introduced the agent threat concept. So agent threat concept

threat concept. So agent threat concept is much more powerful because it allow you to incorporate external memory as part of it. Uh so

uh framework will come out with uh suppose like uh disc based um uh agent thread and you can also hook it up with reddis and or coosmos db for your

external storage and you can also plug in me zero for example or reddis for uh the memory component. So the agent threat concept will capture all these uh

abstraction and uh uh I think in this slide it talks about short-term memory.

So in that sense the short-term memory is a chat history that agents working with and also the long-term memory is something like user preferences you know the past task completions. So those

things are retrieved at the runtime to help agent you know perform better because it doesn't need to repeat the same mistake or can remember the last successful uh task completion so that it

will repeat the same uh success patterns.

Yeah. And for users of semantic kernel I think they'll be familiar with this agent thread but it the Python version at least never was had the full implementation like the .NET version

did. And so there's going to be more

did. And so there's going to be more ways, you know, out of the gate uh to to start to deal as Eric was talking about with these short-term memories, long-term memories, memory providers.

Um, and so having these integrations like Mem Zero is is super important, you know, early on so we can get feedback, help develop, make sure that, you know, agents are using the proper context because it can be frustrating if you're

interacting with it and it loses sight of the conversation or things get out of scope, you know, the token scope or, you know, things are too large. Uh, so I think it's great that we have this this early.

And Evan, just wanted to call out one other thing too on on the road map.

We're working really hard with the AI Azure AAI Foundry team to be able to have agent memory inside of Foundry too.

So if you want like a fully managed service, we're working on that with Foundry team to be able to store your uh memory there. So that's a little sneak

memory there. So that's a little sneak peek into what's coming soon on the road map as well.

Very nice.

Nice. Evan and uh maybe it's a good time to talk about you know the the language that's going to support me on the initial preview.

Yeah, for initial preview we're going to have support for Python and .NET. Um

there is potentially other things coming in the future, you know, on the road map, but but right out of the gate here for for this public pre-release, it it'll be Python and net. And so, you

know definitely users are familiar with both both SDKs, semantic kernel and autogen that'll be familiar for them. Um and so we're yeah,

definitely excited to have have some of these new features available pretty early on. Um, and that that kind of gets

early on. Um, and that that kind of gets us into other things that I think could be top of- mind for people uh, you know, wanting to use an SDK like this because

enterprise enterprise readiness is extremely important aspect of it. The

ability to get tracing uh, and and have guard rails uh, something new, I guess new in the terms of of its functionality is is something we're

introducing as middleware.

uh from semantic kernel we had it called filters. Uh you'll kind of feel familiar

filters. Uh you'll kind of feel familiar in that sense but augmented in the fact that you can now provide middleware at run level and at agent level and so you can make sure you're you're having

things like you're masking PII or you're running certain things that that need to be run as as sort of a hook um during your process or your operation. Um, and

definitely still focused on on security, you know, enterprise security with with Azure, uh, Microsoft Enra ID and and and so on. So, a lot of these things will be

so on. So, a lot of these things will be be familiar, but we're working to improve them to make sure that, you know, really early you can have a good tracing story locally and not just in

the cloud. Um, and so that that the team

the cloud. Um, and so that that the team is is really focused on this and making sure that it's it's a good offering.

Um, anything you want to say, Victor, Eric, about this?

I I just want to point out that the user approval pros and um think about like what we had in autogen was not fully complete in our opinion, but I think we

finally got a chance to have a complete design for how human in the loop, user approval for two calls and workflow suspension like all those things will be

in the new framework. Um, so I'm really excited about that.

I mean that that was also a gap in semantic kernel. We always had this oh

semantic kernel. We always had this oh just you know provide input from the console is is human in the loop and it was like it always left more to be desired

and so I agree that that getting this solved very early and it's such an important aspect. I know in in autogen

important aspect. I know in in autogen you had a a proxy agent which was even a much better step than than semantic kernel and so I think this is going to

solve a lot of a lot of problems for developers uh and and again to get this this early is is really important right I the guard rails are can be

implemented as middleares and so it's really powerful patterns if you want to catch like uh agents produced in pi content control password or secrets or

you want to catch data change switching between boundaries uh as agent working across different tools. So those things are critical for you to build like enterprise ready application.

Definitely. I feel like we've been we've been hyping this up enough. Should we uh should we get into something and actually see some of

it?

Yes. So for this part there is some magic here. Do you want to share your screen with Yeah. So um so in the next couple of

Yeah. So um so in the next couple of minutes I'll walk through a demo and for that demo I'll sort of reiterate some of the similarities between autogen and um

the new agent framework and then I'll sort of show what a simple uh workflow simple agent run looks like. And so if you have used um autogen you'll be super

familiar with the code we have right here. The first thing is that you can

here. The first thing is that you can represent a tool as a function and then we provide a convenience class called the assistant agent. It has a name, a description. We can give the model

description. We can give the model client and we can give it tools as a list and then we can run it and everything is fine. Now if we switch over to um the agent framework, you'll

notice that we have the exact same or very very similar um highle chat agent abstraction. Similarly, we can give it a

abstraction. Similarly, we can give it a name, a description, instructions, which is the equivalent of a system message.

We can give it a model client. Here it's

called a chat client. And we could give it tools. That's a list of tools.

it tools. That's a list of tools.

And once you have all of that set up, you can simply just call uh agent.run or

agent.r runstream. And you get pretty much the exact same behavior.

But importantly, we know that a lot of developers just just don't want to leave alone in the terminal. Um most of the time you want something interesting, something graphical um to sort of

interact with your application. And so

as part of agent framework we we are sort of shipping a sample app called dev UI and if you sort of imported uh the serve construct from dev UI um from

agent framework the dev UI import serve you can sort of give it a list of entities in this case we've given it the agent we've created and we give it a port number and once we run this we get

an uh an experience that looks like this and so what we see is that um we see a new UI and the exact same agent that we've developed we just defined we see

here now typically we can do things like um what is the the weather in SF and as

we'll expect the agent does a model call it does a tool call underneath and then we see the result and get all get streamed in here importantly um we see that there are set of events that are

sort of called as part of that request and we can sort of interrogate all of that there in In this case, we have open telemetry uh sort of set up here. And so

all of the hotel spans that are emitted as part of this agent run, we can sort of investigate and sort of look into exactly what they are here. Um and in addition to that, if we have any tool

calls, all of that sort of gets shown here. Um earlier Eric and Evan sort of

here. Um earlier Eric and Evan sort of alluded to the concept of threads and so that is also fully supported in this sample app and so every time we might switch across multiple threads in this

case we might create a new thread we might do things like let's say in this case we want to add some multimodal inputs so I have a screenshot on my

desktop and I might say describe the content of this image and just like we would expect um all of This goes to the model that's sort of

associated with this agent which is GPT4.1 mini. Um in this case what I

GPT4.1 mini. Um in this case what I actually have is a screenshot from the Microsoft learn page for semantic kernel. We see some uh some text uh

kernel. We see some uh some text uh transcription text extraction just like we would expect and all that good stuff.

So this is just an example of some of the capabilities that we have currently in agent framework and also how we are essentially trying to improve on the

developer experience provide better developer economics and ideally um this experience is not only code based it's also provides a CLI

and so if you structure your agents in a given standard um and you have them organized across folders you could run the CLI pointed at that folder And all of the entities that we support,

agents, workflows, they will show up in this interface. In this case, we have

this interface. In this case, we have only a single agent. But if you had like 10 agents or 10 workflows, they'll all show up here and you can similarly sort

of interact with them. Um, we will continue to build out the sample app too. Uh, we'll get feedback from all of

too. Uh, we'll get feedback from all of you. And the goal here is to provide an

you. And the goal here is to provide an experience that sort of makes it not just interesting, but easy for you to sort of build and prototype agent and multi- aent applications.

Um, that that's exciting, Vito. I don't know if what do you guys think?

Put your hands up. Wow.

And I love it that, you know, all the tracing and the things that Evan was talking about that, you know, production ready, enterprise ready.

Uh, they're going to be able to do that from like from from day one. The

developers can, you know, can track these things on on development side and make sure that they are ready for production. And I love it. I really

production. And I love it. I really

really like it.

There's one thing I wanted to call out that you said Victor that I think is really important. It's around you know

really important. It's around you know kind of the developer ergonomics and things about agent framework at least from semantic kernel. We I think one

issue we had was related to imports. Uh

you could get lost in the all these different imports and not understand how to get in bring a client in. And I think in agent framework, we spend a lot of time designing a package structure that

makes it really intuitive to have the most minimal amount of imports to get up and running with your agents. And you

showed that in the in the code snippet you had that we had a couple imports in that code, right?

Yep. Yep. That is correct. I think um there was a time I built a tool that compared the hello world implementation across multiple frameworks. The semantic

kernel was there. autogen both APIs the high level and the low API was there as a line graph and line chain and career

and all of that and I think um those types of um comparisons really showcase all of the work that we've put into agent framework to improve developer

ergonomics. I think right now agent

ergonomics. I think right now agent framework probably has the least lines of code just to get to run a simple agent that has the standard behaviors

memory tools uh model client um I think it's just great for developer experience and and to to that point about developer

experience too we want your guys' feedback please open GitHub issues let us know in the comments below here um open PRs we just merged a PR from someone who like found an issue the

other day. So like we really want we

other day. So like we really want we really this is we're really committed to the open source nature of agent framework and really want to hear your guys' input, feedback, etc. to make this dev experiences as awesome as possible.

So please reach out. We also have office hours on Wednesdays um 8 a.m. Pacific

time. We'll also include the link below.

Um and we a lot of us we all go to those office hours. So if you have something

office hours. So if you have something feel free to let us know.

Yeah, that sounds great. Do you have any any last like what's next for them? Like

like Elijah said there's any links there. Okay. Yeah, that sounds great.

there. Okay. Yeah, that sounds great.

Definitely.

Yeah. Follow follow follow the project you know follow this show as well because we're going to bring other videos on Asian framework and open your shows that Victor you know the the ID

the the UI that uh don't call ID let's call UI is UI that I think people can help even to improve that a lot and and giving feedback I think that's be the

most important that's why we launching this at open at Microsoft I think is very exciting time and just to Finish.

Evan, you promised to tell me migration.

What's coming from migration? Who wants

to say something about migration?

Yeah. Yeah. We we understand that some of this could look scary in the sense that so many of you built on top of autogen on top of semantic kernel and

you're saying why why do I have to move to this new framework? What do I get out of it? Uh we we really we're going to

of it? Uh we we really we're going to have documentation. We are going to have

have documentation. We are going to have a migration story. We have, you know, that work is in flight. We want to make it as seamless as possible for you. I

think you've seen and as you look through the samples in the repo, a lot of the abstractions will look similar.

It's just a little bit of, you know, different syntax. Um, so there aren't

different syntax. Um, so there aren't this isn't completely, you know, shouldn't be brand new for people. A lot

of the business logic you have in your agents currently you can bring over. Um

and and so we we really want to make it you know as as painfree as possible and that's why we are spending time working on migration. Um and that that'll land

on migration. Um and that that'll land closer to you know you know uh as we're in public preview um later towards GA um

and so that can't promise anything today. The best we can do is you know if

today. The best we can do is you know if you open issues we can work on that or discussions or or help show better samples. feedback is is really paramount

samples. feedback is is really paramount here and and what is not clear. Um so

you know we're trying to have doc strings that show code samples uh how to implement that. I think autogen did an

implement that. I think autogen did an amazing job in in their doc strings of what do I actually do and how do I use this this particular public method things like that. Um and so ready to

kind of hit the ground running with you and the community and and help you uh as you need it. Victor, I think you know I heard you say something about quickly,

Victor, I heard you say something about workflows. I know we barely touched on

workflows. I know we barely touched on that, but what what can you say around that?

Yeah, so um across the presentation you might have had us sort of refer to um both agent autonomous agent orchestration as well as workflows. And

I think Eric mentioned that workflows are how um your path to expressing uh complex business logic in a more

deterministic pattern. Um we've worked

deterministic pattern. Um we've worked very hard on the workflow API. Um and

even though today uh demo we showed was mostly looking at single agents. I think

in the future probably in a future episode we sort of walk you through the details of of all of that. um how you can define independent steps uh represent your business problems as a

graph so to speak and sort of figure out uh just define transitions across multiple steps and so definitely stay

tuned um the link to the repos uh it's open source it's MIT licensed um very similar to autogen and um definitely look through it um if you have questions

which we would expect please reach out to open issues and we would love to hear from you.

Yeah, that sounds great. Thank you so much everyone. That's a big day for those

everyone. That's a big day for those projects and hard work that the whole team is working out on that and together I think be bringing their speed to to a

single project and yeah look forward to see you on the next video and let's say that bring your flow video let's bring

you know more content around agent fo see you guys